Select Committee on Children, Schools and Families Minutes of Evidence

Examination of Witnesses (Questions 220 - 239)



  Q220  Mr. Chaytor: Greg, in your submission, you were critical of the QCA for intervening in too heavy-handed a way, whereas in the AQA submission there were accusations of not regulating strongly enough in respect of the current basic and key skills tests. It said that the problem is that the regulation is weak. Where is the balance? Is it that the QCA has been too interventionist, or has it been too hands-off? Each of you says different things.

  Dr. Bird: I think we were saying that it was conflicted because it was the provider of the tests. It was not a matter of being too hands-off; it was the regulator of itself. That was our point. Those tests are derived—we only deliver them—by QCA itself, so it sets the standard.

  Greg Watson: I would say that it has been a case of too much in some areas and too little in others—too much intervention at the detailed level, getting in the way, I think, of producing more stimulating approaches to geography, more relevant vocational offerings, but on the other hand too little attention to the long-run effects of a series of changes piling up. If you were to look at A-level over time and take modularisation, the move to six units, the balancing of coursework, the splitting of AS and A2 into two lots of 50%, compounded with various tweaks and twists applied along the way; if the regulator is looking after public confidence it should be less concerned with any one of those changes in any one subject and more with looking at the long-term effect over time on actual standards and on public perceptions of standards, and therefore, with measuring the rate of that sort of change in a more moderate way.

  Murray Butcher: The opportunity for the independent regulator may mean that QCA, or whatever it becomes known as, may be able to focus on regulation and refine that process of regulation, which I would say, from City & Guilds experience, has been delivered in fits and starts. There has been urgent attention to this activity, which then suddenly dissipates; then it comes over here somewhere, and gets distracted by some of its curriculum activities. I really look to this as an opportunity to have stable regulation. The one principal issue that we still have, which Andrew mentioned in his opening statement, is the fact that we have four regulators to deal with, and that is a big bureaucratic problem.

  Q221  Mr. Stuart: Following on naturally from that in terms of maintaining standards, should it be an important part of the new regulator's job to look at comparisons internationally? Sir Peter Lampl said, of standards, that "the key question is are they improving fast enough to maintain or improve our position relative to other European countries, America as well as China and India?" Do you agree with that?

  Jerry Jarvis indicated assent.

  Greg Watson: Any sensible regulator in this field, with a primary focus on standards, will want to base its decisions on evidence. That seems a strangely obvious thing to point out, but it is a point worth making. There is good evidence abroad. As I mentioned, we are part of a group operating in 150 countries. It is partly through the process of synthesising research from experience in different countries that we can draw some of the conclusions that we can about the impact of change on standards and so on. I would have thought that any sensible regulator would want to build relationships around the world with all sorts of research communities.

  Q222  Mr. Stuart: In the context of today's evidence, what do you make of this country's tumbling down the international league tables that came out recently?

  Chairman: Do not roam too widely, but comments would be appreciated. Are we tumbling? Is that your fault? Is it just a rotten examination board system causing us to tumble down the international league tables?

  Jerry Jarvis: First, that is a very difficult question to answer here. We should also be very careful about what we compare. When Leach looked at the various standards in different countries, one of the conclusions that we started to come to was that we probably do as much training, but that we do not certificate it in the same way, so we do not count it. Counting eggs and counting eggs is an important issue. We certainly share a massive international business with OCR, and this education system still has huge respect abroad. We cannot be complacent but it can be very dangerous to build any sort of policy on an international comparison because of the sheer difference of literally comparing the same thing.

  Chairman: We must move on to the last section.

  Q223  Mr. Stuart: May I just deal with the key stage and testing at different times? Edexcel's evidence was that that will increase the testing burden and that the Key Stage test should be at the end of the key stage, if retained. Do you think that if we are to have tests when the pupil is ready rather than at the end of the stage, we need to scrap Key Stage tests as such?

  Jerry Jarvis: Forgive me, Barry, I need to make a slightly longer response to this question, because it is something I feel strongly about. We constantly accuse ourselves of over-testing, but summative, quantitative testing needs to be done to ensure that the investment that we put in, and the teaching and learning that is happening, happens. Let me return to what Ken Boston said—three things: personalised learning, continuous—essentially, internal—assessment and training of teachers. The reason why I care so much about it is that after about £80 million of investment, we can do it now, and if we really were interested in shifting that stubborn set of five good A to C grades, we could do it now. It is here, and Ken Boston was absolutely right. So, do we over-test? Yes, in certain areas, but we do not spend enough time on personal, continuous improvement, which is the key to improvement.

  Chairman: Okay. We must move on, and I want to cover the last section. We have the duo of Fiona and David covering it, with Dawn throwing one matter in if she wants.

  Q224  Fiona Mactaggart: We have had quite a bit of evidence about the narrowing of the curriculum and teaching to the test, as you would expect, and in a way, some of your evidence is specifically about that. I am very interested, for example, in your analysis, Andrew, of why aptitude tests are not an appropriate alternative because you can train young people to succeed in them—a fact I know very well, representing a town that still uses the 11-plus. Children of 11 perform less well on Key Stage 2 assessments, I think, because they are training for their 11-plus aptitude tests. I think that I am hearing from you that in some ways it is a good thing if people teach to the test, because then we get better results. However, all of us as educators think it is a bad thing, because the domain that young people learn is narrower.

  Dr. Bird: I think that we all said teach to the curriculum, rather than teach to the test. We want people to cover the whole curriculum and know, as Jerry said, the experience that they will have when they are examined, so that it is not a shock and they can show their best when they are being tested. That is about understanding the style of the examination that they will be required to sit. Our job is to ensure that the tests maintain sufficient variety and coverage of the curriculum over time, so that it is not easy to teach to the test in the narrow sense in which I think you mean it—saying "I am going to predict the six questions that will come up next summer, and teach you seven topics. I guarantee that six will come up." You are leading us towards a very narrow, output-driven thought. We certainly want to give people every encouragement to cover the actual curriculum, which is why we make specifications available, as Greg said. When I was doing A-levels, I had no idea what the curriculum was, which cannot have been a good thing.

  Greg Watson: If I may add to that, we have talked quite a lot about innovation, and it is a real benefit to have more than one awarding body. The competition of ideas has been a powerful driver to keep syllabuses interesting and make subjects stimulating. In the era before micro-regulation, which really started in 1998, there were outstanding partnerships in a whole range of subjects—science, geography, maths—between individual awarding bodies and very creative university departments in places such as York and Cambridge. That was true also between groups of forward-thinking teachers—I can think of the Suffolk science movement. That interplay of ideas put a range of options out there for young people and their teachers. A school would pick its syllabus to suit what kind of school it was and the young people that it had. We are beginning to get some of that back. There have been signs that we have moved back in the right direction, and if we get the form of regulation right and it moves from the micro level to the level of looking after the system, there is scope for the interplay of different ideas among organisations such as ourselves. All the evidence from the work that we have done when we have had a bit more freedom to operate, and been less under QCA diktat, is that we have been able to reignite interest in some subjects. We are doing it again with A-level history, in which we have gone back to an alternative, more research-based approach, sitting alongside what we might think of as a more traditional style. Everything that we hear from the chalk face is that some people learn the skills of history much better by going out and touching it with their hands than by simply learning a load of facts from a textbook.

  Jerry Jarvis: Teaching to the test is a perennial issue for us. When I talk to the Specialist Schools and Academies Trust and say, "Help me to avoid being cast in the role of someone who is encouraging teaching to the test," it slaps me on the back and says, "Get on with it. That is exactly what we are supposed to do. We are supposed to set a standard, teach the syllabus and assess whether students have met that standard." I do not think that, with the enormous stakes that are placed on league tables, you can avoid the accusation that teachers are spending time on examination preparation. It is inevitable, and it was always there. A long, long time ago, when I was doing qualifications, I remember practising on past papers and wishing I had a father who knew more about maths than my friends' fathers did. What has changed is the pressure to succeed. It really defines careers inside schools and colleges, and it is something that we have to guard against. The thing that drives me along is that if we can use technology and all the techniques that we have to make exam preparation relatively simpler, we can spend much more time in enriching education. Why are people educated? Is it for entry to university? Is it in preparation for work? Or is it for the sheer joy of learning? I seem to remember that there was a lot of that in the past.

  Q225  Fiona Mactaggart: Absolutely, Jerry, yet your organisation is producing ResultsPlus, which, as I understand it, will be able to show me, as a teacher, that, fine, I might be wonderful at teaching Jane Austen but when it comes to Shakespeare my students are much more wobbly than Barry's students are, and that I have to shape up on that bit of the curriculum. Do you think that there is a risk that ResultsPlus will actually encourage that sort of thing?

  Jerry Jarvis: What I have had back from a large number of teachers is that we are moving towards true personalised learning. Because such information is available, they can spend less time on revision, which becomes far more to the point. Teachers are able to go at the speed of each different student, and they have more time to indulge in the richer aspects of delivering the syllabus. In fact, in our experience, the process actually works the other way around.

  Q226  Fiona Mactaggart: You make it sound as if it is the child who will be the level of reflection that a teacher will make. Is it not more likely that it will be the teacher who will be the level of reflection in this sort of a system?

  Jerry Jarvis: Having deployed the technology, it is clear that it operates at many different levels. As a tool for the senior management team inside a school or college, it pinpoints exactly how well the school or college is delivering its curriculum across all the subjects. For an individual teacher in the classroom, we are seeing a jump in the performance of their students. Teachers are literally taking the information and using it to their advantage.

  Q227  Chairman: But you do not all agree with that, do you? Some of you are not using the technology.

  Greg Watson: We are looking at similar ideas at the moment.

  Q228  Chairman: So you approve?

  Greg Watson: This is nothing new, in fact. Any teacher who wants to do the best for the young people in their classroom will reflect on how they are doing, and will compare how they did this year with last year. For some time, teachers have been able to get hold of exam answers and have a look at them to see how well they did. For many years, many teachers have come to professional development events—INSET events—that we lay on. Some involve a general run across the syllabus, but some are targeted at particular areas. If we get feedback from teachers that they are struggling with the coursework element, we always lay on support for them to come and talk about coursework and perhaps learn from other teachers. I think that the desire to understand, to diagnose, to target a bit of effort to develop professionally has been with teachers for a long time. We have started to explore whether technology and some of the data that we have could add to that.

  Q229  Chairman: There are no dangers?

  Dr. Bird: There is the issue about releasing detailed results directly to students. We think that that can be mediated through the relationship that students have with the centre and with their teachers rather than giving results to them directly, if that is what you mean by richer information being available. We certainly think that teachers can be assisted by having richer feedback about how the cohort of students is performing so that they can modify their teaching methods. It is obviously better if they get that from formative assessments that they carry out in the classroom rather than from summative results for the cohort that has just gone, as those students already have their qualifications, whether good or bad. Many people's support materials for classroom teachers contain formative tests in the form of homework, which essentially have the teacher ask, "Did I get that algebra across correctly or not?" That must be a helpful process, to assist teachers and curriculum heads in understanding how different teachers are performing in different parts of the curriculum, so that they can play to their strengths and fill in where there might be weaknesses, because people are not universally good at doing everything in the curriculum.

  Q230  Fiona Mactaggart: When you design questions for examinations, which of course is a mystery to most people—we can only imagine the types of questions—you test all sorts of questions and design them to find out if they show specific knowledge or a range of skills, and such like. When you do that, do you think about the skills or knowledge that reveals something beyond the ordinary? We talked earlier about creating an A* grade, for example. Clearly that must have implications for the design of questions, to create the space for a young person to show that ability. None of your evidence tells us how that is done.

  Chairman: One of you answer that question, or we will be here all night.

  Greg Watson: It is a very careful and very rigorous process; a lot of what you suppose to be true is true. In writing a syllabus, we are not simply writing down, "Oh, read this book and you will know all about the Second World War." We will actually unpack that process to say, "By the time someone has successfully completed a GCSE course, they should know the following things and they should be able to do the following things with that knowledge." We will then describe a series of levels, which we call grades, that will differentiate the level of knowledge that you would be likely to have and what skills you would be able to apply to that knowledge. Interestingly, in the context of A* we have been doing a research project in advance of introducing A*, because we have been through the process of redescribing the syllabus. We have now described to ourselves, "What does an A* grade historian have by way of knowledge and skill that you would not have expected of an A grade student?" We are now trying out questions at the moment for exactly that reason; we want to discover what types of questions unleash that potential and what types of questions are sufficiently open-ended to provide that extra stretch and also, importantly, what types of questions successfully differentiate the most able students from the rest and do not lead them all to being clustered around the middle of the mark range. So that is the type of work that we are doing. We have a great army of research people sitting behind everything that we do and that is what they do all day and every day.

  Q231  Chairman: So you want to take over the world, do you not? You do not want the Qualifications and Curriculum Authority or a national curriculum. You want to set the whole thing really, because coming through is the idea that you can do it all. Your exams drive the curriculum.

  Greg Watson: I think that we have very important expertise. We are capable of adding more value in terms of innovation and enriching and discovering new approaches to subjects.

  Q232  Chairman: Is it not dangerous to let you take over the world of education?

  Greg Watson: There is something missing in my equation, which is why we need a regulator, and it is that the public's stake and society's stake in the qualifications system is much bigger now than it has ever been. Because of that, the public want to know that young people's education is safe in our hands and that we are doing what we should be doing absolutely properly. It is quite right that somebody who is not sitting along this line of witnesses here should have an independent view and should be tracking effectiveness.

  Q233  Chairman: So you want a weak regulator to defend you from public criticism?

  Greg Watson: I want a strategic regulator who can really stand up on the basis of hard evidence and support us when we say that we are looking after standards and everybody is getting a fair deal. There should be an independent body that is not a vested interest, which we might be seen as being, and that can say exactly the same thing and say it with confidence, on the basis of hard research.

  Q234  Ms Butler: I just wanted to go back to an earlier question. You talked about Key Stage tests and when those tests are taken. As you know, there is a push for stage not age; the stage when the person is ready to take the tests. I wonder whether the panel could just say what they feel would be any unintended consequences of that.

  Jerry Jarvis: Did you say "unintended"?

  Q235  Ms Butler: Unintended, yes. Will it be an extra burden on the teachers? Is it reliant on the teachers' understanding of that child's abilities, or is the focus all on that teacher?

  Dr. Bird: I take it that you are talking about the confirmatory single level test.

  Ms Butler: Yes.

  Jerry Jarvis: I think that the point that you are making relates to the type of "test when ready" issue and whether that would have unintended consequences. One positive unintended consequence is that it would perhaps take the pressure off that terrible examination point in the cycle. To me, it feels right to test when ready. That would require a much higher degree of administration and control and the provision, perhaps, of technology to assist the teacher in the classroom. It would be bound to have a number of knock-on consequences. My view is that examining when ready would have far more positive than negative outcomes, because it would tend to be about successful delivery. It would be a huge advantage for the student and the teacher.

  Q236  Ms Butler: Are there any other views on that from the other members of the panel?

  Dr. Bird: I agree with Jerry. There is a danger that teachers who lack confidence would fall into a test, re-test, re-test mentality with students who are bouncing along just below a level. That might be a problem but, generally, testing when ready would have many more positive than negative outcomes provided that it is allowed to replace mass testing. There is a danger that we would have single-level tests for one purpose and some sort of national audit test run at the end of the key stages. In many ways, that would be the worst of all worlds.

  Q237  Ms Butler: What do you think would be the best way to ensure that there are more positive than negative outcomes? For those children who are bouncing along and who may be ready but whom the teacher does not think are ready, how can we can build in a caveat to ensure that there are more positives than negatives?

  Greg Watson: There is an important principle, which runs across all assessment, that says that we should be absolutely clear about the purpose of assessment up front. Ideally, there would be one purpose of assessment rather than trying to do many things at once. We should be clear that we are assessing in order to inform and support learning and to help teachers and to guide their efforts. That is a very different purpose from measuring to compare schools against one another to create league tables and support parental choice. Our field does not involve Key Stage testing—we are involved in GCSEs and A-levels—but being clear on what we are testing and why is a really important discipline. It is important to get that right in this context so that we do not have the dangers that have been highlighted about a conflict between an assessment to support learning and an assessment to rate a school's performance overall.

  Jerry Jarvis: Here I am back on my subject. The infrastructure prevents testing when ready. The process is geared to delivering information at a constant rate to all students in the cohort, who will then be examined at the end. We have to do much more than simply say that we will test when ready. The infrastructure—personalised learning—must be in place for such an approach to work. If that happens, it would be worth doing.

  Chairman: We are getting really tight on time.

  Q238  Mr. Chaytor: I have a short question to put to each of the four members of the panel. Does the use of league tables help or hinder your organisation's work?

  Murray Butcher: I probably have the easiest answer of the four panel members. Given that the majority of our work is in vocational qualifications, league tables do not greatly affect our activity.

  Q239  Chairman: Nevertheless, what do you think of them? You are a professional educator.

  Murray Butcher: My anxiety has already been expressed. It depends on the purpose of the assessment and on what are trying to draw from it. I recall that Ken Boston, the chairman of the Qualifications and Curriculum Authority, mentioned just before Christmas that one of his colleagues identified 22 different purposes for assessment. I suspect that 22 is almost a random number and that you could raise it somewhat. I suggest that league tables have far too much pressure put on them and that they are probably not a sufficiently refined instrument to give the exactitude that they purport to give.

  Dr. Bird: I do not think that tables impact on our work particularly strongly. Obviously, our outputs are part of what formulates them and we are aware of pressure around the boundary between D and C grades in some subjects. The inquiries about results and the seeking of scripts after the exam to check on that boundary has a relatively marginal impact as far as operational activities are concerned. I have no other comment really.

  Jerry Jarvis: There is no question that the pressure creates some disadvantage to us—particularly in respect of calls for more research and more opportunities to take—but there is a huge pressure on schools to maximise the sorts of qualifications that are taken. Therefore, we see schools and colleges chasing qualifications in an unhealthy way. However, that same competition demands that we drive up service and standards in a way that we otherwise would not. There is no question that league tables affect what goes on inside a school. However, one that benefits society, if not necessarily Edexcel or my colleagues, is the pressure to continue to compete for those qualifications because they are taken so seriously.

  Greg Watson: Undoubtedly, pressure creates unhelpful tensions in the qualifications system. If our fundamental job is to assess learners and provide assessment that adds to learning and that helps to structure and motivate learning; the other dimension of what we do is used to determine teachers' pay rises and that gets in the way of that relationship. That explains the distance that some teachers feel exists between them and us, and is something that I want to break down. I hope that the new independent regulator—and we have mentioned him several times—will be very aware of that in balancing change and stability in the system. I wonder whether the regulator might run the league tables or develop a carefully defined relationship with those who determine such tables so that the tables do not get in the way of what the qualifications are fundamentally there to do, which is to give young people a portable measure of what they have achieved to take on to the next stage in life.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2008
Prepared 13 May 2008