UNCORRECTED TRANSCRIPT OF ORAL EVIDENCE To be published as HC 1671- i ii

HOUSE OF COMMONS

ORAL EVIDENCE

TAKEN BEFORE THE

EDUCATION COMMITTEE

THE ADMINISTRATION OF EXAMINATIONS FOR 15-19 YEAR OLDS

WEDNESDAY 18 JANUARY 2012

PROFESSOR JO-ANNE BAIRD, DR MICHELLE MEADOWS, TIM OATES and PROFESSOR ALISON WOLF

PROFESSOR STEPHEN J. BALL, PROFESSOR SIR JOHN HOLMAN, PROFESSOR GRAHAM HUTCHINGS and WARWICK MANSELL

USE OF THE TRANSCRIPT

Oral Evidence

Taken before the Education Committee

on Wednesday 18 January 2012

Members present:

Mr Graham Stuart (Chair)

Neil Carmichael

Alex Cunningham

Pat Glass

Damian Hinds

Charlotte Leslie

Ian Mearns

Tessa Munt

Craig Whittaker

Examination of Witnesses

Witnesses: Professor Jo-Anne Baird, Pearson Professor and Director of the Oxford University Centre for Educational Assessment, Dr Michelle Meadows, Director of Centre for Education Research and Policy, AQA, Tim Oates, Group Director, Assessment Research and Development, Cambridge Assessment, and Professor Alison Wolf, Sir Roy Griffiths Professor of Public Sector Management, King’s College, London, gave evidence.

Q329 Chair: Good morning. Thank you very much for joining us today. We have two sessions to get through this morning, so we probably have only about an hour. I am delighted that we have such a high-powered panel. I ask my Committee and the witnesses to try to keep their answers as succinct as possible so that we can get through as much as possible this morning.

The starter for 10 is whether the weight placed on our GCSEs and A-levels within the system is too great and more than any system of examinations, however organised. Is the weight put on it too great? Who would like to have a go at that? Jo-Anne.

Professor Baird: I think that that is a question about purpose really, isn’t it? We have lots of purposes for our qualifications. It is very difficult for them to meet all of those purposes well, so I guess that that would be my answer to the question.

Q330 Chair: How could we change systems of accountability so that we did put less weight on and fewer purposes were attached to current GCSE and A-level results?

Professor Baird: There has been a lot of discussion about performance tables and how that has affected the use of qualifications. There are real issues there.

Q331 Chair: Okay. Michelle.

Dr Meadows: I think it is about knowing the primary purpose. Qualifications can have secondary purposes, but let us have clarity about the primary purpose. Sometimes, what happens over time is that that primary purpose should surround as policies change, but there is a kind of lack of clarity about that decision making.

Q332 Chair: So the current Government have a lack of clarity about the purposes for which GCSEs and A-levels serve.

Dr Meadows: Not particularly this current Government. Generally, policy makers have a lack of clarity and communication. They might have clarity in their own minds, but it is the way that that decision is made and then communicated to those people designing qualifications.

Q333 Chair: Thank you. Alison.

Professor Wolf: I think that we expect too much of exams, in the sense that we expect every exam to do everything. We treat them all in the same way. It is not just about a lack of clarity of purpose, but the fact that we talk about exams as though they were all the same. In that sense, we expect ridiculous amounts of them.

Q334 Chair: What I am getting is that, if you are putting too much weight on the examination system and you sit there reorganising or coming up with radical ideas to change the examination system because you are asking it fundamentally to do something that no examination system could ever do, you are reorganising the wrong thing. I suppose that, before we move into the detail of what we could do with our examination system, we need to check that we have got the big picture right. Tim.

Tim Oates: It is incredibly naïve to assume that an examination system should only carry one purpose. When you look internationally, of course examinations are high-stakes and it is unlikely that a system where you try to establish low-stakes public examinations would work for any purpose. It is a question of looking at the purposes to which qualifications are put, and then to see what the effects of the combination of those particular purposes in that context are.

After all, with examinations you are often dealing with tiny decisions that affect long-term trends. You are deciding between 39 or 38 each year, in terms of a particular grade. You make that decision. If you make the decision in one direction over 10 years because of an overall spirit in the system, it will drive the system in one direction. If you decide t’other way, it will push the system in a different direction.

The point is that the functions combine to construct a complex mix of incentives and drivers. You have got to look to see whether those incentives and drivers are pushing the system over time in the direction that you want to go-in terms of over a policy.

Q335 Chair: Do you feel that, given statements of Government policy, those drivers are aligned with the policy outcomes which the Government desire?

Tim Oates: They certainly were not-and, chronically, were not for quite a long period of time. More sophisticated public policy purposes were being attached to public qualifications over the last two decades. It is my belief that regulation is now beginning to assume a form when many of those problems are recognised and are starting to be acted upon by virtue of regulation and the action of all the agents in the system. Public concern derives from a period when those problems were acute, but I believe that many of the things that have been put in place are beginning to address those problems.

Q336 Alex Cunningham: Tim, you used the word "international", so I shall ask a few questions about international comparison. Much has been made of the unusual features of the English exam system that differ from others: multiple exam boards, high-level transparency, lots of external assessment and close links to accountability. Does that reflect strengths or weaknesses?

Tim Oates: The UK system is not that unique. It is not as if there is a UK system and then absolutely everybody else has a similar system, which is a single-board system with the state having a very prominent or overbearing role. If you look internationally, you find huge subtlety in the variation of different systems. You find huge variation in terms of the role and influence of different agents. We see as well international trends towards greater complexity in the provision of qualifications in different nations. That is not fudging the issue.

In France, for example, you have a long tradition of the Ministry running examinations. It is not free of technical problems. In the States you have a very complex mix, from federal examinations and tests to monitor the system to provide intelligence on individuals, and you then have district arrangements and so on. Although we are unique in the particularities of what we have, it is not as if it is us against the rest of the world.

Q337 Alex Cunningham: But is it strong or is it weak in comparison to others?

Tim Oates: It has a particular blend of strengths and weaknesses, and again that is not dodging the issue. What is crucial in this discussion is, "Are the problems which are perceived to be emergent in the English system capable of being addressed within the current arrangements by changes in regulation, changes in procedures and so on, or is structural change necessary to remedy those problems?" That is the fundamental nut. If you are concerned about particular technical problems with measurement and particular problems with public confidence, you will find those problems emergent in all other systems, too, in different forms.

Q338 Alex Cunningham: Michelle?

Dr Meadows: I broadly agree with Tim, inasmuch as I don’t believe that the structure of the administration of the qualifications is what actually fundamentally determines quality. Different structures might support different aspects. Competing bodies offering qualifications has the advantage of supporting innovation. That is not to say that innovation could not happen in other structures, but it does support innovation. For instance, you get innovations such as electronic marking, which has led to more reliable marking.

With a one-awarding-body structure, you do not have to worry about whether there is competition on standards between awarding bodies. That perception can of course undermine public confidence, but it is not clear how a one-awarding-body structure would support innovation.

Franchising is another possibility, but one might have concerns there that franchisees are competing on standards for the particular subjects that they are offering. You can imagine a debate around whether the organisation offering A-level media studies-I am going to pick on that, but do not read too much into that-is competing with the organisation offering A-level maths. Each of them has strengths and weaknesses, and I just do not think it is the sole determining factor.

Q339 Alex Cunningham: So you are suggesting we can almost learn from what exists in the English system. What about abroad-are there countries we could learn from? Are there particular things that we could learn from particular countries to improve our position here?

Q340 Chair: Shall we come to you, Alison? You are the only one of the four who is not sponsored by one of the awarding bodies.

Professor Wolf: This is absolutely true. I would also like to emphasise that I am also an academic. I work for what we call a recruiting university, where we have lots of people coming in.

It actually goes back to one of the things that I said earlier about looking at different exams differently, and I want to say something specifically about A-levels. In this country, we have a system that relies very much on A-levels, because we have a competitive university system. Many countries do not: they do not have to worry about ranking people in the same way when they get their final qualification coming out of school. I would say the best university systems in the world are those which operate in that way.

It is also important to remember that, although we want to be able to rank people, we are also extraordinarily concerned about the quality of what they are learning in the sixth form. Even if we had a single exam board, it would not actually solve the problem, because an ever increasing proportion of our students come from abroad and from different systems anyway. What I would like to put on the record is that one of the key differences between us and most of the rest of the world is the nature of our university entrants, but the nature of those university entrants does not imply that all our problems will be solved by a single examination board.

Q341 Alex Cunningham: You talk about the quality of learning, but we have had some controversy around the way that different examination boards might not have been awarding grades appropriately because they do not want to lose their business. But we do-I suppose-have a transparent system. Do you think that is why there is not very much public confidence in it-because of this sort of controversy?

Professor Wolf: As I think other people will be able to say, the interesting thing is that there is still quite a lot of public confidence in the system, fortunately. Again, wearing both an assessment and a higher education recruitment hat, I do not think that the quality of the awarding is actually what bothers us most, to be honest. We have had problems in the past in that the grades were so broad that you could not rank people, but we are at least as concerned about the quality of what goes into the examinations.

This is obviously something that you are going to be addressing in detail, but there is an inherent conflict here between having examinations-again, I want to emphasise that I am concentrating mostly on A-levels-which place real demands on people and which are also responsive to changes in what is going on at the higher ends of a subject so that they can change quickly, and an extremely detailed accountability system of the sort we have. At the moment we have this strange, almost, classic of policy: the more you regulate the system and the more you go for accountability and transparency, the more you drive it down a road in which the questions become more and more banal.

Q342 Alex Cunningham: I see you are nodding there, Jo-Anne.

Professor Baird: Can I come back to the innovation issue? I think that there is not enough innovation, actually, in the system. What has squashed that is regulation, because we have become obsessed with a standardised model across all the exam boards so that we can guarantee this consistency. This is a real problem of our system-that we need a way of encouraging that innovation in curriculum and assessment models.

The main change driver across all of these things, such as the introduction of technology in marking, has been policy. Policy is what drives change in this industry. What would worry me here, in all this discussion about the structure of exam boards, is that this is a policy that could be changed, and I think we would actually miss the target.

What we need is to address the sorts of issues that people have about whether the qualifications are at the right standard for us internationally, looking at the content of them as well as the grading. Changing the exam board structure, which has been the main narrative so far in the inquiry, would cause all sorts of resources to be diverted to that, when really what we want to look at is whether the quality of our qualifications is right for our education system and for society more broadly.

Q343 Alex Cunningham: So do you think that there is a link between grade inflation and declining confidence?

Professor Baird: Grade inflation is a narrative in the media. There is less concern about grade inflation more generally among the public.

Q344 Chair: That is not because it does not exist, but because it does not actually concern the public that much?

Professor Baird: I think there is not a great deal of evidence for grade inflation. There is some, and I appreciate the work that Peter Tymms and Rob Coe have done in Durham, which is interesting, but all these studies have methodological issues. When you look qualitatively at the work of students, the question is whether it has improved over time, and we do not have a good answer to that. We can see that the results have gone up over time, but has the education system actually delivered better performance in those qualifications? People call that "teaching to the test".

Q345 Alex Cunningham: But are the young people who are arriving at the door of the university better equipped to start their university education than they were 10 years ago?

Professor Baird: Well, do we have good evidence about that? That is what I am raising. We probably need better studies on that sort of issue.

Q346 Chair: Alison, you have clear views.

Professor Wolf: I am going to go out on a limb and say no, they are not better prepared. I am sorry-I am. It seems to me that when you have the level of consistency in answers from people about this, as we do from academics, and when you get the concrete evidence in some of the quantitative subjects that we are getting, you have to say-not necessarily because of the nature of the questions or anything-that, yes, they are certainly not better prepared.

Q347 Alex Cunningham: Are they less prepared?

Professor Wolf: I am not sure if they are worse prepared, but they are certainly not better prepared. I suppose your question is that, if we have this ever-increasing number of people with As, is there any indication that they are actually better than they were?

Q348 Alex Cunningham: My colleague is asking, "What is the evidence for that?"

Professor Wolf: What is the evidence for the fact that they are not better prepared? I think two things.

Q349 Chair: Surely academics should not go out on a limb. Academics should tell us what the evidence is!

Professor Wolf: Actually, I think academics should-

Q350 Chair: If we want an opinion, we could just bring anyone in!

Professor Wolf: I think there are two pieces of evidence. The first is that a large number of universities that teach quantitative subjects are having to do more lower-level work with students when they come in to bring them up to a certain level, particularly in maths and such particular subjects. That may be changes in content, but I think you also have to say that it is levels.

Q351 Pat Glass: Is that not true right across the sector? Secondary schools say that about children coming in at key stage 2. Colleges say that about children coming in at key stage 4. Universities say that about children, and they always have.

Professor Wolf: No, but they have not always put on extra classes and they have not always changed their first-year syllabuses, and they are now doing both. I think that has to count as concrete evidence.

Q352 Damian Hinds: In what way does that say that children are not better prepared for the questions they are asked? Some might argue it suggests the opposite, that they were well prepared for the specific questions, but not so well prepared for the generality of the subject.

Professor Wolf: And this is when you immediately get into the sort of soup of whether we are talking about standards or are we talking about content? What is it that we are talking about? I think the answer is that children are extremely well prepared for the questions that they are asked in exams.

Q353 Ian Mearns: Teaching to the test.

Professor Wolf: They are taught to the test, and obviously, to the degree that the test is good, that has positive wash-back effects, but in a sense it does come down at the end of the day to whether you believe that people are doing more than the classic "things were always better in the past" response. It seems to me that we do have some concrete evidence of that. Also, the nature of the complaints is so consistent that we have to take them seriously, and the nature of the complaints is consistent with a system in which we have made exam questions so transparent that we have narrowed children’s learning.

Chair: Charlotte, you wanted to come in-briefly.

Q354 Charlotte Leslie: It is very brief and it has mainly been covered. I am going to give you an anecdote, and I want your reaction to it and what it says about the exam system. My subject was classics-Latin-and not an awful lot changes in classics and Latin. There is a certain scope in its translations, but there is not an awful lot of innovation; you just need to be able to translate a piece of grammatical text. I remember when I looked at previous years’ entrance papers at my university, going back 10 or 15 years, those papers, for the kind of Latin that it was and the technicalities, were far more difficult than those that I was prepared for. Looking again over the years, the standard of Latin is not as difficult as that which I did. Certainly, when I was at university and was looking back over the previous entry level, it was very much more difficult. What does that suggest?

Tim Oates: These matters are complex, in as much as a lot of things are happening simultaneously. Therefore your appeal to precision is very well founded, Chair. I will try to link all the points from the last five minutes.

We can look at an independent voice-Rob Coe’s work with the team at Durham, which was previously under Peter Tymms. It is interesting that they have raised questions about the trajectory of standards based on independent measures. They are concerned that there are components of that increase in the numbers getting the higher grades. Components within that relate to grade inflation and not to improved performance in the education system. They have particular concerns about the level of outcome going down in the sciences and approaching that of other subjects over a long period of time. They have raised that consistently in report after report.

However, their conclusion is not that you need a single board; their concern, like Jo-Anne’s and Michelle’s, is very much that there are inadequate qualifications in certain areas. We do not have enough in some areas. That, again, is corroborated by independent work done at King’s by Jeremy Hodgen. We do not have enough maths qualifications, post-16, to meet the genuine needs in society and in the economy.

Q355 Chair: Sorry, I did not follow the link. You were saying that there is some evidence of grade inflations, but you then said that they conclude that there need to be more qualifications. I struggle to follow the link between those two.

Tim Oates: Okay. Taking all the evidence into account, their conclusion is that we have big gaps in our education system in terms of meeting needs. The link is this, and let us take physics, for example. In the time that you are talking about-this is another part of the jigsaw-how many people now aspire to go to university? Many more than did in the 1960s, when the percentage of those going into higher education was in the lower teens. Now, almost 50% go into higher education. Of course, the spread of ability will be greater. Higher education has accommodated that, often by curriculum changes and changes in demand in the first year. I shall come on to Alison’s concerns about number shortly. There have been changes in the cohort going through to higher education, and our notion of what we should assess and put in A-levels has changed commensurately.

Rob Coe’s work on physics A-level shows that the subject appears to have got significantly easier. The discourse around physics is, "We have to make it more accessible. We have to encourage more people to take it-there are not enough as it is. Not enough girls are taking it. We have to do something about all that." Accessibility has become a big issue, so everyone has striven hard to make physics accessible, and to change the nature of the questions and syllabus to get more people interested. The net effect has been to make physics easier.

Q356 Alex Cunningham: Is that because the exam boards are deliberately making it easier so that they can sell the exam to the school?

Tim Oates: This is the point that I am making. There are many things going on here. It is clearly the case, from the stats-I have them all here-that the proportion of people getting grade As over time has increased in almost all subjects. It has increased hugely in areas like mathematics.

Q357 Chair: At the same time, the number of people taking A-levels has massively increased, too. You would therefore expect a diminution in overall quality, yet the percentage getting the top grade has massively increased, so exams must have become easier. One could argue about whether they have or have not-and some people want to argue about it. The question then, I suppose, is: does it matter and is our system better as a result? We want to educate a wider number, therefore we need a metric that differentiates between people in lower achievement than previously would be bothered, because they were just going to leave school.

Tim Oates: This is to do with purpose. We stuck our head above the parapet a couple of years ago. We said, "Look, a number of things have contributed to this rise in the number of people getting the highest grades. One of those things is more efficient teaching and more appropriate provision in schools." People are selecting courses that they are more motivated to do, not least because they can drop one subject in the first year of sixth-form studies and concentrate, in the second year, on those that they are good at and that they are most motivated in, which is a good thing. We also said there are components in there that we are not content with, which are deep in the arcane processes of awarding and which could be removed by more assiduous technical focus and better regulation.

Q358 Chair: Could they be removed by a single examination board, so that we did not have the competition just referred to?

Tim Oates: Rob Coe’s conclusion is that these matters of inter-board, inter-specification comparison are highly technical. What you need is a good apparatus for ensuring inter-specification comparability. In a system where you had one board, you would still need multiple specifications in mathematics, so the problem of having a good apparatus for comparability does not go away. The problem is that, if you change the structures, you may be left with exactly the same problems and have the same grade drift and problems of comparability.

The final issue that Alison raised was whether children are going to university as well prepared in key areas. We have looked at the evidence at Cambridge and we think there are two areas where there are significant problems: mathematical and quantitative skills, and academic writing. Those are reflected in changes in the specifications and the nature of the questions being asked in qualifications and for which people are being prepared. The number of extended essays reduced dramatically over the past two decades but has begun to increase again, thank goodness. Candidates are ill prepared in those two areas, we think.

Q359 Chair: And the reason for that-because we are looking predominantly at awarding bodies in this inquiry, with an insight into the wider system-is because of the specification in policy making rather than the nature of the way we organise exams?

Tim Oates: Modularisation had a profound effect. Everybody blames each other for modularisation.

Q360 Chair: So it was policy rather than structure that led to it. That is what I was trying to get to.

Tim Oates: Yes, precisely.

Q361 Chair: We have a lot more to get through, but I know Alison and Michelle are both waiting to come in.

Professor Wolf: Yes. We keep coming back to the issue of one awarding body. It would be really helpful to distinguish between the question of whether there are particular key examinations where you are very concerned to be certain that there is a very clear national standard, and the question of whether a single awarding body could usefully do everything. It is important to remember that we are a big country, given the level of specificity and reliability we demand of our examinations. If you had a single awarding body-a lot of the things that Jo-Anne and Tim raised the importance of-having new ideas and flexibility would be lost. I cannot think of any reason why, in the process, we would raise substantive standards in Latin, for example, which has probably plummeted more than any other single subject.

Q362 Chair: Thank you. Can I cut you off? Michelle.

Dr Meadows: All I wanted to say was that, where you have examinations that are testing a pre-defined set amount of subject matter, and you have question papers available and mark schemes, then outcomes will go up over time. Performance on those examination scripts will look better. We are calling that rise in outcomes grade inflation, which is kind of a negative terminology.

The other thing I wanted to say was that if one looks at one awarding body countries, such as Scotland, and how their outcomes have gone up over time in their equivalent qualifications, they almost match identically what has happened in England. So, one awarding body does not solve that.

Q363 Alex Cunningham: Are children going through the Scottish system better equipped than those in the English system when they get to the door of the university? Yes or no would be helpful.

Professor Wolf: Not obviously.

Q364 Chair: Jo-Anne, do want to say anything?

Professor Baird: Obviously, I am going to say yes.

Chair: Tessa, do you want to come in?

Tessa Munt: Michelle, could I ask you to move your bottle away from your microphone, because I am finding it hard to hear you?

Q365 Pat Glass: I think what I am getting from that is that none of you would suggest-or rather, you all agree that one awarding body is not the answer. Is that right? Anybody disagree?

Professor Baird: I actually don’t have a view about the number of awarding bodies. That is what I would say to that. I don’t think that is the problem.

Dr Meadows: It is not the answer to the problem that we seem to have.

Q366 Pat Glass: Given that the Secretary of State has said that he will look at this and that one awarding body is clearly on the agenda, what do you think are the key factors that he needs to take into account? Jo-Anne, you mentioned innovation, but there are things like cost, comparability and so on, so what are the key factors that he needs to consider when he is looking at the whole question of having one awarding body or fundamental structural change in the exam system?

Professor Baird: If there is one awarding body, one of the key things that has been raised in many of the submissions to this inquiry is independence from Government. If we want to measure the quality of our education system through exams, which is another purpose to which it has been put, how we can guarantee that there will not be political influence, however indirect, upon a single awarding body?

Tim Oates: For me, the main problem is the issues that have given rise to concerns over public confidence. Would they disappear overnight if you had a single board? No, not at all. In fact, they would all be present in the system still. You would need elaborated statistical apparatus to ensure comparability over time between the specifications. That is the area that we really need to attend to. In addition, where have the biggest administrative failings occurred in public assessment in this country? The single problems that have affected the greatest number of pupils in schools have been failings in national assessment. They operate on a contracted model to single bodies. That is a very salutary fact, I think. You focus risk, it is extremely expensive and you do not render yourself immune from system failure at all. The French system has had some significant failings only in the past couple of years, in security, administration and so on, and that is a very dirigiste top-down model.

Pat Glass: So, operational risk.

Q367 Charlotte Leslie: Just going back to the idea of having one exam board for each subject, to what extent, if at all, do you think that that would contribute to the aim of stopping the so-called race to the bottom?

Professor Wolf: I am not sure that there is any evidence of a race to the bottom fuelled by individual exam boards. What you have done is created a regulatory structure that infantilises the exams. You have created a regulatory structure that treats all exams the same, which puts all the emphasis on procedural matters, which demands a level of clarity, and supposed transparency and reliability in question setting, which forces all exam boards to set papers that, in a sense, become highly transparent and therefore force down standards. That is therefore not relevant. That will happen just as much with a single exam board if that is the policy-making impetus. I have to say that I think that it is impossible to have a system that is not under political influence. If you therefore have Governments that are determined to raise outcome measures and count qualification targets, you will also have a political system that tends to produce that kind of regulatory pressure on the system.

Tim Oates: Maintaining standards in times of change is one of the most significant challenges to any assessment system.

Q368 Chair: Is it the right thing to do? You have said that, because of the changes in the demographics and the numbers going to university, if there has been grade inflation in different components over time, as long as it is fit for purpose who cares whether an A is a lot easier now than it was 20 years ago, if it suits the country, universities and employers better? If it was all right for it to happen in the past, why do we suddenly want to freeze it now and say that maintaining standards trumps every other social and economic need? I hope I am not putting words in your mouth.

Tim Oates: That is a point, absolutely, and one that was well made. I was also going to make a point about the history of change in respect of qualifications. When one looks at the data, some of the points at which it has been most difficult to maintain standards in a way in which we consider to be absolutely watertight is during times of profound structural change: Curriculum 2000, modularisation of GCSEs and so on. The point is, where did all these policy changes originate? As you say, if there are not grounds emerging from the subject itself for fundamental change, I think that that change should be questioned. Many of the changes that have been introduced have been on grounds associated with other than maintaining probity in the qualifications and the subject, ensuring access to the subject and so on. If you repeatedly change qualifications, that is a principal threat to standards. The problem with a single board is that it makes it very prone to constant change in the structure of qualifications, which threatens content standards and, thereby, outcome standards. That is a really critical issue to which we have drawn attention repeatedly.

Q369 Charlotte Leslie: Going back, first, to what we can learn from other countries that have a franchise system, both in terms of faults and strengths, is there any merit in the idea that having a franchise system would help to concentrate examiner expertise, especially in shortage subjects, and to change the nature of those subjects themselves?

Professor Wolf: Yes, I think there is. Interestingly, one of the other problems we have had in this system in the last two or three decades is an insistence from Government that everybody should do everything. So whenever they introduce new qualifications, such as diplomas and so on, there is huge pressure for everybody to offer it and for there to be "competition" in the system. There is a huge benefit in franchising, actually. One of the major mistakes in the current regulatory system is to talk as though this were an ordinary competitive market, like mobile phones, when it is not. So, yes, I think it would be very useful.

Dr Meadows: I was thinking about the Swedish system, which people have been very interested in, where I believe there is a franchising system in the sense that universities have responsibility for various subjects. You are absolutely right: I believe that one of the advantages of that is that you have people with real content knowledge, expertise and pedagogy expertise. However, one of the problems is that they do not have good test development expertise always. It is variable, so there have been some concerns there about the reliability and validity of their examinations. They have had some problems with grade inflation, too.

Tim Oates: That is understating it. There have been massive problems with grade inflation in Sweden-massive.

Dr Meadows: In any franchising system there are two areas of expertise. There is the content-subject knowledge and examiner expertise-and then there is the assessment and test development expertise, and ensuring that there is sufficient of that to go round.

Tim Oates: But do not be fooled as to the possible savings and efficiencies that could be delivered. It depends on the different areas that you are describing or are interested in. The amount of marking that has to be done is just the amount of marking that has to be done. If you have that number of candidates and spread them across three boards, there is still the same amount of marking.

Q370 Chair: But the leadership would be concentrated. If OCR got English GCSE, every marker and every expert on examining English GCSE would be concentrated in that one board. Maybe you would have better leadership and more consistency. I think that was the question, so is there anything in that?

Tim Oates: You have to be very clear about where the benefits derive. I also think you have to be cautious. I absolutely agree with Alison about the notion of it being an imperfect market where market mechanisms do not necessarily drive in the direction you expect. You have a lot of minority subjects that do not generate a surplus at all. They are extremely costly for bodies to put on.

Q371 Chair: So one of the disbenefits of franchising would be that you could have a lot of orphan subjects that no one was prepared to do anymore. Is that right?

Tim Oates: Who would want them?

Q372 Chair: Three of you are nodding; Alison isn’t.

Professor Wolf: I do not understand why that is any different from now. If you put them on at the moment, why wouldn’t you put them on under franchising?

Tim Oates: You have to determine where you allocate them. That is correct, but it gives rise to a problem, which is the concentration of expertise. If you think franchising brings you a benefit in terms of choice as a person contracting, we have seen that the reality of national assessment is that your choices then become profoundly limited. Over a period of time, the expertise becomes entirely concentrated in individual institutions. If you are unhappy with the performance of that institution because of inefficiencies, costs and all sorts of things-the way they have behaved, perhaps, in terms of irrationality, or whatever-and you try to shift it, "Oh my goodness, there is nobody else available with the expertise!" That has happened. That is a reality with national assessment.

Professor Baird: That is absolutely true in terms of the logistics of the operation. What has been quite pleasing in the submissions to this inquiry is that the voice of the subject matter experts-the examiners themselves-has come through to a larger extent than we have seen for a long time in this industry. One of the benefits, perhaps, of franchising would be to concentrate and develop that expertise. A lot of these examiners, as you know, are part-time employees of the examining industry. They have other main jobs, and I think that they have felt disfranchised to some extent by the one-size-fits-all model, and by the concentration in logistics that has actually been a policy push on exam boards in the past decade or so. There could be a clear advantage in having more concentrated subject matter and expertise in the industry.

Tim Oates: But if you professionalise markers to a much greater extent than they are now in terms of regularising their roles and making marking a profession, I do not think that you would see a reduction in costs.

Professor Baird: I think that is absolutely true about markers, but there are thousands of markers for these large-scale subjects. It is the people who design the assessments.

Professor Wolf: Exactly, and you can only professionalise marking with a certain very limited type of question. I have to say that that idea fills me with horror.

Tim Oates: Exactly. I am not advocating it; I am saying that it is a very bad idea.

Professor Baird: It is a bit of a Taylorist model in terms of breaking down the job, but I think there is real expertise in designing the questions and the items in the syllabus, which really need better investment.

Q373 Damian Hinds: Jo-Anne, why are the improvements in GCSE and A-level results not reflected in international tables like PISA, even after adjusting for the usual things that everybody talks about, such as the number of countries in the survey?

Professor Baird: We have seen a very flat profile in PISA. That is true. We are average, and we have stayed average. In some of the other international tests, the results actually look pretty good in comparison with other countries. I think what we have really been saying is that learners and teachers are very strategic these days. We have made it transparent, and we talk about that as though it is a bad thing, but we are actually demonstrating to people what it is they have to do to get the grades in a particular subject. That is why we have seen what we are calling teaching to the test, but throughout our system there was always access to that transparency for some people, who had access to the examiners. It is now the case that it is transparent for everyone. That has its problems, but I do not think that we should be throwing out the baby with the bathwater here, because it also has advantages.

My argument would be that it is likely that this has been produced by teaching to the test and the so-called ratchet effect that you see; Robert Linn has published on this. If you have an examination that stays stable for some time-Michelle Meadows referred to this earlier-you will see the results going up, because the whole system gears itself up to producing better performances in these examinations.

Q374 Damian Hinds: I only have one other question, Chairman, but it is long one. I hope you will forgive me, but I will stop after that.

It strikes me that rising grades are a bit like rising GDP. Part of it is cash inflation and part of it, underlying, is real growth. What you ultimately care about is your real GDP growth compared with other countries. Within that overall thing, the real growth elements could be things like brighter children, more engaged parents, better teachers, better teaching, including better recognition of special needs, and that sort of thing. Within the cash inflation part, you have elements of the syllabus content and breadth, modularisation of coursework, the number of resits, the actual questions set, the stringency of marking, and all sorts of other things like use of calculators, the number of children allowed extra time and so on. That is all within any subject, but at the macro level you could also add in equivalence and subjects and subject mix. Can I ask all of you to give me your top three of those elements-or any other that you think I have missed-that have most contributed to grade inflation?

Professor Baird: To answer the question directly, I would argue that teaching to the test has probably had a huge effect. The education system broadly is now very smart.

Q375 Damian Hinds: So that would be your No. 1?

Professor Baird: Yes, but if it is permissible, I would actually like to answer in a slightly different way.

When the standards are set, we have a variety of sources of information, partly statistical and partly the judgment of the professional examiners-

Q376 Damian Hinds: You are losing me slightly. I do not mind you answering in a different way, but however you want to address the question of what the main component parts are, that is fine.

Professor Baird: When the standards are set, a committee of subject-matter experts sits around with statistical information and information about how the students have performed in a particular exam, so they can see students’ work and they make judgments about that compared with the previous year and so on. I have argued previously that we ought to have supremacy of the statistical information in that process. I think there needs to be quality to the judgment from examiners as well, but if that had been the case, we would have seen more-

Q377 Damian Hinds: Forgive me, Jo-Anne. I am not following you. You have said that teaching to the test is one key element in grade inflation. Are there others that you can identify?

Professor Baird: I am suggesting that I have not always been entirely convinced by the qualitative judgment of examiners when they have told me that their results should go up. I am talking small effects here year on year.

Q378 Damian Hinds: Can you explain that further? I am not understanding.

Professor Baird: I have had statistical information from examiners and qualitative judgments from examiners, and they do not always agree. What we have seen is a small rise year on year in examination outcomes, and this has often been explained by the qualitative judgments about students’ performances in exams.

Q379 Chair: The No. 1 reason, then, for the grade inflation, in your view, is that examiners each year get softer and softer. They see the same answer, and one year they say it is a C. A few years later, it is a B, and a few years after that, it is an A. Why would they respond like that?

Professor Baird: I am saying it is a small effect, and that qualitative judgments are not precise enough to detect small changes.

Q380 Damian Hinds: Is that a posh way of saying that they are getting a bit softer?

Professor Baird: I cannot answer this.

Tessa Munt: Do you-

Damian Hinds: Tessa, do you mind if we go through-

Q381 Tessa Munt: Yes, but I just need to understand something. Are you suggesting that if you took the statistical information about what happened last year away from examiners, and they only judged the quality of answers, you might not see the desire for them to make things marginally easier? If they do not know in broad form what has happened last year-

Tim Oates: Quite the reverse.

Professor Baird: We know the answer to this and, actually, the results would be all over the place. They would be wildly up and down year on year, so we know that without some quantitative component to this process, we would not see-

Q382 Chair: Cohort referencing-is that the term?

Professor Baird: No.

Chair: I knew I shouldn’t have dared to go into that.

Q383 Damian Hinds: Michelle, what are your top three?

Dr Meadows: I would add to Jo-Anne’s list with when qualifications change structure and design. A really good example of this was Curriculum 2000, where the AS-level was introduced into the A-level. What happened there was that students would take generally four ASs, but go on to three A-levels, so they would drop their weakest subject. So, we saw a huge increase there.

Q384 Damian Hinds: Would that be your No. 1?

Dr Meadows: For A-level at that instance, and then what happens is that we carry forward that standard over time. As Jo-Anne has alluded, we get teaching to the test, and within the system of awarding the grade boundaries, the examiners want to give the benefit of the doubt, so they tend to lean on the generous side.

Can I just clarify that very quickly? Increasingly over time, we have used far more rigorous statistical methods and hold the examiners far more to account in relation to the statistics, such that now, even if they are 1% away from what the statistics suggest the result should be, they have to produce reams of very good qualitative evidence of why they think that is the case. So, we have tightened that up.

Q385 Damian Hinds: But that is based on the key stage 2 results of that cohort. Is that correct?

Dr Meadows: For GCSE.

Q386 Damian Hinds: Yes. At GCSE that expectation will be based on the key stage 2 results, and presumably, there is a tolerance either side.

Dr Meadows: A 1% tolerance generally.

Q387 Damian Hinds: Over time, has it tended to err toward the upside or the downside?

Dr Meadows: That is very astute, absolutely. So-

Damian Hinds: You are very kind.

Dr Meadows: Within those 1% guidance limits, yes, there is a tendency to go slightly above.

Tim Oates: Your question has more sophistication, in terms of what you listed, than your requirement that we say three things. What we have said at Cambridge is you need to look at that list and see what research we have in place that tells us to what extent each has contributed. At the moment, we do not think we have enough in place to tell which are contributing in which way.

Q388 Damian Hinds: With the current stock of human knowledge, Tim, and your own skill and judgment, what is your considered assessment?

Tim Oates: I would say that once you have that list, you can then say, "Have we got the appropriate regulatory and technical mechanisms in place to prevent a particular factor that we are worried about from having an undue and inappropriate contribution?"

Q389 Damian Hinds: But Tim, you have spent many years working on all these things. You know more about it than the vast majority of people in the country. Give us a view.

Tim Oates: But I do not want to give a view that would lead the Select Committee in the wrong direction. The point is that 10 years ago, we did not have appropriate apparatus in place in respect of each of those things that you listed, and there are a few more. Therefore, public concern was legitimate over the increasing numbers getting the highest grades. Over the last three or four years, we have really tightened down our statistical processes. The regulator has increasingly said, "If you’re erring, err in this direction, not that direction." But that’s been relatively recent.

Q390 Damian Hinds: Can I push you to give us an indication of what you believe-there is plenty of research on these things; research that your company and others have done-is likely? It doesn’t have to be three; it could be two or four.

Tim Oates: With all those caveats that you have to drive, I believe, sophistication in respect of these matters, change is the top. Change has driven the system in all sorts of directions, which has meant that standards have been technically difficult to control. Teaching to the test and narrow instrumentalism are right the way through the system, and there is a culture of increasing results and an expectation of increased performance in every respect. It kind of pervades the system and insidiously affects individual decision-making on a day-to-day basis.

Professor Wolf: I’m going to give you three. You do have to set priorities, and you’re going to reject mine. At GCSE, the most important thing has been schools giving all their attention to getting GCSE marks, so they are just spending more time on it than they were in the past.

The second thing that is system-wide is the whole constellation of changes, which have tightened reliability, made questions more transparent, introduced modularity and, in so doing, changed the difficulty and demands of the examinations.

The third thing is demographic changes, as you say. It’s a more middle class, more literate, more informed and more ambitious population out there.

Damian Hinds: Thank you.

Chair: Thank you. Ian.

Q391 Ian Mearns: Would you all agree with Cambridge Assessment that the system needs a more robust approach to comparability by the regulator, and that this is more important than the number of exam boards, for instance?

Professor Wolf: Yes.

Q392 Ian Mearns: Excellent. Is that general nods around? Fantastic.

Are you satisfied that Ofqual is taking appropriate action to improve the methodology of its comparability work?

Dr Meadows: I think that Ofqual recognises that it needs in-house technical expertise to be able to run comparability studies to better effect, and to draw on expertise that’s out in the universities and awarding bodies. It is seeking advice when it puts together its studies and listening. I think there is a way to go before you see the robustness of design that we would like, but it is a journey that we are on.

Q393 Ian Mearns: So its methodology is not yet robust enough?

Dr Meadows: I think it is correct.

Q394 Chair: Does it have the capacity?

Dr Meadows: I think that there is recognition that it needs to generate some in-house research expertise to be able to design that kind of-

Q395Chair: Does it have the resource to do that?

Dr Meadows: I don’t know. I think that’s a question for Ofqual.

Professor Wolf: It depends on whether it goes on being asked to do all the things it does at the moment. It seems to me to be perfectly feasible within its current budget to reallocate resources and that that would be the most useful place to reallocate them to.

Q396 Ian Mearns: Ofqual has told us that it is reaching out to assessment experts to help with its work on standards. Have you all had telephone calls from it?

Professor Baird: Can I comment on that? I take from what Michelle has said that it is reaching out. I noticed that Glenys Stacey said that it is considering setting up a standards committee. I think that would be a welcome move. But the problem really is, does it have the in-house resources to respond to the advice that it is given-for example, on the methodological studies that it does on comparability? I think that currently the answer is no.

Q397 Ian Mearns: So it is getting the advice, but it doesn’t have the capacity to take the advice on board?

Professor Baird: That’s right. Partly because it does not have the staff with the expertise, but also partly because of what Alison raises-it has a lot of policy push on it to conduct studies in a particular time scale. These things do take time to design properly. It is also caught because there isn’t a lot of assessment expertise available. People are attracted to different conditions of working than having to conduct studies too quickly.

Q398 Chair: Ministerial diktat?

Professor Baird: I don’t know where the push comes from, to be honest, but I think these are the problems that they face. I also notice that they don’t have an assessment expert on their boards. I think that would be a welcome addition to the board membership.

Professor Wolf: I would like to clarify this. I don’t think it is just that they don’t have the resource because they don’t get given the time; I do actually think they are doing a lot of things, not just because they are told to by Ministers, but because of the complex set of things that were set out in the old legislation, which are probably very low value-added at best and misconceived at worst. That is not the sort of context of this. Therefore, there really does-

Q399 Chair: Such as?

Professor Wolf: Oh, all right, since you’ve asked me, I think all these studies of pricing are completely nuts because this isn’t an ordinary market. I think all that stuff should just come out and then they would have the money to offer a decent salary and set up some sort of contact with the local university. If they get some statisticians they will learn the stuff quite quickly. They basically need some decent in-house statistical help, looking at comparability and technical issues and not wasting time looking at prices. That is my view.

Q400 Chair: Are we at a dangerous moment then? We have the Secretary of State obviously being cross after The Daily Telegraph allegations and with a sense that maybe there is a real need for major change, and you’ve got this Committee doing this inquiry. It is always more attractive to politicians to suggest radical structural change and then watch it happen and have this great sense of authorship as they watch these things occur because they said they should. If Ofqual are not in the best position to counter any misapprehensions, whether of this Committee or Ministers, is this a dangerous moment? Could we see yet another radical reform of structure at great expense with risks for standards brought in because we have not got enough in place to resist it and in fact focus on what the real problems are?

Yes, so all four of you think it could be. The Secretary of State said, "While I have an open mind, I do not rule out that a larger-scale reform might be necessary." From a Minister that suggests that he is itching to do something radical. Do you think he would be unwise? He also said that he was going to come up with some thoughts in the new year. I hope he might wait for the report of this Committee. How dangerous a situation are we in on this front?

Tim Oates: None of us is advising that one does nothing. There are plenty of things that one could do and they would constitute a reform programme. But one needs to focus on the things that would really address the major concerns of both the public and the technicians.

Q401 Ian Mearns: But I was picking up from some of your answers that really what Ofqual needs is some reinforcements in this area.

Tim Oates: Yes, precisely.

Professor Wolf: It needs some very clear restructuring of its priorities and activities-this is a purely personal view-because in a sense what happened in December was a clear indication of the fact that it was so busy looking at the blades of grass that it was missing the forests.

Q402 Ian Mearns: But it was clearly detected that you believed that Ofqual does not yet have the capacity to do everything that is necessary.

Professor Wolf: It has not had time to. It clearly recognises the areas where it is weak-I think it really does-But it is very hard to create a capacity like that overnight.

Tim Oates: I agree absolutely with Alison about the importance of focus in terms of policy attention. We have had too many reforms and I could cite some like, for example, the qualifications and credit framework. It was incredibly expensive. It resulted in a number of small awarding bodies that were servicing the economy in very interesting ways having to reform their qualifications to no good effect. It resulted in the closure of one of them. It cost them hundreds of thousands of pounds, and with no discernable public benefit. That was a structural reform intended to deliver public goods. It did not deliver any. Now some of those reforms are predicated on misconceptions about the system, so there is this mythical figure of 20,000 qualifications, which suddenly entered the discourse about a decade ago. If you look at the number of active qualifications that we had two years ago, it was comparable with Germany.

Now the thing that drove the qualifications and credit framework is that we needed to dramatically contract the number of qualifications that we had in our system. But it is quite clear from evidence from independent researchers that we have huge gaps in our qualifications provision in terms of high-quality vocational programmes from 16, in terms of maths qualifications for those for whom A-level is not suitable and who have already done GCSE. The list goes on and on, so some of these major structural reforms, such as the QCF, were predicated on inaccurate and inappropriate analysis of the problems of the system, resulting in change that was dysfunctional.

Chair: We have very little time left. Jo-Anne?

Professor Baird: I would like to build on that briefly. The changes that we have seen have followed a political time scale and that is not good enough for the education system. We need to look at longer time scales to analyse what the issues are, look at what evidence we have for what works and look at more generational policies.

Q403 Chair: Alison, Tim touched on the number of qualifications. You were widely reported after your report for suggesting that there were thousands of rather poor ones.

Professor Wolf: Again, it is about where the policy decisions get made. It is crazy to have a target for the number of qualifications. What you may well want to do is say that there are parts of the education system where we only want a limited number. I was bound to say that, wasn’t I? That would certainly be in line with the needs of the economy and the needs of the education system. An absolute target is nuts.

Q404 Craig Whittaker: We have touched on grade inflation quite a bit, but do you think-yes or no-that Ofqual’s recent action to contain it was the right thing to do?

Dr Meadows: Yes.

Professor Baird: I’m not sure which action you mean.

Craig Whittaker: To contain grade inflation.

Professor Baird: I think it took appropriate actions, yes.

Q405 Craig Whittaker: Okay. Should somebody be ultimately responsible at national level for the outcomes of GCSE and A-level?

Professor Baird: I am wondering how that would work. Currently, the standards are set by subject committees, so there are thousands of these meetings up and down the country. To unravel that would mean a very different system.

Q406 Craig Whittaker: So no one person ultimately responsible, then?

Professor Baird: Currently no. We would have to radically rethink the system.

Professor Wolf: I can’t think of any country where there is one individual who is responsible for that. I am not sure how that could possibly work in a substantive way. It would probably just create a bureaucracy while you waited for that one person-

Dr Meadows: Currently, responsibility is held at various levels, so the chair of the examiners for a particular GCSE is responsible in one sense, and the responsible or accountable officer of an awarding body is responsible in another sense. I think that is appropriate in terms of how close they are to the awarding of that qualification and what they know about the statistics and judgment of performance for that qualification.

Tim Oates: Craig, an implication of your question is, is it known who is responsible and how the responsibility is enacted in the system? In current arrangements, it is distributed but known. Therefore, I don’t think there should be a policy concern about that. If, for example, there is major concern about the direction of what we call a thousand tiny steps over time in the decision about where to locate the standard-if there is concern that that is leading to grade inflation-do we have an understanding of what we need to do to reverse that trend or change it? The answer is yes, so I think one can offer reassurance on that.

In the past, has there been appropriate policy concern about suppressing grade inflation-that one component among many? I believe that no, there hasn’t, but I think it’s changed and that the combination of regulation and consensual processes in terms of the location of the standard in each year has meant that grade inflation can and could be contained in an entirely appropriate way.

Q407 Craig Whittaker: In your view then, what should Ofqual do if faced with a conflict between its international and standards objectives?

Dr Meadows: It depends on how that international objective is enacted. From what I have observed so far, it seems that what they are really looking at there is the content of the qualifications, and that is about content standards. That is looking at what our question papers assess, which is somewhat different from grading standards. One can change the content of the qualifications while controlling the grading outcomes, such that we get-

Q408 Craig Whittaker: Are saying that standards will drive the other?

Dr Meadows: I’m sorry, I don’t understand.

Craig Whittaker: If they are concentrating on one or the other, will the standards drive the international? Is that what you were trying to say?

Dr Meadows: Yes, sorry. I think there needs to be a clarity about that-which is the driver there. I am assuming that it is content standards.

Tim Oates: Is it sensible to encourage Ofqual to look at what the best in the world are doing? Absolutely. Is it right to make it a narrow technical requirement that it always tracks the standards of other nations? No. That would lead it into dilemmas and contradictions. It is the case that some other jurisdictions have recalibrated in both directions, depending on what their national circumstances are, so it is a finely judged decision. Alison’s point really obtains. If you are interested in getting the right skills and knowledge into the economy and into higher education, is that happening on the back of the qualifications that you have and where you are placing the standards? That is the key question.

Professor Baird: Interestingly, in all this, the curriculum is what we are interested in. Is the curriculum right? We are assessment experts. What Ofqual would need there is some curriculum expertise to make these comparisons between countries. So we have changed the design of Ofqual. It has moved away from a qualifications and curriculum authority.

Professor Wolf: I disagree very strongly with that. I think that it is much better for Ofqual to focus on standards and on technical issues and not to try to second-guess the content of-

Q409 Chair: What about the international peg, which I think was in the Education Act?

Professor Wolf: About looking at things internationally? In a sense I agree with Tim. There are things that it can do and things that it cannot do. It is absolutely right that it should take account internationally. There may be times when you actually want, for policy reasons, to do a really in-depth comparability study on a particular subject and do that internationally. That is a very specific thing that you might want to do sometimes. The more general thing of taking account of it and having a holding brief to look at what is happening for the rest of the world is fine. I rarely disagree so strongly with Jo-Anne, but the idea of recreating a vast curriculum expertise up in Coventry-

Professor Baird: I do not think we are disagreeing, because I do not care who does this. There is a gap in the system for somebody to be looking at the curriculum issues, and I do not think that that gets enough focus currently.

Q410 Neil Carmichael: I was wondering if any of you had any thoughts on the impact of league tables on the pressures of exam boards?

Professor Wolf: I think they are enormous. I will just say this quickly; it was one of my answers to Damian Hinds. One of the most important things that has happened is that schools have been focusing on teaching for exams and to exams more than ever before. That has pluses and minuses, but it is impossible to think about what has been happening with exams and ignore the pressure of the accountability and the league table system.

Chair: Thank you-the four of you-very much indeed for giving evidence to us this morning. It has been a fascinating session. Please do stay in touch with us. The Secretary of State’s appetite for radical change is obviously in place. If you have any thoughts about where that should be directed, further to anything you have already sent us, we would be delighted to hear from you.

Examination of Witnesses

Witnesses: Professor Stephen J. Ball, FBA, AcSS, British Academy representative, Professor Sir John Holman, Senior Fellow for Education, Wellcome Trust, Professor Graham Hutchings, FRS, SCORE Chair, and Warwick Mansell, freelance journalist, gave evidence.

Q411 Chair: Thank you for joining us today. I hope that you were able to hear much of the evidence of the last session as well. We were talking about the Secretary of State’s appetite for radical change. Do you agree with his statement that the exam system is discredited and needs fundamental reform? What do you think The Daily Telegraph investigation in December tells us about inappropriate pressure and competition within the system? Perhaps I will start on the right-Graham?

Professor Hutchings: Thank you, Chair. I think the system is flawed at the present time. In the written evidence we gave, we stressed that from SCORE’s perspective. Of course, I am talking from a science-base point of view, not about the whole spectrum of subjects. In a sense, a lot of the things that have been mentioned before-the teaching to the test, and things like that-have led to what has happened. There has been a disconnect between where the subject expertise is in the UK and how it is being used in getting the assessments and specifications. In science, in particular, the professional bodies have a role to play, but they are used piecemeal at the present time, rather than in an effective way. So the answer is yes, and I could go on for quite a long time about that.

Chair: We will doubtless have time to explore how.

Professor Ball: I would say yes, but not in the way that the Secretary of State intended. I think the system does need radical reform in a way that, in part, disables the Secretary of State from driving the system from the top through the performance-management impact he is able to achieve through changes to things like threshold targets.

Q412 Chair: I think it was Tim Oates who, when pressed by Damian in the last session to name his top three drivers of grade inflation, came out with politically driven change. Is that something you would agree with?

Professor Ball: In a sense, yes. Two would be at the top of my list. One, as the other witnesses emphasised, is the response of schools. The response in terms of the effort that is put into driving up the performance of students is absolutely incredible. The amount of ingenuity, effort, resources, time and energy that are being put into getting more students across the C/D boundary is stunning. From my point of view, that is unequivocally the major driver of the increase in performance, which I would not refer to as standards inflation. But there is a second effect, which is dependent on that, and related to it, but which has not been addressed at all in all this, and that is that that effort relies heavily on targeting some students and neglecting others. There is a systematic effect of concentrating attention on some students.

Q413 Chair: How do you evidence that?

Professor Ball: There is multiple evidence. You have to draw on different evidence-case studies of schools-because you are looking at the practice in institutions. There are multiple examples. Warwick has written about this in his book. I have just finished a study which has been looking at a small number of schools, and what I am saying is overwhelmingly evident in all sorts of ways. I could put together quite a long list of other studies that show that.

Q414 Chair: If that was an offer to provide such a list to the Committee, we would welcome it.

Professor Ball: I would be pleased to put it together. That issue is not unrelated to the nature of our international performance. The issue for me is not simply where we come in the rankings, but what our performance looks like. One of the striking things about our performance is our long tail of poor performance, and part of the reason for that relates back to the way in which, particularly in the last three years of compulsory schooling, more and more attention is paid to fewer and fewer students in an effort to drive up their performance.

Professor Holman: Taking the first point about whether the system is fundamentally flawed-yes, I think it is. We spent a lot of time at the Wellcome Trust preparing our response to you. We looked at the models you were discussing in the earlier session-the single exam board and the franchising-and, of course, at the situation as it is now. What we concluded was that if you were starting out now to design the system, you would not do it this way; you would probably go for a single exam board and take all the steps needed to remove the risks that were discussed earlier, but we are where we are. We concluded that the uncertainties about transiting from where we are now to a single exam board or a franchising system were pretty huge, and the risks of unintended perverse consequences were very high. In the end, we have come down on the side of saying that the system is fundamentally flawed and needs radical change, but that should be done in the current framework, whereby there are three awarding bodies. There does need to be some very radical change, and I did not hear that discussed in the previous session.

Secondly, on your point about The Daily Telegraph revelations, this has just heightened an issue that we know has been going on for a long time. That was probably a fairly extreme example, but there is an enormous amount of sailing close to the wind in the way that those with inside information about the examinations use that knowledge, which, frankly, is in an unprofessional way. The example we saw was simply an extreme.

Warwick Mansell: If I can remember the question correctly, I think there are two things here. You are asking whether the exam system itself is discredited. I would say in general, looking at it, no. GCSEs and A-levels do a pretty good job at ranking candidates. There is an awful lot of technical expertise in the system, and probably a lot more than elsewhere. Also, international A-levels and international GCSEs are very popular abroad as well, so they must be doing something right. In that sense, it is actually quite a good job, if you simply look at the exam system.

There are particular issues around what The Daily Telegraph was exposing, which I have also written about. I have sat in on some of those advice seminars. I chose only two to sit in on, and I found stuff which I think was at least as bad as The Daily Telegraph was reporting on. I think that is quite outrageous really. There is a clear conflict of interest between examiners both setting the exam and then advising people, for money, on how to do well in it; I just do not think it should exist. There are always issues about whether you can actually ban things, but it is morally wrong. I do not think it should happen.

The bigger issue is the pressure on schools to raise results. Having looked at this system for a long time now, I think the system that we have is essentially to set schools performance indicators, however you do that. There are various mechanisms, all still being used at the moment. You have league tables, obviously. The three most important are league tables, Ofsted inspections, which focus very closely still on results indicators, and the closure threats to schools and the various top-down targeting arrangements that are still going on.

Do I think that helps or hinders the system for providing a good education for children? I think it hinders it. It is very English, this system. You can argue the American system has a lot of similarities-and is worse in some places-but, generally, it is what is distinctive about our system: we tell teachers that what matters is this particular number or a set of numbers at the end of it. All kinds of evidence from people like learned societies-for example, the Society of Authors complaining about the issue of textbooks being very exam-focused-are talking about the effect of the learning experience of pupils. I just do not think it is helping our system at all. If you wanted a good system, you would not start from saying that we are just going to assess schools on a few statistics.

I think it goes deeper than what Professor Ball was talking about in terms of focusing on particular pupils. There are huge issues about saying that all that matters about what you learn is that grade at the end of it. I just think that that is a massive issue. What we are essentially saying to the system is that that is all that matters. That is why you get things like these exam seminars, where examiners are operating in a culture which says, "I know this is a bit dodgy, but ultimately all that matters is that grade. You know, that is what we are being judged on. That is what the whole system is being judged on."

Then you look at things like PISA, which finds that there has not been that progress when you change the measure. People acknowledge the side effects and then say that the answer is to look again at the measures and that we need to change the measures, because we cannot change this idea that we have to have statistical monitoring of some kind or put such emphasis on it. I think, actually, no, you need to look at the fundamentals.

Q415 Chair: We probably need to move on.

Professor Holman: Can I just say one quick thing about The Daily Telegraph thing? I think you have to be careful with that one not to throw out the baby with the bathwater. At their best, examiner feedback sessions are good. If they are done professionally, you get the examiner saying, "Oh, we found in question 3 that people didn’t show a clear understanding of the nature of bond breaking. It is very important that this is taught well and so on." And the teachers go away and act accordingly. That is a professional way of doing it. There is a world of difference between that and what The Daily Telegraph revealed, but I don’t think that we should say blanket-fashion that dialogue between examiners and teachers is wrong because, at its best, it can be constructive.

Q416 Chair: All the four examining bodies that came before us when we looked into that said that the provision of those sessions was loss-making. So it is not as if it is a profit driver. The quality of them may help sell their overall package, but in themselves those sessions are not some massive profit-maker.

Q417 Pat Glass: I think you all sat in on the earlier sessions and heard some of the evidence that we were given. People talk about the complexities that sit behind the grade inflation. I think that three of you are scientists, aren’t you? Do you see the conflict that exists? When we taught science and maths in the traditional way, huge numbers of kids were turned off. Yes, there are issues around the modular teaching of maths etc., but are there not children in our universities now studying maths and science who wouldn’t have been there, had we still been teaching in the same traditional way?

Professor Hutchings: I think the way in which children start being interested in science comes a lot earlier than where we are discussing the process before us today. Probably key stage 3 is the crucial area. If you don’t switch them on there, they won’t do key stage 4, and they won’t go through to A-level. That is where the battleground is. You could argue that putting science into the national curriculum and making sure that it was taught at primary level has made sure that we have a far better cohort of people interested in science coming through. I would take it right back. I became enthused as a scientist when I was 11, and that was the first time I was exposed to science.

Q418 Pat Glass: The point I was trying to make is that I have seen classes of children being taught maths when they don’t get decimals. Some of them don’t get it, so they move on and do fractions. Even more of them don’t get fractions, so they move on. The way in which maths is taught now is very different. Some people would call that dumbing down; others would say that it was more accessible.

Professor Holman: Can I just ask what the link is to examinations?

Q419 Pat Glass: Do we have increasing numbers of students achieving higher grades? Therefore, is it easier? The argument that was made earlier was perhaps it is easier, but does it matter? Does it matter because more students are staying with the course longer and are going on to study at high levels?

Professor Holman: If you think about what the situation was before GCSE a long time ago, the GCSE has hugely opened up the territory for people to study, for example, sciences. In particular, it has meant that girls have stuck with physics and boys have stuck with biology, and that has been very important. The same has happened with A-level. It has gone from a sort of minority thing for about 10% of the population to something that is successful for 50%. Those are both very good things.

Not surprisingly, there have been difficulties in going through that 30-year transition in maintaining standards, when you have gone, for example, at A-level, from an exam for a 10% elite who would have all gone to university to one for 50% of the population. There were bound to be difficulties. We heard earlier about the difficulties that accompany change, but the change itself was a good thing.

Q420 Pat Glass: So does grade inflation matter?

Professor Holman: I think that it does.

Q421 Pat Glass: So how do you square that circle?

Professor Holman: You must not let the best be the enemy of the good. It was good that GCSE opened everything up, as good as A-level opening everything up. We must do our best to make sure that grade inflation doesn’t happen because, at its worst, it really undermines confidence, particularly for employers. Universities can keep up with it, but often employers may have a 10-year out of date notion of what a grade C is. Inflation does matter, but we have to manage it and take steps to minimise it. We must not let it stop us making changes to the education system that are more important for fundamental reasons such as widening access to university from 10% to 50% of the population.

Professor Ball: This illustrates one of the issues that the previous witnesses talked about, which is that there are multiple functions and requirements of the exam system. If this Committee were focusing its attention on the number of students who were studying mathematics and science in higher education, the nature of the questioning in relation to assessment may be that the assessment needs to change in order to increase that number, but in fact you are looking at it the other way round, which is whether standards are being inflated. That puts into the background the issues of the participation, involvement and motivation of students and ensuring that we have large cohorts of maths and science students in higher education. You cannot square that circle. It is a policy question in terms of where you put the emphasis.

Q422 Chair: What we are looking at is the organisation of the examination systems. Of course we want to go out to these wider issues, but we come back to the organisation.

Warwick Mansell: I think it is quite revealing that we are even having this discussion about what the improvement of A-level results actually means, because this is the kind of discussion that we have every August. In the media, there is massive debate about that. You are asking whether we have improved the quality of science education and whether we are offering it to more pupils in an attractive way. If you wanted to find the answer to that, you would not use A-levels or GCSEs, because too much has changed about the system. As we have been hearing, there are lots of reasons why those numbers might be going up. We cannot ever be absolutely sure, because there are too many reasons why we cannot be sure what the answer is. Personally, I think that if you wanted national accountability in terms of actually finding out what is going on with education, you would not do it through the current system. You might have a system more akin to PISA, where children are set tests that do not change particularly over the years. There is nothing high stakes about that system, so you can retain questions between years. You could do it in a broader, more in-depth way than PISA by looking at a much broader range of subjects and getting much better information than you get from the system at the moment. That is why this debate continues every year and we never really get to an answer.

Q423 Pat Glass: If we did that, would we not simply start teaching to those tests? The education system is incredibly good at homing in on the accountability measure.

Warwick Mansell: No, because you would not be holding schools accountable through those tests, which is the point. That is part of the reason why the data you get at the end are not particularly good.

Q424 Pat Glass: The exam boards tell us that they do not compete on standards of attainment. Do you accept that?

Warwick Mansell: I have huge respect for all of the previous interviewees, but I end up with the same conclusions that a lot of people reach. I think that the risks and downsides of going to a single board outweigh the advantages, but I think there was not enough discussion about the pressures on schools and what the reality is. One of the main reasons why they look to choose between boards is because they are under huge pressure to raise their results, including the possibility of the head teacher losing their job. That is the case in virtually every school. If the results are not good enough, that will happen.

I have seen examples of a particular exam board essentially sending messages to teachers, "Choose our exams because they are accessible and because a particular format is the right format to raise the results of your pupils." I have written about history teachers who discussed on a forum how they are going to switch boards because a particular board was not providing predictable enough questions and how they would suffer because of that. That is a huge factor, and you would get rid of it with a single board in a franchise system, because they would not be competing on that basis. But, again, I am thinking about a lot of the disadvantages that have been raised.

Professor Holman: Actually, when the awarding bodies were giving evidence to Sir Mark Walport-under the previous Government, he was asked to report on science and mathematics education-he talked to them about this very matter, and they said, "Yes, it is a problem for us." I believe that is written down in the report. Perhaps understandably they would not sit here and say, "Yes, we do it," but they do do it.

Q425 Chair: They wish to have market share and they do compete. As schools are desperate for results, the tendency is to suggest, "Well, actually, you may find that the way we structure it, and all the rest of it, makes it more likely that you will succeed." They will sell schools on all sorts of things-certainty, consistency, and all sorts of other things-as well as suggesting, perhaps, that they do not have to do quite as much. Ofqual’s job is to check whether that is true. Is it possible that there is acceptance among schools, and in the way in which examining bodies present themselves, that there are differences in terms of ease of passing but that the reality is actually altogether different? That is what Ofqual says. They keep looking and they keep doing these standardisation tests, and they say, "Actually, the differentiation is not there. When you compare results between similar cohorts, you do not get different results." So is it just a perception thing?

Professor Holman: It is very difficult to get the evidence, but the pressures are on the awarding bodies to win market share by making their examinations accessible. The example that Warwick gave about the history teachers was a good one, but you can multiply that. People don’t have to sit down and be told, "Make your exam easier," for it to happen. The pressures in the system are all tending to act in that direction, as long as we have a system in which boards compete for market share.

Q426 Pat Glass: So it is all unspoken, but it is there.

Professor Holman: Yes.

Q427 Pat Glass: We asked the earlier panel whether this grade inflation-these increased grades-make students better prepared for university. What is your view on that?

Professor Holman: I am not necessarily talking about grade inflation here, but the cause of it. The main thing that is driving grade inflation is that schools are very focused on exams now. They get very good at preparing for exams, spotting the questions, knowing what you have to say to get the mark and so on. That is making students less prepared for university, because they are being prepared to understand how to answer the question on, for example, chemical equilibrium, but they are not necessarily being prepared for how to understand it, so, when they come to me at the University of York and I teach them about the next stages of chemical equilibrium, I can’t be confident that their knowledge is secure enough, because they will have been prepared to pass the test rather than to have a deep understanding of it. I am not talking about every single school by the way, but this is the risk and that is the direction in which it is driven.

Professor Ball: Can I just say that there is a second order effect of that, namely the extent to which schools concentrate on exam performance. They are not doing other things, and some of those other things are the sorts of skills, thinking abilities and relationships that one would hope that students would be able to make use of when they come into higher education. Those are things that are not being done.

Professor Hutchings: I am in higher education-I am Pro Vice-Chancellor of a Russell Group university-and we have changed the way in which we teach subjects at first year, which is a point that was made earlier, because of the quality of the students and cohorts who come through. There are a few points. One is that there is a legacy issue. Because students are taught to the test and they have the classic question, "Is this going to be in the exam?", that means that we have a non-inquiring cohort of students being brought out from this education system now, compared with, "This is interesting; I need to know about this because I’m going to be studying the subject for a long time." We have to somehow get that across to them very early on in their university career, and I think that that is a very sad thing.

Q428 Chair: Teaching to the test has led to a destruction of curiosity?

Professor Hutchings: It has started-it has not gone totally. There are still very good students and there is not a total erosion, but it is there. As a university teacher, it causes me huge problems, because you are trying to get something interesting across and they say, "Is this in the exam?" They just want to know that.

I also want to go back to the point about grade inflation.

Q429 Pat Glass: Can I just ask you a bit more about that? How much of that is to do with the increase in the number of students going to university? In the past, in the ’60s, it was 7% and they would be the brightest, the best and the most inquisitive. Now, we have 50% of students going to university, so there is bound to be some widening of the ability, skills and curiosity.

Professor Hutchings: That is true, but my comment was made from the perspective of a Russell Group university that is part of the top cohort of universities. We are taking the AAB students that we are meant to be taking.

Q430 Pat Glass: So you haven’t seen any difference.

Professor Hutchings: I would say that, over the past few years, there has been a loss of curiosity.

Q431 Damian Hinds: Just to be clear, although the number of children going to university in general may have increased dramatically, the number going to Cardiff University has-

Professor Hutchings: It has gone from, say, 80 to 100. It is 20%, but it is not a huge amount. In fact, our grades, if we look at the grade points, have gone up during that time, but that is possibly part of the grade erosion that we have been talking about.

Q432 Chair: What about your involvement? The Secretary of State has said that, particularly for A-level, universities and learned societies should have more to do with the construction of the syllabus and so on. What role would you like to have and what needs to happen?

Professor Hutchings: Higher education is one part-this is a point that I wanted to make in relation to the previous part of the discussion. Although it is true that a lot of students coming out go into higher education, a lot do not. From my point of view, we want to have a scientifically literate community, because that is a great benefit to society as we go forward into the future. It is important that we get that scientifically literate society, but higher education is only part of it. They do need to be involved, but I would urge that we involve the professional bodies in the science or, where those do not exist, some sort of excellence framework where we bring together the key people in the subject area, so that you can have them involved. It is not just higher education.

Q433 Chair: Is this like a national subject committee then?

Professor Hutchings: Yes. That is what we would favour, because which part of higher education do you go to? There are at least four groupings. They all have their separate secretariats. Do they speak with one view? No, they do not. Whom you would go to is very difficult.

Professor Holman: In the previous session, you were talking about Ofqual and the question was being asked as to whether Ofqual has enough capacity. Unsurprisingly, the people who were sitting here, who were assessment experts, said that they need more capacity in assessment. Crucially, they need more capacity, or the ability to tap that capacity, in subject expertise. They just do not have it. Its part-predecessor, the QCDA, had a bit of it, but not very much. Predecessor bodies to the QCDA did have it, and there was a time when there were national subject committees, which were truly national and had a strong influence over the exam syllabuses. We need to get that back.

A-level needs to connect much more strongly with the main users of A-levels, which are universities and employers. The way to do that is through national subject committees. How could Ofqual possibly have the kind of depth of expertise to know, for example, what a modern history A-level would be like? You need to have this. In my view, they should be convened by the professional bodies. Any kind of system where you try to have a collection of university heads of departments sitting down together and working out A-levels would not work, but if you take a body like the Institute of Physics, for example, it has strong links to universities and employers and very good education expertise. They would be able to do that job. It would have to be much more than simply saying what goes in the syllabus. It would have to be about an ongoing monitoring of the live examinations. So the national subject committee does not just say what they want to have in the specification; it looks at the individual board’s interpretations of that list and says whether it is good or not. It looks at the sample question papers. It looks at the live question papers. It never stops working. It is always watching and monitoring. That is the way it should be.

Q434 Chair: How would it work for things like maths, where there are lots of bodies?

Professor Holman: Well, it is not going to be perfect. There is not an Institute of Physics for every subject, and I absolutely accept that, but it is not impossible. Maths, for example, has an organisation called the Advisory Committee for Mathematics Education, which is very good.

When a similar question had to be faced about the standard of teaching in higher education, the quality body in some cases went to the Institute of Physics or the Royal Society of Chemistry where such bodies existed. Where they did not, they convened subject bodies to oversee the quality of teaching in higher education. An analogous approach could be taken in this sphere. By the way, such subject bodies should overlook not only A-level, but also GCSE, because the one needs to articulate very well with the other.

Professor Ball: The British Academy would align itself with both of those positions. The British Academy believes it has something to contribute, particularly in the area of humanities, social sciences and statistics. We would not want to see ourselves having too strong a role in the sense that what happens in higher education should not drive what happens at levels lower in the education system, but we would be attracted-

Q435 Chair: How do we address that? Ministers in successive Governments always talk as if everyone is going to get A-levels and go to university. Most people do not, and over 50% will probably continue not to, and yet we talk about the whole system being driven by universities, in part, where most people will not go. Are there dangers in that and how do we avoid them?

Professor Ball: There are clearly. We need a system that is attuned to the needs of all students, not simply those who are going to progress to universities or those who are going to be most successful. One of the problems with our examination system is that it is based on an historical model that was never designed to address the whole of the student population.

We have constantly gone through minor restructurings, which have expanded its range in terms of addressing that population, but there has never been a fundamental reorganisation of what it is we want from exams and who it is they are speaking to. I think it also goes back to Warwick’s point about what we want the system to elicit from students and teachers; what we need them to know and what we need to know about them. The problem with that is the argument that if we wanted to get there we would not start from here.

Chair: Thank you. We have limited time. Ian.

Q436 Ian Mearns: I must admit that as this goes on I am becoming increasingly concerned that people could construe that, instead of producing young people with a good broad knowledge base and inquiring minds who are prepared to learn, we are actually producing-with what we are doing with league tables and the pressure on schools-educational automata who are programmed to answer particular questions in particular exams, and therefore get a result and move on.

John, you mentioned and touched on earlier that the Wellcome Trust has expressed the view that the best way forward to is to retain the current system but with substantial improvements. However, from everything we have heard today it sounds like we really do need substantial improvement.

Professor Holman: We do, but it is not just about the examination system. The results you are talking about are about the very strong accountability framework within which schools operate-league tables and Ofsted, in particular. Maybe you want to make another inquiry into that; I would greatly welcome it. That is what is driving this ever more intense focus on examinations.

Q437 Ian Mearns: I expressed the cynical view that we are not producing youngsters with a broad base of knowledge and inquiring minds who are ready to learn. Would you agree with that?

Professor Holman: You polarised it, which I noted. That is undoubtedly the way that the system is moving. Graham has described a situation that I encounter myself: "Are we going to get this in the exam at the end of this term, Professor Holman?" That is not what you want to go on in our universities. You describe the direction of travel, sadly.

Warwick Mansell: As I am fairly obsessed about this, I want to come back about the accountabilities system. You are talking about inquiring minds, and we have heard about worries from universities that they are not getting independent thinking. A lot of people talk about teaching to the test, but I think there is something quite fundamental going on, in that essentially what the accountability system says is, "We want schools to deliver better results for pupils, come what may." That word "deliver" is often used, which I think is quite revealing. Essentially, if they do not do that, they will be blamed for it.

The production of examination results, in my day anyway, is a system over which teachers have only limited control in reality. I always viewed exam results as mainly a judgment on me, and not on my school or whoever is Prime Minister of the day. We have gone away from that; we have tried to judge all other kinds of people on those results. The reaction to that is that they seek to take more control over that process; they guide closely towards exams, particular exams and content of exams. They are strategic about who they enter for exams. Are those the outcomes we want from the system? I don’t think so; I don’t think that is what universities want. We hear from employers that they want independent thinking. Independent thinking-just letting the kids get on with it-is quite a risky thing if you are a teacher being told that your job is potentially on the line if these kids do not get their results on particular tests.

Having said that, A-levels in particular are reasonably good qualifications, and are certainly regarded that way around the world. That is fighting against that tendency, but it is there. I would say that the transparency of what is being required of students within mark schemes is potentially double-edged. I have seen studies that have shown students looking at the mark scheme and saying, "What I have to do is learn the mark scheme. I don’t need to worry about reading around the subject or anything." The exam board has told you what is going to be in the exam and you focus your attention on that. I don’t think that is the kind of experience that universities are looking for from sixth-form education.

Q438 Tessa Munt: It strikes me that this is just like British Rail in the 1970s, where you are trying to get a train from A to B without actually worrying about the passengers. There is a big similarity, in that what we are trying to do is get kids through exams to get them on to the next bit and not look at the people involved. What I really worry and want to ask you about is what happens when you then introduce wealth inequality, where somebody has the ability to buy cramming expertise. I was going to bring this up with you a little earlier. You were talking about the experience of people coming into top universities. Are there top students? I know that we are constantly trying to level the playing field, but if you introduce wealth and the ability to buy extra tuition-

Professor Hutchings: You can always cram. Any parent who is able to do that for their child is able to help them through the examination system in this country today-definitely. To broaden this slightly, if I am giving the impression that we are getting students coming to us who do not have inquiring minds and are not going to be the great scientists of the future, I want to stop that now-I think we have. We do get a good product out of the system.

Q439 Ian Mearns: I’m thinking of the headline rate, you see, that is the problem.

Professor Hutchings: I know. You talk about grade inflation. It is very small every year and it is perceptible, but is the product that is coming out fit for purpose? That is the question.

Q440 Chair: If we have this narrow, teaching to the test incentive for teachers to control the learning and to keep it focused on what will come up in exams, then it will surely have had the impact of reducing the intellectual curiosity and interest of young people. You have just suggested that that is not the case, and I am confused.

Professor Hutchings: I am just saying it is switched off. You can switch it back on. I do not think it is lost. If it is, then we are doomed as a society, aren’t we? We will have to import all of our top scientists and top innovators in the future, and that has clearly not been the case. We are producing top science still in this country. Clearly, the system is not helping-the system could be improved.

Q441 Chair: Is it that you are a Russell Group university at the top? When I go to an outstanding school or top public school, they ride over all this stuff and do not allow it to narrow everything, because they are self-confident and they know they are not about to be smashed and told they are a failure. If you are serving a very poor community somewhere and you are struggling to recruit the best teachers, however good a head you are, unless you are an absolute genius, the truth is that you are pretty worried. Could it be that the top is all right? The problem, as Warwick said, is what happens to the people at-

Professor Hutchings: We are losing a lot of probably very talented people at the bottom, definitely, that would have come through in the previous system-

Q442 Chair: You probably do not see them, so you are not seeing people who have been damaged.

Professor Hutchings: Nobody sees them, in a sense, because they do not come through.

Professor Holman: You said, "Could it be that the top is all right?" It depends what you mean by the top. Were you talking about the highest-attaining students or the best schools? I think you were talking about the best schools. The best schools will not focus relentlessly and exclusively on preparing for exams. They will, for example in science, do a lot of practical work and a lot of discussion. They will take people out on visits. They will do all those things that enable you to get a broad understanding and an interest, and that is what the best schools do. But you described, Chairman, a school that is firefighting, under pressure and being hammered. What do they do when they are being measured by the number they get at GCSE A to C? They concentrate on the D grades at GCSE, because that is what any rational manager of a company would do in order to maximise the return.

It does come back to the accountability system, but I would not like anyone to go away with the impression that the country is full of schools that are simply sitting children down and forcing exam fodder down them, because they aren’t. The best schools-I use the word confidence-know that if you teach well the exams will come.

Q443 Chair: Under Labour, 30% getting five good GCSEs, including English and Maths, was the floor for national challenge. This Government have said that they will move to 50%, although that is moderated. If you are below 50%, but you show more than average progress-however that is defined-it will not kick in. Is the raising of the floor a good or a bad thing in your view?

Professor Ball: There are two things happening. It is not simply raising the floor, it is changing the indicators that the floor level is based on. There is a narrowing of the range of subjects that are going to be included in the indicator, so there are going to be dual effects in terms of how that plays out in schools. The pressures will increase both to drive up performance, but drive up performance in particular subjects. You may see that as a good or a bad thing. This is one of the ways in which a performance management system is very effective. Perhaps in that respect, it should never be put in the hands of politicians. It is very powerful.

Can I add one other thing to the caveats that my colleagues have put in about what is happening in schools? Teaching for examination performance is not the only thing that teachers do. It may be particularly predominant in years 9, 10 and 11, and increasingly so, but there are other, counter trends. I am particularly thinking of PLTS. I don’t know how many of you have come across Personal, Learning and Thinking Skills, which is a programme and approach that has made enormous progress. It was disseminated widely in schools under the latter period of the new Labour Government. It is aimed at developing the ability to argue and skills of logic, drawing in a programme of philosophy for kids to try to get them working together to think about what they are learning, with an emphasis on learning rather than on assessment. The problem is that that tends to disappear at the end of year 8.

Warwick Mansell: I have been going on a lot about teaching to test, but you have to recognise that Ofsted’s annual report, for example, says that more than 90% of parents answered positively the question, "Are you happy with your child’s experience at this school?" I do not think that accounts of huge numbers of schools being terrible are true at all. I would put that context in.

The floor targets are interesting. It is only one piece of evidence, but there was a report last year from the Advisory Committee on Mathematics Education on early entry patterns for GCSE mathematics. It was very concerned about that, and rightly so. It was worried that some schools in the system are putting children in early for maths. In the worst cases, they get the C and are not even given the chance to improve on that grade later on, which is absolutely scandalous. It looks as though the institution’s need for results is being put ahead of pupils’ needs. But then you look at which schools go in for that most, and a statistic shows that it is the national challenge schools, because the pressures on them to raise results are huge. As I said, judging a school on a couple of indicators will not necessarily produce beneficial education outcomes for the student. It incentivises schools to act in certain ways-sometimes those are in students’ interests and sometimes they are not.

Professor Hutchings: You asked about raising the level. It is likely to exacerbate all the problems that you have been hearing about today. You might want to go from 30% to 50% of a certain grade level, but awarding bodies do not compete on standards, as they say, but on market share. They want to get more of the share of that market.

Research that we have done at SCORE, which we referenced in the report, follows on from the point that has been made on mathematics. Mathematics is not evenly put into the science qualifications. It is not taught, so you can choose an awarding organisation that does not have much maths in its questions. This allows them to get through. This sort of thing is going to drive things in that direction. Having multiple awarding bodies for subjects allows that sort of thing to happen, unless it is regulated properly.

Q444 Chair: The last panel in general were saying that better regulation is the answer rather than changing the structure. What are you saying?

Professor Hutchings: What we want is one body bringing out a specific subject area.

Q445 Chair: For franchising?

Professor Hutchings: Franchising would be a way of doing it, yes. If we had a subject committee for each of the subjects, they would not have to deal with five awarding organisations. They would deal with one awarding organisation, and they could make sure that best practice was driven through. There was a comment in the earlier session about innovation. Innovation is not engendered in the current system because people aren’t encouraged to talk to each other in the old boarding bodies about, "Well, this is a really good idea to do this."

Q446 Ian Mearns: Do you all agree, therefore, that we need some level of organisational reform?

Professor Holman: Absolutely. One thing that has not been touched on, either in the previous session or this one, is the quality of questions. If we have to accept, which we may have to, that we live in a test and exam-driven system, let us have really good tests and exams. Let us make sure that they are very high quality and fit for purpose and, as far as possible, that the questions are such that they lead to the teachers, if they are going to teach to the test, teaching to the test in a good way. Good questions can lead to good teaching. There is no question about that.

One of the problems is that there is such a huge dilution of expertise in question setting. Take science, for example. We have three awarding bodies that between them have five science specifications, each of which has a minimum of two science papers-science and additional science-at two levels, so there are 20 papers in science having to be set each year. It cannot be possible that the questions will always come out absolutely top notch. It cannot be possible, because they are trying to do it at great speed and there are all the things about competing as well. So, a very good reason for reducing the number of awarding bodies, or at least the number of specifications that they produce between them, is to concentrate expertise. This was touched on, but the point that I would want to stress is that perhaps the most important expertise of all is writing the questions that the candidates sit down to-writing, pre-testing, and trying to flush out the glitches and the unforeseen consequences.

Q447 Chair: Is that best served by franchising-giving a subject to an awarding body-or by the creation of a single awarding body?

Professor Holman: Both of those would help, because you would get your best people on to the act of creating the questions, but it is not impossible with the current system. I do not understand why awarding bodies do not collaborate more.

Q448 Ian Mearns: They are in competition.

Professor Holman: Exactly. Okay, let me put it a different way. I do not understand why we allow awarding bodies to carry everything out separately, without any collaboration and behind closed doors.

Q449 Ian Mearns: To be absolutely cynical, some of the exam boards are also owned by publishing organisations that publish the text books. Is there not a conflict of interest there?

Professor Ball: There are issues there. It is important to understand that it is not simply a market in examinations or a market in assessments.

Q450 Ian Mearns: It is a bit incestuous, though, isn’t it?

Professor Holman: It is worse than that.

Q451 Ian Mearns: It’s worse than incestuous?

Q452 Chair: Sir John is a very liberal man.

Professor Holman: I was not going to say there is nothing wrong with incest. It is one thing to say, "It’s insider trading; it’s incestuous; it must be bad." Let us just think for a moment why and what is the risk. The risk is that if an examiner is writing a textbook, he or she may-going through their head when they sit down and write the exam-think, "I won’t put a question on that, because it is not in the book," or "I haven’t done much on that." They will, consciously perhaps, focus the exam on what they covered in their book. That disadvantages people who have not bought the book or schools that have not got the money to do it or whatever. Yes, it is incestuous, but the reason why it is a bad thing is that it carries a risk.

Chair: We have very little time, I am sorry, so we will have to move on.

Q453 Neil Carmichael: I want to talk about Ofqual, but first I want to go back to some of the answers that you have given. What I want to hear is whether or not you think subjects should be treated as a sector. Should there be some sort of structure for that subject, responsible for examinations in total?

Professor Ball: I think that that would be a direction in which some of, if not all, the problems that we have indicated could be addressed.

Q454 Neil Carmichael: Do you all share that feeling?

Warwick Mansell: Are you talking again about having a single exam board?

Q455 Neil Carmichael: For subjects.

Warwick Mansell: On balance, I would not favour that, because for me the disadvantages outweigh the advantages. A major disadvantage is that, in schools, when teachers are dealing with exam boards, if they have a bad experience with one board, they find it very useful potentially to change boards and have that as an exit mechanism.

Q456 Neil Carmichael: So you like the competition element.

Warwick Mansell: In that sense, it is good. There are worries about the standards element, but, in that sense, it is good. Secondly-I think this was mentioned in the previous session-there are massive risks of major structural change in this way and any change to that extent. Thirdly, again you have the risk of greater politicisation of the system-the greater degree of independence than would probably be likely against a system where boards were bidding for the custom, essentially, from the Government rather than from the individual schools. That has worries with it.

Professor Holman: I do not think that we should go for one exam board to one subject, for the various reasons that have been discussed, but I do think that the exam boards have to behave in a totally different way. They need to behave in a way that involves them in collaborating together to produce better examinations in the interests of better teaching. Ofqual, as far as I know, does not have the power to make them do that and I think it should.

Q457 Neil Carmichael: That is partly the next theme of my questions. You think that Ofqual should be completely restructured so that it does have that power effectively to bring about collaboration.

Professor Holman: Yes.

Professor Hutchings: If we default with the current system, then it has to be. I would argue that we should not stick with the current system, and I favour one body for one subject. We can have multiple bodies, but, for a period of time, you want one body for one subject.

The risks that were being pointed out I do not think really exist. I think that this competition between the boards is perceived, because once a school has selected an examining board in a particular thing, it buys the textbooks to go with that-the point about textbooks becomes critical and stops any competition. You have got to break the link between textbooks and the awarding bodies. That is critical. There need to be general textbooks, which schools can go to such that it does not matter if they shift. If we have to have multiple awarding bodies, then if the schools shift, there is real competition, but at the moment they do not because of what is in-built. Do you replace all your textbooks because you are going to go with a different thing? That is a huge cost.

Q458 Neil Carmichael: But what about the accountability mechanism for these sorts of subject area?

Professor Hutchings: Sorry, I don’t understand the question.

Q459 Neil Carmichael: If you are going to have single bodies for each subject, would you say that Ofqual should be the structure to hold them accountable?

Professor Hutchings: There needs to be strong regulation, yes. But we have already discussed, and I think all of us have agreed, on these subject committees, which take the brunt of what is going on, that they need to be regulated, definitely.

Q460 Chair: What is your view, Stephen, or the British Academy’s view, on structure? I am not clear as to what you think we should do.

Professor Ball: I think the British Academy view is that there are some merits in plurality and that there are defences to freedom that lie in the possibility of choice. Perhaps it is that we need to find some kind of middle way, which may be with greater regulation and the role of the subject committees, balanced against some degree of choice. We have to think also that the choice is not simply between boards in relation to different subjects, but is, in relation to subjects, what counts as a subject that is going to be examined. One of the things that exam boards have been doing for the past few years is creating new subjects that can be examined, which in part are being created in order for schools to notch up exam performance statistics. So there are all sorts of factors involved in how the boards operate.

It is also important to recognise that the market in examining is part of a much larger market in educational services, and that market is undergoing massive change in terms of acquisitions, mergers and consolidations, so that we are getting the development of larger and larger education and managerial services companies. The extent to which they would want to seek to compete for contracts for franchised examination systems would, in many cases, inevitably link their activities across areas like text book production, continuing professional development, and the management and improvement of schools. There are all sorts of possibilities for conflicts of interest.

Q461 Neil Carmichael: One or two of your answers just generally have referred to the importance of ensuring that universities and business have an influence in this process. Do you think the single subject route would help or hinder that?

Professor Hutchings: I personally think it would help it. In the case of science, which is what I am talking about, the professional bodies are there. They already have a broad base, as John has said, of representation in higher education, employers and so on. They already have those interests at heart.

Professor Holman: If it were to be a single board-one board, one subject-that would not be sufficient. As we have heard from Graham, it should be linked to a subject committee that had real powers. Might we even consider Ofqual giving all the powers for regulation to that subject body? If that was the starting point, I think you could make that model work. I don’t think the single body is the best model, for the reason I have said, but if you have it you still have to have this strong link with the subject community.

Q462 Neil Carmichael: So you are effectively saying strong subject structure, backed up by the sort of residual powers of Ofqual?

Professor Holman: Yes.

Chair: I think we have to move on. Are there any more questions? No? We are over our time in any case. Thank you very much, gentlemen, for giving evidence to us today. If you have any further thoughts, please be in contact with us because we will obviously be writing a report and making recommendations to Government, and it is important that those recommendations are as well founded as possible. Thank you very much.