Education Committee - Minutes of EvidenceHC 204

Back to Report

Oral Evidence

Taken before the Education Committee

on Tuesday 12 March 2013

Members present:

Mr Graham Stuart (Chair)

Neil Carmichael

Alex Cunningham

Siobhain McDonagh

Ian Mearns

Mr David Ward

________________

Examination of Witnesses

Witnesses: Andrew Hall, Chief Executive Officer, AQA, and Ziggy Liaquat, Managing Director, Edexcel, gave evidence.

Q164 Chair: Good morning, and welcome to this session of the Education Select Committee, looking into the GCSE English exam results in 2012. If you are both happy, we will maintain our informal style, which has the great convenience of meaning we only use first names, and save all pronunciation difficulties. Were last year’s English GCSEs merely a furore, or were they a fiasco, Andrew?

Andrew Hall: What happened last summer was a consequence of something that had been put in a system a long while ago. If you look at the fundamental cause of what occurred, you had the interaction of accountability measures, which have become ever more a focus for teachers and schools over time, with a qualification at the time that was not capable of standing the weight of it, with something like 60% controlled assessment. To a number of people I have described it as taking magnesium and water, both of which are fairly stable components on their own. Put them together and you get an explosion, and that is what we saw last summer.

Q165 Chair: So the primary problem last summer was effectively the cheating within the system, was it?

Andrew Hall: No, it is a combination of accountability measures, those very fine judgments teachers have made, and having a qualification that just cannot stand the weight of it.

Q166 Chair: Yes, but when you say accountability measures, what you mean is that, such was the drive to get the results, in too many cases it was a bit too tempting for teachers, who had 60% of the marks to allocate and their own careers resting on the results of those allocations, to give out marks that were not deserved. Is that not what you are saying?

Andrew Hall: When you talk to teachers day in, day out, as we all do, they make it very clear that they do feel the pressures of the accountability measures, and one of the most common statements you get is: "We, of course, do this properly, but we’re quite nervous about the school down the road." You do wonder how many roads you go down before you come back to yourself.

Q167 Chair: Was it a furore or fiasco, Ziggy?

Ziggy Liaquat: I am really clear that we occupy a privileged position. We are dealing with a precious thing: the life chances of young people. The court and the Ofqual investigation have decided that we carried out our duties professionally. However, we are clear that there was real anxiety and distress for students and parents, and I would like to apologise for that. There are improvements that could be made. Particularly to your point, we could have avoided the situation to some degree by communicating more effectively earlier on, making it clearer that grade boundaries are subject to movement. There is a need for us to really step up our game going forward, particularly as we embark on this substantial reform that is heading our way.

Q168 Chair: So is stepping up the game a technical improvement or primarily one of communication?

Ziggy Liaquat: One of the things this challenge has revealed is that the awarding process is complicated, and we have not done a good enough job in explaining how it works in a very simple way. We thought we had explained it well, but it clearly had not landed, and those grade boundary movements were a surprise. For the January series that we have just issued results for, we called every single one of our customers and said, "Look, we’re not issuing grade boundaries for this series," but we still got calls asking where the grades were. The communication challenge is a significant one.

Chair: Thank you. We have three panels this morning, and a lot to get through. The Committee is a bit denuded by the fact that so many members of this Committee are on the Children and Families Bill Committee, so they are scrutinising legislation as we speak. We have a lot to get through, so I ask for both questions and answers to be as short and succinct as possible. Thank you very much.

Q169 Mr Ward: We have already touched on some of the elements of the first couple of questions, but specifically, for your own organisations, there was clearly a loss of confidence in the system. What specific measures are your own organisations taking to try to restore some of that confidence?

Andrew Hall: There are a number of things I would look at. I will try to be brief on this. Clearly, we along with others recognise that the risk of this particular qualification awarding early units was there. We wrote to the regulator. I was not there at the time; it was something like three and a half years ago. We were not anything like forceful enough, I think, in making our point heard. People have not accused me of not making my point heard, generally, so that is a lesson we have learned. If we have got issues with the design of the qualification, we must be very clear about it.

Much of this issue was around teachers marking to the best of their judgment within a tolerance. Those tolerances were too wide; they were 6%. We have significantly reduced those, and that has been important. The other thing is our feedback to schools; it was previously very much a free text. I have probably read more moderation reports over the last few months than I would care to. They were free texts, and that was designed to be helpful. It had things that had to be covered, but it was possible to interpret those in different ways. You would talk very positively about the administration, and people would maybe interpret that as a positive remark about the marking generally, so we can now go on to a slightly tick-box, less-text approach, but specifically addressing the issues.

Looking forward, we have to protect the next groups of students. I wrote to Ofqual in December, suggesting there should be a radical change to the structure of the GCSE as soon as possible, removing the speaking and listening component, and making that an endorsement on the certificate. It is about having the courage to step up and do those things, and I have been pleased that Ofqual and the other boards seem to think that it is quite a good idea if we do that. So it is that type of approach.

Ziggy Liaquat: So as not to repeat, I will add to what Andrew has said. I will come back to the point around really engaging students, teachers and parents in how the system works. That is going to be critically important as we move to linear, because of course, although modular assessment goes away, there is a residual risk that there are expectations around grade boundaries year on year in a linear system. We should be relentless in communicating the fact that these grade boundaries can move.

The other point I would make is that it would be a real shame if this challenge, this issue that we have faced, is a deterrent to providing appropriate assessment for qualifications. For all of us, the last thing in the world we would want to see is a move to a system where everything is externally assessed in a very safe way, in a kind of multiple-choice way, because we have to offer the right assessments for qualifications that stand up to the assessments that we see in the rest of the world. That is critically important. Our jobs would be a lot easier if we did not do that, but it is right for us to assess the skills and knowledge that employers and higher education need. It is important that we look forward and use this challenge in a positive way.

Q170 Mr Ward: You talk about the system, but do you think your own organisations have suffered a loss of confidence, as organisations?

Andrew Hall: To be clear, there were a number of people that were genuinely upset, and there would be no doubt about that. Yes, there was a loss of trust, and we have to build that back. Part of building it back is being very open and transparent about the issues as we see them. That does not mean just giving in to everything that everybody wants. That does not build trust and confidence over time. You have to accept there was genuine frustration.

One of the most encouraging things, in a way, is that whilst there have been a number of investigations, reports and judicial reviews, the dialogue with teachers has continued. There has been a greater understanding, day in, day out, and the dialogue and the involvement with the leadership of teacher associations and trade unions has continued. We have started that process. Fundamentally, we have to have a system that people trust.

Q171 Mr Ward: I am jumping all over a little bit here, but, Ziggy, you mentioned the issue of not revealing grades for the January assessments. Will that in itself make a difference in terms of avoiding some of the problems?

Ziggy Liaquat: I think so. Fundamental to the challenge we saw was the expectation that was set in January, and certainly for our own organisation, we had to award based on 700 candidates. We did the best job we could in awarding those grades in January, but of course there was substantial movement in June. So moving to a system where we run the awarding or grading of those results at one time gives us more data, more information and more candidate work to look at, to make sure that we get it absolutely right.

Q172 Mr Ward: Moderation seems to have gone wrong here. As I was saying before you arrived, when I was sitting on exam boards, if you are shifting marks at the end, then something has gone wrong in moderation. That clearly must have happened.

Ziggy Liaquat: On the nature of awarding, the central point for me is that the primary focus has to be the quality of the work that you see in front of you; you compare series upon series, year upon year, to make decisions around grade boundaries, and I think that is right. There are improvements to be made in any system, and we are working hard with Ofqual to make sure that we learn the lessons from the challenges that we have faced over the last six months. The spirit of looking forward and looking for improvements in any way we can is consistent across all of the organisations here, and in the Department and Ofqual.

Q173 Mr Ward: More of a technical question now, on the use of comparable outcomes, and in particular the use of Key Stage 2 data as a way of trying to get some consistency year to year. Do you think Ofqual was right to insist on its comparable outcomes approach?

Andrew Hall: From our perspective, AQA has used past performance data for a number of years. This is not something new to us as an organisation. It is very clear to me that you need some sort of guidance as to: "What is there? What is the strength of data now?" Key Stage 2 data is the best available for GCSE.

Mr Ward: It is a long way away.

Andrew Hall: Yes. People will tell you-and they are right-that over five years pupils’ performances change. However, it is valid across a national cohort of 600,000 students. If you are trying to use it to assess what you should reasonably expect in a classroom in the middle of Essex, then it is clearly not appropriate, but using it for national guidance is different. The key is in understanding its uses and limitations, and it is only used as a guide. Examiner judgment is an important part of the overall awarding process.

One of the things potentially misunderstood at some stages was that the Key Stage 2 data was generating a quota. That absolutely was not the case, and the best evidence I can give you on that is for AQA’s GCSE Mathematics, where we went way outside the statistical guidance. We found something in the mix of our cohort, the nature of the entries, and we submitted a very detailed technical report in the summer to Ofqual, as the regulator, and said, "For this reason our examiner judgment, the other evidence we’ve got, says that comparable outcomes isn’t inappropriate here." I would happily share that paper with the Committee afterwards if you want. It proves it does happen. If you use it as an absolute limit, then that is clearly not what it is designed to do.

Q174 Mr Ward: Ziggy, do you think it should continue to be used for GCSE and A-Levels?

Ziggy Liaquat: It is not perfect as a system, and there are improvements. It is as good as any system that I have come across. One of the key points, and potentially one of the misunderstandings publicly, is that somehow an individual’s Key Stage 2 performance caps their particular grade in GCSE. As Andrew said, it is a guide; it is a test to look at the overall performance of the cohort. It is just one of many tests-one of many checks and balances. Like Andrew, for GCSE English in fact, our award was outside of the prediction, and as with AQA, we had good and valid reasons to explain why that was the case. Our award was out of tolerance, and that was okay. Those two examples do demonstrate that it is simply a guide, and that the judgment of senior examiners is key to making those decisions.

Q175 Neil Carmichael: Good morning. With the forthcoming reforms to GCSEs and A-Levels in mind, what are the key lessons for us to take note of following the 2012 GCSE English results?

Ziggy Liaquat: I will return to one of the points I made earlier. Clearly we have learned technical lessons around tolerance. We will be moving to linear, and we need to really engage and educate teachers, parents and students on how the process works; that is key. Fundamentally we share the ambition of the Secretary of State. It is absolutely right to have a qualification system that stands up next to the best systems in world. I personally have been to Singapore and Hong Kong recently to try to understand what that looks like. At Pearson we have got a group of international experts together under Michael Barber to make sure that we build those features into the system.

The key point for me is-and I will say it again-that we do not restrict our ambition in terms of having the right assessment for the right qualifications. Although it may be easier for us, and we may not suffer the bloody noses we have suffered, it is right for students that we assess them in a way that is fit for purpose for what they want to go on to do.

Q176 Neil Carmichael: So in broad terms you agree with the Government’s general direction of travel on this?

Ziggy Liaquat: Absolutely. We are still in the middle of the engagement with the Department around content. We have a lot of research from our international studies that we are providing to the Department. We are also in conversation with Ofqual on more technical issues, so none of those things are set in stone yet. What is positive is that engagement is substantial, and we have a lot of data and information to bring to the table, which we are committed to doing.

Q177 Neil Carmichael: Andrew, you mentioned earlier the issues of accountability, and I certainly have some sympathy with that. So my question to you is: what are the lessons for us in terms of the relationship between Government, the regulators and indeed exam boards?

Andrew Hall: It is one of getting fundamental clarity of purpose on what you are doing. You are right that I refer to accountability. You cannot look to design new qualifications that are going to be used in an accountability system without truly understanding the consequences of that and what will happen in the accountability system. This all needs to be done at the same time. The idea that you can reform a national curriculum, reform qualifications and reform accountability and have them as discrete operations is one I quite openly challenge. You need to have a clear understanding.

It is also about understanding the limitations of what assessment can do. It is to some extent an art, not a science. There will always be margins of error in any assessment, as that happens in anything you do, and you need to understand those. However, it is also about making sure that there is progression in the system-that what you are developing at each stage of the curriculum leads into the qualification. Fundamentally, get it aligned. Let us align accountability.

I have always felt uncomfortable with something like 60% teacher assessment in a qualification that is going to be used for an accountability measure. It is quite interesting; if you take French GCSE, which has an almost identical structure to English GCSE, it does not bear the same accountability rate. It is about looking at that, and understanding what is really going to stretch pupils and encourage them to learn. Having reform is appropriate, but it needs to be reform to a point. The one thing we must do, if we are changing the standard in this reform, is be very clear what the standard change is. If the ultimate goal is to make it more stretching and challenging, if I can say that, be clear about how much more stretching and challenging, and do not have it as some hidden message that creeps out two or three years into the system.

Q178 Neil Carmichael: So you want to see more clarity and transparency in the system, don’t you?

Andrew Hall: Yes, clarity and transparency between the parts of the system. It is impossible, when someone has told you they did not understand something, to say that the clarification is perfect. For example, as the responsible officer for standards in our organisation, every summer I get letters from a teacher that says, "I’m really sorry. I have taught my children the wrong books for the course." Now, that is after a vast amount of communication, so you can always communicate more. However, it is that clarity between the interactions that needs to be tackled.

Ziggy Liaquat: Can I just add something very briefly? One of the things that certainly struck me, having been to Singapore recently, is coherence. There is a single purpose for education, there is absolute clarity, and every organisation is lined up behind it, so you do not have these different tensions that Andrew talks about. In terms of engagement with the Department and Ofqual, we all accept that expertise in qualification and assessment does not reside in one organisation. It is incredibly important that we do work together in the best interests of learners in that respect.

Q179 Neil Carmichael: You have both touched upon this already, but Ofqual’s analysis was essentially that a large number of the problems arose because of a combination of design flaws in the GCSE qualification for English. Do you tend to agree with that?

Andrew Hall: The fundamental problem is the design, set against the accountability measure. You can have an educational debate as to whether controlled assessment is a good thing or a bad thing. You can have a debate about whether modular assessment is good or bad. We might all have particular views around that, but it is that fundamental interaction.

Chair: That is clear, Andrew. Ziggy, do you agree?

Ziggy Liaquat: There are four points Ofqual made; one was around the pressure accountability brings, and one was around design, and I think that is absolutely right. Of course it is quite natural for teachers to strive for their students to do well, and particularly with English, where it is a headline measure. The proposals around the new accountability system are progressive and well thought through, and it is going to be interesting to see how they play out.

Q180 Neil Carmichael: The Welsh exam board raised issues about the 60% weighting, and it was concerned about the controlled assessment aspect of it. Did you have similar concerns, or were you aware of their concerns?

Ziggy Liaquat: At the stage of design, absolutely. Just to be brief, when we were designing the qualification we asked for there to be 40% controlled assessment in English.

Q181 Chair: It was done in about 2009 or something, wasn’t it?

Ziggy Liaquat: Exactly.

Chair: It was the old QCA, so it pre-dated the current Ofqual.

Ziggy Liaquat: It was the interim Ofqual.

Q182 Ian Mearns: Judging from your answer to the earlier question, which was, "With hindsight, what would you have done differently?" isn’t there an element of "you wouldn’t have started from where you started" about that?

Andrew Hall: Probably, yes. There were some fundamental challenges. The point I made before shows the concern we had at the time: our technical team wrote expressing concerns when this qualification was designed and suggested to the then interim Ofqual-which is, to be clear, not the current regime we have-that it would be wise not to award grades but just to issue marks. The regret I have is that we were not more forceful about that, as I said before.

Q183 Chair: It is good to have that mea culpa from you, but WJEC wrote and said, "This doesn’t make any sense." Edexcel wrote and said it did not make any sense, and AQA wrote and said it did not make any sense. You guys can chastise yourselves for not being more forceful, but why were your warnings not heeded? Who called the shots on what led to some of the difficulties we saw last year?

Andrew Hall: The legal reality is that the criteria are designed by the regulator. I just want to draw the distinction between the previous regulator and the regulator we have now, Ofqual. At that time there was an interim regulator that the then Secretary of State had pulled out of the QCA and created as a small body. I am on public record quite often saying it did not have sufficient resources to focus on standards, and that is where the issue arose.

Q184 Chair: Was it independent of Government, or was it basically up to the Secretary of State to decide at that stage?

Andrew Hall: I am not sure I am in a position to judge that. I saw it as an independent organisation. I see the current regulator as more independent.

Ziggy Liaquat: We do bear some responsibility. Ultimately, if we had not met the design criteria, which were 60% control assessment, although we made the case clearly, we would not be offering English GCSE.

Q185 Chair: I am just trying to work out, if all of you were warning against it and someone decided nonetheless to do it, and it was not the current Ofqual, precisely who it was and what status they had at the time. I cannot claim I understand precisely what the legal status of this interim Ofqual was. Was it the Government or was it the Ofqual of the moment?

Andrew Hall: It was the Ofqual of the moment.

Q186 Ian Mearns: From both of your individual perspectives at Edexcel and AQA, are you both absolutely confident that we are not going to find, at any time in the near or medium-term future, a repeat of what has happened over the last 12 months?

Ziggy Liaquat: I am confident that we have looked hard at the issues and challenges, learned lessons, and we are moving to fixing them. Again, central to this was this expectation that grade boundaries would not move substantially, series on series. We are moving to linear, and we are not issuing grades in January. For English, as long as we continue to relentlessly communicate how this works and what it means for students and teachers, I think that will be okay. With a system as complex and high volume as the one that we are involved in, where again, what we are dealing with is something that is so critical to the life chances of students, we have to continue to give the care and attention that we have. However, no system is perfect. I cannot sit here and guarantee we will not face challenges in the future.

Q187 Ian Mearns: You both work for intelligent organisations. Both organisations should have the capacity, in conjunction with your partners in the field, to have learned collective lessons from what has occurred over the last 12 months.

Andrew Hall: My perspective is that you have this: you review what is in your portfolio, you look, and you see if there are other things that concern you right now. We are going through a process of reform of the qualification base that is there now. The most sensitive thing we are doing is the change in standard on GCSE Science, which we have talked about with this Committee before. I believe we have communicated that change as a group of organisations. It was covered in the media beyond belief at the time it was being created.

Do we need to keep reinforcing that? Yes. Is that an area that I have concerns about, in that some schools may not have understood it? It is in the worry mix, of course it is, because you have seen it, so you keep reinforcing the message. Outside that, it is in the process of looking at the GCSE reforms and the A-Level reforms we are going to tackle, to make sure we do not create something again.

Q188 Ian Mearns: From what both of you have said earlier as well, one message you want to get across to everybody out there is: ‘If your exam board is communicating with you as a school, listen very carefully to what they’re saying."

Andrew Hall: Yes, you are right. Schools receive an immense amount of information, not just from us but from Government and from other agencies-teacher agencies. Actually, you go into schools in different places. We will communicate with exams officers, and we will communicate with heads of department. For standardisation for teacher moderation, a member of staff will be sent. We rely on the communication network in the school as well, and that is one of the things we are certainly saying: "Let’s not just rely on one route into a school."

Q189 Ian Mearns: Now, in terms of the technicalities of GCSE English, do you think GCSE English in future should continue to have an element of internal assessment for speaking and listening?

Andrew Hall: I wrote to the regulator in December and said, "I don’t think that’s appropriate going forward." It is an important part of the course, but I believe it should be dealt with through an endorsement, which says you have to teach it; you get a separate recognition on your certificate but it does not count towards the grade. That is where I believe it is, partly because of the challenges of moderating it. It is an ephemeral activity.

Ziggy Liaquat: We definitely need to strengthen the quality arrangements around speaking and listening; that has come through as a clear lesson. You can still do that through internal assessment with strengthened controls, and the key has to be the right kind of assessment for speaking and listening. We just need to maintain that testing and assessing skills in an appropriate way is where our focus should be, and then making sure our controls do exactly that. That has got to be the approach.

Q190 Ian Mearns: Did Edexcel agree with Ofqual’s analysis that teachers over-marked the controlled assessment?

Ziggy Liaquat: We have looked at this, and looked at this very carefully. The thing to bear in mind is that we only work with 10% of the cohort. We have certainly seen inaccurate marking. We have made adjustments to internal assessment decisions both up and down. The thing we have to bear in mind is that Ofqual are looking at the data of the entire cohort, and they have reached the conclusion they have. Although we see inaccuracy of marking in the data we are looking at, it is difficult for us, looking at that data, to make the leap to over-marking. However, we only have 10% of the data to look at.

Q191 Ian Mearns: Yes, but 10% is a pretty good sample, isn’t it? If you do an opinion survey, 10% would be regarded as well within the bounds of discrepancies on either side.

Ziggy Liaquat: It is a substantial number. However, that has to be set against the 100% that Ofqual are looking at. We are in conversation with Ofqual about what we found in our analysis and what they found.

Q192 Ian Mearns: Did you find any evidence at all of significant overmarking?

Ziggy Liaquat: I think we adjusted downwards 8% of marking, and we adjusted up about 5%. So there was inaccurate marking both ways.

Q193 Ian Mearns: Yes, but what I am asking about there specifically is: do you think there was significant over-marking in any individual cases?

Ziggy Liaquat: There was a degree of variety. I would not say there were clear cases of significant over-marking, no.

Q194 Chair: The data shows that outstanding schools tended to do worse in 2012 than in 2011, and in fact the weaker the school, in terms of Ofsted grading, the better it did. You would expect outstanding schools to have better systems of monitoring and maintenance of standards, and is it not the case, therefore, that the weaker and, in some ways, the more desperate the school, the more likely they were to over-mark? Or am I reading too much into it? Are you too reluctant to say it anyway, because these are your customers?

Andrew Hall: No, but if I listen to what a number of head teachers have said to me, they have said similar things, and it is hard, on occasions, to escape that conclusion. After the summer we put a lot of our technical resource into a very detailed investigation looking at mark distributions, and looking at marks needed to get to a particular grade. The thing that has caused me most grief on a personal basis through this is that we think there was some tendency to over-mark in places. We have said so, and I have explained why we think that was. I can well understand that a very confident school that perhaps did not over-mark may feel that its students were somewhat disadvantaged, and it is hard not to have sympathy for that viewpoint.

Q195 Ian Mearns: Were the 2012 papers that you are talking about there, in terms of over-marking or the potential for over-marking, drastically different from those of previous years?

Andrew Hall: It is the first time we have had a modular approach to this qualification in this format; that was the fundamental difference. In the past there has been coursework-it was coursework, not controlled assessment-that had similar degrees of potential challenge in it, but it was not delivered in a modular system. It was delivered with all the grading coming at the end, and therefore it meant some of the issues that could lead people into temptation were not there.

Q196 Chair: So because of the modular nature, it meant they thought they knew where the boundary was.

Andrew Hall: Correct.

Chair: Where there might be a general temptation-because of accountability, to look good-to mark up slightly, without the firm knowledge of what the additional two marks would do, they were less likely to do it. Once they knew that all they had to do was find two more marks and magically the student would become a C instead of a D, perhaps that was another factor that just contributed to over-marking.

Andrew Hall: That information and knowledge must have been a factor in there somewhere.

Q197 Ian Mearns: Did you pass on any information to Ofqual about what, in essence, your customers were doing?

Andrew Hall: The process of how this works, if I can take a minute on this-

Chair: We do not have it, but go ahead anyway.

Andrew Hall: Moderation is a process. It is not re-marking scripts. It is going and trying to see if the school has followed the standard. You have a tolerance because, particularly on subjects like English, History and Geography, there will be a degree of different judgment between teachers. We now know that 6% was too high. Where someone was outside the tolerance, those marks were adjusted; that is a matter of course. The particular challenge was those at 3%, 4% and 5% within the over-tolerances.

We only see that at the time when the award is made. The idea that we have some sort of hidden knowledge halfway through the term of where the grade boundaries are going to be because of the moderation work just is not the fact, so you see that. When we made the award, we adjusted our grade boundaries, as did all the awarding bodies. If this was one awarding body, it might point to something different, but all the awarding bodies did it. In the review meetings we had with the regulator, before the results were issued but after we made our award-which is where, if you like, we are held to account by the regulator-we made specific reference to this point.

Chair: Ziggy might a have different point of view on this.

Ziggy Liaquat: The drivers are there: an incredibly important C grade in English for accountability measures, incredibly important for progress for these people into further education and work, and teachers want their students to do well. The drivers are certainly there. The question I ask myself is: do the data show me evidence of giving marks away to get over a boundary? I am not there based on the sample we have, but I do have to say, again, it is set against the data that Ofqual are looking at, which is 100% of the cohort.

Q198 Chair: Have you seen the WJEC results? In Wales they do not have the same accountability arrangements. There is a chart showing the results of exactly the same exam, and in Wales it is a smooth bell-curve, and the number of people at each mark is exactly where you would expect it to be. Then you see England, and there is a spike at each of the grade boundaries as you go along, which would seem to suggest, unless you are peculiarly different from WJEC, that they provide that snapshot of one world where there is accountability at the boundaries and knowledge, because of modularity, of where those boundaries might be, and one world where it does not matter, and the variation in marks is markedly different. Have you seen that?

Ziggy Liaquat: I am happy to provide the Committee with the data and the analysis that I am talking about. We do see peaks in marks, but way beyond the boundaries eventually set in June. I do not have the data to hand, but I am happy to provide it.

Q199 Ian Mearns: Do you agree with the idea put forward by Ofqual that schools that marked controlled assessment accurately may have seen their students being penalised because the boundaries were raised to address inflation caused by other schools marking generously for their students? Is that an acceptable outcome for an exam system?

Andrew Hall: It is exactly why we should not have 60% controlled assessment in a very high-stakes qualification. We are required to set a national grade boundary for each thing. We cannot set a grade boundary for Sussex and a grade boundary for Surrey, or any particular grouping you like. So absolutely, when you have got that sort of weight-and I know I keep going back to that-you should not have 60% controlled assessment.

Ziggy Liaquat: I will just go back to the point I made earlier, which is that the decisions we make are driven by the quality of work we see in front of us against the grade descriptors. There are other points of data that we consider. Where we ended up in comparing what were two qualifications-there are now three qualifications in English-was outside of the tolerance where we have to explain why we see an increase in the number of students getting to C. That demonstrates the fact that it was not about grade inflation; it was about getting a fair set of grades based on students’ work.

Q200 Siobhain McDonagh: You have already made some reference to trying to tell the regulator about identified problems, but Ofqual has said that as exam boards, you "could have done more to identify and report emerging concerns to the regulators", and that "monitoring by exam boards could have been stronger and more intelligent". What do you understand by this, and what will you be improving as a result?

Chair: Who wants to have a go at that first? Ziggy?

Ziggy Liaquat: Thank you. I have said earlier that I think we identified the problem quite early on. We were quite clear that in a modular system grade boundaries are subject to move, particularly as we are compelled to grade with 700 candidates for every series we offer. As you will have seen from the exchange that we had with Ofqual, which is a matter of public record, we had a very robust and passionate discussion about getting to the right outcome. I accept that there is more that we can do to communicate more effectively with schools, teachers and parents, and there is more we can do to make Ofqual aware.

One of the interesting things in this, and one of the things we need to think about going forward, is that by and large the technical expertise resides within the awarding organisations, and we work together with Ofqual to help them understand how these things could turn out and how we need to manage them. So there is certainly a need and a desire to invest more in that kind of collaboration.

Andrew Hall: From my perspective-and we all have different strengths-the thing that caught us most by surprise was the variability between school groups. The overall English outcome, when we looked at it-and bear in mind we look at the overall outcome-moved by something like 1%, which we could understand from a number of things, such as the changing of the cohort. We had not seen the variability between schools. One of the things I want to look at is if we can find a way-outside the awarding process, because it will be very dangerous if we start to look at school types when we are setting the grade boundaries, and thereby lies all sorts of risk-to have some sort of monitoring about what is happening for types of centres once the award has been made, to get an early warning system. Would it have fundamentally changed the outcomes this summer? No, because there is still a point on which I think pretty much everybody that has looked at this has agreed, which is that the grade boundaries in June were the grade boundaries you should have to get a grade C, but some of the distress could have been mitigated.

Q201 Siobhain McDonagh: How much is cost a consideration when you are establishing moderation procedures for a qualification?

Ziggy Liaquat: I do not think it is a significant consideration.

Chair: If that is your answer, Ziggy, that is clear and fine. Andrew?

Andrew Hall: No, it is not. It is about effectiveness.

Q202 Siobhain McDonagh: The marking tolerance was tightened from November 2012, reducing it from the previous 6%. Did you welcome this change? Who set the new tolerance, and does it now apply across all subjects and exam boards?

Chair: Nice short answers, please.

Andrew Hall: Yes, we did. We all recommended it to the regulator in one roundtable meeting. We needed to tackle it.

Ziggy Liaquat: Yes, and it does not apply to all qualifications.

Q203 Chair: Which qualifications does it not apply to?

Ziggy Liaquat: It applies to English.

Chair: Right, and it does not apply to the other subjects?

Andrew Hall: No. We are doing a piece of work right now testing where it should be for each different subject.

Q204 Chair: When will that be concluded?

Andrew Hall: I need to come back to you on that, unless someone behind me can tell me.

Ziggy Liaquat: It is one of the technical considerations and conversations, particularly around the reform of GCSEs.

Q205 Siobhain McDonagh: Given all your assessment expertise, why did no one question whether a standard tolerance of 6% was appropriate for such a highstakes qualification with a higher proportion of controlled assessment?

Andrew Hall: That was because we did not understand sufficiently when we were setting out on this course the impact of accountability measures. That has been a learning journey for awarding organisations, in my view.

Ziggy Liaquat: That is a fair challenge, and I agree with Andrew. It is something that we need to think carefully about certainly for other GCSEs, but particularly as we look to re-develop GCSEs for 2015.

Q206 Alex Cunningham: As a result of your poor communications in relation to the English exam, many people did not get the A-Level courses they wanted, or they did not get the apprenticeships or the chances that they had there. Some have seen their lives set back a full year because they have to re-sit things, and others have suffered other disappointments. You have said you have learned lessons from this, but what consolation do you think that is to the young people out there who, through your failure, have suffered and seen their lives put on hold?

Ziggy Liaquat: I absolutely understand that it has been a distressing time, and I feel for those individuals. Certainly as a parent, I absolutely understand the distress that it has caused. The key point has to be the point that Andrew made, which is that the decision we made around the grade boundaries in June was the correct decision.

Q207 Alex Cunningham: But you failed to communicate it, so the young people lost out.

Ziggy Liaquat: We did communicate it, but we did not do it effectively enough. Certainly, as I said, we called every single English department we are responsible for and said, "Look, we’re not going to issue grades in January," but we still got calls. So there is much more we can do, and I absolutely accept that.

Q208 Alex Cunningham: So what responsibility are you going to accept? It is a message; we have seen young people’s lives damaged as a direct result of what you say is your failure to communicate correctly.

Ziggy Liaquat: The important point is that those candidates who have a C grade in English have the skills and expertise necessary to warrant that C grade. We have a responsibility to those students, but if we step back, we also have the responsibility to ensure that English as a qualification retains its currency and value as a qualification for the UK system, but also for how we stand against other systems in the world.

Q209 Alex Cunningham: But it is okay for it to have a different value between January and June?

Ziggy Liaquat: No, it is not okay. As I said, with hindsight, because we had to make an award based on 700 candidates, the January award was generous.

Q210 Alex Cunningham: Hindsight is not any use to the young people who got a grade D instead of a grade C. Do you not think somebody in the system should have maybe lost their job because of this?

Ziggy Liaquat: I am not sure. If we take a step back and take a view of where we are now, I completely accept the distress that our lack of communication has caused, and I apologise for that. Everyone in awarding organisations, Ofqual and the Department has learned lessons about how we need to educate people much more effectively about how the process works, and that is the thing we should be focussed on, to make sure that we do not get into this situation again.

Q211 Alex Cunningham: Apologies are no use to the young people out there, but I accept what you said. What steps are you therefore taking to improve communications with schools about the possibilities of changes to grade boundaries along the way?

Andrew Hall: It is about more than just grade boundaries. I made a point before about the number of routes into schools. One of the particular concerns that I have is in the reform process going forward now. We came in front of this Committee once before on the subject of teacher seminars, as I recall, and I think in the end AQA was proved to be innocent on that. However, there is something about how we do launch new specifications in schools, and I really worry that if we overreact to some of the things that others may have done there, we will not have effective communication.

On a number of subjects, I have looked at the launch information we give, and I can provide you with lots of it. It is very clear documentation, and there are clear messages at training seminars: "Grade boundaries will change." Yet I accept not everybody heard that, because they have told me that they did not all hear it. We have to really fight hard when we launch something, because the biggest risk is at the time of change, and it is at the time of change for subjects or new teachers entering the profession that we have to be able to provide really powerful support that does not go to help teachers teaching to the test or fools telling people what an easy exam it is or something.

Q212 Alex Cunningham: I wanted to return briefly to the marking issue, and this over-marking in places. Teachers have been accused of cheating. Is it your view that that has happened, or is it just a case of teachers working within the system but not cheating?

Andrew Hall: "Cheating" is not a word I would use. I have had it made very clear to me by teachers that they do feel the pressures of those accountability systems. When they are making very fine judgments, I can well understand that it influences how they feel. I do not think someone sets out to do something fundamentally bad, but we put them in a position where we are asking them to make really quite complex judgments with 60% of the marks in a high-stakes exam.

Q213 Alex Cunningham: Yes, but the extra mark is the difference between a D and a C. Are teachers cheating by thinking, "Yes, we’ll just allow them the extra mark"?

Andrew Hall: As I say, I would not use the word "cheating".

Q214 Alex Cunningham: What word would you use?

Andrew Hall: I think they are being put in a position where their judgments are influenced by the pressures of the accountability system, and that is natural human behaviour. If you set up a system of measurement, do not be surprised that people use it.

Q215 Alex Cunningham: So they are just being nice? Ziggy, have you got a view?

Ziggy Liaquat: Yes, I think I made my position clear. We have looked at the data and we certainly see inaccurate marking, but I cannot see evidence of teachers giving away marks to get over boundaries. That is based on the 10% of the data that we have.

Q216 Alex Cunningham: So all the stuff we have seen in newspapers and elsewhere suggesting that teachers have cheated is not true? They have just been working within the system?

Andrew Hall: We have not used the word "cheating". What we have talked about is showing you the peaks and troughs of marks near grade boundaries, and that is hard evidence. That is about fine judgments being influenced, but I would not use the word "cheating".

Q217 Alex Cunningham: So, on that basis, do you believe that teachers are very aware of the marking tolerances? There is very clear evidence that they know what they are doing.

Andrew Hall: When we halved the tolerance, what happened was quite noticeable, because you clearly monitor the impact that has, and how many schools went outside tolerance, and the number of schools you need to adjust. Yes, that tolerance was there, and schools were aware of it.

Alex Cunningham: Do you agree, Ziggy?

Ziggy Liaquat: Schools are aware of the tolerance. The thing to bear in mind is that many of these teachers do act as moderators and examiners for organisations like us, so there is an awareness. I will make it clear again. I go back to the data that we have looked at and analysed, and have provided Ofqual; I cannot see that phenomenon happening.

Q218 Alex Cunningham: So you are both satisfied that feedback and moderation will be much sharper in future?

Ziggy Liaquat: The reduced tolerances help focus attention, absolutely.

Q219 Mr Ward: It was almost a blinding flash of the obvious. If the outcomes of football games were determined on the number of corners as opposed to goals, there would be different results. It was so obvious, was it not? Wasn’t there an awareness of how obvious it was that it was going to impact on behaviour?

Andrew Hall: I am in danger of repeating what I said before. When the qualifications were designed, it was driven by curriculum expertise. Whether there was enough expertise in the interim regulator at that time to understand the standards issues around it is open to debate, and I have said so on many an occasion. For awarding bodies the accountability pressures have grown over time. That is not any one party or another; they just have become more and more focussed, and I think there is a recognition of that, which we did not understand three or four years ago.

Q220 Chair: You have taken a lot of stick, and AQA has taken probably a heavier share of the stick. You are the largest awarding body in English. You have relationships with schools who are your customers. Is there not an issue here-when we go back to Alex’s point about miscommunication and children being left with a sense of grievance, which may or may not have been right-that since all this came out, schools have not always behaved appropriately? Do you feel that those who went to court and lost behaved appropriately? Do you feel that they have created misinformation and suggested that, for instance, grades given in June were wrong, when in fact the court, Ofqual and the united bodies have said that the data seem to suggest that broadly, within these tolerances and other issues, they were right? Have you got any criticism for them?

Andrew Hall: What I found interesting was that not every teacher organisation chose to take part in that organisation. Some trade unions did, some trade unions did not, so they all clearly made their individual judgments. I am not always sure about lessons to be learned, because I do not particularly like that phrase. However, what is noticeable now is that for the non-graded exams that were taken in January, the raw marks have been issued to schools. A number of us are already having to hose down the fact that in some institutions people are taking last year’s grade boundaries, turning those raw marks into grades, and saying to students, "This is the grade you’ve got." I have been very pleased that the teacher associations have rallied behind this and are telling their membership that that is exactly the wrong thing to do, because the grade boundaries have not been set.

Q221 Chair: So did some of them use this as a sort of smokescreen? A lot of schools did better last year than they had done the previous year while some did really pretty badly, and that always happens when there is a lot of change brought in. However, fundamentally is that because some schools read the spec and some did not, and those that did not, and let down their children, then used this furore or fiasco as a smokescreen to hide their own underperformance? Is there any fairness to that criticism?

Ziggy Liaquat: It is entirely understandable, for the reasons that Alex outlined-the distress and anxiety-that they represent the views of their pupils and their institutions. One of the helpful things both the Ofqual investigation and the High Court judgment have revealed is how the process works, and how we apply that process professionally and diligently, not for a second undermining the feeling of injustice. So if it helps inform the greater public about how the process works, that has got to be of benefit to all of us in the future.

Chair: A very politic answer.

Q222 Ian Mearns: We now seem to be developing customer and client relationships between exam boards and schools, and between schools and the parents, because the schools want to attract more parents and more pupils at the school, and the exam boards want to attract more schools. Is there not a great concern there that sometimes the truth loses out in all of this, and we are all not willing to criticise each other because we are all in a sort of customer-client relationship with each other? Is that not an overriding concern, when what we are talking about here is a system of trying to educate the nation’s young?

Andrew Hall: All I would say is that it has not felt for the last six months that people are shy of criticising, and we have had our share of both being critical and being criticised. There is something about maturity, of being prepared to say what needs to be said, and that is why I think-it is not just because I am here today-AQA have been prepared to stand up and say: "Actually, before it was fashionable to say it two years ago, we were saying re-sits are really bad, and early entry is really bad. This is hurting our income line, but this is bad education." Organisations have to have the courage to do that, and you have the naïve hope, or the hope, that over time that credibility bears its fruit.

Ziggy Liaquat: Our primary focus is and has to be the longer term value of these students having these qualifications for their careers and their further education. Otherwise, these qualifications have no currency. The challenge and the scrutiny is absolutely right. However, as an organisation, we are in this for the long term, so that relationship issue is also with maintaining a standard, year on year, for qualifications, and also looking at what other education systems are producing for their young people. That has to be our focus, and there will be challenges along the way. I am sure of that.

Q223 Ian Mearns: Andrew, you said earlier on, and it is just something that stuck in my mind, that you did not really like the idea of learning from the experience, or learning a lesson?

Andrew Hall: No, what I meant is that it is the phrase I hate. You do need to learn from everything, but it is too easy a phrase sometimes, and used to deflect: "We’ve learned lessons." It is about really being prepared to learn the lessons, to look at things and account for it. It is just as a phrase.

Q224 Ian Mearns: Do you think it is a throwaway line? Is that what you are saying?

Andrew Hall: It can on occasions be used inappropriately. It is about having the real courage to look at something that has happened and honestly learn it, and just to talk about lessons learned-it is kind of a view. It is a throwaway remark, no more.

Q225 Ian Mearns: So saying it and doing it are two different things?

Andrew Hall: Yes.

Q226 Ian Mearns: Do you think that Ofqual were able to conduct as thorough and impartial investigation into what happened as was necessary or, effectively, do you think it was investigating itself in some respects?

Ziggy Liaquat: You have identified a challenge. Ofqual is obviously deeply involved in the setting of grade boundaries for English. It is an odd situation that they are investigating an issue that involved them. However, if you look at the outcomes of those investigations and the fact that they initiated the process early so that we were able to offer a re-sit in November and to tighten up controls in January, that is a really good outcome from that investigation. If you look at the final outcomes of their investigation against the findings of the High Court, you have to say that they did a good job in identifying some of the issues.

Andrew Hall: It was fine, because you have to look at where the expertise is to carry out an investigation. During the process it certainly did not feel cosy. It did not feel that the investigation was carried out by someone who was integral in the process. It felt very independent, and we were challenged a lot. We put our technical resource behind it, hence the reports you are seeing.

Q227 Ian Mearns: Do you think Ofqual could have been any more preventive in trying to stop all of this happening in the first place?

Andrew Hall: I have made my comment about how I felt when the qualification was designed, and you have to draw the distinction between Ofqual as it is now and Ofqual as it was in its interim phase, particularly in terms of the increased experience and skill that Ofqual have got in the standards arena now, which they did not have when it was formed in 2009 in an interim form.

Ian Mearns: Ziggy, do you have anything to add to that?

Ziggy Liaquat: I would say Ofqual are a young organisation, and it is a challenging time. Like all of us, in hindsight-and I will not say lessons learned-there are things we maybe could have done differently.

Q228 Ian Mearns: Not as a throwaway line, but do you think there are real lessons that Ofqual can learn from this whole sad experience?

Andrew Hall: I think there are. It is about being sure you have the right technical skill in your organisation to understand the issues you are facing and those that are coming. Everything I have seen about the recruitment, frankly including my having to lock down some of my own staff so that they do not get poached, shows that that is happening.

Q229 Ian Mearns: Pretend for a moment that Glenys Stacey is not listening to this. Do you think that they have the wherewithal within them to do the job effectively in the future?

Andrew Hall: They have now, but they did not in their interim form.

Q230 Ian Mearns: Do you think it is right that the fairness of an exam process should ultimately be decided by the courts of justice?

Andrew Hall: I do not think any of us wanted to be there.

Q231 Ian Mearns: As an alternative, for instance, do you think the Secretary of State should have set up an independent inquiry?

Andrew Hall: That is a very different thing. The court is a matter of absolute law, about process and "Did we do the right thing?" There was an Ofqual review, and they did it thoroughly in my view. If you read the reports, they are very detailed. The job was done.

Chair: Thank you both very much for giving evidence to us this morning.

<?oasys [np[pg6,cwe1] ?>Examination of Witnesses

Witnesses: Mark Dawe, Chief Executive, OCR, and Gareth Pierce, Chief Executive, WJEC, gave evidence.

Q232 Chair: Thank you both very much for joining us today. You both listened to the last session. What was said by the witnesses in the last session, either by Ziggy or Andrew, that you most disagreed with, Mark?

Mark Dawe: I do not think I disagreed with too much of what they said. The most fundamental point was around the design of the qualification. If there are lessons learned it is that when all the awarding bodies are saying, "This will not work. This will cause problems," they should not be ignored.

Q233 Chair: Yet they were in 2009 before Ofqual had been made independent, so it was basically a creature of Government, and it nonetheless just ploughed on, and did it anyway.

Mark Dawe: It sounds like they wanted a neat system where controlled assessment was zero, 25 or 60, and you had no option of anything else, so in designing new qualifications we have got to be very aware that every qualification is different. Obviously with threshold measures on English and Maths, there will still be the same pressure on those qualifications. Therefore the design of those probably needs to be different from some of the other GCSEs.

Q234 Chair: Basically, the design at that point was effectively under ministerial control. Do you think we have now got the balance right-before I turn to Gareth-between Government and independent expertise in assessment?

Mark Dawe: You were talking just now about the expertise within Ofqual. We hold years of expertise, with Cambridge offering in around 160 countries. We understand that if we work together and come to an agreement, then that is the best way forward. In the past awarding organisations have made it a skill to make a silk purse out of a sow’s ear, or a quality assessment out of political and regulatory aspirations, so we have had to do that in the past. It is much better if we all agree on what the best way forward is.

Q235 Chair: But is the settlement whereby Ofqual is independent of Government through statute a better situation than we had before?

Mark Dawe: It is. I was not part of that in the past, but from what I have heard, we are in a much better place than we have been.

Q236 Chair: So ultimately having Government deciding on grade boundaries is not healthy?

Mark Dawe: I do not think that is a healthy way forward.

Chair: What do you think, Gareth?

Gareth Pierce: On that point, I agree that regulation should be independent of Government, and also independent of awarding bodies. We have corresponded fairly extensively on that point.

Q237 Chair: Was there anything you particularly disagreed with in what was said?

Gareth Pierce: No, not in terms of disagreement. 2012 was essentially almost a collision between complexity and simplicity. There was huge complexity in terms of the qualifications; there were two parallel but fundamentally different qualifications in English: GCSE English and GCSE English Language. There was complexity in terms of the different uses made of early assessment opportunities, and complexity in terms of the demand in the moderation elements. Then there was simplicity, on the other hand, in the use of data in the regulatory framework for making judgments about awarding standards.

Chair: Excellent. Thank you. We will probably cover some of the same ground we covered in the first session, so please do not feel the need to go on at any length if you essentially agree.

Q238 Mr Ward: We talked earlier on about the need for public confidence and loss of confidence in the system as a whole. You were not involved in the judicial review, but there has now been a judgment. Have you reflected on what that means in terms of public confidence in the system, and what actions are needed?

Mark Dawe: Public confidence is vital and, again, that is the role of the regulator. The regulator and the awarding organisations believed that we followed a process that was appropriate. It is very sad that it went to court, and that is one of the key issues to look at: how on earth we ended up in court-not us, but the other awarding organisations with Ofqual. Where in the process could we have stopped that happening?

It was appropriate that Ofqual investigated it. However, when the interim report came out, it did close down quite a few options. It did not really share the issues and the dilemmas that Ofqual were having to deal with, and at that point, if everyone had been prepared to sit round the table, understand deeply what was going on and realise that there was no perfect answer but that the system needed to come up with an answer, the other bodies would not have been in court.

Q239 Mr Ward: The issue of fairness is often discussed. Russell Hobby from the NAHT said "the system was so flawed that the regulator had no choice but to be unfair to some group of students". Do you think that is true? There was simply no choice-somebody had to be treated badly?

Mark Dawe: If you read the court findings, it reflects what a number have said, which is that the January session was hard to award, and with hindsight was probably slightly generous, and in the summer session the right grades were given and the right boundaries were set. There was an overall fairness, but with some issues around the January session.

Gareth Pierce: From our perspective, we did not have any issues in terms of major differences between January and June with WJEC. However, there are complex questions to do with the impact of the ways in which different schools in England chose between the GCSE English and the GCSE Language route. That was a considerable factor, I think. Linked to that, was the use of data sufficiently robust to ensure that there were not any aspects of unfairness in the way data was used?

Chair: Can you expand on your first point?

Gareth Pierce: I mentioned that there were two qualification routes, and it is unusual in a core subject to have such different routes. Therefore schools were making balanced judgments on which of those routes to use, in most cases, for different subsets of their candidates. Now, on reflection some schools may consider that that was an important factor that may have affected their outcomes and the choices they make this time round; there are those issues as well.

Q240 Chair: One was easier than the other? Is that the point you are making?

Gareth Pierce: It is not so much that they were easier, but they were different. The demands were different, and therefore, in the light of experience, schools will know how different candidates would have fared in those two different routes. Some may be reflecting that first time round they came to one view, and second time round, in the light of that first experience, their view may be different. That is a complexity that is very unusual in a core subject and especially, as we heard before, in one that features so prominently in the various measures of accountability.

Q241 Mr Ward: Still on the issue of fairness, what do you believe should be the overriding fairness consideration when administering examinations?

Mark Dawe: It is about the standard, and the expectation of what a student will have achieved when they get a grade C, a grade B or a grade A. That is where the judgment of the examiners is vital, and we balance the examiner’s judgment, looking at the student’s papers alongside a whole range of statistics, to make sure it is set at the right level.

Gareth Pierce: Fairness also includes ensuring that the learning objectives that teachers and learners have been working towards are fairly reflected in the assessment method used for that course. This is obviously very important in terms of discussing the fate of, for example, the speaking and listening element. If that is a key part of the learning outcomes and objectives, then to be fair to learners, that element must have a fair status in the eventual assessment, so then there is fairness in the design of an assessment to reflect the curriculum intention.

Q242 Mr Ward: I think you were here for the earlier session; I asked a question about the comparable outcomes. It is the same sort of question, but do you have anything new to add in terms of the value? Was too much weight placed upon the Key Stage 2 base for the comparable outcomes?

Mark Dawe: Certainly our approach is, as I said, to look at the examiner’s judgment and then the stats. The judgment normally gives a range, and then the stats help home in on a final boundary or final marks. Regarding the comparable outcomes, if the boundaries had not changed in the summer, the results would have gone up 7%, and we would probably still be sitting here, saying, "How on earth could students’ performance go up by 7% in one year?" There has to be a way of measuring the standard of the students, and it seems to be the best way at the moment, combined with some other factors.

Gareth Pierce: The primary focus has to be on the quality of students’ work, and there is a risk of placing too much reliance on one single statistical indicator. Of course the Key Stage 2 has largely been used in that way, on its own. Therefore the use of that was instrumental in determining whether it was comparable outcomes for England.

Q243 Chair: You do not have that data for Wales-is that right?

Gareth Pierce: No, that is right. The point for England was that comparable outcomes was interpreted as saying the outcomes should come down by one percentage point because of what that predictor model was saying. Now, we do not agree with that view. We think that is overdeterministic as a use of a single predictor model, and our view is that there is always a need for a balanced use of a range of relevant statistical information.

Q244 Chair: The last panel suggested that there was. Surely when you bring in changes in every single aspect of an examination, as we saw last year, then that is where you rely particularly on something like comparable outcomes. Otherwise you have no lodestar by which to make sure that there is not a serious change in boundaries year by year.

Gareth Pierce: Yes, it guides that process, but comparable outcomes is not a neatly defined concept, as we saw in its use last year. For such a large population of candidates, the decision was that the outcomes should come down by a percentage point. That is a particular interpretation of comparable outcomes for England, in the light of, in particular, one piece of predictor methodology, which is the Key Stage 2 one.

Q245 Mr Ward: We have already covered this, but, just quickly, on January/June and the delay, presumably you are supportive of that change, and the delaying of the January grading?

Mark Dawe: In the original discussion we were not. It was a discussion, and we gave our view. We did not feel that OCR had an issue in awarding, and we felt we would have been able to award in January, especially as another year had gone by, and as a qualification gets older it is much easier to award. However, we were not the majority, and it was important that there was a set process for the country as a whole, so we agreed that, overall, that was the way to go. Others felt it was important not to do that.

Gareth Pierce: Our position is identical. The view we expressed was that there was no need to hold back from grading in January but, again, our perspective was based on 2012, where WJEC did not really have any issues between the January and June comparisons. However, we accepted that the ruling had to be binding on all awarding organisations, and across the two countries.

Q246 Mr Ward: Just a different area now: the implications for WJEC and the English exam boards of the recent announcement by the Welsh Government that there will be separate question papers and grade awarding meetings for GCSE English Language candidates in Wales. Any comments on that?

Gareth Pierce: We are obviously acting on that direction or special condition. The view that our awarding committee will take is that they are awarding to a single set of standards, so that will be their point of departure. In a sense, the fact that there is a separate question paper that candidates in Wales would have taken is a bit like having a January paper and a June paper. Their aim will be to set the same standards across both papers, so technically it does not add any particularly different issue for them. It just happens to be another paper for them to look at.

Q247 Chair: So there are not any implications?

Gareth Pierce: At this point there are not any. They will look at it as a paper that has to be awarded to the same standard. Whether there will be any implications or not depends on what the data will show.

Q248 Chair: Are universities, the CBI, IoD and other business organisations and university organisations in Wales happy with the direction of travel and this separation, with Wales having different standards, and arguably possibly lower standards over time than England?

Gareth Pierce: Our line at WJEC is that we are working within a common framework of standards, and that 2012 was one instance where a decision was made by the Welsh Government that in one subject the standard would be different. Now, heading into this year’s awards, we are working within a three-country GCSE framework, and our view is that that three-country framework represents a single set of standards. That is where our awarding committee will start from. If there are issues in the data for our candidates from England and candidates from Wales that pose issues for that awarding committee, we will have to have dialogues with the two regulators. However, at this point, we are working to a single set of standards.

Q249 Chair: Do you think there is a temptation for politicians in Wales? It would appear that Welsh educational outcomes are falling behind England, and do you think that may have played a contributory part in their decision to allow the re-grading and, some would say, the artificial boosting of results in Wales compared with the results in England?

Gareth Pierce: There are two factors we are aware of that were instrumental in relation to the Wales results, and the Welsh Government are aware of these as well. Firstly, schools in Wales made less use of the early entry opportunities-January units and so forth-compared with schools in England. Secondly, there has been a Welsh Government initiative to support some teaching departments of English in Wales in responding to the demands of the new specification. In other words, it is acknowledged that not all schools in Wales responded as perhaps they might have done to the demands of the new specification. So there are those two factors, which did contribute to the scenario we had with the data for Wales last year.

Q250 Chair: Had pupils in Wales been falling behind compared with pupils in England?

Gareth Pierce: Not in the way that we look at the data from an awarding perspective, but of course there is a wider perspective on that, which relates to other aspects of performance data to do with cohorts in Wales and England. I am perhaps not the person that should be asked about that.

Q251 Chair: Going back to my point about universities, have you had people expressing concern, from businesses and universities, about the possible divergence of standards between Wales and England?

Gareth Pierce: No. The 2012 scenario is at the moment recognised as a single situation.

Q252 Chair: No concerns have been expressed to you by universities or employers?

Gareth Pierce: No, not to WJEC, and we are projecting the view that, in these shared brands of qualifications, both GCSE and A-Level, as an awarding organisation working in England and in Wales, we are very much part of that joint use of those brands to a single set of standards. That is our message to universities, employers, parents and learners.

Mark Dawe: The point about a single set of standards is really important, and there is a danger that we end up with a qualification with the same name with different standards, and we have to avoid that at all costs. Really, it is down to the three regulators in Northern Ireland, Wales and England to sort this one out between them, as to how they are going to regulate. Either there are different qualifications in each country with different standards set, or there is an exception, so there is one qualification with one common process for standards.

Q253 Chair: Are you confident that that is going to happen?

Mark Dawe: Not at the moment, because I wrote a while ago to all three regulators asking for clarification, and we have not received that yet. As JCQ, we have done the same, and we do not have clarity yet.

Q254 Mr Ward: The issue is that there was a view, after the judicial review and the deliberations, that that was it. You know, it was a fiasco, we are over that now, and we need to learn the lessons. Isn’t there some temptation to believe that is not the case now-that the uncertainty is going to continue, and comparability will be very difficult, particularly for employers and universities? It is still going to go on.

Gareth Pierce: Comparability is an issue when you have got differences in specifications, and it adds another dimension to the complexity, but it still has to be addressed if it is the same brand. For example, in 2014 there will be a change in the specification for Wales, for GCSE English Language, a year earlier than a similar change in England, if I have got my dates right. Now, they are still going to be within the GCSE brand, so the challenge for regulators and for awarding organisations is to still work together on standards, and still sustain public confidence that a grade C and a grade A, and so forth, means the same even though there are differences in the specifications. As a set of awarding organisations, we have always had this as something we needed to work with, and there are ways of doing so. However, the regulators need to work together on it as well.

Q255 Neil Carmichael: With the forthcoming reforms in GCSEs and ALevels in mind, what do you think the key lessons are from this business with the 2012 GCSE English results?

Gareth Pierce: It is pointing towards greater care being needed in the design of qualifications, and the join between curriculum intentions, assessment design and the pedagogy that is going to be delivering that curriculum has to be thought through pretty carefully when a qualification gets reformed. The choice of assessment method has to be based on fitness for purpose, but also has to be robust, especially in the context in which qualification outcomes are used as data in accountability contexts, and so forth. The lesson learned is that we have to take that rounded view of issues impacting on qualification design from the outset and reach good collective decisions.

Q256 Neil Carmichael: So from what you have seen, do you think the Government is heading in the right direction?

Chair: Mark?

Neil Carmichael: I was thinking Gareth might want to answer that question, but Mark can certainly have a go.

Mark Dawe: The most important thing is what goes on in the classroom, and what the students come out with at the end in terms of their learning, ability, skills, knowledge and understanding. That is what we are looking for, and obviously qualifications have a role to play, but that is about the quality of the teaching, the way of learning and the use of technology. There is a whole range of things. We have got to get that right, and then we have to design an assessment that then facilitates that learning, identifies where it has taken place and encourages high levels of achievement. As long as the different parts of the education system talk to each other and work together, we will get there. The problems have occurred where we have been compartmentalised.

Chair: Thank you. Gareth?

Gareth Pierce: Yes, on the question, "Are Governments going in the right direction?" the answer is "yes". Although there are slightly different emphases in what the purpose of Government policy in England and in Wales might be on aspects of the qualification structure and assessment, by and large both are moving in the right direction.

Q257 Neil Carmichael: The last panel had an interesting discourse about the tensions or relationships between the Government regulator and exam boards. Have you got anything to add to that? What are your thoughts about that relationship, and how do you think it should develop?

Chair: As briefly as you can.

Mark Dawe: It is important that the three work together and sit round a table. Now franchising is out, it means that everyone has come back round the table and is sharing things, sharing their experience and their expertise, rather than all sitting in a dark room preparing for competition. There are definitely benefits for GCSEs there, and we are seeing the same sorts of conversations happening with A-Levels. As long as everyone is willing to put in the work and share, we will get the right outcomes.

Gareth Pierce: The way forward is now allowing awarding bodies to work collectively, in the way we have done in the past, including working collectively in dialogue with regulators and Government Departments. The regulators then need to have a clear understanding within these shared brands of what they need to work on together, what the scope is for doing things differently and, when they do things differently, how they can still join up on a common understanding on standards.

Q258 Neil Carmichael: Mark, the Welsh board did raise concerns earlier about the 60% weighting. Did your organisation do the same?

Mark Dawe: Yes, we did.

Neil Carmichael: What sort of feeling did you have about the response you got?

Mark Dawe: I was not there, but from talking to people we were just told, "This is how it will be." As I said earlier, it was trying to fit a neat model. The regulator at that time said, "We have this model and your qualification must fit into one of these." That forced English to the 60% controlled assessment. It goes back to lessons learned; I quite like the phrase. We must not try to squeeze every qualification and every subject into a small number of rigid models. It is not the way it should be.

Q259 Neil Carmichael: In general you would prefer an examination system rather than an assessment system. Is that the view of both of you?

Gareth Pierce: Are you saying an examination system means written exams at the end?

Neil Carmichael: Yes.

Gareth Pierce: No, I personally do not. Judgment has to be made on the basis of fitness for purpose. Depending on what your learning outcomes are within the curriculum, you design your assessment instrument to reflect those. In many subjects it is important that there are other assessment elements than just an exam at the end of the course.

Q260 Neil Carmichael: So would you want to say that speaking and listening should be included in some sort of assessment in English?

Gareth Pierce: Yes, if they are a key part of the learning outcomes, and therefore they get reflected in the pedagogy for that curriculum, there needs to be a credible way of including them in the overall assessment. Now, obviously that is going to be up for debate, as to whether that can be done robustly. However, fitness for purpose is the guiding rule with assessment design.

Q261 Neil Carmichael: So, Mark, where would you strike the balance between an examination in the traditional sense and assessment?

Mark Dawe: Again, it is subject by subject.

Neil Carmichael: Well, in English, because that is really what we are talking about.

Mark Dawe: In English, like the others, we said that 40% was appropriate. We have to look at whether we really have to assess everything that goes on in the classroom. It is a sad situation if we cannot have teachers teaching in science labs or in speaking and listening, and things like that, unless they are being assessed on it and focusing on that assessment. So we have to look at whether the assessment contributes anything to the grade as well. In some of these cases the assessment does not add anything; it is very narrow, and you do not really get a good result out of it. We have to balance all of those things.

Q262 Alex Cunningham: Way back in the last century when I was at secondary school, I did the GCEs. Alongside that there was a certificate in relation to speaking and listening. Do you think the GCSE English in future should continue to have an element of internal assessment for speaking and listening? If so, could that moderation be strengthened in some way?

Mark Dawe: We both made comments. I would favour having a separate recognition of speaking and listening but not necessarily a contribution to the final grade as one way forward.

Q263 Alex Cunningham: So in the early 1970s it was right and in the 21st century it was wrong?

Mark Dawe: Not all in the past was bad.

Chair: Only the dress sense is regrettable.

Q264 Alex Cunningham: Moving on now, do you agree with Ofqual’s analysis that teachers over-marked controlled assessment? Did you find any evidence of that? Has there been inflated marking?

Mark Dawe: At OCR we did not find evidence. The curves you were talking about earlier were reasonably smooth; there were peaks through the whole range. We had had a modular assessment previously, and we believe that our centres understood how that worked, and how the controlled assessment works in there, so it comes back to Andrew’s point about training and support. We are still waiting, as awarding bodies, to understand what will be seen as appropriate for us to provide to teachers. However, we think it is really important that the teachers understand what is required of them and how they achieve the best for their students.

We are looking at ways of providing as much as we can online, and free, so that teachers can easily access that training and support, but we do need some clarification. With new A-Levels and new GCSEs potentially coming in in the same year, that is an enormous amount of change, and those teachers will need an enormous amount of support. That is normally what we would be giving.

Q265 Alex Cunningham: So you would join others in dismissing the suggestion that teachers have cheated?

Mark Dawe: No, not at all. I do not think I said that.

Alex Cunningham: Sorry, I asked the question the other way round. I was surprised at that answer there. What I was saying is that you would accept that teachers have not been cheating.

Mark Dawe: Sorry, they have not cheated. No, they have not cheated.

Q266 Alex Cunningham: Is that your view as well?

Gareth Pierce: Yes, we would agree with that. The extent to which we had to make adjustments for schools being out of tolerance was no more than we would have expected. Obviously with a narrower tolerance going forward, we expect to make more adjustments in future but, again, we do not regard that as having been a major factor in 2012.

Q267 Alex Cunningham: So there was not anything bizarre about the way that marking took place last year?

Gareth Pierce: No.

Q268 Chair: Are people reading too much into the chart showing the marks for your English exam in England and Wales, and the fact that it looked spiky around the grade boundaries in England and smooth in Wales?

Gareth Pierce: Possibly, yes. The smoothest curve was for our Wales data, the next smoothest was for our England data, and then the least smooth was for AQA’s England data. Our England data was between the smooth and the very obvious patterns. That is why it was less of an issue for us, and it is shown in that data.

Q269 Alex Cunningham: Mark, you have suggested that your teachers got it and were fine, so do you think that some schools have been penalised as a result of problems elsewhere when they have averaged things out or they have raised the boundaries to address inflation?

Mark Dawe: It is really hard to know school by school what was happening. I think it was Andrew who was saying that the biggest surprise to us all when we looked at all the data together was the variability between schools year on year in 2012. There was a concern there about what had happened. It was one of the factors in this-something did not seem right there.

Q270 Alex Cunningham: Ofqual said that as exam boards you "could have done more to identify and report emerging concerns to the regulators", and that "monitoring by exam boards could have been stronger and more intelligent". What do you understand by that, and what are you going to do as a result, bearing in mind that you think you had the ducks in a row?

Mark Dawe: I would disagree with the black and whiteness of that statement. We did a lot, and we do a lot. At the awarding meetings early on, it was clear that English was going to be difficult. We awarded and did not change anything after that awarding. We awarded before this issue really started to flare up. There are regular meetings with Ofqual both during the awarding period-every week we are sitting down together, identifying where there may be issues in particular subjects-and then through the year, where there are particular subjects where we may say, "Well, given what has happened in English, are there others we should be looking at?" We are all doing that, so I do not think it is fair to say that we were not monitoring and keeping a close eye on things.

Alex Cunningham: Is that your view as well?

Gareth Pierce: Yes, I would agree with that. What perhaps took us all by surprise was the significance of the issues that emerged in English, and they did emerge very late. Apart from those design issues, which were way back, it was only very late during the awarding cycle that there was an awareness of the extent of some of these issues. We now know what can become an issue, and therefore there is more of a dialogue on those throughout the year now.

Q271 Alex Cunningham: So the lessons have been learned this time?

Gareth Pierce: Absolutely.

Alex Cunningham: We cannot expect some of the spikes and disappointments that we have seen over the last year. Is that where you are? I see heads nodding. That will be fine.

Mark Dawe: Yes.

Q272 Alex Cunningham: I will go on to the next question. The marking tolerance was tightened from November 2012, reducing it from the previous 6%. Is that a welcome change? Who sets these tolerances, and does it apply across the piece, to all subjects and boards?

Gareth Pierce: Yes. I am just going to confirm the answer you had in the first session: the setting is by agreement across the awarding organisations with regulators, and the change is only for English Language at this stage. However, the benefits that are going to come from that change will cause us, I am sure, to reflect on the extent to which that change should be applied elsewhere.

Q273 Alex Cunningham: If that is such a great idea, why wasn’t someone saying this 6% tolerance was a bit bizarre?

Gareth Pierce: There are issues of practicality whenever a tolerance is set, and also it is in the context of what contribution that element makes. Now, on reflection perhaps we should have realised that an element contributing 60% required narrower tolerances than something contributing half that.

Q274 Alex Cunningham: They existed for a long time, though, didn’t they?

Gareth Pierce: Yes, they did.

Mark Dawe: They did, and there was not a problem. There also comes a point where the narrower it gets, the more you might as well just mark the papers yourselves.

Alex Cunningham: Hopefully we will never get to that stage. The Secretary of State wants to go the opposite way, and just have one exam; that is certainly something that I do not agree with.

Q275 Siobhain McDonagh: What steps are you taking to improve your communications with schools about the possibilities of changes to grade boundaries?

Gareth Pierce: We are building on the dialogue that has been happening over the last six to nine months with schools, especially in terms of English Language. Obviously the unusual arrangements that have surrounded the January series have been part of that communication. There is considerably greater awareness both through our deliberate correspondence with schools, and through the general public discussion. There is a greater understanding now of what grade boundaries mean and the extent to which they do genuinely vary from series to series.

Mark Dawe: Likewise, the concern Andrew voiced around students being told by schools what their grades are, even though we have not issued grades, just shows again that they probably have heard the message, but they are very worried about this threshold measure and the performance of their students. They will be looking to see whether a student has got a grade C or not in their eyes, and whether they should be putting them in again. So it does not matter; we can tell them as much as we like, but there are still other pressures and other issues they will be looking at that will affect behaviour.

Q276 Siobhain McDonagh: Ofqual has said that "moderators’ feedback to schools was not always sharp enough". What do you understand by this, and what steps are you taking to ensure the feedback is sharper in the future?

Gareth Pierce: We have certainly looked at the kind of narrative that we have been putting in moderators’ reports, and one clear distinction we need to make is between saying that their work is perfectly acceptable administratively, which is one thing, and the different comment of saying they were absolutely spot on on the standards. In the way we have been drafting and finalising these reports, we are clearer now in terms of making sure that messages are distinct in that feedback. Some of the messages are to do with process, and some are to do with standards.

Mark Dawe: After the issues arose, we reviewed our guidance and what moderators were writing, and the majority of them were pretty good, and we were not getting this issue of schools feeling like they had been given guidance that they were doing the right thing. However, like all these things, we sharpened it up and we repeated the messages, so that hopefully there was no misunderstanding.

Q277 Mr Ward: I have some questions about the role of the regulator and the ministerial action that was taking place. You did express concerns about the independence of the regulator in its investigation of itself. That was justified by the regulator, but in your view was Ofqual able to conduct a thorough, impartial investigation into what happened?

Chair: Who would like to go first? Were they the right people to investigate this?

Gareth Pierce: In one sense, they were reviewing things that they had not been accountable for from their origins. For example, there are aspects of this that go back to 2009, aren’t there, so the fact that they were not responsible from that stage helped them, in a sense; they could take a detached view on some of the origins of these issues. Of course, they were probably best placed to take an overview of what we had all been doing as awarding organisations, because they had clear access to our information. So probably, on balance, their position was appropriate to report and investigate. Of course, whether that satisfies all parties is a different matter.

Mark Dawe: They were set up to do this sort of job, and to some extent if they are not given that role, then why bother with the regulator? As Andrew said, it felt very uncomfortable in terms of the amount of stuff we were giving and the questions we were being asked, and that was very technical data. You needed someone with the expertise to understand what we were handing over to then be able to judge what had happened, and I do not think anyone outside the regulator would have been able to do that job appropriately.

Gareth Pierce: One other point I could make is a joint regulatory investigation would have been perhaps even better.

Q278 Mr Ward: The regulator argued that the issues surrounded schools and your organisations, rather than the regulator itself. Is there anything the regulator could and should have learned from it?

Mark Dawe: This is where the body changes. They say, "It is not us," but it was the original form of the regulator that caused some of these problems with the original design, so as a whole there is an issue there that the regulator, in its previous form, should take responsibility for.

Q279 Mr Ward: So it was blamefree in terms of what happened last year, in your view. It could not regulate it.

Gareth Pierce: It was independent, to a large extent, of the complexity of the scenario with which we were working. Therefore, it was well placed to comment on that and its implications, but of course it was very much tied into what I described at the outset as too simplistic an approach to the use of data. We are learning a lesson from that as well, and there is a greater openness now to the importance of taking a balanced view of different parts of the statistical evidence.

Mark Dawe: I would go back to the point I made right at the beginning: the regulator had the role of getting everyone around the table and stopping this court case from happening. There possibly were things that could have been done halfway through this process that would have enabled those discussions between all the parties outside court rather than in court.

Q280 Mr Ward: The previous regulator or regime was responsible for the introduction of the system. Was there nothing the current regulator could have done to foresee what would happen?

Mark Dawe: It goes back to Andrew’s point that, staffingwise, the expertise in the previous one was very limited. The current regulator has been building up their staff base. This was happening right at the beginning of the regulator’s life, and I do not think they would have been aware of many of the complexities that they had inherited.

Q281 Mr Ward: Was this just a train crash waiting to happen?

Mark Dawe: We said so in 2009. We said this qualification was going to be a problem.

Gareth Pierce: On the credit side, both Government and regulators in England and in Wales are now recognising what actions need to be taken going forward, and those actions are happening rapidly for English and they are influencing the debate on other GCSEs. It is to the credit of all involved that lessons are being learned and acted upon very quickly.

Q282 Mr Ward: Can I talk to you, Gareth, about the regrading in Wales and your position with regards to that? What are you views on the regrading?

Gareth Pierce: Our position is it is something we were told to do, and therefore we did it, as a direction from the regulator. It is documented that our awarding committee did not find that an appropriate position for them to be in. That is why, in approaching this summer’s awards, they will be getting back to the point of wanting to apply a common set of standards. To an extent, the decision in Wales can be seen as a response to the two factors I mentioned earlier. There were two issues that our data and their investigation of teaching departments would suggest were material factors. It could be interpreted as an approach to give comparable outcomes for the Wales candidates despite those two factors. The two factors I mentioned were: less use of early opportunities, and the issues in terms of some teaching departments in responding to the demands of the new specification.

Q283 Mr Ward: So, in your view, was it the right thing to do?

Gareth Pierce: It was not the right process for our awarding committee to be involved in, because it is not tenable to have different standards for the same qualification. So the question has to be asked in the wider context of those factors affecting candidates in Wales. Having said that we are an awarding organisation that applies common standards, it is for other people to decide whether those other contextual factors made it the right decision.

Q284 Alex Cunningham: Can I just ask a question about the value of the grade now? We have the value of a grade in January, we have the value of a grade in June, and we have the value of a grade remarked in Wales. What is the value of that grade? Is the value of a grade remarked in Wales the same as the mark in January? Where is the value? What I am saying is there are three different elements to that mark and they all have a different value. Would that be your opinion?

Gareth Pierce: Yes, within the WJEC award there are two slightly different awarding standards applied to the grade C, and the grade C alone. In a sense, they reflect the two regulators’ different views on comparable outcomes between 2011 and 2012.

Q285 Alex Cunningham: But to the employer, one is worth more than the other.

Gareth Pierce: I suppose if you get down to that level of detail there is that difference, but it is a small difference. It is statistically a small difference, but you are right that there is a difference.

Q286 Chair: You said that two points contributed. One was that in Wales the nature of entries was different from England a bit, and the other point was...

Gareth Pierce: Teaching departments responding to the new specifications.

Chair: Exactly, but in England exactly the same thing happened. I know in my constituency I had a school that did even better than normal and another one that fell off a cliff. That would be because of their interpretation, understanding and recognition of the new specification, I assume.

Gareth Pierce: Yes. The reason I am mentioning it specifically for Wales is because I am aware that the Welsh Government took a specific initiative to support teaching departments. That is in addition to the support that awarding organisations would give in general terms for teaching departments, so that is why I am mentioning it as a factor. Regarding the other one, the use of early units, there is quite a significant difference statistically in the extent to which Wales centres made less use of the January opportunity compared with England centres.

Q287 Chair: Sorry; so the Welsh Government gave additional help that you would not have seen in England, say, to help schools make sure that they understood the specification of the new qualifications.

Gareth Pierce: They set up an initiative to provide that support in Wales, yes.

Chair: Prior to the whole-

Gareth Pierce: After the 2012 series; well, from the autumn term 2012 onwards.

Chair: Thank you both very much indeed for giving evidence to us this morning.

Examination of Witnesses

Witnesses: Leighton Andrews, AM, Minister for Education and Skills, Wales, and Chris Tweedale, Director of Schools and Young People Group, Welsh Government, gave evidence.

Q288 Chair: Good morning. It is an unusual and pleasant experience to have a Minister from Wales coming before this Committee. Would you like to make any opening remarks, Mr Andrews?

Leighton Andrews: I am very happy to go straight to questions, Mr Stuart.

Q289 Chair: Okay, great. Can you tell me about your relationship with the Secretary of State for Education in England?

Leighton Andrews: I would be very pleased to meet the Secretary of State again. I met him in June 2010, shortly after he became Secretary of State. I first met him when he was a television reporter in Scotland in the late 1980s or early 1990s. I have always found him very charming and engaging but, as I say, both the Northern Ireland Education Minister and I would very much like to meet him to discuss these threecountry qualifications.

Q290 Chair: What are you doing to improve ministerial relations?

Leighton Andrews: We have made very great efforts to have conversations with the Secretary of State. The Northern Ireland Education Minister and I wrote in August last year asking for a meeting to discuss the series of announcements that he had made on GCSEs and, indeed, on A-Levels, which we tended to read about in the pages of the newspapers or heard about on The Andrew Marr Show. Given these are threecountry qualifications, we felt-and I know that your Committee has asked-that there should be a ministerial conference to discuss these threecountry qualifications. We think that would be very valuable indeed.

I have to say I have very good relationships with Ministers in other parts of the UK Coalition Government, and certainly in the case of, say, universities I would expect to have regular dialogue, and indeed I have had several meetings, with Mr Willetts.

Q291 Ian Mearns: From what you have said there, do you take any implication from the Secretary of State’s stance? The fact is you have not met him since June 2010; you have asked for a meeting with him. Do you feel as though in some way he does not have any significant regard for the powers or the remit of the Devolved Administrations?

Leighton Andrews: Well, there is a need for us to meet together as the three Ministers with responsibilities for jurisdictions that have GCSEs and A-Levels. I met with the Northern Ireland Education Minister to discuss this last year. We have had several telephone conversations about it, the last being last Thursday. As I said, we jointly wrote to the Secretary of State in August asking for a meeting. As I said, your Committee has suggested that there should be a ministerial conference on GCSEs and A-Levels on a threecountry basis.

There are divergences between Wales and Northern Ireland on the one hand and England on the other. Of course, the Northern Ireland examining body has withdrawn its GCSE and A-Level qualifications from study in England as a result, to quote them, of the "emerging policy differences between England and Northern Ireland". We are now taking steps, as you are aware, to ensure that we establish different examinations in English Language, for example, at GCSE level. So there are issues here that require discussion. There are also issues, of course, that require discussion between the different regulators.

Q292 Ian Mearns: But regarding the divergence to which you refer, the Secretary of State is on record as saying that in itself is not a bad thing. I think I am right in saying that.

Leighton Andrews: Of course it is entirely for the Secretary of State to determine a course of action in England, but these are qualifications that are jointly owned. They are owned by the CCEA in Northern Ireland, by Ofqual and by the Welsh Government. Now, it is rather strange for us, therefore, to read announcements about GCSEs and A-Levels by someone who does not own the qualification.

Q293 Ian Mearns: So, in education terms, you think he is acting a bit like a high commissioner for education for Britain, dictating power to the Devolved Administrations.

Leighton Andrews: He is certainly not dictating to us. We have the policy autonomy to follow our own course of action, and we are doing that.

Chair: Mr Andrews is more than capable of making his own quotes.

Q294 Ian Mearns: Chair, I was merely suggesting a scenario that had sprung into my mind. The Secretary of State has described your intervention on this issue and on regrading as irresponsible, mistaken and a regrettable political intervention in what should be a process free from political meddling. First of all, I would guess one of your answers would be you did not start it.

Chair: Let him answer.

Ian Mearns: Was your decision based on regulatory or political concerns?

Leighton Andrews: I thought the Secretary of State’s remarks were inconsistent, inflammatory and uncharacteristically ignorant when he said that, because of course I was acting as the regulator in Wales. That is why he was inconsistent. He made a great play in his evidence of saying that he could not comment on matters that were properly for Ofqual and then sought to denigrate the action that had been taken by the Welsh regulator.

I received a report from my regulatory officials in September. It was a sober and serious report. It made it clear that the outcomes of the results for candidates in Wales were unsafe and that we needed to take action. We therefore took that action. Let me say, on a political basis, since the politics of this has been raised, the action I took has been endorsed by three of the political parties in the National Assembly for Wales, so it is not a partisan matter. This is a matter of trying to achieve fairness for candidates in Wales on the basis, let me say, of comparable outcomes, and I would be very happy to discuss the way in which the notion of comparable outcomes has changed over recent years.

Q295 Ian Mearns: You asked for a meeting with the Secretary of State last August.

Leighton Andrews: That is right.

Q296 Ian Mearns: I take it you have asked again for meetings. What has been the response to that?

Leighton Andrews: There has been recent correspondence between me and Mr Gove, and John O’Dowd, the Northern Ireland Education Minister, and Mr Gove, in relation to further recent statements he has made about qualifications, which again we read about in the newspapers or were asked about by the media on the back of the announcements. So you can assume that on the back of that correspondence, we would certainly welcome the opportunity to meet with the Secretary of State.

Q297 Ian Mearns: Of course, not everything that appears in the newspapers necessarily emanates from the Secretary of State himself, although he probably does not know where it does come from.

Leighton Andrews: I could not possibly comment, Mr Mearns.

Q298 Ian Mearns: The regrading of GCSE English Language candidates in Wales means that candidates in Wales will have received a higher grade than their counterparts in England for the same work. Do you consider that fair?

Leighton Andrews: It was action that we took to deal with an unfairness, and if you look at the judgment in the High Court, of course, what they were talking about in relation to Ofqual was how you address a balance of unfairness, to use their phrase. Everybody must accept that 2012 was unhappy all round, and we took action that we felt was appropriate. Ultimately, in the region of 2,300 candidates were regraded as a result of the direction we gave to the WJEC.

Now, I would have to say-it goes to the question that was asked by your colleague earlier on-people will ask what the difference is between a Welsh C and an English C, but then in England they will ask what the difference is between a January C and a June C, so there are issues here. 2012 was clearly an unhappy situation across the piece, and what the regulators have had to do-we have taken our decisions in Wales; Ofqual has taken its decisions in England-is to decide what is the best way to ensure fairness for candidates, and we took the action we did.

Q299 Ian Mearns: Given the portability of the jobs market between Wales and the rest of the UK, and particularly England, you have said that your intervention secured the fairest outcomes for learners in Wales, but do you think there are likely to be any longterm consequences for the currency value and portability of GCSEs achieved by Welsh candidates?

Leighton Andrews: Portability is very important, and we have just had an independent review of qualifications in Wales, which involved people from the further education sector, the higher education sector, employers and others, and that has charted a course forward for our qualifications system in Wales. We want to build on our existing Welsh Baccalaureate but also on GCSEs and Alevels.

One of the things that we asked that group to look at was portability, and they certainly looked at how our qualifications would be seen, for example, by universities across the UK. The evidence that we have is that university admissions tutors have to cope with entry qualifications from a whole series of different countries. Scotland of course has an entirely different set of qualifications; there are international qualifications and so on. But portability is important to us, and that is why we had an independent review of qualifications involving employers and higher education people.

Q300 Ian Mearns: In a nutshell, though, in terms of where we began, would I be right in thinking that you would welcome direct contact with the Secretary of State rather than having communication directed at you via the media?

Leighton Andrews: Indeed.

Q301 Chair: We have heard from the awarding bodies that came before us today that they all told the Government in 2009 that the way the proposed GCSE qualifications in English were being constructed was a mistake, and yet they were all ignored. As a coowner of the GCSE brand, did you say anything to the then Labour Government about the fact that they were going down the wrong track?

Leighton Andrews: Our officials certainly felt that 60% assessment was an issue. We need to be clear: there was, in fact, of course, a regulator called Ofqual back in 2009. I have minutes of the work that was done on maintaining standards in changed qualifications, with representatives of Ofqual not only attending those meetings but chairing them. That is indeed where the first consideration of comparable outcomes comes in, back in 2008. So the discussions would have been between the regulators at that point.

Q302 Chair: Yes, although the regulator at that point was not independent of Government. That was not until the passing of the Apprenticeships Bill.

Leighton Andrews: Well, there was an interim regulator, as one of your previous witnesses said, and it appeared to be called Ofqual.

Q303 Chair: But as a coowner, I just want to explore that a little more. If your experts were telling you there were concerns about it and all the awarding bodies said that it should not have gone ahead, and if that has been a key component in the problems we had last year, trying to get to the bottom of how on earth it happened is important and, as a coowner, you are partially responsible for allowing it to happen, are you not?

Leighton Andrews: I was not the Minister at the time, so I do not have full insight into everything that was going on, but my understanding is that as a regulator we made clear our concerns about the high level of that form of assessment. Indeed, we have now acted, of course, in respect of English Language for the cohort that began in September 2012, to ensure that the level of controlled assessment is 40%, not 60%.

Chris Tweedale: Representing the officials, our subject officer for English Language at the time, going back to the origin of the specification, raised concerns about the 60% controlled assessment, but the officer within Ofqual, which at that time was within the QCA, which was an NDPB, overruled that.

Chair: A nondepartmental public body, aka a quango.

Chris Tweedale: Indeed. So we did make representations at that time, but we were overruled by Ofqual as it was then.

Q304 Chair: Do you accept that the regrading intervention has made it more difficult to maintain common standards across the borders, and will perpetuating a split standard by having separate exam papers and grading arrangements for GCSE English Language candidates in Wales help this situation?

Leighton Andrews: We are seeking to ensure that in future learners in Wales are not subject to a Key Stage 2 indicator that has clearly failed to provide comparable outcomes in 2012 for learners in Wales. Key Stage 2 indicators are not being used in Northern Ireland and they are not being used in Wales, so we are certainly not going to put our candidates in a position where future judgments about grading are made on the basis of a Key Stage 2 indicator, and that is what we have done. Now, it is entirely possible, of course, that we could end up with tougher standards in Wales.

Q305 Chair: Yes, but given your intervention last year, employers and universities will think that unlikely.

Leighton Andrews: I do not see why that is the case. The issue here really is the coherence of the exam system, the curricula and the relationship to the accountability mechanisms. Now, there are serious issues in relation to the current relationship between gradesetting, as has been adopted by Ofqual, and the accountability systems that you have in England. We have seen that explained in the letters from the Ofqual Chief Executive to the Chief Inspector of Ofsted and to the Secretary of State in August-letters that we only became aware of after they were published in September.

Q306 Chair: You have said that Ofqual is in the grip of an ideological spasm. Given that language and the Secretary of State’s language, what is the future for threecountry qualifications and their regulation?

Leighton Andrews: The jury is out on threecountry qualifications. There is a serious issue and it goes to the issue of comparable outcomes. When comparable outcomes was designed as an approach to ensuring that, when qualifications changed, a cohort did not suffer by being the first cohort to sit that new qualification, it was really about ensuring fairness to candidates. Indeed, the quote from those minutes was, "The most important factor is a fair outcome for candidates." These are the minutes of the meeting, chaired by Ofqual in 2008, that came up with the comparable outcomes framework.

Since then, Ofqual, particularly in the last 12 to 15 months, appears to see comparable outcomes as a mechanism for containing grade inflation. That, it seems to me, introduces an entirely new element into the approach, as a result of which we had grade deflation, both for A-Levels and GCSEs, last year.

Q307 Alex Cunningham: Have you any idea why that has happened though? Why have Ofqual all of a sudden gone down a completely different track?

Leighton Andrews: When you held your preappointment hearing for the Chief Executive of Ofqual, she made it clear that containing grade inflation was her overarching priority, so I can only assume that she has established containing grade inflation as a key performance indicator for Ofqual.

Q308 Chair: You said that you thought there had been grade deflation last year. The Ofqual evidence that we have seen suggests marginally the opposite. Admittedly, it gets into the complexities of comparable outcomes and the datasets you use, but if you control for that, the 2012 GCSE English results were slightly more generous, if you like, on a comparable-outcomes basis compared with 2011. You have asserted the exact opposite.

Leighton Andrews: I would ask you this: why is it that the Higher Education Funding Council for England predicted there would be more AAB grades at ALevel than there were at the end of the day?

Q309 Chair: We have not looked into the detail of that. We have looked at GCSE English.

Leighton Andrews: It seems to me it is a relevant consideration, because you found a situation in England last year where quite a number of higher education institutions were clearly expecting rather more AAB entrants than they eventually got. You have to look at this in the round. There is no question but that Ofqual over the last 12 to 15 months has seen dealing with grade inflation as its priority. That is set out very clearly in the letters to the Secretary of State.

Q310 Chair: It is a statutory duty on them in the same Bill that set them up.

Leighton Andrews: I go back to this point about the original conception of comparable outcomes in 2008. The quote from the then Chief Executive of Ofqual was, "The most important factor is a fair outcome for candidates." You have to contrast that with what was said to my regulatory officials over the summer by the Chief Executive of Ofqual: that fairness is not a factor she has to consider because it is not set out in the legislation governing Ofqual.

Q311 Chair: I take you back to the point that you thought there was grade deflation. None of the data we have seen bears that out. We have not looked into the A-Level, although it could be said that the reason why universities would overpredict the results is that, given prior inflation and the fact that they kept being caught out by more people getting it than was expected, they necessarily took a conservative view. If you bring an end to grade inflation, you will start to see an adjustment in universities being able to predict more accurately what is going to happen.

Leighton Andrews: These probably require more time than we have, but let us just say, in relation to A-Levels, this is a factor. The question is, if you are trying to contain grade inflation, should you do it all in one year? That is an issue that has come out in the course of this.

Q312 Chair: We are looking at GCSE English in 2012 and you have said you think there is deflation, and none of the data we have seen bears that out. I am asking you: on what basis do you make that assertion? It is nothing to do with A-Levels: GCSE English 2012.

Leighton Andrews: I will take you to the results of the GCSE English in Wales, where very clearly we had had a very stable cohort over a series of years. We had a stable series of entrants, and if you look at the results, we suddenly saw a 3.9% decline in the results in Wales. There were clearly not comparable outcomes on the basis that we had been expecting, and that is down, I think, to the introduction of the Key Stage 2 indicator, which has acted as a deflator.

Q313 Alex Cunningham: Are your comments in relation to Wales rather than England, Wales and Northern Ireland?

Leighton Andrews: There was a decline in results in England too, but there we are.

Q314 Chair: No, there were not declining results in England according to the Ofqual data. There was that raw change of whatever it was, 1.4%, but they said that when you had gone through the process, disputable though that might be, if anything 2012 was slightly more generous. Is it not possible to read in that in fact Welsh students were falling behind English students, and that was one of the reasons why, because you are the Government rather than just a regulator, you came to the conclusion you did?

Leighton Andrews: No. You have seen from the WJEC evidence, and indeed some of the evidence that Ofqual published before Christmas, that we had fewer early entries in Wales. We had fewer examples of resits. We had far more examples, almost twice as many in Wales, of a linear approach to the taking of these qualifications than in England. The conclusion of the WJEC when it gave evidence to the National Assembly for Wales was that if a similar entry strategy had been followed, on balance, by schools across Wales, there would have been little difference between Wales and England in the overall outcomes.

We took the decision that we did in terms of regrading explicitly on the basis of the regulatory report. There were clearly not comparable outcomes in Wales compared with the previous year. We analysed the data on the cohort for 2012 against the data on the cohort for 2011, and there was no reason to make a judgment that that cohort should have performed worse than the previous cohort.

Q315 Chair: It still seems, to me at least, a thin argument. England is obviously much larger than Wales. You could have gone through and found all those schools that were similar to the Welsh schools and had taken one structural approach as opposed to another, and gone around allocating, marking their points up. Surely the danger is that it undermines the value that is perceived of Welsh qualifications, and you must have weighed that very carefully before you came to the decision you did.

Leighton Andrews: We certainly weighed that judgment very carefully indeed. At the end of the day, as I have said, 2012 seems to me to have been an unhappy experience for everyone. There are clear differences for thousands of students in England as to the grade they got in January and the grade they got in June. Everybody knows that is the case; it is confirmed, indeed, in the High Court judgment. We took a different form of action. One of the advantages of the action we took is that the situation was resolved very early on; it was resolved in midSeptember. There has been great transparency in the process. I have been answering questions on it in the National Assembly virtually every week ever since and been through committee hearings on it. So our process worked, but it worked in the context of an overall situation that everybody, including this morning, has acknowledged was an unhappy one.

Q316 Siobhain McDonagh: You have said that your current regulatory system allows swift resolution of injustice. Are you satisfied that the new regulatory model will also allow this should problems arise in the future?

Leighton Andrews: We are looking to move to a system very like the one they have in Scotland, which is the model of the Scottish Qualifications Authority. We think that that will provide a system that people cannot argue with in terms of independence. I certainly accept that people are able to argue that at the present time. It will also, of course, end any suggestion that in the context of competing exam boards there is, to quote the Secretary of State, a race to the bottom.

Q317 Siobhain McDonagh: This overlaps with your answer, but is there not a strong argument for the regulation of exams to be independent from Government and also from the body that awards grades?

Leighton Andrews: The question of whether it needs to be independent from the body that awards is the one I would focus on. In Scotland, they clearly have a model that works and they have very clear dividing lines between the awarding end of the Scottish Qualifications Authority and the regulatory end. It is important, of course, that we preserve that in any structure that we create. There is a not entirely the same but quite similar model in Northern Ireland too.

We want to be in the situation where there is consistency for learners, teachers know where they stand, and there is proper dialogue within the sector on the development of curricula and assessment and the regulation that goes with that, and that model is one that we can learn from.

Q318 Chair: Just to be clear, are you saying that fairness to students, which is of course an important but nonetheless vague concept, is more important than controlling grade inflation?

Leighton Andrews: I am not sure I am saying that. I do not think anybody wants to see grade inflation.

Q319 Chair: So you explicitly do not want to see grade inflation.

Leighton Andrews: I do not think anyone particularly wants to see grade inflation. You want to see consistency in standards. The issue is if teaching is improving year on year, then you would expect to see growth in the percentage of young people getting good qualifications. So you want a system that allows those improvements in teaching and learning to be expressed.

Q320 Chair: What if teaching is not improving year by year?

Leighton Andrews: If the teaching is not improving, you need to address that through your accountability measures, which we do.

Q321 Chair: That may be harder if there is a presumption and political pressure to make sure the results make it look like that is happening, and is that not a contributor to grade inflation? Most of the independent experts who have analysed the data and have come before this Committee, despite all the heat and light around this issue, suggest there has been grade inflation, which you have said you do not want, and there are political pressures around that have led successive Governments to allow it to happen.

Leighton Andrews: That is a very fair question, and I heard the evidence given by the representative of the AQA right at the beginning, who said there are issues relating to the accountability process and the examination process, and certainly we would accept that. That is why in our grading of secondary schools, unlike, as I understand it, the previous league-table system in England, we have not only a measure that looks at the percentage of young people who get five good GCSEs, to put it crudely, but also the capped point score and other measures too.

Q322 Alex Cunningham: You have made reference to the very different system north of the border-the ScottishEnglish border, of course. Michael Gove has said he does not regard an increasing divergence in qualifications policy between England, Wales and Northern Ireland as a bad thing in itself. Do you agree with that?

Leighton Andrews: I do not regard it as a bad thing in itself. What I do think would be helpful, in a situation where we have commonly branded and commonly owned qualifications, would be more ministerial dialogue when major changes are likely to take place.

Q323 Alex Cunningham: So do you see a situation where there is still the potential that Wales could decide to do their own thing, to have a different system, as they have in Scotland and similarly in Northern Ireland, and separate themselves because of the difficulties that you have faced?

Leighton Andrews: We were going down our own road of a review of qualifications in any case and had embarked on that around 18 months ago. We already have a Welsh Baccalaureate in Wales. We are looking to grade that qualification from this year. Clearly, Northern Ireland is also having a qualifications review at the present time, but the Northern Ireland Education Minister and I both agree that we have confidence in GCSEs and A-Levels.

Q324 Alex Cunningham: You have mentioned the need to improve ministerial relations. You have also said that the current threecountry arrangements and structures are becoming untenable due to unilateral announcements from Westminster about shared qualifications. What more do you think you could do to try to improve that situation?

Leighton Andrews: I have met with the Northern Ireland Education Minister. We have had several telephone conversations on this issue. We have both sought, through the concordat arrangements that exist between our Government and the UK Government, to try to ensure we get better advance information. However, more dialogue would be helpful, and I certainly think a ministerial conference of the kind requested by this Committee would be helpful.

Q325 Alex Cunningham: So your message this morning to the Secretary of State in England is to say, "For heaven’s sake, sit down and talk with us."

Leighton Andrews: Indeed.

Q326 Alex Cunningham: What steps are you taking to work with Ofqual to strengthen the GCSE and A-Level brands to ensure future comparability of standards across the board?

Leighton Andrews: We are in regular meetings with Ofqual. My regulatory officials meet with them on a very regular basis. These issues are considered all the time.

Q327 Alex Cunningham: Finally, the WJEC has increased its market share in England in recent years in some qualifications, particularly GCSE English. We see the Northern Irish exam board have said they are no longer going to offer a GCSE qualification in England. Should there be some sort of containment? Should the Welsh exam boards be sticking to Wales or should we have crossborder examinations and offer a free market?

Leighton Andrews: The current arrangements were developed in the 1990s. Clearly, an organisation called the Welsh Joint Education Committee, as it was originally, evolved to develop qualifications in Wales. It has been very successful in attracting custom from over the border and it has a very high reputation, both in Wales and in England. It also has a high reputation elsewhere.

Chair: Thank you both very much indeed for coming and giving evidence to us today. I am sorry, Chris, we did not hear more from you but, as we have seen, the Minister is more than capable of answering these questions. Thank you very much.