Members present:

Mr Barry Sheerman, in the Chair
Mr David Chaytor
Valerie Davey
Jeff Ennis
Paul Holmes
Ms Meg Munn
Jonathan Shaw
Mr Mark Simmonds
Mr Andrew Turner


Memoranda submitted by AQA, Edexcel and OCR

Examination of Witnesses

KATHLEEN TATTERSALL, Director-General, AQA; JOHN KERR, Chief Executive, Edexcel; and RON MCLONE, Chief Executive, OCR; examined.


  1. Can I welcome you, and start with Kathleen Tattersall, who is Director-General of AQA, welcome Kathleen; John Kerr, who is Chief Executive of Edexcel (in the centre position); and Ron McLone, who is Chief Executive of OCR. We thought we would have you all in together to get a little more spontaneity than just having separate sessions. Just to explain to you that these proceedings are held under Parliamentary Privilege, and so you can say anything you like and have all sorts of protection, but you must not repeat it; if you say anything that you want to be careful about, do not repeat it outside, even though you have said it here. Is that a right explanation; checking with the Clerk. So I want to make it clear before we start that we are not conducting a repeat of the Tomlinson inquiry. Of course, as the elected representatives of Parliament, with the role of inquiring into anything and keeping to account the Department for Education and Skills, and regularly meeting with both yourselves and the QCA, of course, we want to find out not only what is going on in the world of examining boards and the QCA and the relationship between them, but we will be looking to the future, about the way in which we better govern our examination procedures and the way in which perhaps we better organise the accountability of the system. So of course we will be asking you some things that reflect on the past, but we will also be trying to learn lessons. So can I start really by asking you not just for an opening statement, Kathleen Tattersall, but to say, you are something in the public eye at the moment, are you not, as examining boards, and some of us would say better to be out there doing your job in a kind of low-profile way, because what the public want and what parents want and what students want, teachers and everyone else involved in the education system want, is a quiet system that delivers reliability without any fuss, and they do not want to hear a debate on quality of standards on Radio Four every morning, which they have had fairly recently. Why do you think we are where we are at the moment, what do you think has caused these problems?
  2. (Ms Tattersall) I think, in the first year of a new examination, inevitably, there is more of a focus on the examination than might be the case in the examination that has been running for some time; but I think we also know that whenever we publish results then there is an interest in those results and quite a public debate about them. In this, the first year of A level, I think when the results were published on 15 August we were all very pleased that the day passed as well as it did, because the focus has to be on the students who have attained the grades in question, and, indeed, my recollection of that day is that there was a welcome for the new examination. I recall The Guardian leader of the day, for example, that there was a welcome for the examination and that things had gone so well. I think that what happened since was that there was clearly some concern, dissatisfaction, on the part of some schools, with the grades which their students attained and a questioning of those grades, and that has led to the re-opening of various issues, some of which were very much firmly in the past, but nevertheless a concentration on those issues, which has led, I think, to where we are today.

  3. Did this process lead you to feel anxious about your role as an examining board?
  4. (Ms Tattersall) No. Looking at AQA as a board, I believe that the job that we have done in this first year of A level is exactly the same job that we have done in all the previous years of the old A level. And, thinking about it from my own personal perspective, where I have been a chief executive for 20 years, and indeed seen the coming of GCSE, for example, in 1988, the first year of that, and knowing some of the problems that people foresaw at that time, I believe that AQA has done an extremely good job. If you look at what AQA was asked to do, as a result of the Tomlinson inquiry, it was to examine only two of the 1,008 boundaries which we set at A level, and the inquiry, which was very open, very public, very transparent, has reaffirmed the boundaries which I set as a result of looking at the Chair of Examiners' recommendations. So AQA, I believe, can be very proud of its record of bringing in the new A level, and, of course, as a board, we are responsible for something like 45 per cent of the grades awarded in A level this year.

  5. So you are feeling quite comfortable; but it is quite a small world, the examinations, because we are down to three examining boards in England, are we not, and you people meet together a great deal, both informally and formally, and you all have a relationship with the QCA. And how is it that you seem to be very comfortable about the process, but something went wildly wrong, it seems; what went wrong between the three of you? You are all on very close, first name terms, you seem to be great friends, when I look at you chatting together; it is a very small world, very well communicated. What went wrong, in your view?
  6. (Ms Tattersall) It is a small world, in that there are three chief executives, as you say, and, of course, we have also got to remember that the system operates in Wales and Northern Ireland, so there are also two other chief executives who are involved. All of us work within the Code of Practice, which is laid down nationally, it is laid down by QCA, drawn up by QCA in consultation with ourselves, and all of us work against the criteria which are determined for A level. I believe that we were all working together to try to establish the same standards across the awarding bodies, as we are charged to do, because three awarding bodies have to ensure that their grades and their awards are in accord with each other. We met over the period of the four years, or so, leading up to the new A levels, on several occasions, there is the Joint Council for General Qualifications, that is the forum in which we meet together, and also with QCA, to try to establish all those difficult technical issues which have to be resolved when the new qualification comes into being. And this, remember, was a qualification which was quite different from the qualification that went before it; here we have a qualification made up of two parts, the AS examination and the A2 examination, AS being a qualification in its own right, and A2 being the second half that makes up the A level. I believe we worked as best we could to try to establish those standards, and it is only really in retrospect that some of these problems now begin to emerge, which at the time were not seen as real issues.

  7. The people we represent, you would understand them saying to us that everyone knew a new examination system has a lot of problems, its teething problems are obvious, and, you have just said, you have been planning for a long time this transition. Indeed, the Committee has just come back from New Zealand, where we looked at exactly a parallel situation of introducing a new set of examinations in that country, and, yet again, a great deal of work had to go into that transition, and a lot of bad feeling about those guinea-pigs who went through the first years of the transition. If I can turn to Ron McLone then for a moment. Dr McLone, can I ask you, you were at all the meetings, the three of you and the meetings with the QCA, but your board seemed to have more problems and seemed to go off at more doing your own thing than the other two; now can you explain why that was?
  8. (Dr McLone) We do things slightly differently, that is absolutely true. We have all worked, as Kathleen said, to the same Code of Practice, we have worked to the same procedures, and in the end we all come to the same outcome, in terms of the comparability of the results. We do it slightly differently. Where we have started, we start from looking at what the examiners do first and apply statistical evidence afterwards; not all the boards work in exactly the same way, and therefore it becomes more evident in the way, I suspect, we have done it than perhaps in the others. But I think the important thing is that we do work together in looking at the technical issues, that is absolutely true; but it is the way they have been set up in the context of the whole of the implementation of AS and A2 which I think has led us to where we are now.

  9. But, if we look at it forensically, here you are, you have all seen this coming for a very long time, you have all worked together and you all have a relationship with the QCA, indeed you have meetings with the QCA together; how come it seems your interpretation, of your board, seems to have been different? I would not say that Kathleen Tattersall was being smug, she was saying, "I think we did it right; a very experienced board, I am Chief Executive, I have been here 20 years and, more or less, we haven't had any problems." And she has not said anything nasty about the other two boards, certainly, Dr McLone, about you; but you could not say the same thing as Kathleen Tattersall, could you, you have had real problems?
  10. (Dr McLone) I would say that we have not had real problems, but we have worked exactly to defining an A level standard, in the same way that OCR and its predecessors always have. We have always worked to getting to the examiner judgements first and then looking at statistical evidence, to make sure that we can compare year on year that we are getting to the right overall standard. I think I do go back to the question of AS and A2; we did not know exactly, all of us, where exactly A2 was. There is a real tension between trying to set boundaries at A2 and yet carrying forward a standard which is not A2, since we do not have any archive evidence at A2, there is nothing of that kind, but we do have to carry forward the A level standard, which is a combination of the AS and the A2. So therefore it has been a tension, in trying to establish all of that. The setting of the standard is actually QCA's job, of course.

  11. That is exactly where we are trying to get to. If the QCA was setting the standard, and the QCA is talking to all three of you, how come that all three of you do not seem to operate in exactly the same way? It seems, to someone from the outside trying to look in, that two of you seem to read the mind of the QCA in one way, whereas, Dr McLone, you and your board read the QCA's mind in a different way?
  12. (Dr McLone) I think it is possible, in applying the Code of Practice, to be looking for what is the overall standard and trying to define what A2 really means, in a way in which all of us were trying to get to the same place, as Tomlinson said, all of us did our best to get to the same place; if you have not got a definition, and there was no definition written down, as to what you are really trying to get with A2, then I submit that we will be looking to do our best to get there.

  13. Mr Kerr, do you concur with that view?
  14. (Mr Kerr) I have certainly listened very carefully to what my two colleagues have said, and, in fact, I am in full agreement. In terms of setting the standards, I have one year's experience, and clearly I would not claim that Edexcel has not had its problems in the past; but, for this particular year, I am very confident we set the grades professionally, we set them accurately and we set them in accordance with the Code of Practice.

  15. So how do you explain the degree of unhappiness about recent events?
  16. (Mr Kerr) I think, to answer your first question, what has gone wrong here, clearly, 90,000 students had to wait nearly two months to get their grades confirmed, and clearly that is unacceptable. In terms of my own board, we did not change any of the grade boundaries, we co-operated fully with the Tomlinson inquiry, we thought it was very important that we did co-operate and that there was seen to be a public scrutiny of how the grade boundaries were set. At the end of that, I saw no reason to change any of my grade boundaries.

  17. What I am trying to get out of the three of you is, if we know what the events of the last two months have been and you all say, "Well, we operated in terms of our Code of Conduct and full professional standards," what guarantee have the public that this will not all happen again next year? None of you seems to be saying, "It was me, Guv, and we made a mistake and we'll put it right." If none of you admits to any mistakes, how can you improve on what happened this year?
  18. (Dr McLone) The system was flawed, if I may, and I think we are all trying to operate in a flawed system, that really we need to deal with; and I have to say that, personally, I have great confidence in Ken Boston, in putting forward these new committees, that he is putting forward, to try to right what was not done in the past. Tomlinson and Ken have been very clear about that, and I think that we do need to get to the root of those flaws in the implementation of the system that, in my view, and I think in Mike Tomlinson's view, from what he said, exist.

  19. Dr McLone, what I am trying to push you on is the difference between the three boards. I am still not clear, and we are 659 Members of Parliament and I do not want to tell you how many letters we have had from individual MPs, because schools in their constituencies were very much affected by the events of the last two months, and I am still not clear, as Chairman of this Select Committee, what went wrong differently in your board that did not seem to go wrong with John Kerr's and Kathleen Tattersall's boards?
  20. (Dr McLone) It is a matter of how you look at the way it is done and the way in which you can make the measure. Tomlinson, quite properly, put a measure forward for looking at the way it was done; it could have been looked at in different ways. The way we have done it, which is the way consistent - certainly, it shows more in a system where you are in change. When you have consistently an examination that has been taken year after year and everybody is absolutely sure, with the A2, and if I may just use this chart which I sent to you, if everybody is using something where they know the demand is always the same at A level then they have been consistently arriving at it; when you have a demand at AS, to be advanced by A2, it does matter, in the way you are doing it, how it shows, in other words, the perception of what we have been doing. But I do believe that, when it comes to the outcome at the end, you will see, in the comparability of what we do, and we run comparability studies, that we are actually at very closely the same standard all along.

  21. But you have changed lots of results, have you not; the students who thought they had one grade now have a different grade?
  22. (Dr McLone) With respect, we changed 18 out of 1,012, which is a very small number. But, yes, we did, and it is a matter of doing it in a different context; we had a different context, we had different people present, we were making judgements. The judgements that we made, on the evidence, and the summer, stand, the judgements that we made were done in a different context at this particular time, and I judged it right to be able to make the amendments I did in the 18 units that I did, but, nonetheless, that is quite a small number.

  23. So if we were doing a forensic job and we said to you, "It seems that the QCA was terrified about grade inflation," they were terrified of grade inflation, and they said to the three of you, they expressed their anxiety about this, two boards reacted in one way but you seem to have reacted in a different way?
  24. (Dr McLone) I think we reacted all more or less in the same way. I was about setting standards, just as my colleagues were, for the A level examinations. I think there are a lot of lessons to be learned for the future out of this, and I think that, critically, we need to be looking at the lessons that we have in the way we all managed to do it; we are doing it a slightly different way, I grant that.

    Mr Turner

  25. Yours were the 18 units that were revised, and you have said that, essentially, you do the marking first and then the statistics, while the other two boards do the grading and the statistics together. Now that implies, to me, that your actions are more transparent than those of the other two boards; would you agree with that?
  26. (Dr McLone) It could be interpreted so, but I think all the methods that we use are quite proper. I think it is still the case that you could do the statistics, and should perhaps get the statistics first. The key thing is where you select scripts; and I think that this is what we really need to be looking at, in the future, trying to make sure that we all carry out what is a much tighter Code of Practice procedure. The way you select scripts and where you select them depends very much on how much information you put into the system to start with; in a very steady state system, people are very confident about where they might go to select scripts, in a completely new system they are not as confident. Therefore, there are ways in which you have got to get the first set of scripts in which you are actually trying to find out where the boundaries are. We did it, I have to say, by saying to the examiners, "Well, where do we look?" and then we looked at the statistical evidence, where the GCSE performs, there are all sorts of things that you properly do every year. I have to say that you could do it, and we could have done it, by looking first of all at statistical evidence, to say, "Well, let's think about this; if we want a new A2 standard then where will the scripts come from."

  27. That is the first time I have heard anyone mention selecting scripts.
  28. (Dr McLone) It is critical.

  29. So how do you select them differently from the other two boards?
  30. (Dr McLone) We have a set of procedures laid down, that is absolutely true; what turned out, and the way we have actually operated in the last few years, is that we have looked very closely at what the examiners have said, that is the principal examiner, with not very much, I have to say, statistical evidence applied at that time. We then apply statistical evidence, GCSE performance, we look at performance from year to year, of course, to see whether or not we are agreeing. This year, some subjects, they would obviously work something out to get to the right place, and other subjects have not. My job is to bring all the subjects together, just as it is Kathleen's and John's.

  31. I did not quite hear your words. You said they work something out?
  32. (Dr McLone) What I am saying is that when the examiners are looking at the scripts they are looking at where they should be selecting the scripts from. If, in a new system, we are not sure where the A2 is to be set then they are working in the dark, to some extent, if we do not have some statistical evidence applied; there is no archive, there is nothing of that kind.

  33. Could I ask just one other related question. This is to do with whether what you are trying to do is possible, actually, because the former Secretary of State told me, a few days ago, that an AS level is worth half an A level; do you agree with that?
  34. (Dr McLone) Technically, it is, because of the 50 per cent weighting. But the problem is that, if you have got a 50 per cent weighting, and, I have to say, we argued very strongly for 40/60, my own experience, from being an academic, is that if you have a part one and a part two, and, many universities, the one I was at was certainly like this, you got the balance right between a part one and a part two one year, and then a harder exam the next year, by a 40/60 weighting. It was changed to a 50/50 weighting. A 50/50 weighting has an impact; of course, it all depends on the demand that is set in question papers, but if you look at a 50/50 weighting and you set one at an AS, which everybody said would be at a lower standard, then, the other one, by definition, you are asking for something that is more demanding. You hope to get question papers that are more demanding, but, if you have got to get the higher demand, that 50/50 weighting, in my view, totally distorted really what we have all been asked to do. Now you could actually look at statistical evidence, to start with, to try to get that distortion out of the way; we look at it afterwards.


  35. Who did you argue with for the 60/40?
  36. (Dr McLone) We argued with QCA and with, well, I suspect we argued with - there was a lot of big four that went on, at the time, I remember, and we certainly argued about 40/60 at that time.

  37. Did any of the other examining boards think that 50/50 was unwise?
  38. (Ms Tattersall) We all started off by arguing for a 40/60 relationship, that is absolutely true, but as the debate went on other factors came into play, one of them being the points which UCAS proposed to award to AS and to A2, and since AS was half of the other then 50 per cent seemed to be an appropriate percentage. I think we were all asked to, as it were, square the circle, but, I have to say, we did not find it as difficult in AQA as Dr McLone is saying that the problem is in theory; for us, we saw that the AS and the A2 made up the old A level standard, that AS was clearly at a lower standard because it was after one year's work in the sixth form, and that the A2 had somehow to bridge that gap between the full standard and, as it were, the half standard. And so we saw it in terms of the less difficult parts of the old syllabuses being in the AS, and the more difficult parts being in the A2, and the whole being the old A level standard.

  39. I think we sent you the letter from Alan Stitchcombe, I think all three boards were sent his letter, where he says that, his argument was that all this was predictable, all the problems were entirely predictable, that you were going to run into these problems, you were going to have these difficulties, and he is a chap that is a sort of voice of commonsense, pointing out, what you just said, that it was going to be a totally different examination, it was going to be an easier first year, you were going to be able to retake it, so that that was going to push up the passes. What Stitchcombe says is that all this was predictable, that these, you, highly sophisticated examination boards, with the QCA, did not get it right?
  40. (Dr McLone) I must apologise, we have not seen it; (and we have got it open, the e-mail ?).

    (Mr Kerr) I think, what Kathleen has said, we did get it right. We are actually not making any apologies for the standards set this year; the standards set, certainly by Edexcel, were correct.

    Mr Chaytor

  41. Chairman, I am getting very confused, because what we are hearing from each of the three boards is that the difficulties that we experienced this year were really entirely technical matters; but that is not really what the Tomlinson report concludes, because Tomlinson says, quite specifically, that the actions of the boards during the grading exercise arose from the pressure they perceived they were under from the QCA both to maintain the standard and achieve an outcome, more or less in line with the results in 2001. So what Mike Tomlinson is saying is that the problem was not merely a technical issue but it was an issue that arose directly because of the pressure from the QCA and the way in which each of the three boards responded to that pressure. Now the submission from the AQA says: "And at no time were we unduly influenced by any external pressure or agency to act differently this year, when awarding grades." So my first question to Ms Tattersall is, are you saying Mike Tomlinson's conclusions are completely wrong; and then I would like to hear from the other two boards as to this pressure from the QCA, when was it applied, and in what form did it occur?
  42. (Ms Tattersall) Let me start by saying that AQA did not respond, as it were, to any pressures, of any kind.


  43. That was not the question. Were you pressured, was the question?
  44. (Ms Tattersall) Let me come back then to the discussions that took place between QCA and the awarding bodies, and they are on record, in two letters, which I think you have, a letter of 22 March and a letter of 19 April; 22 March, myself, and 19 April from Bill Stubbs. And they relate to a meeting that we had on 12 March, when inevitably the issue of standards and the new examination came up; and all of us, as awarding bodies, are charged with the maintenance of standards, year on year, and indeed between ourselves, and the issue is, how do you measure those standards. One measure, you could say, is the percentage of candidates who receive a given award in a given year, the outcomes, I will call them, and the discussion that we had on 12 March focused very much on the outcomes, the expectation being that in 2002 they would be very similar to 2001; that worried many of us, because, clearly, many will see the outcomes only as indicators, not as real examples of standards, and the issue is what are the standards. And that gave rise to the letter that I sent to Bill Stubbs, which was really setting out our position, as awarding bodies, that we judged the standards from the evidence, and the prime evidence is the candidates' work, and the subsidiary evidence is the statistical information that Dr McLone referred to; and we had a letter back from Bill Stubbs, which concurred with that view. Now, as far as AQA was concerned, that clarified the issue, we were all talking the same language; we were not talking about outcomes being the same, we were talking about judging the evidence on the basis of what candidates actually did in the examination.

    Mr Chaytor

  45. But was that the end of the story with the QCA, or was there further intervention following the completion of the marking and the early results coming forward?
  46. (Ms Tattersall) We had a further meeting, firstly of ourselves and secondly with QCA, on 26 July, and the reason for that meeting was that it was very clear, certainly from the awards in AQA, that the pattern of the outcomes was going to be very different in 2002 than it had been in 2001. What I was anxious to ascertain was whether this was something peculiar to AQA, or whether it was something which my fellow chief executives were also experiencing in their awards; and so we called a meeting of the boards, we ascertained that we were all experiencing the same sort of pattern of results, and we identified the reasons for that pattern of results. One of the major reasons being that there is a big drop-out rate between the old AS and the full A level, people who had performed to the best of their ability at the AS level and then not gone on to take it at A level. And so, as a result of that meeting, we were very comfortable that the results we were seeing were indeed representing truly the true standards that we were expecting, the carrying forward of standards, and we then shared that information with QCA. There was no pressure from QCA to intervene and change the results after that point.

  47. So there was no further communication with yourself after the meeting on 26 July?
  48. (Ms Tattersall) I think I wrote to Bill Stubbs following that meeting, again to clarify what we were doing. We were a little worried when QCA had mentioned an inquiry, because that seemed to undermine confidence in results, although all of us recognised that with a new system it is inevitable that people want to see how the system has worked, and therefore an inquiry in the general sense was quite acceptable, but in the specific sense, as to whether the grades were right or not, we thought would really have cast doubt, which would have rebounded very badly on the candidates.

  49. What you are saying there is completely at odds with what Mike Tomlinson is saying in the Conclusion to his report?
  50. (Ms Tattersall) I do not think it is. Mike Tomlinson is recognising that there were clearly pressures, and all of us operate in a very pressurised society, particularly when results are being published, we have to stand up and be counted; and that, of course, is a pressure, that you have to stand up and be counted, you have to be able to say that the standard of the award in this year is the same as the standard of the award in previous years, regardless of what the statistics actually say. And so that is the sort of pressure that we were all operating in. And, certainly, in terms of specifically bringing in a particular percentage, no, that was not the case, as far as AQA was concerned; we followed our procedures and our practices in the normal way.

  51. So William Stubbs ought to be in the job still?
  52. (Ms Tattersall) I cannot comment on William Stubbs not being in his job; clearly, that was a matter for the Secretary of State.

  53. You are saying to the Committee, there was no undue pressure brought to bear by the QCA?
  54. (Ms Tattersall) In terms of that particular issue, I did not see that as a pressure to actually bring in awards at a particular level, once we had clarified that we were talking the same language, and we were not actually saying that the outcomes for 2002 had to be exactly the same as the outcomes of 2001.

  55. But were there other issues in which the QCA brought pressure to bear? I am just trying to put some flesh on this Conclusion to the Tomlinson report.
  56. (Ms Tattersall) I do appreciate that. I can only say that, within the context we work, the main pressure on us is to be able to demonstrate that the standard of our awards is commensurate with the standard of previous awards; and in the first year of an examination that inevitably is difficult, because the syllabuses are different, the structure of the examination is difficult, you do not have the same reference points as you had in the past. But that is the sort of pressure that I would describe, but it is a pressure of which we were very aware, even without QCA saying it.

    Ms Munn

  57. I want to clarify something which is in Ron McLone's written submission to us, in terms of this issue about maintaining the standards. Because what you have said here is: "There is a fundamental tension inherent in the awarding process between the current Code of Practice (CoP) requirement to maintain year-on-year standards at qualification level" which is what we have just been talking about, and which I understand very well, "whilst making examiner judgements on the basis of script evidence at unit level." What does that mean? I do not understand what you actually mean by that.
  58. (Dr McLone) I am sorry if it is vague, but what it is really saying is that the standard that we are required to take forward is that of the whole A level, and that, in fact, is what is in the Code of Practice. The Code of Practice does not mention the AS or the A2; whether it should or not, I think, maybe it ought. But the judgements that are required to be made were not at A level, they were at A2, now, by definition, which is not at A level; there has been a debate as to where it should be, that has been the whole issue about what the whole thing has been about, what should be the A2. And, therefore, the senior examiners, in looking at scripts, were being asked to look at something which was not the exact continuation of the A level, by definition, because of a 50/50 weighting and because of what we have been doing with AS. In my view, that would have been a lot easier if it had been a 40/60 weighting, because they could certainly have continued with a view about what was going on in the past A level and drawn a similar conclusion, the weighting would have taken account of it. I do not think, with a 50/50 weighting, and I think this is pretty clear now, that you could expect that; and the examiners themselves, to be fair, understood that they had got to go for a different level. But there is a tension in that.

  59. So is 'tension' a nice way of saying it was problematic from the start?
  60. (Dr McLone) I think the whole of the implementation scheme was flawed from the start, yes, and I believe, and certainly my board believe, that what we should have been seeing was a much cleaner, crisper definition for all of us of what that actually meant, that tension explained. It is inevitable, I think, and Tomlinson said so in his report, that you are going to get different interpretations, I accept ours looks a different interpretation. I have also to say, what matters is the outcomes, and I believe that, the outcomes, actually, if you take a look at where our boundaries have been set, compared with, say, Kathleen's boundaries, you will probably find them in very much the same place.

  61. So is it, was it, should it be in the future, QCA's responsibility to sort that tension out?
  62. (Dr McLone) It is, and I do think that Ken Boston is intending to do so.

  63. And it would have made your life a lot easier, if that had been done last year rather than the next year?
  64. (Dr McLone) Most importantly, it would be right for all the students and teachers out there; that is what actually matters. Whether life is easier for us, we are exam boards, it is our job, but it must be right and clear for all the students out there.

  65. So that was one problem that you could have identified at the outset, that one issue that was going to cause you a problem. Given that this was a new process anyway, what are the other teething problems that you expected there to be, given that you were changing to a new system and that examination boards have had that experience in the past, I think Kathleen Tattersall referred to that earlier?
  66. (Dr McLone) It has been a big change this year, so the first change we have had such a change to one of our major qualifications, and the first really big change to A level in 50 years, of course, splitting it into two. The issue that I think was important to getting it sorted out, apart from that, was the technicalities all to do with whether there should be a trial or not; there were no pilots, we had some pilots to AS, which helped, I think, with AS, we had one or two, I think, Kathleen did one, we did one, I think John probably did one, maybe, at A2, but no consistent pilot at A2. In the end, of course, what actually happened was the pilot of A2. I think that if we had got some pilot, we had tried to define some archive evidence on which we could base a moving forward, I think that would have been immensely helpful. We could also have done something different, I suspect, in the structure of AS and A2 and whether they were linked, because another problem, again, we wrote about this earlier, a problem about this is we had one qualification, AS, embedded in another one; that is a very difficult thing actually to sort out in the end, too, it is much better if you sort out the two and split them up.

  67. I just wonder, Chair, whether John Kerr or Kathleen Tattersall want to say anything on the pitfalls that they saw at the outset, in terms of this being a new examination?
  68. (Ms Tattersall) One of the pitfalls that has not been mentioned is the number of units that formed the qualification. In the early stages, when the qualification was being discussed, we certainly argued for four units, not for six, and, one of the difficulties I think there was, that had there been a unit devoted, as it were, to coursework it would have actually exceeded the limits which at that time were being laid down nationally for coursework. So, in a sense, six became the norm as opposed to four. And that, of course, has brought about other problems, like examiner recruitment, and that is a real issue, in terms of the new qualification. So that, for us, was one of the issues, the number of units, the fragmentation of the curriculum; and I think we have got to remember this is a curriculum problem, not just an examination problem. So we felt that, in some subjects more than in others, for example in English, History, the splitting into that number of units was in itself a problem in curriculum terms.

    (Mr Kerr) Having piloted material, having exemplar material, out in the schools, would have made it clearer. I think clearly it was a mistake to launch A2 without going through that; and also I think that Kathleen's view on four units is probably easier for everyone to grasp, rather than six.


  69. Where were you three in terms of, the general impression we get, in terms of this Committee, is that there is this great discontent, about the old A level being too narrow, too specialist too early, three, sometimes four, three subjects, at 16, and, can I call them, the chattering classes, right across the piece, people wanted change, they wanted a broadening; where were you, did you want to stick with the old system, or were you champions of a new system?
  70. (Ms Tattersall) AQA only came into being in 2000, so I am now speaking from a different board, like the NEAB, which was one of the boards that formed AQA, and we argued for many years for a change to the system, in particular actually to have an examination which was at a lower standard than A level following GCSE, because there was such an appalling drop-out of young people between GCSE and A level, with no record at all of their achievements, and we felt that that was such a waste of talent. So, as a board, we piloted what we called the E examination, I think it was Extended, I think that was the name, the Extended examination, which was piloted with many independent schools, because they were the only ones free actually to take an examination which was not a formal qualification. And I think that demonstrates where we stood on the issue of change.

  71. Dr McLone, you were around at that time; where were you?
  72. (Dr McLone) Yes, I was, indeed. I do believe that Curriculum 2000, as a curriculum driver, has been a great success, because it has allowed students to move into a broader number of subjects, it has allowed students to develop as they can over 16 to 18; it is a time when students change, to think about all sorts of things that they can suddenly decide. And if they are choosing subjects at 15, as, of course, they have to, and then there is at 17 and they have still got a year to go, and "This really wasn't the subject I wanted to do," this was certainly a real success, it seems to me. So I do not think we should be throwing anything out, we certainly should not be pulling up plants yet again to inspect the roots.

  73. So let the thing settle down, is your view, not switching to an International Baccalaureate immediately?
  74. (Dr McLone) I think we need to be just a bit calm about it.

    Paul Holmes

  75. We have just heard two of you welcoming the new system, the fact that it is broader and it is a test in different ways, and allowing Year 12 students, who were finding difficulty from GCSE to A level, actually to show what they can do. But is not one of the problems that we have had this summer the very fact that the exam boards and the QCA and the Government did not adjust to that, that, the 4 per cent increase in A level passes that we had this summer, really is not that too small? If you introduce a modular system which allows students to resit their modules, and therefore obviously do better, and if you introduce a system which allows students to drop their weaker subjects before they go on to the final A2 stage, and therefore do better, should we not have seen a much bigger jump in pass rates than the 4 per cent we achieved?
  76. (Dr McLone) Of course, you expect those to have an effect. I have to say that, of course, an awful lot of people were already doing modular A levels, they are not new. So the whole business about whether they actually have dropped the units, or they have dropped out, that has been going on for some time, over half of our A levels before were already modular; so we were not experiencing that massive shift because we were going to modular. The fact that there were five, and that therefore they could drop different subjects, that of course is new, and we expected an increase at E because of that.

  77. I would be interested in the figures across the exam boards, because you were saying over half of yours were doing modules already. I was an A level teacher, I was a head of sixth form, and certainly over half of ours were not doing modular, although there were more modular courses around; but when I was teaching in the first year of AS level, and when I was talking to A level teachers last summer, before the results came out, generally everybody in the education world expected there to be a significant jump in the number of children, students, achieving. Have we not artificially depressed that achievement, for whatever reason? Mike Tomlinson thinks, he says twice in his Conclusion, that that was what happened, because the exam boards perceived this pressure to keep the "grade inflation" down.
  78. (Dr McLone) I do not perceive that we have artificially depressed anything. I do believe that we have been looking very hard as to what this A2 standard would be, and A2 is, indeed, a good deal harder than AS, and I think has been differentially seen, across different schools and different colleges, as to what that really meant. And I think Kathleen and John are right, that if we had had more exemplar materials and more worked out then it would certainly have helped.


  79. Can I just intervene, on that question, Paul. If you had, whose duty was it to do this, to have the pilots; who let us down, in terms of the system, was it the QCA, was it the Department for Education and Skills, was it your own boards?
  80. (Dr McLone) The boards clearly cannot operate in their own vacuum; obviously, to have exemplar material for a new standard, you have got to do it collectively and you have got to do it all together. Therefore, I perceive that it should be down to the regulator to be driving that forward. Whether it was at anywhere else, I would not like to speculate.

    Paul Holmes

  81. So the root problem, as teachers found, and as, in various evidence, we have had submitted, and Mike Tomlinson points this out, is that the thing was introduced too quickly, without piloting, and so, two years on, that is why we have got the problem we have got?
  82. (Dr McLone) Absolutely; completely, yes.


  83. Will not the people that we represent feel a bit aggrieved that you, as the great examining boards, the three great examining boards of this country, did not squeal a bit louder to warn the educational world and parents and students? If Mike Tomlinson has been saying it was a disaster waiting to happen, to broadly quote him, why were you not shouting, why were you not jumping up and down; where are the letters, why did you not knock on our doors, as politicians, and say "Look, this is going to be a real problem for students"?
  84. (Dr McLone) We certainly, all of us, were very concerned when this was being introduced, and there are minutes of the joint meetings that we had, that it was being rushed.

  85. Being rushed; who did you say that to?
  86. (Dr McLone) We said it, I am sure, in meetings we have had with the QCA, but we certainly had it in meetings we had with the Joint Council together; so I am sure we pushed it forward.

  87. So you had been saying it to Sir William, whoever was the Chief Executive?
  88. (Dr McLone) Whoever is in receipt.

  89. Did you find it difficult, because there was no Chief Executive?
  90. (Dr McLone) I am talking about two years ago.

  91. Well, two years ago there was a Chief Executive for only a year, was there not?
  92. (Dr McLone) There was, yes.

  93. And then there was an interregnum of nine months?
  94. (Dr McLone) Yes, something like that.

  95. So was it difficult to communicate with the QCA?
  96. (Dr McLone) I think I would ask my colleagues what they feel about that. I think it is important - - -

  97. You are being a bit difficult to pin down on this?
  98. (Dr McLone) We do communicate, yes; the question is, the response.

  99. Yes, but, Dr McLone, I am asking you a straight question. You are saying to me you did see the problems and you stood up and said, "There are going to be problems with this, there are going to be real problems here," you said that to the QCA; you are saying that you do not know what came back, or if anything came back?
  100. (Dr McLone) What I am saying is that we all perceived that there would be problems; we were working with these problems but we did not conclude any answers.

    Mr Chaytor

  101. Chairman, can I just pursue this question of the relationship between the examining boards and the QCA, because I do think this is the heart of the issue, really. And what we have been told so far is that on 12 March there was a meeting with the QCA, at which the line was given out to the boards that there should be no grade inflation this year; on 26 July, there was a meeting with the QCA, at which the boards told the QCA, essentially, there was going to be grade inflation this year, and after that there was nothing and there was no communication, no further meetings, no undue pressure. Now the submission to the Committee by OCR and Edexcel, and this is many, many pages of documentation, is very direct in its criticisms of the QCA, it is sometimes vitriolic in its criticisms of the QCA as being bureaucratic, unresponsive, divided, unsure of where they stand on particular issues, and the Edexcel submission particularly. But here you are almost ignoring what you said - - -
  102. (Mr Kerr) You asked one question to AQA, you did not ask the rest of the panel.

    Mr Chaytor: This is why I am now turning to you, Mr Kerr, and Dr McLone as well.

    Chairman: Mr Kerr, we are giving you the opportunity to contribute.

    Mr Chaytor

  103. Because what I am concerned about is that you are very critical of the QCA in your written submission, but in front of the Committee you are saying, "Well, it's purely a technical matter"?
  104. (Mr Kerr) I must interject. In terms of the events that Kathleen outlined, I agree entirely, the 12 March meeting, the letter that the three chief executives wrote, because we were extremely concerned by what was said at the 12 March meeting; the letter of 19 April did not reassure me, I felt the pressure, I am sure the integrity was clearly there but the pressure put on by QCA was inappropriate. That was my evidence to Tomlinson, that has been repeated in the press outside.

  105. So you disagree with Ms Tattersall, you are saying there was inappropriate pressure?
  106. (Mr Kerr) The events are the same, I think our interpretations are the same; the pressure was clearly inappropriate. To link grades this year back to legacy A levels was only one factor; the most important factor, from Edexcel's point of view, was the student performance, and to depress students' performance based on Government statistics would be unethical.

  107. Was there any further communication between yourselves and QCA after 26 July, before the publication of the results?
  108. (Mr Kerr) No.

    (Ms Tattersall) Can I just say, I had forgotten this when I was answering your earlier question, I am sorry; we had a routine meeting, a meeting that was in the diary for a long time, on 6 August, in anticipation of the results. Now by that time, of course, everything is done, dusted, we know where we are, and all we are talking about is what the results are and the sorts of explanations that are available for those results; so there was that third meeting, which, I am sorry about, I had forgotten.

  109. But there was no inappropriate pressure at that meeting on 6 August?
  110. (Ms Tattersall) No. As I say, by then, it is impossible anyway, even if anybody had wanted to, by then, your results are ready to go out, they are ready to go out to UCAS. So the answer is, no, there was not.

  111. Could I just follow that and ask Dr McLone about the broader issue of the relationship with the QCA, because I think the picture that is emerging is of quite a profound breakdown in communication with the QCA, not only over the issue of this year's results but over the whole issue of the design of the new qualification. Are you saying that you have been arguing your case for a longer trial period, for a different weighting between the two parts of the new qualification, and the QCA has been completely unresponsive?
  112. (Dr McLone) I think what I am saying is that there have been flaws in the system, which it is QCA's responsibility to deal with, and that that has not been sorted, and that is now clear. What I think I am also saying is, what we need to do is look forward, we have got a new Chief Executive in QCA, I think we have every confidence in that, and we must build that new relationship; and I think that we should look forward, in doing that.

  113. But you, as an examining board, are completely exonerated?
  114. (Dr McLone) I say that what we should be doing is working to a completely new system. I think that QCA have the responsibility to define what it is that we have to do in a new structure like AS and A2, it is their responsibility.

  115. That does not answer my question. Are you, as an examining board, completely exonerated?
  116. (Dr McLone) I say that we operated, as Tomlinson said, with integrity, in a proper procedure, to deliver what we had to.


  117. But what we are trying to get out of you, Dr McLone, is, what is your opinion, in terms of the right relationship with the QCA? If it was wrong in the past, is that because of personality, or is it structural; is there something fundamentally wrong with a regulator that, as I look at it, has got an executive role, it has got an advisory role, it has got a combination of roles, has it not, and also it has this relationship with examining boards, it has a bit of a relationship with Parliament, it has a relationship with the Secretary of State? It could not really be called an independent body. Would you like it to be more independent?
  118. (Dr McLone) We certainly would; in our submission to the QCA quinquennial review, we made it quite clear that we thought the QCA was trying to fulfil a whole host of roles, some of which were incompatible. For example, it has an operational role in the Key Stage tests, it has a regulatory role for the awarding bodies, and yet at other times it is making its own awards for tests, for example. It defines operations sometimes very crisply, very precisely, in places where, as awarding bodies, we would say that is not their role; but it is their role to set and define a standard, it is their role to regulate that, it is their role to set that very crisply. That, I would have to say, was not as successful as it should have been. And I think, possibly you were perhaps suggesting it, at the root of it is where its independence is. We believe strongly that we should have an independent, responsible to Parliament, preferably, robust regulator, who will be clear - - -

  119. Why? In the past, have you been frightened of them, because they are so close to the Government?
  120. (Dr McLone) I would not say that we have been frightened of them, at all, but it is quite clear that that has a different impact than if it is independent.

  121. What was your view of the fact that the Acting Chief Executive was a secondee from the Department for Education and Skills?
  122. (Dr McLone) I have no view on that.

  123. No view; you have no view. We are a bit worried about you not having a view on anything. Would you have preferred not to have a senior civil servant seconded from the Department as the Acting Chief Executive?
  124. (Dr McLone) As an issue of principle, yes; but I do not wish to make any comment about individuals.

  125. You do not, and we would not want to do that in this Committee, but what it signifies is a very close relationship between the QCA and the Department, does it not?
  126. (Dr McLone) It is our view that the QCA should be independent and should be responsible to Parliament.

  127. Alright. John Kerr, what is your view on that?
  128. (Mr Kerr) Absolutely, coming from a regulatory background, I do believe that an independent, robust regulator, with one caveat, a right of appeal, particularly as it is not envisaged at the moment, answerable to Parliament, would help to restore public confidence.

    (Ms Tattersall) In general terms, I agree with what my colleagues are saying here. Certainly, we have argued for a very independent regulatory role for QCA, and we believe that that would be the most accountable and transparent way of demonstrating that. If I could say though, where I disagree somewhat with my colleague is in terms of the discussions which did take place between QCA and ourselves about the big issues of timing, and so on.

  129. Which colleague are you disagreeing with?
  130. (Ms Tattersall) With Dr McLone; on the question of the timing issue, for example, we did press our case very, very strongly to QCA. What we do not know is what QCA advised Government on those issues, because those matters are not made public; and this again comes back to the issue of the independence and the transparency of the process. I am sure QCA will speak for itself on that issue, but we did, as awarding bodies, make very clear that more time was needed for the new qualification, and, in particular, to test out some of the technical matters. As I say, it is then for QCA to advise, in the current terms, Government on what to do, and I am sure QCA will speak for itself on that particular matter.

    (Dr McLone) I do not think we are that far apart, Chairman.

  131. You are very keen to agree, Dr McLone, on some of these things?
  132. (Dr McLone) No, no; sorry about that, if I gave a misinterpretation. I do believe that the pressure was there, and I was trying to say the pressure was there.

    Valerie Davey

  133. Is there a formal relationship between your bodies and the QCA, such that if, in these discussions, which seem very general to me, you wish to make an appeal, can you formally appeal, and what formalities are there between yourself and the QCA?
  134. (Ms Tattersall) There is not a formal appeal process; the formal relationship is certainly between QCA and ourselves in the Joint Council for General Qualifications, and, I have to say, that is at a policy level, but we also have several working relationships and working groups, of a technical nature, between QCA and ourselves. So there is very regular communication. But, in terms of an appeal process, that is something that we do not have and which we have argued for, and have been told that the only way in which we could make an appeal is through a judicial review.

  135. But the regular communication is with the Council, rather than you as individual boards?
  136. (Ms Tattersall) No; sorry, with the Joint Council, yes. I misunderstood you.

  137. So your Joint Council is the body that would appeal, in the future, if that was what emerged, or individual boards?
  138. (Ms Tattersall) It would depend on the issue. If it is an issue to do with the system, where we are acting collectively as individual boards, then that would be a matter, I think, for us to act collectively in that context. If it is a matter which affects an individual board, for example, a matter of accreditation, either the qualification or of the individual organisation itself, that will be for the individual organisation to take up.

  139. It seems to me that if you had a robust and, I would appreciate too, a more independent QCA, then having determined that these exams are 50/50, that should be the end of the argument; it seems to me that on this occasion, in what seems to be a very fluid dialogue, still going on, one board has accepted, in AQA, it was 50/50, whereas OCR is still - - -
  140. (Mr Kerr) (? Edexcel - Inaudible - very clear at) 50/50.

  141. And Edexcel; it seems to me that OCR is still arguing the case. Now, if you are still arguing the case, does that not affect, can I ask you, the way in which you are dealing with this matter?
  142. (Dr McLone) No. I do understand where you are coming from; but, no. Our job has been to apply 50/50, and that is what we have done, and that is why I have said that we are looking for A2 to be a considerably higher standard than the old A level, that is part and parcel of the 50/50. My argument was that if we had gone for 40/60, if, then it would have made it easier to deal with in a different way; and I think it would. But that is not affecting the outcome now, we have to deal with 50/50, you are quite right.

  143. A last question. This newly constituted, potentially, QCA would then have more power, or less power, vis-à-vis the Joint Council?
  144. (Ms Tattersall) I think, in terms of acting as a regulator of the system, acting as a regulator laying down the rules for the system, the powers would be similar to what they were, but I think more clearly expressed, and I hope more clearly focused, in terms of the way in which QCA activated those powers in respect of the boards. We think that that will be the cleanest and the most transparent way of carrying out those responsibilities.

    Jeff Ennis

  145. All three witnesses have advocated that we do need to change and redefine the existing role of the QCA and make it more independent from Government, etc., etc. Could I ask our three witnesses, do you have confidence in the QCA as it currently stands, in their role and capacity as regulator of the current exam system?
  146. (Ms Tattersall) Clearly, QCA has had some difficulties, both in terms of not having a Chief Executive and also its Chair, only recently a new Chair being appointed. But I have to say that, in terms of the general working relationship that we have with QCA, at a very practical, logistical, technical level, we have very good relationships and confidence in the people that we deal with. I think, as Ron McLone has said, we now have an opportunity to rebuild relationships at the political, as it were, level with the new Chair and the new Chief Executive. So I have every confidence that we will actually strike a proper relationship with the newly constituted team and, if the QCA does change, with any newly constituted QCA.

    (Mr Kerr) I think I would agree with that. We have a new Chief Executive in place now, and that all three boards should work very closely with him to achieve some aims. But if there is an opportunity to redesign QCA, perhaps it is removing its role on the design of qualifications and focus much heavier on the regulation side.

    (Dr McLone) I agree with everything that has been said; particularly I agree with Ken Boston, when in his speech at the QCA Annual Conference he talked about the restructuring of QCA that was necessary and the reconstituting of it, and I think we have every confidence in Ken that he will deliver that.

  147. It has already been said, Chair, that when we adopt any new examination system you will get teething problems; would it not be fair to say that one of the ways we could cut down on the teething problems with new examinations being brought in would be to cut down on the number of examination boards that we have got, and cut down from either three to two, or even now to one? I wonder what the witnesses have got to say about that, Chair.
  148. (Mr Kerr) The teachers tell us that they value the choice and innovation that three exam boards bring.

  149. I am not on about the teachers, I am on about you?
  150. (Mr Kerr) And we would echo that; we are here for educational services to students and to the teachers, and I think we continue as three boards.

    (Ms Tattersall) I do not really think the main issues that we are dealing with are problems arising from three boards; what I think it is arising from is the problem of a new system, and, inevitably, in a new system you do get the issues that we have been talking about, about interpretation of standards, and I am quite certain that would occur even if you had the one board dealing with it. Furthermore, the volume of candidates that we have in this country taking qualifications, I think, if you put it all into one board, the risk of things going wrong will be far greater than the risk of things going wrong with three boards.

    (Dr McLone) We are, of course, talking about England here, are we not, the English awarding bodies and QCA; but I do not think I have to remind you about that.


  151. This Committee's remit is for England only, not for Scotland.
  152. (Dr McLone) But I do not have to remind you, I am sure, about what happened in Scotland two years ago, when we had one board and one set of problems; and it is not a matter of whether it is three boards or not, one board can make all the problems. And, in truth, if you have only got one board, you could argue that it might all get hidden.

    Mr Simmonds

  153. You have all said, categorically, that you think there was no either perceived or other pressure put down upon you, as examining boards, from QCA; you have also stated that very few grades were changed, at the end of the day. Do you actually think the Tomlinson inquiry was necessary at all?
  154. (Ms Tattersall) I think, by the time the Tomlinson inquiry was set up, the lack of confidence, as it were, in the public perception was such that there needed to be some objective look at what had gone on, and some objective recommendations for action. And, therefore, in that sense, I do think the Tomlinson inquiry was necessary actually to break the deadlock, which we had to, in terms of the public perception and the public confidence in the system. As a board, we were very confident ourselves in the grades that we had awarded, and, indeed, in the interventions that I had made, as a chief executive, which, just to remind the Committee, most of the interventions I made were to lower the grade boundaries, not to raise the grade boundaries, in other words, in favour of the candidates, we were very confident of that, but, nevertheless, we felt it important that we co-operated in full with the Tomlinson inquiry, in order to restore the public confidence that was so necessary.

    (Mr Kerr) There is no real other effective mechanism for reviewing grade boundaries. The inquiries by results procedure will look at re-marking only. So this was the only mechanism really to try to demonstrate to the public how the grade boundaries were set, and, as Kathleen said, to diffuse what was clearly a tension out there.

    (Dr McLone) I think we all welcomed the fact that we had the Tomlinson inquiry. I think the most important thing that was unfortunate was that the whole expectations had been raised of so many students that we had to do something; and the sad thing, and which we are all concerned about, is how many students have had, as it were, two goes, having their expectations raised and then dashed. That clearly is very sad.

  155. There does seem to be a contradiction also in what the three of you are saying about the future confidence that you have in the QCA. On the one hand, you seem to be saying that you prefer it to be an independent body that is responsible to Parliament, and yet, on the other hand, at the same time, you are saying you are happy with the new team that is there, you are happy to continue as it is existing. Perhaps you could explain that contradiction?
  156. (Ms Tattersall) I think we are facing the situation as it is, namely, QCA as it is; there is no indication, as far as I am aware, that QCA is going to be changed in the near future. And I think it is right that we, as examining boards, work in a proper, professional way with the body which exists to regulate the system, with all its other activities, which we somewhat question. So I do not see, myself, a contradiction between a willingness to make the system work as it is, but also saying, as certainly we did in February, when we made our submission to the quinquennial review, that we would prefer to see a QCA that was totally independent, in the way we have been talking.

  157. Before the other two gentlemen reply, could I just say that my question was whether you would have more confidence if it was independent, rather than a willingness to work with what is there already?
  158. (Ms Tattersall) Yes, we would certainly feel that the system could be accounted for and be demonstrated to be more transparent and free of any sort of interference if it were a new system; so the answer is yes.

    (Mr Kerr) The answer is, very simply, yes, we would have more confidence with an independent regulator.

    (Dr McLone) We certainly would.


  159. Can I just take you back for a moment to the unhappiness that so many students felt in the summer, because, in a sense, I get the feeling that it is not that you want to sweep it under the carpet, any of you, but you would rather get on with looking at the future rather than what happened in the past. Something that members of this Committee expressed to me, privately, has it really come out of Tomlinson or anywhere else, why. And next Wednesday, Wednesday week, we will be having Sir William Stubbs and representatives of the headmasters organisation in, both the independent sector and the state sector, to talk to them; those people, the heads associations, they got very, very unhappy about what was going on, in an unprecedented way. Now was that all nonsense; can you explain to us why there was this deep unhappiness? Dr McLone, you were in the forefront of that, were you not?
  160. (Dr McLone) I think there was deep unhappiness because many people were unsure, just not clear, about what was happening and why it was happening; and I understand that.

  161. But, Dr McLone, they had good students, they had predicted they had good results, who did not get them?
  162. (Dr McLone) With due respect, some schools are like that, a good deal of the time; if we take a look at our forecast grades, we would never say, I do not think any of us would say, they have an expert prediction of what the outcomes will be, and that has always been the case.

  163. But you have the whole university system of acceptance, the whole university allocation system is based on predicted grades?
  164. (Dr McLone) It is, it is; but we have done an analysis of the forecast grades, and some of them are very good and some of them are not so good, and it depends, and that has always been the case. It has been true that there have been some people very, very unhappy; there have been some people, I have to say, who have been very happy. We have had a number of letters saying how well they thought the thing had gone, with us. I go back to the fact that a lot of the students out there have done very well, a lot of the colleges worked very hard, and there was a demand, in some way, with Curriculum 2000, that, as an organisation, you gave commitment to Curriculum 2000. And there is a good deal of evidence, on what we have seen, that those organisations that spent time with their students, worked out how they were going to do it over the two years, put some people in for the first January, then into June, then into January, then into June, have actually done very well; the colleges, I have to say, some of them, in particular, have done very well, because they planned it. Not everybody planned it.

  165. Is that the reason why some colleges found that students that were performing exceedingly well in terms of their examination results got an unclassified mark for their coursework?
  166. (Dr McLone) There are very few of those. I know there has been a lot in the press about it, but we did an analysis for QCA, in their inquiry, and, as you will have seen in that report, there are actually very few who actually got a U in coursework; and, in fact, in English, which was the subject which was most under review, nobody got a U in coursework who got As in the examination, when you actually analysed the results. So I do not think that there is actually an issue which goes along with what we are talking about.

    Paul Holmes

  167. As a result of all that has happened over the last few months, you have got a lot of teachers out there who are not at all sure whether they are teaching the right things, whether they understand the system, and yet, within the next few weeks, they have got to predict grades for the students who want to sit modules in December. What are you doing collectively to train, to work with those teachers, to reassure them about what is going on, to avoid all this happening again?
  168. (Dr McLone) First of all, the most important thing is that we are working with QCA exactly on looking at the main points that we need to do to establish confidence and to give guidance; that we have to do fast, and I know Ken Boston says we have got to do it fast, and we will want to do that. We are establishing many more INSET sessions, to be able to advise teachers and to give teachers help. That sort of thing is very important. It is also done subject by subject; it is not done, as it were, globally, we are giving advice to individual subjects where we know there are issues out there.

  169. And how well is that being co-ordinated across the three of you? And, again, I can remember teaching the first year of AS levels and teachers were coming back from different INSET, in different subjects, with different exam boards, with totally different stories of what was going on. Is that still happening, or are you avoiding that now?
  170. (Ms Tattersall) I think the main point, as Ron said, is that we are working with QCA to define more closely the standard that we were talking about earlier in this session; but each of us does run our own INSET meetings, in the case of AQA, we have something like 1,000 meetings lined up in the next two or three months to help teachers to understand better the system. And, of course, we do have now archive material to draw on; so that is a better position than we were in last year, when the INSET material was being provided for the first time. In response though to the first question, if I may, AQA did not have a grading problem, I really must make that clear, and I must make clear that, as I said earlier, we deal with something like 45 per cent of the grades awarded in this country. And so we did not perceive we had a grading problem, we certainly did not have any regrading, arising from the Tomlinson inquiry, and we have many letters from schools, who are saying that they are very content with the service which AQA provided for them and their examination results. So, in a sense, while we are talking about all the problems that have occurred, and quite rightly so, I think one has also got to remember that there are people there who have actually performed in the way that they expected in the exam and been rewarded appropriately for their performance.


  171. Can I just ask you, Kathleen Tattersall and Ron McLone, to stand down for a moment. Would you like to sit to one side, you could even have a breather outside; do whatever you like. We would like five minutes with John Kerr, and then, alphabetically, Ron McLone and then Kathleen; just five minutes, so that we can concentrate questions on particular issues that affect your position. Mr Kerr, we were interested in seeing your interview, with The Times report this morning. Is there anything that, what particularly, at this stage, made you feel strongly enough to say the sort of thing you said this morning, in that very interesting article?
  172. (Mr Kerr) Thank you, Chair. I disagreed with your opening comments, where you said the exam boards should keep quiet, exam boards should keep below the surface and should not have a voice. I really do not agree with that. I think that is perhaps one of the lessons we do have to learn from this activity. We are not very good at explaining to people what we do. I am looking at the faces, going round, and there is still a lack of understanding here, there is clearly a lack of understanding on the press bench, exactly what we do, and it is not that difficult. And I do think that certainly both Kathleen and Ron have far greater experience, and that, together, or individually, we can actually restore that public confidence by explaining what we do. That was why I agreed to do the interview with The Times, and I think it is something I wish to continue to do.

    Ms Munn

  173. So what do you do?
  174. (Mr Kerr) We are a large awarding body, of which half of the qualifications are involved with general qualifications, at A levels; the other half are the B-Tech qualifications, the vocational qualifications, which receive no publicity at all, and these are very good, solid qualifications, which we firmly believe in, these are the qualifications that are in demand internationally, they are in demand from employers and from fellow education specialists.

  175. And, in terms of where you see yourselves going, presumably, like any organisation, you have some sort of development plan, or vision statement, or something like that?
  176. (Mr Kerr) Yes, we are still working on our vision statement; but I think it is really to deliver great qualifications, qualifications that enthuse the learner, the qualifications that teachers find it enjoyable to deliver.

    Mr Chaytor

  177. What are the most important steps to be taken by your examining board and by the QCA to avoid a repeat of this year's affair next year?
  178. (Mr Kerr) As my colleagues have already stated, it is working with Tomlinson, it is working with QCA, to get the standard communicated better to schools and colleges. It is to enhance the training that is provided; we have already provided training to 40,000 teachers this year, we will probably have to do more. And it is getting our message across, that people can trust the grades that are set by the exam boards, and these are very important qualifications.

  179. But you are adamant there is nothing in your existing systems that is at fault?
  180. (Mr Kerr) I am not adamant at all on that. The exam system is still essentially Victorian, it is a large number of pieces of paper; in our own exam board, it is ten million marks, five million pieces of paper, scripts, in a large warehouse, there is little technology that has been applied there. Certainly, the question for me is investment, who is going to pay for that investment and bring technology in; otherwise, we are going to continue with errors and mistakes, which clearly we will strive to minimise, and it is important that we do so, but there are limited reserves within the exam boards, as charitable organisations.

    Jonathan Shaw

  181. If you had the opportunity to pinpoint what you do, if you had the opportunity to pinpoint one particular event, over the course of the summer, that would have been done differently, what would that be?
  182. (Mr Kerr) From Edexcel's point of view, I was very, very pleased with the results this summer, given the publicity surrounding the board earlier in the year.


  183. You were pleased to be out of the public eye?
  184. (Mr Kerr) I was very pleased to be out of the public eye. I was very pleased to see that we delivered the results on time and that we delivered them accurately. So, actually, this whole grading issue came as a bit of a surprise to us. In terms of what I would want to change, piloting of the A2s would have helped enormously.

    Paul Holmes

  185. You talked about the half of your business that goes without any comments, any problem, all the vocational courses; is that because the vocational courses are essentially criterion referenced, and they are not bedevilled all the time by the harp back to the old days of norm reference in A level, and the idea that if there are any improvements in grade passes it must be because things are getting easier?
  186. (Mr Kerr) I think some of the aspects from the B-Tec qualifications are that they are essentially assessed by the teachers, the scripts never leave the college, we have an internal verification system, we have an external verification system, teachers and lecturers have a great deal of confidence in applying these qualifications, and they feel confident and they pass on that confidence to the children.


  187. When, in my introductory remarks, I suggested that most people do not really want to know much about examining boards, I said that in the sense that they want a quiet confidence but they do not want really to hear exam results questioned, as they always are, round about August, which comes at a particularly slow news time. Is part of what you were saying, in terms of your method of explaining, or your mission to explain, if you had a mission to explain, how would it be better done, because in view of the very good article by Len Masterman, regarding, I think he said, "How the papers saw it: press coverage of the A-level controversy (up to the publication of the Tomlinson inquiry)," and if there are ridiculous articles in the Daily Mail that occurred, and Simon Jenkins in The Times, and then Melanie Phillips, again in the Daily Mail, those are articles that are really not based on any reality, you could have explained, how would you have come back and explained yourself?
  188. (Mr Kerr) We get the press we deserve; it is our job to try to educate the press, it is our job to make clear what we do, how we do it, and that is bearing fruit.

  189. And you could see it all going away from you, in the summer, that run of articles, because it was all moving away from you, was it not? Who should have stepped in and said, "Actually, these are the facts; this is what's happening here," who should have done that, you, the QCA, who?
  190. (Mr Kerr) Referring again, an independent regulator, a regulator who was respected by the public, could stand up and say, "Don't worry; these are the issues, we can resolve it." But I think also a Government spokesman standing up and saying that they had confidence in the exam system, confidence in the exam results, would have gone a long way to allay press speculation.

  191. So the Government did not give you enough support; the Government should be out there, batting for that?
  192. (Mr Kerr) I would prefer to see the Government taking forward the lead in promoting the qualifications, and promoting confidence in the qualifications.

  193. Mr Kerr, thanks very much. And can I have Dr McLone back in the seat. Dr McLone, the reason I really wanted to get you on your own was because you are a consensus builder, and I felt that I wanted really to find out more in depth what you individually thought about what had been going on in the last couple of months, and also your view of how you could better do your job. How do you think you can better do your job as an examining board?
  194. (Dr McLone) I think that it is absolutely essential that we have a clear remit in which to operate, given by a regulator. I also believe that what we have in the system that we have got is something, as John has said, which is not transparent, and that we need to move on the examination system we have got in this country to make it more transparent, but also to bring it into the 21st century. That will make it a better job.

    Mr Turner

  195. I am still worried about your chart, Dr McLone, because Val asked you, essentially, were you arguing with the 50/50 split or not, and you said, no, you were not, and then proceeded, in my view, to do so, by saying it is still 40/60. And, putting it at its simplest, what we are saying is, an A level is worth 1, an AS level, according to the Secretary of State, is worth 50 pence, but, according to you, it is worth 40 pence?
  196. (Dr McLone) I do apologise, if I have not explained it properly. What I am saying is that we would have preferred it to be 40 pence, because that would have been a recognition that it was not half an A level. But we worked on it being 50/50, in terms of having to get a balance between the two, yet it is something which is not worth 50 pence but you are having to call it 50 pence; that being so, you have got to have something which really should be 60 pence, and you are going to call that 50 pence. All of that means that you have got this complexity of where you are at; and I am sorry if the arithmetic does not add up, but I think it does.


  197. I think we are getting to the heart of this problem.
  198. (Dr McLone) We have to apply 50/50, and, in my view, and I think in lots of other people's, the 50/50 meant the A2 was harder than A level, otherwise it did not stack up.

    Mr Turner

  199. Ms Tattersall, I think, used an expression relating to the maturation of the candidates, maybe she did not but somebody did; no, I wrote it down, actually, on the basis of what she said. This is making assumptions about the maturation of the candidates over the two-year period, is it not, and I still do not see how you can say, on the one hand, that your chart shows 40 pence because the student in the lower sixth is only broadly capable of achieving a lower standard, and therefore you have got to top it up with a greater achievement in the upper sixth?
  200. (Dr McLone) Indeed; and that, I think, is part of the flaw in the system, which we referred to earlier. And, also, if I go back to my analogy with the university world, which I was in, if we had said it was 50/50 we would have been asking too much of the final year students, that is absolutely true, but the 40/60 made that balance work. The fact that you say 50/50 means that you are really asking an awful lot, because you are asking something that is not A level; it is this comparison with what we were doing before which is the problem, because many students in the modular course took these three units in the first year sixth before, yes, and they were A level standard, by definition. Now we have them taking it, and we say, "No, it's not A level standard." That has been the problem; it is a flawed process, and it has been flawed, I think, because we have not had the right definition, and the definition should come from the regulator.

  201. But when answering David Chaytor, you were asked about communication with the QCA, effectively, over the last two years; as I read it, it was before the last two years began that the failure of communication, or, at least, of agreement, took place?
  202. (Dr McLone) The roots of the problem certainly happened then. What needed to be done, in my view, was, over that time, to have recognised that the roots of the problem were going to be difficult. Now we spent a lot of time, of course, in 2001, focusing on AS properly, because we have not run A2; and, properly, I think we got AS right, it was welcome. What we did not do, and what I think everybody feels we should have done, is have some exemplar material, so that everybody understood what A2 was, we did not have it, and if I go back and think about it, collectively, or individually, I think, driven by QCA, we should have had exemplar material.

    Valerie Davey

  203. You say there was not pressure from QCA; was there pressure from the universities, in any way, in any way over this new process?
  204. (Dr McLone) I would not like to say. There was certainly no pressure on the individual awarding, absolutely not; but, in terms of that 50/50 decision, I suspect it was a contributory factor, yes.

  205. So universities, which we have not really mentioned very much, are the other factor in determining how they value the outcome of your exams?
  206. (Dr McLone) I think there was a concern within the universities, and, as I said, I come from the sector, I can understand it, there was a concern about what on earth AS was supposed to be; and if it was not valued at 50, I think the understanding of the universities was that it would not have been something that they could value. That must have been, in the end, a political decision, of one sort or another, and I am using 'p' with a small 'p', not a large 'P'. But, nonetheless, that must have had an effect on the final decision about whether it is 40/60 or 50/50; they tend to be decisions that are not made on the assessment structure but on other decisions.

  207. So where does the university influence come into this debate; is there any debate from your Council, or the QCA, or is it all done then by the Government?
  208. (Dr McLone) I would say it was done directly, myself, it would not have been through our Council; directly to QCA or to Government, I would think.

  209. To QCA or Government?
  210. (Dr McLone) Or. I would not know.

  211. Are we saying that this is another body that does not understand the system that you are operating?
  212. (Dr McLone) We need a lot more transparency for everybody. I think the business about whether the AS will count as one point, or not, was something which certainly did not get thought through alongside what that would mean if you had put it in terms of assessment structure; and, of course, there is an argument which says the two should be divorced anyway.


  213. But, Dr McLone, the worry the Committee would have, from your evidence today, particularly after your remarks just now, would be that here is a flawed system, you have said it is a flawed system; on the one hand, some of your colleagues have said, "But we've got great confidence in the new team in QCA," but you are saying, "It's a flawed system, we have not said it right, the super-tanker is on its way, kids are doing this AS level, they are on their way, they are on the new A level system, they are on their way, very soon we'll be in June again." But you are saying you are happy with this system. On the one hand, you are saying, "It's a deeply flawed system, we're all on the way to the next disaster, the next iceberg;" what are you saying?
  214. (Dr McLone) I do understand the question, and I think there is a tension; there is a tension between picking up plants, as I said, and inspecting the roots, because you want to embed something that is already going. There are flaws which I believe will be put right through Ken Boston's procedures over the next few months; those are the flaws, and that is the way it must be put right.

  215. Right; so the QCA can get it right. We are not suggesting that you pick it up by the roots and replace it by the International Baccalaureate immediately, but what we are saying is, you have identified the flaw but you have not actually said, at which I am surprised, how you will put it right, by next year?
  216. (Dr McLone) By next year, I think we will put it right, by talking to Ken Boston's task group and Ken Boston's Programme Board and the arrangements he is getting right, and Tomlinson's inquiry, I think we will be looking to get things out that can match next year and get it on the road. I think there are longer-term issues that we will have to address, like six/four units, because, eventually, there is really too much assessment going on, and it is overburdening our teachers and students.

  217. Dr McLone, we will have you back to talk about that at greater length; thank you. Kathleen Tattersall, can I ask you, for the final spot. Can we just, seamlessly, sort of move from that question to Dr McLone to you, in the sense that, right at the end there, he said, "far too many examinations." As I say, the Committee has been to New Zealand, where they are really at the opposite end, hardly any examination and testing, a large number of educators there saying, "We ought to have more appropriate and accurate evaluation of how students are doing; we don't have it." And they are looking at our system, they do not want our system because they think we have gone to an extreme, but they would like something. Are we at the extreme, should we be fighting back and getting less examination, I know it is difficult, you are in the exam business, are we overexamining our students in this country?
  218. (Ms Tattersall) I think we are externally overexamining our students, and that, I think, is where the problem lies; and certainly there has been a trend to external examining, over the last 14, 15 years, which actually has swept away some very good coursework-based examinations. So I distinguish between assessing of students and externally examining our students, and I think the balance, as it were, has tipped too far to external assessment, and, as I said earlier, that brings with it some problems, such as recruiting examiners to fulfil our requirements. So I would prefer to see the pendulum swing somewhat back, to enable teacher assessment to take place, but, in order for that to happen, I think we would have to recognise that there would have to be a lot of training of teachers, for a start, in assessment methods, there would have to be very robust systems of moderation, and there would have to be, I think, a turnaround of public perception of the value of teacher assessment, because that was where it went wrong 14 or 15 years ago, when people really started questioning whether that was valuable.

  219. Yes; in New Zealand, they said every school thought they were assessing objectively, independently, but they were not?
  220. (Ms Tattersall) And the important thing, therefore, is moderation; and, therefore, as an examination board, we would certainly welcome more emphasis on the teacher involvement in assessment, but it would have to be in a context where there was a very clear framework of moderation provided by some external body, of which we would be one of them, we hope, and we would provide the exemplar material for teachers, we would do the training of teachers, and we would moderate samples of the work which they assessed.

    Ms Munn

  221. John Kerr said nobody really understands what you do, as examining bodies, and Ron McLone said earlier that OCR were doing things differently from the other two bodies. Can you just explain, what you told us right at the outset was that you complied with the Code of Practice in terms of setting the grade boundaries, and just explain how you do it, so we understand really very clearly how that is different from how OCR do it?
  222. (Ms Tattersall) I think what Ron McLone was talking about was the information that we provide to the awarding meetings when they are making their decisions about grade boundaries. And we provide, at the outset, both the candidates' scripts and some statistical information which will help the awarders come to a judgement about those scripts; and that statistical information, as Ron McLone said, is GCSE mean grades, it is AS performance of the candidates who are being judged on the A2, it is information in normal circumstances relating to the previous examination, in terms of how candidates performed, and we provide that from the outset. So we provide parameters, which show very clearly what the impacts of judgements are going to be and how they might compare with, let us say, the GCSE mean grade data. We believe that interaction between the evidence is important for people actually to understand the scripts themselves; because there is an awful lot of research evidence to show that if you simply present scripts in a vacuum then people are not very sure exactly where they relate to the different grades. So we provide as much information as possible from the outset, and that is the difference between ourselves and what I think Ron McLone was saying.

  223. Just explain that to me again, because what I understood Ron McLone to be saying was there is a clear difference, and this is something which has been confused throughout this debate between marking and grading, and he was saying, as I understand it, that OCR mark the scripts and then use statistical information to help with the grading and the grading boundaries. Now you are saying something different?
  224. (Ms Tattersall) No, I am not really. I am taking it from the point of the grading boundaries. All of us have a very clear procedure, in terms of the marking of the scripts, the standardisation of examiners, they all have a meeting where they are standardised.

  225. So when you talk about the awarding meeting, that is the grading, that is not the marking?
  226. (Ms Tattersall) It is the end of the process. No, all the marking has been done, or should have been done, by that stage; and then there is a group of people whom we call 'awarders', there will be a Chair of Awarders, who is the person who makes recommendations to the Chief Executive, and there are the various chief examiners, or principal examiners, for the different components of the examination, and they will have made recommendations as to where they think the grade boundaries should rest. And the awarders as a whole will get that information, together with the statistical data that I have just mentioned, together with a range of scripts, which cover the various recommendations which have been made; and, within that, the awarders have got to try to find the defining mark between one boundary and another, and it is not easy. And if you actually look at the range of decisions which awarders make, some will believe that a mark of, let us say, 40 is the mark, others will believe a mark of 39, others believe a mark of 41, and so on, and somewhere somebody has to come down and make a judgement on that matter.


  227. We understand that; but what happens when a senior examiner, the most senior examiner, of a board, gets in touch with this Committee and says, in the whole 30 years that he has been in the examining business, he has never known a year where, in the final meeting, after the marks have been agreed, that they then are especially called back, as chief examiners, to a meeting, to be told that all the marks in that subject have to be changed? What is going on out there, when that can happen? This is something that was communicated to this Committee, anonymously, because the person, in terms of the chief examiner of that subject, did not want to be identified. What is going on, if that happens?
  228. (Ms Tattersall) I have to say, that did not happen in my board, so I cannot actually account for what you are actually describing; and it does not happen in that way in my board, the recommendations come through to me, as the accountable officer, and in the vast majority of cases the recommendations stand, I accept them. In the very small number where I say, "I am not quite certain whether all the evidence has been properly taken into account," then the normal procedure in AQA is for that then to be discussed with the Chair of Examiners, and some accommodation of view is arrived at. In some instances, I might have to decide that a different mark, and it is usually one or two marks that we are talking about here, would prevail, and, as I said earlier, some of the decisions which I took, in the very, very small number of cases where I made a decision, the majority of my changes were actually in favour of the candidates, they were actually to drop the mark, not to raise it.

    Ms Munn

  229. I am just getting even more confused now, because I am not sure how your process is different from OCR's, it does not sound different?
  230. (Ms Tattersall) Obviously, I cannot account for what Ron is saying, in terms of it being different. I think what he was saying was that some of the statistical information, which we introduce at the very outset of the awarding meeting, because we believe that to be transparent, open, above board, everybody knows what is going on, might have been introduced into the OCR procedure at a later stage.

  231. And do you think there is something different about script selection, which was the other bit, where it starts to get very technical but which seems to be very important in terms of grade boundaries; is it different?
  232. (Ms Tattersall) The Code of Practice very clearly lays down that the script selection is made by the awarding body staff, in connection with the recommendations which have been made by the principal examiners for the unit or the paper concerned; so I doubt very much if there are differences really in how we operate there, because there is a very clear statement in the Code of Practice.

    Paul Holmes

  233. Can I ask you, again, individually, the question I asked everybody collectively. We heard a few minutes ago from OCR, we were talking about the grid, showing whether 40 and 60 adds up to 100, or 50 and 50 does, and we have heard you talking about whether a boundary should be 39 or 40 or 41. Should not the exam boards this year, or in the last two years, have been saying, to the media, to the QCA, to the Government, that, that thinking, really you have got to move on from that, because there should have been a quantum shift upwards, as a result of the new exam system that has been introduced, that it should not just be measured within 1 or 2 per cent against last year's and the year before and the year before?
  234. (Ms Tattersall) I think that really is precisely the issue that, as awarding bodies, we took up with QCA in March, when some language, which might have suggested that we ought to be having the same percentage of candidates, was being used, and we took up that issue very firmly and very clearly in the letter that we sent on 22 March, and which then, in my judgement, was resolved by the letter which we had back from William Stubbs. In terms of the quantum shift up, as it were, you referred earlier to a 4 per cent rise, and I think you were suggesting that perhaps it ought to have been a 9 per cent rise, or whatever; now, if you actually look at individual subjects, you will find that there are 8 per cent here, 9 per cent there. Four per cent is the general, overall, national shift across the three awarding bodies; look at it in individual subjects and you will find some very different patterns emerging. And we have not done this analysis yet in AQA, but I am suggesting to my colleagues that we do it, as to whether those shifts were different in those subjects which were modular beforehand from those subjects which moved to a modular system in 2000; and that is an analysis which certainly we can have a look at, and we will be happy to provide the Committee with information later on.

  235. If the average pass rate this year was a 4 per cent increase, what were the sorts of averages increases over the last four or five years?
  236. (Ms Tattersall) It has been at round about the1 per cent, sometimes less than 1 per cent, level, but it has been a very marginal change over the years. But, again, if I can pick up on the modular theme, if you go back to round about 1993, when many of the subjects, particularly in the sciences, started to, as it were, go modular, you did actually find the shift then at the Grade E and above level, which was greater than the normal pattern in other exams; and that was in a system where you had a greater facility for retaking than now, because there was a limit on the retakes.

    Chairman: I have asked Dr McLone to come back, and he has very kindly agreed, because I think Meg Munn was not happy that she quite fully understood the difference between the two approaches of the two boards; so would you like to rephrase your question, Meg Munn?

    Ms Munn

  237. What I am trying to get at is understanding the point that you made earlier, Dr McLone, which was about saying that you were doing it differently. Now Kathleen Tattersall has explained to us what they do; is that different?
  238. (Dr McLone) I do understand what Kathleen is saying, because we have had these discussions many times. They are both within the Code of Practice; the whole thing about the Code of Practice says that you have got a balance between examiner judgement and statistical evidence. I have to say, I am going back a few years now, back to the Midland Examining Group, which was part of one of the first GCSE groups, along with the NEAB and SEG, and so on; we always took a view then that what we wanted to do was to make sure that the examiners had the first go and talked about it and then looked at what the impacts were. It is sort of very much a bottom-up process; in a sense, I think the Midland Examining Group said it was an accountable process, because you could see what was happening with the statistics, because then it was evident. It is true, there are other ways of doing it, and one of the other ways is, as Kathleen has said, to produce a good deal more of the statistics to inform where the scripts are selected in the first place. That, essentially, is where we are at.

    Ms Munn: Thank you. I understand.


  239. Can I come back on a question I gave, that it was one of your examiners, chief examiners, I was talking about, who approached this Committee, who approached me, as the Chairman of the Committee, because, in the 35 years, I think it was, he had been an examiner, and now chief examiner, of a subject, he had never had the process that occurred this summer ever before, to have had the final meetings of his exam board, to have come to some conclusions about the marking, and then to be pulled in by a conference of heads of examining boards to be told that grade boundaries were going to be moved. Everything you have said today has said it has been business as usual, it has never been any different; but here is one of your chief examiners who said something very different happened this year?
  240. (Dr McLone) Indeed; and, without knowing the subject, of course, I cannot actually directly comment on what an individual would say. The difference this year has been, it is the first time in 50 years to have such a fundamental change of A level; it is not different in the practice, and it may well have been that, in his subject, or her subject, I would not like to say whether it is his or her, nothing has happened significantly, but this year, in a few subjects, I have to say, in most of the subjects, did not get such substantial issues that have arisen, but in some subjects, obviously in this subject, there was this difference which has come by looking at what they have suggested against statistical evidence that has been more dramatic than in the past. And we have said, and I say again, it is a major shift this year. The way we do it obviously works very well in circumstances when it is maintained year on year and it is a regular, consistent standard, but if you are working, again, with an A2 standard, which, I still submit, we did not know, we had no exemplars, that has provided the issues for some of our examiners, it is absolutely right. Remember, I look across all subjects, he is looking at his own subject.

  241. So it is not surprising that some of these people, that saw themselves as guinea-pigs, might now consider themselves sacrificial lambs?
  242. (Dr McLone) I think it is unfortunate that we had no trialling done before we made such a major change.

    Mr Chaytor

  243. Yes, but, to Kathleen particularly, is not the root cause of the problem the fact that A level has this unique means of assessing the grade boundaries, we do not have this in awarding university degrees, the degrees are not moderated by students' performance at A level, we do not have it in the standard attainment test; and do you not think there has to be in the future a move to a criterion reference system for AS and A2?
  244. (Ms Tattersall) We are not a norm reference system, I think that is the first thing that I would wish to say; we moved away from norm referencing many, many years ago. I think, at some point in the 1980s, A level ceased to be a norm referenced examination. Nor are we fully a criterion reference examination, but we have, as it were, moved along the scale more to that by defining some criteria to underpin the grades, and those criteria are defined at A level overall. If we moved entirely to a criterion reference exam, then I think you have got to take the consequences of that; namely, if you have not mastered whatever is determined for the grade, you will not get that grade, however, as it were, good you are.

  245. But most parents would assume that should be the case, would they not?
  246. (Ms Tattersall) But what we do have is a system which is a soft criterion referencing, for want of a better term, where there is some compensation for a weakness in one skill area, with strength in another, and, in that sense, you could say, it is a little bit of a fudge, when it comes to the criteria. But it is a system which does reward attainment at the more general level than some very specific criteria would do, and I think it is a system which has served students exceedingly well over the years, and, indeed, if you look at GCSE it is exactly the same sort of system, if you look at Key Stages 2 and 3 then I think what we are talking about there is a pre-determined level of attainment, which is only slightly moderated when the students have actually done their SATs. Some of us would say that, in some ways, criterion referencing is fine, but it is when the students actually do the exam that some of the criteria begin to break down, because it is not like that in the real world. So, in a sense, I would argue very strongly for the soft criterion referencing system that we have, provided we have a little bit more definition of those criteria, but not so specific that we are going to cut people out of the grades.

  247. But do you think we have this soft criterion referencing because we have this overemphasis on external assessment, and if we had more internal assessment there would not be the need to have the methodology for the external assessment that was designed to compensate for any ( protection ?) ?
  248. (Ms Tattersall) I think we would have exactly the same issue, but we would have to have descriptors which enabled teachers to mark work consistently; and the fact is that students do not perform in consistent ways, and, therefore, there has to be, as it were, some sort of compensation for the way in which students strive to meet the criteria, and that is what our system does. And I think it is irrelevant, whether it is internal or external assessment, to actually apply the criteria that we have. I am all for making the criteria more explicit, trying to reach criteria which are better understood by everyone, but I really do think that we would be in trouble if we tried to rely entirely on criteria for our system.

    Chairman: I think that we have to end the session now. Thank you, all of our witnesses today, who have taken the time also to enjoy a rather different format we have played with today, and thank you very much for being so flexible. Can I say, to quote John Kerr, perhaps a mission to explain, I was thinking this when you were talking about, of course, everyone knows we have moved from norm referencing to soft criterion referencing, but there are a few people in my constituency who did not quite realise that that had occurred. Perhaps it is part of the role of the QCA and the examining boards to tell parents and students that that is the case. Thank you.

    KEN BOSTON, Chief Executive, BEVERLEY EVANS, Deputy Chief Executive, and KEITH WELLER, Head of Qualifications Division, QCA, examined.


  249. Can I welcome Ken Boston to the Committee, and say, when we met the QCA in May of this year, we were trying to persuade the then Chairman, Sir William Stubbs, to get on with getting a new Chief Executive, and we are delighted that, shortly after that, he succeeded in doing so; so welcome aboard, at a particularly interesting time in the development of the QCA, so welcome indeed. And Keith Weller and Beverley Evans we have met before; but welcome. You have been sitting there listening to the evidence that we have been taking from the examining boards, and I hope that that will give you a clue as to the sorts of questions that we are going to be asking you. We pushed them pretty hard, in terms of where they were coming from, in terms of their relationship with the QCA, and there did seem to be a deep ambivalence; on the one hand, they wanted to work with you, obviously, as a new broom, a new Chief Executive, a different personality at the helm, and with the new Chair that has just been announced, but there was also unease, was there not, about the status, the independence, the split roles of QCA. Dr Boston, could you tell us how you view getting the show back on the road; what is your vision of how you will sort all this out?
  250. (Dr Boston) Well, Mr Chairman, I will start by saying that I certainly have no magic wand, and I am not at all sure that the path into the examinations in January and in June will be smooth; there are some major problems and some major risks ahead of us. Certainly, we will be able to respond to the Tomlinson recommendations by the end of November; we will have before us then better generic statements of standard, we will have a revised Code of Practice, we will have made considerable progress in getting specific exemplars from this year's exams, for A2, which we did not have before, but getting that all understood in time for the 2003 examinations is a big challenge. We also have some enormous logistic problems ahead of us, in running the examinations. We have been stretched in the past, or the awarding bodies have been stretched, to find sufficient examiners; we have this incredible process where 24 million scripts go round the country in a matter of weeks. We have little control at the moment, or virtually no control, but need some control, I believe, on the number of late entries for examinations; and I am not at all sure, unless we pull all this together into a better managed system, that we will not have strife ahead of us again. Now so far as the QCA is concerned, I think it needs to take a far more directive and management role, so far as its powers allow, in determining what goes on. I am not at all sure, for example, that there is real benefit in having awarding bodies able to take slightly different, but nevertheless significant, approaches to implementation of the Code of Practice. I am not at all sure, at the moment, until we have done further work, that we might not run into trouble with one awarding body, or several, again. All of these things urgently need to be attended to. Now so far as the QCA is concerned, as the independent regulator, it needs to have the authority and the credibility to be able to make statements publicly about the state of the examination system, be believed and have the power to fix them; it needs, in my view, to have some degree of greater distance from Government. I do not believe that there has been any evidence of Government interference in standards, or in the work of the QCA, or in the work of awarding bodies, but if it is to be a credible public authority there needs to be the appearance of independence. The other side of that is, there needs to be greater distance, I think, too, between Government and the awarding bodies. If I were here to regulate a financial market, a market in financial services, I would expect the providers of financial services to be totally at arm's length from Government, and for the regulator to bridge the distance between them. Similarly, there must be, in my view, conspicuously, clear blue water between the awarding bodies and Government, and the bridge across that is the regulator; now that is not conspicuously apparent at the moment. I believe the QCA has acted independently, from my reading of all the documentation, and, believe me, my mind has been concentrated wonderfully on the documentation over the past few weeks. I do not believe there is evidence of political interference or pressure on QCA. I see no evidence, but I take Mr Tomlinson's report, of pressure from the QCA on the awarding bodies. But it is clear that the independence of the organisation is not transparently there, it is not unambiguously accepted, and it needs to be, in a far stronger and clearer way.

  251. Would you like to see it on parallel lines with Ofsted?
  252. (Dr Boston) I think Ofsted is a very good model in the education area, yes. There are other models of regulators, I guess, both here and overseas, in completely different sectors, in my home country, for example, in the transport sector, the financial services sector; there is not in the education sector because education, in the states (where I grew up ?), in Australia, is not run on the basis of a competitive market between organisations, competing on the basis of not price but quality of service.

  253. Let us just probe you a little bit though. You are saying, you, the QCA, should be the bridge between the awarding bodies and Government; what is the relationship, as you perceive it, and has been, between the awarding bodies and the Department then?
  254. (Dr Boston) There seems to be, from the evidence I see, quite close contact between DfES (the Department for Education and Skills) officers and individual officers in awarding bodies, at a variety of levels and for a variety of purposes, all of which, I am sure, are benign. But, nevertheless, in a situation where there is a regulator, I believe that relationship is not a desirable one.

    Jonathan Shaw

  255. You described a situation that is going to require QCA to undertake a considerable amount of work to restore public confidence. When you applied for the job and you had the interview, what did QCA, Sir William Stubbs, say to you, "This is a well-oiled machine," or "We're in a hell of a mess"?
  256. (Dr Boston) The reason I became interested in the job was because the QCA does have a very high international profile, it is the international benchmark, as a qualifications and curriculum authority, far broader than simply something concerned with A level examinations. There are, in fact, 117 different awarding bodies, many of them, the majority of them, by far, in the vocational area. Qualifications and curriculum authorities have been introduced in many western countries fundamentally for the purpose of maintaining standards, enhancing standards and responding to the workforce skills needs of the countries in which they operate, the building of social and human capital; that is the job that I came to do. I also came to manage the operation. Now I have walked into the situation where I find, since taking up the position, which I took up on 12 September, but, being here in August, just privately, watching the examination system and the results come out, that a whole new set of priorities have emerged, as a result of this real problem that has occurred with the A level examinations, and which is the product of a series of mistakes made by Government, by QCA, by awarding bodies, and a lack of common understanding across the country about what standards are and how they are determined, and we have seen that lack of understanding in here today. Here is a real problem that needs to be addressed; and that is my task, to take that on. I am simply saying, there is no magic solution here, it is a long, hard row ahead of us, and I can give no guarantees, except the capacity to point the organisation in the right direction, work with the awarding bodies and Government and the Head Teachers Association and teacher associations to try to get it right.

  257. You have been fairly direct, in response to the Chairman's questions about how you see the organisation should be set up; are you going to continue to be as direct, if Government makes recommendations, or you make recommendations to Government and they do not accept them, will you stay?
  258. (Dr Boston) The job is that of a regulator, I report to a board, the board is appointed by the Secretary of State, but it would seem to me that the QCA is an organisation outside the Civil Service per se, it is a non-departmental government body, it is there to maintain and defend and protect standards, it is there to guard standards, it is the watchdog, and the watchdog occasionally must bark.

  259. Just to move on, did you bark; were the examination boards barked at too much, do you think that there was undue pressure put on them?
  260. (Dr Boston) I was not here when the events that were discussed by Mr Tomlinson, and have been referred to in the discussion today, were alleged to have taken place; all I can do is read the documentation available to me, and I find no evidence there, I read Mr Tomlinson's report and I take that at face value. It is not for me to say, in that instance, when I was 12,000 kilometres away, whether undue pressure was put on them or not.

  261. And do you think the removal of Sir William Stubbs will restore the confidence that you spoke about?
  262. (Dr Boston) That was not what I said, and I do not have a view on that matter.

    Valerie Davey

  263. You have talked about the international element, I would like to pick up that, just quickly, before we move back to the main issue of today. Has this issue within Britain affected the international nature of your work, as a QCA?
  264. (Dr Boston) I would guess the answer would be, yes, in that all our attention has been focused on dealing with this issue, of which we are, along with others, partly the cause. So that the key priority at the moment, the real key priorities are to overcome the problems in the examination system, urgently. There are two levels with that, implementing Tomlinson, the first part of his inquiry, and attending to the immense risks that reside in the logistics of the operation, and restoring the credibility and authority of the organisation; they are the two key priorities. And the international work takes, clearly, second priority, second preference, to those.

  265. You were very clear in saying that the QCA had made mistakes; what were the mistakes that QCA made, and how are you going to put them right?
  266. (Dr Boston) I think that the key mistake was not perhaps being vocal enough with the time-line issue, when one goes back to look at the way in which this was implemented. A change of this magnitude really should take three years of development and two years of piloting, and the piloting is so critical, to get the exemplar papers, which set the standard. We have a generic statement about standards, about the A2 being harder and the AS being an easier paper, as it were, we have grade descriptions for English, for History, for Physics, for Chemistry, for Mathematics, which describe in terms of perhaps 200 or 300 words quite analytical, well put together documentation of the sorts of knowledge, understanding and skills and analytical abilities a student should have at each level of a grade description; but you cannot take that any further and operationalise it until you have actually got exemplar material. Now that is what we did not have; and, in fact, the time-line was very rushed. The final specifications were finalised in January 2000, teaching began in September 2000 and the first examination was in June 2001. Now it is quite clear that that sort of time-line makes it very difficult to introduce a change of this type without real risk occurring. Reading through the documentation, it drew to the attention of Government several times the impact of the time-line, so did the awarding bodies, but perhaps we should have been more strident at that time about where we were heading. I think that was the key problem. At the same time, of course, as all this was happening, a whole lot of other things were going on; there was Key Stage testing going on, there were changes to the GCSE, to the GNVQ, there were examiners being stretched to the limit in all sorts of things other than A levels, specifications, or syllabuses, were being written everywhere, 3,500 new specifications were written between 1998 and 2002, when you go to the exact dates, it works out at an average of 15 new specifications, new syllabuses, a week; now that is stretching an examination system to its absolute limits. And it is quite clear that we have reaped the fruit of it in this most recent event; and there is no sort of quick fix, that it is only a matter of attending to that, or attending to that to get it right.

  267. The examining bodies, therefore, the awarding bodies, were right in their criticisms of you, and they made those points, I think, in a different way, perhaps. What would your criticism be of them?
  268. (Dr Boston) I do not have any specific criticism of the awarding bodies; one can point to areas where, with the benefit of hindsight, there could have been better performance. The problem really only lies with one awarding body, and there are two issues that seem to me to reside there; one is the different approach to implementing the Code of Practice, which seems minor, and might be minor, but if it did have an impact, and I do not know whether it did, but we would need to sort that out, then that impact clearly had a very bad effect upon a generation of young people. The other is this issue of standards, and the definition; with respect, I do not agree with my colleague, Dr McLone, that we are really talking about a new standard, with A2. In my view, it is better to look at there being two examinations, of different levels of difficulty or demand. For 50 years, the A level has been judged by one examination at the end of two years; we have now changed to a system where there is a less demanding examination at the end of the first year, or, if you like, in the language of the students, an easier examination, and at the end of the second year there is a harder examination. I do not believe it is an examination pitched halfway between the old A level and the end of first year university standard, that is inappropriate for these young people.


  269. But you were sitting in this room when Dr McLone gave his evidence, he seemed to think that that was the case, that we had two examining boards marking on the A level concept, with Dr McLone not sure what level he was marking on?
  270. (Dr Boston) I think that we would all be more certain, and I agree with Dr McLone and others, if we had the exemplar material that would actually show you. I have been back through, say, some History papers, I looked at the A2 History papers and the AS History papers, and I looked earlier today at a question on Nazi Germany. Now the AS paper was a question that took students to some source material, one was a cartoon and some short pieces of text, and then had a 30-mark question which was broken up into sections of five marks, five marks, ten marks, five marks, and something else. The A2 paper is a paper which also had source material but it was heavy-duty, political literature, and then the question was only in two parts, a ten-mark question and a 20-mark question, making the 30 part, and was far more demanding. Now, the young people who do that exam, that is a hard exam, and the other one was an easier exam; but the two of them, when they come together, and you will perform probably not as well on the harder exam as on the other, when the two of them come together, that gives you the A level, and it is our job to ensure that the awarding bodies, that the standard of that is the same now as it has been for 50 years. Now we will only do that with certainty when we have got the exemplar material, and we did not have the exemplar material for this round of exams; but my starting-point is, let us talk about level of demand rather than a standard, there is one standard, and that is the A level standard.

    Ms Munn

  271. Coming back to this issue then of different approaches to the grading process, which we were exploring before the break, you said at the outset you do not think it is a good thing, or you probably do not think it is a good thing, for there to be different approaches among the examining boards. How would you determine which process of the two, or indeed a third process, which I cannot think of, but which process you would want to use, what would you use to determine that?
  272. (Dr Boston) I think the key to that lies in the revision of the Code of Practice, that Mr Tomlinson has asked, or directed, that we undertake, which should remove the capacity for different approaches in that way.

  273. I accept that, I am taking that as a read, that you have said there should be one approach. How would you decide which was the fairest approach?
  274. (Dr Boston) I would seek the advice of the experts in my organisation, and I am not in a position tonight to say which is the better approach. But, clearly, in the revision of the Code of Practice, this is an issue we need to address, and we are addressing that through the A Level Programmes Board, which meets tomorrow again, its second meeting since Tomlinson came down, it includes the heads of the regulatory bodies of England, Northern Ireland and Wales, the heads of five awarding bodies, representatives of the head teacher associations, and the teacher associations, and it is in that body we are going to sort it out.

  275. So are you saying to me that that would be on the basis of some expert evidence as to which was the fairest approach?
  276. (Dr Boston) I might ask my colleague, Mr Weller, who is, in fact, doing the detail of this, to come in, Mr Chair, if you would be agreeable.

    (Mr Weller) The Code of Practice is revised every year, actually, for A levels, and for other examinations, and it is done in the light of what happens operationally; we look at the Code in action, we look at it with the awarding bodies, and we adjust it, if it is not doing the job effectively. There are always areas, I think, that the Code cannot go into in the finest level of administrative detail, we would be doing the awarding bodies' job then, and there are always areas where you decide there are various ways of doing the job perfectly reasonably, and that is the way the Code operates. If it transpires that, in doing the job reasonably, through various methods, we have some cause to doubt whether they are equally efficacious, we will talk with the awarding bodies about that and establish which of those methods should be codified. We do seem to have an issue here where there might be some of that kind of discussion necessary.

  277. Efficacious, or fair?
  278. (Mr Weller) Both.

  279. Can I just follow that up and ask Beverley Evans, because when you came to see us in May 2002 you said to us that, A levels, they had had a great deal of change, with the introduction of the reforms, and that you were spending a lot of time focusing on the arrangements to make sure they were being conducted properly. Did you identify, in that focusing exercise, the likelihood of the particular problems that arose, or did you identify other particular teething problems, which, in actual fact, because of identifying them, did not subsequently become problems?
  280. (Ms Evans) I think, when you spoke to the QCA in May, it would be fair to say that the problems that we thought were going to create most difficulty in the A level system this year were ones to do with the delivery of the exams, the physical delivery, making it happen, getting papers into schools and colleges.

  281. Is that because of Edexcel problems that we spoke about at length at that time?
  282. (Ms Evans) Certainly, there were specific problems with Edexcel, but we did not just concentrate on Edexcel, we were looking at the issues right across the range of awarding bodies. I think it was referred to earlier, examiner supply, for example, was an issue that we had to pay very great attention to, and, indeed, one of the awarding bodies, in the end, was not able to have all its papers marked to the right timescale; that was a matter of great concern to us, we were managing it very directly, right up until the results were announced, in fact. So I think it is fair to say that our attention was focused on those issues, because we had identified those as being the highest risk issues to the development of the system.

  283. So you had not identified a likely problem around grading, although, given that this was a new process and therefore there were not exemplars, there was not previous information, did not that alert you to a possible problem, along with the rushed timescales we have heard about?
  284. (Ms Evans) As with Ken Boston, I have not been here for the whole of the period when QCA has been working on the new exam system.


  285. You have been here two years?
  286. (Ms Evans) I have been here two years, and what I was going to say is, looking back at the record to 1998, what I see is an enormous amount of exchange, dialogue, correspondence between QCA and the awarding bodies and the teacher associations, and head teacher associations, in examining the very issues that you are describing, what the standard should be, what the A level standard will be in future, was it going to be a different standard, or was it going to be the same standard, and how the AS and the A2 components of those fitted in. So that dialogue had been taking place since 1998, and, in fact, I think, since I have been in the QCA, since the year 2000, that dialogue had seemed almost to have come to an end by the time I joined QCA. There is a record in the early part of the teaching of the first A levels, in the autumn of 2000, and that, to me, would appear to be the tail-end of that debate and that discussion; it was not an issue that was continuing to be discussed between ourselves and the awarding bodies during the last two years, the main issues for us were delivery.

  287. So you are at the helm of the QCA in this period, and you seem to be concentrating on whether the exams, physically, a sort of Consignia role, making sure everything got delivered, and so on, whereas, this disaster, you are going right to the iceberg; and you seem to be saying that, here you are, in a sense, your fingerprints are all over this, Ms Evans, you have been there the longest, and you are saying that you did not see any sign of the problems of the awarding bodies, the examining bodies, you did not see any of this coming?
  288. (Ms Evans) Perhaps I could correct what I said earlier, because that was not what I meant. Certainly, the organisation has a whole division of staff who attend awarding body meetings to observe what happens when decisions are taken about awarding, and their role there is to determine that our Code of Practice is being followed correctly; that was going on throughout the whole of the period. I am sorry if I gave you the impression that that was not happening.

  289. But what we are trying to push you on is, you heard the evidence earlier about the deep unhappiness, what Dr McLone called this "flawed process", yet you, as QCA, you were the Acting Chief Executive, you did not see that this flawed process (a) was flawed, or (b), if it was flawed, was going to cause a great deal of unhappiness?
  290. (Ms Evans) Dr McLone was in a lot of correspondence with the QCA since 1998 on what he believed were aspects of the process that he wanted to clarify; but I think it is fair to say that, certainly during 2001 and 2002, there was very little exchange that went on, the awarding bodies were not saying to QCA, during 2002, "We're very unhappy and very uncomfortable about how to do awarding this year," that was not being said at all.

    (Mr Weller) Chairman, may I comment. I have to confess to having been there longer than Beverley. Over the period from May '98, when the decision was made to proceed with the new curriculum, until about May of this year, we had some 35 meetings of a technical kind with the awarding bodies, and they went right up to April or May this year, and they were concerned with all the technicalities of the examinations. I think it would be true to say that the issues with grading and awarding were resolved, we believed, with the awarding bodies ahead of time. I think we were all conscious that we would have liked more exemplar material, that point has been made a number of times, but I do not think we were leaving the issue as though it had been finally resolved early at all, we went on meeting, indeed, up until April this year, on those issues.

  291. But what about the central problem, that Dr McLone pointed out, of treating this exam as a 50/50, first year 50 per cent, second year 50 per cent, which seemed, from his evidence, at the real heart of the problem; did you not worry about this, did it not concern you?
  292. (Mr Weller) We worried about it a good deal, and we had a very extensive discussion on it at the time that the first advice went to ministers at the time on the reforms; and, indeed, that advice was public, as was the Minister's response, that is all public. We talked long and hard, in a whole host of consultation conferences and through written consultation, with all the users, all the stakeholders, including the schools and colleges, about the options of a 40/60 or a 50/50 balance. It has to be said that, while two-thirds of those responding were supporting the introduction of an AS examination, it was a very popular reform, still is, I believe, a popular reform, the one thing that they really worried about was the 40/60 split proposed by Ron Dearing, they worried about it in terms of what it would mean in relation to perceived value, by universities, by parents, they worried about the complexity, that 40/60 would have to sit alongside 50/50 in the university tariff system, 50/50 performance tables, and the main message from that consultation was, this is a bridge too far, in terms of complexity. So the decision was that we make it a 50/50 balance, we have an easier part, or a first-year part, and we have a second-year part, and the combination of that first-year and second-year part, more and less demanding, makes the A level standard. Once the decision was made, it was important for us to get it right and work at it with the awarding bodies, and that was what we did, over those 35 meetings.

    Jeff Ennis

  293. In earlier examination, we heard from the three examination boards that, in their opinion, the remit of the QCA was now too broad and woolly and needed to be redefined, and that you needed to concentrate on your core business, as it were. What comment would you make on that particular line, Mr Boston?
  294. (Dr Boston) I think the comment has a good deal of substance to it, and, in fact, it relates to the quinquennial review which took place earlier this year and made some similar points. We are looking at that very closely, and are about to respond to Government on how to implement the quinquennial review, or to deal with it in another way. One of the key issues was that the regulator should not also be the provider of examinations, and with our statutory tests we are the provider of the examinations; the suggestion, or the recommendation, was made that we look closely at separating that and at moving it away. Now we either grow some capacity outside the organisation to take on the statutory tests, we hand it to an awarding body, and I do not think there would be necessarily immense public confidence in that, at the moment, or we contract it out in some other way, by tender, or we live with the ambiguity and continue to run the Key Stage tests ourselves, until such time as they can be moved elsewhere. The public has confidence in those tests, and they are running smoothly; it is not a problem that needs to be fixed, it is a philosophical point that a regulator should not also be a provider, and I agree with that, but, nevertheless, there must be an element of pragmatism in the implementation. I have no doubt that the QCA will change and develop over a period of time, particularly if it has a different reporting relationship, in due course. None of us is in the business of defending the status quo, in fact, in my view, the status quo is the only option that is not on the table; another way has to be found.

  295. Generally speaking, when problems occur within organisations or across organisations, generally, it can be perceived to be a poor communication strategy, or the fact that you are not getting your message across to whoever it may be. Do you agree with me that poor communication has been a major factor in the events that have unfolded this summer?
  296. (Dr Boston) I think all of us failed, in the key strategic thing we all have to achieve, and, by that, I mean Government, the QCA, the awarding bodies, teacher associations, and everything else, we failed to communicate the nature of the new standard, and we are still grappling with it tonight, and that is the key issue that we have to resolve. The Programme Board was set up to do that, it is working very hard, we will meet the Tomlinson deadlines, we will get clarification, but the promulgation of that and growing it into a community understanding of what it is about is going to be a very complex process, and I doubt that we are going to have that delivered into the national psyche by the time of the January examinations.

  297. Just one supplementary question, on the answer you have just given me, Mr Boston. Is there any agency or organisation, in the ones in which are involved, which should be taking the lead, in terms of improving the communication strategy across the agencies involved, or should it be just a sort of partnership approach?
  298. (Dr Boston) I think the QCA should be taking the lead in it, and that is what we intend to do; but, of course, the training and development, the in-service education training programmes for teachers, are delivered by the awarding bodies. But we need to be far more rigorous, I believe, in our scrutiny of that, just as we have been rigorous in our scrutiny of the actual awarding process. We need to have a far greater eye to the support that goes into working with teachers and getting a general community understanding; and there are some things we must do ourselves, as the regulatory body.

    Paul Holmes

  299. A few months ago, this Committee did an inquiry into ILAs, another inquiry came out on Friday, into ILAs, and one of the findings of that was that it was a good scheme that was ruined by being rushed in too quickly, to meet political targets, really. And you said, a little while ago, that the introduction of Curriculum 2000 was rushed in, far, far too quickly, when there should have been five years of development and piloting; and you also said that there should be a clearer, blue-water barrier between the exam bodies and the Government and the QCA, it should be there. So do you see the QCA, from now on, standing up and taking a very public role, and saying, perhaps, to the new, ambitious Secretary of State for Education, "No, you shouldn't be doing that"?
  300. (Dr Boston) I would not put it that way. I would put it as the QCA properly carrying out the role for which it was established, and that role, from time to time, will involve saying to Government, "This is not a possibility; if you proceed to go down that track, or on that time-line, you're going to run into trouble." It seems to me that that is what the nature of a regulator is, and that, unless a regulator is prepared to do that, credibility and authority will never be established; you cannot legislate for credibility and authority, you have actually to demonstrate it by performance, by the quality of what you do, and that is where the QCA must position itself, in the public arena.

  301. That is good to hear. Would you say, as a newcomer, again, looking back at what has happened here, that it was really an abject failure by the QCA not to have put in place very clear guidelines on what the A2 standards were and should be?
  302. (Dr Boston) No, I cannot say that. The QCA did everything it could to make a clear statement about how the A2 standard and the AS standard related, and, indeed, I understand it has been on our website since 2000; we did everything we could to write rich, meaningful statements about grades in all the subjects, which examiners and teachers could understand. What we did not do was deliver on exemplars. You do not really know what a grade B is at A2 level until you have looked at a thousand scripts, from young people across the country, how did they handle that question on Nazi Germany, what was the depth of the analysis, the depth of the synthesis of argument, how did they deal with the synoptic issues, you do not know that until you have got that material in front of you. Now that was not a fault of the QCA or of the awarding bodies, that was a fault of the time-line; we launched into the first A2 examinations in summer without that pilot material behind us, and we should have had two years of it. We were okay with AS, because that has been sat three times, but we were not able to deliver on the A2; even though the A2 had been sat during the trial process, the grading had not been sorted out, and so the exemplars simply could not be used.

  303. But, given that the QCA had the situation as it was, not as you might have wanted it to be, the thing was introduced, the exams were being set. Now the Tomlinson inquiry says, in paragraph 16, that, quite clearly, there was no common understanding on how much greater the demands of A2 should be, compared with AS. So, given that we were in the middle of this process, should not the QCA have done more to try to establish that common understanding?
  304. (Dr Boston) The QCA should have done more, but it is not in the sense of being able to write down, in specific detail, other than the generic statement, what the standard is in History at A, at B, at C, using real substance, that has to come from the exemplar material. But the fact that we even have a discussion now about AS and A2, and whether there are standards or level of demand and how they relate to each other, indicates that the QCA, along with other agencies, has not delivered adequately in implanting that understanding in the minds of the profession and the community.


  305. We have a system in this country, as there is also in Australia, of accountability; you know, when things go wrong, people we represent tend to want people to say, "Well, who was responsible for these problems?" and to home in on who it was and to exact some sort of retribution. Who were the guilty people, who should now carry the can, using terribly blunt language, who should carry the can for what happened over these last two months?
  306. (Dr Boston) Mike Tomlinson has produced a report which has looked at those issues, and I really do not want to add anything to what he has said, because he has looked at a wider range of evidence than I have, I have simply looked at what has happened in the QCA.

  307. He has not looked at the political accountability of who now carries the can, who pays the price. Now the battlefield is littered with corpses, we have got a new Secretary of State, we have a new Chairman of the QCA; have the right people disappeared from the scene?
  308. (Dr Boston) I would not want to comment one way or the other in relation to individuals.

  309. So you will not be making any personnel changes in the QCA?
  310. (Dr Boston) I did not say that. I have not discussed the structure of the QCA. The QCA is an organisation which, as I have said publicly, now has to reinvent itself, it has to establish very clearly what its priorities are, it has to be very clear about what its strategy is, it needs to align its structure to deliver that strategy. There will be changes in the QCA, but I do not believe that the problem which has emerged here is a problem which can be driven home to particular individuals, either in the QCA or elsewhere, and say they were guilty. It is a compound of a series of things that should not have happened, rushed time-lines and other things, which, with the benefit of hindsight, coming in as a new person, I can see, and others are seeing at the same time. I am not so sure, if I had been here, whether I would have seen all of those problems emerging, but, the fact is, they happened.

    Mr Turner

  311. You have made it pretty clear, tell me if I am wrong, that you do not think that an AS level is worth half an A level. Is it, therefore, not entirely wrong for UCAS to treat an AS level as worth half an A level?
  312. (Dr Boston) What I am saying is that that is the wrong question to ask, if we are going to make progress with this. The issue is, we are dealing with A levels as they have been for 50 years; the change is, we are arriving at it now from two papers, one of them is a hard paper, one of them is an easy paper, relatively. But that is language which every student who takes the course understands, there are two papers, and you add together the scores on the two of them, one you take a year before the other, and you get a result.

  313. But the universities, or, at least, the university admissions system, is treating an AS level as if it is not an easy paper, as if it is half an A level; surely, that is wrong?
  314. (Dr Boston) Mr Chairman, I am not wishing to comment on that issue, because I have been rather preoccupied with things other than UCAS and university entrance, and I have not thought that fully through; but, clearly, it is part of the work that we have to do in implementing Tomlinson, because, clearly, this is a major purpose to which the result is put.

  315. Can I ask you another question, which relates to what Sir William Stubbs wrote in the Sunday Times. Do you recall being told what you were told by a senior official in the Education Department, about them approaching chief executives of boards with a view to what might happen in certain circumstances; would you like to recount that, if you do?
  316. (Dr Boston) Yes, I do, I do recall that. Mr Chairman, I was made aware, by a senior official of the Department, that discussions were occurring between members of the Department and the awarding bodies on what would happen if Tomlinson sought to recommend regrading, and that was accepted by the Secretary of State. I was concerned about that, as the regulator, and concerned because, earlier that same day, we had been examined by Mr Tomlinson, and we had made it clear, in response to questioning from him, that we ourselves were having no contact with the awarding bodies while his work was in progress. Now I contacted a more senior official at the Department to express concern at that, as the regulator. I have no objection at all to the Secretary of State sorting out the various scenarios, as it were, that might emerge from an inquiry and seeking advice on what to do with each one of them, but, the reality is, the conversation should not have been with the awarding bodies, by the Department, but with the regulator. Because the regulator is thoroughly across, because of its role as a scrutineer and day-to-day regulator, issues of the capacity of the awarding bodies to deliver, and would be able to advise Government on whether or not regrading was possible; indeed, we were, in fact, doing that, as a result of another request. My concern was not that the work was being done, but that it was being done directly with the awarding bodies rather than the regulator.

    Chairman: We are going to suspend the session for 15 minutes.

    The Committee suspended from 7.02 pm to 7.15 pm for a division in the House


  317. Thank you very much for being patient. I am sorry to delay everyone's dinner. There is now legislative power that you have, Dr Boston, in terms of actually intervening, as I understand the new legislation, in an examining board you are unhappy with, discontent with. Can you see the QCA using that power?
  318. (Dr Boston) Yes. There are three amendments to the Act; the most substantial one is a new section, 26(a), which does give us the power to intervene, to direct, and we do see ourselves using that power, not necessarily always only to correct what might be some mistake or aberration but to manage the system a little better. I referred earlier to, one of the problems in dealing with the examinations is the large number of late entries that can occur, in fact, there are sometimes young people who turn up on the day of the examination, and papers are photocopied and given to them. Edexcel had, over all qualifications, over half a million late entries at the last examination; now, if we got something like that scale with the A levels, even in proportion to it, it just simply becomes unmanageable, the number of markers that you require goes up enormously. Now, if we are to model the system and manage it correctly, one of the things we are looking at is using that new power to determine that there be no late entrance, or no late entrance after a particular date, except for young people who might be in particular categories, awaiting remarks, or something like that. Now we have not committed to that, but that is an example of the sorts of things the new powers could be used for.

  319. Would not a lot of people be a bit worried, in a sense that some people would have thought that, if you were going to assess most recent problems, it is the QCA that have got the problem rather than the examining boards, and you have now the power to go in and interfere with the running of exam boards; some people might see that as a nightmare scenario?
  320. (Dr Boston) They might. I think the community would see the fundamental test any regulator has to face, or pass, or, indeed, a Government has to pass, is, well, it is the equivalent of making the trains run on time, make sure the examination system works; and we have seen in this last month an examination system that has been under extraordinary pressure. The priority now, I think, is to make sure that never happens again, and we do that both by addressing the Tomlinson recommendations and, on the other hand, addressing the issue of logistics.

  321. Do you anticipate any new legislation that will affect QCA in the new session of Parliament?
  322. (Dr Boston) I have no expectation of that, at the moment. It will depend very much, I think, on what Mr Tomlinson finds as he addresses his second term of reference, and it might be that legislation flows from that, possibly in relation to the role of the QCA itself.

    Chairman: In terms of the Queen's Speech, I think he would have to hurry up with that. Andrew, you were in the process of finishing your questions, I think. I think you were in full flow.

    Mr Turner

  323. Yes, I was, and I apologise for returning late. I had only one other question at that time, and that was, did you perceive the response of the Permanent Secretary to your inquiries to be appropriate?
  324. (Dr Boston) I think the answer is, no. Mind you, I came to that conclusion on the basis of experience in another country, where there are ways in which these matters are handled, and I was coming from that background; but, because of my concern about the matter, I did telephone Mike Tomlinson and report it, because of the discussion, or the examination that he had given us earlier in the day, when the issue of contact was raised. I had no thought that it compromised the integrity of this inquiry, and he quite properly came out the next day and made a statement on precisely that point, and that was fine. Nor, as I said earlier, did I have any reason to think that the Secretary of State acted inappropriately; of course, she was sensible, to look at all possible things that could come out of the inquiry and know how she was going to deal with them. It is just the wrong bodies were consulted.


  325. So there was a clear division on that subject between you and your Chairman?
  326. (Dr Boston) My Chairman. I do not understand the assertion, Mr Chairman.

  327. I am just seeking what your views were on the actions of the Secretary of State, and the Chairman's?
  328. (Dr Boston) The then Chairman, Sir William Stubbs?

  329. The then Chairman; you must have discussed it, surely?
  330. (Dr Boston) I did discuss it with Sir William Stubbs, indeed, before calling Mike Tomlinson.

  331. So his protest about the Secretary of State was something that you were in accord with him on?
  332. (Dr Boston) No, I was not in accord with any protest against the Secretary of State. I was concerned that the legitimate request of the Secretary of State had been dealt with by approaching the awarding bodies to ask them whether they could handle a regrading, which was still being considered by an independent inquiry in progress. My concern was that the QCA, as the regulator, had not been the body that was consulted. I had no criticism at all of the Secretary of State.

  333. How different was that from Sir William's point of view there?
  334. (Dr Boston) I cannot speak for Sir William, Mr Chairman.

  335. What about Beverley Evans, you must have been heavily involved in this, and you are seconded from the Department, you must have been involved; did Sir William consult you before he made his allegations about the inappropriate behaviour of the Secretary of State?
  336. (Ms Evans) I was present when the discussions were taking place between Ken Boston and Sir William Stubbs.

  337. And there was a disagreement between those two?
  338. (Ms Evans) No. I think, my recollection of Sir William Stubbs's view, as we were discussing it, was that it was inappropriate of the Secretary of State to have discussed, or to have asked two officials to discuss, those matters with the awarding bodies, rather than discuss them with ourselves.

  339. And did Sir William say he was going to make those views of his public?
  340. (Ms Evans) He then proceeded to speak to a number of journalists about a number of matters, including the evidence that we gave earlier that day to the Tomlinson inquiry, and it was in the course of those discussions with journalists that that came out.

  341. So, as you had those discussions, as a very senior secondee from the Department, did you give him any warning on what would be the repercussions, if he made that sort of public statement?
  342. (Ms Evans) There was not a discussion of that sort that took place, I am afraid.

  343. But you knew that he was going to make that?
  344. (Ms Evans) I am a member of the Department, as you have referred to before in this Committee, but for the period that I have been working in QCA then my role is as a member of QCA's staff, and that is the way in which I have acted.

  345. No. What we are seeking to discover is, many of us who know Sir William were surprised at the way in which he spoke, because it did seem inevitable that if he spoke in that way there could be only one resolution to that action. Did no-one in the QCA at that time counsel him that that would be one of the possible repercussions?
  346. (Ms Evans) The discussion that took place between us was on the appropriateness of DfES officials having those discussions with awarding bodies and not having those discussions with ourselves.

  347. And what was your view?
  348. (Ms Evans) I think it was inappropriate that the discussions were had with awarding bodies and not with ourselves.

    Chairman: Right; so you are in accord with your new Chief Executive on that.

    Mr Chaytor

  349. If I can refer to the speech you gave at the QCA Annual Conference earlier this year, where you talk about the annual problems of the examination system, which are quite separate from the specific issue of grading this year, what are the annual problems, over and above those we have discussed this afternoon?
  350. (Dr Boston) It is the shortage of examiners, and I think that is going to be exacerbated this year by many people not wishing to examine again, or perhaps examine for one board again; the sheer volume of the assessment that occurs across the country. I do believe examinations here are probably the most excessive in the world for young people, and that we could get equally valid measurements of student performance and progress with less examination. The reliance so strongly on external examinations, rather than some component of it, at least, being internally examined. The notion of having internal assessments externally moderated, which the Secondary Heads Association is advancing in the form of chartered examiners, is, in fact, the norm for many examinations in many western countries and produces valid results. John Kerr referred earlier to issues of technology. The technology that we use is very simple, and it was the subject of some comment in the report, maintaining A level standards, that Peter Baker chaired earlier this year. Our scripts are all marked by single markers, no script is marked by two markers; the scripts from centres move by post to a marker's home, usually, we do not use, although we have trialled, as a general rule, marking centres, where markers are brought in to mark under supervision, and one marks questions 5a and 5b, and another marks questions 6a and 6b, and you get consistency that way. Very little application of technology. We are running here a 21st century education system on a huge cottage industry, in the marking process, and it is just going to fail, unless we move to change the way that operates. Now that cannot be done for the summer examinations next year, we do not have that capacity to move that quickly; but that is the longer-term issue, we have got to get the examination system logistically and technically on a much firmer basis.

  351. Now some of the points you have listed are issues of management, or issues of technique, but other points are matters of huge importance in terms of policy, and, if there were a move to a greater degree of internal assessment and a reduction of the overall volume of assessment, that would be a reversal of the policy in this country, under successive governments, over the last 20 years. So, if that is what you identify as the annual problems of the examination system, how are you going to influence the Government to bring about that kind of change of policy?
  352. (Dr Boston) I cannot say that I have a developed strategy for doing that, at this stage; but I think it needs to be put on the agenda for public discussion, backed up with a lot of evidence and with alternatives, and become a subject with which the community as a whole occupies. I have followed very closely the developments in education over the years, and I understand the pressures that have led to this highly intensive testing programme. As a person who is experienced in this field, although in other contexts, but has been in it all my life, I think there are major problems here, and I am actually more concerned about those problems than the A level issues. From the point of view of the A level and the marking and the limited amount of regrading that has gone on, this is not a system in disarray from that point of view, it is a system that has been through a difficult passage but the causes of that can be addressed, and can be addressed probably in the relatively short term. But addressing the bigger issues of the potential for the system actually to break and not be capable of being delivered, they must be urgently on the public agenda; and I think one of my roles, and one of our roles, in the QCA, as an independent regulator, is to lead that public debate.

  353. And you think that criticism applies to A level, to GCSE and to the Key Stage tests as well, you are including all phases of the education system?

(Dr Boston) I think we should be looking at all phases. The 7, 11, 14, certainly there are intervals there, and other systems have similar intervals, but we also have an extraordinary number of optional tests, some of which, many of which, QCA, in fact, develops, that are administered to children. There has to be a balance between assessment for formative purposes, for aiding learning, assessment for summative purposes, so that Government, that has made the investment in education, knows whether they are getting the outcomes that they are investing in; but it does not have to be this extraordinarily intense programme, we have got to leave some time for teaching, not testing.

Chairman: Dr Boston, I think that that is a good note on which to end this session, and, certainly, if you are coming out of your corner fighting for those principles, you will get a lot of support from this Committee. And I hope you have enjoyed your first session in front of this Committee, and we look forward to a long and creative partnership. Thank you.