WEDNESDAY 4 DECEMBER 2002

__________

Members present:

Mr Barry Sheerman, in the Chair
Mr David Chaytor
Valerie Davey
Paul Holmes
Ms Meg Munn
Mr Kerry Pollard
Jonathan Shaw
Mr Mark Simmonds
Mr Andrew Turner

__________

Memorandum submitted by Mr Roger Porkess

Examination of Witness

MR ROGER PORKESS, a Principal Examiner to OCR Mathematics and Director, Mathematics in Education and Industry, examined.

Chairman

Mr Porkess, welcome to our meeting. Thank you very much for attending. I am sorry this is a brief session but we will rattle through and try to get as much out of it as we can. Obviously we wanted to talk to you because of our very short inquiry into the A- level problems that were encountered this summer. It did seem that we were just getting onto an even keel, and then suddenly you burst into the media expressing your unhappiness. I know in the document you sent to the Committee you talked about the argument "let sleeping dogs lie". Is there not a problem in the sense that we were getting to the stage where parents and students were feeling that what had happened had somehow been resolved by the Tomlinson inquiry and suddenly your head is above the parapet saying, "No, no, great injustice has been done to a great number of candidates". Could you tell us why you said that?

(Mr Porkess) My motivation is that I want an A- level that has integrity and I want something that I can believe in. This is an area that I have been working in for a long time, and I want an A- level that works to proper principles and at the moment we seem not to be getting that. There are two principles involved: one is the technical matter which is very important, not that you should not use the thresholds on modules to influence the A-level outcome; the second is that candidates' interest should be paramount. Those two principles at the moment have not been upheld and I am very concerned that we are setting a precedent which is going to mean that we cannot rely on any grade in the future. I want out of this process that we have an exam system that has an integrity that we can all believe in that we can build on for the future, and I do not think at the moment were are quite getting it.

But we have just now had the second part of the Tomlinson inquiry reported for this week: it was received positively by the Secretary of State who is going to action most of the recommendations across the piece: are you still unhappy after yesterday's statement and the publication of Tomlinson mark 2?

(Mr Porkess) Yes, I am unhappy. Mark 2 does not say very much about the problems of mark 1 - they are only really en passant - and in particular in the section that deals with accountable officers it does not make the point that accountable officers should not use module thresholds to influence qualification outcomes. That was something that I had expected it to say and that I had expected to see written into the new QCA code of practice, and it is not.

But is there not a view in Tomlinson that the QCA and the examining boards will now sort this out?

(Mr Porkess) I am sorry - we have a precedent at the moment that says that this does not matter, and if it is not written down as a principle then I fear that the precedent will stand and we will have lost a major principle.

So how many candidates are you saying this summer got the wrong grade? You are talking mainly about mathematics, are you not?

(Mr Porkess) No, I am talking across the board. In the scheme of things mathematics was probably relatively lightly affected, but across the board it is tens of thousands but I do not know how many tens of thousands. Without the full information from the exam boards one cannot tell. Whether that full information can ever be fully available is also doubtful.

Mr Turner

Mr Porkess, have you seen the memorandum produced by Mr Seager, the chair of the mathematics examiners with your board?

(Mr Porkess) I received that when I arrived at the hotel late last night. I have prepared a response to it which I tried to get typed up before this meeting but I did not succeed.

Chairman

We have only just received it and it will be added to the evidence to the Committee.

(Mr Porkess) Could I make some major points?

Mr Turner

Yes.

(Mr Porkess) Brian Seager accuses me of using extrapolation and that is untrue. OCR has about 50 syllabuses; 22 of those were identified by Tomlinson Part 1 as having threshold movements of 6 marks or more so that the case that I took of a 5-mark movement is pretty much in the middle and it is not extrapolated out into some extreme region, which is what Brian Seager is implying so on that point he is wrong. The other point that he makes is that the example that I worked was using adjustments to an 'A' grade threshold only. It seems from the evidence that he has produced from OCR that for most of the 97 modules that were identified in Tomlinson 1 with movements of 6 marks or more those movements were the 'E' grade. Now the effect will be the same in terms of numbers of people losing a grade but it will be that people are losing an A- level grade 'E' going from 'D' into 'E' rather than at the other end. It will be a mirror image but it does not affect the total numbers involved. The third point that he makes is he questions my statement that the effect on coursework is greater and that is not justified in the Tomlinson final report. In section 73 there is a reference to a syllabus where 9 marks separate 'A' and 'U' and that means that the multiplying-up factor going from a raw mark to a uniform mark is about 4.5 or 5, which is the sort of thing that I set. I think that syllabus was probably psychology. Salter's chemistry was another one that had very similar tight impact thresholds.

Mr Porkess, I have really only got two lines of questions, one arises out of what you just said. Is not an examination which produces such tightly packed thresholds rather an unsatisfactory examination?

(Mr Porkess) Entirely so, and I do not know how such a thing has got approved by QCA.

Chairman

You are saying this has been going on for years?

(Mr Porkess) No, I am not.

You are saying it is only this year?

(Mr Porkess) This is a new specification. It is a new syllabus for Curriculum 2000.

I thought you said in your answer to Andrew Turner that this methodology had been developed over a number of years? Am I wrong in understanding that?

(Mr Porkess) That was not what I said in respect of my answer there. The point I had made in my general comments were that the modular syllabuses have been around for about ten years and the methodology has existed for ten years and worked perfectly well. What has happened this year with Curriculum 2000 is totally new. What has gone wrong has gone wrong because principles that were adhered to have no longer been.

Chairman: And in a moment you are going to tell us how to put it right.

Mr Turner

The numbers are interesting but what is surely more important is the principle and what you are asserting in your memorandum is that certain modules were marked down "in order to" reduce the number of candidates getting particular A-level grades. You assert that on a number of occasions in different places. What is the evidence for "in order to"?

(Mr Porkess) I gave you the example of our own MEI syllabus. That syllabus has been running a long time and most of the modules in it were virtually unchanged going into Curriculum 2000 so we have a very long history of awarding that. If, for instance, I take the statistics 2 module which has been unchanged for twelve years, so there is a long history of awarding that, we awarded the grade 'A' on that at a mark of 46 out of 60 as an awarding committee and that was in line with everything that had happened before. That mark went to the accountable officer for the exam board and was accepted as being a right mark for that module. Later, it was then changed and increased by 3 marks to 49, and in the scheme of things 3 marks is a very big adjustment there. Now, whatever the motivation was it was not that the module was incorrectly awarded at 46 because that was accepted.

So you are saying that you do not know what the motivation was but you know that the change took place and you have imputed this motivation?

(Mr Porkess) Yes and no. I am imputing it but I also know that it was part of the air that everyone in the exam world was breathing at the time - that there was pressure to keep the results down.

Your chief executive has told this Committee effectively that an A2 has to be harder to balance the AS being easier - in other words, the grade threshold has to be higher. Do you see that as an improper objective?

(Mr Porkess) It is inconsistent with what the constructions were at the time that Curriculum 2000 syllabuses were submitted. Remember that this is a point I put in the papers in advance because I did not feel it was a point that has been properly brought out - that at the time of submission of these syllabuses a great deal of effort went into producing specimen papers and mark schemes, and QCA spent ages and ages pouring over these trying to ensure the standard. At that point there was no suggestion that the standard that was required of new A2 modules should be any different from that required of the legacy A2 modules. Indeed, the design thresholds in our case are written into our syllabus and approved by QCA, the same as they always were.

Paul Holmes

You have submitted evidence in your experience as an examiner that normally grade boundaries might move by one or two marks in any given year but the Tomlinson inquiry was only allowed to look at grade boundaries of six marks or more, and this led to huge distortions because they were not looking at the majority of the unusual changes in grade boundaries. Can you explain that a bit more?

(Mr Porkess) If I could explain what happens at, first of all, the awarding committee, you are giving the grade 'A' and the grade 'E' on each module threshold and it is a very painstaking business looking at a lot of evidence. Typically you are taking about half an hour on each threshold that you are looking at and you are taking into account candidates' work, the design thresholds, centres' comments, centres' predictions, the principal examiner's suggestions, any evidence about the population and historical data, and an experienced committee will come out pretty much with the right mark. Now, it may be that, say you have a situation where you are looking at an 'E' grade threshold, and one mark would give you 80 per cent passing but if you go down a mark - which is your next option because you cannot have half marks - you might have 83 per cent passing, so you have a lot of candidates around there. Last year there were, say, 81 per cent who passed. Now you are going to be out of line a bit on last year whatever judgment you make and you make a decision. Now, it may well be that the accountable officer would say, "Sorry, I think you chose the wrong way there. You chose for the more generous; I am going to choose for the harder", or the other way, and you had Kathleen Tattersall's evidence where I think pretty well half her decisions for one mark adjustments were up and half were down. Now, that is normal for an accountable officer to look at. Occasionally two marks will happen but that is really the limit of changes that an accountable officer would make, and if an accountable officer is making changes of 5 and 6 marks regularly then something is desperately wrong inside the exam board. The awarding committee should not be that inaccurate and something is really seriously wrong with the direction and the personnel and whatever if the accountable officer does not have that level of confidence in them.

So if this year there were a lot of unusual moves between 2 and 6 marks rather than just 1 and 2 marks, and the Tomlinson inquiry only looked at one and 6 marks a board, does that mean they were missing the point in what they were looking at?

(Mr Porkess) Yes. If they had set a limit of 2 marks that would have been fine. One mark no one is going to question, as I have explained, but a movement of more than one mark really would be unusual, and more than 2 really worthy of comment.

The Tomlinson inquiry, the final version published yesterday, seems effectively to say that AQA and Edexcel more or less got it right but OCR were responsible for all the wide discrepancies. Would you think that was correct?

(Mr Porkess) I do not have evidence of what went on in AQA and Edexcel and the Tomlinson inquiry did, but I do not have any evidence to the contrary.

Chairman

Why are you such a lone voice, Mr Porkess? You seem to be out there on your own. There are thousands of examiners and experts and statisticians out there who are not making the same voice that you are making about this?

(Mr Porkess) Everyone involved is subject to confidentiality agreements with the boards and because I am employed by an independent organisation I have a bit more freedom to speak out than others.

There are other independent souls out there. This Committee has been inundated with information from people who you might think were bound by confidentiality, but you still are up there on your own. Why are you so deeply unhappy?

(Mr Porkess) I am unhappy because I do not see that, as we are, we are going to have A- levels that have credibility in the future.

What is your passion about? The injustice done this summer to students or about what might continue to be a problem in the future?

(Mr Porkess) It may sound discreditable to me but probably the latter is the greater - that I am more concerned about getting system that is going to work properly in the future. However, having said that, to get that I think we have to sort out what happened to candidates this summer as well. You cannot really separate the two but in the long term what happens in the future is really crucial to our country. We cannot have an exam system that does not have integrity.

Mr Chaytor

Mr Porkess, one of the main principles you identified earlier was that the accountable officers should not be manipulating grade thresholds to influence the outcomes, but is that not what happens every year?

(Mr Porkess) No. It has not happened in modular syllabuses.

But in terms of the history of A-level examinations, you have described to us a process whereby every year the accountable officer has the power, if he or she chooses to use it, to change the recommended grade thresholds made by the awarding committees?

(Mr Porkess) Yes. It is a question of where quality lies.

But is it not a matter of degree, not a matter of fundamental principle?

(Mr Porkess) No, it is a matter of fundamental principle. In a modular syllabus, you set your standards with the modules and having given the students credit for the modules, the final outcome is then outside your control. You set your standard on the modules so that is where the control is exerted and that is how every modular system works. It is how Open University works, for instance, with its degrees and in that it is different from a linear system.

In the normal year when you say that the grade thresholds may be adjusted by one or two points maximum per module, how many outcomes would that influence? For this last year you have given an estimate of somewhere between 20,000 and 35,000 but how many would it be in a normal year if the adjustment was only by 1 or 2 marks?

(Mr Porkess) On an exam module, if you had one module moved by one mark, that would affect about three quarters of a per cent of candidates when it came through to the A- level, roughly.

And in terms of raw numbers, how many would that be?

(Mr Porkess) Well, you are talking then about one syllabus so if you have a syllabus with 10,000 candidates you are talking about 75 people, and that would be a big syllabus.

Separate from that, one of the issues you raise in your submission is the question of comparability of standard between different subjects which has not been an issue that has featured in the public debate over the last few months nor I think in the first part of the Tomlinson report, and yet you made quite an important point of this. Is it possible to establish a system where there is precise comparability between subjects, or do we not simply have to accept that high achievement in certain subjects, be it maths or physics, is a rarer skill than in other subjects?

(Mr Porkess) I think there ought to be a methodology to get a lot closer than we are at the moment. In maths at AS last year, 2001, we had 30 per cent failures where most art subjects were single figures of failures, yet in maths we would normally think that we are probably getting the brighter children, and that is an extreme injustice and really QCA should be setting up procedures that are advising the boards, "Look, your subjects are not working the same way".

Is the variation of pass rate between the different subjects in that order every year, or was that peculiar to this particular year?

(Mr Porkess) In maths in 2002 the pass rate went up a bit, but maths still came 31st out of 31 and the order of subjects was virtually unchanged.

Valerie Davey

Probability between subjects has not been touched but certainly comparability between examining boards has. Would it be fairer to both the individual students and, indeed, the integrity of A- levels if there were only one examining board?

(Mr Porkess) I think if you only had one you would end up with fossilised exams - you would lose the creativity that is there. Remember that a lot of your subjects are evolving - maths, science, technology and so on are evolving subjects - and you need the variety so you can represent that evolution and not just end up with a static syllabus. I am not quite answering your question but I think there is a bigger principle there of keeping our school syllabuses alive.

In which case, given that there was a new syllabus this year and it would appear that one of the three was out of line, would you not therefore have expected that the accountable officer might have made a greater variety of change within that year's marking?

(Mr Porkess) I would not have expected that that would happen with the accountable officer, no. I do not see that that would be for him. I can see that the awarding committees would have to think carefully but I am sure that they did so.

But if the awarding committees on the basis of the whole ethos of that particular examining board, OCR, was out of line then potentially the accounting officer did have to make that change at the end? I am talking theory: I am not competent enough to be talking as an expert but in theory that potentially could have happened this year?

(Mr Porkess) I do not think there is ever any evidence to suggest that the awarding committees of OCR were out of line with anyone else. It was what happened subsequently that was a quite different procedure that happened with OCR than happened with the other two boards.

But potentially would that not have created a greater fairness at the end of the day?

(Mr Porkess) What I would say is that we have QCA observers at awards, and I would very much like to see that QCA observers are more helpful in making sure we are awarding to the same standards. They come; they check that you have followed procedures; they do not give you any indication, "Look, I think Edexcel would have set that threshold a mark higher", and actually that information would be very helpful to an awarding committee. It is actually QCA's job and it is something that they could do a lot better.

You are saying QCA could have improved its performance and would have had a better effect for both students and the A-levels this year?

(Mr Porkess) Yes.

Chairman: Mr Porkess, thank you very much for your attendance. We have learnt a lot in this brief session, and we will get your written comments typed up and taken in evidence. Thank you.

MR MIKE TOMLINSON, Chairman, Inquiry into A-level standards, examined.

Chairman

Mr Tomlinson, welcome. We thought when we said goodbye to you as the chief executive of OFSTED that we would not see you so regularly but we are obviously going to see more of you than ever before! You are very welcome to this Committee but are you not becoming a "man for all seasons", to an extent? I was in the radio/television studio this morning and they complained that the Tomlinson report had not given them enough blood on the floor, and I am looking at this painting behind you and there seems to be blood on the wall in this particular room! Is there not a problem? Knowing you well, you have a personality that is likeable, if I may say so, and you have come up with two reports that do not say anything nasty about anyone. In a sense people are perhaps saying - parents, students - that we went through this terrible trauma during the summer yet when you read Mike Tomlinson's report basically no one is to blame and everyone has got off scot free. Is that a fair comment on you being too nice to everyone?

(Mr Tomlinson) I do not think so, no. I think I try to be fair in the sense of where the evidence allows me to go rather than where my own personal views might want to take me, and those are two different things. This inquiry was seeking to get to the bottom of what happened. I think my report pointed clearly to where there were inadequacies in the system which allowed the position we reached this summer to occur. I do not find that attaching personal blame is a particularly helpful activity. The issue was about the systems and the behaviours that those systems allowed, and nothing that was done this summer was outside of the code of practice and the frameworks which govern that.

But how do we get to such a state where you come up with some remarkable recommendations for change and they, as we have heard yesterday, are going to be mainly accepted by the Secretary of State and implemented, and indeed you are going to take a significant role in the improvement of the system? How did we get to the state of what went wrong with the system, the relationship with QCA and the examining boards?

(Mr Tomlinson) I think probably it is long coming in history but the particular point really is that, first of all, the introduction of AS and A2, as I said in the interim report, was rushed. A2 was not piloted which it should have been, and there was no script of material available to the QCA to inform and use with teachers, lecturers and students. Secondly, I believe that, though the QCA issued some guidance, that guidance in my view was not satisfactory and sufficient to clearly define the standard of AS and A2 and to exemplify it by material not only with reference to the criteria but also to students' work. That was missing as well. Then we get into a third area which has been going on for a long time and that is the annual August frenzy that says, if more students have achieved the standard then the only way that could have been done is by somehow lowering the requirements they had to meet, and I find that a very unsatisfactory situation. So it is a combination of a whole range of factors, some of which have been with us for a while and others of which are particular to Curriculum 2000, and more broadly some of which are particular to the way we tend to see the introduction of innovation and new policy requirements.

You will know that certainly the Chairman of this Committee agrees with your comments on the summer frenzy, and what this Committee is very keen on is maintaining confidence in the system; that students who have worked so hard to pass their exams feel confident that the qualification is a good one and endures for years to come. But you heard Mr Porkess give evidence to this Committee in the last half hour: here you have conducted what we all assume is a thorough inquiry, in two parts, and there is Mr Porkess, a respected and well known statistician, who says, "Come on, you missed the point"?

(Mr Tomlinson) I do not think I did. First of all, the awarding committees do make recommendations about mark grade boundaries for each and every unit. Sometimes at those Committees they are specific to a mark: sometimes they give a range of marks and do not come down on a firm mark, and I am talking about the system as a whole - not the syllabus with which Mr Porkess is associated. In the case of the particular board that administers Mr Porkess' syllabus there is a second stage, and that is something called the GEM (Grade Endorsement Group) and that takes the recommendations of the awarding committee and involves the chairman of examiners of the subjects concerned. It has also available to it not only scripts but other data about performance and it can make recommendations on the movement of grade boundaries. Those committees are often attended by very senior people in the board, sometimes indeed the chief executive but not at that point acting as the accountable officer, and then those recommendations go to the accountable officer and are moved again. I think what is important to accept is that there is nothing sacrosanct about the recommendation of the awarding committee. It is their view and it is a respected view and an important one, but to suggest that no changes can being made to those mark grade boundaries flies in the face of what has happened consistently over time and no doubt will continue to happen in the future. So it was a new situation this year. The other point that has to be stressed is that at the accountable officer level, too, there is that one and only opportunity to look across the suite of syllabuses. In mathematics there are a number of syllabuses all under the heading "Mathematics", and the necessity there of ensuring that an 'A' in that syllabus in terms of the standard of students' work and in a syllabus in that suite is the same is a key role for the accountable officer.

So Mr Porkess is plain wrong? He is wrong to believe there are thousands of students who had an injustice delivered to them this summer, and he is plain wrong that there are serious problems for the future?

(Mr Tomlinson) I do not accept some of the assumptions that he makes in his paper and hence his calculations. I am not pretending, either, that this year or any other year there may not have been students who did not get the grade that they may have thought they got - or, indeed, deserved. That is the nature of examining. It is not a science, it is an art, and you make decisions of grade boundaries. Now, that main sound shocking but it is the reality. We have a criterion reference system but it is not a perfect one. Nothing of a perfect criterion reference system exists, and you have each year, when you have got the data and the results, to have a look to see whether or not applying the criteria and judging where the grade boundaries are is right. In many instances they do need movement and those movements vary between syllabuses and between boards, in part because the arrangements for the process are different in themselves.

Ms Munn

One of the things that we have struggled with to some extent in talking to the different examining boards is understanding the whole process that the examining boards go through in arriving at first the marks and then the grade boundaries, and understanding that there is a lot of confusion around that. When we had the three boards here, there was a discussion which indicated that two boards came to their conclusions in one way. What they said, if I recall, is that they introduced statistical information at a different point. Now, OCR have helpfully given us a memorandum which sets out their process and includes the process you have just described but we have not got one from the other boards so I am still at a bit of a loss as to how that happened. Did you as part of your inquiry form a view about whether either of those ways is better, or is it just that they are different?

(Mr Tomlinson) I came to the view they are different but would not of necessity lead to necessarily different outcomes. I think that the difference in terms of how much statistical data is available at various stages is correct and certainly at the awarding stage in the AQA and Edexcel there appears to be more statistical data available at that point than in OCR, but that additional data becomes available at the GEM stage and even more at the final stage - more in the sense that the accountable officer is looking across all the suite of syllabuses in a particular subject, which is not something easily done at the other two stages.

Would having that statistical information earlier, as the two exams boards do, in your view mean there would be more likely to be a positive or negative influence on people's thinking in terms of where the grade boundaries should fall?

(Mr Tomlinson) If I take the balance of opinion of the chairs of examiners that I have spoken to then I think the provision of as much data information as possible at that awarding stage is regarded as beneficial to their work. That is their view and I respect their view as very experienced chairs of examiners.

So by bringing it in later what is the effect upon the OCR process, in your view?

(Mr Tomlinson) I think it could lead to mark grade boundary changes which are more numerous and potentially more in number than at the other stages, and that was certainly the evidence I was presented with by the three boards when I asked for their most recent 2001 data movement in mark grade boundaries.

But what you said earlier still would hold true - that the outcome is not better or worse; it is just a different process?

(Mr Tomlinson) It is different. There are some studies being done by Professor Carol Fitz-Gibbon in Durham which looks at the performance of different boards with students of equivalent GCSE grades and what they get at A-level, and certainly mathematics shows a close correlation between the results of mathematics across the three boards, which is reassuring.

Is one of the outcomes of your report that the process should be standardised across all the boards so that the pointing of fingers in terms of "more grades are moving here", which is what you seem to be saying is not justified, would not happen, or can we live with two different processes?

(Mr Tomlinson) I think we can live with two different systems as long as at each stage and particularly at the final stage changes to mark grade boundaries are not made without recourse to discussion with the chair of examiners, who of course has been intimately involved in looking at students' work and therefore brings that important dimension to that discussion. That is something, of course, which following the interim report the QCA has moved to make a requirement. There were a number of cases this last summer that I investigated where those changes made had not been discussed and agreed by the chair of examiners. In the case of Mr Porkess' syllabus both the chief examiner and the chair of examiners had agreed the mark grade boundary movements that were recommended. If I go and investigate them, they are going to say, "I agreed with these for the following reasons", and how do I gainway that they were wrong without going back and marking every single paper myself which is clearly impracticable.

Mr Pollard

The essence of all this is about resotring credibility which we all support. Mr Porkess very clearly in his evidence a few minutes ago indicated that others may be keeping quiet. You have spoken to lots of people. Is there any evidence? Are you confident that Mr Porkess is a lone voice in this?

(Mr Tomlinson) I would never put my head on the line and say he is the "lone voice". I think I should remind the Committee that I asked for the boards to relieve the chairs of chief examiners of the confidentiality clause. They were free to speak to me and to offer me written and oral evidence if they so wished, and the confidentiality clause did not count, and a large number of them did submit evidence to me. In some cases it was very supportive of what had happened and their belief that it was correct; others, as you well know, did not agree. So the boards have not sought to gag anyone at all. There are some issues which came out of the inquiry which some examiners, and indeed some schools, continue to feel concern about and I referred to a number of those in my report of yesterday - in particular the fact that in one syllabus the marks separating 'A' and 'U' were very small in range and therefore gave rise to some difficulties. Now, like Mr Porkess, I am surprised by that because, of course, not only had that whole assessment proposal to go through the board itself but it also had to go through QCA, and it raises some questions, shall we say. There are schools still worried about that - and quite rightly so. But the problem is it is not about the grading issue but about the whole marking and assessment arrangement. Those are being tackled by QCA in conjunction with the board and there will be changes not only to the psychology but the English literature syllabus, which suffered in a similar way, for the examinations next year. So there are people concerned about those issues and it did spark off concerns about the grading issue.

Mr Porkess could keep niggling away whilst everybody else is trying to draw a line and move forward and restore the credibility that everybody needs. If you just keep niggling away, does that not undermine what you and others are trying to do? How do we close that gap?

(Mr Tomlinson) Well, it is not going to help, is it, and certainly it does run the risk of undermining efforts to restore credibility which I think, and I have said in my report, is absolutely paramount: that people feel - students, their parents and teachers - that next year's examinations are absolutely secure and they are going to get the grade their work deserves, and I have every confidence that what is happening in the QCA, with the boards and others means we are going to be quite clearly able to say that next year, and I hope I will be able to say that. I have spoken once with Mr Porkess and we have had a number of telephone calls. I might suggest to him that with the OCR and the QCA we sit down and look at this and see if we can find a solution which is acceptable to all parties. I do not mean a fudge - I think there is a need here to understand better and to have all the evidence in front of people such that we can make sensible decisions.

Chairman

So you are suggesting a meeting?

(Mr Tomlinson) I am suggesting perhaps a meeting with the QCA, the OCR board and particularly with the chair of examiners.

And Mr Porkess?

(Mr Tomlinson) And Mr Porkess, to look at this issue as clearly as we are able to.

Chairman: "Blessed are the peacemakers" - and I mean this Committee!

Mr Simmonds

Do you think your report would be more complete and have a greater holistic approach had you considered movements in all grade thresholds and not just in extreme ones?

(Mr Tomlinson) No, because as I have already indicated movements of mark grade boundaries have been something which are part and parcel of the examination systems - and justifiably so. You cannot set a paper year on year which has the same level of demand of difficulty. It is not humanly possible to do that. Therefore you have to look at the marks and compare them with the past and ask yourself whether you are still pitching at the same standard. So there is always going to be mark grade boundary movements. In terms of my inquiry I got the data for all of the mark grade boundary movements for every unit done by all three boards this year, and I asked them what the mark movements had been in the previous, 2001, which was the only basis because that was the first of the AS systems as well - for most. Mr Porkess quite rightly says there have been modular syllabuses for some while. I equally wanted to know whether those changes had been agreed after discussion with the chair of examiners whose responsibility it is, and I did not want an assurance from the board but a written assurance from the chair of examiners that that had been the case, and I got those assurances in the very large majority of cases. Where I did not, it was part of the stage 1 regrading exercise.

So with the exception of Mr Porkess, who we have heard from this morning, there is no evidence you have come across either directly or anecdotally that suggests there was greater movement - not in the extreme grade thresholds but the smaller ones - this year than in previous years?

(Mr Tomlinson) No, no evidence whatsoever. Remember that Mr Porkess is a principal examiner; there is a chief examiner and there is a chair of examiners for the subject, and the chair of examiners is the key person who takes forward the views of the awarding committee and certainly, as you have seen from the response, he was party to the discussions about movements in that particular set of syllabuses - and supported them. Now, if I had gone to him and said do you agree with the boundary changes that were made, then the answer would have been, "Yes, and this is why" - and where would I have turned?

Chairman

If you had gone to him -- what do you mean?

(Mr Tomlinson) If I had gone to Dr Seager and included those units he would have said "I agreed those changes because...", and he would have produced the evidence for me. So I believed I did do all that was possible to identify where movements were made which were outside the norm and had not been agreed with the chair of examiners.

Mr Simmonds

In your report you recommend that the examiners are professinalised. Where do you see those new professionals coming from? Out of the existing teaching profession or as a new graduate intake as professional examiners from the day they leave university? Will this not impact on the teaching profession by extracting numbers from it?

(Mr Tomlinson) No. I am not talking about a separate cadre of people; I am saying I want to professionalise the examining process which at the moment is quite rightly dealt with largely by teachers in our system - both in schools, in colleges and in universities for that matter. What concerns me at the moment is that their work gets little or no credit: their training is, I believe, not as thorough and as consistent as I would hope it ought to be, and what I am looking for is good quality training to be provided for examiners and for examination secretaries in schools and colleges who have a significant role in all of this, and that that training should be properly accredited and that that accreditation should be part of the individual's career and professional development, and I think it would be quite right to think about in the future that a head of department in a secondary or a head of faculty in a college should be someone who has had experience of examining who can advise his or her colleagues and new teachers in what is a very important activity - not just in the public examination sense but in the internal school examination sense as well. I am not looking to pull teachers out of school; I am looking to give teachers a real professional status as an examiner in the system.

Many heads of department in secondary schools say they have quite enough to do as it is without laying more professional work on to them through this examination process you are talking about.

(Mr Tomlinson) My reaction to that is to say at the moment that is where the vast majority of our examiners come from each year. I also have met a number who are no longer examiners, and their reasons for not doing it any longer are very much along the lines that it just does not get the credit it deserves, and if we do value our examination system - and I think we should - then we should ensure that the people doing it receive the credit that it deserves and the training and support that they need to do the job effectively, a job which is changing quite significantly as time passes.

And paid?

(Mr Tomlinson) And paid too, yes.

Chairman

Some of us might say that if you had come from Mars and made these comments we would understand but, come on - you have been a senior education official for many years and chief executive of OFSTED. All the time you were in OFSTED and in other senior education roles, did you never worry or have concern about the professionalism, and the way in which you ran out examining and examination training for examiners?

(Mr Tomlinson) Yes, not in recent times because certainly OFSTED did not have access to the examining process, but when OFSTED was created in 1992 we continued then alongside QCA to have involvement in monitoring the examination system, and certainly I was very much involved at that stage in the work that was done in the reports produced following the introduction of GCSE and indeed also at A-level. We were, and our reports then were, critical of what was happening at that point in time so it is not a new call. I think it has become heightened, however, by the expansion in the number of examinations that are sat and marked and upon which so much depends for both schools and individual students.

So do you think we should have less exams or even more exams that are moderated internally in schools?

(Mr Tomlinson) In my report I have suggested strongly that there needs to be a serious look taken at the burden of examinations from GCSE through to A-level. This is not a personal view but a result of a lot of discussions with students, their parents, teachers and the like over recent weeks, and there is other evidence that has been presented in the press and to me by letter and the like. I think there is an issue to be looked at there and I recommend it is but as part of the 14-19. What I do not want to see is a piecemeal approach to this; I want to see a co-ordinated approach looking across the 14-19 field such that whatever happens is a rational approach to the issue. So yes, I do think it needs to be reduced. Whether or not that reduction is to move the responsibility from external examination to internal assessment on the school I think bears much upon the point made by Mr Simmonds. There is a burden on teachers then that that would bring about, and also there are some serious questions to be resolved about coursework in order to give everyone the assurance that it truly represents the work of the student, and only the student.

When I was a struggling young university lecturer I think external examining was thought of as outdoor relief for struggling young lecturers. You were a bit reticent about pay ----

(Mr Tomlinson) I am not reticent - I think they should be paid appropriately for the task that they do.

Mr Turner

In paragraph 9 of your recommendations you are taking into account the view expressed by Dr Boston that officials of DfES have too many bilateral relationships with examination boards and that those relationships should be conducted through the QCA.

(Mr Tomlinson) I am clear that there were contacts between officials and the DfES and the examining boards, yes. I am equally clear in some instances those contacts were quite proper and legitimate, and I would not wish to see them cease. For example, they might want to seek information about the policy which is after all set by the DfES and any advice ought to come from the Department on that. What I am wanting to see is a very, very clear and transparent set of responsibilities which people understand, who is doing what to whom, when and how, rather than at the moment those boundaries being somewhat vague. I think the argument that some have put forward that we should change the status of QCA it seems to me that there is a tendency to rush to say, to solve a problem you change the status of something, however the important things are the behaviour of people inside those organisations, changing the name will not necessarily itself change behaviour. What I am trying to do here is say that behaviour needs to be changed in such a way that everyone understands what is happening and how it is happening. If a remit letter is sent to the Secretary of State from the QCA to do whatever, if it is the view of the Secretary of State that he or she wishes to involve another party in that then that should be part of the remit. If that party is DfES officials it should say so - that is what I am getting at - then everyone knows and there can be no conspiracy theories.

You said in some instances these contacts were quite proper and legitimate, does that mean in some instances they were not, or in some instances you have no evidence?

(Mr Tomlinson) I have no evidence, no.

You only know of some instances where they were?

(Mr Tomlinson) I did see an awful lot of written exchanges, all of which seem to be quite legitimate, I was told there were a lot of telephone calls, but I cannot say what was said during those. I am not by nature a member of the "Conspiracy Theory Club".

On paragraph 82 you say, "It is self-evident that ministers should be responsible for key decisions which shape the qualification system". Why?

(Mr Tomlinson) As the elective Parliament they are determining the policy.

That is circular.

(Mr Tomlinson) If the Government decide to introduce a new system call Curriculum 2000 that is a policy decision.

You are still within the circle.

(Mr Tomlinson) I do not think I am. I am saying policy, I am not saying they should actually be closely involved in all stages following that.

Presumably at one stage - you may know the date which ministers took responsibility for the qualifications system - there was a date before that when qualifications were not the responsibility of ministers, certainly not A-Level and O-Level qualifications, they were the responsibility of the Examination Board. Why is it self-evident ministers should have this responsibility?

(Mr Tomlinson) I cannot think back. I have to say I have been at the table of all secretaries of state since Keith Joseph and I cannot remember a time when a secretary of state did not feel that they had some responsibility for policy - he introduced GCSE.

He said that the only power he had was to decide whether to sign or not sign an examination certificate. Surely before that ministers did not feel responsible? What I am asking you is, why is it self-evident? The fact that it always happened does not mean it is self-evident. Why is it self-evident?

(Mr Tomlinson) Simply because at the moment in law the Secretary of State has that power.

We made the law we have to try and make it right.

(Mr Tomlinson) We are getting into territory -

Chairman

We do not want an argument. Questions and answers please.

(Mr Tomlinson) I am simply saying at the present time it is clear that the Secretary of State is determining policy on qualifications and I can see why that happens, given the responsibility they currently have. If you are arguing that Parliament wants to change them that is up to Parliament, not me.

Our original meeting with Sir William Stubbs back in May gave us some cause for concern because it did seem there was a relationship between the QCA and Government, it was not quite well defined. I remember at that stage asking Sir William why he did not go in as an independent regulator, high profile, bang on the table and say he was unhappy with the situation and to say to the Secretary of State very clearly "I am unhappy". Taking a high profile approach, being more proactive rather than looking like the relationship was extremely close. After all the secondment a senior official from the Department was his acting chief executive. It all seemed too cosy to for us. The QCA did not seem as independent and as rugged as it should be. Does your Report really grasp that? You do not recommend that they have the same relationship with Parliament as OFSTED has?

(Mr Tomlinson) I do not recommend that. As I have already said I think changing the legal status of the body would not necessarily of itself change the behaviours and relationships. What you want are changes in those relationships. That is what I have said should happen. I also said in their, quite clearly, that the QCA must be a rigorous regulator and must be fully involved throughout the warning process, fully involved throughout, which at the moment is not the case. I also recommend that some activity of the QCA should no longer be part of the remit because they run the risk of contaminating that role as regulator. I am very much in favour of being rigorous.

Mr Tomlinson, if you remember your days in OFSTED, is it not the fact that it did give you that mark of independence that you were responsible to Parliament through this Committee, did it not give you that status as security of having that independent challenge, accountability was not just pleasing your paymaster in the Department?

(Mr Tomlinson) It did give me a certain amount of comfort, yes. It also, of itself was not, in my view, sufficient. What was necessary as well was, and I go back to the behaviour of all concerned, was to recognise that fact and behave accordingly. It was a matter of being diligent at all times. Hence, for example, I did always request and, indeed, I always got, a full remit from the secretary of state for any particular activity they wanted OFSTED to undertake - not how it should be done, I always resisted that - in particular the involvement of other parties, if other parties were to work with OFSTED. It is that clarity we want.

Jonathan Shaw

Mr Tomlinson, you said that AS and A2 systems should be uncoupled. There have been calls from some quarters for ASs to be scrapped completely, what is your view on that?

(Mr Tomlinson) I would not argue for ASs to be scrapped. The views' of students, and taking account of my own experience, is that there are students who want to gain credit for what they have studied in their first year of sixth form because they are not going to continue it in their second year of sixth form. ASs do have a very important, strong role. In the past students have left after one or two year's of study with nothing to show for what they have achieved. I think the ASs have a very important part to play, it has an important part to play in enhancing the breadth by giving due recognition to those subjects. I would not advocate the loss of ASs. It could also form an important part of any future development in our assessment system.

Do you think we need GCSEs and AS Levels?

(Mr Tomlinson) I think that question has to be looked at, apart from the 14 to 19. I think there is a difference between a public examination at 16 and a question of having some assessment of the progress made by the student at that point in time in order to help and inform decisions about where they go from there. Those are two different things that might be achieved by two different means. There will be students who will legitimately want to have a public qualification at the age of 16 simply because they were not going to continue with studying. I go back to my own days at sixth former when whatever you studied at A-Level your O-Level disappeared with it, in other words it no longer counted. For matriculation purposes you had to have the necessary O-Level plus your A-Level. It was an interesting system and that is how it applied to what was a joint matriculation board.

Do you think that the AS Level standing alone is going to provide the necessary incentive for young people to stay on post 16? This Committee, and a lot of people, are really concerned about the number of youngsters staying on beyond 16. Is it going to have the weight and credibility for youngsters to stay on?

(Mr Tomlinson) I do not think youngsters stay on at sixth form because of the possibility of having an AS.

No, they stay on to get an A-Level and go on to university.

(Mr Tomlinson) It is still less than 50 per cent that take the route of getting an A-Level and going on to higher education. Remember A-Level is not just the traditional, it is also the vocational A-Level as well. One of the challenges that has had to be met by A-Levels is to meet a population which is very much different from the population of which the original A-Levels were designed for.

You said somewhere in region of five years in your Report for them to be uncoupled and you talk about there being a due process. You say, "The necessary design, development and testing for schools and colleges to familiarise themselves with any changing..." Do you include piloting?

(Mr Tomlinson) Yes.

You do include piloting?

(Mr Tomlinson) Somewhere else I do refer to piloting. The 5 years is not plucked out of the air, AS and A2 were three years, GCSE was four. I do not think I need to say any more in quoting those two. I think there needs to be a proper time scale. It was also informed by my view that we need initially to have the AS and A2 firmly established on their standards as well before we can move forward.

Will there be 50/50?

(Mr Tomlinson) I think that is a decision to be made when that is looked at. I am not making the decision.

We heard from Sir William Stubbs he thought piloting of the A2 would have been very difficult, do you agree with that?

(Mr Tomlinson) I do not know why it would have been difficult. It would have been difficult in the time scale given, it would not in essence have been difficult.

Given that there was no historical data to compare ---

(Mr Tomlinson) There was not for AS, that was piloted.

That was, but the A2s were not.

(Mr Tomlinson) I said in my interim report I thought that was one of the mistakes made, A2 should have been piloted.

Paul Holmes

Very briefly to go back to the regrading exercise, who was it that took the decision that you and the regrading panels would only look at the minority of cases that were changed with six marks and above rather than the majority of the changes with a range of three, four or five marks? Was that your decision or the recommendation of the Examining Board?

(Mr Tomlinson) That was me. The three boards gave me the data for their movements of grades, marked down grades this year, and they gave me data from 2001 and because most of the other stuff, remember the time scale, was archived and not easy accessible, they did refer to it orally but I did not see it on paper. It is not the case with all three boards I worked on the plus or minus five mark because the three boards were working differently and had different boundaries, one board has plus or minus two, one board had plus or minus three and the third board was plus or minus five. Most of it was bound up in the way the system operated. The decision to look at it was mine alone, based on that evidence and, as I already said, the evidence from documentation, which indicated whether or not the chair of examiners had been consulted about the changes and had agreed them. That was the basis.

When the regrading panels had finished, they looked at 75 different units covering 21 different subjects, in the end the person who decided whether to accept and implement the change was the accounting officer, the chief executive, which was the very people you were investigating in the first place?

(Mr Tomlinson) They were the people. That is what the code of practice requires. In my letter of 2 October to the then secretary of state I made it clear that that would be the case. It was a public statement, it was not challenged by anyone as being the right way to go about it. That decision by the accounting officer was not made out of that meeting, the accounting officer made the decision in front of everyone else who was present, including the chair of examiners for another board, including QCA observing, including an independent teaching association representative. If he or she wished to maintain the grade mark they had to put their arguments forward and at the same time it was looked at to see whether or not the chair of examiners present was satisfied with the argument as well. Where that was not the case further work was done, and it was.

When Ken McLone sat here in front of the Committee and said really the inquiry had vindicated him because there was not that many changes to grades, he was the person who decided there would only be a limited number of changes to grades.

(Mr Tomlinson) He had to sign off the ultimate decision. What I am saying was very different from what was done during the main part of the summer, that decision was made by him, and him alone. In some cases there was no reference to other people, certainly no people present. What I am saying on the regrading is his decision was made in front of and argued in front of all of those other people and there needed to be agreement and ultimately there was in all cases. In one or two it required further work to be done, beyond that the regrading meeting was in order to satisfy everyone that the evidence substantiated the decisions made.

In your response to a question from Kerry Pollard you were saying, yes, we do need to draw a line and restore confidence in the system. In paragraphs 73 and 74 of yesterday's Report one of the issues you talk about is about course work, you say that was not the thrust of what you were looking at, it was the issue of regrading, you talk about course work. There are a number of schools that we have heard about, Knights Templar School was visited by this Committee, where, for example, 14 out of 20 of their students got U grades on their course work and that brought it down, where they were getting As and Bs, they got Us for course work. You have said in paragraph 75 that you are concerned about the quality of communication and the feedback from schools and colleges about the course work and what went wrong. The head of Knights Nice Temple School was saying his teachers are still no wiser as to what was supposed to have gone wrong. He said they have had the course work back now with not one comment or mark on it. I have marked course work for 26 years and the rules are very strict, you have to annotate the work, you have to say why you are giving the marks. Here you have an example at the centre of a major control circuit about why they give 14 out of 20 kids U grades on one subject. They are not answering letters. They have not answered three letters. They have sent back course work with no evidence of being remarked, no comments on why it was wrong and yet these kids are re-sitting in January or the same teachers are teaching kids who are going to do the exams next summer and they have no idea.

Chairman

That cannot be right.

(Mr Tomlinson) That is not right. They deserve and we need a better quality of communication and feedback from schools and colleges. That school is not the only school that is complaining about these issues. I have had a number of letters. As you rightly point out, it is not within my remit to deal with this. I have, in fact, by raising it here and with side communication, it is not just an OCR issue, it goes more widely than that, it may be the volume this year, I am not sure, it is certainly the case that schools do deserve full and clear communication of these matters such that they can deal with any issues that may be about their understanding of the standard, but equally importantly it may be issues that the Board have to deal with. The QCA and the Board are, I believe and I know, looking at this issue of guidance and criteria for course work. What I found was I could not locate it to say it was a system-wide issue. If they had all been brought down, if there had been a total pulling down of grades associated with course work one would have seen very high levels of failure across the course work module, that was not the case, it was individual schools, clusters of schools, individual pupils which forced us into that conclusion that I have come to. Your fundamental point that schools deserve and need that should be annotated such that they could be understood. My suspicion is that the fact that papers are now returned makes examiners less willing to annotate their papers.

Paul Holmes

Yet they require the teachers to annotate and explain why they are rewarding the grades.

(Mr Tomlinson) This is about confidence in each other and systems basing.

Valerie Davey

The whole Report is, I think, based on a change of ethos that you are looking for. You are looking for robustness in the QCA, you are looking for greater openness and dialogue between the Examining Board, you are looking for a different status for the Joint Council and throughout the Report it is based on greater trust, greater understanding, greater communication. How is all that going to be enforced? Who is going to be essentially responsible for taking forward your recommendations now?

(Mr Tomlinson) The Qualifications Curriculum Authority has the main responsibility for that. I have every confidence in the new leadership of the Qualifications Curriculum Authority, Sir Anthony Greener and Ken Boston, and that derives from the way they tackled the issue round my interim report, they have tackled them with vigour and rigour, as well. It is obviously, as you quite rightly point out, a change of attitude, a change of ethos, a change of behaviour is really what this is all about. It must be QCA, equally the Secretary of State is also involved in this and has to be, quite rightly. He has already indicated he has made money available to help on one front, he has equally indicated he would be interested in costing for the increased use of ICT, which I think is very important. Going back to marking, with ICT you would be able to easily allocate scripts not just on the basis of a schools' package but on providing the examiner with a full range of the performance spectrum so that they were able to see As and able to see Us. At the moment you get the whole schools. Equally it would also mean that marks go up on the ICT and you can then identify if you have any rouge markers and deal with it. There is a whole set of things which would improve the consistency and reliability of the marking and examining process. I was pleased when he announced he was willing to look at that as well.

The time scale for that, how do you see this working and how long will it take to achieve the kind of examining bodies, admittedly it being an art still and not a science, which I take, as a former teacher, very much to heart. How long is that going to take to achieve?

(Mr Tomlinson) Some of it will be achieved I anticipate over the next months, because in January what will go into schools and colleges and it will be very, very, very clear statements of the standards associated with AA and AS supported by a whole range of exemplifications, including student work from examination papers. That will be there in January. Further material will come in later in the year. There is a training programme for examiners, markers, and so on, in place to take effect for next year's examinations. It will start there. I very much hope that the code of practice changes will have an impact through that. I cannot say how long it will take to fully gain the confidence of every party involved, it is impossible to answer that. It is important that it is done as quickly as possible. I do think it is crucial in this. I have stressed this. I know that QCA has a thorough plan in place for communication. I think that it is vital over the coming weeks that we find a way of communicating with all students currently in the sixth form in a very simple, post-card way that says what has changed, what is to change and how it will make sure that what happened this year does not happen again. They need to understand that, and their parents. Then we need to get through to the institutions or directors of chambers of commerce and the CBI about how they could work to get communication to employers. Equally, their confidence in what they are seeing on a certificate has been grappled with. There is a huge communication issue that has to start now.

Ms Munn

I wanted to clarify one issue, we were told by the head teachers who came to us that there were fewer re-markings and re-gradings this year than in a normal year, what was the process previously if a school was unhappy about the mark that a young person had received?

(Mr Tomlinson) The school can make an enquiry but it has to have the approval of the student before that happens, it did not used to require that but it does now. You have to get the approval of the student and that can sometimes cause difficulty because there are time limits and they could be away on holiday. You get approval from the student concerned and you then submit a request for remarking. At that point it is understood that that request could result in the mark going up or down and have the consequent impact on grades, it is not an assurance that it will always go up. The difference for me in my regraded process was the only movement could be upwards.

Mr Chaytor

Mr Tomlinson, you have talked this morning in your evidence about schools and sixth forms, a huge proportion of A-Level candidates come from A-Level colleges ---

(Mr Tomlinson) I mentioned colleges a number of times.

Obviously I was not listening carefully enough. In the regrading exercise was there a distinction between candidates in schools and in sixth form colleges? The impression is certain schools have made more noise about this whereas sixth form colleges seem generally content?

(Mr Tomlinson) There has been evidence presented to me that colleges - I am broadening it beyond sixth form, to FE generally - that they spend an enormous amount of time and effort getting ready for Curriculum 2000 and ensuring adequate training of staff and all of the rest of it. They felt, according to them, particularly well prepared for that. I think that from the schools' side, I am going to resist being critical, some of the issues are about time for teacher release, and all of the rest, given the pressures in schools. There is some evidence that some schools did not participate in the training for Curriculum 2000.

That does not feature in your Report.

(Mr Tomlinson) I do mention the fact that not all attended. I do understand their reasons, this is about the fact that at the times they offered training it is very often hard to get teacher release and the necessary cover.

Do you think in retrospect that needs a higher degree of emphasis than you have given to it or has been given by media coverage of these events?

(Mr Tomlinson) It might. You may well be right on that. That has to be part of this whole issue that I dedicate one chapter to, that is professionalisation of training. That is an issue I think cannot be tackled on its own, it may have to be linked with discussions about teachers' contracts, and all of the rest of it, that are going on at the moment. You may know that the FE does say it is slightly easier on occasion for them, given their size and capacity. It was not a great difference, it was a slight difference.

You said the examining boards gave you statistical information from 2001. In 2001 there were O-Level exams and AS exams, so which was it?

(Mr Tomlinson) It was both. In some cases there were already module syllabuses, the administration was slightly different because of the fact we had AS and A2.

The question of the modular syllabus, how do you respond to that? The unique thing about this year was that the grade boundaries were sifted for individual modules, it was not just for the aggregate scores?

(Mr Tomlinson) Because the aggregate scoring derived from the marks of individual grade boundaries those had to be fixed.

Is there not inevitably a cumulative effect?

(Mr Tomlinson) There is. It is in some sense a perverse effect. What you get is a regression. What you find is if you use only the marked grade boundaries for the units and you did you not look at the broader statistics, this is something which people need to understand, then you would have ended up with much lower numbers of A grades because the regression causes that. That was one of the reasons why you have to look not just at the mark, the unit grade boundaries, but the aggregate as well, the code of practice requires that to happen.

Was it unique the grade thresholds were changed this year for individual modules?

(Mr Tomlinson) It was not unique, no.

In paragraph 64 of your Report you talk about criterion-referencing and you say, "Effective use of statistical information will provide results which are closer to those that would result from effective criterion-referencing". Is that not like saying that genetically modified food is more authentic than the real thing?

(Mr Tomlinson) No. What that is saying is there is no such thing as a perfect criterion-referencing system.

Nowhere. Nowhere in the world?

(Mr Tomlinson) No. Once you have criteria you are open to different interpretations of those criteria by different people and different interpretations of the work they are looking at against those criteria. It is not an absolute science. You can get close, we are close in this country, possibly closer than many others, but at the end of the day you cannot be perfect, however statistics help you to get closer to that perfection.

Are you satisfied that overall in looking at the syllabuses of all three examining boards across all subjects the detail of the specifications are sufficiently close to criteria and reference principle or is there room for a greater degree of specificity?

(Mr Tomlinson) I think in some subjects that I have seen, I must say I have not seen and read every single one of them, in those I have seen, it is a small minority of cases, there could be much tighter specifications to help. That, of course, relates to some of the issues that have been raised in the reports.

Mr Chaytor: Thank you.

Chairman

You talked about the "frenzy" in the summer, who is responsible for stoking that frenzy, was it the Headmaster and Headmistresses Conference, was it the Today programme?

(Mr Tomlinson) The frenzy that I refer to is an annual one, the annual frenzy as soon as results come out, how some people are unwilling to accept that as a result of harder work and better teaching more students can achieve the standards. We cannot call for improved standards and then as soon as we begin to have them appear, and they are appearing, we suddenly decide they cannot be real, somebody has lowered the boundary. I find that very, very unacceptable. If that boundary, that standard, is not being maintained year-on-year then I think those people are right to raise those questions. One of the issues I raised very clearly in my Report is I do not think we can lurch from answering that question from crisis to crisis, there needs to be a systematic, consistent approach to looking at where the standards are being maintained all of the time. If they are we have to be honest and do something about it. If they are we have to accept the outcome and we have students achieving better than they did previously. After all that is what we want. We do not want it at the cost of lowering standards.

When the second part of your inquiry was published Sir William Stubbs reported your inquiry exonerated him by implication, he should never have been sacked.

(Mr Tomlinson) I make no comment on Sir William Stubbs. There is a process in train.

You can exercise parliamentary privilege. We cannot get you to say anything nasty, even about the Today programme!

(Mr Tomlinson) I apologise for not putting blood on the carpet. I am more interested in making sure students get what they deserve and that is not achieved by putting blood on the carpet, it is about dealing with the system.

You banged the table with your finger, Mr Tomlinson!

Mr Turner

Mr Tomlinson, is an AS level worth half an A-Level?

(Mr Tomlinson) It is at the moment, yes, by definition.

Even though both the former secretary of state and Dr Boston say that the AS paper is easier than the A2 paper?

(Mr Tomlinson) Of course it is, an A-Level paper in the past contained an easier group of questions and a hard group of questions. When you are testing over a two year period any A-Level paper, any student and any teacher will point out, there are an easier set of questions and there are harder ones.

One qualification is based on easier ---

(Mr Tomlinson) It is based on one year of study, not two. I would argue that your maturity level, your capacity to synthesise and to analyse increases and improves not necessarily linearly but it does improve. You can ask more difficult and demanding questions after two years than you can after one.

What about somebody who takes an A-Level at the age of 30?

(Mr Tomlinson) They are judged by that standard and very often they do well because they bring to bear an awful lot of maturity and experience.

Mr Pollard

Are you satisfied that vocational examinations are okay?

(Mr Tomlinson) I am insofar as I have looked at them partly because they do not follow the model of the AS and A2, all units are graded at the same level.

Chairman: Mr Tomlinson, we promised to release you at 10.45, it is now 10.45. We have found this a most useful session. Thank you very much.