Select Committee on Children, Schools and Families Minutes of Evidence

Examination of Witnesses (Questions 180 - 199)



  Q180  Chairman: Before you move on, Greg, can you tell us whether you share Andrew Bird's view? He said in his introduction that he was worried about the regulator, who might be a tick-boxing person with no strategic role at all. You are very complacent about that.

  Greg Watson: No, I think that is absolutely the right move to make. There is a lot in the detail, and we are all involved in a consultation process with the Department. There are some important nuances about where the line should be drawn between what remains under a Minister's direct control and what is the responsibility of the independent regulator. I have some views on that, and I guess that we do not have enough time to explore them in detail here, but, yes, the fundamental point in terms of getting the regulator right is to put the right responsibilities and activities on the other side of that independence line so that we get the building of confidence that we are looking for.

  Chairman: We will come back to that. Sharon is off to some important duty on the Front Bench, so I call John Heppell.

  Q181  Mr. Heppell: I am left feeling that there is no proof that standards have dropped, but also that there is no proof that they have been maintained. You said that it was difficult to see what you could do about that, and I find it difficult to see what measurement you could make, given the change in the system. However, you may be right to suggest that the regulator will ease some people's difficulties. But what about the implications for the validity and reliability of the work you all do in providing sample questions and answers and targeted syllabus training for teachers, including comprehensive teaching and learning materials about what exams will be about? Does that not encourage people to get children to study those things that are relevant to the test, rather than to the broader education that all of us are looking for?

  Dr. Bird: As you rightly said, we all provide curriculum support materials, training for teachers and such like. The publishers of books in support of our products also include sample questions, tests and so forth. I do not detect that that is very different from what it has ever been. Past exam textbooks have always included past questions to help people prepare effectively and it is surely better that people understand the style of questioning that they are going to face than that they do not. Our challenge in assessment terms is to make sure that we do not become formulaic, that we cover, over time, the whole curriculum— although we do not cover the whole curriculum necessarily every time we set a particular test—so that it does not become predictable and that it discourages people from things like question-spotting, which is not a new phenomenon either. I think teachers have always done an element of that. What we are trying to do is provide the very best access for teachers and the very best guidance and support, so that teaching experiences in the classroom can be exciting and informative and can carry students' interest forward, so that they are successful. We are all interested in children being successful, but we separate that very carefully; we do that through separate parts of the organisation. The setting of the exam is an independent operation.

  Chairman: John?

  Mr. Heppell: I thought Greg wanted to say something.

  Greg Watson: Just to add to that—

  Chairman: Greg, you have to catch my eye.

  Greg Watson: I forgot my flag today. I have a couple of thoughts. The first is that I want young people to learn to the syllabus. The syllabus is an important definition of a programme and the whole point of having a syllabus is to structure and motivate someone to want to learn. I do not think that is any different for a professional exam in later life or for a school exam. I think it is a good development that whereas I never did see my A-level syllabus and wondered till the bitter end what exactly it was I was heading towards, young people today have a pretty good idea from the start. That has been healthy. Secondly, I also want young people to demonstrate what they can actually do, in the exam. By being a bit more open, a bit more transparent, and providing a few more clues, we enable people to feel well prepared, and what we are actually is assessing what they can do not how successfully they have guessed what they are about to do, or their ability to cope with the surprise of what they have been faced with. I think that that has also been a positive development. But I would set against that the fact that there is a challenge and we employ expert people in the field of assessment to make sure that we keep the level of challenge right, and that it does not become too formulaic and too predictable, which obviously would mean beginning to lose some of that effect.

  Q182  Mr. Heppell: I want to add a further thing about the idea that there is a sort of competition to pick the easier course. We actually had someone in this Select Committee telling us that that was common practice in schools; that schools would look at the syllabuses and pick the easiest. What about from the other side? What about universities that blacklist qualifications because they say they have been made too easy?

  Chairman: That is common practice, is not it?

  Mr. Heppell: Is it common practice?

  Dr. Bird: I think some of the universities have got a list of subjects that they would encourage people not to do more than one of, in the sense that if you were doing a number of these newer subjects—media studies and something else, maybe—they feel that that might unbalance your curriculum. I do not think they have actually blacklisted A-levels. They have discouraged people from doing too many of them.

  Q183  Chairman: It is a bit of a mafia, though, isn't it—all the senior tutors in charge of admissions at Oxford and Cambridge get together on a regular basis. They do not have to have it written down on a list.

  Greg Watson: I think everyone who—

  Mr. Stuart: Greg is on the high table.

  Greg Watson: Bearing in mind that there are over 100 universities in this country and 50,000 different degree courses, I am not surprised that different universities and different departments in universities come to a particular view on what is the most appropriate basis for being prepared for their courses. It would be pretty extraordinary to imagine that a maths degree would be available to someone who had not done maths A-level. That is perfectly proper, and perhaps that is a question for the universities to answer, rather than us. Our job is to run A-levels to a common standard, right across the subject range, and to make sure that the A-level is worth the same, as a qualification, regardless of which subject you do it in. What use universities subsequently make of it is down to them trying to target the right kind of young people to get on to the right kind of courses.

  Q184  Mr. Heppell: I was thinking about common qualifications. I understand that some universities differentiate the courses and say, "We will take the OCR one as one we accept but we won't accept another." Does that happen? I thought it did.

  Greg Watson: Not to a great degree that I am aware of.

  Jerry Jarvis: If it does, it must be very limited. There are areas of the country where there are affinities between certain universities and certain exam boards. There are traditions, and there are preferences in the selection process, but I cannot believe that it is widespread. It has not come to us.

  Q185  Chairman: What does Murray think? One thing that comes through from reading all this stuff, as we have got into the inquiry, is that even if a parent does not remember their A-levels and GCSEs very well, they remember their degrees. I was leafing through my examination results from when I was a small person right through to my postgraduate time. I looked at the year in which I got my degree from the London School of Economics, and eight people got firsts throughout the whole LSE, in all subjects. That does not happen now; the figure would be one third. It is not your area to set degree papers, but surely you must be worried about that. Does it not raise a big question about whether a first is good as it used to be, if a third of people get firsts compared with a small number not very many years ago?

  Murray Butcher: By definition, yes. You are either talking about an elite, or you are not. I spent many years as an employer of degree-holding candidates, and the university from which the degree is gained and the subject always creates a choice for employers. Gosh, we are moving off the subject.

  Q186  Chairman: No, I think it is very relevant, because what part of testing and assessment is there to try to dispel this view—if it is wrong among parents—that standards have got easier? I pitched to you the idea that in degrees that it is the case, with so many people getting firsts compared with lower and upper seconds.

  Greg Watson: I honestly do not think I could comment, not being involved in university-level assessment.

  Q187  Chairman: That is why I was going to ask a second question. What I am getting from you is that you do not really know because you do not have that much of a link with universities. Surely, part of your job, and one criticism of the examining boards, is that you do not any longer have enough of a relationship with the teachers who teach the subject, or with the people who take the products after they have taken your examination—that you have become rather isolated, both from the teachers below and from the university teachers above. That surely cannot be good if that is what people are saying about you.

  Dr. Bird: Is that the evidence you have had presented? I would certainly reject that on behalf of AQA. We have active teachers and head teachers on our governing council all the way through to people involved in our subject committees. They are active teachers who act as senior examiners, preparing and developing our material. From the classroom perspective, we have got that active involvement with the chalk face—the whiteboard face as it now more correctly is—and HE is represented from admissions and senior academic perspectives on our governing council.

  Q188  Chairman: So most of your examiners will be teachers, will they?

  Dr. Bird: Yes.

  Jerry Jarvis: All of them.

  Q189  Chairman: All of them?

  Jerry Jarvis: Decisions in all our awarding bodies are taken essentially by practising senior teachers—from the grading decisions to the writing of examination papers and so on.

  Greg Watson: I am actually quite concerned about what you say, though, because we could not possibly compete if we did not have an intimate relationship with people who are ultimately our customers—the takers of our qualifications.

  Q190  Chairman: Any hints that you would knock that to one side?

  Jerry Jarvis: Certainly, in the development and creation of curriculum materials and in supporting a teacher in delivery, I cannot subscribe to that view. However, remember that we are required to demonstrate rigour, and part of that demonstration necessitates distance. I could well understand that when it comes to a disagreement over an examination outcome, we have a role to play that is authoritarian, and we must maintain some rigour and some distance from the process of learning.

  Greg Watson: As awarding bodies, we occupy a unique position. We sit in the middle of lots of different stakeholders in the qualification system. On the one hand, we have schools and colleges, teachers, young people and parents. On another side, we have subject groups, subject associations, leading thinkers in a subject and groups in universities that do research on particular subjects such as the Salters group at York. On the other hand, we have the Government of the day and their political drives of various sorts, and, of course, we as a charity also have our own education mission. We have a lot of experience—150 years—and a lot of research to tap into. They give us good clues about the right direction of travel. Inevitably, sitting in that situation, we do not please everybody all the time, but we do play an important role in drawing on all the different views and trying to square the circle in a way that I do not believe anybody else in the system does.

  Chairman: Because I was following the run of your answers, we have been stealing Lynda's questions. We are over to Lynda now.

  Q191  Lynda Waltho: I was actually going to extend the question, although I wanted to deal in particular with differentiation. What input do you get from employers? There is a general feeling, certainly within the trade press—specific evidence has not been given to this Committee—that there is not sufficient input from academics and employers, particularly in respect of the closed questions that make up a large part of exams. They may be deskilling our school leavers, and not expanding their analytical skills. What input do you get from employers, who are receivers as well?

  Greg Watson: A couple of comments. First, there has been direct input from employers in respect of many of our longest standing and most successful qualifications. It is still the case that the single most heavily used qualification in the history of this country is an adult IT qualification called CLAiT, which was developed almost 20 years ago in direct response to trends in the employment market and signals that were coming from employers and further education colleges speaking on behalf of employers. More recently, very successful new vocational programmes called OCR Nationals have been rolled out to schools. We built those—again, sitting in that unique position that I mentioned—listening, on the one hand, to schools and colleges and what they hoped they could offer to young people, and, on the other hand, to employers and their views on what was relevant to modern employment. I think that we do play that role. That said, I spoke in my opening statement about the impact of the past 10 years and the growing use of qualifications as a public policy tool—perhaps more change has been driven directly by public policy. In my position sitting in the middle of everybody, I would say that in the past 10 years we have probably been drawn closer to satisfying the Government's direction of travel, and the trade-off has been that we have given less time than we would have done 10 years ago to consulting directly with universities and employers. There are only so many hours in the day, and only so much time available to develop new qualifications. I hope that, with the arrival of an independent regulator, there will be the opportunity to rebalance that slightly, because there are times when I would like to be closer to employers and universities in developing some of our qualifications.

  Q192  Lynda Waltho: That is the general view?

  Jerry Jarvis: I would endorse to a large extent what Greg said. Half of Edexcel's provision is in the vocational workspace with BTECs. I believe half a million students are doing them. Those qualifications would not work if they had not been developed in conjunction with employers. They are vocationally oriented qualifications for the marketplace. The degree to which employers play a part in the setting of criteria for what is taught at GCE and GCSE is open to question. Examination organisations such as ours are required to create qualifications that conform very closely to criteria set down by the regulator. There is always a case for better collaboration with all stakeholders, including parents, in the putting together of criteria. We are all constrained by, and must work within, the criteria that are set down but, certainly, when you move away from GCE and GCSE, the degree of employer collaboration is massive.

  Murray Butcher: My answers would be from a vocational context. Employers can contribute in two ways. First, they can contribute through their role on sector skills councils, in which the creation of national occupational standards takes place. They lead to the creation of national vocational qualifications. Secondly, they can contribute through the provision of supervision of that activity. The key person within vocational qualifications is the external verifier. He or she works to a particular awarding body and visits locations to ensure that practice accords to standards. They may be practitioners from further education, or people drawn straight from employment. There is considerable opportunity for employers to play a role, but I concede that it is quite difficult on occasions to gain such contact with employers because, naturally, they see their principal activity as earning their particular income from their role—finding time to engage in education can be quite difficult for them.

  Q193  Lynda Waltho: That finishes that part, but I would like to extend the question and to look at the differentiation of students for selection purposes. Certainly, top universities argue that too many applicants attain the top grades, which often leads to more testing. The Committee has heard that high-stakes testing and the need for schools to show well in league tables has resulted in teaching to test. What are your views on that? Does the league table culture in schools distract teachers from the task of preparing school leavers with the deep knowledge and the independent analytical study skills that are necessary for higher education and higher training?

  Jerry Jarvis: Let me take the first question on whether testing has an effect on the behaviour in the classroom. There is huge pressure on schools and teachers to increase performance. When we have one of our regular focus groups, I sit with a panel of teachers and ask them to introduce themselves. Almost without exception, they tell me their name, what school they are from, the subject that they teach, and the pass rate in their subject. Unfortunately, there are huge issues at stake in most schools, and teachers are human. Having said that, the huge overwhelming majority of teachers aim to deliver on education—that aim comes across strongly in what they do. However, there is no question that there is pressure. We are talking about teaching to test. If we can write examination materials that cover the whole syllabus in a way that means that people cannot predict how something will be questioned or what the questions will be about, it would be necessary for schools to teach the whole syllabus. When I came here today, you very kindly issued me with some instructions and notes about what was going to happen today; you made me feel comfortable, and you gave me a brief outline of the lines of questioning that I would be given. You were either leading me and helping me to respond, or you were just trying to set down the rules of engagement. Providing we stick to the notion that we are talking about the rules of engagement, surely we are right, but the pressure is massive.

  Dr. Bird: I think the pressure is massive. My concern as a former sixth-form governor is that it is too much about output measures and not about added value. There is so much in output measures that is to do with the inputs that the students bring with them and less about what the school adds through its teaching and learning processes. So there is a place for more sophisticated league tables if they will be used to assist schools in improving. As Jerry said, we aim to provide a broad curriculum. We aim to test all of the student's skills. We are a strong supporter of the introduction of A* at A-level. We certainly believe that there is enough evidence in the current distributions of marks that that could be done from the current exam papers, if the Government were minded to do so. We feel that it is a cause for celebration that more than 20,000 people are getting three As. It is a very small proportion of the total candidate group and it is a very small number of university departments that have a problem with that. We appreciate that it is a problem, but those departments that select have known for many years that it is not just three A-levels that you need to be a good vet or a good lawyer—you need a range of other things—and they go out of their way to establish the other skills that good sixth-form curriculums provide, such as extension work, project work, community work and work experience. Those things can also be certificated. We have a qualification that we are trying to get approved that would support the bundling up of those activities.

  Q194  Lynda Waltho: So it would be a combination of those extended activities plus A*. What do your colleagues think about the A*? I would be quite interested to hear that.

  Greg Watson: The A* is the right development at the right time. Andrew has mentioned that the number of candidates getting three As at A-level is small; we are talking about 3% of the 18-year-old age group. So it is still a pretty tall order to end up with three As. Nevertheless, there are now enough young people in that category that certain universities are saying that that achievement is not enough to differentiate. As I said earlier, I think that we have made a bit of a trade-off in the evolution of A-level, between broadening it as a general qualification and maintaining it as the most stretching assessment for getting into university. I think that reintroducing an element of stretching in each subject and marking out an A* grade as a higher bar for the best candidates to get over is a perfectly sensible thing to do, and I know that a lot of universities will welcome that development. I think that the big pressure in this area is the GCSE A* to C measure. When Jerry is talking about the teachers round the table, that is the thing that I know drives a lot of institutions. That is probably the greatest risk of creating any distorting incentive and of distorting things in two ways potentially. One is that I think that if I were in school today, I would rather be a D-grade candidate who was very close to the C grade boundary than a D-grade candidate who was at risk of ending up in the Es. There is always the danger that the last half an hour of the teacher's attention will inevitably be drawn towards the candidate who has a good chance of getting over the C grade hurdle. The second issue is that, as the curriculum evolves and we have new qualifications coming along, the pressure to find new qualifications that can be treated as equivalent to a C grade at GCSE can itself become a distorting incentive in the way that those new qualifications are developed; you can see that from much of the public debate about Diplomas and how they are going to be treated. Attention could become concentrated on that issue, rather than on whether it is a good curriculum innovation and will be a good new offering for young people.

  Lynda Waltho: That is fine. I think that Dawn wanted to come in on Diplomas.

  Q195  Ms Butler: I just wanted quickly to touch on Diplomas, because I know that Graham is going to talk a little about them; he thinks that I am stealing his thunder. Do you think that the new Diplomas will help to bridge the gap between school and higher education? We know that young people who come from more affluent families already have this type of mapping; going to school, then going on to higher education and going to university. Will Diplomas help to bridge that gap?

  Jerry Jarvis: One thing that the Diploma may do is broaden the experience and learning of students who enter university. They might have taken a much broader, different curriculum, and perhaps one that was closer to practical learning than previous syllabuses were. There is an opportunity there, although a huge number of students enter university through BTEC nationals and, by proportion, do very well in taking degrees. I am interested in your identification of the gap.

  Q196  Ms Butler: As I was saying, we are trying to inject aspiration into the learning agenda. We want to give those who might not have thought about going on to higher education the opportunity to do so, as the BTEC does. The question is partly how we can ensure that employers take Diplomas seriously. Also, how can we ensure that a Diploma is equivalent to three and a half A-levels?

  Jerry Jarvis: We have to work very hard to have that Diploma earn its spurs and get it the reputation that it needs. It must cover a great deal of ground, and it must be valued by everyone, not just those who would not necessarily have considered themselves able to go into higher education. It has the potential to offer a very different form of learning on the way to either higher education or employment. The Diploma is a big ask, and no subject is dearer to us right now than to make it work, as the first teaching starts in September.

  Chairman: We are deep into Diplomas now, so I shall share the Diploma questions around. Fiona first.

  Q197  Fiona Mactaggart: Greg, in your evidence you quoted a delegate at a conference that you organised, who said that the Diplomas were examination officers' worst nightmare—a telling phrase, not usual for evidence to a House of Commons Select Committee. You clearly feel strongly about this. What is nightmarish, and what would you have done differently?

  Greg Watson: Just by way of explanation to members of the Committee—I am conscious that our sector, like many, is full of all sorts of strange language—exams officers are the administrative hub in a school or college, responsible for making exam entries, ensuring that results are dished out in time and so on. I mentioned Diplomas because they are the most complicated qualification that I have ever seen. A typical learner is likely to take part of their Diploma in a school and part in a college, and they will do different elements of the Diploma with different awarding bodies. The exams officer will be faced with a brand new IT system built at great speed, and just about in time, with which to administer all this, and any given exams officer will probably have to work with an exams officer in at least one other school or college to ensure that they use the same reference number for a candidate and keep track of what a candidate has done so far. They must also know exactly what the candidate still needs to do to complete the qualification. Running the exams office in a school or college is a tough job anyway. It is often a temporary or part-time job, and has a churn rate of about 50% of individuals in any given year. In many places the exams officer struggles to have authority with the senior management team in the school or college to get things changed. We need to recognise the complexity of the qualification in its own right, the practical logistics of how it will be taught and the fact that it needs to be sorted out in pretty short order for the first learners, who will be going through the programmes in September 2008.

  Q198  Fiona Mactaggart: That comment is about the administration of it; let us consider the assessment of it. Murray, most of your assessment experience is in the thing that is worrying us in some ways about the Diplomas—whether the person can do the extended portfolio. Do we have people who can carry out that quality of assessment? Will it work?

  Murray Butcher: At this stage, a great number of questions remain on the form of assessment that will take place within the various themes of activities, from the principal learning and the additional specialist learning to the extended project and the functional skills. We are likely to find a range of assessment practices that will cover all of those. Some will be internal, some external, some will be moderated and some will be verified by the awarding body. Even though we are planning to release the first teaching in September, there is still quite a long way to go, and quite a lot of discussion to have with the regulator to agree on the types of assessment that will take place. I believe that quite a bit of responsibility will fall on the schools and the delivery consortiums, and funding is already in place to support that. It will be fairly heavily pressurised in order to ensure that the teaching staff at schools and colleges and even the local employer have the necessary skills. It is a very big question.

  Chairman: May I ask everyone to be quite short? My colleagues are tending to ask all of you questions on each section.

  Dr. Bird: Very quickly, I agree with all that Murray said. I would just add that none of those means of assessment are new. We have experience of using them in centres now. We have lots of experience in training teachers to be assessors; we have done it for many years. It is a big ask and a lot to do. That is why we are encouraged by the Government's process of controlling volume in the first few years with Gateway centres and so on and by the money that they are providing to help with that verification and assessment development process.

  Q199  Fiona Mactaggart: I was interested in the bit of your evidence in which you said that coursework had changed over time and that assessment used to be more embedded in courses. You also said that the development of extended portfolios had created greater opportunities for plagiarism, and that some of the risks that people have suggested has occurred with coursework. I hope that I have not misread you. I wonder how you assess the extended project element of the Diploma and whether such an element holds the same risk.

  Dr. Bird: The extended project is a piece of coursework that is chosen by the student and arises out of their other studies. It is not a subject set by us, but it is approved by us. It then requires a student to do a piece of work and present some material. Clearly, that involves them doing research and using the Internet and so on. By its very nature, it is a unique piece of work that is based on a student's interests and the A-levels that they are doing alongside the project. The chance of copying great chunks of it, therefore, is massively reduced. That was our point. Old-fashioned coursework embedded in the curriculum arising out of a student's experience requires higher quality assessment skills. However, because it is different for every student, such work reduces the chances of plagiarism being a serious problem. With coursework in general qualifications, we have reduced its variety, standardised it and made it a task. That task, therefore, is difficult to differentiate, which means that you are looking at lots of very similar things. The chances of people sharing that and copying are therefore increased because it is the same piece of work. The thing that we find disappointing about coursework in general qualifications is that just as a number of us are launching the portfolio products that allow students electronically to publish evidence of their work and have it assessed remotely, which means you can provide photographs, audio clips, short videos and texts, the system is closing coursework down and making it a fairly repetitive and limited task. The technology enables things to be checked, which, we think, would allow coursework to flourish. We hope that that will be the case in the Diploma environment.

  Chairman: Very interesting.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2008
Prepared 13 May 2008