Select Committee on Children, Schools and Families Minutes of Evidence

Examination of Witnesses (Questions 200 - 219)



  Q200  Fiona Mactaggart: I have heard that administration is hard, and that we need particular skills to assess the can-do bits of the coursework of the Diplomas. Do you think they are going to work? A yes or no answer will be sufficient.

  Dr. Bird: Yes.

  Murray Butcher: Yes, we are working very hard to ensure that.

  Jerry Jarvis: I would rather come back with the statement that they have to work. We are making a huge investment, and we have made a very bold decision. We all feel exactly the same way. We all suffer from trepidation in a whole series of different areas, but we have to make this Diploma work.

  Greg Watson: I will say yes, with one important proviso. I return to an earlier question about whether employers and universities will really feel that the Diplomas are worth while. The answer is that they will see the proof with their own eyes, because they will get to know young people who have been through these programmes. Whether the Diplomas will succeed in the early years will be more about how they are taught, and then about the kind of young people that they produce. Regardless of all the technicalities of assessment—and there will be issues that we have to iron out over time—if the Diplomas are really to succeed in the first two or three years, it will be all about the teaching. That means that support for teaching has to be exemplary and that we should be sensible about the number of students that we want to see on those programmes early on. As Andrew said, it has been helpful that the Government have put some criteria in place. If the numbers do not grow too fast, and we devote an extraordinary effort—not just the Government, but people like us—to supporting teachers in the first few years of teaching, Diplomas will succeed.

  Chairman: Murray, may I ask you to be one of the first respondents to Graham?

  Q201  Mr. Stuart: Fiona has given you a pretty good work through. We ended there on a positive note that, "We are going to make it work because we've got no choice." City & Guilds has the greatest experience of vocational training; may I put it to you that there has been insufficient time for component awarding bodies to research and develop new assessment approaches? Are Diplomas being introduced to a political, rather than educational, timetable?

  Murray Butcher: Timetable, perhaps. In terms of trying to draw—

  Q202  Mr. Stuart: Is that a yes or a no? Would any one of you, or anyone whom you know in the educational establishment, have gone for this timetable if it had not been dictated by the Government?

  Murray Butcher: I think I would need to collaborate with my colleagues on either side to reflect on how some of the Curriculum 2000 qualifications were introduced. That was also an extremely short time scale, but it was achieved. It is possible that we can achieve this, but some very concentrated thought is needed. It is, in historical terms, more of a political timetable than a curriculum development one—that is true.

  Q203  Mr. Stuart: Yes; it comes in in a few months, and you say that there is a lack of clarity over the purpose of the qualification and its underpinning curriculum.

  Murray Butcher: Yes—

  Q204  Mr. Stuart: That is pretty damning is it not?

  Murray Butcher: There are considerable concerns—if I can explain. As a vocational awarding body, we see this as an opportunity to introduce a hint of vocationalism into the school and post-school curriculum. We carry considerable anxieties that, as we go through that process, the pressure on the award may limit the amount of vocational experience available and that the award may drift back to the standard, general qualifications—much as we believe GNVQs did. My anxiety is that if it is successful, this qualification will also be a challenge for higher education, because I believe that it will draw in a slightly wider cohort than has been experienced up to now. There is a lot of pressure on HE to increase its intake, and it must be borne in mind that the nature of the intake, as well as the skills and approach of young people going into higher education, may change. There will need to be a response to that.

  Q205  Mr. Stuart: The advanced Diploma is supposed to be worth 420 UCAS points, whereas three As at A-level are worth 360 points. Is that realistic? Will it carry public confidence?

  Murray Butcher: I do not see why it should not. We are devising a new qualification that is trying to attract young people with a range of skills. I would not wish to compare the skills of this young person to that young person. If they are successful in the award, they deserve the three or three and a half A-levels that they achieve.

  Q206  Mr. Stuart: People have had theories over the years and they have always come up with conceptually excellent ideas that, like GNVQs, end up not working. The two biggest challenges, according to The Guardian, are university endorsement and parental endorsement. Is not the real challenge in getting the new qualification going that you have to get enough good pupils and good teaching to get it off the ground? The tendency is that a qualification comes in and it is the poorest cohorts at the poorest establishment who are often the first adopters. Is that not a danger? How can we prevent it from happening, so that the Diploma can flourish?

  Jerry Jarvis: You are absolutely right. I made the point earlier that the reputation of this qualification will be critical to its success. The programme of introduction is too fast. There is too much involved in this programme, and too many other parallel changes are going on at the same time. We would not have done it this way, but we have made a commitment. We have had our arguments with our regulator, with the Department, with each other and inside our own organisations, and we have made a commitment, so there is a point at which we have to say, "We wouldn't have started from here, but we have to make this work." So much is at stake. The ultimate success of the qualification—

  Q207  Mr. Stuart: What if we had had you here when we were talking about GNVQs? We can look at every previous initiative, because we have always recognised we do not have the right system for vocational education. If we had had your equivalents in the past year, you would have all said the same thing, wouldn't you—about the need to make it work? And they have not worked. I am trying to tease out just how serious the problems are with the lack of clarity over the purpose, the timetable, and the risk of undermining GCSEs and the AQA coming in.

  Greg Watson: I am not sure I necessarily agree with the statement that GNVQs did not work in the end. Through a process of iteration over about 10 years, GNVQs became confidently used in some schools and colleges, but it took 10 years. When the Education and Skills Committee looked into Diplomas—we offered evidence about the speed—there was, I think, recognition in the Committee's report that going at this speed does not help and putting pressure on numbers early on does not help.

  I mentioned OCR Nationals earlier. When we were developing those, we probably took a year longer than we took over Diplomas. That gave us a year to triangulate more effectively between what employers wanted and what schools and colleges thought they wanted, and to look hard at assessment structures and ensure that grading arrangements and the like were going to work well. I think we would have all wished for a little more time. On a positive note, there has been an excellent dialogue between ourselves, the Department and the QCA about the first phase of Diplomas and how that process works. I think everybody has recognised that it was very loaded towards the employer view and we did not take on board enough about teaching and pedagogy and did not, early on, spend enough time on the business of assessment and standards and ensuring that the outcome of the Diplomas would command confidence. There is a commitment all round, among all the people I have just talked about, to go about the second phase in a rather different way. That in itself shows that we are perhaps recycling the learning faster than we might have done with the GNVQ.

  Chairman: There is a bit of pressure on time. We have to move on to coursework.

  Q208  Annette Brooke: You have already hinted that there are quite a lot of advantages in using coursework in the examinations system. Given the rather dramatic moves, particularly with the GCSE, is this a question of throwing the baby out with the bathwater?

  Dr. Bird: In my early remarks about coursework and the extended project, I was, I suppose, broadly hinting in that direction. Very few people in their daily lives do not discuss with colleagues an issue they are trying to tackle, or use the Internet or any of those wider research skills. Clearly, properly constructed coursework questions allow people to demonstrate, in preparing a document for moderation, those wider study skills and extended communication skills, which it is very hard to assess in a quite short, formal examination.

  Q209  Annette Brooke: I shall ask another question and perhaps somebody else can pick it up. Is it simply that you guys have not been smart enough in setting the coursework?

  Chairman: Who is that question to?

  Annette Brooke: I am looking for volunteers. I though that it would be a bit unkind to pick on somebody.

  Chairman: Jerry can go first, followed by Greg.

  Jerry Jarvis: I think that you have two volunteers here. It is probably a case of throwing the baby out with the bathwater. It is not that difficult to detect plagiarism. After all, the same mechanism that allowed the student to find the material to plagiarise makes it just as easy for the moderator to find it. We believe that coursework makes a hugely valuable contribution to the way in which we conduct assessments. It is a far richer method than an end-of-term examination. Last year, Edexcel did a huge amount of work on detecting cheating throughout the UK. We examined all sorts of issues, but detecting cheating in coursework is easy, actually.

  Greg Watson: Yes, I am disappointed with our direction of travel. Hopefully, some of what we learn from the extended project within the Diploma will flow back into coursework. Ironically, the extended project could prove to be one of the most exciting parts of the Diploma development. It could be a genuinely personal and independent project that would build evaluation, research, thinking and self-organisation skills, which are exactly the sort of skills that coursework—done well—has the power to develop. Coursework is not right for every subject, and in some ways we have suffered from having veered from one side of the road to the other. At one time, we decided that coursework was a good thing and, through over-mighty regulation, we were compelled to put coursework into subjects such as maths, where it does not sit readily. There has been a sudden change of tack and now we are compelled to take coursework out, even when, in subjects such as geography or history, it can breed fundamental skills that, incidentally, carry on very well into higher education, too.

  Murray Butcher: I was just reflecting on some good, simple assessment reasons for maintaining coursework. We are trying to get as broad a picture as we can of individuals' abilities. If you take out coursework, you are focusing on just one or two other forms of assessment, which gives you a biased picture of the individual. I share the view that by withdrawing coursework, we would be going in the wrong direction.

  Q210  Annette Brooke: On gender balance, I have the impression that girls started performing rather well in some GCSE coursework. Is that statistically true? I suppose that I should look to the awarding bodies for that information, but I thought that you might have the gender breakdown of results with and without coursework.

  Jerry Jarvis: We could provide it but, anecdotally, it is generally the case that girls do better in coursework, as well as the other subjects, although that varies enormously in coursework subjects. However, yes, they are better at coursework.

  Q211  Annette Brooke: I find that interesting. I wonder whether getting rid of coursework would solve one of the Government's problems by potentially closing the gender gap—I accept that that is rather cynical. Something that concerns us greatly—this might be in line with our questions and other lines of inquiry on social mobility—is that young people from different backgrounds and with differing levels of family support clearly will have different opportunities within their coursework. For instance, I recall that the Prime Minister's wife could get help with certain coursework, and I used to take my children to museums and so on. How do you factor in those things for a true assessment?

  Chairman: You cannot, can you? Is it fair? Is it biased towards middle-class kids? Do less privileged kids find it harder to achieve?

  Greg Watson: That is very hard to iron out. However, returning to something that a couple of us have said, the notion of a truly personal piece of learning could get over that problem, because it would be in the bounds of the imagination and creativity of the young person and their teacher. I can think of good examples in GCSE business studies, in which coursework is open-ended. A student goes to a local business, works up a project on one aspect of the business, and writes it up. A teacher who encourages young people to think imaginatively, or a school that maintains good links with local employers, is able to do that, regardless of any advantage or disadvantage in the home background. Part of what creates the difficulty is the tendency, through regulation, to drive coursework towards being a standardised task. If anything, those other advantages kick in to a greater degree when that is the case.

  Chairman: David, do you want to come in on coursework?

  Q212  Mr. Chaytor: Yes. Specifically, what are the most effective means of preventing plagiarism in the first place? I take the point about making coursework more personalised, as against standardised, but how do you prevent someone from accessing the Internet or from getting a disproportionate advantage from having well-informed parents?

  Jerry Jarvis: The view that we have taken is that the best way to prevent plagiarism is the likelihood that your efforts will be detected—it is like the idea that you will probably drive slower if you suspect that there is a traffic camera around. The best way to do that is to sample students' work. As I said earlier, moderators and examiners can use the same methods to access non-original material as the students did. These days, even with material that is being shared among students, which might not be generally available, we have deployment technology and we digitally scan everything that we do, so we have the ability to have a machine literally looking for similarities that could not otherwise be detected without the deep reading of work. For me, the way to discourage plagiarism is through the probability that you will be caught.

  Chairman: Andrew, do you want to come in on that?

  Dr. Bird: I agree with some of that. I think the real way of inhibiting plagiarism is to make sure that you have set the right task. If we set a task that is basically a knowledge task, we are encouraging somebody to go to the Internet and find out all they know, so we need to set a task that requires them to acquire some knowledge, from wherever they get it, and then to do something with it—to process it and convert it into something that is about them and their insight into that knowledge. The task needs to be about underlying ability, rather than a recounting of what they can find in a book or on an Internet site. Tasks that require people to give empathy of analysis are the sort of coursework tasks that we ought to be setting. That, in my view, is the strongest way of avoiding plagiarism, followed by the sense that there is a speed camera around the corner.

  Q213  Mr. Chaytor: City & Guilds is particularly keen on the role of coursework, and local assessment of coursework is more frequent in vocational qualifications than in more academic qualifications. What are your views on all that? Do different standards apply to courses that are defined as vocational, as opposed to those that are defined as academic?

  Murray Butcher: There are similarities and differences. Within a National Vocational Qualification course, you are dealing with the fact that the curriculum contains certain criteria. When you come to assess what, in vocational qualifications, is called the portfolio, which is a collection of evidence, you are testing that evidence against certain specified criteria and looking for the supporting evidence within it. That does not mean to say that there are not occasions when you find that individual students and trainees come up with extremely similar evidence. Supported by regulation, we must carry an investigations team within City & Guilds so that whenever questions occur that might suggest plagiarism—or not so much plagiarism as the possibility that the centres providing the documentation are not encouraging individuals to do things themselves—we need to explore that and confirm whether something untoward has taken place. So, in a sense, plagiarism is possible in NVQs, and we have to make sure that we eliminate it.

  Q214  Mr. Chaytor: Broadly, what proportion of awards are rejected because plagiarism has been identified by City & Guilds and the other boards?

  Murray Butcher: Within City & Guilds, it is going to be an extremely small percentage—less than 1%.

   Greg Watson: For OCR, 192 candidates in 2007 had sanctions applied for plagiarism.

  Q215  Mr. Chaytor: As a percentage of the total?

  Greg Watson: Minute.

  Q216  Mr. Chaytor: Broadly, on the use of the Internet in coursework and to support learning for external examinations, is there any evidence that the digital divide between families is leading to a widening gap in the achievement of young people? Is access to the Internet a factor that drives up standards very quickly for the most affluent families, leaving the children of poorer families behind? Is anyone doing any research on this, or do you have any thoughts about it?

  Greg Watson: That is actually a very difficult question to answer from where we are sat. We assess what is put before us. Given that most of it arrives in a digital format, you could say, prima facie, that there is no obvious divide, but I cannot gauge—I am not sure that I even want to—who has used what means to get there. We simply mark it all to the same standard when we are presented with it. There is an interesting research subject there, but I cannot comment from the evidence that we see.

  Jerry Jarvis: Following on from an earlier question about setting an examination to ensure equality of access, particularly for disadvantaged children, we do quite a lot in setting all our examination and assessment work on the more obvious things, such as language and culture. We would not want to set a paper in geography in which someone who was a non-UK resident would be disadvantaged, so we think very carefully about that. We would not require someone to do a field trip to France, so we can make sure that when we set our assessment instruments, we do not. However, I am afraid that one or other of the population will be inadvertently disadvantaged. Geography comes in as well, with remote villages and so on where access to appropriate research facilities is limited. We have to be careful but, inevitably, some students have greater advantages than others.

  Q217  Mr. Chaytor: May I move on to the QCA? All four boards agree that the changes to the QCA are welcome. That is right, is it not?

  Dr. Bird indicated assent.

  Greg Watson indicated assent.

  Jerry Jarvis indicated assent.

  Murray Butcher indicated assent.

  Q218  Mr. Chaytor: But what has it done wrong? Where has it gone wrong?

  Chairman: And why were you not lobbying for the change before?

  Greg Watson: I was.

  Chairman: Oh, you were. Okay.

  Dr. Bird: We were as well. The QCA has not done an awful lot wrong, but having the two activities together can generate a conflict of interest, and separating curriculum development from the regulation of assessment can only be a good thing. The regulators spent a lot of time thinking about the development of the new A-levels, but never mentioned what it would cost to assess those various models of assessment. Sometime later on, QCA quite properly wanted to explore our pricing strategy for those A-levels—after we had been in development for some time and the die had been cast. That is the very worst set of arrangements, because we are already committed to a style of assessment in association with the curriculum development side of QCA at a time when the regulatory side started talking about pricing, which is influenced by what it costs to deliver the style of assessment. Separation of the two roles should lead to a more robust dialogue between the regulatory side and the development side so that we triangulate the issues more effectively, because the consequence of development requirements on us changes what we can deliver and what it might cost to put it into the marketplace. One can see a more robust triangular relationship across the industry, which is why we favour it.

  Q219  Mr. Chaytor: The example that you have given could equally be used as an argument to scrap the curriculum development agency function completely and devolve it to each of the four boards.

  Dr. Bird: It could be—

  Mr. Chaytor: There is no guarantee that a separate development agency will also not overlook the practical implementation costs of the financial arrangements

  Dr. Bird: I agree, and I think some of us might say that that might be quite nice, from the innovation perspective. Certainly we would encourage the development side of the organisation to try and be strategic and output-driven, rather than detailed and prescriptive, in its future thinking. We do suffer from the very detailed and prescriptive approach to what A-level English looks like, which does not help innovation across this table.

  Greg Watson: I think, to go right back to what I said at the start this afternoon, that managing a large and high-stakes qualification system means constant trade-offs between change in order to keep qualifications relevant, to respond to changes in the economy, and to adapt to the different needs of higher education versus stability, which is the thing that builds public confidence and familiarity. I think that a good regulator would constantly be balancing those two drivers. I think that QCA, because of the position it has occupied very close to Government, has tended to find that its role in being a sponsor of change has far outweighed, over time, its responsibility for stability. I hope that the one difference that we will see with the new regulator is a greater weighing of the pros and cons of change at any given moment, whether it is at a system-wide level in introducing a whole new qualification, in the form of Diplomas, or at an individual qualification level, in deciding whether to have more or less coursework in GCSEs. I think that that would be a very positive development.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2008
Prepared 13 May 2008