Examination of Witnesses (Questions 180
- 199)
MONDAY 21 JANUARY 2008
DR ANDREW
BIRD, MURRAY
BUTCHER, JERRY
JARVIS AND
GREG WATSON
Q180 Chairman: Before you move on,
Greg, can you tell us whether you share Andrew Bird's view? He
said in his introduction that he was worried about the regulator,
who might be a tick-boxing person with no strategic role at all.
You are very complacent about that.
Greg Watson: No, I think that
is absolutely the right move to make. There is a lot in the detail,
and we are all involved in a consultation process with the Department.
There are some important nuances about where the line should be
drawn between what remains under a Minister's direct control and
what is the responsibility of the independent regulator. I have
some views on that, and I guess that we do not have enough time
to explore them in detail here, but, yes, the fundamental point
in terms of getting the regulator right is to put the right responsibilities
and activities on the other side of that independence line so
that we get the building of confidence that we are looking for.
Chairman: We will come back to that.
Sharon is off to some important duty on the Front Bench, so I
call John Heppell.
Q181 Mr. Heppell: I am left feeling
that there is no proof that standards have dropped, but also that
there is no proof that they have been maintained. You said that
it was difficult to see what you could do about that, and I find
it difficult to see what measurement you could make, given the
change in the system. However, you may be right to suggest that
the regulator will ease some people's difficulties. But what about
the implications for the validity and reliability of the work
you all do in providing sample questions and answers and targeted
syllabus training for teachers, including comprehensive teaching
and learning materials about what exams will be about? Does that
not encourage people to get children to study those things that
are relevant to the test, rather than to the broader education
that all of us are looking for?
Dr. Bird: As you rightly said,
we all provide curriculum support materials, training for teachers
and such like. The publishers of books in support of our products
also include sample questions, tests and so forth. I do not detect
that that is very different from what it has ever been. Past exam
textbooks have always included past questions to help people prepare
effectively and it is surely better that people understand the
style of questioning that they are going to face than that they
do not. Our challenge in assessment terms is to make sure that
we do not become formulaic, that we cover, over time, the whole
curriculum— although we do not cover the whole curriculum
necessarily every time we set a particular test—so that
it does not become predictable and that it discourages people
from things like question-spotting, which is not a new phenomenon
either. I think teachers have always done an element of that.
What we are trying to do is provide the very best access for teachers
and the very best guidance and support, so that teaching experiences
in the classroom can be exciting and informative and can carry
students' interest forward, so that they are successful. We are
all interested in children being successful, but we separate that
very carefully; we do that through separate parts of the organisation.
The setting of the exam is an independent operation.
Chairman: John?
Mr. Heppell: I thought Greg wanted to
say something.
Greg Watson: Just to add to that—
Chairman: Greg, you have to catch my
eye.
Greg Watson: I forgot my flag
today. I have a couple of thoughts. The first is that I want young
people to learn to the syllabus. The syllabus is an important
definition of a programme and the whole point of having a syllabus
is to structure and motivate someone to want to learn. I do not
think that is any different for a professional exam in later life
or for a school exam. I think it is a good development that whereas
I never did see my A-level syllabus and wondered till the bitter
end what exactly it was I was heading towards, young people today
have a pretty good idea from the start. That has been healthy.
Secondly, I also want young people to demonstrate what they can
actually do, in the exam. By being a bit more open, a bit more
transparent, and providing a few more clues, we enable people
to feel well prepared, and what we are actually is assessing what
they can do not how successfully they have guessed what they are
about to do, or their ability to cope with the surprise of what
they have been faced with. I think that that has also been a positive
development. But I would set against that the fact that there
is a challenge and we employ expert people in the field of assessment
to make sure that we keep the level of challenge right, and that
it does not become too formulaic and too predictable, which obviously
would mean beginning to lose some of that effect.
Q182 Mr. Heppell: I want to add a
further thing about the idea that there is a sort of competition
to pick the easier course. We actually had someone in this Select
Committee telling us that that was common practice in schools;
that schools would look at the syllabuses and pick the easiest.
What about from the other side? What about universities that blacklist
qualifications because they say they have been made too easy?
Chairman: That is common practice, is
not it?
Mr. Heppell: Is it common practice?
Dr. Bird: I think some of the
universities have got a list of subjects that they would encourage
people not to do more than one of, in the sense that if you were
doing a number of these newer subjects—media studies and
something else, maybe—they feel that that might unbalance
your curriculum. I do not think they have actually blacklisted
A-levels. They have discouraged people from doing too many of
them.
Q183 Chairman: It is a bit of a mafia,
though, isn't it—all the senior tutors in charge of admissions
at Oxford and Cambridge get together on a regular basis. They
do not have to have it written down on a list.
Greg Watson: I think everyone
who—
Mr. Stuart: Greg is on the high table.
Greg Watson: Bearing in mind that
there are over 100 universities in this country and 50,000 different
degree courses, I am not surprised that different universities
and different departments in universities come to a particular
view on what is the most appropriate basis for being prepared
for their courses. It would be pretty extraordinary to imagine
that a maths degree would be available to someone who had not
done maths A-level. That is perfectly proper, and perhaps that
is a question for the universities to answer, rather than us.
Our job is to run A-levels to a common standard, right across
the subject range, and to make sure that the A-level is worth
the same, as a qualification, regardless of which subject you
do it in. What use universities subsequently make of it is down
to them trying to target the right kind of young people to get
on to the right kind of courses.
Q184 Mr. Heppell: I was thinking
about common qualifications. I understand that some universities
differentiate the courses and say, "We will take the OCR
one as one we accept but we won't accept another." Does that
happen? I thought it did.
Greg Watson: Not to a great degree
that I am aware of.
Jerry Jarvis: If it does, it must
be very limited. There are areas of the country where there are
affinities between certain universities and certain exam boards.
There are traditions, and there are preferences in the selection
process, but I cannot believe that it is widespread. It has not
come to us.
Q185 Chairman: What does Murray think?
One thing that comes through from reading all this stuff, as we
have got into the inquiry, is that even if a parent does not remember
their A-levels and GCSEs very well, they remember their degrees.
I was leafing through my examination results from when I was a
small person right through to my postgraduate time. I looked at
the year in which I got my degree from the London School of Economics,
and eight people got firsts throughout the whole LSE, in all subjects.
That does not happen now; the figure would be one third. It is
not your area to set degree papers, but surely you must be worried
about that. Does it not raise a big question about whether a first
is good as it used to be, if a third of people get firsts compared
with a small number not very many years ago?
Murray Butcher: By definition,
yes. You are either talking about an elite, or you are not. I
spent many years as an employer of degree-holding candidates,
and the university from which the degree is gained and the subject
always creates a choice for employers. Gosh, we are moving off
the subject.
Q186 Chairman: No, I think it is
very relevant, because what part of testing and assessment is
there to try to dispel this view—if it is wrong among parents—that
standards have got easier? I pitched to you the idea that in degrees
that it is the case, with so many people getting firsts compared
with lower and upper seconds.
Greg Watson: I honestly do not
think I could comment, not being involved in university-level
assessment.
Q187 Chairman: That is why I was
going to ask a second question. What I am getting from you is
that you do not really know because you do not have that much
of a link with universities. Surely, part of your job, and one
criticism of the examining boards, is that you do not any longer
have enough of a relationship with the teachers who teach the
subject, or with the people who take the products after they have
taken your examination—that you have become rather isolated,
both from the teachers below and from the university teachers
above. That surely cannot be good if that is what people are saying
about you.
Dr. Bird: Is that the evidence
you have had presented? I would certainly reject that on behalf
of AQA. We have active teachers and head teachers on our governing
council all the way through to people involved in our subject
committees. They are active teachers who act as senior examiners,
preparing and developing our material. From the classroom perspective,
we have got that active involvement with the chalk face—the
whiteboard face as it now more correctly is—and HE is represented
from admissions and senior academic perspectives on our governing
council.
Q188 Chairman: So most of your examiners
will be teachers, will they?
Dr. Bird: Yes.
Jerry Jarvis: All of them.
Q189 Chairman: All of them?
Jerry Jarvis: Decisions in all
our awarding bodies are taken essentially by practising senior
teachers—from the grading decisions to the writing of examination
papers and so on.
Greg Watson: I am actually quite
concerned about what you say, though, because we could not possibly
compete if we did not have an intimate relationship with people
who are ultimately our customers—the takers of our qualifications.
Q190 Chairman: Any hints that you
would knock that to one side?
Jerry Jarvis: Certainly, in the
development and creation of curriculum materials and in supporting
a teacher in delivery, I cannot subscribe to that view. However,
remember that we are required to demonstrate rigour, and part
of that demonstration necessitates distance. I could well understand
that when it comes to a disagreement over an examination outcome,
we have a role to play that is authoritarian, and we must maintain
some rigour and some distance from the process of learning.
Greg Watson: As awarding bodies,
we occupy a unique position. We sit in the middle of lots of different
stakeholders in the qualification system. On the one hand, we
have schools and colleges, teachers, young people and parents.
On another side, we have subject groups, subject associations,
leading thinkers in a subject and groups in universities that
do research on particular subjects such as the Salters group at
York. On the other hand, we have the Government of the day and
their political drives of various sorts, and, of course, we as
a charity also have our own education mission. We have a lot of
experience—150 years—and a lot of research to tap
into. They give us good clues about the right direction of travel.
Inevitably, sitting in that situation, we do not please everybody
all the time, but we do play an important role in drawing on all
the different views and trying to square the circle in a way that
I do not believe anybody else in the system does.
Chairman: Because I was following the
run of your answers, we have been stealing Lynda's questions.
We are over to Lynda now.
Q191 Lynda Waltho: I was actually
going to extend the question, although I wanted to deal in particular
with differentiation. What input do you get from employers? There
is a general feeling, certainly within the trade press—specific
evidence has not been given to this Committee—that there
is not sufficient input from academics and employers, particularly
in respect of the closed questions that make up a large part of
exams. They may be deskilling our school leavers, and not expanding
their analytical skills. What input do you get from employers,
who are receivers as well?
Greg Watson: A couple of comments.
First, there has been direct input from employers in respect of
many of our longest standing and most successful qualifications.
It is still the case that the single most heavily used qualification
in the history of this country is an adult IT qualification called
CLAiT, which was developed almost 20 years ago in direct response
to trends in the employment market and signals that were coming
from employers and further education colleges speaking on behalf
of employers. More recently, very successful new vocational programmes
called OCR Nationals have been rolled out to schools. We built
those—again, sitting in that unique position that I mentioned—listening,
on the one hand, to schools and colleges and what they hoped they
could offer to young people, and, on the other hand, to employers
and their views on what was relevant to modern employment. I think
that we do play that role. That said, I spoke in my opening statement
about the impact of the past 10 years and the growing use of qualifications
as a public policy tool—perhaps more change has been driven
directly by public policy. In my position sitting in the middle
of everybody, I would say that in the past 10 years we have probably
been drawn closer to satisfying the Government's direction of
travel, and the trade-off has been that we have given less time
than we would have done 10 years ago to consulting directly with
universities and employers. There are only so many hours in the
day, and only so much time available to develop new qualifications.
I hope that, with the arrival of an independent regulator, there
will be the opportunity to rebalance that slightly, because there
are times when I would like to be closer to employers and universities
in developing some of our qualifications.
Q192 Lynda Waltho: That is the general
view?
Jerry Jarvis: I would endorse
to a large extent what Greg said. Half of Edexcel's provision
is in the vocational workspace with BTECs. I believe half a million
students are doing them. Those qualifications would not work if
they had not been developed in conjunction with employers. They
are vocationally oriented qualifications for the marketplace.
The degree to which employers play a part in the setting of criteria
for what is taught at GCE and GCSE is open to question. Examination
organisations such as ours are required to create qualifications
that conform very closely to criteria set down by the regulator.
There is always a case for better collaboration with all stakeholders,
including parents, in the putting together of criteria. We are
all constrained by, and must work within, the criteria that are
set down but, certainly, when you move away from GCE and GCSE,
the degree of employer collaboration is massive.
Murray Butcher: My answers would
be from a vocational context. Employers can contribute in two
ways. First, they can contribute through their role on sector
skills councils, in which the creation of national occupational
standards takes place. They lead to the creation of national vocational
qualifications. Secondly, they can contribute through the provision
of supervision of that activity. The key person within vocational
qualifications is the external verifier. He or she works to a
particular awarding body and visits locations to ensure that practice
accords to standards. They may be practitioners from further education,
or people drawn straight from employment. There is considerable
opportunity for employers to play a role, but I concede that it
is quite difficult on occasions to gain such contact with employers
because, naturally, they see their principal activity as earning
their particular income from their role—finding time to
engage in education can be quite difficult for them.
Q193 Lynda Waltho: That finishes
that part, but I would like to extend the question and to look
at the differentiation of students for selection purposes. Certainly,
top universities argue that too many applicants attain the top
grades, which often leads to more testing. The Committee has heard
that high-stakes testing and the need for schools to show well
in league tables has resulted in teaching to test. What are your
views on that? Does the league table culture in schools distract
teachers from the task of preparing school leavers with the deep
knowledge and the independent analytical study skills that are
necessary for higher education and higher training?
Jerry Jarvis: Let me take the
first question on whether testing has an effect on the behaviour
in the classroom. There is huge pressure on schools and teachers
to increase performance. When we have one of our regular focus
groups, I sit with a panel of teachers and ask them to introduce
themselves. Almost without exception, they tell me their name,
what school they are from, the subject that they teach, and the
pass rate in their subject. Unfortunately, there are huge issues
at stake in most schools, and teachers are human. Having said
that, the huge overwhelming majority of teachers aim to deliver
on education—that aim comes across strongly in what they
do. However, there is no question that there is pressure. We are
talking about teaching to test. If we can write examination materials
that cover the whole syllabus in a way that means that people
cannot predict how something will be questioned or what the questions
will be about, it would be necessary for schools to teach the
whole syllabus. When I came here today, you very kindly issued
me with some instructions and notes about what was going to happen
today; you made me feel comfortable, and you gave me a brief outline
of the lines of questioning that I would be given. You were either
leading me and helping me to respond, or you were just trying
to set down the rules of engagement. Providing we stick to the
notion that we are talking about the rules of engagement, surely
we are right, but the pressure is massive.
Dr. Bird: I think the pressure
is massive. My concern as a former sixth-form governor is that
it is too much about output measures and not about added value.
There is so much in output measures that is to do with the inputs
that the students bring with them and less about what the school
adds through its teaching and learning processes. So there is
a place for more sophisticated league tables if they will be used
to assist schools in improving. As Jerry said, we aim to provide
a broad curriculum. We aim to test all of the student's skills.
We are a strong supporter of the introduction of A* at A-level.
We certainly believe that there is enough evidence in the current
distributions of marks that that could be done from the current
exam papers, if the Government were minded to do so. We feel that
it is a cause for celebration that more than 20,000 people are
getting three As. It is a very small proportion of the total candidate
group and it is a very small number of university departments
that have a problem with that. We appreciate that it is a problem,
but those departments that select have known for many years that
it is not just three A-levels that you need to be a good vet or
a good lawyer—you need a range of other things—and
they go out of their way to establish the other skills that good
sixth-form curriculums provide, such as extension work, project
work, community work and work experience. Those things can also
be certificated. We have a qualification that we are trying to
get approved that would support the bundling up of those activities.
Q194 Lynda Waltho: So it would be
a combination of those extended activities plus A*. What do your
colleagues think about the A*? I would be quite interested to
hear that.
Greg Watson: The A* is the right
development at the right time. Andrew has mentioned that the number
of candidates getting three As at A-level is small; we are talking
about 3% of the 18-year-old age group. So it is still a pretty
tall order to end up with three As. Nevertheless, there are now
enough young people in that category that certain universities
are saying that that achievement is not enough to differentiate.
As I said earlier, I think that we have made a bit of a trade-off
in the evolution of A-level, between broadening it as a general
qualification and maintaining it as the most stretching assessment
for getting into university. I think that reintroducing an element
of stretching in each subject and marking out an A* grade as a
higher bar for the best candidates to get over is a perfectly
sensible thing to do, and I know that a lot of universities will
welcome that development. I think that the big pressure in this
area is the GCSE A* to C measure. When Jerry is talking about
the teachers round the table, that is the thing that I know drives
a lot of institutions. That is probably the greatest risk of creating
any distorting incentive and of distorting things in two ways
potentially. One is that I think that if I were in school today,
I would rather be a D-grade candidate who was very close to the
C grade boundary than a D-grade candidate who was at risk of ending
up in the Es. There is always the danger that the last half an
hour of the teacher's attention will inevitably be drawn towards
the candidate who has a good chance of getting over the C grade
hurdle. The second issue is that, as the curriculum evolves and
we have new qualifications coming along, the pressure to find
new qualifications that can be treated as equivalent to a C grade
at GCSE can itself become a distorting incentive in the way that
those new qualifications are developed; you can see that from
much of the public debate about Diplomas and how they are going
to be treated. Attention could become concentrated on that issue,
rather than on whether it is a good curriculum innovation and
will be a good new offering for young people.
Lynda Waltho: That is fine. I think that
Dawn wanted to come in on Diplomas.
Q195 Ms Butler: I just wanted quickly
to touch on Diplomas, because I know that Graham is going to talk
a little about them; he thinks that I am stealing his thunder.
Do you think that the new Diplomas will help to bridge the gap
between school and higher education? We know that young people
who come from more affluent families already have this type of
mapping; going to school, then going on to higher education and
going to university. Will Diplomas help to bridge that gap?
Jerry Jarvis: One thing that the
Diploma may do is broaden the experience and learning of students
who enter university. They might have taken a much broader, different
curriculum, and perhaps one that was closer to practical learning
than previous syllabuses were. There is an opportunity there,
although a huge number of students enter university through BTEC
nationals and, by proportion, do very well in taking degrees.
I am interested in your identification of the gap.
Q196 Ms Butler: As I was saying,
we are trying to inject aspiration into the learning agenda. We
want to give those who might not have thought about going on to
higher education the opportunity to do so, as the BTEC does. The
question is partly how we can ensure that employers take Diplomas
seriously. Also, how can we ensure that a Diploma is equivalent
to three and a half A-levels?
Jerry Jarvis: We have to work
very hard to have that Diploma earn its spurs and get it the reputation
that it needs. It must cover a great deal of ground, and it must
be valued by everyone, not just those who would not necessarily
have considered themselves able to go into higher education. It
has the potential to offer a very different form of learning on
the way to either higher education or employment. The Diploma
is a big ask, and no subject is dearer to us right now than to
make it work, as the first teaching starts in September.
Chairman: We are deep into Diplomas now,
so I shall share the Diploma questions around. Fiona first.
Q197 Fiona Mactaggart: Greg, in your
evidence you quoted a delegate at a conference that you organised,
who said that the Diplomas were examination officers' worst nightmare—a
telling phrase, not usual for evidence to a House of Commons Select
Committee. You clearly feel strongly about this. What is nightmarish,
and what would you have done differently?
Greg Watson: Just by way of explanation
to members of the Committee—I am conscious that our sector,
like many, is full of all sorts of strange language—exams
officers are the administrative hub in a school or college, responsible
for making exam entries, ensuring that results are dished out
in time and so on. I mentioned Diplomas because they are the most
complicated qualification that I have ever seen. A typical learner
is likely to take part of their Diploma in a school and part in
a college, and they will do different elements of the Diploma
with different awarding bodies. The exams officer will be faced
with a brand new IT system built at great speed, and just about
in time, with which to administer all this, and any given exams
officer will probably have to work with an exams officer in at
least one other school or college to ensure that they use the
same reference number for a candidate and keep track of what a
candidate has done so far. They must also know exactly what the
candidate still needs to do to complete the qualification. Running
the exams office in a school or college is a tough job anyway.
It is often a temporary or part-time job, and has a churn rate
of about 50% of individuals in any given year. In many places
the exams officer struggles to have authority with the senior
management team in the school or college to get things changed.
We need to recognise the complexity of the qualification in its
own right, the practical logistics of how it will be taught and
the fact that it needs to be sorted out in pretty short order
for the first learners, who will be going through the programmes
in September 2008.
Q198 Fiona Mactaggart: That comment
is about the administration of it; let us consider the assessment
of it. Murray, most of your assessment experience is in the thing
that is worrying us in some ways about the Diplomas—whether
the person can do the extended portfolio. Do we have people who
can carry out that quality of assessment? Will it work?
Murray Butcher: At this stage,
a great number of questions remain on the form of assessment that
will take place within the various themes of activities, from
the principal learning and the additional specialist learning
to the extended project and the functional skills. We are likely
to find a range of assessment practices that will cover all of
those. Some will be internal, some external, some will be moderated
and some will be verified by the awarding body. Even though we
are planning to release the first teaching in September, there
is still quite a long way to go, and quite a lot of discussion
to have with the regulator to agree on the types of assessment
that will take place. I believe that quite a bit of responsibility
will fall on the schools and the delivery consortiums, and funding
is already in place to support that. It will be fairly heavily
pressurised in order to ensure that the teaching staff at schools
and colleges and even the local employer have the necessary skills.
It is a very big question.
Chairman: May I ask everyone to be quite
short? My colleagues are tending to ask all of you questions on
each section.
Dr. Bird: Very quickly, I agree
with all that Murray said. I would just add that none of those
means of assessment are new. We have experience of using them
in centres now. We have lots of experience in training teachers
to be assessors; we have done it for many years. It is a big ask
and a lot to do. That is why we are encouraged by the Government's
process of controlling volume in the first few years with Gateway
centres and so on and by the money that they are providing to
help with that verification and assessment development process.
Q199 Fiona Mactaggart: I was interested
in the bit of your evidence in which you said that coursework
had changed over time and that assessment used to be more embedded
in courses. You also said that the development of extended portfolios
had created greater opportunities for plagiarism, and that some
of the risks that people have suggested has occurred with coursework.
I hope that I have not misread you. I wonder how you assess the
extended project element of the Diploma and whether such an element
holds the same risk.
Dr. Bird: The extended project
is a piece of coursework that is chosen by the student and arises
out of their other studies. It is not a subject set by us, but
it is approved by us. It then requires a student to do a piece
of work and present some material. Clearly, that involves them
doing research and using the Internet and so on. By its very nature,
it is a unique piece of work that is based on a student's interests
and the A-levels that they are doing alongside the project. The
chance of copying great chunks of it, therefore, is massively
reduced. That was our point. Old-fashioned coursework embedded
in the curriculum arising out of a student's experience requires
higher quality assessment skills. However, because it is different
for every student, such work reduces the chances of plagiarism
being a serious problem. With coursework in general qualifications,
we have reduced its variety, standardised it and made it a task.
That task, therefore, is difficult to differentiate, which means
that you are looking at lots of very similar things. The chances
of people sharing that and copying are therefore increased because
it is the same piece of work. The thing that we find disappointing
about coursework in general qualifications is that just as a number
of us are launching the portfolio products that allow students
electronically to publish evidence of their work and have it assessed
remotely, which means you can provide photographs, audio clips,
short videos and texts, the system is closing coursework down
and making it a fairly repetitive and limited task. The technology
enables things to be checked, which, we think, would allow coursework
to flourish. We hope that that will be the case in the Diploma
environment.
Chairman: Very interesting.
| 
