UNCORRECTED TRANSCRIPT OF ORAL EVIDENCE To be
published as HC 169-iv
House of COMMONS
MINUTES OF EVIDENCE
TAKEN BEFORE THE
CHILDREN, SCHOOLS AND FAMILIES COMMITTEE
Testing and Assessment
Monday 21 January 2008
DR ANDREW BIRD, MURRAY BUTCHER, JERRY JARVIS and GREG
WATSON
Evidence heard in Public Questions 173 -
250
USE OF THE TRANSCRIPT
1.
|
This is an uncorrected
transcript of evidence taken in public and reported to the House. The
transcript has been placed on the internet on the authority of the Committee,
and copies have been made available by the Vote Office for the use of Members
and others.
|
2.
|
Any public use of, or reference
to, the contents should make clear that neither witnesses nor Members have
had the opportunity to correct the record. The transcript is not yet an
approved formal record of these proceedings.
|
3.
|
Members who receive this for the purpose of correcting questions
addressed by them to witnesses are asked to send corrections to the Committee
Assistant.
|
4.
|
Prospective witnesses may receive this in preparation for any
written or oral evidence they may in due course give to the Committee.
|
Oral Evidence
Taken before the Children, Schools and Families Committee
on Monday 21 January 2008
Members present:
Mr. Barry Sheerman (Chairman)
Annette Brooke
Ms Dawn Butler
Mr. David Chaytor
Mr. John Heppell
Mrs. Sharon Hodgson
Fiona Mactaggart
Mr. Graham Stuart
Lynda Waltho
Examination of Witnesses
Witnesses:
Dr. Andrew Bird, Deputy Director
General, Assessment and Qualifications Alliance, Murray Butcher, Director, Assessment and Quality, City &
Guilds, Jerry Jarvis, Managing
Director, Edexcel, and Greg Watson, Chief
Executive, Oxford, Cambridge and RSA Examinations, gave evidence.
Q173 Chairman:
May I welcome our witnesses to this session of the Children, Schools and
Families Committee? We are very pleased to have such a talented group of
experts with us this afternoon and we hope to learn a lot from them. As we have
at least six sections to cover, I hope that you will not mind if we cut a section
to move on to the next. We really could spend a couple of hours on each
section. Sometimes I will rather rudely say, "Quick questions and quick
answers." Do not get upset about that.
Will
you all introduce yourselves? We have your CVs, so there is no need for you to
say anything about them. Starting with
Andrew, have you any one thought that you would like to raise before we start
the questions and answers?
Dr.
Bird: I am Andrew Bird from the Assessment and Qualifications Alliance.
I take it that you are looking for an opening statement from us?
Chairman: It depends how long your
opening statement is.
Dr.
Bird: A minute.
Chairman: You can have a minute. Before
you came in, I was saying to my colleagues that you used to do a really useful
job in a fantastic chemical company in Huddersfield.
Dr.
Bird: That was a few years ago.
Chairman: They did not believe that I
was going to say that.
Andrew,
you are very welcome. Please give me your minute.
Dr.
Bird: First, AQA is an independent charity. The board of trustees is
drawn from the teaching profession, higher education and business. Our one
purpose is to deliver a good education. We aim to discharge that by giving high
quality qualifications in respect of teachers, parents, employers and HE, by delivering
new qualifications and modes of assessment that meet the needs of today's and
tomorrow's learners, by providing the best level of training support to
teachers who deliver our specifications, and by carrying out and publishing
research into educational assessment.
May
I draw your attention to a couple of points that we raised in our written
evidence, and then two that have arisen since? The first one, drawn from our
written evidence, concerns functional skills and hurdles for GCSE. The policy
intention is to impose a functional skills hurdle at level 2 of GCSE on
English, maths and ICT. On considering our research, we are concerned that when
it is introduced, it will de facto be a change of standard. From our modelling
work, it will suppress the pass rate for A to C at GCSE. The policy position is
that making such things explicit will lead to more discreet, direct teaching of
those skills and, hence, a rise in performance. That might be true, but we need
to consider the policy implications of that. We have no problem with raising
the standard. In fact, we think that that is a good idea, providing that we
understand it and we all know what will happen as a consequence of that at the
transition point.
Secondly, throughout our evidence, we are
quite keen on diversity of provision: giving the choice to teachers and
advisers to give students the widest range of curriculum opportunities. It is
important to remember that only a small number of our students do three
A-levels. Many do one or two A-levels, and they would find the diploma to be
too much at level 3. Diversity providers-meaning people such as ourselves-drive
competition in service delivery and support, which, we believe, helps
innovation. Evidence from contractual models suggests that, in so doing, the
Qualifications and Curriculum Authority drives out innovation.
I also want to raise two items that are
drawn from the more recent past. Clearly, we welcome the Government's intention
to separate the regulator. As we mentioned that in our written evidence, we
cannot do anything else at this stage. However, we want to draw two points to
your attention. One is the need for a
willingness to co-ordinate and integrate regulation across the three countries
of England, Wales and Northern Ireland.
There is a de facto single market in qualifications but, as you will be
aware, the policy position in those countries is diverging. Therefore, from a regulatory point of view,
there needs to be a bringing together of regulation.
We
are also concerned that as the shadow regulator-sorry, it is the interim
regulator at this stage, I am told. As
that process has got under way, the focus has seemed to be mainly on picking a
new name for it, rather than considering the technical capability and capacity
it requires to be an effective regulator.
We are concerned that the result will be a stifling, box-ticking
bureaucracy, rather than a strategic regulator of our activities.
Finally,
we think there is an emerging dilemma between two terms that we hear a lot from
the Government and regulator. This is
the whole high-stakes environment versus light-touch regulation. We obviously welcome appropriate and
sensible regulation that aligns to the five principles, but we see the
intention to extend the availability of qualifications from colleges and
workplaces and, in those situations, encourage light-touch regulation-which one
can understand to help people enter the market and to ensure that those
qualifications are acquired and certificated-as working against those qualifications
being portable and having utility. If
regulation is only light touch, it is in danger of not meeting the standard of
regulation that the high-stakes qualifications are put under. Qualifications need to command respect, and
not just from the initial provider of those qualifications. Thank you, Chairman.
Chairman: I let you get away with that
even though it took more than a minute.
Murray
Butcher: Good afternoon. Thank
you for inviting me. I am Murray
Butcher, director of assessment and quality at City & Guilds, which is the
UK's largest vocational awarding body.
Established in 1878, it received a royal charter from Queen Victoria in
1900. We provide about 500 vocational
qualifications in diverse occupational areas, ranging from agriculture to zoo
keeping. City & Guilds currently
comprises four qualification brands: the City & Guilds, which is the wide
range of vocational qualifications; the Institute of Leadership and Management,
which covers first-line management and beyond; the Hospitality Awarding Body,
which relates primarily to hotel and catering qualifications; and the National
Proficiency Tests Council, which covers all our land-based awards. We operate in the UK and internationally,
covering about 100 countries and working through about 8,000 centres. A centre can be a school, college,
university, training provider or employer.
I will seek to make any other points I can during the general
questioning.
Chairman: Fine.
Jerry
Jarvis: Good afternoon, ladies and gentlemen. I will be brief. I am
managing director of Edexcel, which is a Pearson company. We are known principally for our
technology. The only thing I would like
to say in an opening statement is how struck I was by what Ken Boston said in
his evidence on 17 December. He picked
up three key issues that he said were critical in improving attainment in this
country. First was the provision of
personalised learning; the second was the provision of continuous analytical
testing and evaluation; and the third was the professional training of
teachers. We strongly endorse that
view. We also believe that those three
factors are the issues that would most quickly improve attainment in this
country, and we have invested massively in the provision of a framework to do
that. Like my colleagues, I am very
pleased to participate in this inquiry and I look forward to the
recommendations and the outcome.
Chairman: Thank you.
Greg
Watson: Good afternoon. I am
Greg Watson, the chief executive of OCR-Oxford, Cambridge and RSA Examinations,
to give it its full title. We are a
major UK awarding body that principally makes qualification awards to 14 to
19-year-olds in schools and colleges.
We are exactly 150 years old and a not-for-profit social
enterprise. For a century and a half,
we have developed assessments of various types to structure, motivate and
reward learning. We are a member of the Cambridge Assessment Group, which is a
well known international education group that operates in more than 150
countries. In many of those countries, it offers assessments similar to the
style of assessment that we have here in the UK.
In
the past 10 years there have been three developments of note, which perhaps we
shall have an opportunity to explore in this inquiry. The first is a growing
use of qualifications as a public policy lever, and with that a widening of the
uses to which assessment is put beyond the original purpose of structuring,
motivating and rewarding learning. I am thinking of uses such as measuring
school performance. Secondly, there has been frequent change at both the
system-wide level and that of individual qualifications, and some
short-circuiting of long-established disciplines of evaluation and research
based on hard evidence. Thirdly, and connected with the previous two, there has
been an imperceptible but worrying loss of public confidence and a feeling that
somehow things are not quite what they used to be. That concern has become
harder to deal with because of the many uses to which assessment and
qualifications have been put and the difficulty of explaining and assessing the
impact of change.
We
very much welcome the Secretary of State's announcement before Christmas that a
new independent exams regulator will be created. We see in that a
once-in-a-generation opportunity to deal with the three issues that I have
mentioned and to put the exams system in a position of being seen to be
sufficiently independent while commanding public confidence, and for the
regulator to have a key role in balancing the desire to innovate and keep pace
with society with a desire to maintain stability and integrity over time.
Q174 Chairman:
Thank you very much for those openers.
May
I open the questioning by asking Jerry Jarvis something? I shall start with
him, as he is in the middle. People used to say that the trouble with our
examinations was that we had a number of boards, and that what we needed was
one big board that did everything. That would stop competition and prevent
people from switching from one examining board to another, and everything would
be a lot tidier if one board did the job that the four of you do. Is that not
an unanswerable proposition?
Jerry
Jarvis: Inevitably, I have a personal view. I spent a long career
outside education before coming into it, and I am used to competition being
used to drive up standards and reduce costs. My observation is that we benefit
massively from having competition in the marketplace. The huge majority of
teachers who choose the specifications of examination systems tell us that they
value the choice that they have. That choice certainly makes me compete
strongly with the colleagues who are sat beside me. Without it, we would not
have the degree of ingenuity, purpose and lead that we have in this country,
nor the stability and reliability. Competition has been very good for
education.
Q175 Chairman:
What if a teacher or head teacher said to you that the danger of competition
between examination boards is that everybody knows that if you are being pushed
and pushed to raise standards and raise the number of people getting grades A
to C and so on at GCSE, A-level and other levels, people go for the easiest
pass? Reputations go around, and people say, "It is easier to take English at
GCSE or A-level with that board," and they switch around. If you are really
going to compete, you will just become known as the easiest board from which to
get qualifications so that you can wipe out the other three.
Jerry
Jarvis: That is a popular view, but it is generally not held up by
fact. I am one of three accountable officers in this country. I am responsible
for ensuring that each award made by Edexcel is made under strict scrutiny and
that the standard is maintained across time and in comparison with other
awards. I do not have the ability to
interfere with that standard. If you
look at the appearance of so-called easy qualifications, the arguments tend to
break down when you get into some of the detail. For example, the pass rate in GCSE maths is higher than the pass
rate in media. Does that mean that
maths is easier than media? Because of
choice, these days, students will take the qualifications that they enjoy and
are good at. Ken Boston put it
eloquently when he drew the difference between the standard that is the hurdle
that students must achieve, and the standard that is expressed as the number of
students who have actually achieved that standard. We do not and cannot compete by producing easy
qualifications.
Q176 Chairman:
Why then, Greg, are so many people and parents out there, let alone the poor
old editor of the Daily Mail, unhappy
and feeling that standards have gone down and that kids do not work as hard or
get qualifications of the same standard as when they were at school? Why is there a general feeling that things
ain't what they used to be?
Greg
Watson: Let me offer two possibilities. First, qualifications have changed and evolved. The A-levels that young people sit today are
not the same as those I sat, with good reason.
The need for the routine replaying of a large body of knowledge has
probably weakened slightly as access to information has become easier. On the other hand, industry says that it
wants people who are more skilled in using that information-in applying it and
being able to think for themselves. In
A-level, we have seen a shift over time so that the body of knowledge in any
given subject is probably a bit smaller, but the skills needed to apply that
knowledge have moved to a slightly higher level of demand. Some of the commentary is simply an
unfamiliarity with how the qualifications have changed-they do not feel the
same.
Secondly,
I think that there is a misunderstanding about the nature of competition among
the people sat at this table. Ours is a
competition of not standards, but ideas.
Because we are all independent organisations, because we are all close
to the business of teaching and learning, and because we find ourselves between
schools and colleges on one hand and universities and employers on the other, I
think that we feel driven to look for new approaches. Look at what has happened with GCSE science recently. There has been a real rejuvenation of
science in the classroom because of a particularly innovative programme that we
at OCR have developed in partnership with the Nuffield Foundation. Look at what is happening with geography at
the moment. We are running a
groundbreaking pilot with a different approach to geography that reconnects the
concept of geography with a study of the real world. That helps young people to make more sense of the subject. One reason why we have a 150-year tradition
in this country that so many other countries overseas want to buy into is that
we have had the power to innovate and a competition of ideas and subjects. They have helped us to keep subjects fresh
and interesting and to offer different approaches to different young people who
want to learn in different styles.
Q177 Chairman: Andrew, you are a scientist by training, are
you not? Why do people say that people
are shifting to what are perceived as easier subjects? We are still having difficulty in attracting
enough people to carry on with maths, physics and the sciences in general. Even geography seems to be losing students,
despite the new course that Greg Watson has just described to us. With what you are providing, are you not
colluding to stimulate movement away from the hard scientific subjects to
subjects that are perceived to be easier?
Dr.
Bird: Perception is everything, is it not? We are trying to reflect those things that students want to study
that are relevant to commerce and work today.
Media studies qualifications meet a student need, and teachers feel that
students would enjoy learning it.
Through it, students can collect important basic study skills and skills
for future employment. Is that easier
than science? I did French and science
at school and found the former incredibly hard. Was the French exam easier than the science exam? It was much harder for me, because of my
ability. People find what they enjoy
easier, so I found maths quite straightforward, whereas other people find it
very difficult.
A
lot of this is about perception. We
work extraordinarily hard to ensure that the level of demand among subjects is
maintained over time, and we do that by using experts in the classroom and
expert examiners-people who are knowledgeable about their subject. We cannot undo the perception of, "Well, it
is not what I was taught at school"- nor should it be, because times, demand
and needs have moved on-"and I cannot connect with or understand it, so I do
not appreciate that it is as difficult as, say, physics."
Chairman: Right, I have warmed you
up. Murray, you will have to wait for a
moment, because people will get testy if I carry on. Now we will drill down a bit, and John Heppell will lead us.
Mr. Heppell: Sharon can go first
because she has to leave.
Chairman: Sorry, Sharon, you are next.
They have switched; they have done a secret deal.
Q178 Mrs.
Hodgson: I want to ask a couple of
quick questions and then go, because I am due in the Chamber on Bench duty.
I want to talk about evidence that we
received from Professor Dylan Wiliam, who argues that although A-levels have
not necessarily become easier, examinations no longer measure what they used
to. From that, he infers that a pupil
achieving a top grade does not necessarily have the same skills as a pupil who
achieved a top grade years ago. How are
the gatekeepers to further and higher education, and employers, to compare
students in similar subjects, but from different years, given the changes in
qualifications?
Jerry
Jarvis: Our examination system
is complicated and driven by populism.
It is actually very difficult to compare an A-level taken in 2007 with
one taken pre-Curriculum 2000. The
structure is different, and we are examining different things. Access to A-level education was different
some 10, 15 and 20 years ago, so the cohort taking those examinations was also
different. However, we can perhaps see
a continuing thread through the regulator's work in attempting to maintain a
standard in A-levels over the years.
Truly speaking, however, we can compare precisely only A-levels that
were taken since the introduction of Curriculum 2000.
I
shall return to the mantra that I am sure that you will hear time and again
when speaking to anyone from an examination board or awarding body: we have
attempted to maintain the hurdle at the same height, even though the features
that we are examining are different.
What has changed quite dramatically is access. There is far more choice, so, for example, students can take a
number of AS-level examinations and continue the AS-level studies that they are
best at. There is multiple access to
resits, modular variants, and so on, which have increased the probability of
students attaining that same fixed standard.
There is a very difficult notion to get across, so it is easy to say,
"When I did A-levels, they were much harder."
Our students work very hard for A-levels today, and something in the
region of only 3% of 18-year-olds achieve three A grades.
Q179 Mrs. Hodgson:
If you are saying that standards have been maintained and the hurdle is still
at the same height, how can you counter the claims by some universities that
school leavers entering the first year do not have the same depth of knowledge
that students with the same grades had years ago? The universities are saying that.
Greg
Watson: I think that you have to bear in mind that the role of A-levels
in the education system has changed over time.
There was a time when A-levels were purely for those entering higher
education and they were actually offered to a pretty small part of the 17 and
18-year-old age group. A-levels moved
over time to become the standard school-leaving qualification, in many ways,
and that will be even more the case if the rate of those staying on to 18
continues to rise. As A-levels have
evolved, there has inevitably been a trade-off between ensuring that the
qualification is suitably motivating and providing the right structure for
learning for a wide range of young people, and making sure that it is a good
basis for university entrance. I would
recognise that, in the drive to widen the use of A-levels, we have lost a
little, and that is why we have come back and started to look at the stretching
of the upper end of A-levels to make sure that we reintroduce a little more
stretch for the most able exam candidates and give some of universities more of
an ability to see who the most able are.
We
should also draw a distinction between the year-on-year reliability of
standards and the long progress of the history of standards. Year on year, all of us at this table go to
great lengths-in fact, the QCA's independent review last year said that we go
to greater lengths in this country than anywhere-to guard standards in our
subjects. What Jerry said was
right. Over the decades, we have seen a
number of structural changes, which have all been there for a good reason, but
we have lacked an independent assessment of the impact of those changes. As I said in my opening remarks, I see it as
a positive development that the Secretary of State wants to put the regulator
some distance from the Government of the day and Ministers. One role that the regulator will be able to
play will be to look at change in the system, to evaluate the impact that any
change might have on standards and public perceptions, and to see whether we
are happy to make a trade-off for the benefit that we get from making that
change. That will be very welcome.
Chairman: Sharon, have you
finished?
Mrs. Hodgson: Yes. That is great. Thank you.
Q180 Chairman:
Before you move on, Greg, can you tell us whether you share Andrew Bird's
view? He said in his introduction that
he was worried about the regulator, who might be a tick-boxing person with no
strategic role at all. You are very
complacent about that.
Greg
Watson: No, I think that is absolutely the right move to make. There is a lot in the detail, and we are all
involved in a consultation process with the Department. There are some important nuances about where
the line should be drawn between what remains under a Minister's direct control
and what is the responsibility of the independent regulator. I have some views on that, and I guess that
we do not have enough time to explore them in detail here, but, yes, the
fundamental point in terms of getting the regulator right is to put the right
responsibilities and activities on the other side of that independence line so
that we get the building of confidence that we are looking for.
Chairman: We will come back to
that. Sharon is off to some important
duty on the Front Bench, so I call John Heppell.
Q181 Mr.
Heppell: I am left feeling that there is no proof that standards have
dropped, but also that there is no proof that they have been maintained. You said that it was difficult to see what
you could do about that, and I find it difficult to see what measurement you
could make, given the change in the system.
However, you may be right to suggest that the regulator will ease some
people's difficulties. But what about
the implications for the validity and reliability of the work you all do in
providing sample questions and answers and targeted syllabus training for
teachers, including comprehensive teaching and learning materials about what
exams will be about? Does that not
encourage people to get children to study those things that are relevant to the
test, rather than to the broader education that all of us are looking for?
Dr.
Bird: As you rightly said, we all provide curriculum support materials,
training for teachers and such like.
The publishers of books in support of our products also include sample
questions, tests and so forth. I do not
detect that that is very different from what it has ever been. Past exam textbooks have always included
past questions to help people prepare effectively and it is surely better that
people understand the style of questioning that they are going to face than
that they do not.
Our
challenge in assessment terms is to make sure that we do not become formulaic,
that we cover, over time, the whole curriculum- although we do not cover the
whole curriculum necessarily every time we set a particular test-so that it
does not become predictable and that it discourages people from things like
question-spotting, which is not a new phenomenon either. I think teachers have
always done an element of that.
What
we are trying to do is provide the very best access for teachers and the very
best guidance and support, so that teaching experiences in the classroom can be
exciting and informative and can carry students' interest forward, so that they
are successful. We are all interested
in children being successful, but we separate that very carefully; we do that
through separate parts of the organisation.
The setting of the exam is an independent operation.
Chairman: John?
Mr. Heppell: I thought Greg wanted to
say something.
Greg
Watson: Just to add to that-
Chairman: Greg, you have to catch my
eye.
Greg
Watson: I forgot my flag today.
I
have a couple of thoughts. The first is
that I want young people to learn to the syllabus. The syllabus is an important definition of a programme and the
whole point of having a syllabus is to structure and motivate someone to want
to learn. I do not think that is any
different for a professional exam in later life or for a school exam. I think it is a good development that
whereas I never did see my A-level syllabus and wondered till the bitter end
what exactly it was I was heading towards, young people today have a pretty
good idea from the start. That has been
healthy.
Secondly,
I also want young people to demonstrate what they can actually do, in the
exam. By being a bit more open, a bit
more transparent, and providing a few more clues, we enable people to feel well
prepared, and what we are actually is assessing what they can do not how
successfully they have guessed what they are about to do, or their ability to
cope with the surprise of what they have been faced with. I think that that has also been a positive
development. But I would set against
that the fact that there is a challenge and we employ expert people in the
field of assessment to make sure that we keep the level of challenge right, and
that it does not become too formulaic and too predictable, which obviously
would mean beginning to lose some of that effect.
Q182 Mr.
Heppell: I want to add a further thing about the idea that there is
a sort of competition to pick the easier course. We actually had someone in this Select
Committee telling us that that was common practice in schools; that schools
would look at the syllabuses and pick the easiest. What about from the other side?
What about universities that blacklist qualifications because they say
they have been made too easy?
Chairman: That is common practice, is not it?
Mr. Heppell: Is it common practice?
Dr.
Bird: I think some of the universities have got a list of subjects that
they would encourage people not to do more than one of, in the sense that if
you were doing a number of these newer subjects-media studies and something
else, maybe-they feel that that might unbalance your curriculum. I do not think they have actually
blacklisted A-levels. They have
discouraged people from doing too many of them.
Q183 Chairman:
It is a bit of a mafia, though, isn't it-all the senior tutors in charge of
admissions at Oxford and Cambridge get together on a regular basis. They do not have to have it written down on
a list.
Greg
Watson: I think everyone who-
Mr. Stuart: Greg is on the high table.
Greg
Watson: Bearing in mind that there are over 100 universities in this
country and 50,000 different degree courses, I am not surprised that different
universities and different departments in universities come to a particular
view on what is the most appropriate basis for being prepared for their
courses. It would be pretty extraordinary to imagine that a maths degree would
be available to someone who had not done maths A-level. That is perfectly proper, and perhaps that
is a question for the universities to answer, rather than us.
Our
job is to run A-levels to a common standard, right across the subject range,
and to make sure that the A-level is worth the same, as a qualification,
regardless of which subject you do it in. What use universities subsequently
make of it is down to them trying to target the right kind of young people to
get on to the right kind of courses.
Q184 Mr.
Heppell: I was thinking about common qualifications. I understand that some
universities differentiate the courses and say, "We will take the OCR one as
one we accept but we won't accept another."
Does that happen? I thought it
did.
Greg
Watson: Not to a great degree that I am aware of.
Jerry
Jarvis: If it does, it must be very limited. There are areas of the country where there are affinities between
certain universities and certain exam boards.
There are traditions, and there are preferences in the selection
process, but I cannot believe that it is widespread. It has not come to us.
Q185 Chairman:
What does Murray think? One thing that
comes through from reading all this stuff, as we have got into the inquiry, is
that even if a parent does not remember their A-levels and GCSEs very well,
they remember their degrees. I was
leafing through my examination results from when I was a small person right
through to my postgraduate time. I
looked at the year in which I got my degree from the London School Of
Economics, and eight people got firsts throughout the whole LSE, in all
subjects. That does not happen now; the
figure would be one third. It is not
your area to set degree papers, but surely you must be worried about that. Does it not raise a big question about
whether a first is good as it used to be, if a third of people get firsts
compared with a small number not very many years ago?
Murray
Butcher: By definition, yes.
You are either talking about an elite, or you are not. I spent many years as an employer of
degree-holding candidates, and the university from which the degree is gained
and the subject always creates a choice for employers. Gosh, we are moving off the subject.
Q186 Chairman:
No, I think it is very relevant, because what part of testing and assessment is
there to try to dispel this view-if it is wrong among parents-that standards
have got easier? I pitched to you the
idea that in degrees that it is the case, with so many people getting firsts
compared with lower and upper seconds.
Greg
Watson: I honestly do not think I could comment, not being involved in
university-level assessment.
Q187 Chairman:
That is why I was going to ask a second question. What I am getting from you is that you do not really know because
you do not have that much of a link with universities. Surely, part of your job, and one criticism
of the examining boards, is that you do not any longer have enough of a
relationship with the teachers who teach the subject, or with the people who
take the products after they have taken your examination-that you have become
rather isolated, both from the teachers below and from the university teachers
above. That surely cannot be good if
that is what people are saying about you.
Dr.
Bird: Is that the evidence you have had presented? I would certainly reject that on behalf of
AQA. We have active teachers and head
teachers on our governing council all the way through to people involved in our
subject-to-subject committees. They are
active teachers who act as senior examiners, preparing and developing our
material. From the classroom
perspective, we have got that active involvement with the chalk face-the
whiteboard face as it now more correctly is-and HE is represented from
admissions and senior academic perspectives on our governing council.
Q188 Chairman:
So most of your examiners will be teachers, will they?
Dr.
Bird: Yes.
Jerry
Jarvis: All of them.
Q189 Chairman:
All of them?
Jerry
Jarvis: Decisions in all our awarding bodies are taken essentially by
practising senior teachers-from the grading decisions to the writing of
examination papers and so on.
Greg
Watson: I am actually quite concerned about what you say, though,
because we could not possibly compete if we did not have an intimate
relationship with people who are ultimately our customers-the takers of our
qualifications.
Q190 Chairman:
Any hints that you would knock that to one side?
Jerry
Jarvis: Certainly, in the development and creation of curriculum
materials and in supporting a teacher in delivery, I cannot subscribe to that
view. However, remember that we are
required to demonstrate rigour, and part of that demonstration necessitates
distance. I could well understand that
when it comes to a disagreement over an examination outcome, we have a role to
play that is authoritarian, and we must maintain some rigour and some distance
from the process of learning.
Greg
Watson: As awarding bodies, we occupy a unique position. We sit in the middle of lots of different
stakeholders in the qualification system.
On the one hand, we have schools and colleges, teachers, young people
and parents. On another side, we have
subject groups, subject associations, leading thinkers in a subject and groups
in universities that do research on particular subjects such as the Salters
group at York. On the other hand, we
have the Government of the day and their political drives of various sorts,
and, of course, we as a charity also have our own education mission. We have a lot of experience-150 years-and a
lot of research to tap into. They give
us good clues about the right direction of travel. Inevitably, sitting in that situation, we do not please everybody
all the time, but we do play an important role in drawing on all the different
views and trying to square the circle in a way that I do not believe anybody
else in the system does.
Chairman: Because I was following the
run of your answers, we have been stealing Linda's questions. We are over to Linda now.
Q191 Lynda
Waltho: I was actually going to extend the question, although I wanted to
deal in particular with differentiation.
What input do you get from employers?
There is a general feeling, certainly within the trade press-specific
evidence has not been given to this Committee-that there is not sufficient
input from academics and employers, particularly in respect of the closed
questions that make up a large part of exams.
They may be deskilling our school leavers, and not expanding their
analytical skills. What input do you
get from employers, who are receivers as well?
Greg
Watson: A couple of comments.
First, there has been direct input from employers in respect of many of
our longest standing and most successful qualifications. It is still the case that the single most
heavily used qualification in the history of this country is an adult IT qualification
called CLAiT, which was developed almost 20 years ago in direct response to
trends in the employment market and signals that were coming from employers and
further education colleges speaking on behalf of employers. More recently, very successful new
vocational programmes called OCR nationals have been rolled out to
schools. We built those-again, sitting
in that unique position that I mentioned-listening, on the one hand, to schools
and colleges and what they hoped they could offer to young people, and, on the
other hand, to employers and their views on what was relevant to modern
employment. I think that we do play
that role.
That
said, I spoke in my opening statement about the impact of the past 10 years and
the growing use of qualifications as a public policy tool-perhaps more change
has been driven directly by public policy.
In my position sitting in the middle of everybody, I would say that in
the past 10 years we have probably been drawn closer to satisfying the
Government's direction of travel, and the trade-off has been that we have given
less time than we would have done 10 years ago to consulting directly with
universities and employers. There are
only so many hours in the day, and only so much time available to develop new
qualifications. I hope that, with the
arrival of an independent regulator, there will be the opportunity to rebalance
that slightly, because there are times when I would like to be closer to
employers and universities in developing some of our qualifications.
Q192 Lynda
Waltho: That is the general view?
Jerry
Jarvis: I would endorse to a large extent what Greg said. Half of Edexcel's provision is in the
vocational workspace with BTECs. I
believe half a million students are doing them. Those qualifications would not work if they had not been
developed in conjunction with employers.
They are vocationally oriented qualifications for the marketplace.
The
degree to which employers play a part in the setting of criteria for what is
taught at GCE and GCSE is open to question.
Examination organisations such as ours are required to create
qualifications that conform very closely to criteria set down by the
regulator. There is always a case for
better collaboration with all stakeholders, including parents, in the putting
together of criteria. We are all
constrained by, and must work within, the criteria that are set down but,
certainly, when you move away from GCE and GCSE, the degree of employer
collaboration is massive.
Murray
Butcher: My answers would be from a vocational context. Employers can contribute in two ways. First, they can contribute through their
role on sector skills councils, in which the creation of national occupational
standards takes place. They lead to the
creation of national vocational qualifications. Secondly, they can contribute through the provision of
supervision of that activity. The key
person within vocational qualifications is the external verifier. He or she works to a particular awarding
body and visits locations to ensure that practice accords to standards. They may be practitioners from further
education, or people drawn straight from employment. There is considerable opportunity for employers to play a role,
but I concede that it is quite difficult on occasions to gain such contact with
employers because, naturally, they see their principal activity as earning
their particular income from their role-finding time to engage in education can
be quite difficult for them.
Q193 Lynda
Waltho: That finishes that part, but I would like to extend the question
and to look at the differentiation of students for selection purposes. Certainly, top universities argue that too
many applicants attain the top grades, which often leads to more testing. The Committee has heard that high-stakes
testing and the need for schools to show well in league tables has resulted in
teaching to test. What are your views
on that? Does the league table culture
in schools distract teachers from the task of preparing school leavers with the
deep knowledge and the independent analytical study skills that are necessary
for higher education and higher training?
Jerry
Jarvis: Let me take the first question on whether testing has an effect
on the behaviour in the classroom.
There is huge pressure on schools and teachers to increase
performance. When we have one of our
regular focus groups, I sit with a panel of teachers and ask them to introduce
themselves. Almost without exception,
they tell me their name, what school they are from, the subject that they
teach, and the pass rate in their subject.
Unfortunately, there are huge issues at stake in most schools, and
teachers are human. Having said that,
the huge overwhelming majority of teachers aim to deliver on education-that aim
comes across strongly in what they do.
However, there is no question that there is pressure.
We
are talking about teaching to test. If
we can write examination materials that cover the whole syllabus in a way that
means that people cannot predict how something will be questioned or what the
questions will be about, it would be necessary for schools to teach the whole
syllabus. When I came here today, you
very kindly issued me with some instructions and notes about what was going to
happen today; you made me feel comfortable, and you gave me a brief outline of
the lines of questioning that I would be given. You were either leading me and helping me to respond, or you were
just trying to set down the rules of engagement. Providing we stick to the notion that we are talking about the
rules of engagement, surely we are right, but the pressure is massive.
Dr.
Bird: I think the pressure is massive.
My concern as a former sixth-form governor is that it is too much about
output measures and not about added value.
There is so much in output measures that is to do with the inputs that
the students bring with them and less about what the school adds through its
teaching and learning processes. So
there is a place for more sophisticated league tables if they will be used to
assist schools in improving.
As
Jerry said, we aim to provide a broad curriculum. We aim to test all of the student's skills. We are a strong supporter of the
introduction of A* at A-level. We
certainly believe that there is enough evidence in the current distributions of
marks that that could be done from the current exam papers, if the Government
were minded to do so.
We
feel that it is a cause for celebration that more than 20,000 people are
getting three As. It is a very small
proportion of the total candidate group and it is a very small number of
university departments that have a problem with that. We appreciate that it is a problem, but those departments that
select have known for many years that it is not just three A-levels that you
need to be a good vet or a good lawyer-you need a range of other things-and
they go out of their way to establish the other skills that good sixth-form
curriculums provide, such as extension work, project work, community work and
work experience. Those things can also
be certificated. We have a
qualification that we are trying to get approved that would support the
bundling up of those activities.
Q194 Lynda
Waltho: So it would be a combination of those extended activities plus
A*. What do your colleagues think about
the A*? I would be quite interested to
hear that.
Greg
Watson: The A* is the right development at the right time. Andrew has mentioned that the number of
candidates getting three As at A-level is small; we are talking about 3% of the
18-year-old age group. So it is still a
pretty tall order to end up with three As.
Nevertheless, there are now enough young people in that category that
certain universities are saying that that achievement is not enough to
differentiate. As I said earlier, I
think that we have made a bit of a trade-off in the evolution of A-level,
between broadening it as a general qualification and maintaining it as the most
stretching assessment for getting into university. I think that reintroducing an element of stretching in each
subject and marking out an A* grade as a higher bar for the best candidates to
get over is a perfectly sensible thing to do, and I know that a lot of
universities will welcome that development.
I
think that the big pressure in this area is the GCSE A* to C measure. When Jerry is talking about the teachers
round the table, that is the thing that I know drives a lot of
institutions. That is probably the
greatest risk of creating any distorting incentive and of distorting things in
two ways potentially. One is that I think
that if I were in school today, I would rather be a D-grade candidate who was
very close to the C grade boundary than a D-grade candidate who was at risk of
ending up in the Es. There is always
the danger that the last half an hour of the teacher's attention will
inevitably be drawn towards the candidate who has a good chance of getting over
the C grade hurdle. The second issue is
that, as the curriculum evolves and we have new qualifications coming along,
the pressure to find new qualifications that can be treated as equivalent to a
C grade at GCSE can itself become a distorting incentive in the way that those
new qualifications are developed; you can see that from much of the public
debate about diplomas and how they are going to be treated. Attention could become concentrated on that
issue, rather than on whether it is a good curriculum innovation and will be a
good new offering for young people.
Lynda Waltho: That is fine. I think that Dawn wanted to come in on
diplomas.
Q195 Ms
Butler: I just wanted quickly to touch on diplomas, because I know that
Graham is going to talk a little about them; he thinks that I am stealing his
thunder. Do you think that the new
diplomas will help to bridge the gap between school and higher education? We know that young people who come from more
affluent families already have this type of mapping; going to school, then
going on to higher education and going to university. Will diplomas help to bridge that gap?
Jerry
Jarvis: One thing that the diploma may do is broaden the experience and
learning of students who enter university. They might have taken a much
broader, different curriculum, and perhaps one that was closer to practical
learning than previous syllabuses were. There is an opportunity there, although
a huge number of students enter university through BTEC nationals and, by
proportion, do very well in taking degrees. I am interested in your
identification of the gap.
Q196 Ms
Butler: As I was saying, we are trying to inject aspiration into the
learning agenda. We want to give those who might not have thought about going
on to higher education the opportunity to do so, as the BTEC does. The question
is partly how we can ensure that employers take diplomas seriously. Also, how
can we ensure that a diploma is equivalent to three and a half A-levels?
Jerry
Jarvis: We have to work very hard to have that diploma earn its spurs
and get it the reputation that it needs. It must cover a great deal of ground,
and it must be valued by everyone, not just those who would not necessarily
have considered themselves able to go into higher education. It has the
potential to offer a very different form of learning on the way to either
higher education or employment. The diploma is a big ask, and no subject is
dearer to us right now than to make it work, as the first teaching starts in
September.
Chairman: We are deep into diplomas
now, so I shall share the diploma questions around. Fiona first.
Q197 Fiona
Mactaggart: Greg, in your evidence you quoted a delegate at a conference
that you organised, who said that the diplomas were examination officers' worst
nightmare-a telling phrase, not usual for evidence to a House of Commons Select
Committee. You clearly feel strongly about this. What is nightmarish, and what
would you have done differently?
Greg
Watson: Just by way of explanation to members of the Committee-I am
conscious that our sector, like many, is full of all sorts of strange
language-exams officers are the administrative hub in a school or college,
responsible for making exam entries, ensuring that results are dished out in
time and so on. I mentioned diplomas because they are the most complicated
qualification that I have ever seen. A typical learner is likely to take part
of their diploma in a school and part in a college, and they will do different
elements of the diploma with different awarding bodies. The exams officer will
be faced with a brand new IT system built at great speed, and just about in
time, with which to administer all this, and any given exams officer will probably
have to work with an exams officer in at least one other school or college to
ensure that they use the same reference number for a candidate and keep track
of what a candidate has done so far. They must also know exactly what the
candidate still needs to do to complete the qualification.
Running
the exams office in a school or college is a tough job anyway. It is often a
temporary or part-time job, and has a churn rate of about 50% of individuals in
any given year. In many places the exams officer struggles to have authority
with the senior management team in the school or college to get things changed.
We need to recognise the complexity of the qualification in its own right, the
practical logistics of how it will be taught and the fact that it needs to be
sorted out in pretty short order for the first learners, who will be going
through the programmes in September 2008.
Q198 Fiona
Mactaggart: That comment is about the administration of it; let us consider
the assessment of it. Murray, most of your assessment experience is in the
thing that is worrying us in some ways about the diplomas-whether the person
can do the extended portfolio. Do we have people who can carry out that quality
of assessment? Will it work?
Murray
Butcher: At this stage, a great number of questions remain on the form
of assessment that will take place within the various themes of activities,
from the principal learning and the additional specialist learning to the
extended project and the functional skills. We are likely to find a range of
assessment practices that will cover all of those. Some will be internal, some
external, some will be moderated and some will be verified by the awarding
body. Even though we are planning to release the first teaching in September,
there is still quite a long way to go, and quite a lot of discussion to have
with the regulator to agree on the types of assessment that will take place.
I believe that quite a bit of
responsibility will fall on the schools and the delivery consortiums, and
funding is already in place to support that. It will be fairly heavily
pressurised in order to ensure that the teaching staff at schools and colleges
and even the local employer have the necessary skills. It is a very big
question.
Chairman: May I ask everyone to be
quite short? My colleagues are tending to ask all of you questions on each
section.
Dr.
Bird: Very quickly, I agree with all that Murray said. I would just add
that none of those means of assessment are new. We have experience of using
them in centres now. We have lots of experience in training teachers to be
assessors; we have done it for many years. It is a big ask and a lot to do.
That is why we are encouraged by the Government's process of controlling volume
in the first few years with gateway centres and so on and by the money that
they are providing to help with that verification and assessment development
process.
Q199 Fiona
Mactaggart: I was interested in the bit of your evidence in which you said
that coursework had changed over time and that assessment used to be more
embedded in courses. You also said that the development of extended portfolios
had created greater opportunities for plagiarism, and that some of the risks
that people have suggested has occurred with coursework. I hope that I have not
misread you. I wonder how you assess the extended project element of the
diploma and whether such an element holds the same risk.
Dr.
Bird: The extended project is a piece of coursework that is chosen by
the student and arises out of their other studies. It is not a subject set by
us, but it is approved by us. It then requires a student to do a piece of work
and present some material. Clearly, that involves them doing research and using
the internet and so on. By its very nature, it is a unique piece of work that
is based on a student's interests and the A-levels that they are doing
alongside the project. The chance of copying great chunks of it, therefore, is
massively reduced. That was our point. Old-fashioned coursework embedded in the
curriculum arising out of a student's experience requires higher quality
assessment skills. However, because it is different for every student, such
work reduces the chances of plagiarism being a serious problem. With coursework
in general qualifications, we have reduced its variety, standardised it and
made it a task. That task, therefore, is difficult to differentiate, which
means that you are looking at lots of very similar things. The chances of
people sharing that and copying are therefore increased because it is the same
piece of work. The thing that we find disappointing about coursework in general
qualifications is that just as a number of us are launching the portfolio
products that allow students electronically to publish evidence of their work
and have it assessed remotely, which means you can provide photographs, audio
clips, short videos and texts, the system is closing coursework down and making
it a fairly repetitive and limited task. The technology enables things to be
checked, which, we think, would allow coursework to flourish. We hope that that
will be the case in the diploma environment.
Chairman: Very interesting.
Q200 Fiona
Mactaggart: I have heard that administration is hard, and that we need
particular skills to assess the can-do bits of the coursework of the diplomas.
Do you think they are going to work? A
yes or no answer will be sufficient.
Dr.
Bird: Yes.
Murray
Butcher: Yes, we are working very hard to ensure that.
Jerry
Jarvis: I would rather come back with the statement that they have to
work. We are making a huge investment,
and we have made a very bold decision.
We all feel exactly the same way.
We all suffer from trepidation in a whole series of different areas, but
we have to make this diploma work.
Greg
Watson: I will say yes, with one important proviso. I return to an earlier question about
whether employers and universities will really feel that the diplomas are worth
while. The answer is that they will see
the proof with their own eyes, because they will get to know young people who
have been through these programmes.
Whether the diplomas will succeed in the early years will be more about
how they are taught, and then about the kind of young people that they produce. Regardless of all the technicalities of
assessment-and there will be issues that we have to iron out over time-if the
diplomas are really to succeed in the first two or three years, it will be all
about the teaching. That means that
support for teaching has to be exemplary and that we should be sensible about
the number of students that we want to see on those programmes early on. As Andrew said, it has been helpful that the
Government have put some criteria in place.
If the numbers do not grow too fast, and we devote an extraordinary
effort-not just the Government, but people like us-to supporting teachers in
the first few years of teaching, diplomas will succeed.
Chairman: Murray, may I ask you to be
one of the first respondents to Graham?
Q201 Mr.
Stuart: Fiona has given you a pretty good work through. We ended there on a positive note that, "We
are going to make it work because we've got no choice." City & Guilds has the greatest
experience of vocational training; may I put it to you that there has been
insufficient time for component awarding bodies to research and develop new
assessment approaches? Are diplomas
being introduced to a political, rather than educational, timetable?
Murray
Butcher: Timetable, perhaps. In
terms of trying to draw-
Q202 Mr.
Stuart: Is that a yes or a no?
Would any one of you, or anyone whom you know in the educational
establishment, have gone for this timetable if it had not been dictated by the
Government?
Murray
Butcher: I think I would need to collaborate with my colleagues on
either side to reflect on how some of the Curriculum 2000 qualifications were
introduced. That was also an extremely
short time scale, but it was achieved.
It is possible that we can achieve this, but some very concentrated
thought is needed. It is, in historical
terms, more of a political timetable than a curriculum development one-that is
true.
Q203 Mr.
Stuart: Yes; it comes in in a few months, and you say that there is a lack
of clarity over the purpose of the qualification and its underpinning
curriculum.
Murray
Butcher: Yes-
Q204 Mr.
Stuart: That is pretty damning is it not?
Murray
Butcher: There are considerable concerns-if I can explain. As a vocational awarding body, we see this
as an opportunity to introduce a hint of vocationalism into the school and
post-school curriculum. We carry
considerable anxieties that, as we go through that process, the pressure on the
award may limit the amount of vocational experience available and that the
award may drift back to the standard, general qualifications-much as we believe
GNVQs did. My anxiety is that if it is successful,
this qualification will also be a challenge for higher education, because I
believe that it will draw in a slightly wider cohort than has been experienced
up to now. There is a lot of pressure
on HE to increase its intake, and it must be borne in mind that the nature of
the intake, as well as the skills and approach of young people going into
higher education, may change. There
will need to be a response to that.
Q205 Mr.
Stuart: The advanced diploma is supposed to be worth 420 UCAS points, whereas
three As at A-level are worth 360 points.
Is that realistic? Will it carry
public confidence?
Murray
Butcher: I do not see why it should not. We are devising a new qualification that is trying to attract
young people with a range of skills. I
would not wish to compare the skills of this young person to that young
person. If they are successful in the
award, they deserve the three or three and a half A-levels that they achieve.
Q206 Mr.
Stuart: People have had theories over the years and they have always come
up with conceptually excellent ideas that, like GNVQs, end up not working. The two biggest challenges, according to The Guardian, are university endorsement
and parental endorsement. Is not the
real challenge in getting the new qualification going that you have to get
enough good pupils and good teaching to get it off the ground? The tendency is that a qualification comes
in and it is the poorest cohorts at the poorest establishment who are often the
first adopters. Is that not a
danger? How can we prevent it from
happening, so that the diploma can flourish?
Jerry
Jarvis: You are absolutely right.
I made the point earlier that the reputation of this qualification will
be critical to its success. The
programme of introduction is too fast.
There is too much involved in this programme, and too many other
parallel changes are going on at the same time. We would not have done it this way, but we have made a
commitment. We have had our arguments
with our regulator, with the Department, with each other and inside our own
organisations, and we have made a commitment, so there is a point at which we
have to say, "We wouldn't have started from here, but we have to make this
work." So much is at stake. The ultimate success of the qualification-
Q207 Mr. Stuart:
What if we had had you here when we were talking about GNVQs? We can look at every previous initiative,
because we have always recognised we do not have the right system for
vocational education. If we had had
your equivalents in the past year, you would have all said the same thing,
wouldn't you-about the need to make it work?
And they have not worked. I am
trying to tease out just how serious the problems are with the lack of clarity
over the purpose, the timetable, and the risk of undermining GCSEs and the AQA
coming in.
Greg
Watson: I am not sure I necessarily agree with the statement that GNVQs
did not work in the end. Through a
process of iteration over about 10 years, GNVQs became confidently used in some
schools and colleges, but it took 10 years.
When the Education and Skills Committee looked into diplomas-we offered
evidence about the speed-there was, I think, recognition in the Committee's
report that going at this speed does not help and putting pressure on numbers
early on does not help.
I
mentioned OCR Nationals earlier. When
we were developing those, we probably took a year longer than we took over
diplomas. That gave us a year to
triangulate more effectively between what employers wanted and what schools and
colleges thought they wanted, and to look hard at assessment structures and
ensure that grading arrangements and the like were going to work well. I think we would have all wished for a
little more time.
On
a positive note, there has been an excellent dialogue between ourselves, the
Department and the QCA about the first phase of diplomas and how that process
works. I think everybody has recognised
that it was very loaded towards the employer view and we did not take on board
enough about teaching and pedagogy and did not, early on, spend enough time on
the business of assessment and standards and ensuring that the outcome of the
diplomas would command confidence.
There is a commitment all round, among all the people I have just talked
about, to go about the second phase in a rather different way. That in itself shows that we are perhaps
recycling the learning faster than we might done with the GNVQ.
Chairman: There is a bit of pressure on
time. We have to move on to coursework.
Q208 Annette
Brooke: You have already hinted that there are quite a lot of advantages in
using coursework in the examinations system.
Given the rather dramatic moves, particularly with the GCSE, is this a
question of throwing the baby out with the bathwater?
Dr.
Bird: In my early remarks about coursework and the extended project, I
was, I suppose, broadly hinting in that direction. Very few people in their daily lives do not discuss with
colleagues an issue they are trying to tackle, or use the internet or any of
those wider research skills. Clearly,
properly constructed coursework questions allow people to demonstrate, in
preparing a document for moderation, those wider study skills and extended
communication skills, which it is very hard to assess in a quite short, formal
examination.
Q209 Annette
Brooke: I shall ask another question and perhaps somebody else can pick it
up. Is it simply that you guys have not
been smart enough in setting the coursework?
Chairman: Who is that question to?
Annette Brooke: I am looking for volunteers. I though that it would be a bit unkind to
pick on somebody.
Chairman: Jerry can go first, followed by Greg.
Jerry
Jarvis: I think that you have
two volunteers here.
It
is probably a case of throwing the baby out with the bathwater. It is not that difficult to detect
plagiarism. After all, the same
mechanism that allowed the student to find the material to plagiarise makes it
just as easy for the moderator to find it.
We believe that coursework makes a hugely valuable contribution to the
way in which we conduct assessments. It
is a far richer method than an end-of-term examination. Last year, Edexcel did a huge amount of work
on detecting cheating throughout the UK.
We examined all sorts of issues, but detecting cheating in coursework is
easy, actually.
Greg
Watson: Yes, I am disappointed
with our direction of travel. Hopefully, some of what we learn from the
extended project within the diploma will flow back into coursework. Ironically, the extended project could prove
to be one of the most exciting parts of the diploma development. It could be a genuinely personal and
independent project that would build evaluation, research, thinking and
self-organisation skills, which are exactly the sort of skills that coursework-done
well-has the power to develop.
Coursework
is not right for every subject, and in some ways we have suffered from having
veered from one side of the road to the other.
At one time, we decided that coursework was a good thing and, through
over-mighty regulation, we were compelled to put coursework into subjects such
as maths, where it does not sit readily.
There has been a sudden change of tack and now we are compelled to take
coursework out, even when, in subjects such as geography or history, it can
breed fundamental skills that, incidentally, carry on very well into higher
education, too.
Murray
Butcher: I was just reflecting
on some good, simple assessment reasons for maintaining coursework. We are trying to get as broad a picture as
we can of individuals' abilities. If
you take out coursework, you are focusing on just one or two other forms of
assessment, which gives you a biased picture of the individual. I share the view that by withdrawing
coursework, we would be going in the wrong direction.
Q210 Annette
Brooke: On gender balance, I have
the impression that girls started performing rather well in some GCSE
coursework. Is that statistically
true? I suppose that I should look to
the awarding bodies for that information, but I thought that you might have the
gender breakdown of results with and without coursework.
Jerry
Jarvis: We could provide it
but, anecdotally, it is generally the case that girls do better in coursework,
as well as the other subjects, although that varies enormously in coursework
subjects. However, yes, they are better
at coursework.
Q211 Annette
Brooke: I find that
interesting. I wonder whether getting
rid of coursework would solve one of the Government's problems by potentially
closing the gender gap-I accept that that is rather cynical.
Something
that concerns us greatly-this might be in line with our questions and other
lines of inquiry on social mobility-is that young people from different
backgrounds and with differing levels of family support clearly will have
different opportunities within their coursework. For instance, I recall that the Prime Minister's wife could get
help with certain coursework, and I used to take my children to museums and so
on. How do you factor in those things
for a true assessment?
Chairman: You cannot, can you? Is
it fair? Is it biased towards
middle-class kids? Do less privileged
kids find it harder to achieve?
Greg
Watson: That is very hard to
iron out. However, returning to
something that a couple of us have said, the notion of a truly personal piece
of learning could get over that problem, because it would be in the bounds of
the imagination and creativity of the young person and their teacher. I can think of good examples in GCSE
business studies, in which coursework is open-ended. A student goes to a local business, works up a project on one
aspect of the business, and writes it up.
A teacher who encourages young people to think imaginatively, or a
school that maintains good links with local employers, is able to do that, regardless
of any advantage or disadvantage in the home background. Part of what creates the difficulty is the
tendency, through regulation, to drive coursework towards being a standardised
task. If anything, those other
advantages kick in to a greater degree when that is the case.
Chairman: David, do you want to come in
on coursework?
Q212 Mr.
Chaytor: Yes. Specifically, what
are the most effective means of preventing plagiarism in the first place? I take the point about making coursework
more personalised, as against standardised, but how do you prevent someone from
accessing the internet or from getting a disproportionate advantage from having
well-informed parents?
Jerry
Jarvis: The view that we have taken is that the best way to prevent
plagiarism is the likelihood that your efforts will be detected-it is like the
idea that you will probably drive slower if you suspect that there is a traffic
camera around. The best way to do that
is to sample students' work. As I said
earlier, moderators and examiners can use the same methods to access
non-original material as the students did.
These days, even with material that is being shared among students,
which might not be generally available, we have deployment technology and we
digitally scan everything that we do, so we have the ability to have a machine
literally looking for similarities that could not otherwise be detected without
the deep reading of work. For me, the
way to discourage plagiarism is through the probability that you will be
caught.
Chairman: Andrew, do you want to come
in on that?
Dr.
Bird: I agree with some of that.
I think the real way of inhibiting plagiarism is to make sure that you
have set the right task. If we set a
task that is basically a knowledge task, we are encouraging somebody to go to
the internet and find out all they know, so we need to set a task that requires
them to acquire some knowledge, from wherever they get it, and then to do
something with it-to process it and convert it into something that is about
them and their insight into that knowledge.
The task needs to be about underlying ability, rather than a recounting
of what they can find in a book or on an internet site. Tasks that require people to give empathy of
analysis are the sort of coursework tasks that we ought to be setting. That, in my view, is the strongest way of
avoiding plagiarism, followed by the sense that there is a speed camera around
the corner.
Q213 Mr.
Chaytor: City & Guilds is particularly keen on the role of coursework,
and local assessment of coursework is more frequent in vocational
qualifications than in more academic qualifications. What are your views on all that? Do different standards apply to courses that are defined as
vocational, as opposed to those that are defined as academic?
Murray
Butcher: There are similarities and differences. Within a national vocational qualification
course, you are dealing with the fact that the curriculum contains certain
criteria. When you come to assess what,
in vocational qualifications, is called the portfolio, which is a collection of
evidence, you are testing that evidence against certain specified criteria and
looking for the supporting evidence within it.
That does not mean to say that there are not occasions when you find
that individual students and trainees come up with extremely similar
evidence. Supported by regulation, we
must carry an investigations team within City & Guilds so that whenever
questions occur that might suggest plagiarism-or not so much plagiarism as the
possibility that the centres providing the documentation are not encouraging
individuals to do things themselves-we need to explore that and confirm whether
something untoward has taken place. So,
in a sense, plagiarism is possible in NVQs, and we have to make sure that we
eliminate it.
Q214 Mr.
Chaytor: Broadly, what proportion of awards are rejected because plagiarism
has been identified by City & Guilds and the other boards?
Murray
Butcher: Within City & Guilds, it is going to be an extremely small
percentage-less than 1%.
Greg Watson: For OCR, 192 candidates
in 2007 had sanctions applied for plagiarism.
Q215 Mr.
Chaytor: As a percentage of the total?
Greg
Watson: Minute.
Q216 Mr.
Chaytor: Broadly, on the use of the internet in coursework and to support
learning for external examinations, is there any evidence that the digital
divide between families is leading to a widening gap in the achievement of
young people? Is access to the internet
a factor that drives up standards very quickly for the most affluent families,
leaving the children of poorer families behind? Is anyone doing any research on this, or do you have any thoughts
about it?
Greg
Watson: That is actually a very difficult question to answer from where
we are sat. We assess what is put
before us. Given that most of it
arrives in a digital format, you could say, prima facie, that there is no
obvious divide, but I cannot gauge-I am not sure that I even want to-who has
used what means to get there. We simply
mark it all to the same standard when we are presented with it. There is an interesting research subject
there, but I cannot comment from the evidence that we see.
Jerry
Jarvis: Following on from an earlier question about setting an
examination to ensure equality of access, particularly for disadvantaged
children, we do quite a lot in setting all our examination and assessment work
on the more obvious things, such as language and culture. We would not want to set a paper in
geography in which someone who was a non-UK resident would be disadvantaged, so
we think very carefully about that. We
would not require someone to do a field trip to France, so we can make sure
that when we set our assessment instruments, we do not. However, I am afraid that one or other of
the population will be inadvertently disadvantaged. Geography comes in as well, with remote villages and so on where
access to appropriate research facilities is limited. We have to be careful
but, inevitably, some students have greater advantages than others.
Q217 Mr.
Chaytor: May I move on to the QCA?
All four boards agree that the changes to the QCA are welcome. That is right, is it not?
Dr.
Bird indicated assent.
Greg
Watson indicated assent.
Jerry
Jarvis indicated assent.
Murray
Butcher indicated assent.
Q218 Mr.
Chaytor: But what has it done wrong?
Where has it gone wrong?
Chairman: And why were you not lobbying
for the change before?
Greg
Watson: I was.
Chairman: Oh, you were. Okay.
Dr.
Bird: We were as well. The QCA
has not done an awful lot wrong, but having the two activities together can
generate a conflict of interest, and separating curriculum development from the
regulation of assessment can only be a good thing. The regulators spent a lot of time thinking about the development
of the new A-levels, but never mentioned what it would cost to assess those
various models of assessment. Sometime
later on, QCA quite properly wanted to explore our pricing strategy for those
A-levels-after we had been in development for some time and the die had been
cast. That is the very worst set of
arrangements, because we are already committed to a style of assessment in
association with the curriculum development side of QCA at a time when the
regulatory side started talking about pricing, which is influenced by what it
costs to deliver the style of assessment. Separation of the two roles should lead to a more robust dialogue
between the regulatory side and the development side so that we triangulate the
issues more effectively, because the consequence of development requirements on
us changes what we can deliver and what it might cost to put it into the
marketplace. One can see a more robust
triangular relationship across the industry, which is why we favour it.
Q219 Mr.
Chaytor: The example that you have given could equally be used as an
argument to scrap the curriculum development agency function completely and
devolve it to each of the four boards.
Dr.
Bird: It could be-
Mr. Chaytor: There is no guarantee that
a separate development agency will also not overlook the practical
implementation costs of the financial arrangements
Dr.
Bird: I agree, and I think some of us might say that that might be
quite nice, from the innovation perspective.
Certainly we would encourage the development side of the organisation to
try and be strategic and output-driven, rather than detailed and prescriptive,
in its future thinking. We do suffer
from the very detailed and prescriptive approach to what A-level English looks
like, which does not help innovation across this table.
Greg
Watson: I think, to go right back to what I said at the start this
afternoon, that managing a large and high-stakes qualification system means
constant trade-offs between change in order to keep qualifications relevant, to
respond to changes in the economy, and to adapt to the different needs of higher
education versus stability, which is the thing that builds public confidence
and familiarity. I think that a good
regulator would constantly be balancing those two drivers. I think that QCA, because of the position it
has occupied very close to Government, has tended to find that its role in
being a sponsor of change has far outweighed, over time, its responsibility for
stability.
I
hope that the one difference that we will see with the new regulator is a
greater weighing of the pros and cons of change at any given moment, whether it
is at a system-wide level in introducing a whole new qualification, in the form
of diplomas, or at an individual qualification level, in deciding whether to
have more or less coursework in GCSEs.
I think that that would be a very positive development.
Q220 Mr.
Chaytor: Greg, in your submission,
you were critical of the QCA for intervening in too heavy-handed a way, whereas
in the AQA submission there were accusations of not regulating strongly enough
in respect of the current basic and key skills tests. It said that the problem is that the regulation is weak. Where is the balance? Is it that the QCA has been too
interventionist, or has it been too hands-off?
Each of you says different things.
Dr.
Bird: I think we were saying that it was conflicted because it was the
provider of the tests. It was not a
matter of being too hands-off; it was the regulator of itself. That was our point. Those tests are derived-we only deliver
them-by QCA itself, so it sets the standard.
Greg
Watson: I would say that it has been a case of too much in some areas
and too little in others-too much intervention at the detailed level, getting
in the way, I think, of producing more stimulating approaches to geography,
more relevant vocational offerings, but on the other hand too little attention
to the long-run effects of a series of changes piling up. If you were to look at A-level over time and
take modularisation, the move to six units, the balancing of coursework, the
splitting of AS and A2 into two lots of 50%, compounded with various tweaks and
twists applied along the way; if the regulator is looking after public
confidence it should be less concerned with any one of those changes in any one
subject and more with looking at the long-term effect over time on actual
standards and on public perceptions of standards, and therefore, with measuring
the rate of that sort of change in a more moderate way.
Murray
Butcher: The opportunity for the independent regulator may mean that
QCA, or whatever it becomes known as, may be able to focus on regulation and
refine that process of regulation, which I would say, from City & Guilds
experience, has been delivered in fits and starts. There has been urgent
attention to this activity, which then suddenly dissipates; then it comes over
here somewhere, and gets distracted by some of its curriculum activities. I really look to this as an opportunity to
have stable regulation. The one
principal issue that we still have, which Andrew mentioned in his opening statement,
is the fact that we have four regulators to deal with, and that is a big
bureaucratic problem.
Q221 Mr.
Stuart: Following on naturally from that in terms of maintaining standards,
should it be an important part of the new regulator's job to look at comparisons
internationally? Sir Peter Lampl said,
of standards, that "the key question is are they improving fast enough to
maintain or improve our position relative to other European countries, America
as well as China and India?" Do you agree with that?
Jerry
Jarvis indicated assent.
Greg
Watson: Any sensible regulator
in this field, with a primary focus on standards, will want to base its
decisions on evidence. That seems a
strangely obvious thing to point out, but it is a point worth making. There is good evidence abroad. As I mentioned, we are part of a group
operating in 150 countries. It is
partly through the process of synthesising research from experience in different
countries that we can draw some of the conclusions that we can about the impact
of change on standards and so on. I
would have thought that any sensible regulator would want to build
relationships around the world with all sorts of research communities.
Q222 Mr.
Stuart: In the context of today's
evidence, what do you make of this country's tumbling down the international
league tables that came out recently?
Chairman: Do not roam too widely, but comments would be appreciated. Are we tumbling? Is that your fault? Is it
just a rotten examination board system causing us to tumble down the
international league tables?
Jerry
Jarvis: First, that is a very
difficult question to answer here. We
should also be very careful about what we compare. When Leach looked at the various standards in different
countries, one of the conclusions that we started to come to was that we
probably do as much training, but that we do not certificate it in the same
way, so we do not count it. Counting
eggs and counting eggs is an important issue.
We certainly share a massive international business with OCR, and this
education system still has huge respect abroad. We cannot be complacent but it can be very dangerous to build any
sort of policy on an international comparison because of the sheer difference
of literally comparing the same thing.
Chairman: We must move on to the last
section.
Q223 Mr.
Stuart: May I just deal with the key stage and testing at different times?
Edexcel's evidence was that that will increase the testing burden and that the
key stage test should be at the end of the key stage, if retained. Do you think that if we are to have tests
when the pupil is ready rather than at the end of the stage, we need to scrap
key stage tests as such?
Jerry
Jarvis: Forgive me, Barry, I need to make a slightly longer response to
this question, because it is something I feel strongly about. We constantly accuse ourselves of
over-testing, but summative, quantitative testing needs to be done to ensure
that the investment that we put in, and the teaching and learning that is
happening, happens. Let me return to
what Ken Boston said-three things: personalised learning,
continuous-essentially, internal-assessment and training of teachers. The reason why I care so much about it is
that after about £80 million of investment, we can do it now, and if we really
were interested in shifting that stubborn set of five good A to C grades, we
could do it now. It is here, and Ken
Boston was absolutely right. So, do we
over-test? Yes, in certain areas, but
we do not spend enough time on personal, continuous improvement, which is the
key to improvement.
Chairman: Okay. We must move on, and I want to cover the
last section. We have the duo of Fiona
and David covering it, with Dawn throwing one matter in if she wants.
Q224 Fiona
Mactaggart: We have had quite a bit of evidence about the narrowing of the
curriculum and teaching to the test, as you would expect, and in a way, some of
your evidence is specifically about that.
I am very interested, for example, in your analysis, Andrew, of why
aptitude tests are not an appropriate alternative because you can train young
people to succeed in them-a fact I know very well, representing a town that still uses the 11-plus. Children
of 11 perform less well on Key Stage 2 assessments, I think, because they are
training for their 11-plus aptitude tests. I think that I am hearing from you
that in some ways it is a good thing if people teach to the test, because then
we get better results. However, all of us as educators think it is a bad thing,
because the domain that young people learn is narrower.
Dr.
Bird: I think that we all said teach to the curriculum, rather than
teach to the test. We want people to cover the whole curriculum and know, as
Jerry said, the experience that they will have when they are examined, so that
it is not a shock and they can show their best when they are being tested. That
is about understanding the style of the examination that they will be required
to sit. Our job is to ensure that the tests maintain sufficient variety and
coverage of the curriculum over time, so that it is not easy to teach to the
test in the narrow sense in which I think you mean it-saying "I am going to
predict the six questions that will come up next summer, and teach you seven
topics. I guarantee that six will come up." You are leading us towards a very
narrow, output-driven thought. We certainly want to give people every
encouragement to cover the actual curriculum, which is why we make
specifications available, as Greg said. When I was doing A-levels, I had no
idea what the curriculum was, which cannot have been a good thing.
Greg
Watson: If I may add to that, we have talked quite a lot about
innovation, and it is a real benefit to have more than one awarding body. The
competition of ideas has been a powerful driver to keep syllabuses interesting
and make subjects stimulating. In the era before micro-regulation, which really
started in 1998, there were outstanding partnerships in a whole range of
subjects-science, geography, maths-between individual awarding bodies and very
creative university departments in places such as York and Cambridge. That was
true also between groups of forward-thinking teachers-I can think of the
Suffolk science movement. That interplay of ideas put a range of options out
there for young people and their teachers. A school would pick its syllabus to
suit what kind of school it was and the young people that it had.
We
are beginning to get some of that back. There have been signs that we have
moved back in the right direction, and if we get the form of regulation right
and it moves from the micro level to the level of looking after the system,
there is scope for the interplay of different ideas among organisations such as
ourselves. All the evidence from the work that we have done when we have had a bit
more freedom to operate, and been less under QCA diktat, is that we have been
able to reignite interest in some subjects. We are doing it again with A-level
history, in which we have gone back to an alternative, more research-based
approach, sitting alongside what we might think of as a more traditional style.
Everything that we hear from the chalk face is that some people learn the
skills of history much better by going out and touching it with their hands
than by simply learning a load of facts from a textbook.
Jerry
Jarvis: Teaching to the test is a perennial issue for us. When I talk
to the Specialist Schools and Academies Trust and say, "Help me to avoid being
cast in the role of someone who is encouraging teaching to the test," it slaps
me on the back and says, "Get on with it. That is exactly what we are supposed
to do. We are supposed to set a standard, teach the syllabus and assess whether
students have met that standard." I do not think that, with the enormous stakes
that are placed on league tables, you can avoid the accusation that teachers
are spending time on examination preparation. It is inevitable, and it was
always there. A long, long time ago, when I was doing qualifications, I
remember practising on past papers and wishing I had a father who knew more
about maths than my friends' fathers did. What has changed is the pressure to
succeed. It really defines careers inside schools and colleges, and it is
something that we have to guard against.
The
thing that drives me along is that if we can use technology and all the
techniques that we have to make exam preparation relatively simpler, we can
spend much more time in enriching education.
Why are people educated? Is it
for entry to university? Is it in
preparation for work? Or is it for the
sheer joy of learning? I seem to
remember that there was a lot of that in the past.
Q225 Fiona
Mactaggart: Absolutely, Jerry, yet your organisation is producing
ResultsPlus, which, as I understand it, will be able to show me, as a teacher,
that, fine, I might be wonderful at teaching Jane Austen but when it comes to
Shakespeare my students are much more wobbly than Barry's students are, and
that I have to shape up on that bit of the curriculum. Do you think that there is a risk that
ResultsPlus will actually encourage that sort of thing?
Jerry
Jarvis: What I have had back from a large number of teachers is that we
are moving towards true personalised learning.
Because such information is available, they can spend less time on
revision, which becomes far more to the point.
Teachers are able to go at the speed of each different student, and they
have more time to indulge in the richer aspects of delivering the
syllabus. In fact, in our experience,
the process actually works the other way around.
Q226 Fiona
Mactaggart: You make it sound as if it is the child who will be the level
of reflection that a teacher will make.
Is it not more likely that it will be the teacher who will be the level
of reflection in this sort of a system?
Jerry
Jarvis: Having deployed the technology, it is clear that it operates at
many different levels. As a tool for
the senior management team inside a school or college, it pinpoints exactly how
well the school or college is delivering its curriculum across all the
subjects. For an individual teacher in
the classroom, we are seeing a jump in the performance of their students. Teachers are literally taking the
information and using it to their advantage.
Q227 Chairman:
But you do not all agree with that, do you?
Some of you are not using the technology.
Greg
Watson: We are looking at similar ideas at the moment.
Q228 Chairman:
So you approve?
Greg
Watson: This is nothing new, in fact.
Any teacher who wants to do the best for the young people in their
classroom will reflect on how they are doing, and will compare how they did
this year with last year. For some
time, teachers have been able to get hold of exam answers and have a look at
them to see how well they did.
For
many years, many teachers have come to professional development events-INSET
events-that we lay on. Some involve a
general run across the syllabus, but some are targeted at particular
areas. If we get feedback from teachers
that they are struggling with the coursework element, we always lay on support
for them to come and talk about coursework and perhaps learn from other
teachers. I think that the desire to
understand, to diagnose, to target a bit of effort to develop professionally
has been with teachers for a long time.
We have started to explore whether technology and some of the data that
we have could add to that.
Q229 Chairman:
There are no dangers?
Dr.
Bird: There is the issue about releasing detailed results directly to
students. We think that that can be
mediated through the relationship that students have with the centre and with
their teachers rather than giving results to them directly, if that is what you
mean by richer information being available.
We certainly think that teachers can be assisted by having richer
feedback about how the cohort of students is performing so that they can modify
their teaching methods. It is obviously
better if they get that from formative assessments that they carry out in the
classroom rather than from summative results for the cohort that has just gone,
as those students already have their qualifications, whether good or bad. Many people's support materials for
classroom teachers contain formative tests in the form of homework, which
essentially have the teacher ask, "Did I get that algebra across correctly or
not?" That must be a helpful process,
to assist teachers and curriculum heads in understanding how different teachers
are performing in different parts of the curriculum, so that they can play to
their strengths and fill in where there might be weaknesses, because people are
not universally good at doing everything in the curriculum.
Q230 Fiona
Mactaggart: When you design questions for examinations, which of course is
a mystery to most people-we can only imagine the types of questions-you test
all sorts of questions and design them to find out if they show specific
knowledge or a range of skills, and such like.
When you do that, do you think about the skills or knowledge that
reveals something beyond the ordinary?
We talked earlier about creating an A* grade, for example. Clearly that must have implications for the
design of questions, to create the space for a young person to show that
ability. None of your evidence tells us
how that is done.
Chairman: One of you answer that
question, or we will be here all night.
Greg
Watson: It is a very careful and very rigorous process; a lot of what
you suppose to be true is true. In
writing a syllabus, we are not simply writing down, "Oh, read this book and you
will know all about the second world war."
We will actually unpack that process to say, "By the time someone has
successfully completed a GCSE course, they should know the following things and
they should be able to do the following things with that knowledge." We will then describe a series of levels,
which we call grades, that will differentiate the level of knowledge that you
would be likely to have and what skills you would be able to apply to that
knowledge.
Interestingly,
in the context of A* we have been doing a research project in advance of
introducing A*, because we have been through the process of redescribing the
syllabus. We have now described to
ourselves, "What does an A* grade historian have by way of knowledge and skill
that you would not have expected of an A grade student?" We are now trying out questions at the
moment for exactly that reason; we want to discover what types of questions
unleash that potential and what types of questions are sufficiently open-ended
to provide that extra stretch and also, importantly, what types of questions
successfully differentiate the most able students from the rest and do not lead
them all to being clustered around the middle of the mark range.
So
that is the type of work that we are doing.
We have a great army of research people sitting behind everything that we
do and that is what they do all day and every day.
Q231 Chairman:
So you want to take over the world, do you not? You do not want the Qualifications and Curriculum Authority or a
national curriculum. You want to set
the whole thing really, because coming through is the idea that you can do it
all. Your exams drive the
curriculum.
Greg
Watson: I think that we have very important expertise. We are capable of adding more value in terms
of innovation and enriching and discovering new approaches to subjects.
Q232 Chairman:
Is it not dangerous to let you take over the world of education?
Greg
Watson: There is something missing in my equation, which is why we need
a regulator, and it is that the public's stake and society's stake in the
qualifications system is much bigger now than it has ever been. Because of that, the public want to know
that young people's education is safe in our hands and that we are doing what
we should be doing absolutely properly.
It is quite right that somebody who is not sitting along this line of
witnesses here should have an independent view and should be tracking
effectiveness.
Q233 Chairman:
So you want a weak regulator to defend you from public criticism?
Greg
Watson: I want a strategic regulator who can really stand up on the
basis of hard evidence and support us when we say that we are looking after
standards and everybody is getting a fair deal. There should be an independent body that is not a vested
interest, which we might be seen as being, and that can say exactly the same
thing and say it with confidence, on the basis of hard research.
Q234 Ms
Butler: I just wanted to go back to an earlier question. You talked about key stage tests and when
those tests are taken. As you know,
there is a push for stage not age; the stage when the person is ready to take
the tests. I wonder whether the panel
could just say what they feel would be any unintended consequences of
that.
Jerry
Jarvis: Did you say "unintended"?
Q235 Ms
Butler: Unintended, yes. Will it be
an extra burden on the teachers? Is it
reliant on the teachers' understanding of that child's abilities, or is the
focus all on that teacher?
Dr.
Bird: I take it that you are talking about the confirmatory single
level test.
Ms Butler: Yes.
Jerry
Jarvis: I think that the point that you are making relates to the type
of "test when ready" issue and whether that would have unintended
consequences. One positive unintended
consequence is that it would perhaps take the pressure off that terrible
examination point in the cycle. To me, it
feels right to test when ready. That
would require a much higher degree of administration and control and the
provision, perhaps, of technology to assist the teacher in the classroom. It would be bound to have a number of
knock-on consequences. My view is that
examining when ready would have far more positive than negative outcomes,
because it would tend to be about successful delivery. It would be a huge advantage for the student
and the teacher.
Q236 Ms
Butler: Are there any other views on that from the other members of the
panel?
Dr.
Bird: I agree with Jerry. There
is a danger that teachers who lack confidence would fall into a test, re-test,
re-test mentality with students who are bouncing along just below a level. That might be a problem but, generally,
testing when ready would have many more positive than negative outcomes
provided that it is allowed to replace mass testing. There is a danger that we would have single-level tests for one
purpose and some sort of national audit test run at the end of the key
stages. In many ways, that would be the
worst of all worlds.
Q237 Ms
Butler: What do you think would be the best way to ensure that there are
more positive than negative outcomes? For those children who are bouncing along
and who may be ready but whom the teacher does not think are ready, how can we
can build in a caveat to ensure that there are more positives than negatives?
Greg
Watson: There is an important principle, which runs across all
assessment, that says that we should be absolutely clear about the purpose of
assessment up front. Ideally, there
would be one purpose of assessment rather than trying to do many things at
once. We should be clear that we are assessing
in order to inform and support learning and to help teachers and to guide their
efforts. That is a very different
purpose from measuring to compare schools against one another to create league
tables and support parental choice.
Our
field does not involve key-stage testing-we are involved in GCSEs and
A-levels-but being clear on what we are testing and why is a really important
discipline. It is important to get that
right in this context so that we do not have the dangers that have been
highlighted about a conflict between an assessment to support learning and an
assessment to rate a school's performance overall.
Jerry
Jarvis: Here I am back on my subject.
The infrastructure prevents testing when ready. The process is geared to delivering
information at a constant rate to all students in the cohort, who will then be examined
at the end. We have to do much more
than simply say that we will test when ready.
The infrastructure-personalised learning-must be in place for such an
approach to work. If that happens, it
would be worth doing.
Chairman: We are getting really tight
on time.
Q238 Mr.
Chaytor: I have a short question to put to each of the four members of the
panel. Does the use of league tables
help or hinder your organisation's work?
Murray
Butcher: I probably have the easiest answer of the four panel members. Given that the majority of our work is in
vocational qualifications, league tables do not greatly affect our activity.
Q239 Chairman:
Nevertheless, what do you think of them?
You are a professional educator.
Murray
Butcher: My anxiety has already been expressed. It depends on the purpose of the assessment
and on what are trying to draw from it.
I recall that Ken Boston, the chairman of the Qualifications and
Curriculum Authority, mentioned just before Christmas that one of his
colleagues identified 22 different purposes for assessment. I suspect that 22 is almost a random number
and that you could raise it somewhat. I
suggest that league tables have far too much pressure put on them and that they
are probably not a sufficiently refined instrument to give the exactitude that
they purport to give.
Dr.
Bird: I do not think that tables impact on our work particularly
strongly. Obviously, our outputs are
part of what formulates them and we are aware of pressure around the boundary
between D and C grades in some subjects. The inquiries about results and the
seeking of scripts after the exam to check on that boundary has a relatively
marginal impact as far as operational activities are concerned. I have no other
comment really.
Jerry
Jarvis: There is no question that the pressure creates some
disadvantage to us-particularly in respect of calls for more research and more
opportunities to take-but there is a huge pressure on schools to maximise the
sorts of qualifications that are taken. Therefore, we see schools and colleges
chasing qualifications in an unhealthy way. However, that same competition
demands that we drive up service and standards in a way that we otherwise would
not. There is no question that league tables affect what goes on inside a
school. However, one that benefits society, if not necessarily Edexcel or my
colleagues, is the pressure to continue to compete for those qualifications
because they are taken so seriously.
Greg
Watson: Undoubtedly, pressure creates unhelpful tensions in the
qualifications system. If our fundamental job is to assess learners and provide
assessment that adds to learning and that helps to structure and motivate
learning; the other dimension of what we do is used to determine teachers' pay
rises and that gets in the way of that relationship. That explains the distance
that some teachers feel exists between them and us, and is something that I
want to break down. I hope that the new independent regulator-and we have
mentioned him several times-will be very aware of that in balancing change and
stability in the system. I wonder whether the regulator might run the league
tables or develop a carefully defined relationship with those who determine
such tables so that the tables do not get in the way of what the qualifications
are fundamentally there to do, which is to give young people a portable measure
of what they have achieved to take on to the next stage in life.
Q240 Mr.
Stuart: Just to go back to Edexcel ResultsPlus, will you look at common
fields and common analysis? Referring to the earlier question about the
unintended consequences, could it be that every teacher in the future will be
able to have marks for their entire career? Would you be able to categorise
them as a four-plus teacher, a three-and-a-half plus teacher and a three-plus
teacher? Now that the Prime Minister has said that he will not put up with
schools that are below a certain level and they will have to be closed, will we
move to a stage in which politicians in here will say that any teacher below a
certain level will be fired within two years or brought up to scratch?
Jerry
Jarvis: Right now, those teachers are liable to be sacked anyway for
their performance as defined by the league tables. Teachers, management teams
and students can avoid that if they use ResultsPlus to continuously assess
their performance and improve it-so it would have the opposite effect.
Mr. Stuart: And to continuously avoid
splitting infinitives.
Q241 Fiona
Mactaggart: Do exams have to be in the summer when hay fever and hot
weather cause problems to some candidates?
Jerry
Jarvis: That is a much broader question because of university entrance
and the competition for places. As someone said to me when I joined Edexcel,
the reason why we have university entrance at that time of the year is so that
children can bring in the harvest. We seriously should look at the overlap that
exists between the teaching curriculum and the examination process that leads
to higher education. Right now, I do not think that any of us can see a viable
methodology for moving that examination time.
Q242 Chairman:
Even trying to get Oxford and Cambridge to have the same application process as
other universities has been very difficult. You looked rather
uncomfortable-Greg particularly, and also you Jerry-when I said that perhaps
the opinion is that you are getting out of touch with what universities and
teachers are saying. You seemed more
hurt at that than at any other question.
May
I take you back to Kevin Stannard, the director of international curriculum
development at Cambridge University? He
said: "Through the late 1980s and the 1990s, the gap between academics and
schools got wider and wider in terms of academics disengaging from exam boards. I think what we are living with now are the
implications of that". Why would he say
that, Greg, if it was all nonsense?
Greg
Watson: My discomfort is that getting qualifications right is the
business of trade-offs. There are so
many different stakes in the qualification system. For it to work well, and for it have everyone's confidence,
everybody has to have their say; and those different and sometimes competing
demands have to be balanced-and balanced well.
I
recognise that universities have a bit less of a stake as a result of the last
10 years; employers will feel in some areas that they have a bit less of stake
in some parts of the system, although it has grown in others. We are sitting in the middle, and inevitably
juggling all those. We are certainly
making a lot of effort to build more relationships back with higher education;
they have weakened a bit, as we have had more of our focus drawn towards the
Government of the day, and a not particularly independent regulator.
Q243 Chairman:
Is there anything you want to say to the Committee before we wind up-anything
that you think you should inform the Committee about what you have not yet said
because our questions were inadequate?
Jerry
Jarvis: If I had not got across my point about the importance of the
things that Ken Boston said and about all the money that I have spent, I would
be very disappointed.
Chairman: I think that we got that.
Jerry
Jarvis: But we have a serious opportunity to shift that stubborn A to C
statistic that has dogged us for many years.
Murray
Butcher: A point came from Fiona about releasing information and the
problems it might cause. I think that
we should look at it from the reverse direction. It is sensible for awarding bodies like ourselves to find ways to
be more and more open in our processes.
If part of that is giving feedback, as mentioned, with all the data that
we collect from various examining activities, there is a growing duty and
responsibility, at least on the four of us, to be more creative and more open
with that information. It will only
make the system more robust.
Q244 Chairman:
I feel a bit guilty about you, Murray, because we have not asked you as many
questions as others. Surely we should
have asked you about this vast expansion of apprenticeships. Are they going to be linked to
qualifications in a serious way-or should they
be?
Murray
Butcher: I regard apprenticeships as they stand as serious
qualifications.
Q245 Chairman:
I am sorry, but they are not naturally linked to qualifications, are they?
Murray
Butcher: They are an amalgam of two or three different bits.
Q246 Chairman:
A lot of them are still time-served, with hardly any paper qualification. Evidence given to the Committee suggests
that that is the case.
Murray
Butcher: I would be extremely surprised, because the QCA and the
Learning and Skills Council have identified apprenticeships as involving national vocational qualifications, possibly
what is called a technical certificate, which underpins knowledge, and
functional and basic skills. That is
the apprenticeship.
Q247 Chairman:
So you are totally happy with what is happening with apprenticeships?
Murray
Butcher: I would not say that I was totally happy. They probably need more support in the
workplace, and still more understanding by employers as to what they
offer. I know that the achievement rates
remain relatively low-at about 40%.-partly because young people who begin
apprenticeships achieve employment and leave the programme. So there are obviously some problems about
ensuring continuance into employment.
There
are some structural issues that need to be resolved, but as a product they
provide a good background and a good basis for employment. What is planned at the moment is increasing
flexibility in the structure of the apprenticeship. That is what QCA and others are currently looking at.
Greg
Watson: Just a final thought on standards, which is right at the heart
of the review. In general in this
country, we spend too much time having a debate about standards at the low
level of an individual question paper, the marking of this year's examination
and small changes here or there in the percentage of candidates getting a given
grade. We are having that debate at the
wrong level. The potential for
standards to move and for public confidence to be shaken is greatest when there
is wholesale, system-wide change or major structural changes to
long-established qualifications. The
acid test for looking at the move to an independent regulator is whether we
will have a body that is sufficiently able to look at the macro-level changes
and the effect that they may have on standards and public confidence and worry
much less about the detail of which individual qualification is which. As I hope you have heard, that is something
on which we have tremendous expertise, tremendous power to innovate and a
tremendous ability to add to learning by making exams sympathetic to the
business of learning, rather than something that sits outside it.
Q248 Chairman:
But, Greg, you are still selling a product.
We on this Committee and our predecessor Committee have argued for a
long time that you do not need a new A* plus and that you can just let the
universities have the full marks at A-level.
Would that not be just as good as an A*? Why do you want an A*?
Greg
Watson: We do not just want the A* grade, but the new style of syllabus
that we have developed to support it. We have been back to every single A-level
subject and-to answer your question, Fiona, about where these questions come
from-defined in new terms what you have to be able to do to get the highest
grade in an A-level. Now, we are
setting questions to match that.
Q249 Chairman:
Will that inevitably lead to even more children from elite independent schools
dominating the leading research universities in our country?
Greg
Watson: We will just mark what we are faced with and give the A*s to
the A*-standard candidates.
Dr.
Bird: There is a forecast in our evidence to that effect. I cannot remember which paragraph it is in,
but we have provided a forecast of that in our evidence.
Q250 Chairman:
Jerry, do you think that, too? Do you
think that will be the inevitable conclusion?
Will an A* mean that even more children from the independent sector and
the best of the state school system will dominate?
Jerry
Jarvis: That is certainly a tendency and something that we really have
to work on. However, it is striking
that if you look at individual schools' performance, you can see that two
schools facing each other in the same street and drawn from the same community
can have dramatically different outcomes, and we need to understand much more
why that is. The fact that schools that
have dramatic advantages will do better is almost an inevitability. The private sector already has the vast
majority of passes at A-level, so you would expect it to excel at A*, but a
number of other schools also excel and can also differentiate themselves
hugely. In the evidence that the
awarding bodies gave during last summer's results, we faced the press together
and pointed out that there were differences in different school types and that
the overall achievement rates at some school types were doing down, not
up. However, there is, of course, a
general tendency that the better-off and the more advantaged will gain higher
grades.
Chairman: Thank you very much for that
evidence. It has been a long session,
but it has been a good one from our point of view. Sorry if we pushed you too hard at any time or if we have been
hard on you, although I do not think that we were. You have been excellent witnesses. Thank you very much for your contributions.