Examination of Witnesses (Questions 151-159)
JERRY JARVIS,
SIMON LEBUS
AND DR
VIKKI SMITH
22 APRIL 2009
Q151 Chairman: Could we have
the next set of witnesses. I am sorry that you have had a slight
delay. May I welcome Jerry Jarvis, Simon Lebus and Dr Vikki Smith
to our proceedings. I am sorry that we are going to have a shorter
session than we planned. You know exactly why, because you were
sitting there listening earlier. We usually give people a chance
to say something about accountability and the inspection system
and how you view it. You are in a very powerful but privileged
position in your organisations. Can I start from the left, Jerry,
Simon, Vikki, if you do not mind me using your first names? Do
you want to say something to get us started Jerry, or do you want
to go straight into questions? It is up to you.
Jerry Jarvis: I have not prepared
anything in advance. I am very comfortable to take the questions
as they are.
Q152 Chairman: Why did you
come here?
Jerry Jarvis: I came here, first,
because I was invited. I am head of one of the principal examination
boards in the country. We have a huge responsibility. We have
just gone through a very important set of evidence in the previous
session. It is very important that people like me are held to
account and make as big a contribution as we can to the well-being
of the system. I am here out of duty.
Q153 Chairman: Thank you very
much, Jerry. I just say to all the witnesses that if you feel
that anything asked by members of this Committee touches on a
commercially sensitive area we understand that you might not be
able to answer. One of you mentioned to me that there are some
sensitivities in one particular area. Just make that clear in
terms of your response. Simon?
Simon Lebus: I am the group chief
executive of Cambridge Assessment, a department of the University
of Cambridge, which owns the exam boards OCR, Cambridge International
Exams and Cambridge ESOL. We operate in 150 countries throughout
the world, as well as in the UK, so have a very good perspective
on the situation internationally. I am here, likewise, because
I was asked—inevitably out of interest in your previous
session—but also because I think the whole issue of accountability,
and the use to which exams are put in terms of their application
as an accountability measure, is critical. In terms of the overall
system and the wash-back effect on educational exams that arise
from their use as a measure of accountability, there are a number
of impacts that are of concern and need reflection.
Dr Smith: I am Vikki Smith, director
of assessment and quality at City & Guilds. Again, we were
invited to submit evidence, and we were pleased to be invited.
For us, it signals a potential blurring of historical boundaries
that tend to see a separate vocational qualification. I hope that
the issue to be discussed will be how we move to a more holistic
picture and better sharing of data that is of use and more accessible.
Also, City & Guilds has made a very firm commitment to diplomas.
If the market leads how diplomas develop and they become more
vocational, as we believe they should, that will be core to us,
and we will need to look at different ways of managing that accountability,
because the diplomas will demand that. We are very pleased to
be here.
Chairman: Good. Let's get into the questions.
Graham.
Q154 Mr Stuart: In your view,
what aspects of provisions should a school be accountable for,
and to whom?
Chairman: Who wants to take that? Jerry?
Jerry Jarvis: I would almost prefer
to answer that as someone from the street, if you like, rather
than as head of an exam board. I believe that they have a responsibility
to prepare students for higher education, but also to prepare
for broader issues, such as the ability to take a place in society,
to be ready for work and perhaps to develop those characteristics
that engender achievement in people—to celebrate and develop
things that people are good at. Part of the reason for this today,
I guess, is that the achievement of academic qualifications clearly
dominates, so I think that it would be advantageous if we could
broaden the scope of that accountability away from the narrow
focus on academic qualifications.
Q155 Mr Stuart: It is a very
broad question. I think when we did our testing and assessment
report, there were 23 purposes of examinations and accountability
that we came up with. I was trying to get your point of view,
less as ordinary citizens, but more as experts in this area. What
accountability can examinations provide, and what are the areas
where examinations cannot provide that, and it would be better
provided using some other method, such as sampling? Would one
of you deal with that broad issue?
Simon Lebus: I think, in a sense,
the other name for exams is qualifications, and they are about
the qualifications that individuals need to succeed in the various
routes that they choose to pursue with their career and their
life—that is their primary purpose and function. I think
that a lot of the difficulty arises when multiple functions are
then heaped on top of that. Clearly, parents, teachers, taxpayers
and citizens all have an interest in seeing how well schools equip
children to be successful in life, and exams have become a form
of proxy for that. That, in itself, is not necessarily damaging.
What is damaging is the apparatus that is put in behind that.
Once that comes to be done in a systematic and mechanical way,
all sorts of distorting factors come into play: various artificial
equivalences, a whole philosophy of credentialism and an approach
to the design of qualifications, all of which interfere with that
primary, educational purpose. I do not think that it is an illegitimate
thing for a variety of interested parties to be looking at qualifications
and results to evaluate how well an institution is succeeding
in its task of equipping learners for their later life. I think
the difficulty arises when a whole edifice of construction is
built on that using rather elaborate and artificial equivalences
and measures.
Q156 Mr Stuart: Is there room
for greater teacher assessment in place of the formal examinations
that you provide?
Simon Lebus: There is no question
that there is room for greater teacher assessment. I think the
difficulty, as ever, is the question of public trust. There have
been various debates about coursework and the extent to which
people are schooled in coursework so that they can do very well
in it, and then how that compares to written qualifications. There
is nothing educationally wrong with teacher assessment at all.
The question is how ready people are to trust that. Also, just
thinking from an international perspective, and looking at what
has happened in qualifications over the last 10 years, we live
in a global economy. People are increasingly mobile. Qualifications
are a form of currency and a support for them in their mobility
and their careers, and they need to be trusted. I think it is
a case that where systems have very large elements of teacher
assessment, degrees of trust tend to be slightly reduced.
Q157 Mr Stuart: At primary
level, for instance, do you have a view on the fact that there
could be more teacher assessment and, in terms of schools accountability,
that we would be better using alternative methods such as sampling?
If the teachers are not contributing to their own assessment,
so to speak, through the assessment of pupils, then that distortion
will be removed and there would be less teaching to the test,
and hopefully the assessments provided to secondary schools would
be more useful than they currently are with a supposedly independent
external examination. Do you agree with that?
Jerry Jarvis: Ken repeatedly made
the case for onscreen marking. We have virtually 100% onscreen
marking running at the moment, and I provide a complete breakdown
and analysis of every teacher's own performance in delivering
the curriculum that they are required to deliver. It is, however,
the case that probably less than 10% of those teachers actually
use that analytical information sensibly and sincerely. Part of
it is because of the way that we come about it. We expect a great
deal of our teachers—we expect them to be the sorts of individuals
who can inspire and lead and give us values. Certainly teachers
did that for me when I was young. But we also need them to be
accomplished managers of processes as well, because we have huge
examination processes going on. The current system, as Simon is
alluding to, clearly separates the role of an awarding organisation
such as mine so that there is clearly regular separation from
the delivery of the process, almost to the exclusion of a teacher
unless it is to do with coursework and so on. But I will hark
back to the technology again: you can blend the two if you use
the technology intelligently. Continuous personalised assessment
is a key issue of learning and yet we separate that from the formal
process that we engage in, and the technology could actually blend
those to great effect. As I say, we expect a great deal of our
teachers. We expect them to be able to do both. Let me risk an
analogy. If you were running an art gallery in which the material
was hung by artists who were really committed to the purpose of
their art and so on, you would not necessarily ask one of them
to run the art gallery and take the money at the door. But we
do expect our learning institutions to do both of those things
that I have talked about, and we separate the way in which we
measure those things to a huge extent.
Q158 Mr Stuart: Can I move
on to contextual value added. There seems to be more and more
criticism that it gives no more accurate an assessment of a school's
performance than conventional league tables. What views do you
have on CVA?
Simon Lebus: I think the issue
to some extent is that it becomes very confusing. The more measures
that are introduced the less clear the picture. There is a sense
that one set of measures is introduced that does not necessarily
give people the information they feel they want, or does not necessarily
give the result, so another set of measures is introduced, and
then a third set of measures is introduced. If you take something
like CVA you can have the peculiarity of a school that performed
very well on the CVA but not very well on the five to eight A*-C
at GCSE. What conclusion do you draw from that? It is difficult
to know what conclusion can be drawn from it as a taxpayer, a
member of an LEA, a parent or a teacher. All those different groups
will draw different conclusions. I do not think that there is
anything wrong with CVA as such, but it is not clear that it adds
a lot of value in terms of clarifying the position and enhancing
understanding. That is simply a function of the replication of
measures, not necessarily that measure itself.
Q159 Annette Brooke: I should
particularly like to ask Cambridge Assessment about its comments
regarding "perverse incentives" for schools to choose
easier qualifications as an outcome from the performance tables,
and the game playing. First, has this intensified over recent
years? I recall that in the past schools played examination boards,
but perhaps now we are talking about subjects as well.
Simon Lebus: League tables are
a relatively recent phenomenon—they are only 15 years old.
With the passage of time, institutions that are judged in performance
terms by those league tables become more sophisticated at how
they play the game. More and more judgment of school performance
is based on performance in league tables, to the extent that schools
have become very sophisticated. However, whether they have become
more sophisticated over the last two or three years I could not
say. Once the incentives are put up and the equivalences are created—so
that one GNVQ is equivalent to four GCSEs or whatever, and there
is a five-GCSE threshold—we set in train a pattern of behaviour
that is bound to arise from using the results for accountability
in that way. Whether that has intensified over the last two or
three years I am not sure.
|
