Examination of Witnesses (Questions 220
- 239)
MONDAY 21 JANUARY 2008
DR ANDREW
BIRD, MURRAY
BUTCHER, JERRY
JARVIS AND
GREG WATSON
Q220 Mr. Chaytor: Greg, in your submission,
you were critical of the QCA for intervening in too heavy-handed
a way, whereas in the AQA submission there were accusations of
not regulating strongly enough in respect of the current basic
and key skills tests. It said that the problem is that the regulation
is weak. Where is the balance? Is it that the QCA has been too
interventionist, or has it been too hands-off? Each of you says
different things.
Dr. Bird: I think we were saying
that it was conflicted because it was the provider of the tests.
It was not a matter of being too hands-off; it was the regulator
of itself. That was our point. Those tests are derived—we
only deliver them—by QCA itself, so it sets the standard.
Greg Watson: I would say that
it has been a case of too much in some areas and too little in
others—too much intervention at the detailed level, getting
in the way, I think, of producing more stimulating approaches
to geography, more relevant vocational offerings, but on the other
hand too little attention to the long-run effects of a series
of changes piling up. If you were to look at A-level over time
and take modularisation, the move to six units, the balancing
of coursework, the splitting of AS and A2 into two lots of 50%,
compounded with various tweaks and twists applied along the way;
if the regulator is looking after public confidence it should
be less concerned with any one of those changes in any one subject
and more with looking at the long-term effect over time on actual
standards and on public perceptions of standards, and therefore,
with measuring the rate of that sort of change in a more moderate
way.
Murray Butcher: The opportunity
for the independent regulator may mean that QCA, or whatever it
becomes known as, may be able to focus on regulation and refine
that process of regulation, which I would say, from City &
Guilds experience, has been delivered in fits and starts. There
has been urgent attention to this activity, which then suddenly
dissipates; then it comes over here somewhere, and gets distracted
by some of its curriculum activities. I really look to this as
an opportunity to have stable regulation. The one principal issue
that we still have, which Andrew mentioned in his opening statement,
is the fact that we have four regulators to deal with, and that
is a big bureaucratic problem.
Q221 Mr. Stuart: Following on naturally
from that in terms of maintaining standards, should it be an important
part of the new regulator's job to look at comparisons internationally?
Sir Peter Lampl said, of standards, that "the key question
is are they improving fast enough to maintain or improve our position
relative to other European countries, America as well as China
and India?" Do you agree with that?
Jerry Jarvis indicated assent.
Greg Watson: Any sensible regulator
in this field, with a primary focus on standards, will want to
base its decisions on evidence. That seems a strangely obvious
thing to point out, but it is a point worth making. There is good
evidence abroad. As I mentioned, we are part of a group operating
in 150 countries. It is partly through the process of synthesising
research from experience in different countries that we can draw
some of the conclusions that we can about the impact of change
on standards and so on. I would have thought that any sensible
regulator would want to build relationships around the world with
all sorts of research communities.
Q222 Mr. Stuart: In the context of
today's evidence, what do you make of this country's tumbling
down the international league tables that came out recently?
Chairman: Do not roam too widely, but
comments would be appreciated. Are we tumbling? Is that your fault?
Is it just a rotten examination board system causing us to tumble
down the international league tables?
Jerry Jarvis: First, that is a
very difficult question to answer here. We should also be very
careful about what we compare. When Leach looked at the various
standards in different countries, one of the conclusions that
we started to come to was that we probably do as much training,
but that we do not certificate it in the same way, so we do not
count it. Counting eggs and counting eggs is an important issue.
We certainly share a massive international business with OCR,
and this education system still has huge respect abroad. We cannot
be complacent but it can be very dangerous to build any sort of
policy on an international comparison because of the sheer difference
of literally comparing the same thing.
Chairman: We must move on to the last
section.
Q223 Mr. Stuart: May I just deal
with the key stage and testing at different times? Edexcel's evidence
was that that will increase the testing burden and that the Key
Stage test should be at the end of the key stage, if retained.
Do you think that if we are to have tests when the pupil is ready
rather than at the end of the stage, we need to scrap Key Stage
tests as such?
Jerry Jarvis: Forgive me, Barry,
I need to make a slightly longer response to this question, because
it is something I feel strongly about. We constantly accuse ourselves
of over-testing, but summative, quantitative testing needs to
be done to ensure that the investment that we put in, and the
teaching and learning that is happening, happens. Let me return
to what Ken Boston said—three things: personalised learning,
continuous—essentially, internal—assessment and training
of teachers. The reason why I care so much about it is that after
about £80 million of investment, we can do it now, and if
we really were interested in shifting that stubborn set of five
good A to C grades, we could do it now. It is here, and Ken Boston
was absolutely right. So, do we over-test? Yes, in certain areas,
but we do not spend enough time on personal, continuous improvement,
which is the key to improvement.
Chairman: Okay. We must move on, and
I want to cover the last section. We have the duo of Fiona and
David covering it, with Dawn throwing one matter in if she wants.
Q224 Fiona Mactaggart: We have had
quite a bit of evidence about the narrowing of the curriculum
and teaching to the test, as you would expect, and in a way, some
of your evidence is specifically about that. I am very interested,
for example, in your analysis, Andrew, of why aptitude tests are
not an appropriate alternative because you can train young people
to succeed in them—a fact I know very well, representing
a town that still uses the 11-plus. Children of 11 perform less
well on Key Stage 2 assessments, I think, because they are training
for their 11-plus aptitude tests. I think that I am hearing from
you that in some ways it is a good thing if people teach to the
test, because then we get better results. However, all of us as
educators think it is a bad thing, because the domain that young
people learn is narrower.
Dr. Bird: I think that we all
said teach to the curriculum, rather than teach to the test. We
want people to cover the whole curriculum and know, as Jerry said,
the experience that they will have when they are examined, so
that it is not a shock and they can show their best when they
are being tested. That is about understanding the style of the
examination that they will be required to sit. Our job is to ensure
that the tests maintain sufficient variety and coverage of the
curriculum over time, so that it is not easy to teach to the test
in the narrow sense in which I think you mean it—saying
"I am going to predict the six questions that will come up
next summer, and teach you seven topics. I guarantee that six
will come up." You are leading us towards a very narrow,
output-driven thought. We certainly want to give people every
encouragement to cover the actual curriculum, which is why we
make specifications available, as Greg said. When I was doing
A-levels, I had no idea what the curriculum was, which cannot
have been a good thing.
Greg Watson: If I may add to that,
we have talked quite a lot about innovation, and it is a real
benefit to have more than one awarding body. The competition of
ideas has been a powerful driver to keep syllabuses interesting
and make subjects stimulating. In the era before micro-regulation,
which really started in 1998, there were outstanding partnerships
in a whole range of subjects—science, geography, maths—between
individual awarding bodies and very creative university departments
in places such as York and Cambridge. That was true also between
groups of forward-thinking teachers—I can think of the Suffolk
science movement. That interplay of ideas put a range of options
out there for young people and their teachers. A school would
pick its syllabus to suit what kind of school it was and the young
people that it had. We are beginning to get some of that back.
There have been signs that we have moved back in the right direction,
and if we get the form of regulation right and it moves from the
micro level to the level of looking after the system, there is
scope for the interplay of different ideas among organisations
such as ourselves. All the evidence from the work that we have
done when we have had a bit more freedom to operate, and been
less under QCA diktat, is that we have been able to reignite interest
in some subjects. We are doing it again with A-level history,
in which we have gone back to an alternative, more research-based
approach, sitting alongside what we might think of as a more traditional
style. Everything that we hear from the chalk face is that some
people learn the skills of history much better by going out and
touching it with their hands than by simply learning a load of
facts from a textbook.
Jerry Jarvis: Teaching to the
test is a perennial issue for us. When I talk to the Specialist
Schools and Academies Trust and say, "Help me to avoid being
cast in the role of someone who is encouraging teaching to the
test," it slaps me on the back and says, "Get on with
it. That is exactly what we are supposed to do. We are supposed
to set a standard, teach the syllabus and assess whether students
have met that standard." I do not think that, with the enormous
stakes that are placed on league tables, you can avoid the accusation
that teachers are spending time on examination preparation. It
is inevitable, and it was always there. A long, long time ago,
when I was doing qualifications, I remember practising on past
papers and wishing I had a father who knew more about maths than
my friends' fathers did. What has changed is the pressure to succeed.
It really defines careers inside schools and colleges, and it
is something that we have to guard against. The thing that drives
me along is that if we can use technology and all the techniques
that we have to make exam preparation relatively simpler, we can
spend much more time in enriching education. Why are people educated?
Is it for entry to university? Is it in preparation for work?
Or is it for the sheer joy of learning? I seem to remember that
there was a lot of that in the past.
Q225 Fiona Mactaggart: Absolutely,
Jerry, yet your organisation is producing ResultsPlus, which,
as I understand it, will be able to show me, as a teacher, that,
fine, I might be wonderful at teaching Jane Austen but when it
comes to Shakespeare my students are much more wobbly than Barry's
students are, and that I have to shape up on that bit of the curriculum.
Do you think that there is a risk that ResultsPlus will actually
encourage that sort of thing?
Jerry Jarvis: What I have had
back from a large number of teachers is that we are moving towards
true personalised learning. Because such information is available,
they can spend less time on revision, which becomes far more to
the point. Teachers are able to go at the speed of each different
student, and they have more time to indulge in the richer aspects
of delivering the syllabus. In fact, in our experience, the process
actually works the other way around.
Q226 Fiona Mactaggart: You make it
sound as if it is the child who will be the level of reflection
that a teacher will make. Is it not more likely that it will be
the teacher who will be the level of reflection in this sort of
a system?
Jerry Jarvis: Having deployed
the technology, it is clear that it operates at many different
levels. As a tool for the senior management team inside a school
or college, it pinpoints exactly how well the school or college
is delivering its curriculum across all the subjects. For an individual
teacher in the classroom, we are seeing a jump in the performance
of their students. Teachers are literally taking the information
and using it to their advantage.
Q227 Chairman: But you do not all
agree with that, do you? Some of you are not using the technology.
Greg Watson: We are looking at
similar ideas at the moment.
Q228 Chairman: So you approve?
Greg Watson: This is nothing new,
in fact. Any teacher who wants to do the best for the young people
in their classroom will reflect on how they are doing, and will
compare how they did this year with last year. For some time,
teachers have been able to get hold of exam answers and have a
look at them to see how well they did. For many years, many teachers
have come to professional development events—INSET events—that
we lay on. Some involve a general run across the syllabus, but
some are targeted at particular areas. If we get feedback from
teachers that they are struggling with the coursework element,
we always lay on support for them to come and talk about coursework
and perhaps learn from other teachers. I think that the desire
to understand, to diagnose, to target a bit of effort to develop
professionally has been with teachers for a long time. We have
started to explore whether technology and some of the data that
we have could add to that.
Q229 Chairman: There are no dangers?
Dr. Bird: There is the issue about
releasing detailed results directly to students. We think that
that can be mediated through the relationship that students have
with the centre and with their teachers rather than giving results
to them directly, if that is what you mean by richer information
being available. We certainly think that teachers can be assisted
by having richer feedback about how the cohort of students is
performing so that they can modify their teaching methods. It
is obviously better if they get that from formative assessments
that they carry out in the classroom rather than from summative
results for the cohort that has just gone, as those students already
have their qualifications, whether good or bad. Many people's
support materials for classroom teachers contain formative tests
in the form of homework, which essentially have the teacher ask,
"Did I get that algebra across correctly or not?" That
must be a helpful process, to assist teachers and curriculum heads
in understanding how different teachers are performing in different
parts of the curriculum, so that they can play to their strengths
and fill in where there might be weaknesses, because people are
not universally good at doing everything in the curriculum.
Q230 Fiona Mactaggart: When you design
questions for examinations, which of course is a mystery to most
people—we can only imagine the types of questions—you
test all sorts of questions and design them to find out if they
show specific knowledge or a range of skills, and such like. When
you do that, do you think about the skills or knowledge that reveals
something beyond the ordinary? We talked earlier about creating
an A* grade, for example. Clearly that must have implications
for the design of questions, to create the space for a young person
to show that ability. None of your evidence tells us how that
is done.
Chairman: One of you answer that question,
or we will be here all night.
Greg Watson: It is a very careful
and very rigorous process; a lot of what you suppose to be true
is true. In writing a syllabus, we are not simply writing down,
"Oh, read this book and you will know all about the Second
World War." We will actually unpack that process to say,
"By the time someone has successfully completed a GCSE course,
they should know the following things and they should be able
to do the following things with that knowledge." We will
then describe a series of levels, which we call grades, that will
differentiate the level of knowledge that you would be likely
to have and what skills you would be able to apply to that knowledge.
Interestingly, in the context of A* we have been doing a research
project in advance of introducing A*, because we have been through
the process of redescribing the syllabus. We have now described
to ourselves, "What does an A* grade historian have by way
of knowledge and skill that you would not have expected of an
A grade student?" We are now trying out questions at the
moment for exactly that reason; we want to discover what types
of questions unleash that potential and what types of questions
are sufficiently open-ended to provide that extra stretch and
also, importantly, what types of questions successfully differentiate
the most able students from the rest and do not lead them all
to being clustered around the middle of the mark range. So that
is the type of work that we are doing. We have a great army of
research people sitting behind everything that we do and that
is what they do all day and every day.
Q231 Chairman: So you want to take
over the world, do you not? You do not want the Qualifications
and Curriculum Authority or a national curriculum. You want to
set the whole thing really, because coming through is the idea
that you can do it all. Your exams drive the curriculum.
Greg Watson: I think that we have
very important expertise. We are capable of adding more value
in terms of innovation and enriching and discovering new approaches
to subjects.
Q232 Chairman: Is it not dangerous
to let you take over the world of education?
Greg Watson: There is something
missing in my equation, which is why we need a regulator, and
it is that the public's stake and society's stake in the qualifications
system is much bigger now than it has ever been. Because of that,
the public want to know that young people's education is safe
in our hands and that we are doing what we should be doing absolutely
properly. It is quite right that somebody who is not sitting along
this line of witnesses here should have an independent view and
should be tracking effectiveness.
Q233 Chairman: So you want a weak
regulator to defend you from public criticism?
Greg Watson: I want a strategic
regulator who can really stand up on the basis of hard evidence
and support us when we say that we are looking after standards
and everybody is getting a fair deal. There should be an independent
body that is not a vested interest, which we might be seen as
being, and that can say exactly the same thing and say it with
confidence, on the basis of hard research.
Q234 Ms Butler: I just wanted to
go back to an earlier question. You talked about Key Stage tests
and when those tests are taken. As you know, there is a push for
stage not age; the stage when the person is ready to take the
tests. I wonder whether the panel could just say what they feel
would be any unintended consequences of that.
Jerry Jarvis: Did you say "unintended"?
Q235 Ms Butler: Unintended, yes.
Will it be an extra burden on the teachers? Is it reliant on the
teachers' understanding of that child's abilities, or is the focus
all on that teacher?
Dr. Bird: I take it that you are
talking about the confirmatory single level test.
Ms Butler: Yes.
Jerry Jarvis: I think that the
point that you are making relates to the type of "test when
ready" issue and whether that would have unintended consequences.
One positive unintended consequence is that it would perhaps take
the pressure off that terrible examination point in the cycle.
To me, it feels right to test when ready. That would require a
much higher degree of administration and control and the provision,
perhaps, of technology to assist the teacher in the classroom.
It would be bound to have a number of knock-on consequences. My
view is that examining when ready would have far more positive
than negative outcomes, because it would tend to be about successful
delivery. It would be a huge advantage for the student and the
teacher.
Q236 Ms Butler: Are there any other
views on that from the other members of the panel?
Dr. Bird: I agree with Jerry.
There is a danger that teachers who lack confidence would fall
into a test, re-test, re-test mentality with students who are
bouncing along just below a level. That might be a problem but,
generally, testing when ready would have many more positive than
negative outcomes provided that it is allowed to replace mass
testing. There is a danger that we would have single-level tests
for one purpose and some sort of national audit test run at the
end of the key stages. In many ways, that would be the worst of
all worlds.
Q237 Ms Butler: What do you think
would be the best way to ensure that there are more positive than
negative outcomes? For those children who are bouncing along and
who may be ready but whom the teacher does not think are ready,
how can we can build in a caveat to ensure that there are more
positives than negatives?
Greg Watson: There is an important
principle, which runs across all assessment, that says that we
should be absolutely clear about the purpose of assessment up
front. Ideally, there would be one purpose of assessment rather
than trying to do many things at once. We should be clear that
we are assessing in order to inform and support learning and to
help teachers and to guide their efforts. That is a very different
purpose from measuring to compare schools against one another
to create league tables and support parental choice. Our field
does not involve Key Stage testing—we are involved in GCSEs
and A-levels—but being clear on what we are testing and
why is a really important discipline. It is important to get that
right in this context so that we do not have the dangers that
have been highlighted about a conflict between an assessment to
support learning and an assessment to rate a school's performance
overall.
Jerry Jarvis: Here I am back on
my subject. The infrastructure prevents testing when ready. The
process is geared to delivering information at a constant rate
to all students in the cohort, who will then be examined at the
end. We have to do much more than simply say that we will test
when ready. The infrastructure—personalised learning—must
be in place for such an approach to work. If that happens, it
would be worth doing.
Chairman: We are getting really tight
on time.
Q238 Mr. Chaytor: I have a short
question to put to each of the four members of the panel. Does
the use of league tables help or hinder your organisation's work?
Murray Butcher: I probably have
the easiest answer of the four panel members. Given that the majority
of our work is in vocational qualifications, league tables do
not greatly affect our activity.
Q239 Chairman: Nevertheless, what
do you think of them? You are a professional educator.
Murray Butcher: My anxiety has
already been expressed. It depends on the purpose of the assessment
and on what are trying to draw from it. I recall that Ken Boston,
the chairman of the Qualifications and Curriculum Authority, mentioned
just before Christmas that one of his colleagues identified 22
different purposes for assessment. I suspect that 22 is almost
a random number and that you could raise it somewhat. I suggest
that league tables have far too much pressure put on them and
that they are probably not a sufficiently refined instrument to
give the exactitude that they purport to give.
Dr. Bird: I do not think that
tables impact on our work particularly strongly. Obviously, our
outputs are part of what formulates them and we are aware of pressure
around the boundary between D and C grades in some subjects. The
inquiries about results and the seeking of scripts after the exam
to check on that boundary has a relatively marginal impact as
far as operational activities are concerned. I have no other comment
really.
Jerry Jarvis: There is no question
that the pressure creates some disadvantage to us—particularly
in respect of calls for more research and more opportunities to
take—but there is a huge pressure on schools to maximise
the sorts of qualifications that are taken. Therefore, we see
schools and colleges chasing qualifications in an unhealthy way.
However, that same competition demands that we drive up service
and standards in a way that we otherwise would not. There is no
question that league tables affect what goes on inside a school.
However, one that benefits society, if not necessarily Edexcel
or my colleagues, is the pressure to continue to compete for those
qualifications because they are taken so seriously.
Greg Watson: Undoubtedly, pressure
creates unhelpful tensions in the qualifications system. If our
fundamental job is to assess learners and provide assessment that
adds to learning and that helps to structure and motivate learning;
the other dimension of what we do is used to determine teachers'
pay rises and that gets in the way of that relationship. That
explains the distance that some teachers feel exists between them
and us, and is something that I want to break down. I hope that
the new independent regulator—and we have mentioned him
several times—will be very aware of that in balancing change
and stability in the system. I wonder whether the regulator might
run the league tables or develop a carefully defined relationship
with those who determine such tables so that the tables do not
get in the way of what the qualifications are fundamentally there
to do, which is to give young people a portable measure of what
they have achieved to take on to the next stage in life.
| 
