Examination of Witnesses (Questions 300
- 319)
MONDAY 28 JANUARY 2008
DAVID BELL,
SUE HACKMAN
AND JON
COLES
Q300 Stephen Williams: You think
that standards have been held, but not necessarily improved?
Sue Hackman: I definitely think
that standards have improved. We can corroborate that by the patterns
that exist in other surveys of pupil abilities, and in the PIRLS
and PISA tests, which also give us another take on how standards
are developing as do Ofsted reports. We are pretty certain that
standards are rising. However, having said that, no one will say
that every test is perfect. There must be year on year fluctuations.
It is the task of the QCA to ensure that they are regulated and
watched.
Q301 Stephen Williams: If Sue Hackman
thinks that standards are improving, how does that stack up with
the evidence from Professor Smith in the earlier session? He said
that universities increasingly have to lay on remedial classes,
particularly in maths, or for anyone who wishes to do a physics
degree, because the standard of entrance—even if a pupil
has three straight As—is not sufficient to be a first year
undergraduate. Therefore, how could standards have gone up?
Sue Hackman: National tests test
every single child in the cohort. University intakes vary from
year to year, so it did occur to me that university fluctuations
might be to do with which pupils are choosing to come to your
university over time. Having said that, I agree that when pupils
are studying a science degree at university, they need a high
level of mathematics, and we must strive to produce pupils who
can do that.
Q302 Stephen Williams: But if you
look at the number of people applying to do a subject such as
physics, it has gone down considerably over the last 20 years.
So it is safe to assume that the people who are applying to do
physics now are the hard core who want to do that subject and
are committed to it. Yet university departments say that the standard
coming in is not what it was—not only at Exeter, I have
also heard it from many other admissions tutors around the country.
Jon Coles: There are some important
considerations relating to how many people are doing a STEM subject,
which is broadly science, technology, engineering, or maths, and
there is a need to increase that number. You will know from previous
inquiries, and from documents that the Department has submitted
to the Committee, some of the background to what we are doing
about the matter.
One thing that has been happening over the last
20 years at A-level and post-16 years old, is a significant widening
of the choices available to young people and a significant increase
in the range of options that they can take. A feature of that
has been a decline in the number of people doing some of the traditional
subjects, particularly science subjects. That is unquestionably
a cause for concern. In recent years we have seen some reversal
of that trend in relation to some key science subjects and to
maths, which was affected after curriculum 2000. A range of issues
needs to be addressed. I do not think that any of what I have
said suggests that standards themselves have fallen, but rather
that there is a wider range of young people doing a wider range
of subjects. Therefore, people who might once have chosen to do
science subjects are choosing other subjects. That is a feature
of our system, and you have to judge at every stage to what extent
to allow people free choice of subjects or to attempt to constrain
their choice.
Q303 Chairman: If you took students
from the independent sector and grammar schools out of the number
of young people qualifying to come through in STEM subjects, you
would be very worried indeed. So something is going on in the
state sector that you surely need to take seriously.
Jon Coles: That is absolutely
right. I do not have the figures in front of me for numbers of
school leavers doing STEM subjects by different types of school,
although the Department certainly has them. That concern is an
important policy direction. As you know, a huge amount of work
is going on to raise the numbers of students doing the sciences,
particularly the physical sciences, maths and other technology
subjects. That is important work. The good news is that, in the
last couple of years, some reversal in that decline has begun
and numbers are beginning to come up again. Numbers in maths have
now recovered to above their pre-curriculum 2000 levels, which
is serious progress. There is more to do, and of course the new
science Diploma will be part of encouraging more and a broader
group of young people into science.
Q304 Stephen Williams: I think that
we are drifting on to the choices people make at 16, rather than
the standards once people have exercised those choices, which
is what we are asking about. Looking at A-levels in the round
and going back to some of the comments made in the earlier session
by Professor Smith again, I wrote down what he said: increasingly
students mark hunt; they do not have independent thinking and
are afraid to give critical answers; and they play safe. Do you
accept that it is a fault with the existing modular A-level system
that students are simply trying to leap each hurdle in order to
get the grade at the end, rather than having an in-depth understanding
of the subject?
Jon Coles: As I was trying to
say in an earlier answer, the A-level system tests people's knowledge
of the whole syllabus better than before, and ensures that more
young people have to know more from all of it.
I think that we agree there is a need to have
more questions that test the abilities to think independently
and critically, to analyse in depth, to give extended answers
and so on. That is the purpose of the reforms in train at the
moment, which will reduce the number of A-level modules from six
to four and increase the amount of synoptic assessment. That is
absolutely a driver of policy to get more of that into A-level.
Q305 Stephen Williams: I have one
and a half questions. This is the last time that I will ever ask
a question in this Committee.
Chairman: On this day.
Stephen Williams: First of all, it has
been put to us that as a result of this teaching to the test,
teachers have become more skilled at getting children or young
people through exams, but the downside is that the students themselves
become deskilled in the area I have just asked you about. Given
that universities have these worries and employers have these
worries, as we heard from the CBI, and given that universities
are increasingly introducing or reintroducing their own supplementary
exams and tests, are A-levels fit for the purpose for which they
are perceived to be designed?
Jon Coles: Yes, I think they are.
The changes we are making now will make them more fit for that
purpose. The tougher questions that follow from the changes that
I have been talking about and the introduction of an A* will address
the concerns of those universities which struggle to discriminate
between candidates. Those changes, together with the extended
project, will mean that more and more young people are required
to learn independently, to study in depth and to pursue their
own thinking and ideas. Those are crucial things. All of those
things are being designed into Diplomas from the outset. So the
size of Diploma assessment unit will be the size of the new, bigger
A-level assessment units rather than the existing smaller ones.
They will be designed to encourage critical thinking and reflection.
That broad set of personal learning and thinking skills—the
things about self-management you were pursuing in the last session—the
ability to work in teams, to learn in depth and to research critically
are all built into the Diplomas and into the new extended project.
So all young people pursuing that route will be required to pursue
in depth and under their own steam an area of learning that is
particular to them. It will force them to become independent learners
in order to be successful. That is a set of changes that we have
under way which are very important.
Q306 Chairman: The awarding bodies
said right at the end of our session last week that they thought
an A* would certainly favour the independent sector in getting
into the research-rich universities. Does it not concern you that
they said that?
Jon Coles: We have analysed that.
It was a concern that we all had. It was something that was important
to look at. At the moment the analysis I have suggests that just
over 9,000 of the 26,000 of the candidates for A-level—some
may be over 18—who get three As come from the independent
sector. That is just over a third. Our analysis based on last
summer's exam results of what proportion would get three A*s,
suggests that about 1,150 of just over 3,050 candidates who would
have got 3 A*s had that grade been introduced last year would
be from the independent sector. Again, that is just over a third.
It is a slightly greater proportion from the independent sector,
but it is not dramatic.
Q307 Chairman: But that is not the
point, is it? The point is that what we know in this Committee,
and you must know it well, is that the longer you extend and the
higher the hurdles you put up in that process of coming through
education to higher education, the more kids from less privileged
backgrounds drop by the wayside or lower their sights. That is
why we are so obsessed as a Committee in all its incarnations
that, for example, Oxford and Cambridge still get away with having
a different application system. As you said, David Bell, we all
go to schools and we know that as soon as the kids from the poorer
backgrounds find out that there is this posh route to Oxford and
Cambridge they say, "I'm not going to be different from my
friends and I'm not going to do that." So you lose them there.
Then you are going to give them A*s, and that will put them off
even further. That is the problem, is it not? It is more psychological
than the careful analysis of figures you have just given us.
Jon Coles: I think that the point
the awarding bodies were making in your discussion—it is
absolutely right to raise it as an issue—is whether you
would have a disproportionately greater proportion of people from
the independent sector getting three A*s. This analysis suggests
there is a slight increase in that proportion, but it is very
slight.
Q308 Chairman: Why not just publish
the results, give them the figures and let them judge? What is
wrong with the figures? Why do you have to have an A*? Give them
the figures. They can judge who has got what in their A-levels
at present.
Jon Coles: Well—
Chairman: David, you are looking very
quizzical.
David Bell: I will come in in
a minute.
Jon Coles: At the moment, we certainly
allow universities to have information about performance in AS
modules and to have the marks in individuals AS modules if they
want them—that is available to them.
Q309 Chairman: In AS?
Jon Coles: Of course, it is only
AS marks that exist at the moment of making offers, which is the
crucial moment for many of the selecting courses in HE. So, they
do have that information available to them. The issue, of course,
is that the rules around AS and cashing in mean that not every
candidate has their AS grades at that point. We make available
all the information that exists in the system now, but the point
you are making is that some universities would like more information
and a greater ability to make choices. This is an important issue.
The crucial point about the A* is that it is designed with respect
to the more stretching assessments that we are putting in place;
it is designed to make sure that the things that we are trying
to do to stretch students and the broader range of skills that
you are rightly focusing on are taught, rewarded and recognised
in the assessments. It is no good putting in those more stretching
questions, which require people to show that they can analyse
in depth, if, having demonstrated that they can do so, they do
not get the reward for that. That is an important part of why
having an A* makes a difference educationally, as well as in terms
of selection.
Q310 Chairman: We do fear that there
will be an A** and an A*** on the way shortly.
David Bell: Very briefly, I just
wanted to defend the honour of many universities, which actually
go out of their way to put into place programmes—some are
funded nationally and some are introduced on the instigation of
universities themselves—to open up access as best they can.
But, of course, the universities themselves rely on the supply
of students from schools and colleges, which is why a lot of our
attention has equally got to focus on encouraging schools to ensure
that their youngsters get the right opportunities. The assessment
is an important issue, as Jon has said, but a lot is being done
across the system to ensure that students are given the best possible
opportunities to go on to higher education.
Chairman: David, we take that point,
but you know how this Committee has felt historically about the
dual application process. We move on to single level tests.
Q311 Lynda Waltho: I think that we
are expecting the results of the first round of the pilot single
level tests on 18 January. I have a quote. Apparently, a DCSF
spokeswoman said: "there are some differences between subjects,
levels and performance in different key stages that we need to
understand better before we are confident about releasing results".
Are you any closer to understanding the results and publishing
them?
David Bell: I am pleased to confirm
that that quote was correct, and we said in a letter to the schools
that participated that there had been some unexpected patterns
in the results. The first thing to say is that we would not and
should not be surprised that, when you pilot a new form of testing,
you might need to see what actually happened. We are doing some
further work, and we have asked the National Assessment Agency
to do some further work. We are not ready yet to come back with
the results of that analysis to say what has happened. To reinforce
the point, however, we will make these findings public in due
course. Obviously, we want to enable those schools and students
who took part in the pilots to receive their results in due course.
I cannot say any more than that at the moment. It would be unfortunate,
to say the least, if I misled the Committee by starting to speculate
on the results of that kind of analysis.
Q312 Lynda Waltho: It is about the
time frame, really. The Government have actually stated that the
single level test will be rolled-out depending on positive evidence
from the pilot, but what are we talking about when we say, "positive
evidence"?
David Bell: We said two things:
first, as you suggested, there was positive evidence from the
pilot, to which I will return in a moment; secondly, there is
an endorsement of the approach from the regulator. It is a very
important second point that we would need to ensure that the tests
passed the standard. In terms of positive evidence, there are
a number of things that you want them to do. You ask whether they
are robust, reliable and valid, what is their impact on students'
and teachers' behaviour, and so on. We have to consider a range
of things in this round of testing. We are very fortunate in having
40,000 students taking part at this stage, so there is a good
base. We also said that we would expect at least four rounds in
the pilot phase, so that we learn as we go—I did a bit of
homework on this before I came in—and we should not be surprised
that it takes time to do that, because if you look back to the
introduction of the national curriculum tests, it took quite a
bit of time to get them right. We are quite careful in saying
that we are doing a national pilot; we will examine it carefully;
it will be independently reviewed; and we will need the advice
of the regulator before we go forward nationally. However, as
Sue said earlier, we think that the basic principle is sound in
offering youngsters a test when they are ready to take it, to
build it much more naturally into the flow of teaching and learning.
You have to go from what seems the right principle to something
that works in practice, hence the need for the national test in
the pilot.
Q313 Lynda Waltho: As the head of
the whole thing, do you have a time frame that you think would
be a good idea to aim at—a time when you would want it to
be finished?
David Bell: We have the four rounds
of the pilots, and perhaps Sue will give more detail on the timetable.
We are cautious about saying that it will be done by this summer
and that we will try to roll it out and so on, because we want
to be very clear that we have the evidence to make a decision
should Ministers decide to go in this direction.
Q314 Chairman: Should Sue's head
be on the block for this? Sue, is it your section's responsibility
that this project is not making sufficient progress?
Sue Hackman: Team work is my responsibility.
David Bell: I must come in and
defend a member of staff. In the end, the Permanent Secretary
has to take responsibility in the civil service.
Sue Hackman: We have given ourselves
two years. There are two test windows a year for test runs. At
the end of that time, we would know if this thing is workable,
but then there is a run-in time to prepare future tests, so it
would not automatically happen that, at the end of two years,
you would go straight into a single testing system. We would commit
to saying that, after two years, we would take a decision on whether
to carry on or not.
Q315 Lynda Waltho: You were quite
definite about it being possible by the 18th—
David Bell: Just to be clear,
we had given a date for the release of the results to individual
schools and pupils on the basis of the December pilot. That is
the work we need to do, based on the first pattern of results.
As Sue said, in terms of making a decision about the future of
the whole programme, we are working to that longer time scale.
Q316 Ms Butler: So is it intended
that, at the end of two years, the single level test would replace
all the other Key Stage tests nationally?
Sue Hackman: Yes. The pupils in
the pilot are doing both; they are testing the new tests and sitting
the Key Stage tests. If it were successful, the aim would be that
it would replace the current testing regime and there would be
single level tests for pupils as they move through the system.
Q317 Ms Butler: What safeguards are
in place—I ask this in the context of Bernard Coard's 1970
report on the education of black boys—to ensure that black
boys will not be held back by teachers' perception that they are
not ready to take the test?
Sue Hackman: Underpinning the
single level test is a big project to secure assessment for learning
in schools based on a piece of work that QCA is doing for us called
APP—Assessing Pupil Progress. That is a ladder of progress
in reading, writing and mathematics, and in due course we think
it will apply to other curriculum subjects. Those criteria will
help teachers to arrive at accurate rather than impressionistic
judgments, and they are the same criteria that underpin the single
level tests, so there will be a close tie between teacher assessments.
Single level tests are attractive and attract a lot of attention,
but the progression pilot is a project that starts with classroom
assessment with strong strategies for knowing exactly what children
can do and what they need to do next. It also helps with periodic
judgments based on the criteria that I have just described. When
the teacher and the pupil feel that the pupil is ready for the
next level, they would be entered for the next level, a little
like they do in music tests. The test will confirm the teacher
assessment. With this model, teacher assessment will have more
credibility and more importance than currently. The answer to
your question is that assessment for learning is really important,
and the test and the teacher assessment go hand in hand on the
ground as part of the pilot.
Chairman: A lot of people are worried
about that comparison with music tests. The Secretary of State
uses it often—sorry Dawn.
Q318 Ms Butler: May I drill down
on that a little? Are you saying that you are introducing further
smaller tests to assess whether pupils are ready for the single
level test?
Sue Hackman: No. I am describing
the single level test. There is just one set of tests.
Q319 Ms Butler: So what is the APP?
Sue Hackman: The APP is the material
to help teachers to arrive at accurate judgments in their everyday
assessments. It is classroom assessment—ongoing assessment.
| 
