Examination of Witnesses (Questions 340
- 359)
MONDAY 18 FEBRUARY 2008
JIM KNIGHT
MP AND RALPH
TABBERER
Q340 Mr. Chaytor: Minister, the Department's
submission to the inquiry describes the arrangements at Key Stage
1, saying that, "The child will not necessarily recognise
a difference between the formal tests and tasks he/she completes
for other classroom exercises." If that is important at Key
Stage 1, why is it not important at Key Stages 2 or 3?
Jim Knight: I will let Ralph make
a contribution, because he has been sitting here very patiently,
but I would say that there is a difference. When you look at the
age at Key Stages 1, 2 and 3, there is clearly a significant age
difference, and with that, in general terms, there is a difference
in maturity. There comes a point when it is appropriate to start
introducing young people to the pressures of assessment. Those
are pressures that we all live with throughout our educational
careers; we have to start getting used to that at some point,
and I think 11 is a better age than seven.
Q341 Chairman: Ralph, I hope that
you do not feel neglected. The Minister has said that you have
been very patient. Will you catch my eye if you want to say something,
and we will welcome you in?
Ralph Tabberer: Thank you, Mr.
Chairman. I endorse what the Minister has said. We try to take
decisions about assessment that suit the context and the particular
teaching and learning environment. We will perhaps look at the
use of more controlled assessment and teacher assessment, where
they offer us a better alternative. That might, for example, be
when young people are younger; there may be more variation in
their performance on particular days, and such assessments may
be more sensitive. We would also look at the use of more controlled
assessment or teacher assessment in areas such as applied learning.
There are aspects of applied learning in Diplomas that will not
be as susceptible to an external test.
Q342 Mr. Chaytor: When we get to
Key Stage 2, the judgment is made on tests that last about 45
minutes. How does that equate with the Minister's criticism a
few moments ago of what is now the old system of language orals?
You said, "We want to move away from the hit-and-miss, 20-minute
test in which you are coached to learn." How can it be wrong
that there is a hit-and-miss, 20-minute test, but right that there
is a hit-and-miss, 45-minute test?
Jim Knight: In respect of the
oral examinations for GCSE, those are the qualifications that
you take with you through your life. I cannot remember whether
I got a B or a C for the oral.
Mr. Chaytor: I am sure it was a B.
Jim Knight: Well, I got a C for
the written one and a B for the oral or vice versa. I cannot
remember which way round it was, but I do remember the oral exam.
You carry that with you. I cannot imagine that many people remember
their SATs scores—I do not reckon many of us were young
enough to take them.
Q343 Mr. Chaytor: But the tests determine
the primary school's position in the league tables and the pupil's
self-esteem when they enter secondary school. My question is why
are the Government so hung up on the single test at the end of
Key Stage 2.
Jim Knight: It may be that we
are not. It may be that if testing when ready and the single level
tests prove effective in raising standards, we will be able to
move to a position in which you have a number of test windows
during a year—there are currently two, but we might be able
to develop that further—and it is not necessarily all about
how everyone did on a rainy Monday afternoon in English and a
rainy Friday afternoon in maths at the end of Key Stage 2; it
can be throughout that key stage.
Ralph Tabberer: I add to that
that the judgment we are making is about the context—the
type of learning taking place—and an oral assessment looks
to us better placed as a teacher assessment rather than as an
external exam. In relation to the end of Key Stage tests, there
is also an issue in every assessment of manageability. If we go
back far enough in the history of the testing regimes, at key
stages there was experience of using teacher assessment. That
proved, in the early days of the national curriculum, very unmanageable.
It meant that we were losing huge amounts of teacher time to moderation,
which was not proving terribly effective. It is about trying to
judge the best kind of measurement—the manageability of
the measurement and the validity of the measurement.
Q344 Mr. Chaytor: In getting an accurate
picture of a child's ability in year 6, is it not more valid to
have the result of teacher assessment throughout the year as well
as an external test, rather than relying simply on the external
test?
Chairman: What is the point of personalised
learning?
Jim Knight: That is the excitement
of the progression pilots. The current situation with SATs is
that everyone takes the test and then the examiner decides which
grade you are at based on your response in that test, whereas
the single level test is a scenario whereby the teacher assessment
informs whether the child is ready and what level the child is
put in for, so the test is used as part of teacher assessment
for learning, rather than sitting alongside it as it does at the
moment.
Q345 Mr. Chaytor: My other question
is this. In moving away bit by bit from the regime that was inherited
in 1997, will you accept that there is a link between a very rigid
testing regime and disaffection and demotivation among children
who do not perform well under that kind of regime?
Jim Knight: I think that it would
be a very tenuous link. You see schools that are performing very
well in very difficult circumstances. Obviously, part of what
they are doing in performing well is that a large number of their
pupils are doing well in tests. Why are they doing well? Which
comes first, the chicken or the egg? I think in this case it is
getting the behaviour, ethos and atmosphere in the school right,
and getting people focused on their learning, which means that
they are not disengaged. What then subsequently happens is that
they do well in their tests, but my own feeling would be that
you would be getting it the wrong way round if you said that because
they are not doing well in tests, they are disengaged.
Q346 Mr. Chaytor: In any high-stakes
testing system, 80% pass, but 20% fail. I am interested in whether
there is any link between the sense of failure and loss of self-esteem
of those who so publicly fail, and disaffection in the early years
of secondary—Key Stage 3—which is a major concern
of the Government.
Ralph Tabberer: First I question
the premise that these are, in conventional terms, high-stakes
tests. We normally talk about high-stakes tests as determining
for pupils the school they go on to within a selective system.
Within our assessment history, if we go back 20 or 30 years and
look at tests such as the 11-plus, those might legitimately be
called high-stakes tests for children, because they were so determining
of the next stage. We have got medium-stakes tests for our students
that allow them to show what they can do and give them and their
parents a sense of where they are. They also happen to give us
very useful information, as Mr. Slaughter has indicated, for policy
development and accountability. The Minister is right to point
to the progression tests as an interesting experiment. What we
have been keen to do is to offer Ministers alternative approaches.
We have listened like you to comments over the years about the
possible downside of what you termed rigidity. We have listened
to people talking about the possible effect on the year 6 curriculum
and the possible effect on the pace of the whole key stage. So
looking at a progression test as an alternative, the idea of actually
being able to draw down a test to be ready on time for pupils
may give teachers and pupils a different environment. We think
it is appropriate to pilot that, but we do not think it appropriate
to rush for that solution. We want to give Ministers alternative
options, so they can deal with just that sort of question.
Chairman: We now move on. John Heppell
will lead us on the notion of centralised control and validity
versus reliability.
Q347 Mr. Heppell: Looking at how
the Government are addressing A-levels and Diplomas, people might
think that there has been a subtle change, in that whereas we
were moving towards greater reliability at the expense of validity,
there has been a slight move the other way. Many universities
have said to us that people come to them without a sufficient
breadth of skills in a particular subject. We have heard from
examination boards that, instead of being closed, the questions
are now, if you like, opened out, or open-ended. Obviously there
is a consequence to that, and I know that such things are finely
balanced, but can you confirm that there is a move to ensure that
validity goes up a bit in the rank, as against reliability?
Jim Knight: We want both, obviously,
and we will continue to evolve and improve the A-level as we introduce
the Diplomas. We are mindful of the criticism, which we have heard
from both employers and universities, that people may be well
versed in the particulars of their subject, in which they perhaps
took an A-level, but they need to do better in terms of some of
the wider, softer skills. That is why we are introducing measures
such as the extended project into A-levels. That is also why we
have introduced personal learning and thinking skills and why
the work-related learning that runs through the Diplomas in order
to offer what both universities and employers are saying they
want more of from our young people.
Q348 Mr. Heppell: Moving on from
that slightly, you now have what is called controlled internal
assessment. However, we are told by our advisers that there has
always been controlled internal assessment. You have not actually
taken the coursework into account in assessing that. You are taking
an add-on to the coursework and assessing that. Is that not the
case? Are you not interfering, from a centralised position, with
what should be a creative process? I understand that you set down
the guidelines fairly rigidly in respect of what the controlled
internal assessment—the very name says it—does. Is
it a lack of faith in teachers?
Jim Knight: No, I do not think
that it is a lack of faith in teachers. Again, we had to design
an examination system that retains the confidence of everyone
that it is serving and those involved, including the pupils—most
importantly—the teachers and parents, the employers and
those in further and higher education. We found that an over-emphasis
on coursework in some subjects was problematic, so we have moved
away from that. The use of controlled internal assessment is,
perhaps, a halfway house between the examination hall one afternoon
and the continuous assessment of coursework. Ralph, do you want
to add anything to that?
Ralph Tabberer: Again, I think
it is a question of looking at different subjects and seeing which
is the right design that suits them. There are some subjects for
which coursework is a natural or highly desirable assessment method—art,
for example. There are other subjects for which external assessments
work almost fully. For example, we have moved more of maths into
that realm. We have been trying, with the controlled assessments,
to create a more controlled environment where it is more likely
that the assessments made by one teacher will be replicated by
another. That addresses public questions about coursework, as
the Minister suggests, and about the possibility that there is
variability, which affects GCSE results.
Q349 Chairman: Is this an endless
search for an accurate method of evaluating the teaching and the
quality of knowledge that the child assumes? It is endless, is
it not? Does it not squeeze out the thing that John is pushing
you on: the creativity—the breadth, depth and the imagination
of it? The Welsh have got rid of it. Are they struggling because
they have got rid of this testing?
Ralph Tabberer: Any assessment
system is a design where you are trying to balance validity, reliability
and manageability. You try to get the best design for your whole
system. I think that we have been very consistent, actually, with
the principles that were set out at the start of the introduction
of the national curriculum assessment. We have tried to stick
to those principles in measuring and giving parents and pupils
information about what they can do. We have made changes when
there has been a build-up of concern and we have felt that it
has not been possible to answer that. So we have moved when things
have been unmanageable. We have not been inflexible. The basics
are still there. Again, if we find better ways of assessing, we
will put those options to Ministers. I suppose that one of those
areas in future will be IT-delivered testing. We should certainly
keep our eyes open for alternatives that give us the best balance.
Q350 Chairman: Is there a horrible
generation in the Department that read, as I did as a young man,
a thing called "The One Minute Manager", the central
theme of which is that, if you cannot measure it, you cannot manage
it? It seems to me that the Department is still desperate to measure
all the time. They do not measure so much in the independent sector,
do they? That is not the way they get good results, is it? Not
through constant measurement.
Jim Knight: I am a product of
an independent school where I was tested an awful lot. That is
part of a traditional elitist education, I think. It is an endless
pursuit because the economy is ever-changing and we are ever-changing
socially. The environment in which schools and the education system
are operating is ever-changing, and education has to respond to
that. It therefore has to keep changing. The environment that
I grew up in and in which I went to school was one in which, if
you were lucky, 10% went to university. However, skills needs
have changed, as we discussed in other evidence sessions. We therefore
need to change the qualifications to respond to that change—and
as you change the qualifications, you change the forms of assessment.
Q351 Mr. Heppell: Have you actually
come across problems on the ground? Has somebody that is studying
or somebody that is teaching said, "Look, it doesn't work
like this. We need to have more flexibility in the way we deal
with it"? Are there problems on the ground?
Jim Knight: Specific problems?
Mr. Heppell: Problems specific to the
controlled assessment rather than just an assessment of coursework.
Chairman: Ralph, perhaps you should answer
that. You used to be in charge of teacher training.
Ralph Tabberer: I am trying to
think of any particular cases where people have brought up additional
problems relating to controlled assessment, but I cannot think
of a piece that does that. In general, though, I am clear that
we do monitor the impact of assessment; we monitor not only the
impact on pupils and schools but the opinions of different parties.
We keep that in view, and as I tried to say earlier we are willing
to change, and when we can we put alternative proposals to Ministers.
We think we have something with the progression tests that might
give an alternative approach, and Ministers have been quick to
say, "Well, let's pilot it. Let us not implement it until
we know more about how it might impact." That is all evidence
of us being intelligent and open. We keep on looking for improved
solutions, but not moving away from the basic principles on which
the current model was developed.
Q352 Chairman: The desire to measure
is not driving out the imagination and joy of education?
Ralph Tabberer: It should not,
no.
Jim Knight: I thought of this
Committee when the announcements were made last week around the
culture of entitlement and the importance of greater partnership.
Similarly, we have a commitment to sport in the curriculum that
we are developing, and we have a cooking announcement. Some of
these things are not that easily measured. The pilots on culture
will test how easy it is to measure the five hours in the curriculum.
We are ever-evolving about this. Some things are much easier to
measure than others, but the pressure that I was talking to John
about in respect of employers and universities around the softer
skills is more difficult to measure—but that does not mean
that we are not committed to trying to work harder to develop
that.
Ralph Tabberer: Of all the schools
that I have visited, I cannot think of one that is immensely creative
that is not also interested in the tests and doing their best
by them. I cannot think of a school that I have visited that does
well in tests that does not have a strong creative side as well.
Sometimes we set these aspects as alternatives, but I do not think
that that is always fair. There are plenty of schools that manage
to do both very well indeed. They are well led, and they know
what they are doing. There is plenty of evidence that you can
have creativity, a lot of autonomy and a lot of self-determination
by teachers and that you can have a properly assessed system that
gives parents a good account of what is happening in the school.
Chairman: Let us drill down into testing
and school accountability with Annette.
Q353 Annette Brooke: I want to look
at whether the test is fit for all the purposes that we try to
use it for. I do not think anyone would disagree that there should
be some form of measurement of pupils' progress. But is not the
difficulty that the Government are placing so much reliance on
a test that was designed for one purpose but which is now being
used to measure whole school performance? Do you not have any
concerns about that?
Jim Knight: If it were the only
measure of school performance and the only aspect of accountability,
one would have to be concerned that we were putting all our eggs
in one basket. But we are not. We have inspection and we look
at performance in other areas. We only test a few subjects through
the SATs. We are still looking at how people are doing on other
things. We also have national strategies working in some other
areas. So I would say that it is a critical measure but it is
not the only measure. Therefore, I am happy with how it sits.
Q354 Annette Brooke: Many parents
will focus only on this as a measure. Personally, I can feel fairly
relaxed that a local authority is looking at the whole school,
because it might set off some warning signs that need to be dipped
into. However, the majority of parents are not going to dip below
what they see in their local newspaper. Therefore, do you not
think that this measure is harmful to the idea of parental choice?
Jim Knight: It goes back to what
I was saying before. We do not publish ranked tables; the newspapers
choose to rank locally what we publish. I have spoken to various
people who work in the media who were a little sceptical about
whether they should publish the tables, but when they saw how
well rival newspapers sold when they published them they soon
leapt at it and published them too. There is no doubt in my mind
that if we did not publish the tables someone else would. As I
said before, if we as a Department do it, we can be scrutinised;
the process is carried out independently using national statistics,
and we know that it will be done objectively and fairly, rather
than someone who is not subject to as much scrutiny being able
to do it.
Of course, our local newspaper, The Dorset
Echo, regularly reports the Ofsted results of schools as and
when they achieve them. They usually publish the successes and
the pictures of celebrating pupils, teachers, head teachers and
governors, rather than those schools that get satisfactory ratings,
but obviously they will occasionally run stories on those schools
that are not doing so well. I think that those stories, along
with what parents say to each other, are as informative of parental
choice as the tables.
Q355 Annette Brooke: I would still
disagree that parents have full information. For example, someone
told me recently that a certain selective school has 100% five
A to C grades at GCSE, and said, "Isn't that fantastic?"
That is the perception, is it not, because of the way that league
tables are presented?
Jim Knight: Again, I cannot be
accountable for how league tables are presented in every single
newspaper. We publish the contextual value added as well as the
raw scores. We are now moving to progression targets, so that
we are looking at the proportion of schools that progress children
through two levels for each key stage. So we will be reporting
a number of different things. We are putting a science and language
into the indicator, so that all that data will be available in
the attainment and assessment tables. However, we cannot tell
newspapers how to report the tables.
Q356 Annette Brooke: May I just dig
in to the Contextual Value Added measure? We have had varying
evidence on this measure. I think that there was one throwaway
comment that it was really just a measure of deprivation. There
is also the issue that not all parents will understand the significance
of the measure. Is there more that you could do as a Department
to make the measure stack up better and be genuinely more informative
for parents?
Jim Knight: I would not say that
the measure is perfect, and I will let Ralph give the technical
answer on CVA in a moment. However, one of the reasons why I have
been pushing on the progression measure is that it is slightly
easier for people to get their head round, as to how every single
pupil is progressing. So it is not a threshold but something that
applies across the board. I think that that will help in respect
of the concerns that you raise.
Q357 Annette Brooke: I would like
the answer to my question. However, I would like to pick up on
that particular point about progress. An admirable school may
have a high percentage of children with special educational needs.
Up to 40% of its children may have special educational needs.
It is quite likely that those children will not be able to progress
more than one level in the standard assessment tests over a given
period. If you use that information across the whole school it
will add even more distortion to the picture.
Jim Knight: I am not sure whether
there will be any more distortion than there is at the moment.
It is a reasonable criticism. When we introduce the foundation
learning tier, which can accredit progression and learning below
Level 1 in national vocational qualification terms—it is
very confusing having national curriculum and NVQ levels—we
may be able to look at whether that can be reflected. At the moment,
if you have a high proportion of children with SEN, you will not
do as well in the raw scores as those schools with a lower proportion.
Ralph Tabberer: The most important
thing is to see the Ofsted inspection as the top of the tree.
For parents, who are your concern here, the Ofsted inspection
is probably the most rounded, richest and most comprehensive assessment
that they will get of a school's strengths and weaknesses. I would
always point parents to that assessment as the best thing to consult.
When we publish results, we try to ensure that the public can
see raw results and that they can look at comparators and benchmarks.
We have had a lot of discussion about which way to approach value
added. In our consultations on that, we have settled on contextualised
value added as the most fair. In trying to publish series of data,
we are following the principle of being transparent about all
of the analyses so that parents can access the information that
they understand or the information that they want. I have to say
that we get very few complaints about the testing regime.
Q358 Chairman: That is because they
cannot understand a word of it. You have to understand it to be
able to complain about it.
Jim Knight: They could complain
that they cannot understand it.
Q359 Chairman: That is true. Come
on, look at your site. Get a group of parents to look at the site
and evaluate how much they understand the contextual value added
score. It is very difficult to understand. Why present it in that
form?
Ralph Tabberer: Equally, why hold
it back? What I am saying is that we know that many parents consult
Ofsted reports. We know that in putting those Ofsted reports together,
the school, in its self-evaluation, and the inspectors will draw
on all those analyses. There is no reason for us to hold back
those analyses. What we do is make them transparent.
| 
