Examination of Witnesses (Questions 128
- 139)
MONDAY 14 JANUARY 2008
DR MARY
BOUSTED, MICK
BROOKES, BRIAN
LIGHTMAN AND
KEITH BARTLEY
Q128 Chairman (Fiona Mactaggart):
Good afternoon, everyone. We have a rather interesting situation
here. We are missing our Chair, because this session coincides
with the debate on the Education and Skills Bill in the House,
so I have agreed to act as Chair. We are also missing a bunch
of witnesses. I am afraid that Steve Sinnott of the National Union
of Teachers and Chris Keates of the National Association of Schoolmasters
Union of Women Teachers pulled out. They would have provided two
thirds of the witnesses in the second part of this evidence session,
so we have decided to put together all four witnesses who are
here—thank you for agreeing to this, Mary—for one
slightly truncated session. It is important that we speak to the
NASUWT and the NUT, and it is striking that in its evidence the
NUT specifically asked to come before the Committee to give evidence.
We want to speak to senior officials, and not to junior substitutes,
so we will arrange an alternative date for them to appear. In
the meantime, in this session—I imagine that it will finish
at about 5.15 pm if that is convenient for all our witnesses—we
will look at testing and assessment as part of our inquiry. It
is usual for the Chair to offer witnesses an opportunity to make
brief preliminary remarks about the issues before them, which
can help the Committee to zero in on its main concerns. If any
of you would like to do that, I would welcome your contribution.
Dr Bousted: The key issue for
this Committee is that proposed by Dylan Wiliam, who said that
the challenge that we have as a country is to have tests that
are worth teaching to. At present, the view of the Association
of Teachers and Lecturers is that we do not have tests that are
worth teaching to. The current testing system is highly unsatisfactory.
Some 30% of pupils will be awarded the wrong level at Key Stage
tests. That is an issue for standard assessment tests and GCSEs.
Another issue is that, because of over-teaching to the tests,
six months on from being tested at Key Stages 2 and 3, 25% of
children do not maintain the same level. For a Government who
are keenly interested in raising pupils' standards and system
levels of attainment and achievement, that is not good. What is
striking from the evidence that has been presented by the people
representing our organisations is the degree of consensus in the
submissions. There is consensus that tests are used for too many
different purposes, and because of that their value is corrupted.
There is consensus on the inadequate relationship between the
national curriculum and the tests. In other words, the tests cover
very narrow aspects of the national curriculum, which leads to
worries about validity. There is also striking evidence that because
we test seven out of 11 years of compulsory schooling, there is
a demotivating impact on pupils, which leads to a very instrumental
view of learning. I was interested to read in The Times Higher
Education Supplement that this instrumental view of learning
is even affecting the most academic pupils—those who go
on to higher education. They arrive at university without the
necessary research skills and skills for independent learning,
which then have to be taught in the first year of university.
Therefore, the tests have a severe effect on all children in the
curriculum. Even in the Government's own terms, the tests do not
do the job and, more significantly, they militate against assessment
for learning, which we need to encourage. This is a highly significant
inquiry. I am glad that the Select Committee wants to consider
the matter. I know that you were going to do it and then the inquiry
was halted and you will come back to it. It is highly significant,
and we will await your final report with interest because you
are commenting on something for which the public perception is
now changing. We are coming to an interesting time in the assessment
and testing debate. There is beginning to be more of a clamour
to do things differently.
Mick Brookes: I am very pleased
to be here as well. It is important that we get beneath the headlines
of what all the associations have been saying. The impression
that the teaching unions are against assessment is palpably not
true. We are for assessment, but it has to be assessment for the
right reasons and with the right instruments. If we do not have
that, we end up, as Mary said, corrupting the curriculum. You
should have received the book from the Commission on Testing by
the National Association of Head Teachers.[3]
In that Commission, views were gathered from the wider community,
and not just the teaching community. It included views from the
National Confederation of Parent Teacher Associations and governing
bodies. It was not just our association that was represented.
Anthony Seldon stated: "Children are encouraged to develop
an attitude that, if it is not in the exam, it doesn't matter.
Intellectual curiosity is stifled and young people's deeper cultural,
moral, sporting, social and spiritual faculties are marginalised
by a system in which all must come second to delivering improving
test and exam numbers." That is where we are. I know from
my colleagues that it has altered the curriculum, particularly
in the primary sector but also in the secondary sector. I am sure
that Brian will say more about that in a minute. It is timely
that we come to this now to look at where we go to continue to
raise standards in education.
Brian Lightman: What is interesting
is the degree of consensus that is already here. I have heard
nothing that I disagree with and would not have wanted to say
myself. That is a very important message: that we really do feel
strongly about this. I want to home in on two things. The first
is the examinations system, which has become so costly and complex
and is at a point that is completely unsustainable in its current
format. There seems to be an assumption that everything has to
be externally assessed, which is having all kinds of implications
in terms of what we are doing in school, what we are doing for
the children, the pressure we are putting them under and the disaffection
that we are causing as well as the unhappiness and stress of children.
That sounds as if I am going to speak against assessment and I
am certainly not going to do that. Like the NAHT, we are far from
opposed to assessment. In fact, we are saying that assessment
is an integral part of the teaching and learning process. It is
absolutely the bread and butter of what every teacher does. It
is strange that that aspect of our work has almost been taken
away from the professional skills of teachers, because it has
been handed over to people outside the classroom. I want to talk
about the problem with that. True assessment for learning is something
that we are genuinely excited and passionate about as school leaders
because, when you introduce those types of technique, you can
see immediately improvements in the motivation of the students
and quite enormous improvements in the quality of the learning
that goes on. That is a terribly important aspect of what we are
doing and we need to re-professionalise teachers and train them
so that they can use those methods in their teaching. That would
have an enormous impact on things like low-level disruption in
the classroom, the motivation of students and the progress that
they make. ASCL is providing in its paper a proposal for chartered
assessors that we see as a solution to the problem. We do not
want just to talk about a problem. We are saying that we understand
the need for assessment for accountability and we understand that
assessment needs to be robust, valid, reliable and so on. Therefore,
we propose a model whereby teachers can be trained in their skills
and assessment and we can build that into our work. That would
be much better value for money and a much more efficient system.
I could say a lot more about that, but by way of introduction,
that will do for the moment.
Keith Bartley: I became Chief
Executive of the General Teaching Council in March this year and
one of the first things that impressed me was the range and the
nature of the evidence and research work with teachers and with
parents that underpinned our submission to you. I hope that you
have had access to that. The General Teaching Council was founded
and exists in the public interest, and I share the consensus that
you have heard about already. We feel strongly that this country
needs an assessment system that more effectively supports learning
and promotes higher achievement. We are very much here in terms
of a statement of intent to help the Government to find a means
by which benchmarked information about schools in the public domain
is valid, reliable and illustrative of the progress made by children.
I will not go through them now, but all our proposals were submitted
with that objective in mind.
Q129 Chairman: Thank you all very
much. It could be understood that the arguments that you have
all made—I doubt that this is what you believe—are
against all forms of externally moderated examinations. I would
like you to talk about where you feel externally moderated examinations
ought to fit into the system and why.
Mick Brookes: You are quite right;
that is not that case. We think that we should place greater reliance
on teacher assessments, as Mary said, but it would need to be
moderated. I know, from my experiences as a head teacher, that
you can have two teachers with a parallel year group, one of whose
glass is half-full and the other whose glass is half-empty, and
who might assess something such as writing, for instance, which
has a degree of subjectivity about it, at different levels. There
needs, therefore, to be something there. The Scottish system is
worth looking at. They have a bank of benchmarked tests, from
which schools can draw, in order to check on the validity of teacher
assessment. We are not against external assessments; in fact,
it is important to have some benchmarking. Nobody in our association
wants to return to the 1970s when you did not know what the school
up the road was doing, let alone a school at the other end of
the country. There needs to be some benchmarking and an idea of
where schools should be, but we are saying that we need to test
for the right purpose. The current testing regime is used for
far too many purposes.
Brian Lightman: I am sure that
we would agree with that. There is certainly a place for external
assessment, which will increase as you go higher up the age range.
We do not suggest removing A-levels because they are important
external benchmarks, but there should be an appropriate range
of assessment methods. However, going further down the age range,
we need to think whether it is really necessary for material at,
say, Key Stage 3, to be marked externally, bearing in mind that
it is marked by the same people as those in the school at the
time. Does everything need to be externally arranged? Do we need
a system by which we send things away? Given technology, should
we not, as the NAHT suggested, adopt a system by which, for example,
you could download assessment material? We should use new technology
to download new material and use it when we are genuinely ready.
That does not mean that everybody does the same test on the same
day and in the same room, but that when you are ready, you draw
down those resources to a certain standard.
Q130 Chairman: Sorry to interrupt
you, but is that not exactly what is proposed in the single-level
test?
Brian Lightman: No, I do not think
that it is. At the moment, everybody across the country is doing
the same test on the same day, so it is still an external test.
We suggest having a bank of assessments, which the professionals
should be trusted to draw down and use. When a class, or group
of students within a class, is ready to be assessed, they could
draw down those materials and apply the assessments to those students.
We would have to ensure the appropriate external moderation. That
could be helped by the model that we put forward of chartered
assessors, whereby qualified people moderate both within and outside
the school.
Q131 Chairman: In St. Cyres school,
which you headed, did you find that occasionally you would pick
different examination boards for different subjects, because of
questions about whether a board is easier in some subjects than
in others?
Brian Lightman: I am sure that
that has happened in every school in the country.
Q132 Chairman: I am not picking you
out, but just asking for your personal experiences as a head teacher.
Brian Lightman: In my experience,
in all of my schools, including when I was a head of department
in Surrey, we would change our syllabus, partly depending on how
we felt that we could get the children through exams. That is
bound to happen in a culture in which everything is looked at
and accountable. We should be choosing assessment materials that
reflect the kind of teaching and learning that we want to have.
Given that you mentioned St. Cyres, I should add that it is in
Wales, where we do not have Key Stage 3 tests. Interestingly,
given the changes there, there is now a genuine debate among heads
of different subjects about what constitutes effective learning
at a particular level within each subject. Heads of department
of different schools are getting together and really thinking
about that moderation process in a way that I have not seen in
the past 15 years or so.
Q133 Chairman: It does not seem to
be producing better results in Wales and the rest of the country,
but nevertheless—
Brian Lightman: Well, I think
it is.
Keith Bartley: Going back to your
question about external moderation, our research tends to suggest
two things. One is that we think that public exams should be more
about learning, which means that what is examined needs to be
broader, and that more account needs to be taken of how they represent
what has been learned. The second is linked to that. At the moment,
most of our public exams have extremely high stakes. They are
used for many purposes—that is the point that Mary started
with. An externally moderated examination tells us how well a
young person has achieved, comparatively. The scores are then
aggregated to give us a sense of how well a school has done, and
aggregated further to give us an idea of how well young people
of a particular age have done across the country. That multiplicity
of uses to which a single examination is put represents stakes
that are too high, and it tends to subvert part of the original
purpose of evaluating learning.
Chairman: Andy, perhaps this is the moment
at which you would like to come in.
Q134 Mr Slaughter: Yes, I was very
interested in what was said at the beginning about the types of
test and whether they are of a good standard. I shall come to
that in a second, but first I shall return to an even earlier
stage and see whether I have understood what you are saying about
testing in general.From my lay understanding, you are essentially
talking about two different types of testing. The first is testing
that is internal to an institution—the type of testing that
I remember from when I was at school, which is a tool for teachers
to use on their pupils to determine whether they are progressing
and learning according to the curriculum that they are being taught.
I would have thought that it is also used to encourage them to
learn, because it provides an incentive, rather than their staring
at a blank piece of paper in a test. I assume that that still
goes on; it went on a lot when I was at school. I assume that
you do not object to it. National tests—I am not so much
talking about exams such as A-levels, which have been mentioned—seem
to perform a wholly different function: to test whether an institution
and its teachers are performing. Do you see testing as I have
just explained it, and, if not, what is your analysis? Do you
think that the first type is good and the second bad?
Dr Bousted: Well—
Chairman: Sorry, do come in there.
Mick Brookes: We are very well
behaved.
Dr Bousted: Yes, we are. We are
not going to speak without the teacher letting us speak. If your
statement was right and the national tests were used to decide
how good an institution was, that would cut down their purpose.
You might be able to look at national tests that decide how good
an institution is, but that is not the case. The national tests
are used to give the individual performance of each child. Our
argument is that in looking at an assessment system, we must consider
two things: is it valid—testing the key, essential core
abilities in a subject that are defined in the national curriculum—and
is it reliable? ATL's contention is that the current system is
neither valid—it does not test the essential core attributes
of a subject—nor reliable. I return to the fact that up
to 25% of children, maybe more, get the wrong grade, which has
profound consequences for them. Also, a child who only just gets
a Level 4 and one who nearly gets a Level 5 might be at very different
stages in their learning. It is a broad brush stroke. The other
problem is that although tests are meant to be used to give individual
level data, school level data and so on, at Key Stage 2, which
has one of the most pernicious stages of testing, the results
are given far too late to be any good for a child. The child goes
on to secondary school, and secondary schools do not believe the
grades because they think that primary schools train children
to take tests. Indeed, independent evidence from the QCA proves
that to be true. The children are then retested at secondary school
because there is no confidence in the grades given by primary
schools. The idea that the national tests are used just for the
national picture is not right. The problem for children is that
if you are told at seven, at 11 and then at 14 that you are not
very good, it is perfectly logical to say, "Well, if I'm
not very good, I won't try." If individual children are told
that they are not very good and not told why they are not very
good—they might actually be quite good, but they might have
been given the wrong grade—that will have a profoundly pernicious
effect on lots of them as individuals. So, I would contend with
your outline statement that the two types of testing are for two
completely different purposes. I do not think that is the case.
Q135 Chairman: How much are the figures
that you are quoting a reflection of the fact that the levels
and the curriculum are different at Key Stage 3 and Key Stage
2, although the assessment levels are the same?
Dr Bousted: What, if you get a
level—
Q136 Chairman: If you get a Level
4 in the Key Stage 2 test and a Level 4 in the Key Stage 3 test,
you are being tested on a different curriculum, are you not?
Dr Bousted: Yes, you are, and
it does not mean the same thing. Level 4 at Key Stage 2 and Level
4 at Key Stage 3 do not mean that progress has not been made.
You are being tested against a different curriculum and a different
assessment framework.
Q137 Chairman: Were you quoting those
figures to suggest that the original test was wrong? I was wondering
whether the figures that you were quoting were a reflection of
the different tests, rather than the wrongness of the first result.
Dr Bousted: The figures that I
am quoting are a measure of the confidence that you can have in
the fact that pupils are getting the right grade in the tests
at each level. The reason why there is such a problem in the level
of confidence is this. At 11 and 14, we assess whether a child
is proficient in English, maths and science through pencil-and-paper
tests. That has particular problems for the science curriculum,
because you can test only a very narrow element of the science
curriculum with pencil and paper. There are also huge problems
with the English tests at Key Stage 3. There have been problems
with their validity and reliability since their inception, and
there has been a lot of political fury about them. The issue in
terms of validity is whether the test actually relates to key
concepts in the national curriculum. There is an argument that
it does not, because what you can test with the test items is
so narrow. That means that although a child might get a certain
level in a test, it might not be—our argument is that, too
often, it is not—reflective of their ability. That is equally
damaging regardless of whether that goes up or down—whether
they are assessed at too high or too low a level.
Mick Brookes: To come back to
your question, some proof of the value that the profession attaches
to the results of testing can be seen in the number—QCA
will provide this data—of year 3, year 4 and year 5 tests
that are purchased by schools to check on progress and teacher
assessments at the end of the year. The difficulty with testing
is not so much with those things that are easy to test, such as
mathematics and comprehension. To pick up what Mary was saying
about the validity of the tests, there can be really interesting
variations, with the same teacher at the end of primary school—and
perhaps all the way through—scoring something like 85% with
their children in a reading test, but only 75% or less for writing.
The variation in the national levels achieved in reading and in
writing, which are often misunderstood by the press, is huge.
Why is it that these results are so different if the same teacher,
with the same skills, is teaching the same children for all those
tests? I think that it has something to do with the assessment
of writing. To take just one example, I know of an extremely good
school that had very good writers, but the whole year group misunderstood
the genre of the writing that they were supposed to be producing
for Key Stage 2 SATs and none of them achieved their levels. That
meant that the school was in deep trouble with the inspection
system—quite unnecessarily so, given that there is over-reliance
on the results of testing and not enough attention given to teachers'
assessment of the actual ability of children, who, in this case,
just made a mistake on a particular day.
Keith Bartley: I would like to
go back to the premise about the assessment that is undertaken
to inform learning and the assessment that is undertaken perhaps
to give information about the effectiveness of a school. Parents
told us very clearly that they felt that the information that
was published about tests in a school was about the school justifying
itself publicly in terms of its place in the national pecking
order. Actually, the information that they valued about how well
their pupils were doing was that which teachers gave them, which
was very largely drawn from the teachers' own interactions with
the pupil, whether that information was in written format or—most
particularly—they were given the opportunity to talk to
teachers about how well their children were doing. As well as
questioning validity, we would question the utility of those tests
in terms of the audience.
Q138 Mr Slaughter: That was really
where I wanted to go. I was a little concerned about a comment
that was made about failure. Obviously, one does not want to label
children as failures, but I assume that it is common ground that
we regard testing as part of teaching and learning and an essential
tool, and that therefore there are going to be people who succeed
or fail—that is what happens as a consequence. I want further
comment on that, but I would have thought that it was a starting
point. Accepting what you just said about the validity of different
types of tests, do you think that there is any validity to national
testing in that way, or can the positive aspects of testing simply
be dealt with at school level, with the judgment with respect
to institutions being dealt with in other ways, such as through
Ofsted?
Brian Lightman: I think that one
of the problems with national testing is that you are applying
it across the board and taking a snapshot of everybody at the
same time, and there are other ways of sampling what progress—what
learning—has taken place, if we want to have those national
benchmarks. That is one of the things that we proposed in our
paper. We used to have the assessment of performance unit, which
sampled children's progress in different areas and looked across
the whole country at one aspect of their learning. By doing that
you can really see what children have actually learned, rather
than trying to test across the board by giving everybody the same
test on the same day and trying to cover everything, which, of
course, you cannot possibly do in an hour, or an hour and a half.
The other point I want to make is that testing is only a small
part of assessment. There are other very valid and effective—and,
in fact, proven—methods of assessment, like approaches to
assessment such as externally moderated portfolios. Within things
like BTEC at the moment there are some very successful models.
The people who are doing that assessment have to be trained and
accredited, and they have to meet very rigorous standards. You
are then able to assess the work of students over a longer period,
within that very rigorous framework, to make sure that you are
not just doing a snapshot on a particular day. There are all kinds
of things that come up, and we have experienced them over the
years. Every teacher will tell you about results of tests that
they have been mystified about and when they just do not understand
the result that a student got in the test, given what they have
seen every day in the classroom, because obviously they see the
child over a longer period of time. Children get nervous in tests
and underperform in tests, and so on. I think that we have to
be very careful about how much credence we attach to one method
of assessment.
Mick Brookes: Just on the headlining
of what happens, I think that children who have overcome significant
special educational needs and have reached Level 2 or upper Level
3 at the end of Key Stage 2 are what we have called the invisible
children. They do not appear. While people say, "Well, they
are in the contextual value added tables", what newspaper
picks those up? What is reported is simply those children who
have achieved Level 4-plus at the end of Key Stage 2, which also
gives a misleading view, so this is not just at a pupil basis,
it is also at a schools basis. I have the permission of head teacher
William Ball to tell you this: New Manton primary school in Worksop,
Nottinghamshire, has always been down at the bottom end of the
league tables because they are norm referenced. You nevertheless
get ill-informed people saying that it and others like it are
failing schools. Here are three sentences from New Manton school's
Ofsted report: "The excellent leadership of the head teacher
is largely responsible for the good level of improvement in all
areas of school life ... The staff show a strong commitment to
the personal development of individual pupils and, as a result,
most make good progress ... The very effective governing body
is showing an equal determination to bring about change for the
benefit of all pupils." The school is good, but it is down
at the bottom end of the league tables, so there has been a distortion
of fact. There are very good schools that work against the odds
to produce higher educational qualifications than they have ever
had in their areas, but they are disabused of that excellent work
on an annual basis.
Q139 Mr Slaughter: I have a lot of
sympathy for what you are saying—I am sure that we all know
of similar schools in our constituencies. In shorthand, there
are good schools that provide a good level of education, but are
not in the top quarter of the tables—they might even be
in the bottom quarter. I am asking about very basic stuff, and
I shall shut up after this, but I want some clarity. I felt that
by dissing everything about the tests—their quality, reliability
and so on—you were not confronting the question of whether
we should get rid of them altogether. Obviously, there must be
some way in which to assess institutions; there are many ways,
but there must also be some oversight—I used the example
of Ofsted. What would you like to see? I would like you to confirm
that there is a positive role for testing in schools, including
primary schools. Are you saying that testing should be conducted
entirely within an institution? What would you like to see done
about external accountability, including nationally?
Chairman: Each of our witnesses would
like to respond to that question, so you should all be quite brisk.
Keith Bartley: There will never
be a time at which information about testing in schools is not
in the public domain and viewable. We accept that and, indeed,
we support it, for comparability purposes. However, the information
conveyed by tests should be accurate and valid in terms of what
they measure, and tests should not distort the curriculum and
learning. At present, the multiplicity of uses to which a single
test is put narrows the curriculum and distorts the outcome. To
pick up on the question that the Chairman asked Dr Bousted earlier,
Dylan Wiliam's view is that some of the fall-off between Key Stages
2 and 3 occurs because most of year 6 is spent drilling youngsters
for Key Stage 2 tests, and they forget completely over the summer
because they were coached only to climb that hurdle on that day.
Removing the high-stakes nature of the testing will be valuable
in future.
Brian Lightman: We are not arguing
that you should get rid of testing. As others here have said,
we are concerned about how test results are used: they produce
simplistic league tables and feed misunderstanding. That has been
evident in the coverage that we have seen in the past week. The
publication of the league tables has been completely misleading
as to the actual meaning of the tests. There should be testing.
Using a model whereby people can download high-quality assessment
materials and use them at the right time would measure different,
important things.
Mick Brookes: I agree with my
colleagues. There is already a company that does online testing.
Children take the test online, and they are assessed not only
at their own level, but that of the whole cohort. There is already
a quick and easy expedient. There must be testing, and it must
be nationally benchmarked—otherwise, schools will not know
where they are—but I agree that the testing system goes
wrong because of the multiplicity of purposes to which the tests
are put.
Dr Bousted: Yes, assessment including
testing is a key part of the repertoire that teachers must have
at their disposal, but that does not go for the tests that we
have at the moment. The tests must be as valid and reliable as
they can be but, at the moment, our testing system is corrupting.
It does not just corrupt the results; it corrupts all the other
things that teachers are trying to achieve, like a broad and balanced
curriculum, a varied menu for pupils, and valid assessments of
where pupils are and what they can do. You said that pupils have
to experience failure. At some point, yes, they do. At some point
there has to be sifting—a proper sifting. Children and young
people, no matter how they look on the outside, are fairly fragile
on the inside, just like the rest of us. What is more important
than failure—failure does nobody any good in the end—is
that pupils need to know where they are now, and what they need
to do to be better. They need to know where they are at. They
do not need to know that they have failed. In the end, failure
does not get anyone anywhere—they just fail. It does not
teach them how to do better. What they have to know more of is
why they are at a particular stage and what they need to do to
get better. I would say that the over-emphasis on testing means
that where a child is now and what has to be done to enable that
child to learn better is the most undeveloped aspect of our education
system. It is one of the reasons why, in the PISA league tables,
we are not performing as we should. We have one of the most undeveloped
systems of assessment for learning among developed countries.
It is parlously poor in our country.
Chairman: On that note, I am going to
ask David to speak.
3 Commission of Inquiry on tables, targets and testing Back
|
