Memorandum
submitted by Professor Peter Tymms,
Durham University
1. Summary:
The questions posed by the Inquiry are vital to the future of our educational
system, but it is clear that most of them cannot be answered satisfactorily
given our present state of knowledge. What follows is a justification of this
statement and a suggestion for a way forward.
2. Background
and expertise: Professor Peter Tymms
is an educational researcher based at Durham University
where he directs the Centre for Evaluation and Monitoring (CEM) which works with thousands of schools
providing direct feedback to them and their teachers in an attempt to improve
the educational system. It runs parallel to the national system of examinations
and inspection and has generated a considerable quantity of data. Analyses
stemming from the monitoring have provided an independent view of the English and
other education systems. Professor Tymms also contributes to and engages in
debate outside the UK
and is aware of the research carried out into accountability, monitoring and
designs for school improvement
worldwide.
3. Responses:
(i) "Under
the accountability system, what should be the consequences?"
3.1) That is very clear. We should see an improving educational system in the sense that we should
see slow but steady improved
attainment levels amongst our pupils. We should see improved
behaviour and social orientation. Indeed we should see improvement
in all areas for which schools are responsible.
(ii)
What is the value of:
o the
school's self-assessment;
o the
results of national tests; and
o the
school's contextual value added scores;
3.2) These questions are hard to answer
with any certainty. Whilst there is no shortage of verbal accounts,
questionnaire results and inspectors' opinions it is not at all clear where the
truth lies. Take, for example,
questions about the consequences of schools using national test results and/or
contextual value-added scores. To what extent have they made a difference? We
simply cannot tell. This is because
so many other things are happening simultaneously in our society and in our
schools. There have been numerous initiatives: inspections have changed, the nature
of the tests has changed, the population of school children has changed and so
on. We are seeing changes in the schools but what has caused what? We simply cannot know, and that is a problem that faces
us nationally and internationally. There are two very relevant publications. One
is from forty years ago and one very recent. They are:
Campbell,
D. T. (1969). Reforms as experiments. American Psychologist, 24,
409-429.
Issue number 2 of the The
Psychology of Education Review Volume (2008) 32 issue 2
(iii)
Is the school report card potentially a
sound basis for:
o informing
parents;
o providing
a set of prioritised outcomes for schools;
o providing
a starting point for Ofsted inspection; and
o providing
a management tool for government?
3.3) It is fairly easy to survey parents to
see what they say they want or to ask Ofsted what they would like, but it is
much harder to know the consequences of using such a report card. We can look
and see what people say has happened in New York
with their report card, but New York does not
know for sure what impact it has
had, although there are opinions and there are report cards all over the US. Which is
best and are there better ways of doing things? An evidenced based assessment
is lacking.
3.4) There are, however, clear ways forward
and these are outlined in Campbell's paper "Reforms as
Experiments" which was referenced above. It would be a major advance to
consider his ideas seriously and his suggestions for ways forward. In essence what
he says is this: Governments across the world in many areas of policy really do
not know what the consequences of their policies will be. They know where they
want to go and they know what people say they want but when we put a policy
into place we rarely know its consequences. What we should be doing is to
formulate policies from which we learn and explicitly aim to change them in the
light of evidence. The best way forward is to try out several different things
systematically. When we are thinking about report cards we should have trials
with a variety of systems. We need that diversity so that we can learn.
3.5) I have recently been at an invited
conference in Germany
where they are thinking of setting up national testing and high-accountability
systems. I was privileged to be part of an international delegation including
people from Sweden, Holland, the United States,
the UK, and various Länder
in Germany
discussing what we know about the way forward. The one thing that was clear was
our ignorance and that we need to learn from each other. We need to co-ordinate
our efforts, investigate systematically and build a knowledge base so that our
educational systems can make the kinds of advances that we all want.
January
2009
| 