Memorandum submitted by Professor Peter
Tymms, Durham University
1. Summary
The questions posed by the Inquiry are vital
to the future of our educational system, but it is clear that
most of them cannot be answered satisfactorily given our present
state of knowledge. What follows is a justification of this statement
and a suggestion for a way forward.
2. Background and expertise
Professor Peter Tymms is an educational researcher
based at Durham University where he directs the Centre for Evaluation
and Monitoring (CEM) which works with thousands of schools providing
direct feedback to them and their teachers in an attempt to improve
the educational system. It runs parallel to the national system
of examinations and inspection and has generated a considerable
quantity of data. Analyses stemming from the monitoring have provided
an independent view of the English and other education systems.
Professor Tymms also contributes to and engages in debate outside
the UK and is aware of the research carried out into accountability,
monitoring and designs for school improvement worldwide.
3. Responses
(i) "Under the accountability system,
what should be the consequences?"
3.1 That is very clear. We should see an improving
educational system in the sense that we should see slow but steady
improved attainment levels amongst our pupils. We should see improved
behaviour and social orientation. Indeed we should see improvement
in all areas for which schools are responsible.
(ii) What is the value of:
— the school's self-assessment; —
the results of national tests; and
— the school's contextual value added scores;
3.2 These questions are hard to answer with
any certainty. Whilst there is no shortage of verbal accounts,
questionnaire results and inspectors' opinions it is not at all
clear where the truth lies. Take, for example, questions about
the consequences of schools using national test results and/or
contextual value-added scores. To what extent have they made a
difference? We simply cannot tell. This is because so many other
things are happening simultaneously in our society and in our
schools. There have been numerous initiatives: inspections have
changed, the nature of the tests has changed, the population of
school children has changed and so on. We are seeing changes in
the schools but what has caused what? We simply cannot know, and
that is a problem that faces us nationally and internationally.
There are two very relevant publications. One is from 40 years
ago and one very recent. They are:
(iii) Is the school report card potentially
a sound basis for:
— informing parents; — providing
a set of prioritised outcomes for schools;
— providing a starting point for Ofsted
inspection; and
— providing a management tool for government?
3.3 It is fairly easy to survey parents
to see what they say they want or to ask Ofsted what they would
like, but it is much harder to know the consequences of using
such a report card. We can look and see what people say has happened
in New York with their report card, but New York does not know
for sure what impact it has had, although there are opinions and
there are report cards all over the US. Which is best and are
there better ways of doing things? An evidence-based assessment
is lacking.
3.4 There are, however, clear ways forward
and these are outlined in Campbell's paper "Reforms as Experiments"
which was referenced above. It would be a major advance to consider
his ideas seriously and his suggestions for ways forward. In essence
what he says is this: Governments across the world in many areas
of policy really do not know what the consequences of their policies
will be. They know where they want to go and they know what people
say they want but when we put a policy into place we rarely know
its consequences. What we should be doing is to formulate policies
from which we learn and explicitly aim to change them in the light
of evidence. The best way forward is to try out several different
things systematically. When we are thinking about report cards
we should have trials with a variety of systems. We need that
diversity so that we can learn.
3.5. I have recently been at an invited
conference in Germany where they are thinking of setting up national
testing and high-accountability systems. I was privileged to be
part of an international delegation including people from Sweden,
Holland, the United States, the UK, and various Länder in
Germany discussing what we know about the way forward. The one
thing that was clear was our ignorance and that we need to learn
from each other. We need to co-ordinate our efforts, investigate
systematically and build a knowledge base so that our educational
systems can make the kinds of advances that we all want.
January 2009
| 
