Memorandum submitted by Dr A
Gardiner, University of Birmingham
Executive summary
1. All comments in this
submission derive from experience with mathematics at school and
university level.
2. (i) Testing has a
useful function: it provides an occasional "focus", requiring teachers and
pupils to "get it all together on the night" - which is quite different from
"thinking one understands", or managing to use a technique effectively provided
the wind is behind one.
(ii) Testing also provides
a potential mechanism for certifying in a reasonably objective way (for the
benefit of employers and others) that certain simple things have been attained
- subject only to the requirement that these "things" are inevitably restricted
to what can be reliably tested through centrally controlled tests.
These remarks indicate the limitations
of this useful function, and hence the potential for abuse. The present system has ignored these dangers
- with serious consequences.
3. Education must
recognise the need to achieve robust mastery of certain key "atomic" tasks because
they constitute essential components of more important,
and more interesting, challenges.
These more delicate challenges
presuppose certain "atomic" skills, but demand something much harder to
achieve: namely the willingness to struggle in order to select and to apply the
relevant simple procedures in order to solve a genuine problem. That is, the true goals of education are
like learning to enjoy tackling, and completing, jigsaws with a significant
number of pieces: to achieve this it may be important to be able to
recognise a single piece, and to learn to fit two given pieces together;
but these two "atomic skills" omit the most important requirements for success
in the more interesting challenge.
Education loses its raison
d'être, and loses the respect of pupils - and ultimately of end-users and
of the public, if it allows the focus of attention to shift
- from achieving the
more important challenges (of learning to integrate simple skills into
simple "wholes"),
- to documenting levels
of mastery in completing predictable "one-piece jigsaws".
4. The significance of
this for the current inquiry arises from:
- the pressure on central tests to be reliable
and fair (which obliges them to concentrate
on items which are predictable, and
which are relatively straightforward);
- political pressures to "demonstrate"
year-on-year "improvement";
- the need to cultivate a professionalism among teachers and
schools which recognises
the limitations of testing, and the impossibility of cultivating
such professionalism in a
climate where performance on central tests is the main indicator
of professional
competence.
The first two pressures make it
difficult for centrally controlled tests to focus on more delicate challenges
which candidates may fail to complete (jigsaws with a larger number of pieces).
The third pressure means that the
higher goals (which are crucial both culturally - in passing on what our
civilisation is about, and economically - in preparing pupils to
understand and to contribute to the modern world, and which are the only
justification for having an education system) are now no longer addressed by
ordinary teachers in England, with profound
consequences.
5. Notwithstanding likely
submissions from other "experts", what is missing is not "extended
tasks" (which lead to chaos in the average classroom, and which leave pupils
totally ignorant of what they are supposed to have learned), but
- attention to
lots of simple problems, with short solutions,
- that require pupils
(and teachers!) to think rather than merely to implement some one-
step knee-jerk technique in a predictable way, and
- to master the art of
"selecting and coordinating" the simplest methods to tackle and
solve mildly unfamiliar problems.
6. The evidence for these
claims is clear
- when one works with
pupils and teachers in schools;
- when one talks to
those (for example, in QCA, in MTDT - the centralised testing
contractors, or in the Awarding Bodies)
whose job it is to design, to implement and to
monitor the system;
- when one looks at
independent measures of pupil attainment over the years (such as the
national pyramid of competitions);
- when one looks
closely at the results of the best international comparisons; or
- when one teaches
those who enter university having emerged from the system at age 18
with the best qualifications.
7. The current focus on
preparation for centrally controlled tests has led to a "de-natured" school
mathematics which serves the average pupil rather poorly; but the consequences
are especially unfortunate for those at either end of the ability spectrum. In particular, there is a profound need for
a suitable "curriculum and assessment framework" for more able pupils (roughly
speaking the top 25%), which addresses the omissions alluded to in 3. and
5. above.
8. The current system is
too often driven by a bureaucratic and political preference (which has never
been openly debated) to conceal differences, and to force all assessment
into a single framework.
I cannot speak for all subjects,
but the needs of elementary mathematics - whether for all to the age of 14
(basic arithmetic, measures, fractions, ratio; geometry and mensuration;
algebra; and the art of using this basic material to solve the simplest written
problems), or for those who continue to age 16 or age 18 - cannot obviously be
met within a framework designed mainly to accommodate more practical
training. If we value elementary
mathematics, then we need to consider a simple structure which clarifies and
respects such differences.
9. Central testing should
be of two quite different types:
- testing at one or two specified "endpoints"
in the educational cycle, whose function is to
certify that an individual has achieved certain simple
skills to a specified standard;
- sampling (in
the spirit of the old Assessment of Performance Unit APU), which is
designed to inform teachers, schools, inspectors and others
of areas requiring attention
(locally and/or nationally).
10. Education remains a
subtle art which depends on good teaching within the context of a curriculum
and assessment framework that supports high quality instruction. It is a serious mistake to imagine that
genuine educational improvements can result from a central data-base which
tracks each pupil's progress in jumping through a series of relatively trivial
hoops, and which forces schools to concentrate on the most superficial aspects
of education at the expense of the most profound.
Brief introduction
The author is a university research
mathematician, who has devoted his career to trying to make sense of the role
played by mathematics in our society "from the cradle to the grave". He has worked extensively in curriculum
development, and is the author of numerous books of materials for pupils and
teachers. His full-time position is that of Reader in Mathematics and
Mathematics Education in the University of Birmingham (since 1988).
He is the Founding Director of the UK
Mathematics Foundation (which works with pupils and teachers, and which
established the current pyramid of national mathematics competitions and ran it
from 1988-1996, before setting up the UKMT and handing over the competitions -
which now involve 650 000 pupils each year - as a going concern).
He was President of The Mathematical Association
(1997-8), Chair of the Education Committee of the European Mathematical
Society (2000-2004), is Senior Vice President of the World Federation of
National Mathematics Competitions, and is the UK's next nominee to the International
Commission on Mathematical Instruction - the ultimate international body in
the world of mathematics education. He
is respected not only for his breadth of experience and competence, but also
for having remained fiercely independent of all educational factions.
More extended analysis
1. All comments in this
submission derive from experience with mathematics at school and
university level. The author's experience at primary level is limited, but is
extensive at secondary level and at university level - in the UK and abroad.
2. Testing has a useful
function.
It can provide an invaluable
"focus", challenging teachers and pupils (who can easily slip into the cosy
belief that certain material has been mastered) to demonstrate that basic
techniques can be used effectively in limited settings without falling apart.
At the same time it provides a
valuable potential mechanism for certifying in a reasonably objective way (for
the benefit of employers and others) those simple things that
have been attained and that can be tested. The limitations of this certification process need to be clearly
understood by society, so that pupils, teachers and senior management in
schools recognise
- the very limited
nature of those things that can be most easily tested on central tests;
- that "atomic" skills
constitute a rather small part of what needs to be learned,
- that learning to
integrate simple techniques into longer chains of calculation and
reasoning is considerably more important and considerably
more demanding, and
- that subsequent
employers/universities depend on pupils' ability to "integrate" basic
techniques to solve simple problems of a less standard kind,
and that they may have to
assess this in ways that are far less predictable than
standard centralised assessments, so
that student long-term progression demands that considerable
time and effort be devoted
to tackling such problems.
3. Thus education must
recognise that robust mastery of certain key "atomic" tasks is important because
they constitute the simplest (but essential) components of more
important, and more interesting, challenges.
For example, the task of summarising the contents of
a two page essay presumes all manner of prior experiences (from the ability to
read, a basic working vocabulary and the ability to look up one or two
unfamiliar words; a grasp of grammatical structure; comprehension; etc. to the
ability to construct a précis), but cannot be reduced to these "atomic" skills;
it needs extensive practice in order to cultivate the relevant elusive
"ability".
Similarly solving problems in mathematics cannot be
reduced to "atomic" skills - whether
- calculating 15 ´ 9 = ? (which the Numeracy
Strategy pretended could be reduced to
standard "atomic" skills, only to discover
in TIMSS 2003 how ineffective this approach
is compared to that adopted in other
countries , who achieve much better results in a
much shorter time);
- or answering a simple word problem, such as
Two
cyclists cycle towards each other along a road. At 8am they are 42km apart; at
11am they meet. One cyclist pedals at an average speed of 7.5 km/h. What is the
average speed of the other cyclist?
where the "atomic skills" are those of Year
8 or Year 9 pupils, but where the necessary
information has to be extracted from three
or four sentences, and then used in simple
ways; but where the ability to complete
these steps correctly eludes 25-30% of those
currently emerging from school with an A
grade at A level;
- or needing to identify
"[(sinq)´(2cosq) -
(sin2q)´(-cosq)]/(sin2q)" as
a complicated way of writing "-2sinq";
- or facing the requirement to prove an unfamiliar simple result in
plane geometry.
Instead of understanding this
crucial distinction we have established a system whereby potentially useful
tests of simple techniques are being abused to exercise control over
schools and teachers. The result has
been to replace genuine education with a generation of school-leavers who have
become experts in "101 silly ways to complete one piece jigsaws", but who are
neither interested in completing, nor able to complete, the simplest
interesting jigsaws.
The evidence of this profound
transformation is to be found in the intakes of our better universities (I have
just marked 400 scripts from three very different first year university
courses); but no-one dare speak out. We
have wasted billions of pounds and millions of lives.
4. The awkward truth is
that high quality education depends on high quality teaching: central
testing needs to be designed (and
timed) to support and to encourage such high quality
teaching.
Centrally imposed tests can
support, but can also undermine, good teaching; they cannot be
used to "drive up standards".
In most schools, the effect of
such tests has been to undermine good teaching, and to make it
impossible for ordinary teachers
and pupils to discover at first hand what has been achieved
by mankind through the
combination of discipline, technique, persistence and imagination.
We need to devise a system which
recognises:
- the value of central
monitoring of that attainment which is measurable;
- the limitations of
this kind of central monitoring;
- the fundamental fact
that pupils' subsequent progress depends on the subtle art of
selecting and combining simple techniques to achieve higher
goals;
- the fact that these
more important goals oblige us to cultivate a higher professionalism
among teachers which takes for granted that the
responsibility for teaching and assessing
these "higher goals" lies with them (as illustrated by the
much vaunted example of
Finland).
5. QCA and others are
vaguely aware of the fact that current assessments concentrate on
assessing "predictable one-piece
jigsaws". But they jump to the
conclusion that this
weakness can be fixed by
supplementing the assessment of "one piece jigsaws" with
"extended tasks" (5000 piece
jigsaws!).
This touching belief in "extended
tasks" lay behind the original commitment to coursework.
It ignores the fact that one can neither
"practice" nor assess the ability to complete very large
jigsaws; what one can teach and
learn is the art of completing 20 piece and 100 piece jigsaws
- where the basic strategies
required prefigure those required for very large jigsaws.
The current assessment system
totally neglects this crucial domain: simple problems that can
be stated in a few sentences, and
which have short solutions - but whose solutions cannot be
instantly reduced to one-step
routines, and which therefore force pupils (and teachers) to
think.
"Extended tasks" have their
place: (the author is a recognised expert in devising such tasks, in
advocating their use, and in
analysing how they can be used most effectively). But they place
considerable demands on both
pupils and teachers. There may be
grounds for requiring all
pupils to engage in such work,
provided it is assessed internally by the teacher; but such tasks
are totally irrelevant within a
centralised system.
6. The author spends much
of his time working with pupils and teachers, and marking the
work of students at all ages from
11 to 22. He would be happy to provide
documentary
evidence of his claims. The extent of the increasing failure of the
current, QCA-controlled
system is clear
- from official
examination statistics,
- from the numbers of
independent schools abandoning GCSE in favour of IGCSE,
- from olympiad scripts,
- from international
studies such as TIMSS,
- from the pattern of
recent Oxbridge admissions;
- from the performance
of our best school-leavers;
- from undergraduate
exam scripts; etc..
Yet those who are most directly
affected by these dramatic changes are constrained by the
effect of "market forces" in
education into collective collusion and concealment:
- schools compete, so
teachers collude with their pupils in all sorts of ways (well-illustrated
by the widespread cheating associated with, but in no way
restricted to, coursework);
- Exam Boards now
compete, so the horror-stories of their examiners are systematically
suppressed;
- and to provide their
"customers" with a better service, Exam Board chief examiners are
officially encouraged to write "the book of the syllabus"
that promises an inside track
to the best marks on
that particular Board's exams, ignoring any important aspects of
mathematics that happen not to feature on that Board's
exams;
- no university dare
speak out, in case potential applicants draw the conclusion that the
outspoken university is somehow worse than others;
- no university
department dare fail incompetent students because of the consequent loss
income, so extreme pressures are exerted from above to
"minimise wastage" (that is, to
pass incompetent students through the system, and ultimately
to certify them as genuine
"graduates").
7. The need for a
"curriculum and assessment framework" designed for more able students
was recognised in Recommendation
4.5 of the Smith report Making mathematics count
(HMSO 2004;
<http://www.mathsinquiry.org.uk/report/>).
The initiative resulting from
this recommendation was completely botched by QCA (the
author was the lead contractor in
the relevant project to "devise suitable materials").
The professional consensus now
matches the judgement of OfSTED that to be effective, such
a "curriculum and assessment
framework" has to guide teaching Monday-Friday, week-in and
week-out, in ordinary classrooms
in ordinary schools. Hence it needs to
serve around 25% of
each cohort - a group which
includes most of those who might later proceed to study a
significantly numerate discipline
at university, or who might need to use elementary
mathematics in a moderately
serious way in their employment.
The move to "double award maths
GCSE" from 2010 offers a unique opportunity to satisfy
the needs of the ordinary punter
(in the basic GCSE1) and to devise a second GCSE (GCSE2)
aimed at 50+% of each cohort, but
which includes a component specifically designed to lay a
stronger foundation for the top
25% or so. This opportunity is
currently being wasted because
of bureaucratic obstruction and
official failure of imagination.
8. The bureaucratic
determination to force all assessment into a single framework is a
major concern - but one which I
cannot analyse fully here.
The assessment needs of
"academic" subjects are quite different from those of "vocational"
diplomas (for example, effective
basic assessment of academic subjects like mathematics has
to be largely through timed
written examinations). And those whose
qualifications are of one
kind (whether academic or
vocational) are often ill-equipped to "cross over"; so attempts to
establish "equivalences" between
qualifications of different types and to insist that these
equivalences be used for
decisions concerning "progression" are misguided.
The needs of elementary
mathematics cannot obviously be met within a framework designed
to accommodate more practical
training. If we value elementary mathematics,
then we need
to consider a simple structure
which clarifies and respects these basic differences.
9. The work of the
Assessment of Performance Unit (APU) is well documented. It not only
provided invaluable data, but
demonstrated a broad-mindedness which included exploring
ways to assess some of the more
elusive aspects of mathematics alluded to in previous
sections. An improved version could do even better.
The whole ethos of the APU and
its work was to support teachers and good teaching with the
minimum of intervention. If we manage to recognise the damage done by
"ranking" schools
and pupils on the basis of tests
of dubious value, then an institution like the APU could
provide all the information
schools and inspectors need in a manner that supports good
teaching, and at a fraction of
the cost!
June 2007
