Memorandum submitted by Dr A Gardiner, University of Birmingham
1. All comments in this submission derive from experience with mathematics at school and university level.
2. (i) Testing has a useful function: it provides an occasional "focus", requiring teachers and pupils to "get it all together on the night" - which is quite different from "thinking one understands", or managing to use a technique effectively provided the wind is behind one.
(ii) Testing also provides a potential mechanism for certifying in a reasonably objective way (for the benefit of employers and others) that certain simple things have been attained - subject only to the requirement that these "things" are inevitably restricted to what can be reliably tested through centrally controlled tests.
These remarks indicate the limitations of this useful function, and hence the potential for abuse. The present system has ignored these dangers - with serious consequences.
3. Education must recognise the need to achieve robust mastery of certain key "atomic" tasks because they constitute essential components of more important, and more interesting, challenges.
These more delicate challenges presuppose certain "atomic" skills, but demand something much harder to achieve: namely the willingness to struggle in order to select and to apply the relevant simple procedures in order to solve a genuine problem. That is, the true goals of education are like learning to enjoy tackling, and completing, jigsaws with a significant number of pieces: to achieve this it may be important to be able to recognise a single piece, and to learn to fit two given pieces together; but these two "atomic skills" omit the most important requirements for success in the more interesting challenge.
Education loses its raison d'être, and loses the respect of pupils - and ultimately of end-users and of the public, if it allows the focus of attention to shift
- from achieving the more important challenges (of learning to integrate simple skills into
- to documenting levels of mastery in completing predictable "one-piece jigsaws".
4. The significance of this for the current inquiry arises from:
- the pressure on central tests to be reliable and fair (which obliges them to concentrate
on items which are predictable, and which are relatively straightforward);
- political pressures to "demonstrate" year-on-year "improvement";
- the need to cultivate a professionalism among teachers and schools which recognises
the limitations of testing, and the impossibility of cultivating such professionalism in a
climate where performance on central tests is the main indicator of professional
The first two pressures make it difficult for centrally controlled tests to focus on more delicate challenges which candidates may fail to complete (jigsaws with a larger number of pieces).
The third pressure means that the higher goals (which are crucial both culturally - in passing on what our civilisation is about, and economically - in preparing pupils to understand and to contribute to the modern world, and which are the only justification for having an education system) are now no longer addressed by ordinary teachers in England, with profound consequences.
5. Notwithstanding likely submissions from other "experts", what is missing is not "extended tasks" (which lead to chaos in the average classroom, and which leave pupils totally ignorant of what they are supposed to have learned), but
- attention to lots of simple problems, with short solutions,
- that require pupils (and teachers!) to think rather than merely to implement some one-
step knee-jerk technique in a predictable way, and
- to master the art of "selecting and coordinating" the simplest methods to tackle and
solve mildly unfamiliar problems.
6. The evidence for these claims is clear
- when one works with pupils and teachers in schools;
- when one talks to those (for example, in QCA, in MTDT - the centralised testing
contractors, or in the Awarding Bodies) whose job it is to design, to implement and to
monitor the system;
- when one looks at independent measures of pupil attainment over the years (such as the
national pyramid of competitions);
- when one looks closely at the results of the best international comparisons; or
- when one teaches those who enter university having emerged from the system at age 18
with the best qualifications.
7. The current focus on preparation for centrally controlled tests has led to a "de-natured" school mathematics which serves the average pupil rather poorly; but the consequences are especially unfortunate for those at either end of the ability spectrum. In particular, there is a profound need for a suitable "curriculum and assessment framework" for more able pupils (roughly speaking the top 25%), which addresses the omissions alluded to in 3. and 5. above.
8. The current system is too often driven by a bureaucratic and political preference (which has never been openly debated) to conceal differences, and to force all assessment into a single framework.
I cannot speak for all subjects, but the needs of elementary mathematics - whether for all to the age of 14 (basic arithmetic, measures, fractions, ratio; geometry and mensuration; algebra; and the art of using this basic material to solve the simplest written problems), or for those who continue to age 16 or age 18 - cannot obviously be met within a framework designed mainly to accommodate more practical training. If we value elementary mathematics, then we need to consider a simple structure which clarifies and respects such differences.
9. Central testing should be of two quite different types:
- testing at one or two specified "endpoints" in the educational cycle, whose function is to
certify that an individual has achieved certain simple skills to a specified standard;
- sampling (in the spirit of the old Assessment of Performance Unit APU), which is
designed to inform teachers, schools, inspectors and others of areas requiring attention
(locally and/or nationally).
10. Education remains a subtle art which depends on good teaching within the context of a curriculum and assessment framework that supports high quality instruction. It is a serious mistake to imagine that genuine educational improvements can result from a central data-base which tracks each pupil's progress in jumping through a series of relatively trivial hoops, and which forces schools to concentrate on the most superficial aspects of education at the expense of the most profound.
The author is a university research mathematician, who has devoted his career to trying to make sense of the role played by mathematics in our society "from the cradle to the grave". He has worked extensively in curriculum development, and is the author of numerous books of materials for pupils and teachers. His full-time position is that of Reader in Mathematics and Mathematics Education in the University of Birmingham (since 1988).
He is the Founding Director of the UK Mathematics Foundation (which works with pupils and teachers, and which established the current pyramid of national mathematics competitions and ran it from 1988-1996, before setting up the UKMT and handing over the competitions - which now involve 650 000 pupils each year - as a going concern).
He was President of The Mathematical Association (1997-8), Chair of the Education Committee of the European Mathematical Society (2000-2004), is Senior Vice President of the World Federation of National Mathematics Competitions, and is the UK's next nominee to the International Commission on Mathematical Instruction - the ultimate international body in the world of mathematics education. He is respected not only for his breadth of experience and competence, but also for having remained fiercely independent of all educational factions.
More extended analysis
1. All comments in this submission derive from experience with mathematics at school and university level. The author's experience at primary level is limited, but is extensive at secondary level and at university level - in the UK and abroad.
2. Testing has a useful function.
It can provide an invaluable "focus", challenging teachers and pupils (who can easily slip into the cosy belief that certain material has been mastered) to demonstrate that basic techniques can be used effectively in limited settings without falling apart.
At the same time it provides a valuable potential mechanism for certifying in a reasonably objective way (for the benefit of employers and others) those simple things that have been attained and that can be tested. The limitations of this certification process need to be clearly understood by society, so that pupils, teachers and senior management in schools recognise
- the very limited nature of those things that can be most easily tested on central tests;
- that "atomic" skills constitute a rather small part of what needs to be learned,
- that learning to integrate simple techniques into longer chains of calculation and
reasoning is considerably more important and considerably more demanding, and
- that subsequent employers/universities depend on pupils' ability to "integrate" basic
techniques to solve simple problems of a less standard kind, and that they may have to
assess this in ways that are far less predictable than standard centralised assessments, so
that student long-term progression demands that considerable time and effort be devoted
to tackling such problems.
3. Thus education must recognise that robust mastery of certain key "atomic" tasks is important because they constitute the simplest (but essential) components of more important, and more interesting, challenges.
For example, the task of summarising the contents of a two page essay presumes all manner of prior experiences (from the ability to read, a basic working vocabulary and the ability to look up one or two unfamiliar words; a grasp of grammatical structure; comprehension; etc. to the ability to construct a précis), but cannot be reduced to these "atomic" skills; it needs extensive practice in order to cultivate the relevant elusive "ability".
Similarly solving problems in mathematics cannot be reduced to "atomic" skills - whether
- calculating 15 ´ 9 = ? (which the Numeracy Strategy pretended could be reduced to
standard "atomic" skills, only to discover in TIMSS 2003 how ineffective this approach
is compared to that adopted in other countries , who achieve much better results in a
much shorter time);
- or answering a simple word problem, such as
Two cyclists cycle towards each other along a road. At 8am they are 42km apart; at
11am they meet. One cyclist pedals at an average speed of 7.5 km/h. What is the
average speed of the other cyclist?
where the "atomic skills" are those of Year 8 or Year 9 pupils, but where the necessary
information has to be extracted from three or four sentences, and then used in simple
ways; but where the ability to complete these steps correctly eludes 25-30% of those
currently emerging from school with an A grade at A level;
- or needing to identify
"[(sinq)´(2cosq) - (sin2q)´(-cosq)]/(sin2q)" as a complicated way of writing "-2sinq";
- or facing the requirement to prove an unfamiliar simple result in plane geometry.
Instead of understanding this crucial distinction we have established a system whereby potentially useful tests of simple techniques are being abused to exercise control over schools and teachers. The result has been to replace genuine education with a generation of school-leavers who have become experts in "101 silly ways to complete one piece jigsaws", but who are neither interested in completing, nor able to complete, the simplest interesting jigsaws.
The evidence of this profound transformation is to be found in the intakes of our better universities (I have just marked 400 scripts from three very different first year university courses); but no-one dare speak out. We have wasted billions of pounds and millions of lives.
4. The awkward truth is that high quality education depends on high quality teaching: central
testing needs to be designed (and timed) to support and to encourage such high quality
Centrally imposed tests can support, but can also undermine, good teaching; they cannot be
used to "drive up standards".
In most schools, the effect of such tests has been to undermine good teaching, and to make it
impossible for ordinary teachers and pupils to discover at first hand what has been achieved
by mankind through the combination of discipline, technique, persistence and imagination.
We need to devise a system which recognises:
- the value of central monitoring of that attainment which is measurable;
- the limitations of this kind of central monitoring;
- the fundamental fact that pupils' subsequent progress depends on the subtle art of
selecting and combining simple techniques to achieve higher goals;
- the fact that these more important goals oblige us to cultivate a higher professionalism
among teachers which takes for granted that the responsibility for teaching and assessing
these "higher goals" lies with them (as illustrated by the much vaunted example of
5. QCA and others are vaguely aware of the fact that current assessments concentrate on
assessing "predictable one-piece jigsaws". But they jump to the conclusion that this
weakness can be fixed by supplementing the assessment of "one piece jigsaws" with
"extended tasks" (5000 piece jigsaws!).
This touching belief in "extended tasks" lay behind the original commitment to coursework.
It ignores the fact that one can neither "practice" nor assess the ability to complete very large
jigsaws; what one can teach and learn is the art of completing 20 piece and 100 piece jigsaws
- where the basic strategies required prefigure those required for very large jigsaws.
The current assessment system totally neglects this crucial domain: simple problems that can
be stated in a few sentences, and which have short solutions - but whose solutions cannot be
instantly reduced to one-step routines, and which therefore force pupils (and teachers) to
"Extended tasks" have their place: (the author is a recognised expert in devising such tasks, in
advocating their use, and in analysing how they can be used most effectively). But they place
considerable demands on both pupils and teachers. There may be grounds for requiring all
pupils to engage in such work, provided it is assessed internally by the teacher; but such tasks
are totally irrelevant within a centralised system.
6. The author spends much of his time working with pupils and teachers, and marking the
work of students at all ages from 11 to 22. He would be happy to provide documentary
evidence of his claims. The extent of the increasing failure of the current, QCA-controlled
system is clear
- from official examination statistics,
- from the numbers of independent schools abandoning GCSE in favour of IGCSE,
- from olympiad scripts,
- from international studies such as TIMSS,
- from the pattern of recent Oxbridge admissions;
- from the performance of our best school-leavers;
- from undergraduate exam scripts; etc..
Yet those who are most directly affected by these dramatic changes are constrained by the
effect of "market forces" in education into collective collusion and concealment:
- schools compete, so teachers collude with their pupils in all sorts of ways (well-illustrated
by the widespread cheating associated with, but in no way restricted to, coursework);
- Exam Boards now compete, so the horror-stories of their examiners are systematically
- and to provide their "customers" with a better service, Exam Board chief examiners are
officially encouraged to write "the book of the syllabus" that promises an inside track
to the best marks on that particular Board's exams, ignoring any important aspects of
mathematics that happen not to feature on that Board's exams;
- no university dare speak out, in case potential applicants draw the conclusion that the
outspoken university is somehow worse than others;
- no university department dare fail incompetent students because of the consequent loss
income, so extreme pressures are exerted from above to "minimise wastage" (that is, to
pass incompetent students through the system, and ultimately to certify them as genuine
7. The need for a "curriculum and assessment framework" designed for more able students
was recognised in Recommendation 4.5 of the Smith report Making mathematics count
(HMSO 2004; <http://www.mathsinquiry.org.uk/report/>).
The initiative resulting from this recommendation was completely botched by QCA (the
author was the lead contractor in the relevant project to "devise suitable materials").
The professional consensus now matches the judgement of OfSTED that to be effective, such
a "curriculum and assessment framework" has to guide teaching Monday-Friday, week-in and
week-out, in ordinary classrooms in ordinary schools. Hence it needs to serve around 25% of
each cohort - a group which includes most of those who might later proceed to study a
significantly numerate discipline at university, or who might need to use elementary
mathematics in a moderately serious way in their employment.
The move to "double award maths GCSE" from 2010 offers a unique opportunity to satisfy
the needs of the ordinary punter (in the basic GCSE1) and to devise a second GCSE (GCSE2)
aimed at 50+% of each cohort, but which includes a component specifically designed to lay a
stronger foundation for the top 25% or so. This opportunity is currently being wasted because
of bureaucratic obstruction and official failure of imagination.
8. The bureaucratic determination to force all assessment into a single framework is a
major concern - but one which I cannot analyse fully here.
The assessment needs of "academic" subjects are quite different from those of "vocational"
diplomas (for example, effective basic assessment of academic subjects like mathematics has
to be largely through timed written examinations). And those whose qualifications are of one
kind (whether academic or vocational) are often ill-equipped to "cross over"; so attempts to
establish "equivalences" between qualifications of different types and to insist that these
equivalences be used for decisions concerning "progression" are misguided.
The needs of elementary mathematics cannot obviously be met within a framework designed
to accommodate more practical training. If we value elementary mathematics, then we need
to consider a simple structure which clarifies and respects these basic differences.
9. The work of the Assessment of Performance Unit (APU) is well documented. It not only
provided invaluable data, but demonstrated a broad-mindedness which included exploring
ways to assess some of the more elusive aspects of mathematics alluded to in previous
sections. An improved version could do even better.
The whole ethos of the APU and its work was to support teachers and good teaching with the
minimum of intervention. If we manage to recognise the damage done by "ranking" schools
and pupils on the basis of tests of dubious value, then an institution like the APU could
provide all the information schools and inspectors need in a manner that supports good
teaching, and at a fraction of the cost!