Education CommitteeWritten evidence submitted by Dr Tony Gardner, University of Birmingham
Executive Summary
A: A single awarding body may seem attractive, and you may conclude that we should work towards such an arrangement. However, the premature imposition of such a structure would involve clear dangers. We first need an understanding of, a clear commitment to, and a cultivation of, the essential concomitant centralised specialist competence (in each discipline such as mathematics, and in the intricacies of how to design and run an assessment system that supports good teaching). The current problems appear to derive not so much from having three main awarding bodies, as from the fact that one of the three is driven by principles that appear incompatible with “public service”, and the others then see no alternative but to “compete”.
B: Assessment is a craft. The current shortcomings at 16-19 are much more profound than a matter of “accuracy”. Assessment is inevitably approximate, and depends on professional judgement. One should work to minimise errors. But attempts to guarantee the level of “accuracy” (in an attempt to avoid the possibility of “errors”) lead to trivialised assessment items and rigid mark schemes that are in no-one’s interests.
C: The expanding commercial activities of awarding bodies are a national disgrace; the phenomenon is most acute with one of the three main awarding bodies, but the others often see no alternative but to follow suit. If the police were to publish and profit from books and courses telling motorists “how to avoid conviction for speeding or driving offences”, this might well lead to a reduction in the number of recorded motoring offences; but no-one would defend the practice on those grounds – for it would be clear to all that any consequent reduction was a dishonest fiction. Similarly those bodies that provide public assessment (and their employees) should be banned from giving advice on how to improve grades, from endorsing exam-related materials, and from profiting from such provision. Moreover, we must somehow escape from the current situation where an awarding body can be owned by a (large and powerful) business, whose ultimate primary is financial rather than educational, and which can use its power to face down government agencies (such as QCDA and Ofqual).
Introduction to the Witness
I am a research mathematician who has spent 40 years straddling the gulf between universities and schools. Since 1989 I have been Reader in Mathematics and Mathematics Education at the University of Birmingham. I was President of the Mathematical Association in 1997
A. The arguments in favour of (and against) having a range of awarding bodies for academic and applied qualifications etc
A1: We need a permanent core of subject expertise – within the DfE, and in some related agencies (including Ofqual). Cultivating the necessary expertise is a long-term process: we cannot neglect the responsibility to cultivate such experience and competence, and imagine one can then “outsource” such matters, or “train up” civil servants at short notice. (This should have become clear both from QCDA’s attempts to revise the 2007 National Curriculum having previously sacked their “subject teams”, and from the current attempt to review the curriculum without a national curriculum agency.)
A2: At one stage, KS2 and KS3 SATs were an embarrassment. But at some point in the late 1990s or early 00s, QCA established a dedicated group, who both developed in-house expertise, and established constructive links with the wider mathematical community. The result was that the quality of the test papers, and that the professional perception of the resulting product (especially at KS3) improved markedly. Instead of being protected and nurtured, this core group was then hived off to Pearson (publishing and training provider who control Edexcel), their ruminations became inaccessible, the experts were put out to grass, and the short-lived “tradition” was lost. Similarly, awarding bodies such as AQA (previously NEAB, JMB) used to involve academics from the constituent universities and a remarkable bunch of experienced teachers from schools; but the links with universities have been systematically dismantled over the last 30 years, and the nature of teacher-involvement has changed. As a result, many of those who contribute to the setting and marking processes no longer feel that their professional competence is being valued; instead, the process is increasingly driven by marketing, by bureaucratic concerns relating to managing the (increasingly mechanistic) marking process, and by a determination never to require candidates to “join the dots” for themselves.
A3: Before we can effectively debate the first of your three questions, we need to take stock and to:
(a)
(b)
A4: While we engage with these two long-term projects (A3(a) and (b)), we must pursue a path that minimises the danger of centralised disasters (such as “Curriculum 2000”, or the last KS3 SAT fiasco). The situation is far from ideal; but the tradition of pluralism would tend to suggest that, until we are in a position to trust a competent central “authority”, it makes sense to continue with a degree of appropriately monitored variety, giving each operator sufficient freedom for the community to benefit from a degree of choice and innovation (whilst at the same time avoiding centralised folly).
A5: You may need separate confidential sessions with officials and ex-officials from the three main awarding bodies and relevant civil servants to clarify the way the current situation has been distorted by the pure commercialism of one of awarding bodies (forcing the others to follow suit).
B. How to ensure accuracy in setting papers, marking scripts, and awarding grades
B1: This question is not as simple as it may appear: “accuracy” alone is a poor guide to quality. The fact that some of our best schools have abandoned A levels in recent years tends to confirm the clear impression that the quality of these assessment instruments has indeed been degraded. You need to talk (a) to suitable head teachers, and (b) to experienced examiners who have recently “retired” (or resigned) in order to get a better feeling for the nature of the perceived change, and to the pressures which have led to the perceived decline.
B2: You may discover that the frequency of assessment and the way setting and marking have been “administratively mechanised” in recent years have combined in ways that allow question setting errors to be overlooked (because the processes often focus on checking procedures, rather than content – an observation which is linked to our response to A above).
B3: Ex-examiners may alert you to a more serious degradation in the overall quality of exam papers, and the awarding of grades. Downward pressures on the quality of papers seems to result (in part) from the impact on awarding bodies of excessive central pressures (from Ofqual etc.) for their exams and their marking to be “fair” – pressures which your own eventual report could exacerbate if carelessly worded! For excessive focus on “accuracy” and “fairness” leads to boringly predictable questions, in which candidates are never required to “join the dots”, and in which mark schemes treat markers as automatons (who are then merely expected to apply rules, even where these run counter to their professional judgement: for example, many candidates are trained, when in doubt, to present two or more conflicting solutions, and I know that markers used to be instructed to give the marks if one of them was roughly correct – even though the candidate has no idea which solution was correct). The awarding of grades has obvious moved dramatically: Dr. Robert Coe is reported as having declared (at the Ofqual standard summit on 13 October 2011) that “data showed grade inflation was continuing. In A level mathematics, the level of ability worth a grade D or E twenty years ago would now secure an A or a B”.
B4: Examining is a craft rather than a science: examination results are never wholly reliable. An effective system requires an effective bureaucracy. But an effective bureaucracy is not sufficient: one still needs competent professionals to exercise judgement – both at the setting/checking and at the marking stages.
B5: The use and abuse of league tables (by government and by the media – neither of whom seem interested in grappling with nuances) has encouraged a simplistic view of exam results which ignores the fact that the numbers that emerge from any assessment process are approximate artefacts. Provided the assessment process is allowed to do its job unmolested, it can provide approximate individual results, and much more accurate statistical information; but this function is undermined, and the public misconceptions are exacerbated, if the assessment process is compromised by the belief that raising target scores is a way of “driving up standards”. As the old Soviet “five year plans” showed, raised targets can generally be met (and surpassed) without any real improvement. In education, the pressure to “meet targets” regularly deflects attention from the kind of instruction that contributes to students’ long-term mathematical development.
B6: For decades, concerned teachers have written quietly to exam boards each year complaining (often with considerable justification) of “unfair” or inaccurate examination questions. And each year the examiners and the relevant committee would produce a report in which such complaints were addressed. More recently one has the impression that such “errors” have been communicated directly to the national press! It is hard to be sure, but it may well be that the number of serious errors is not very different from what it has always been, and that the real decline is in the overall quality and demand of exam papers (cf B3). I encourage the Committee to talk to awarding bodies (and critical ex-examiners who may feel more able to “spill the beans”) to get an accurate picture of such trends.
C. The commercial activities of awarding bodies, including examination fees and textbooks, and their impact on schools and pupils
C1: All responsible members of the mathematical community are concerned at the increasing involvement of examination boards in the preparation, endorsement and publication of textbooks, which is having a seriously detrimental effect on secondary education.
C2: We are alarmed that textbooks are prepared with contents limited to what is needed for a specific examination and are then endorsed by the awarding body. Such textbooks regularly lack depth and cover a limited range of subject content (being designed narrowly towards passing a particular exam); they also give students little incentive or opportunity to engage with broader and richer material, and fail to foster an appreciation of the subject’s subtleties.
C3: This pattern has had an invidious impact on publishers, who appear no longer interested in publishing serious mathematics textbooks which seek to present elementary mathematics in a way that is independent of any particular syllabus.
C4: The precise history needs to be clarified. But the summary is clear: one awarding body was much quicker to see the commercial (and anti-educational) “opportunities” in the early 1990s; their moves were at first resisted by the other two awarding bodies, who still recognise that these moves are against the public interest, but appear powerless to resist; the trend was exacerbated when the “trend-setting” awarding body was allowed to be bought out by a large commercial publishing group – who now appear to be more powerful than the regulating agencies. Until the awarding bodies (whether one or three) operate once more as a public service, one can expect English “public” examinations to continue their decline.
C5: The practice of reducing mathematics to “exam-fodder” reduces candidates’ experience of learning to a “boring drill”, whose only perceived purpose is to pass a particular examination. When combined with a modular system and regular examinations, the traditional eye-opening experience of A level studies is completely lost. The resulting pressure to focus on a particular set of exams, rather than on the discipline ostensibly being studied, narrows students’ horizons and sets them false criteria of personal achievement. And it de-professionalises teaching.
C6: “Teaching to the test” is destructive in every subject area, but it is especially harmful in mathematics education. In the field of mathematics, the current National Curriculum Review and the promised review of A Levels will not achieve the ambitious goals set by the Government if this urgent matter is not addressed.
C7: To this has recently been added another concern – namely the increasing involvement of examination boards in Continuing Professional Development (CPD) of teachers. Just as we need a core of professional competence in key subject areas (cf A: above), so a system needs to cultivate a system which guarantees high quality local CPD provision. This used to be provided through Local Authorities and through local universities; both have been squeezed without any alternative provision. So we are left with no effective mechanism for assessing local needs and for providing the necessary support. The vacuum has been filled by unmonitored private providers and by exam boards.
C8: Exam boards should clearly be free to present and to explain their syllabuses to interested teachers. The problem arises when such marketing sessions are advertised as “CPD” (in a climate where real dispassionate CPD opportunities are increasingly rare). The additional link between exam boards and the production of textbooks only serves to reinforce the plague of “teaching to the test”: CPD sessions run by exam boards can too easily slip into explaining to teachers how to train students to maximise marks in exams set by their particular board, while a textbook endorsed by the same board often restricts the teaching to coaching students for the relevant examination.
C9: Many would welcome amendments to the regulatory framework to forbid the involvement of examination boards in the production of textbooks, the endorsement of textbooks, and the running of courses and events advertised or recognised in any way as “CPD”. In particular we would welcome an introduction of a rule (similar to the one existing in Civil Service) whereby senior employees and examiners from awarding bodies should be forbidden from publishing educational material, or from benefitting materially from “exam-linked CPD” during their employment by an awarding body and for a specified period after the termination of their contract.
November 2011