Education CommitteeWritten evidence submitted by Institute of Mathematics and its Applications

The arguments in favour of and against having a range of awarding bodies for academic and applied qualifications (including A Levels, GCSEs, Diplomas, BTECs and OCR Nationals), and the merits of alternative arrangements, such as having one national body or examination boards franchised to offer qualifications in particular subjects or fields.

1. The Institute is clear that the present competition between awarding organisations does not promote excellence in teaching mathematics in schools, and may even drive down standards.

2. Public examinations should meet three main purposes:

accrediting achievement;

indicating preparedness for progression; and

providing accountability within the education system.

3. The Institute considers that all three of these purposes would be better met by a system without competitive pressures.

4. Stakes in public examinations are so high that teachers put considerable effort into identifying the awarding organisation and specification most likely to maximise the outcomes for their students. At the simplest level, this may mean seeking out the examination papers that appear most straightforward. This in itself provides a pressure on awarding organisations to set papers that appear reasonably straightforward. The Institute has a number of concerns about the impacts of competition on the education system.

5. Our principal concern relates to the kinds of challenges set to candidates in GCE and GCSE mathematics examinations. We consider that mathematics assessment should reflect what end users and the mathematics education community value about the subject, and should therefore provide substantial questions, rich in challenge and problem-solving opportunity, that leave teachers and their students in no doubt that solving problems in context and demonstrating reasoning ability are highly valued mathematical skills. Our concern is that this kind of rigorous assessment is gradually disappearing from GCE and, particularly, GCSE mathematics examinations in England.

6. We have considered what might be the causes of this dilution of mainstream public examinations. We acknowledge that the examinations criteria that awarding organisations use as a basis for preparing papers do emphasise the importance of solving problems and reasoning clearly. Nevertheless, the papers that emerge fail to reflect the central significance of these essential aspects of mathematics.

7. It is clear that awarding organisations have significantly improved their technical performance in the preparation of examinations that offer candidates the opportunity to demonstrate what they know, understand, and can do. So the question arises why they have not also increased the extent to which examinations reflect the mathematical qualities that end users and the mathematics education community would most welcome.

8. At a technical level, examiners are rightly wary of questions in which candidates might possibly go badly wrong, as these are both difficult to mark and are likely to be wasteful of marks. But this caution should not be allowed to override the importance of developing questions that assess the kind of qualities that success in these examinations should denote.

9. The Institute takes the view that the introduction of appropriately challenging mathematics papers, for both GCSE and GCE, will not happen whilst multiple examinations exist that are intended to assess the same curriculum in the same way. We do not believe that awarding organisations will be willing to set more demanding questions enabling proper assessment of higher level skills for higher attaining students if by so doing they would risk losing market share.

10. On the other hand, a system without competitive pressures would offer a level playing field for candidates, and would permit the kinds of examinations that the mathematics education community would support. This could be achieved in a number of ways. The Institute considers that any of these approaches would improve on the present situation.

11. One approach would be to have a single awarding organisation and a single specification. This would ensure fairness for candidates. A single award would allow for far greater scrutiny, and would concentrate the talents of the best examiners. It would permit a wide variety of curricula and teaching approaches to flourish, and a range of textbooks to support this variety. It would make the case that teaching to the test is not regarded as providing a good curriculum.

12. An alternative is to franchise one of several awarding organisations to provide mathematics examinations for a number of years. This could involve existing awarding organisations tendering for a single national contract to award mathematics examinations for a period of time.

13. A more radical approach would involve separating examining processes that determine standards from those that are more to do with marketing qualifications. The former set of responsibilities includes drafting specifications, paper setting, mark scheme writing, marking and awarding. A public body (perhaps DfE or Ofqual) would award contracts for these processes for fixed periods to contractors for specific subjects or groups of subjects. The price structure of the contracts should be such as to give no incentive to the contractor to increase or decrease the uptake of their subject(s). The present awarding organisations would be able to bid for the contracts but would need to separate the operation from their other activities. Others would also be able to bid (eg a university education department, or perhaps a subject grouping/HEI consortium). The contracts would be awarded on a quality basis rather than value for money alone. The awarding organisations would continue to badge the papers and market specifications (more than one could offer the same specification), distribute question papers, distribute results and certificates, as well as providing customer service. Under such an arrangement, awarding organisations would compete on price and quality of service. As changing awarding organisation would no longer be so traumatic, a more fluid market would exist in which centres or consortia of them could seek to drive down costs.

How to ensure accuracy in setting papers, marking scripts, and awarding grades

14. The present approach to setting papers relies entirely on the professional experience of examiners who draft the questions, and revisers and assessors who check their papers before printing. In mathematics, these processes are even more important than in other subjects, as it is the demands of the questions themselves that provide the examination with its challenge, or lack of challenge. This is different from most other subjects in which a question is capable of being answered at a variety of different levels, and where the differentiation comes from the mark scheme.

15. The costs of exam writing have been held down by awarding organisations and papers are therefore constructed too hastily, usually by a single person, with questions tending to be too similar to those of earlier years. This makes the papers highly predictable. Questions tend to avoid significant challenge for candidates, as this can lead to very low marks on the question, which reduces the effectiveness of the paper. A more thoughtful approach is needed that allows examiners the time to develop new ideas carefully, taking research evidence (eg about common misconceptions) into account. The nationally produced National Curriculum tests for Key Stages 2 and 3 form an excellent model in which all items are developed by a team of writers and pre-tested. The additional expense of such an approach is justified in mathematics where the stakes in GCSE and GCE assessment are so high. It is worth pointing out that higher quality mathematics examinations form an essential element in increasing take up of science and mathematics, on which the UK’s economic future depends so heavily.

16. In some instances, mathematics papers are re-typeset by the awarding organisation after they have been set and revised. This is not necessary in the 21st century, and can introduce inaccuracy, as great expertise is needed to typeset mathematical expressions correctly.

17. The Institute considers that onscreen marking is advantageous as it permits more, and more flexible, scrutiny of marking by the awarding organisation.

18. Awarding issues are critical, since the award is intended to ensure that, whatever paper has been sat, in whatever specification, the grade awarded can be regarded as having the same meaning in terms of achievement in the subject. In practice, however, awarding is far from an exact science. In seeking to meet these key objectives, therefore, awarding organisations use two principal approaches. The first is to assume that the candidature will remain broadly comparable from year to year, justifying the stipulation that the proportions achieving each grade should also compare closely to those of the previous year. The second approach is to give statistical information about the impact on grade distributions a much greater role than examiners’ judgements about candidates’ actual performances in setting grade boundaries.

19. The effect of variability in awarding is that there is insufficient comparability of standards across awarding organisations, between specifications, and between subjects. A level playing field for our young people as they sit these critical examinations is too important to continue to permit the substantial differences that currently persist.

The commercial activities of awarding bodies, including examination fees and textbooks, and their impact on schools and pupils

20. The Institute of Mathematics and its Applications issued a Position Statement on Awarding Bodies and Teaching Resources in September 2009. The Institute’s views on the links between awarding organisations and textbooks remain as they were then. The statement reads.

21. The IMA is concerned about developments in the market for textbooks and other teaching resources (in what follows “textbooks” should be construed as including other teaching resources) for courses preparatory to public examinations which it believes to be detrimental to the public good. The practices have arisen of the endorsement of textbooks by awarding bodies and the publicising of authors being examiners.

Many centres feel obliged to purchase textbooks which have been written by examiners and have been endorsed by awarding bodies in case they confer some advantage and to protect themselves from criticism for failing to have done so. Even for those centres wishing to use broader criteria in the choice of resources this restricts their choice as other publishers find they cannot achieve commercial viability and withdraw from the market.

The versioning of textbooks to particular specifications fragments the market which makes it difficult for a variety of approaches, especially innovative ones, to be published. It also causes a culture to arise in which textbooks are considered obsolete when specifications change, with the consequent cost to centres of replacing stock.

Examination specifications need to be self-contained and sufficiently detailed so that all are confident what is required and there should be no suspicion that dependence must be placed on de facto amplification provided by texts where a financial relationship exists between the awarding body and the publisher.

Good examiners are not necessarily good writers of textbooks. Although there are exceptions, there is a tendency for the books to be very exam focused which is a further factor encouraging teaching to the test; those teachers who feel the most need for endorsed texts are those least likely to be confident to break free from the textbook.

Endorsement is effectively acting as a restraint to trade rather than as a kite-marking system. Whilst innovative projects and pilots should quite properly be able to publish linked supporting material, and awarding bodies should be able to support innovative or small-entry examinations with booklists and free materials, anything from which awarding bodies gain commercial advantage should be prohibited as a condition of accreditation by the regulator.

For the maintenance of public confidence in standards there must be a clear separation, between teaching and learning and the public assessment thereof, which does not admit of any suspicion of conflict of interest. It is in the public interest that the resources published for use in our schools and colleges are supportive of good teaching and learning, and it is proper that the regulatory authorities should take steps to prevent practices not conducive to this.

November 2011