Education CommitteeWritten evidence submitted by the Wellcome Trust

Key Points

1. The key messages of this response are:

There are significant problems arising from the current model of multiple awarding bodies for academic qualifications for 15-19 year olds.

If we were establishing the examination system from scratch, a single awarding body would be most favourable. However, in the interest of stability the current model should probably be retained, but only with substantial improvements (paragraph 33), specifically:

(i)There needs to be greater consistency across awarding bodies in the process of awarding grades, and much more openness about how it works.

(ii)Awarding bodies must communicate better with each other, especially in sharing best practice and introducing innovations.

(iii)National subject committees should be established to oversee the standard of examinations of major subjects across all awarding bodies.

(iv)The changes to specifications every five years should be discontinued and awarding bodies given the ability to make incremental changes to examinations as and when needed, under the guidance of national subject committees.

The Government should use Ofqual’s Codes of Practice to stop awarding bodies endorsing textbooks (paragraphs 28-29).

The value of STEM subjects (science, technology, engineering and mathematics) to Higher Education Institutions (HEIs) and employers should be made more evident, to ensure that pupils are not put off taking them due to perceived difficultly in achieving higher grades as compared to other subjects (paragraphs 25-26).

Awarding bodies must be held to account for the standards of the examination system. If substantial improvements are not seen in the short term, Ofqual must take action.

Introduction

2. An appropriately run examination system in England is vital for qualifications across the breadth of education. Confidence in the system is essential, yet the perception from teachers, the general public and other stakeholders, is that the standard of grades has fallen over time; this can be particularly difficult for students who have worked hard to achieve their grades and whose future is at stake.

3. There are many findings to support the perception of “grade slippage”. The Chief Executive of Ofqual recently noted that of their approximately 50 studies, standards were in many instances maintained, sometimes increased, but that most recent studies tended to show that standards are on the slide. She specifically cited studies that had found declining standards in mathematics, chemistry and design and technology.1

4. It seems likely that grades have reduced in part because the awarding bodies are competing for custom and teachers are likely to choose those qualifications that will yield the best performance for their schools and for their students. This process could happen without conscious direction from the awarding bodies. However, this process may be more explicit, as suggested by the fact that at least one awarding body uses grade improvement in its marketing, stating that its science GCSE course is “Proven to help improve grades...” backed up by a teacher’s comments that she had “seen a big 18% increase in C+ grades”.2 Furthermore, Sir Mark Walport, chair of the Science and Learning Expert Group observed that, when giving evidence, awarding bodies openly admitted that they struggle to avoid competing with each other on grade standards.

5. Other serious concerns arising from competition across awarding bodies include:

(i)Variation in awarding processes across the bodies, and lack of transparency about how grades are arrived at;

(ii)Errors in examination papers and the quality of the questioning in exams;

(iii)Endorsement of textbooks by awarding bodies;

(iv)The low level of teacher, HEI and professional body engagement in development of examinations.

6. The issues associated with multiple awarding bodies are not new. However, many recommendations to improve the system, such as those from the Science and Learning Expert Group report in 2010, have clearly not been implemented. The high-profile errors evident in examinations in 2010 and 2011 highlight the urgent need for vital improvements.

7. Given the remit of the Wellcome Trust, we are particularly concerned with examinations in STEM subjects, including the insufficient mathematics content in science GCSEs and A levels. This response focuses on academic rather than vocational qualifications. In a recent speech, the Secretary of State for Education acknowledged concerns from the business community, universities and professional bodies over declining standards in school science.3

8. Another apparently conflicting concern is the relatively greater difficulty of achieving high grades in science and mathematics.4 While we would absolutely not wish to see a “dumbing down” in the quality of science qualifications or assessment, this issue needs to be addressed. Either grade boundaries need to be raised in some of the less stringently marked subjects, or there needs to be much greater and more overt recognition of the high value of the skills and knowledge developed by science and mathematics. Without these steps, students may be discouraged from post-16 science and mathematics, exactly the opposite of what the UK needs, as highlighted in a recent report from the Royal Society.5

Arguments for and against having a range of awarding bodies and the merits of alternative arrangements

9. We set out arguments for and against a number of different models for awarding bodies below. We then consider these arguments, along with international comparisons, to inform our position on the best way forward.

Model 1: The status quo: a range of awarding bodies

10. A range of awarding bodies (currently AQA, Edexcel and OCR for academic qualifications) competing for custom has existed for many years in England.

For:

Allows for a diversity of provision, delivering a range of skills and knowledge in the population (although, individual students are rarely matched to examination).

Gives capacity for the large number of examinations required in England, reducing the associated risk (ie a problem with one body would only affect a proportion of students).

Can promote innovation.

Against:

Limits the transparency of awarding bodies leading to variation in grade standards and reduced comparability across and within subjects.

Arguably drives down the rigour of examinations (paragraphs three to five).

Limits the spread of innovations and best practice.

Requires an examination (and often more than one, due to multiple syllabuses) to be set for each subject, thereby diluting the expertise of examiners.

Model 2: A single awarding body

11. For:

Removes competition on standards.

Increases confidence in comparability across subjects.

Gives better economies of scale including simplifying the incorporation of changes in curricula into examinations, and reducing the workload and therefore dilution of examiners.

Provides a single point of contact for external partners such as HEIs.

Against:

Any problems would affect all students.

Could reduce the diversity of provision unless the body offers a range of examinations within individual subjects.

Could reduce innovation, unless clear drivers were in place.

Model 3: Contracting subject groups to different awarding bodies

12. Ofqual regulates the work of awarding bodies, allowing them the exclusive right to run examinations for different groups of qualifications for a time—awarding bodies would have to bid for each subject contract and then run the examinations in their contracted subjects for, say, five years.

For:

Gives many of the advantages of a single body model, with inbuilt innovation and some risk reduction.

Eliminates competition in examinations for each subject and external drivers for variability in standards.

Allows each awarding body to concentrate on one particular area of education at a given time, drawing upon all expertise available.

Against:

May be difficult to assure continuity over transfer of groups of examinations to another awarding body after the lifetime of a contract.

Could reduce comparability of grades across subjects set by different awarding bodies.

Could be financially challenging for awarding bodies given that the profits generated from a small number of large entry subjects (eg English and mathematics) currently offset any losses incurred from smaller subjects.

International comparisons

13. The Government has put much emphasis on international comparisons since coming into office. We therefore thought it appropriate to highlight the model of working in other countries. We have yet to identify a country, other than England, that operates a model of multiple competing awarding bodies. Countries such as Finland, Singapore and Korea, who are all higher that England in the PISA standings for science and mathematics, operate a single awarding body model.6

14. Scotland also has a single awarding body, the Scottish Qualifications Authority (SQA), and due to its cultural similarity should be used as a key comparator with England (notwithstanding the difference in educational structure). Having one body running all examinations arguably allows SQA to:

(i)Incorporate new curricula into examinations more easily and make incremental revisions to examinations more regularly to reflect progression in subjects.

(ii)Facilitate interactions with stakeholders who only have to consult with one body.

(iii)Ensure stability of grade standards.

Observations

15. A single awarding body for England could solve many of the problems associated with variability in standards and grade slippage that we currently experience. However, moving to a single body system would require overhaul of the entire examination system in England. In a time when the English education system is changing with, for example, the review of the National Curriculum and the Academies Act, such disruption may not be wise.

16. Given the considerable problems associated with having multiple awarding bodies, this would not be the model of choice if we were designing the system from scratch. However, the best way forward may be with resolving the problems within this system.

17. The contracting model seems like a good idea in principle. However, the practical implications of awarding bodies bidding for subjects every five years could have a detrimental impact on their ability to run examinations effectively. It would be difficult to ensure continuity in subjects when contracts are changed, as well as comparability across the subject groups.

Ensuring accuracy in setting papers, marking scripts, and awarding grades

Accuracy in setting papers and marking scripts

18. The quality of the examination system can never be better than the quality of examination papers. Appropriate expertise in setting papers is essential for the delivery of high quality examinations. The sheer scale of the examination system in England, and the need for awarding bodies to each produce separate examination papers, produces a dilution of expertise in setting examination questions. This could reduce the quality of the questions and even result in errors. However, proposals to reduce the number of re-sits, including ensuring all GCSE examinations are taken at the end of the two years rather than some taken part way through the course, should address some of these issues as fewer papers would be needed.7

19. The role of awarding bodies in recruiting and training examiners is very important in ensuring that examination questions are of high quality.8 This is particularly important in the sciences, where a recent report has shown that incorporation of “How Science Works‘ into awarding bodies” specifications at GCSE has been variable.9 The need for continuing professional development of examiners should be explored.

20. Another problem lies with the rotation of specifications by Ofqual every five years. This is disruptive to schools and unhelpful for awarding bodies as it usually requires major changes to examination papers. This puts more burden on examiners setting a new range of questions (including sample questions). In addition, not enough time is given for accurate incorporation of the new specifications. Critically, although lip-service is paid to consultation, it often amounts to inviting subject experts to rubber-stamp near-final proposals. We therefore urge Ofqual to discontinue the rotation of specifications every five years.

21. We also propose the establishment of national subject committees to provide expert input for incremental revision of examinations (further remit of these national subject committees is outlined in paragraph 33). Awarding bodies should be required to use this expertise to regularly revise examination specifications and papers to keep up with progress in the subject and its teaching. This might involve small changes, more often, but would increase the quality of examination questions, give more continuity, and keep pace with developments in the subject and its applications in industry and research.

Awarding grades

22. It is important that the public, employers and universities understand and have confidence in the process by which grades are arrived at. Yet little is known publicly about how awarding bodies proceed from marked scripts to final grades.10

23. All boards use a combination of criterion referencing (setting a fixed standard for each grade) and cohort-referencing (comparing individual performance against that of the overall cohort). Although criterion referencing is attractive, in practice there are serious difficulties, arising from the fact that – in contrast to, say, the national driving test—it is well-nigh impossible to write down statements of attainment criteria that are not wide open to subjective interpretation. In practice, all boards resort to moderating criterion-referenced marks statistically. The Secretary of State recently suggested that A* grades could operate entirely on a norm-referenced system, with a fixed percentage of students receiving the grade within each subject, and that there could be more information on ranking to sit alongside the grading system.11

24. Ofqual should be charged with identifying the best system of grading, which can recognise when students have reached a certain standard, can reflect if there is a marked increase in standards but guard against grade drift, and that delivers comparability across years and subjects. This system should be implemented across all awarding bodies and clearly communicated to the public.

25. It is apparent that some A levels are marked more strictly than others. For example, a study undertaken by Durham University12 concluded that science and mathematics A level were significantly more severely marked on average than many non-science subjects, with A level Physics, for example, being some two grades harder than A level Art. This was also seen in science and mathematics at GCSE, but to a lower degree.

26. Perhaps even more problematic, is the fact that students perceive it to be harder to get high grades in science and mathematics and this perception is likely to be factored into their subject choices. It is therefore vital that the high value of studying science and mathematics is overtly recognised and communicated so that student uptake is not affected. Pupils should be made aware that taking these subjects would provide them with skills that are highly valued by HEIs and employers alike—a recent report by the Russell Group provides a useful starting point.13

Mathematics content in science specifications and examinations

27. A specific area of concern is the low mathematical content of science specifications and examinations at GCSE and A level, highlighted by the report of the Science and Learning Expert Group.14 Mathematical content should be strengthened within the science specifications and, most importantly, required in the actual examination questions. This is something that Ofqual can and should demand as a matter of urgency.

The commercial activities of awarding bodies, including examination fees and textbooks, and their impact on schools and pupils

28. The danger with any examination system is that learning becomes directed towards achieving the best examination results rather than giving students a broad understanding of a subject—“the tail that wags the dog”. Textbooks have increasingly become “examination guides” instead of providing broad and deep knowledge.15 The endorsement of textbooks by awarding bodies exacerbates this problem by promoting teaching to the test. Of particular concern is that examiners are commissioned to write textbooks that are endorsed by an awarding body. This carries the risk that examinations could be used to maximise sales of books rather than in the public interest.

29. In Scotland, the SQA does not endorse specific textbooks, but supports a wide range of resources, including electronic resources, that can be used to teach the courses effectively. This is a sensible approach that does not unduly influence teachers or schools when purchasing appropriate materials. We therefore urge the Government to take action to stop awarding bodies endorsing textbooks through inclusion in Ofqual’s Codes of Practice.

University Admissions Process

30. Although not directly asked about in this inquiry, we would like to comment on the recent proposal from UCAS to move to Post Qualification Admissions, a model where the decision on admission to full time undergraduate courses is made following the award of A levels.16 We support this proposal and believe moving to a process like this would:

(i)widen participation, as admission would be based on actual results rather than predictions;

(ii)reduce the pressure on students to take more and more GCSEs as admission to university would be based on actual A level results; and

(iii)limit the need for multiple examinations throughout the two year A level course.

31. However, advice and support for students going through the application process is already variable. It will be essential to ensure that schools sufficiently support their students through the process, even if this is occurring over the summer holidays.

Conclusions

32. Now is the time for real action to improve the examination system in England. If we were establishing a new examination system from scratch, a single awarding body would seem the most favourable. However, we are not in this situation, and unless there was wholesale overhaul of the education system, the disruption required to move to a single body would not be worthwhile.

33. We believe that the range of awarding bodies that are currently in place for academic qualifications should remain, but with urgent improvements. These essential improvements must ensure transparency, comparability and consistency in examinations across the awarding bodies, and promote raising standards. To achieve this we recommend that:

(i)All awarding bodies must be transparent about their process of awarding grades, and there should be consistency across boards in these processes.

(ii)Awarding bodies must communicate better with each other, especially in spreading best practice and introducing innovations in the system. Commercial confidentiality should not be used as an excuse for withholding information that would act in the interest of improving the quality of the whole examination system.

(iii)National subject committees should be established for each major subject to oversee standards across all awarding bodies. These committees should be convened by professional bodies and contain representatives from HEIs, employers and practicing teachers.

(iv)The rotation of specifications every five years encourages wholesale changes to examinations and is disruptive for schools. Instead, awarding bodies should have the ability to make incremental changes to examinations, keeping them up-to-date with developments in the subject. The national subject committees should provide expert input to this process. This is particularly important in science subjects where developments are seen continuously.

34. Awarding bodies must be accountable for the performance of the examination system and should be given a strict timeline for reform. They should be in no doubt of the seriousness of the situation they are in. Should they fail to make these vital improvements, Ofqual must take decisive action, including consider the other models discussed.

35. Finally, if an improved system produces higher standards there must be a mechanism of acknowledging these improvements so they are not perceived as “grade slippage”. Real improvements would produce better grades, and how this is achieved must be communicated to the public effectively.

November 2011