Education CommitteeWritten evidence submitted by the Joint Council for Qualifications (JCQ)

About JCQ

1. The Joint Council for Qualifications (JCQ) is a membership body comprising the seven largest national awarding bodies offering qualifications in the UK. These are AQA, CCEA, City & Guilds, Edexcel, OCR, SQA and WJEC.

2. The JCQ exists to provide:

(a)An opportunity for strategic debate, information exchange and expression of common interest amongst awarding bodies.

(b)A means to enable the awarding bodies jointly to express views on, and respond to, national issues as they arise.

(c)A vehicle for collective approaches, where relevant, to government, political and other influential parties.

(d)A means of collective discussion with teachers, their organisations and other valued stakeholders, on matters of common concern.

(e)Common administrative arrangements for schools and colleges.

3. Its members cover a range of qualifications that include: Entry Level; GCSE; Diplomas; GCE; Advanced Extension Award; Vocational; Extended Project; Scottish Highers; Basic and Key Skills and Functional Skills.

Overview

4. This submission provides an overview of the examination system in the UK, the role of awarding bodies, and information relating to the errors in this year’s examination papers. It supports submissions made separately by some of JCQ’s members. The JCQ does not enter into discussions of awarding bodies’ commercial activities.

The Scale of the Examinations System

5. The public examination system is a vast and highly complex operation deploying some of the nation’s best educational expertise in an intensive period to produce results across a range of qualifications.

6. Each year, it employs over 50,000 examiners who mark over 25 million separate examination scripts and pieces of coursework. Awarding bodies set over 60,000 examination questions each year and issue over two million GCE results and around six million GCSE results.

7. Students have a choice of more than 70 different subject titles at both GCSE and AS/A-level as well as a range of vocational and Entry Level subjects. In most subjects there is a choice of syllabus or specification and a choice of content. This provides the opportunity for innovation, centre choice, and competition between awarding bodies to meet the needs of centres, learners, universities and employers.

The Role of Awarding Bodies

8. Awarding bodies have a wide range of roles and responsibilities. At a basic level they:

(a)design syllabuses—which are descriptions of what skills, knowledge and understanding are required in a particular course. This is often done in collaboration with education partners such as Salters Foundation, Nuffield Foundation, Institute of Physics, Schools History Project, and universities;

(b)write examination questions and develop mark schemes—which are based on the whole course and are set at a range of different levels of difficulty. This process starts nearly two years before the examination is taken and involves a large number of specialists including chairs of examiners, chief examiners, principal examiners, revisers and scrutineers; and

(c)mark, grade and award qualifications—which involves a huge logistical operation to gather completed scripts, get them marked by examiners and then sent to awarding bodies for the awarding process and then grading. All of this is completed in a 12 week window from the date of the first examination to the publication of results for all examinations. The examinations themselves take place in the first half of this period.

9. Awarding bodies undertake much more than these core activities. They provide training for teachers, give a huge amount of administrative support to centres, invest in improving the use of technology, deliver innovation to the system and the qualifications, and carry out important research, all with the aim of developing and improving the UK’s examination system.

Ensuring Accuracy in Setting Papers, Marking Scripts and Awarding Grades

10. Awarding bodies understand that accuracy is essential in delivering a successful world-class examinations system that has the confidence of all stakeholders.

11. Detailed and well-tested procedures are in place to eliminate errors that may occur and ensure standards are maintained in what is a long and complex process that relies on human input and judgement.

12. This year saw too many major errors appearing on examination papers. Awarding bodies deeply regret these errors but were able to react swiftly to ensure students were not disadvantaged.

13. Awarding bodies undertook their own investigations into the errors, which found that there was no systemic failure of the examination system. The number of major errors reported in 2011 represented an unusual cluster of unrelated errors appearing in one year with a separate explanation for each individual error.

14. The investigations did highlight areas in which awarding bodies can make improvements to their own processes to minimise the risk of mistakes and these are being implemented.

15. Such a large and complex system that has human involvement will always carry an element of risk. The UK examinations system does operate to a very high standard with an extremely high level of accuracy, but further improvements are being made to prevent a repeat of 2011 and so that confidence can be restored.

16. An example of how awarding bodies ensure standards are maintained is by undertaking statistical screening each year, which comprises a post hoc analysis of how statistically aligned the results for awarding bodies’ specifications in a subject were. The method compares specification outcomes, adjusted to take into account any difference in the ability of their entry cohorts. Those measures of ability for GCE and GCSE specifications are derived respectively from candidates’ mean prior and mean concurrent GCSE score. In the event of any unexplainable misalignment, remedial action is recommended for the following year.

Conclusion

17. The current set-up allows and incentivises innovation in the development of qualifications and the system as a whole, within a regulated framework. It provides a broad choice for learners and schools and colleges.

18. The UK examinations system is large and complex. There are processes in place to ensure accuracy and that standards are maintained. The errors on examination papers this year are not a consequence of a fault in the system, but a series of unrelated mistakes appearing in one year. The processes and procedures employed have been scrutinised by awarding bodies and changes are being made to further minimise the risk of errors occurring again in the future.

November 2011