Education CommitteeWritten evidence submitted by David Leitch

Summary

1. The marking of examination scripts has many risks attached to it. In some subjects there is a shortage of experienced, competent examiners. Less competent examiners make mistakes which, if undiscovered, may result in the issuing of results to candidates which do not reflect their abilities. Once results have been issued, awarding bodies may not have sufficient will to uncover such mistakes and, if uncovered, to rectify them, leaving the responsibility of querying results to centres and candidates.

Introduction

2. I work for Cambridge Assessment, OCR’s parent organisation, as a senior supervisor in the Post Examinations department. The majority of my time is spent dealing with OCR examination scripts—receiving, storing, despatching and otherwise handling them. I have been in this role for five years.

Information

3. Once examination results have been issued to centres, the Access to Scripts (ATS) process begins. This process allows centres and candidates access, for a fee, to their scripts. OCR introduced a new method of dealing with ATS requests for the recent June 2011 series. The new procedure involves scanned pdf images of the requested scripts being produced by staff within my department. These pdfs are uploaded to an internet-based portal to which the centres have access. The previous procedure involved photocopies of scripts being sent to centres for Priority requests and, at a later date, original scripts being sent by post to centres for Non-Priority requests. As part of the new OCR process for dealing with ATS requests, the pdfs are checked to ensure that all of the pages of the scripts have been scanned and that they are as legible as the original scripts themselves.

4. Soon after this checking process began, in the third week of August, it was discovered by one vigilant member of staff that some of the scripts had marks which either had been added up incorrectly or had not been transferred correctly to the front of the scripts by the examiners, thereby resulting in incorrect total marks being awarded to the candidates. “Clerical error” is the term we use to describe such an error. The Qualifications Managers (QMs) who are responsible for the syllabuses were informed and OCR senior managers also became aware of the issue. Over 100 scripts requested by centres as part of the ATS process contained clerical errors. Over 80 examiners were responsible for these errors.

5. My team and I considered it necessary to carry out supplementary checks on the other scripts marked by these examiners in order to discover the extent of the problem. When senior OCR managers became aware of these supplementary checks, they tried on several occasions to stop them taking place. On the 22 of September an OCR assistant director informed my line manager by email that all “unauthorised” checks for clerical errors should stop and that the QMs should not be contacted, a senior manager to be contacted instead. Covertly, we continued to investigate. About 200 further scripts with clerical errors have been found during these supplementary checks.

6. OCR’s chief executive was sent an email on 23 September in which the discovery of clerical errors was brought to his attention. On the 14 October the OCR assistant director and OCR’s director of standards explained, in a meeting with me and two others, the position to be adopted by OCR. The suggested resolution concentrated on the steps to be taken to reduce the scale of the issue for future series. With regard to the current June 2011, they stated that supplementary checks on scripts should not be carried out because these had not been authorised either by OCR or by the centres or by the candidates. Also, they were not willing to inform the centres of the mistakes which had been uncovered in the supplementary checks. They mentioned that, firstly, not all the scripts in storage had been checked for clerical errors and, secondly, the supplementary checks had not been undertaken in previous series. Apparently, these are reasons for not setting right the errors which we have discovered in this June 2011 series. Reluctantly, I said I would not carry out any more supplementary checks. At the end of the meeting, I said I was unhappy with OCR’s proposed resolution and wanted to discuss the matter with Ofqual. On 17 October I informed Ofqual of my concerns. The matter has been passed to its investigations team. Subsequently, OCR informed my line manager that particular supplementary checks should take place. This has been done and the information passed back to OCR. There are, however, still many examiners who made clerical errors for whom full supplementary checks have not taken place.

7. We have found about 300 scripts with clerical errors for the June 2011 series. Less than 5% of the scripts we have in storage have been checked for clerical errors by my team.

8. Ofqual’s General Conditions of Recognition, with which all awarding bodies must comply, states that: “Where the application of an appeals process in the case of a Learner leads an awarding organisation to discover a failure in its assessment process, it must take all reasonable steps to—(a) identify any other Learner who has been affected by the failure, (b) correct or, where it cannot be corrected, mitigate as far as possible the effect of the failure, and (c) ensure that the failure does not recur in the future.” This refers to the “appeals process”. Does any Cambridge Assessment or OCR representative coming before the committee believe this instruction extends, in spirit at least, to the Access to Scripts process? If so, is the Cambridge Assessment or OCR representative happy that, not only have several senior OCR staff not abided by the instruction, they have also forced, or tried to force, others not to abide by it?

9. With reference to paragraph 8, how can an awarding body ensure that the failure of an examiner to add up their marks correctly does not recur if the full extent of the examiner’s errors is not uncovered? Decisions regarding the suitability of examiners to mark need to be as informed as possible.

Recommendations

10. Closer scrutiny of examiners’ marking of examination scripts should take place, by the awarding bodies themselves and by an outside agency. The outside agency should be allowed full, unfettered access to the script storage area to undertake its own checks.

11. After examination results have been issued, any errors discovered in the results issued to centres and candidates should be notified to the regulator, Ofqual, and to the centres and candidates. Any candidate whose work merits a higher grade should be awarded a new, higher grade.

12. Awarding bodies should offer full support to all members of staff who discover errors in the marking of examination scripts, whether pre- or post-results issue.

November 2011