House of Commons - The administration of examinations for 15-19 year olds in England

The administration of examinations for 15-19 year olds in England - Education Committee Contents

5 The role of Ofqual

Introduction

96. Ofqual is pivotal in the current examination system. The Secretary of State has said that Ofqual needs to move from being an organisation that provides reassurance to one that provides challenge to politicians and exam boards.[147] There were recurring calls in both oral and written evidence for a stronger Ofqual, whatever organisational model is adopted. SCORE told us that "whatever system is in place, there is a need for some form of external regulation or scrutiny". It criticised Ofqual for being "a crash scene investigator rather than an air traffic controller".[148] Professor Nick Lieven of Bristol University told us that "Ofqual needs more teeth. Whether you have one examination board or 10, it is the checks and balances you have in the system that are important".[149]

97. The Secretary of State has also said that if Ofqual is to be an effective watchdog, it needs "sharper teeth".[150] To this end, the Government has legislated to increase Ofqual's regulatory powers, most notably giving it the power to fine exam boards up to 10% of their turnover. It also made an amendment to Ofqual's qualifications standards objective in the Education Act 2011, requiring it to ensure that attainment in English qualifications is consistent with that required by comparable qualifications in other countries, as well as over time in England. From May 2012 Ofqual will be moving to tighter regulatory requirements and, as Chief Executive Glenys Stacey repeatedly told us, it will be "crawling all over" the exam boards.[151] Regarding its powers, Ms Stacey reported to us that Ofqual is "shortly to be awash with them".[152]

98. In this chapter we consider Ofqual's regulation of grading and content standards and whether this is sufficiently robust. We also examine Ofqual's use of assessment and subject expertise and its relationship with the Joint Council for Qualifications (JCQ).

Ofqual's regulation of standards

99. Assessment experts and exam board chief executives have recommended that Ofqual should focus on its standards objective, building in-house assessment expertise and improving the robustness of its comparability work.[153] This, they argue, would serve its public confidence objective, by improving confidence in the system. Mark Dawe, Chief Executive of OCR, told us:

Ofqual's core role should be around [...] standards and comparability. That should be their focus. If they do that properly, it gives us all the approval, in a sense, that our qualifications are appropriate, and it gives the public the confidence they are looking for.[154]

100. Assessment researchers have suggested that Ofqual needs to build its in-house assessment expertise, in order to regulate standards more effectively. They acknowledged that Ofqual has taken steps to improve the methodology of its comparability work although, as Dr Michelle Meadows of AQA's Centre for Research, Education and Policy (CERP) told us, there is "still a way to go before you see the robustness of design that we would like, but it is a journey that we are on".[155] Professor Alison Wolf told the Committee that Ofqual would benefit from "some decent in-house statistical help, looking at comparability and technical issues and not wasting time looking at prices".[156]

101. There are signs, as the DfE suggested to us, that Ofqual "has begun to show a real willingness to tackle awarding bodies on the key issues of standards".[157] Since summer 2010, Ofqual has taken action to contain grade inflation at A level. This has proved effective, and it plans to do the same at GCSE from summer 2012. In October 2011, Ofqual hosted a standards summit to stimulate and inform public debate on exam standards. It has also recently set up a standards advisory group of assessment specialists to "consider and advise it on qualification and assessment standards issues".[158] Amanda Spielman, Chair of Ofqual, told us that "without a shadow of a doubt, we need more assessment expertise in the oversight of Ofqual" and that they are also considering how to bring in such expertise at board level.[159]

102. We agree with assessment experts that Ofqual should focus more tightly on its qualification standards objective, taking steps to improve the methodology of its comparability work and building its in-house expertise. We appreciate the point made by Glenys Stacey that "assessment expertise is quite a rare thing",[160] residing mainly in the exam boards and some universities. We were therefore disappointed to note the absence of an assessment expert in Ofqual's recent announcement of new appointments to its Board.[161] We recommend that Ofqual seek to build its assessment expertise and finds the resources to do so. We further recommend that Ofqual appoint an assessment expert to its board as soon as possible.

OFQUAL'S INTERNATIONAL STANDARDS OBJECTIVE

103. The recent amendment to Ofqual's qualifications standards objective in the Education Act 2011 with its international dimension presents significant challenges to Ofqual in its regulation of standards. Ofqual must ensure that attainment in English qualifications is consistent with that required by comparable qualifications in other countries, as well as over time in England. Glenys Stacey has acknowledged that this could lead to a potential conflict, should standards in England be found to be out of line with those in other countries, telling us that "there is a tension, but I think it is a healthy one".[162] When we asked Ofqual how it would deal with this tension, whether it would prioritise standards over time or internationally, Ms Stacey responded that "the answer is that we want the best [...] we are going to pick up these things, learn, play them out for people, discuss them and then take a view".[163]

104. We recognise that it is important to take account of best practice in assessment in other countries and welcome the work that Ofqual has done in this area so far. We are concerned that the amendment to Ofqual's qualification standards objective could over a period of time pull it simultaneously in different directions and recommend that the Government give a clear indication to Ofqual about which should be the priority: the comparability of standards over time in England or benchmarking against the standards of qualifications in other countries.

GOVERNMENT POLICY CHANGES

105. Similar difficulties may emerge with the Secretary of State's suggestion that "we are going to make exams tougher" and that there may be years "where GCSE and A level results dip".[164] Ofqual's current approach is to maintain standards from year to year. For many years this has been the overriding concern at times of change in the exam system, in order to be fair to students from one year to the next. As journalist Warwick Mansell has questioned, "if Mr Gove does introduce changes which do make it "tougher" to get a good grade at GCSE or A-level one year than it was the previous year, how can this be justified to individual students?"[165] As stated in chapter four, occasional explicit recalibration of grading standards may be required. We recommend that the Government make its priorities clear to Ofqual, whether these are the maintenance of standards over time or making exams tougher, and that both the Government and Ofqual be open about the consequences of these policies for young people.

106. The Secretary of State's recent letter on A level reform would suggest that the Government is prepared to see more diversity in the examination system. Professor Jo-Anne Baird et al noted in evidence to us "there is a tension between the regulator upholding content standards and allowing variation in the syllabus and examination offer".[166] She went on to say that "Ofqual needs to be empowered to foster more diversity in the examinations system, whilst ensuring that evidence is collated to reassure stakeholders that standards have been upheld".[167] If A levels are going to become more varied in structure, Ofqual needs to ensure that its collection of evidence and monitoring of standards are sufficiently robust to provide convincing reassurance that content standards are being maintained.

GCSE CHANGES AND DEVOLUTION

107. The Government has announced that GCSEs in England will be changing from modular to linear courses (a return to end-of-course exams), taking effect for courses starting in September 2012. These changes will not apply in Wales or Northern Ireland, where schools will be able to choose between linear and modular GCSE courses.[168] It will be the responsibility of Ofqual, in conjunction with the regulators in Wales and Northern Ireland, to ensure that GCSEs awarded in each nation are of a comparable standard. According to Cambridge Assessment, "the interrelationship of England's qualifications system with those of Wales, Scotland and Northern Ireland has never been seriously addressed".[169] We can see that increasing diversity between qualifications across the UK could present greater challenges to Ofqual in ensuring comparability of standards. Cambridge Assessment suggests that "it may no longer be tenable for the English regulator to accept by proxy decisions made by the other UK regulators and vice versa".[170] We recommend that Ofqual review its arrangements for ensuring comparability of standards between England, Wales and Northern Ireland, and that it continue to monitor standards in GCSE and A level examinations offered by WJEC and CCEA, as well as the English providers as part of its ongoing regulation of standards. We also believe that a debate is needed on the importance of standards comparability between the home nations, with a Ministerial conference to decide whether and what action is necessary.

ACCREDITATION

108. While Ofqual has been proactive in regulating grading standards, it has been less obviously so in respect of content standards until very recently. As noted in chapter four, several learned bodies have been critical of Ofqual's accreditation procedures. Professor Jo-Anne Baird et al told us that "the process by which content standards of qualifications are judged by Ofqual could be more robust and transparent. A review of methodologies and publication of the process generated by this work is warranted".[171]

109. With reform of A levels imminent and a further full reform of GCSE anticipated following the National Curriculum review, Ofqual's regulation of content standards within subjects, through its accreditation procedures, is likely to feature prominently. Glenys Stacey suggested to us that "there is every incentive for an awarding body to want to get to the point where it does not have to come to us for accreditation [...] there is a real incentive in it getting an endorsement from the regulator on an annual basis that its products pass muster across the spectrum".[172] She also suggested that "the real control over standards should not, in the long term, be through an accreditation process. It should be by us placing requirements on awarding bodies that they must demonstrate they meet time after time in [...] close and continuous monitoring".[173]

110. We have serious concerns about this approach for GCSEs and A levels. We are convinced that the regulator has an important role in ensuring that content standards are appropriate and comparable across the exam boards, at the point of syllabus development through an accreditation process as well as through its ongoing regulation. We recommend that individual accreditation of all new syllabuses, including our recommended national syllabuses, remain a part of Ofqual's continuing regulation of GCSEs and A-levels and, indeed, of any qualifications that are deemed equivalent to GCSEs and A-levels. With this in mind, Ofqual needs to demonstrate that its accreditation procedures are rigorous and transparent, and that it draws on appropriate respected subject and assessment expertise when reviewing draft syllabuses and their associated materials. We recommend that Ofqual review and strengthen its regulation of content standards, including accreditation procedures, seeking and acting upon advice from its standards advisory group as appropriate.

Ofqual and subject expertise

111. We received recurring criticisms of Ofqual's lack of in-house subject expertise and of a lack of transparency in its use of external subject experts.[174] The Advisory Committee on Mathematics Education (ACME) stated that "there is a no transparency in Ofqual's operation in terms of the choice of external subject experts consulted. This results in a lack of confidence in Ofqual's ability to regulate key national examinations effectively".[175] According to SCORE, syllabuses are required to have "official support from their subject community before they are accredited"[176] but the Wellcome Trust told us that "critically, although lip-service is paid to consultation [by Ofqual], it often amounts to inviting subject experts to rubber-stamp near-final proposals".[177] Ofqual told us "we choose not to invest heavily in subject expertise on our staff at Ofqual [...] we prefer to buy it in [...] we choose to broker that in a way that we think is in the best interests of qualifications and standards".[178]

112. Action is clearly needed to define the role of subject communities within the exam system. As SCORE told us, "this lack of engagement with subject communities results in a lack of confidence from users of the system, including HEIs and employers".[179] While we accept Ofqual's rationale for its lack of in-house subject expertise, criticisms from the subject communities lead us to conclude that Ofqual needs to be more transparent about its consultation with and use of external subject experts.

NATIONAL SUBJECT COMMITTEES

113. Several organisations, such as SCORE, ACME and the Wellcome Trust, argued strongly for the establishment of national subject committees, comprising representatives from higher education, employers and subject communities. It was suggested that the remit of such committees should include syllabus development and accreditation, as well as on-going monitoring, to help oversee standards across exam boards. As Professor Sir John Holman of the Wellcome Trust proposed to us, "the national subject committee does not just say what they want to have in the specification; it looks at the individual boards' interpretations of that list and says whether it is good or not. It looks at the sample question papers. It looks at live question papers. It never stops working. It is always watching and monitoring".[180]

114. On the matter of question papers, several learned bodies expressed concern about the type of questions set and what they assess. SCORE, along with several mathematics subject associations and specialists, suggested to us that question papers often do not test important aspects of a subject, although these are in the syllabus.[181] This was a feature of SCORE's recent research on mathematics content in A level science examinations. Dr Ian Jones of the University of Birmingham told us that exams "fail to gather evidence of the very conceptual knowledge and higher order thinking skills that our country values and needs".[182] These issues also emerged in Ofqual's most recent set of standards reviews.[183] Ofqual has indicated that it will use the findings from these reviews to inform the development of regulations for revised A levels and GCSEs. National subject committees would provide a way for subject communities to be involved formally in scrutinising question papers and mark schemes. They would also be a mechanism for Ofqual to gather regular feedback on the type of concerns described above.

115. We consider that national subject committees, convened by Ofqual, would offer a way to formalise Ofqual's engagement with subject communities, as well as improving the involvement of higher education and potentially employers with GCSEs and A levels. National subject committees would help Ofqual to counter criticisms about its lack of in-house subject expertise and to be more transparent in its use of external subject experts. We are aware that "the strength and representation of professional bodies/learned societies vary across different subjects" and this means that national subject committees may work better in some subjects than others.[184] Nonetheless, we consider that there is merit in the idea and we recommend that Ofqual convene national subject committees in large entry GCSE and A level subjects, drawing their membership from learned societies, subject associations, higher education and employers. Such committees should include in their remit syllabus development and accreditation, as well as on-going monitoring of question papers and mark schemes, and oversight of comparable qualifications offered in the devolved nations.

Ofqual and the Joint Council for Qualifications

116. The Joint Council for Qualifications (JCQ) is a membership body, consisting of AQA, City and Guilds, CCEA, Edexcel, OCR, SQA and WJEC. It was formed in 2004 and describes itself as "the single voice for member bodies".[185] The JCQ is not regulated by Ofqual, but its member bodies are. The JCQ collates and publishes results for GCSEs, GCEs and other national qualifications. It issues guidance to schools and colleges on administrative matters common across exam boards, such as access and special consideration, entries, timetabling, instructions for conducting exams and post-results services.

117. Glenys Stacey told us that there is "a grey area between us and JCQ".[186] She explained that "there are some areas where we have a joint interest, where we need to discuss our approach and also whether the prime responsibility is with JCQ or with us. A good example of that is the proper controls over requests for special consideration or extra time, where we have a joint interest but we are the regulator".[187] We agree that a discussion about responsibilities in this "grey area" is needed. Ofqual should instigate discussions with the JCQ to clarify roles and responsibilities in areas where there is a joint interest and publish information about this to schools and colleges as appropriate.

Conclusion

118. Overall the evidence we received suggests that the exam system needs a stronger regulator, whatever organisational model is adopted. Glenys Stacey told us that "historically, the sector has been under-regulated or not firmly regulated" and that she has regarded her work as "turning a ship".[188] We recognise that this is not just about increasing Ofqual's powers and tightening its regulatory requirements, but may also involve a change in the focus and resourcing of Ofqual as an organisation. Although the exam system has been regulated for some time, we appreciate that Ofqual is a relatively new independent regulator (it was established officially on 1 April 2010) and its chief executive and chair have been in post since March 2011 and July 2011 respectively.

119. We believe that Glenys Stacey's decision to concentrate initially on improving Ofqual's regulation of standards was the right one.[189] We are particularly encouraged by evidence that Ofqual is taking more robust action in its regulation of grading standards and we welcome the steps it has taken to engage publicly with the "standards debate". It is clear from the issues raised with us that further improvements are needed if Ofqual is to be a stronger, more challenging and more effective regulator. As AQA's Andrew Hall put it "Ofqual is, in fairness, on a journey".[190] We believe that there is a strong argument in favour of allowing time for a strengthened Ofqual to take effect, as the changes it is making will take time to settle and bear fruit. But Ofqual must demonstrate that it is collecting the right sort of qualitative and quantitative evidence and using robust methodology to regulate effectively. Details of the evidence used by Ofqual in the regulation of standards, and any specific findings and regulatory action on standards, should be set out clearly in annexes to Ofqual's annual report to Parliament. Ofqual must continue to show that it is prepared to take vigorous action when needed, in order to help increase public confidence in the exam system.

147 Michael Gove speech to Ofqual standards summit, 13 October 2011 Back

148 Ev 153 Back

149 Q136 Back

150 Michael Gove speech to Ofqual standards summit, 13 October 2011 Back

151 Q591, Q630 and Q636 Back

152 Q591 Back

153 See Ev 144, Q391 & Q392 Michelle Meadows, Q399 and Q401 Alison Wolf Back

154 Q532 Back

155 Q392 Back

156 Q399 Back

157 Ev 170 Back

158 http://www.ofqual.gov.uk/news-and-announcements/83-news-and-announcements-news/869-standards-advisory-group Back