5 The role of Ofqual |
96. Ofqual is pivotal in the current examination
system. The Secretary of State has said that Ofqual needs to move
from being an organisation that provides reassurance to one that
provides challenge to politicians and exam boards.
There were recurring calls in both oral and written evidence for
a stronger Ofqual, whatever organisational model is adopted. SCORE
told us that "whatever system is in place, there is a need
for some form of external regulation or scrutiny". It criticised
Ofqual for being "a crash scene investigator rather than
an air traffic controller".
Professor Nick Lieven of Bristol University told us that "Ofqual
needs more teeth. Whether you have one examination board or 10,
it is the checks and balances you have in the system that are
97. The Secretary of State has also said that if
Ofqual is to be an effective watchdog, it needs "sharper
this end, the Government has legislated to increase Ofqual's regulatory
powers, most notably giving it the power to fine exam boards up
to 10% of their turnover. It also made an amendment to Ofqual's
qualifications standards objective in the Education Act 2011,
requiring it to ensure that attainment in English qualifications
is consistent with that required by comparable qualifications
in other countries, as well as over time in England. From May
2012 Ofqual will be moving to tighter regulatory requirements
and, as Chief Executive Glenys Stacey repeatedly told us, it will
be "crawling all over" the exam boards.
Regarding its powers, Ms Stacey reported to us that Ofqual is
"shortly to be awash with them".
98. In this chapter we consider Ofqual's regulation
of grading and content standards and whether this is sufficiently
robust. We also examine Ofqual's use of assessment and subject
expertise and its relationship with the Joint Council for Qualifications
Ofqual's regulation of standards
99. Assessment experts and exam board chief executives
have recommended that Ofqual should focus on its standards objective,
building in-house assessment expertise and improving the robustness
of its comparability work.
This, they argue, would serve its public confidence objective,
by improving confidence in the system. Mark Dawe, Chief Executive
of OCR, told us:
Ofqual's core role should be around [...] standards
and comparability. That should be their focus. If they do that
properly, it gives us all the approval, in a sense, that our qualifications
are appropriate, and it gives the public the confidence they are
100. Assessment researchers have suggested that Ofqual
needs to build its in-house assessment expertise, in order to
regulate standards more effectively. They acknowledged that Ofqual
has taken steps to improve the methodology of its comparability
work although, as Dr Michelle Meadows of AQA's Centre for Research,
Education and Policy (CERP) told us, there is "still a way
to go before you see the robustness of design that we would like,
but it is a journey that we are on".
Professor Alison Wolf told the Committee that Ofqual would benefit
from "some decent in-house statistical help, looking at comparability
and technical issues and not wasting time looking at prices".
101. There are signs, as the DfE suggested to us,
that Ofqual "has begun to show a real willingness to tackle
awarding bodies on the key issues of standards".
Since summer 2010, Ofqual has taken action to contain grade inflation
at A level. This has proved effective, and it plans to do the
same at GCSE from summer 2012. In October 2011, Ofqual hosted
a standards summit to stimulate and inform public debate on exam
standards. It has also recently set up a standards advisory group
of assessment specialists to "consider and advise it on qualification
and assessment standards issues".
Amanda Spielman, Chair of Ofqual, told us that "without a
shadow of a doubt, we need more assessment expertise in the oversight
of Ofqual" and that they are also considering how to bring
in such expertise at board level.
102. We agree with assessment experts that Ofqual
should focus more tightly on its qualification standards objective,
taking steps to improve the methodology of its comparability work
and building its in-house expertise. We appreciate the point made
by Glenys Stacey that "assessment expertise is quite a rare
mainly in the exam boards and some universities. We were therefore
disappointed to note the absence of an assessment expert in Ofqual's
recent announcement of new appointments to its Board.
We recommend that Ofqual
seek to build its assessment expertise and finds the resources
to do so. We further recommend that Ofqual appoint an assessment
expert to its board as soon as possible.
OFQUAL'S INTERNATIONAL STANDARDS
103. The recent amendment to Ofqual's qualifications
standards objective in the Education Act 2011 with its international
dimension presents significant challenges to Ofqual in its regulation
of standards. Ofqual must ensure that attainment in English qualifications
is consistent with that required by comparable qualifications
in other countries, as well as over time in England. Glenys Stacey
has acknowledged that this could lead to a potential conflict,
should standards in England be found to be out of line with those
in other countries, telling us that "there is a tension,
but I think it is a healthy one".
When we asked Ofqual how it would deal with this tension, whether
it would prioritise standards over time or internationally, Ms
Stacey responded that "the answer is that we want the best
[...] we are going to pick up these things, learn, play them out
for people, discuss them and then take a view".
104. We recognise that it is important to take account
of best practice in assessment in other countries and welcome
the work that Ofqual has done in this area so far. We
are concerned that the amendment to Ofqual's qualification standards
objective could over a period of time pull it simultaneously in
different directions and recommend that the Government give a
clear indication to Ofqual about which should be the priority:
the comparability of standards over time in England or benchmarking
against the standards of qualifications in other countries.
GOVERNMENT POLICY CHANGES
105. Similar difficulties may emerge with the Secretary
of State's suggestion that "we are going to make exams tougher"
and that there may be years "where GCSE and A level results
current approach is to maintain standards from year to year. For
many years this has been the overriding concern at times of change
in the exam system, in order to be fair to students from one year
to the next. As journalist Warwick Mansell has questioned, "if
Mr Gove does introduce changes which do make it "tougher"
to get a good grade at GCSE or A-level one year than it was the
previous year, how can this be justified to individual students?"
As stated in chapter four, occasional explicit recalibration of
grading standards may be required. We
recommend that the Government make its priorities clear to Ofqual,
whether these are the maintenance of standards over time or making
exams tougher, and that both the Government and Ofqual be open
about the consequences of these policies for young people.
106. The Secretary of State's recent letter on A
level reform would suggest that the Government is prepared to
see more diversity in the examination system. Professor Jo-Anne
Baird et al noted in evidence to us "there is a tension between
the regulator upholding content standards and allowing variation
in the syllabus and examination offer".
She went on to say that "Ofqual needs to be empowered to
foster more diversity in the examinations system, whilst ensuring
that evidence is collated to reassure stakeholders that standards
have been upheld".
If A levels are going to
become more varied in structure, Ofqual needs to ensure that its
collection of evidence and monitoring of standards are sufficiently
robust to provide convincing reassurance that content standards
are being maintained.
GCSE CHANGES AND DEVOLUTION
107. The Government has announced that GCSEs in England
will be changing from modular to linear courses (a return to end-of-course
exams), taking effect for courses starting in September 2012.
These changes will not apply in Wales or Northern Ireland, where
schools will be able to choose between linear and modular GCSE
courses. It will
be the responsibility of Ofqual, in conjunction with the regulators
in Wales and Northern Ireland, to ensure that GCSEs awarded in
each nation are of a comparable standard. According to Cambridge
Assessment, "the interrelationship of England's qualifications
system with those of Wales, Scotland and Northern Ireland has
never been seriously addressed".
We can see that increasing diversity between qualifications across
the UK could present greater challenges to Ofqual in ensuring
comparability of standards. Cambridge Assessment suggests that
"it may no longer be tenable for the English regulator to
accept by proxy decisions made by the other UK regulators and
that Ofqual review its arrangements for ensuring comparability
of standards between England, Wales and Northern Ireland, and
that it continue to monitor standards in GCSE and A level examinations
offered by WJEC and CCEA, as well as the English providers as
part of its ongoing regulation of standards. We also believe that
a debate is needed on the importance of standards comparability
between the home nations, with a Ministerial conference to decide
whether and what action is necessary.
108. While Ofqual has been proactive in regulating
grading standards, it has been less obviously so in respect of
content standards until very recently. As noted in chapter four,
several learned bodies have been critical of Ofqual's accreditation
procedures. Professor Jo-Anne Baird et al told us that "the
process by which content standards of qualifications are judged
by Ofqual could be more robust and transparent. A review of methodologies
and publication of the process generated by this work is warranted".
109. With reform of A levels imminent and a further
full reform of GCSE anticipated following the National Curriculum
review, Ofqual's regulation of content standards within subjects,
through its accreditation procedures, is likely to feature prominently.
Glenys Stacey suggested to us that "there is every incentive
for an awarding body to want to get to the point where it does
not have to come to us for accreditation [...] there is a real
incentive in it getting an endorsement from the regulator on an
annual basis that its products pass muster across the spectrum".
She also suggested that "the real control over standards
should not, in the long term, be through an accreditation process.
It should be by us placing requirements on awarding bodies that
they must demonstrate they meet time after time in [...] close
and continuous monitoring".
110. We have serious concerns about this approach
for GCSEs and A levels. We are convinced that the regulator has
an important role in ensuring that content standards are appropriate
and comparable across the exam boards, at the point of syllabus
development through an accreditation process as
well as through its ongoing regulation. We
recommend that individual accreditation of all new syllabuses,
including our recommended national syllabuses, remain a part of
Ofqual's continuing regulation of GCSEs and A-levels and, indeed,
of any qualifications that are deemed equivalent to GCSEs and
A-levels. With this in mind, Ofqual
needs to demonstrate that its accreditation procedures are rigorous
and transparent, and that it draws on appropriate respected subject
and assessment expertise when reviewing draft syllabuses and their
associated materials. We recommend that Ofqual review and strengthen
its regulation of content standards, including accreditation procedures,
seeking and acting upon advice from its standards advisory group
Ofqual and subject expertise
111. We received recurring criticisms of Ofqual's
lack of in-house subject expertise and of a lack of transparency
in its use of external subject experts.
The Advisory Committee on Mathematics Education (ACME) stated
that "there is a no transparency in Ofqual's operation in
terms of the choice of external subject experts consulted. This
results in a lack of confidence in Ofqual's ability to regulate
key national examinations effectively".
According to SCORE, syllabuses are required to have "official
support from their subject community before they are accredited"
but the Wellcome Trust told us that "critically, although
lip-service is paid to consultation [by Ofqual], it often amounts
to inviting subject experts to rubber-stamp near-final proposals".
Ofqual told us "we choose not to invest heavily in subject
expertise on our staff at Ofqual [...] we prefer to buy it in
[...] we choose to broker that in a way that we think is in the
best interests of qualifications and standards".
112. Action is clearly needed to define the role
of subject communities within the exam system. As SCORE told us,
"this lack of engagement with subject communities results
in a lack of confidence from users of the system, including HEIs
While we accept Ofqual's
rationale for its lack of in-house subject expertise, criticisms
from the subject communities lead us to conclude that Ofqual needs
to be more transparent about its consultation with and use of
external subject experts.
NATIONAL SUBJECT COMMITTEES
113. Several organisations, such as SCORE, ACME and
the Wellcome Trust, argued strongly for the establishment of national
subject committees, comprising representatives from higher education,
employers and subject communities. It was suggested that the remit
of such committees should include syllabus development and accreditation,
as well as on-going monitoring, to help oversee standards across
exam boards. As Professor Sir John Holman of the Wellcome Trust
proposed to us, "the national subject committee does not
just say what they want to have in the specification; it looks
at the individual boards' interpretations of that list and says
whether it is good or not. It looks at the sample question papers.
It looks at live question papers. It never stops working. It is
always watching and monitoring".
114. On the matter of question papers, several
learned bodies expressed concern about the type of questions set
and what they assess. SCORE, along with several mathematics subject
associations and specialists, suggested to us that question papers
often do not test important aspects of a subject, although these
are in the syllabus.
This was a feature of SCORE's recent research on mathematics content
in A level science examinations. Dr Ian Jones of the University
of Birmingham told us that exams "fail to gather evidence
of the very conceptual knowledge and higher order thinking skills
that our country values and needs".
These issues also emerged in Ofqual's most recent set of standards
has indicated that it will use the findings from these reviews
to inform the development of regulations for revised A levels
and GCSEs. National subject committees would provide a way for
subject communities to be involved formally in scrutinising question
papers and mark schemes. They would also be a mechanism for Ofqual
to gather regular feedback on the type of concerns described above.
115. We consider that national subject committees,
convened by Ofqual, would offer a way to formalise Ofqual's engagement
with subject communities, as well as improving the involvement
of higher education and potentially employers with GCSEs and A
levels. National subject committees would help Ofqual to counter
criticisms about its lack of in-house subject expertise and to
be more transparent in its use of external subject experts. We
are aware that "the strength and representation of professional
bodies/learned societies vary across different subjects"
and this means that national subject committees may work better
in some subjects than others.
Nonetheless, we consider that there is merit in the idea and we
recommend that Ofqual convene national subject committees in large
entry GCSE and A level subjects, drawing their membership from
learned societies, subject associations, higher education and
employers. Such committees should include in their remit syllabus
development and accreditation, as well as on-going monitoring
of question papers and mark schemes, and oversight of comparable
qualifications offered in the devolved nations.
Ofqual and the Joint Council
116. The Joint Council for Qualifications (JCQ) is
a membership body, consisting of AQA, City and Guilds, CCEA, Edexcel,
OCR, SQA and WJEC. It was formed in 2004 and describes itself
as "the single voice for member bodies".
The JCQ is not regulated by Ofqual, but its member bodies are.
The JCQ collates and publishes results for GCSEs, GCEs and other
national qualifications. It issues guidance to schools and colleges
on administrative matters common across exam boards, such as access
and special consideration, entries, timetabling, instructions
for conducting exams and post-results services.
117. Glenys Stacey told us that there is "a
grey area between us and JCQ".
She explained that "there are some areas where we have a
joint interest, where we need to discuss our approach and also
whether the prime responsibility is with JCQ or with us. A good
example of that is the proper controls over requests for special
consideration or extra time, where we have a joint interest but
we are the regulator".
We agree that a discussion about responsibilities in this "grey
area" is needed. Ofqual
should instigate discussions with the JCQ to clarify roles and
responsibilities in areas where there is a joint interest and
publish information about this to schools and colleges as appropriate.
118. Overall the evidence we received suggests that
the exam system needs a stronger regulator, whatever organisational
model is adopted. Glenys Stacey told us that "historically,
the sector has been under-regulated or not firmly regulated"
and that she has regarded her work as "turning a ship".
We recognise that this is not just about increasing Ofqual's powers
and tightening its regulatory requirements, but may also involve
a change in the focus and resourcing of Ofqual as an organisation.
Although the exam system has been regulated for some time, we
appreciate that Ofqual is a relatively new independent regulator
(it was established officially on 1 April 2010) and its chief
executive and chair have been in post since March 2011 and July
119. We believe that Glenys Stacey's decision to
concentrate initially on improving Ofqual's regulation of standards
was the right one.
We are particularly encouraged by evidence that Ofqual is taking
more robust action in its regulation of grading standards and
we welcome the steps it has taken to engage publicly with the
"standards debate". It
is clear from the issues raised with us that further improvements
are needed if Ofqual is to be a stronger, more challenging and
more effective regulator. As AQA's Andrew Hall put it "Ofqual
is, in fairness, on a journey".
We believe that there is a strong argument in favour of allowing
time for a strengthened Ofqual to take effect, as the changes
it is making will take time to settle and bear fruit. But Ofqual
must demonstrate that it is collecting the right sort of qualitative
and quantitative evidence and using robust methodology to regulate
effectively. Details of the evidence used by Ofqual in the regulation
of standards, and any specific findings and regulatory action
on standards, should be set out clearly in annexes to Ofqual's
annual report to Parliament. Ofqual must continue to show that
it is prepared to take vigorous action when needed, in order to
help increase public confidence in the exam system.
147 Michael Gove speech to Ofqual standards summit,
13 October 2011 Back
Ev 153 Back
Michael Gove speech to Ofqual standards summit, 13 October 2011 Back
Q591, Q630 and Q636 Back
See Ev 144, Q391 & Q392 Michelle Meadows, Q399 and Q401 Alison
Ev 170 Back
Chief Regulator of Qualifications and Examinations, Eighth
Report of Session 2010-12, HC 1764-i, Q34 Back
Michael Gove speech to Ofqual standards summit, 13 October 2011
and "Michael Gove: Get set for new age of exam failures",
The Independent, 22 February 2012 Back
Ev 179, paragraph 5.1 Back
Education Minister says no to "linear only" GCSEs and
yes to choice, CCEA press release, 12 March 2012 and Changes to
GCSEs in England, Wales and Northern Ireland, WJEC, http://www.wjec.co.uk/index.php?nav=14&news=206
Ev 148 Back
Ev 181, paragraph 6.8 Back
See for example Ev w35, Ev w45, Ev w59, Ev w78 and Ev 153 Back
Ev w78 Back
Ev 153 Back
Ev 131 Back
Ev 155 Back
Ev 153-6, paragraph 2, 7 and 10, Ev w116 paragraph 6, Ev w40,
paragraph 32, Ev w36, paragraph 20, Back
Ev w37 paragraph 1 Back
Standards reviews: a summary, Ofqual, 1 May 2012 with links
to reports on individual reviews see: http://www.ofqual.gov.uk/news-and-announcements/130/888 Back
Ev 153 Back
http://www.jcq.org.uk/about us/index.cfm Back
See Q624 Back