Education CommitteeWritten evidence submitted by Ofqual

Introduction

1. Ofqual aims to secure the standards, fitness for purpose and quality of regulated qualifications and to regulate in a way that promotes a robust, secure and efficient qualifications system. We welcome the opportunity to participate in the Committee’s inquiry and contribute to further work in this area, given our responsibilities and expertise.

2. The structure of the market for qualifications has evolved over many years. There are five main providers of GCSEs and A levels, which are the most high-profile and widely taken qualifications by this age group. Three awarding organisations (AQA, Edexcel and OCR) account for over 85% of awards. Alongside these, there is a wide range of other qualifications, most of which are proprietary and offered by only one awarding organisation. Annex A sets out the landscape of qualifications for 15–19 year olds in more detail. There are strong interdependencies with the market for adult qualifications, which are out of scope of this inquiry but which Ofqual also regulates.

3. This submission sets out our views on the outcomes that an effective qualifications system should deliver. It is against these outcomes that alternative arrangements would need to be evaluated. The submission sets out the benefits and risks presented by the current model, and how Ofqual’s regulatory approach seeks to secure the benefits and manage the risks. In putting in place a new, risk-based approach to regulation we have set out to understand and evaluate the health of qualifications markets. We set this out in more detail below.

4. There are many alternative models for delivery of qualifications. No delivery model is risk-free and there are many factors that could influence the pros and cons of each. These need to be considered carefully, so that if changes were made, they were not based on the assumption that we would be able to create a perfect system.

5. An effective qualifications system must:

Secure qualifications standards;

Secure delivery standards;

Incentivise value for money;

Be transparent; and

Support innovation.

Secure Qualification Standards

6. Ofqual recently hosted a standards event, looking at some of those aspects of standards said to be of concern. We are building and exploring the evidence base. We know that the commercial pressures inherent in any market (particularly if combined with the wrong incentives on purchasers) risks downward pressure on standards. The perception is that by its very nature, competition risks the development of more “accessible” qualification specifications (syllabuses) creating incentives on centres to switch to these less demanding qualifications. Whether schools are regularly switching between different specifications in particular subjects and the rationale for switching needs further analysis—but it is important to note that the overall awarding organisation share of GCSE and A level qualifications has remained relatively stable over a number of years (as set out in figures 2 and 3 in Annex A).

7. However, there are broader influences, including the impact of the use of qualifications as accountability measures for schools, which may create incentives to lower standards, and various systemic pressures which risk small annual changes in standards that add up to more substantial reductions over time. Our role as regulator is to understand these risks and pressures, and design our regulatory arrangements to mitigate them. The current Education Bill proposes a revision to our standards objective, so that we have to seek to secure comparability of standards with qualifications awarded outside the United Kingdom. Assuming that the Bill is passed, we will consider whether we need to recalibrate qualification standards in the light of our international studies, and also whether we can use international comparisons to help manage the pressures and risks to standards of regulated qualifications.

8. We require awarding organisations to use methodologies that provide assessments that minimise bias and are valid, reliable, comparable and manageable. We use a risk-based approach, and our regulatory arrangements include qualification and subject criteria which set out the minimum requirements for particular high-stakes qualifications, notably GCSEs and A-levels. Awarding organisations’ specifications must meet those minimum requirements in order to be accepted onto the register as regulated qualifications, and are then subject to Ofqual’s monitoring.

9. Ofqual now monitors the interim outcomes from summer awards to secure comparability between awarding organisations and minimise any “grade inflation”. Since we have been doing this, the rate of increase in students achieving the top grades has slowed to such an extent that in summer 2011 there was no increase in those achieving grade A or above at A-level. Ofqual is committed to containing grade inflation whilst making sure that awards reflect accurately students’ achievements.

Secure Delivery Standards

10. It is fundamentally important that the system delivers high quality assessment including (so far as is humanly possible) error-free papers, accurate marking and fair awarding. Our powers, based on conditions of recognition, allow us to intervene if we find weaknesses in awarding organisations’ approaches. There are also mechanisms in place to ensure inaccuracies can be dealt with, for example the Enquiries About Results (EAR) appeals mechanism. For the summer 2010 GCSE and GCE series there were 171,700 enquiries about results,1 resulting in a total of 34,800 grade changes. The general trend has been for an increase in both the number of appeals, up from 119,300, in 2006 and the number of grades revised, up from 25,900 in 2006. It is difficult to attribute this increase to a single factor because of the changes to the system (such as the change to unitised qualifications and the introduction of the A* for GCEs), and since 2009 the changes include cases where grades have decreased as well as increased, meaning that direct comparisons are not available. The Examination Appeals Board (EAB), which is independent from the awarding organisations and the regulator, handle appeals where the candidate is not satisfied by the EAR process. In 2010 the EAB received 16 appeals (an increase from 11 in 2011 and six in 2009). Nine applications to appeal were declined in 2011 (five in 2010).

11. The qualifications delivery system is complex. Each summer examinations period requires the awarding organisations that offer GCSE and A level qualifications to set over 60,000 examination questions, mark over 25 million separate examination scripts and items of coursework, and issue over 8 million GCE (AS and A Level) and GCSE results. In dealing with the consequences of the unacceptable number of significant errors in live examination papers this summer, we promoted a principles-based approach to ensure that, as far as possible, no candidate was unfairly disadvantaged or advantaged. We have also launched an inquiry to provide recommendations directed at driving improvements for the future. Our interim report can be found http://www.ofqual.gov.uk/files/2011-10-31-exam-errors-inquiry-summer-2011-interim-report.pdf. We will produce our final report in December.

12. A major strength of the current system is that for high-stakes qualifications, multiple providers can serve to reduce delivery risk since failure in one provider does not affect a whole cohort of students. The failure of National Curriculum test delivery in 2008 and the events in France this summer demonstrate some of the risks of relying on a single provider. In France, 165,000 candidates were affected when one question from the French baccalaureate mathematics paper appeared on the internet the day before the test. Many of the live exam errors affected very small number of candidates. The largest, by far, in one paper affected nearly 42,000 candidates.

Incentivise Value for Money

13. Competition can incentivise awarding organisations to implement more cost efficient processes for the development and delivery of qualifications and lower prices. The qualifications market continues to grow in terms of qualifications, awards and revenue. In a healthy market there will be incentives on organisations to meet the needs of purchasers, learners and the end-users of qualifications in the most efficient way possible. Competing providers can lead to greater choice and incentivise the tailoring of qualification products and services to meet the diverse range of needs of millions of students. The current system enables awarding organisations to respond to market need in particular in vocational and industry-led qualifications.

14. Expenditure on examinations and assessments in state secondary schools in England has nearly doubled from £154 million to £303 million over the period 2002–03 to 2009–10. A high quality, internationally competitive, system of qualifications provision has an associated cost. There is a high level of assessment in the current system and changes in the structure and the mix and type of qualifications being taken in schools and colleges has compounded the pressures on costs. Ofqual’s international comparisons work evidences England’s unusual reliance on examination as opposed to other forms of assessment. Ofqual’s market report shows that whilst unit fees have fallen for A levels they have risen slightly above inflation for GCSEs.

15. The current data available do not allow us to make a robust overall judgement on the efficiency of awarding organisations’ costs or cost structures, including their ability to exploit economies of scale and scope. But we need to be able to make this judgement: we have an efficiency objective to secure that regulated qualifications offer value for money.

16. Ofqual’s recognition conditions already require awarding organisations to make fee information available and to be transparent about how their products and services are bundled. They are also required to notify Ofqual if they anticipate cost pressure that could result in fee increases above the level of inflation.

17. As a new regulator our early focus has been on delivery and qualification standards. But as evident in our healthy markets work our economic regulation and market work is taking a higher profile. We will be undertaking work to better understand what is driving costs in the sector, the scope for efficiency and on pricing principles and structures. Where the market fails to deliver qualifications that offer value for money, we will intervene, where necessary and appropriate, using our fee-capping power.

Transparency

18. Effective and appropriate choices either by those actually taking a qualification or on their behalf require a system of provision that is transparent and navigable. A system where purchasers of qualifications know what they are buying and are able to buy only those services they want to buy is a sensible goal. The total number of regulated qualifications has risen from 7,400 in 2006 to 15,400 in 2010. Of the 15,400 currently available, some 8,000 are in regular use, and therefore there are several thousand on our register that are not in common use.

19. Competition in the market may incentivise awarding organisations to diversify the range of qualification types provided. Whilst this may have benefits, as the volume of qualifications rises, there is a risk that the system may become more complex and difficult to navigate. There is also an associated risk that some qualifications may lose their value as a signal of attainment. Regulation, notably the introduction of the Qualifications and Credit Framework (QCF), has also impacted on qualification numbers. There are other questions about the QCF including whether the arrangements are too cumbersome, too prescriptive, or if they unduly constrain choice, for example choice of provider. We intend to find the best way to enable a unitised and credit-based approach to qualifications to work well in the interests of users.

20. For vocational qualifications awarding organisations have welcomed early proposals by Ofqual to increase transparency by bringing more relevant information into the public domain to aid choice.

Supporting Innovation

21. An effective qualification system must enable innovative approaches to be introduced safely to examinations and assessments, reflecting the needs of the future. Competition can encourage greater levels of investment and innovation over time, for example in marking technology and systems. Our innovation stock-take aims to gather information on the drivers and barriers of innovation in the qualifications sector and understand better the investment incentives faced by the sector.

Independence from Government

22. A key benefit of the current structure is that awarding organisations are independent of Government and many of them have long histories; the large awarding organisations that provide qualifications for 15–19 year olds have their roots in the universities and industries that remain key users of these qualifications. This helps give their qualifications credibility with the independent sector, employers and universities. In considering potential alternative arrangements for provision of qualifications, the Committee will want to consider how such credibility could be maintained.

Healthy Markets

23. We want to develop our regulatory arrangements to enable and incentivise a healthy market for qualifications that:

Provides mechanisms for those operating in the market to interact with each other in a way that provides qualifications that meet the needs of users and is clearly understood; and where

Intervention is directed to those areas where the market by itself would not produce desired outcomes (ie) there is a likelihood of market failure).

24. Ofqual is embarking on a continuous programme of work to set out what a healthy market(s) would look like and where we may need to intervene.

25. The current operation of qualifications markets and their ability to deliver desired outcomes (“market health”) is affected by a range of factors relating to:

supply and demand side market structures—such as the concentration of awarding organisations, the extent of vertical integration in provision and the nature of purchasing decisions on the demand side;

supply side interaction between awarding organisations—such as the intensity and form of competition, collusion or collaboration;

the interaction between the supply side and demand side—for example determined by the degree of information provision and procurement practices; and

Broader systemic factors - including political (and institutional), economic, sociological, technological, legal (and regulatory) and environmental factors.

26. Ofqual plans several projects relating to healthy markets:

Conflicts of interest—to determine the issues and potential problems associated with conflicts of interest in the qualifications sector, in particular in relation to study aids such as textbooks and training services, and to develop a long term action plan to address issues where they exist.

The role of market forces—to consider the appropriate role of competition and market forces in the qualifications sector, and develop an approach to promoting competition where it is beneficial in delivering desired outcomes. This work will include consideration of, for example:

the benefits of competition in delivering innovation in the sector and containing costs;

the risks of downward pressure on standards of qualifications;

the risk of duplication of resources in qualification provision; and

the risk that market forces create too many qualifications.

Market distortions—to identify existing policy and regulatory distortions to the effective operation of market forces. This work will include consideration of, for example:

the role of some Sector Skills Councils—which are themselves evolving—in the operation of the market; and

the impact that requirements in relation to shared qualification units in the Qualifications and Credit Framework are having on incentives for awarding organisations to invest and innovate.

27. Given the range of forces that impact on the health of markets, many of these projects will require a long term collaborative approach with other bodies across the sector, including DfE, BIS, UKCES, the OFT and the funding agencies.

The Commercial Activities of Awarding Bodies and their Impact on Schools and Pupils

28. The activities undertaken by many awarding organisations (and their parent companies) extend beyond qualification provision and include, for example, provision of training courses, publication of textbooks and development of educational IT platforms. These activities are an important part of many awarding organisations’ current business models.

29. There is a perception that some such activities risk creating conflicts of interest with qualification provision. One such area is the publication of textbooks by awarding organisations (or in exclusive partnership with other publishers).

30. The textbook issue is complex and has two key dimensions:

The potential impact on competition—if awarding organisations produce textbooks to accompany their examinations, it may be more difficult for others to develop and sell high quality alternative textbooks. This problem may be exacerbated if awarding organisations “bundle” qualifications and textbooks (and other related products), selling them together as a package, even if bundling has other benefits.

The potential educational impact on standards—if awarding organisations produce textbooks, there is a risk that the material in the textbooks will be considered to be the only material candidates need consider in preparing for examinations, leading to an undesirable narrowing of teaching.

31. The latter issue does not arise just from the production of textbooks related to specific qualifications, which if produced and used appropriately can support and enhance learning. Rather it arises when textbooks provide too much information about likely questions and the structure of answers, facilitate “teaching to the test” or otherwise distort candidate learning.

32. Ofqual requires assurances through the recognition conditions from awarding organisations that connected activities do not affect the standards or quality of qualifications, unduly limit learners’ programmes of study or prevent other awarding organisations from offering similar products.

33. But given stakeholders’ concerns, in particular in relation to textbooks, Ofqual intends to investigate the evidence to determine what regulatory action, if any, we should take. Our investigation will be part of our broader healthy markets programme (see above) and will consider both the potential competition and educational impacts of textbooks.

Conclusion

34. Ofqual welcomes the Select Committee’s interest in this area. Securing the efficient delivery of rigorous qualifications is an essential part of the education system. This evidence seeks to demonstrate two things in particular to inform the Committee’s deliberations:

First, that the issues involved are complex. Whatever arrangements are put in place to provide qualifications, they will need sophisticated and intelligent oversight to secure the right outcomes, particularly the maintenance of standards. And any decision to reform the system—in whatever direction—would need to take into account the interdependencies and pressures within and beyond the market for 15–19 qualifications.

Second, that Ofqual is absolutely committed to the challenge of making the system work in the interests of users and the wider education system. As a new regulator, we acknowledge that we have much to do. In particular, we will:

complete the work to put in place the foundations of our regulatory arrangements, including to reflect the proposed change to our objectives and powers in the Education Bill;

continue to develop our analysis of the qualifications system, to inform the debate about the development of regulation and the delivery of qualifications reform; and

continue our programme of work to identify the key areas of risk where we need to focus our regulatory effort, in order to secure robust standards and a healthy qualifications market.

35. Ofqual looks forward to discussing this evidence and the wider issues further with the Committee.

February 2012