Education CommitteeWritten evidence submitted by Pearson
As always, we worked closely with the regulator to determine the grade boundaries for this year’s GCSE English and all other subjects. The decision making process and its outcome are in line with Ofqual’s Code of Practice, and the imperative to maintain standards year on year. However, it is important that schools and students are satisfied that the right processes have been undertaken, and that we capture the lessons which may be learned. We are hopeful that the Committee’s session will support this.
To that end, the paper below sets out the approach Pearson and our fellow awarding organisations have taken to date. We hope this will be helpful to the Committee in providing some more context to this year’s award, explaining the processes undertaken, and making recommendations for the future.
Standard Setting
Awarding organisations, as you know, have a responsibility to maintain standards year on year, and this is overseen by Ofqual. It is important that standards are comparable over time in order for a qualification, and awards at different grades, to retain any intrinsic value for the learners who achieve them and therefore enable them to progress. This is the guiding principle in standard setting.
During the awarding process, senior examiners consider a wide range of materials and reach decisions on the number of marks that candidates need to get certain grades. These materials include copies of candidate work from the current and previous years, as well as relevant statistical data. Decisions reached during awarding must ensure that overall outcomes reflect consistent standards for candidates past and present.
Awarding the new English GCSE in 2012
2012 marked the first award of a new English specification. At a time of new specifications, setting grade boundaries is especially challenging, and the familiarity of teachers with specifications can also have an impact on results.
2012 is also the second year in which an additional statistical measure has been used to predict the outcomes of GCSE cohorts. The “prediction matrix” gives examiners more data about the prior performance of cohorts. In GCSE, the reference point used is performance at Key Stage 2. This means that, at awarding, we are better able to see if a group of students’ performance at qualification level is coming out of line with what could reasonably be expected given their overall ability (and therefore if the demands being placed on them to gain particular grades may not be comparable to those placed on peers in previous years). Reasonable expectations as to outcomes for a given year are set by Ofqual prior to examinations, in the form of “tolerances” for awarding. Awarding organisations are required to ensure awarding outcomes are aligned to these predictions.
For the new GCSE English, the summer 2012 examination series was the first opportunity where all awarding bodies could consider how the grade boundaries set for individual units contributed to the overall qualification grade. As we began to make summer awards, it became clear that all awarding organisations were reporting results which suggested that the demands placed on students to reach particular grades for some units was insufficient to maintain the standard at qualification level.In other words, awards were moving out of “tolerance”.
Changes to Grade Boundaries
Grade boundaries set with the regulator for the January cohort—a much smaller number of students than completed in the summer (in the case of Edexcel, just 750 students versus 24,000)—did not therefore give confidence that the same performance was being required of students for the achievement of particular grades as in previous years.
We agreed with the regulator that the shifts in outcomes we were seeing on January grade boundaries could not be accepted if the principle of ensuring the intrinsic value of performance at particular grades was to be upheld. In order to maintain standards it has been necessary for all awarding organisations to move grade boundaries.
In Edexcel’s case, in three instances out of nine, the grade boundaries have increased compared to the January 2012 examination series, by 2, 4, and 10 marks. This does not mean that the overall qualification standard has changed, nor that it is any easier or any harder to get a grade in GCSE English.
These changes were discussed with the regulator at some length, because it is also important, of course, that students are rewarded for the genuine performance they show on the day and that examiner judgement on quality of work, rather than a statistical model is the key factor in grades awarded.
Ofqual, Pearson and other awarding organisations therefore worked together to determine what grade boundary shift was required to report results which represented student achievements fairly and delivered comparability year on year. In Edexcel’s case, for example, the boundary shift decided upon reported results slightly above the tolerances determined by the prediction matrix.
We are happy to share our exchange of letters with the regulator on the changes, in which different grade boundary movements were discussed before reaching a decision as to the right award. This is usual practice where an award outside tolerance is being made.
Teacher Reaction to Changes
In considering the reaction of past weeks, it is critical to reinforce the point that the discussion of grade boundaries with the regulator between series, and their revision, is not unusual. Schools and teachers are aware that grade boundaries are always provisional and may move from series to series. We do not advise teachers to use previous grade boundaries as a guide for students as to the grades they may achieve. Finally, it should also be noted that Edexcel’s grade boundary decisions on this qualification are not severe in terms of the outcomes expected by the prediction matrixes for the cohort.
What has contributed to the strength of feeling shown in recent days is the extent of the changes made, and that some grade boundary changes have been to “controlled assessment” modules, marked by teachers who may have used previous grade boundaries as a guide and communicated expected grades to students on this basis.
In the case of the Edexcel English GCSE, both can be clearly explained:
First, due to the structure of the qualification, changes to boundaries in controlled assessment units were unavoidable if standards were to be maintained at qualification level. The size of the shift, too, is partly attributable to the structure of the qualification, which has only a limited number of grade boundaries on which a change can be made (though in other awarding organisations, changes of around 10% have had to be made even where the qualification structure is different).
A second factor in the size of the shift, as noted above, is the very small size of the cohort upon which the original grade boundary for this new qualification was determined. The shift to the terminal rule and the fact that there are now three English GCSE options on offer to students (English, English Language and English Literature) also meant that the composition of the cohort sitting in both January and June had differences compared to previous years, complicating awarding.
The Resit Opportunity
Whatever the explanation, we understand that students not receiving the results anticipated can be unsettling, for them and for their teachers. However, we take very seriously our responsibility to ensure that students are rewarded correctly for their performance, and treated comparably to their peers in previous years.
It is in seeking to reconcile the above that Pearson made the suggestion to Ofqual that a further resit opportunity be provided. The perceived unfairness which stems from the difference in the boundaries between January and June cannot be addressed directly at this point without compromising standards or undermining the broad approach we take to awarding. However, the impression of unfairness also comes from the belief that young people may have been working to a boundary which then moved significantly—exacerbated by the concentration of changes on controlled assessment units. For this reason we have suggested that students have another opportunity to sit the same task, with June grade boundaries maintained, free of charge.
By providing a resit opportunity in the Autumn term, we hope to alleviate the sense of unfairness and to limit the impact on affected students who may be “at risk” of disengagement from education, which has been a key concern cited in communications from schools. The decision to provide resits demonstrates awarding organisations’ collective willingness to enable more students to reach the required standard but, crucially, not to compromise it.
Lessons Learned
There are a number of lessons that can be learned from the issues that have been experienced this summer.
1. Clarity on the status of January grade boundaries
Schools are aware and used to grade boundaries changing between series.
However, new specifications always have an impact on performance as they settle and, given the extent of the changes with the new English specification, and the small cohort in January, we would have expected it to be particularly so in this case.
Communication with schools could with hindsight have been more explicit about the dangers of presuming January grade boundaries stood for the summer. Pearson will ensure this is explicitly addressed in all future series.
2. Accountability
There is a wider point to be made around the issue of the accountability and the extent to which it drives a focus on grade boundaries which is unhelpful. Giving transparency on assessment to teachers and holding them to account for performance, whilst avoiding too much focus on the test is an issue worthy of consideration.
There is a need to balance a right focus on standards and floor targets with the reward of broader measures of improvement and all stakeholders need to work together to develop a system which better facilitates this.
3. Impact of change
The issues we have seen this summer are related to a change in the structure and content of qualifications in a single subject. This is indicative of the complexity of making changes to the examinations system, and the associated risks to standards and potential implications for public confidence.
Proper time and consideration needs to be given to the long term impact of changes at the point they are suggested, in order that they can be resolved. For example, some possible issues around maintaining standards in modular and composite qualifications, relevant to this summer’s events, were explored in 2009 and with hindsight would have merited further debate.
At a time when system wide reforms are being considered, both to the way the exams system is run and to the qualifications which sit within it, we should be cognisant of the likelihood that issues such as this may well arise in any transition to new qualifications and systems. This will be exacerbated if both things happen at the same time and if there is insufficient engagement between all stakeholders on associated risks.
I hope the above reassures the Committee that the right processes have been followed with regard to this year’s results. We are working hard to talk directly with schools and their representative groups to explain our decisions.
We feel acutely the disappointment of learners and teachers, but are confident that the steps we have taken are to the good of the credibility of the GCSE qualification awarded to students this year and for the long term.
September 2012