Education CommitteeWritten evidence by AQA

The GCSE English qualification was redesigned for first teaching in September 2010 and was awarded for the first time this summer. This new suite of English GCSEs includes English, English language and English literature. Students in England take either English, or English language and English literature. So while in the past students simply studied GCSE English, teachers are now offered a choice of routes to fulfil the requirement for English set out in the floor target. In a further change, this new GCSE English qualification also required students to study literature which was not the case in its predecessor.

It is the maintenance of standards from the previous GCSE English qualification to the two new GCSE English and English Language qualifications that has caused concern amongst teachers. To understand the issues, it is useful to understand the change in the structure and assessment of the specifications.

Qualification Design—Structure and Assessment

The structures of AQA’s GCSE English and GCSE English Language qualifications are summarised in Table 1 below. All students sit three units: the first is a written examination, which is taken at either the higher or foundation tier; the second is a compulsory speaking and listening assessment, carried out by teachers; and the third is a written controlled assessment, also carried out by teachers, with different versions for students taking GCSE English and those taking GCSE English Language.

There are aspects of this design that were intended by the regulator (through the qualification criteria) to create a positive teaching and learning experience and to meet the various needs of students. However, they make the setting of comparable standards complex and contribute to the volatility in results that teachers are experiencing.

Only 40% of the assessment is tiered and carried out by conventional examination, while 60% is controlled assessment that is marked by schools. Thus, the ability to set controlled assessment boundaries, exam by exam, is key. Another consequence of this aspect of the design is that it is possible for students entered into the foundation tier of the examination to achieve a grade A overall and although few students will (it would indicate that their ability had been dramatically under-recognised in school), this was not possible in the previous qualification. It is also important to note that the unit 3 written controlled assessments are the key differentiator between the qualifications and hence critical to achieving comparable standards between them. Adjusting the unit 3 controlled assessment standards is necessarily the method by which comparable qualification level standards are set. Clearly, the choices that are made in how to balance outcomes across the assessment units impact on the overall rank-ordering of the candidates.

Table 1

UNITISED STRUCTURE OF AQA’S GCSE ENGLISH AND GCSE ENGLISH LANGUAGE QUALIFICATIONS

			Availability
Unit	Mode of Assessment	Weighting	Jan 11	Jun 11	Jan 12	Jun 12
English unit 1 foundation tier	Written Examination	40%	√	√	√	√
English unit 1 higher tier	Written Examination	40%	√	√	√	√
English unit 2 untiered	Ephemeral Controlled Assessment	20%	x	√	√	√
English unit 3 untiered	Written Controlled Assessment	40%	√	√	√	√
English Language unit 3 untiered	Written Controlled Assessment	40%	√	√	√	√

By comparison, the structure of AQA’s previous (legacy) GCSE English qualification is summarised in Table 2 below.

Table 2

LINEAR STRUCTURE OF AQA’S LEGACY GCSE ENGLISH A QUALIFICATION1

			Availability
Component	Mode of Assessment	Weighting	Jun 102	Nov 10	Jun 11	Nov 11
Component 1 foundation tier	Written Examination	30%	√	√	√	√
Component 1 higher tier	Written Examination	30%	√	√	√	√
Component 2 foundation tier	Written Examination	30%	√	√	√	√
Component 2 higher tier	Written Examination	30%	√	√	√	√
Component 3	Ephemeral Coursework3	20%	√	√	√	√
Component 4	Written Coursework	20%	√	√	√	√

AQA’s previous qualification comprised 60% tiered, written examination. The November awarding series was used as an opportunity for both resitting and early entry. It is important to note that the grade boundaries for the coursework were agreed at an inter-awarding body level and did not change during the lifetime of the qualification. This was unique to GCSE English. Having 60% of the assessment via written examination was the means by which comparable standards were maintained. This was more acceptable to teachers than a change to the coursework boundaries in which the task had not changed. This, however, also had the unfortunate effect of creating, over time, an imbalance in the outcomes on the examined and coursework assessments: coursework grades increased over time and so the grades on the examinations decreased to compensate. This has consequences for the validity of the qualification.

Awarding the New Qualifications

In awarding the new qualifications, it was necessary to maintain comparable standards on several (sometimes conflicting) axes, including:

from the legacy to the new qualifications;

between awarding organisations;

between the two qualifications (English and English Language);

between the modules that comprise the qualifications;

between the tiers of the examination papers; and

across the multiple assessment opportunities (January and June).

The approach taken to these issues is briefly outlined below.

In order to maintain comparable standards between the previous (legacy) and new qualifications, and between the awarding bodies, statistical predictions based on the prior attainment (KS2 test results) of the students entered for the new qualifications were used to support the judgements of the awarding committee in the subject awards made this summer. This method was successfully applied in the maintenance of standards of those new GCSEs awarded for the first time in summer 2011.

The predictions took into account how the general ability of the students entered for the new qualifications compared with that of the students entered for the previous qualification. Crucially the predictions were used by all awarding bodies and were based on the national outcomes in the previous GCSE English qualification, rather than each awarding body using outcomes in its own exam only, and so ensured that the different awarding bodies’ standards were aligned. It is important to note that although the predictions were an important source of evidence, the awarding committee were able to recommend boundaries that fulfilled their professional expectation of an appropriate standard of student work.

The predictions were also very useful in allowing the awarding committee to understand the very different kinds of students entered for the new English and English Language qualifications. In terms of prior attainment, the students taking GCSE English Language were significantly more able than those taking GCSE English. The disparity in the results of the two groups is stark but reflects differences in ability (for example, 31.3% grade Cs in English and 74.7% grade Cs in English Language). The challenge of the new literature element of GCSE English would have been considerable for these students.

When there are significant changes to qualifications, both in terms of their content and their assessment, there is an expectation that outcomes may dip as teachers and students become familiar with the new demands. Because it is considered unfair to those students unfortunate enough to be in the first cohort taking the new qualification, the use of statistical predictions is considered a useful method of ensuring that this extraneous change in demand is compensated for. Conversely, in this instance it seems likely that the move from 40% to 60% teacher assessment might have led to an expectation that results would improve. The purpose of the awarding process, however, is to take into account such changes in demand and assessment style.

The early awards of units in January 2011, June 2011 and January 2012 were challenging. While statistical predictions can be useful in supporting awarders’ judgements at a qualification level, they provide only very weak support in making early unit awards. This is because the students entered at this early point are a relatively small and unrepresentative sample of the cohort and assumptions can not therefore be made about the likely relationship between their prior attainment and GCSE outcomes. It is also the case that the students entered early will have had varying levels of preparation and will have matured, in both the academic and developmental sense, to varying levels. Hence, these awards relied almost exclusively on the awarding committee’s judgement of candidate performance.

This is problematic because, it must be noted, standards reside at the level of the qualification. The way in which candidate performance on the individual assessment units aggregates to give an overall grade is crucial. For example, if a group of students were to take a qualification made up of six assessments, the grade boundaries would need to be lower at the top end, and higher at the bottom end, than if they were to take a qualification made up of fewer units; these are known as regression effects. Even when it is possible to estimate statistically these aggregation effects, it is impossible for awarders to factor them into their judgement of candidate performance and thus the setting of the grade boundaries. The mental agility required to compare student performance, which is often uneven, to the generic grade descriptions is in itself challenging enough.

In sum, while we are entirely confident that the awarding committee made the best possible judgments given the information that was available to them, the level of confidence we could have in the early unit awards was inevitably lower than that in this summer’s awards in which students certificated. The setting of the controlled assessment boundaries in June 2011 and January 2012 was, of course, particularly problematic because the publication of these grade boundaries naturally influenced the way in which teachers marked student work thereafter. Teachers having a clearer idea of the standard impacted on the distributions of marks seen in later examination series. Despite repeated communications to make the likely change of these boundaries clear to all teachers, the change has nonetheless caused immense concern and has undermined for some their trust in the qualification system. The bottom line is that, on the basis of the boundaries set for previous controlled assessments, teachers thought they knew with some degree of accuracy what grades their students were going to get.

There is concern that those students taking a modular route through the qualification have benefited from both lower controlled assessment boundaries and lower examination boundaries, particularly on the foundation tier. We are very much in the hands of the judgement of the awarders in assessing the relative standards between those students making use of the early units and those who took their assessments linearly. This cannot be done with confidence statistically as the two groups are selected samples of the overall cohort so one would not necessarily expect their outcomes to be the same (even taking into account measures of their general ability such as KS2). Moreover, one group had the opportunity to resit units and the other did not, this is particularly problematic since only the better of the attempts counts towards grading.

Some ideas for the Immediate Future

During the development of modular GCSEs, in 2009 AQA proposed to the regulator some approaches to the grading (or not) of early units. At the time these were rejected but are worth revisiting for GCSE English before the January 2013 examination series, particularly in relation to the controlled assessment units:

1.Early unit grades could be issued as provisional and only finalised in the summer when candidates certificate.

2.Early unit marks (either raw or standardised, but not uniform) could be issued instead of grades. These would allow teachers and students to assess performance without implying any reference to grade boundaries.

We accept that these options are likely to be unpopular with teachers. An alternative is not to offer the opportunity to submit controlled assessment in the January series, though we recognise that that too would be unpopular with some schools that have planned to submit at that point in the year.

Teachers would clearly need to be consulted on the impact of these options on teaching and learning before any decision is made. We would be happy to discuss these possibilities further.

September 2012