Memorandum submitted by OCR
1. INTRODUCTION
1. Assessment is a complex, long established,
yet ever-developing science which is used throughout education
and working life in a myriad of ways. In giving evidence, we have
sought to confine ourselves to the context of the 14-19 qualifications
system. In doing so we have sought to avoid highly technical issues,
and have concentrated on systemic issues, seeking to point up
some of the many strengths in the system as well as highlighting
the tensions and weaknesses which need to be addressed.
2. In setting the context, it is important
to establish the purpose of assessment. It can be used to provide
ongoing feedback to the learner on their performance; it can be
used to monitor progress and for diagnostic purposes, highlighting
learners' strengths and where further work is needed.
3. A key purpose of formal assessment is
to be able to issue qualifications. Qualifications provide structure
to programmes of learning, ensuring coverage of the curriculum,
and can influence the style and approach to delivery. Above all
a qualification serves to confirm and celebrate the achievement
of an individual. Qualifications must have value and recognition
in society which requires that they are robustly assessed to ensure
a comparable national standard across location, institutions and
time, despite a constantly changing and evolving curriculum.
Important measures of success are that there
is general trust in the system and that qualifications are valued
by HE institutions and employers for recruitment and selection
purposes. This requires a powerful but delicately balanced collaboration
between teachers, employers, higher education and assessment experts
whenever new qualifications are developed. The extent to which
this delicate balance is achieved can be assisted or inhibited
by the involvement of governments and their agencies.
The UK 14-19 Qualification System—A Success
Story
4. It is important at the outset to emphasise
the many strengths of the existing system. The system is well
established and widely, if not perfectly, understood. In our experience
it takes at least 10 years for a new qualification to become accepted
and take root, so "heritage" is an essential feature
of a qualifications system. It is no coincidence that GCSEs, GCEs
and their equivalent forbears are the most widely recognised qualifications
in the UK—they have been around for generations of learners.
Nor is their success limited to the UK; in an increasingly knowledge-based
global economy, it is no trivial mater that the international
GCSEs and GCEs are offered in excess of 150 other countries.[1]
5. But the system is not stuck in the past.
Over the years it has proven remarkably adaptable to social and
economic change. At A level, it has moved from a 1950s model of
providing a service to a tiny minority of aspiring university
entrants, to a mainstream, backbone of a comprehensive education
system where a university place is a realistic aspiration for
all. It has adapted to accommodate the increase in school leaving
age from 14 to 16, it has reflected every change in teaching styles
and pedagogy, and most remarkable of all, it has accommodated
massive changes to the curriculum.
6. The system remains highly adaptable,
constantly introducing incremental change, such as the revisions
to new A Levels, changes to GCSE coursework, the introduction
of cutting edge technologies. At the same time, exam boards have
quietly introduced new choices and flexibility within the curriculum,
leading to highly successful new qualifications such as OCR Nationals.
7. Despite the difficulties associated with
the introduction of Curriculum 2000 and a growing culture of general
mistrust in public services, trust in the exams system remains
remarkably robust:
"The level of support for the A level qualification
remains high and unchanged since March 2003 .... Indeed among
A level students, there has been a significant decline in the
proportion who believe A levels should be abolished (falling from
14% in 2004 to just 3% now) .... There has been an increase in
confidence in the GCSE ...".[2]
8. The regulatory bodies have played a significant
role in providing a sound regulatory framework and responding
to public concerns. It is still the case that, in the UK, the
most critical question asked by employers and HE is not where
did you learn, but what did you achieve? Nor is this just a matter
of public perception. The independent review of standards commissioned
by QCA concluded:
"It is our considered judgement that QCA
has done a commendable job in its effort to assure quality of
the A level examinations, especially as QCA is a developing organisation.
In addition, it must contend with a raft of notable changes: in
curriculum, examination practices, consolidation of awarding bodies,
policies seeking to expand upper secondary and university enrolment,
and increased school accountability, among others".[3]
9. The system is supported by a range of
independent exam boards, each with their distinctive strengths
and heritage, competing to deliver efficiencies and to modernise
infrastructures. The accuracy and precision of the examination
system is one that would be the envy of almost any other industry:
|
| | Performance in 2006
|
| QCA expectation
| AQA | Edexcel
| OCR |
|
| % of question papers dispatched to centres on time
| 100 | 100 (100)
| 100 (100) | 100 (100)
|
| % of question papers without errors | 100
| 99.1 (99.0) | 98.7 (99.1*)
| 98.6 (99.2) |
| % of examination results issued to centres on time
| 100 | 100 (100)
| 100 (100) | 99.9 (100)
|
| % of priority enquiries about examination results completed within 20 days
| 100 | 100 (100)
| 100 (100) | 100 (100)
|
| % of examination papers copied and sent out at least 10 days before the deadline for enquiries about results
| 100 | 100 (100)
| 100 (99.8) | 99.7 (100)
|
|
Equivalent figures for June 2005 are provided in brackets. Percentages shown to nearest 0.1%.
* The 2005 figure for Edexcel has been revised from 98.0% to 99.1% to enable like for like comparisons between awarding bodies and years.[4]
|
10. The examination system has seen unprecedented investment
in technological advances estimated at around £150 million
over the last 10 years with relatively little direct support from
the taxpayer.
11. The exam boards are now well-placed to take on the
challenges and opportunities that new technologies will provide.
Many innovations are already in place, such as on-demand electronic
tests; adaptive tests, e-portfolios, and industrial scale electronic
script management. It is important that a measured and long term
view is taken with technology which isn't just about relatively
simple on-screen objective tests or replicating paper-based conventions
in an electronic format. Eye-catching initiatives, such as the
Adult Basic Skills tests, require greater scrutiny and analysis.
The real opportunities for assessment lie in the creation of new
virtual environments and interactive processes. A glimpse of what
is possible is the virtual geography field trip created by Cambridge
International Assessment, which makes it possible for students
to interact with a simulated tropical rain forest, enabling their
skills to be tracked as well as their answers logged—a significant
enhancement of pen and paper work.
A Far from Perfect System
12. Despite its many clear benefits, there is growing
unease from many stakeholders that there is something not quite
right with the system. The sources of these tensions are not straight
forward but we believe the system is most dysfunctional when policy
makers and regulators begin to intervene too closely in assessment
design, set prescriptive and unnecessary requirements, or actively
participate in the development and implementation of qualifications.
This dissatisfaction tends to cluster around the following
issues:
"The system is too burdensome"
13. The system is sometimes caricatured as being heavily
bureaucratic and expensive and it cannot be denied that the administration
of valid and reliable examinations comes with an associated amount
of bureaucracy and cost. The submitting of accurate candidate
data, the administering and timetabling of exams, the collection
and distribution of results will never be entirely free from bureaucracy.
14. There have been huge advances in recent years such
as:
— on-line entries and results services;
— the expansion of Exam board customer support services;
— the NAA's programme to "professionalis"
exams officers;
— direct capital spending on exams offices;
— completion of the JCQ initiative "Eight Pledges
to reduce Bureaucracy"; and
— the centralisation and consolidation of exams
offices within large institutions.
15. Yet there is still a feeling that more can be done
to take pressure out of the system and the Joint Council for Qualifications
(JCQ) and the Federation of Awarding Bodies (FAB) remain committed
to working together wherever possible to ensure that the systems
and processes we require customers to use are aligned.
16. To some extent, the level of bureaucracy is determined
by the assessment model used for any particular qualification,
along with its structure and design. It should be noted that,
as a rule, locally assessed, centrally moderated models (such
as coursework) tend to be more bureaucratic than examinations.
Yet often Government requirements or regulatory mandates have
been at one remove from the impact on organisations delivering
qualifications. It was the regulator, supporting government policy,
that determined in its 2002 GCSE criteria that each GCSE must
contain coursework. This was driven by a perfectly valid view
that coursework supported the important practice of "learning
by doing" or "applied learning", but the mandatory
requirement across all subjects was disproportionate to the gain,
made the volume of coursework unmanageable, and brought a perfectly
valid form of assessment into disrepute.
17. Too often policy has driven solutions which take
no account of their cost, manageability and impact. Key Skills,
the first qualification to be designed and run by the regulator,
became the only single unit qualification that had both an external
test and a locally assessed portfolio, both designed to assess
exactly the same thing. This belt and braces approach derived
from a ministerial view that external tests were "rigorous",
but with an acceptance that local assessment was the best way
of confirming that learners could actually apply their Key Skills
to real tasks.
18. The blueprint for "specialised" Diplomas,
as laid down in the 14-19 Education and Skills White Paper, appears
not to have taken into account cost and complexity or to have
acknowledged at all the bureaucracy that would be needed to underpin
it. A delegate at a recent OCR conference described the Diplomas
as "every exam officer's worst nightmare". The requirement
to provide an overall grade for the Diplomas has been additional
decision which does not seem to take into account the additional
burden on schools and colleges.
19. Once initiatives have been put in place, the regulator
and the system in general is capable of learning from, and correcting,
mistakes. The reduction of A levels from six units to four and
revisions to GCSE criteria are examples of this. But QCA has been
less successful in trying to reduce the burden at the implementation
end. The English regulator's project on common centre approval
(widely believed to have cost £1 million pounds) seems unlikely,
after more than two years, to deliver any tangible benefits. It
is just one of a series of misguided attempts to find quick fixes
and magic bullets where there are none.
20. Whilst much has been made of the costs of assessment
in our system, and there are undoubtedly further efficiency savings
to be made, more attention needs to be paid to evaluating the
alternatives. A substantial case could be made to show that qualifications
developed for large national uptake, in a competitive market,
will always generate greater efficiencies than a more local, devolved
model (ie teacher assessment). Although we will highlight the
risks of over-dependence on exams as an assessment model, they
are undoubtedly the most cost efficient way of assessing large
numbers of people. Discussions with the Learning and Skills Council
have led us to believe that the spend on exam board fees is a
miniscule proportion of their overall spend. When looking at cost,
we would recommend that some consideration is given to the costs
of alternative systems operated in other countries.
"There is too much assessment"
21. This is intrinsically linked to the theme of burden
and bureaucracy but refers to the view that learners themselves
are subject to too much assessment. Again, much of this is a consequence
of policy decisions or too inflexible, one-size-fits all, regulatory
requirements. As already stated, many of the forthcoming reforms
will address this issue. Whilst we are likely to see increased
modularisation in the new GCSEs, which will lead to more ongoing
assessment, this will be balanced against a reduction in coursework.
22. Moreover, we are developing a view that the spreading
out of assessment over a longer period of time is less stressful
than a concentrated period of assessment at the end of a two year
period. In addition, our conversations with teachers lead us to
believe that it is the sustained, unnecessary and inappropriate
mass testing of very young people through the key stage national
tests that is the single biggest cause of the view that there
is too much assessment. (This is discussed at greater length in
Cambridge Assessment's submission to the Committee.)
23. The amount of qualifications taken and achieved,
and therefore the amount of assessment undertaken, has risen significantly
in recent years.[5] A greater
number of more qualified people must largely be a good thing,
although there are some behaviours, such as the amount of re-sits
taken, which need further analysis also there is a tendency for
some candidates to take increasing numbers of GCE and GCSEs, when
higher level qualifications, requiring greater stretch and challenge,
or different more skills- based qualifications, might be more
appropriate.
The Range of Qualifications is too Narrow, Stifling Innovation
and Choice
24. This phenomenon of learners being entered for increasingly
large numbers of GCSE/GCEs may be a symptom of the view that the
current curriculum is too narrow.
25. In truth, the curriculum has been getting broader
and richer exponentially over the last 10 years. With pathfinder
projects to `flex' the curriculum and the introduction of increasing
numbers of alternative qualifications on section 96 (the list
of qualifications approved for use in schools), we have seen a
transformation of what many schools and colleges now offer. The
`Entitlement Framework' in Northern Ireland and the Welsh Baccalaureate
are commendable examples of proportionate initiatives to broaden
the curriculum offer.
26. However, many traditional institutions have yet to
embrace these new opportunities; the dominance of General Qualifications
in terms of recognition and the value placed on them, combined
with poor levels of advice and guidance, means that some learners
are still condemned to an inappropriate diet solely of General
Qualifications. This may be why some employers and HE institutions
still complain that current qualifications do not provide rounded,
multi skilled, motivated young people.
27. Government policy on assessment has tended to reflect
a nervousness of any form of assessment other than a formal examination.
A belief that this is the only `rigorous' way of assessing achievement
has led to many alternative qualifications which assess the practical
application of skills with suspicion. This was one of the key
reasons for endless tinkering with the GNVQ and its final withdrawal
and why we ended up with Vocational GCSEs were developed in such
a way as to look the same as any other GCSE. Some have accused
the current Diploma developments of showing signs of `academic
drift'.
28. Tomlinson recommended the withdrawal of A Levels
and GCSEs as the most certain way of resolving the domination
of these qualifications over the rest. OCR's view has always been
that this solution is too drastic, and that it is possible to
work towards a fuller curriculum by gradual and careful enhancement
of the provision. Single, one hit initiatives, such as GNVQ, or
possibly even Diplomas, are not easy or guaranteed solutions.
There are other ways of implementing policy, as Cambridge International
Examination's submission highlights.
29. Nor should we assume that general qualifications
need to be wholly knowledge based, purely academic qualifications.
Once again, the system continues to adapt and improve in response
to change and to learn from its mistakes. The last round of GCSE
developments took place against fairly prescriptive regulatory
criteria which, narrowed the opportunity to develop stimulating
and imaginative qualifications that would engage learners and
allow teachers to bring a wide range of teaching styles to the
classroom. We believe the new criteria will allow us to develop
far more engaging new GCSEs and that their introduction in 2009
will be something of a watershed. A precursor has been the new
GCSE in Science suite `Twenty First Century Science', introduced
in September 2006, ahead of the other new GCSEs. This highly practical
suite, rooted in contemporary, relevant contexts has proven a
runaway success with teachers and learners alike at a time when
interest in Science appeared to be in terminal decline.
Using assessment to measure the wrong thing
30. The purpose of qualifications has already been set
out. Problems arise when they are used as a proxy measure for
completely different things. The use of qualifications in school
performance tables, national targets, OECD comparisons etc leads
to misinformation and drives undesirable behaviours. The use of
performance tables in particular, leads to cynicism and a devaluing
of qualifications and learning. Assessment is designed around
that which it is intended to assess—it is no doubt wholly
possible to devise mechanisms (many of which already exist) to
assess the performance of schools and colleges, but qualifications
are the wrong instrument. Cambridge Assessment's submission deals
with this area in more detail.
2. SECTION 2: QUESTIONS
Is the testing and assessment in "summative" tests
(for example, GCSE, AS, A2) fit for purpose?
31. By and large the assessments used in GCSE, AS and
A2 are wholly fit for purpose and are adapted to suit the different
nature of what is being assessed, albeit within a framework of
regulatory criteria. Where wider skills and different learning
styles come into play, other assessment models and qualifications
are available outside of those offered by general qualifications.
Are the changes to GCSE coursework due to come into effect
in 2009 reasonable? What alternative forms of assessment might
be used?
32. It is a shame that coursework has been dropped across
many subjects. All assessment requires trade-offs between validity
(its ability to reflect the genuine level of achievement of a
learner) and its reliability (its ability to produce the same
outcome for learners who reach the same level of performance).
Well designed coursework can sometimes deliver validity which
sit-down examinations do not because of the inherent artificiality
of the examination environment. Much of the criticism of coursework
relates to scope for plagiarism; this is only a risk under the
increasingly prescriptive regulation applied to coursework which
means that all candidates undertake similar tasks, rather than
the highly personalised work which characterised the original
introduction of coursework. It is also disappointing that coursework
has been scaled back at a time when technology, in the form of
e-portfolios which OCR has already trialled in several subjects,
offers the opportunity for learners' work to be monitored and
audited in ways which were not previously possible.
What are the benefits of exams and coursework? How should they
work together? What should the balance between them be?
33. In reality, there are a great many assessment models—examinations
and tests come in many forms and can involve controlled assignments,
practical activities, orals etc. Coursework might include the
creation of artefacts, musical compositions, major project management
activities and research projects. Often assessment takes place
whilst a learner is performing a task, say in a dramatic performance,
conducting a live experiment during a field trip, dealing with
customers during work experience and so forth. It is necessary
to design assessment around what is to be assessed.
Will the ways in which the new 14-19 diplomas are to be assessed
impact on other qualifications, such as GCSE?
34. So far it has been the other way round. In seeking
parity with GCSE and GCE, the main parts of the Diplomas have
increasingly adopted models which mirror the models for GCSE/GCE
laid out in the regulatory codes of practice. The grading structures
have also been adopted to mirror GCSE/GCE scales. The diplomas
already have a minimum of four different assessment models within
them. The increased modularity of GCSEs from 2009 should encourage
greater take up of components of GCSEs within Diplomas. With careful
management and a vision that goes beyond 2013, it should be possible
to see components of GCSEs, Diplomas and other qualifications
being combined to create new choices which are not as narrow as
the prescribed Diploma routes, and not as unvaried as a diet entirely
of General qualifications.
Is holding formal summative tests at ages 16, 17 and 18 imposing
too great a burden on students? If so, what changes should be
made?
35. Evidence suggests that the majority of learners prefer
assessment to be spread out over time and closer to the learning
experience. OCR has supported QCA in developing guidance to teachers
on synopticity at GCE to ensure that candidates are able to demonstrate
a full grasp of a subject across and between topics.
How does the national assessment system interact with university
entrance? What does it mean for a national system of testing and
assessment that universities are setting entrance tests as individual
institutions?
36. It has never been claimed that A levels should be
the only tool used to determine university entrance (or recruitment
to employment for that matter). The Cambridge Assessment response
provides information on how university tests provide additional
information about potential undergraduates, and where they seem
to merely replicate existing 14-19 assessments (as with SAT 1)
GENERAL QUESTIONS
Who is the QCA accountable to and is this accountability effective?
37. We understand that QCA is accountable to the Secretary
of State, who is accountable to Parliament. What is less clear
is precisely what QCA is accountable for. This is a consequence
its somewhat open-ended statutory remit as defined in the 1998
Education Act, a tendency for ministers to add to QCA's remit
over time through ad hoc remits and funding, and a tendency over
time for QCA to shed its independence from Government and to act
as the Government's implementation arm for qualifications policy.
What role should exam boards have in testing and assessment?
38. We believe our evidence sets out clearly the key
role of examination boards in developing and delivering qualifications
for 14-19 year olds. We would like to emphasise again the delicate
balance of stakeholder interests that must be obtained to ensure
a valued qualification system, and the unique position of examination
board, which possess deep understanding of assessment and qualifications,
and sit independently of government and between the many stakeholders
enables them to play the role of consensus builder, provided that
regulation does not prevent them from doing so.
June 2007
1
Source: Cambridge International Assessment Submission to the Committee. Back
2
GCSEs and A levels: the experiences of teachers, students, parents
and the general public, QCA February 2006 (results of an independent
survey commissioned with MORI). Back
3
Maintaining GCE A Level Standards, QCA October 2001. Back
4
Report on the performance of awarding bodies for general qualifications
in 2006, QCA, 2007. Back
5
"The success rate for 16-18 year-olds taking full Level 2
qualifications has improved by 9 percentage points over a two
year period to 67% in 2005-06, with the success rate for adults
following the same programmes increasing by 11 percentage points
to 66% over the same 2 year period".
Further Education and work-based learning for young people-learner
outcomes in England 2005-06, Learning and Skills Council,
April 2007. Back
| 
