|
1. Introduction
1. Assessment is a complex, long established, yet ever-developing science which is used throughout education and working life in a myriad of ways. In giving evidence, we have sought to confine ourselves to the context of the 14-19 qualifications system. In doing so we have sought to avoid highly technical issues, and have concentrated on systemic issues, seeking to point up some of the many strengths in the system as well as highlighting the tensions and weaknesses which need to be addressed.
2. In setting the context, it is important to establish the purpose of assessment. It can be used to provide ongoing feedback to the learner on their performance; it can be used to monitor progress and for diagnostic purposes, highlighting learners' strengths and where further work is needed.
3. A key purpose of formal assessment is to be able to issue qualifications. Qualifications provide structure to programmes of learning, ensuring coverage of the curriculum, and can influence the style and approach to delivery. Above all a qualification serves to confirm and celebrate the achievement of an individual. Qualifications must have value and recognition in society which requires that they are robustly assessed to ensure a comparable national standard across location, institutions and time, despite a constantly changing and evolving curriculum. Important measures of success are that there is general trust in the system and that qualifications are valued by HE institutions and employers for recruitment and selection purposes. This requires a powerful but delicately balanced collaboration between teachers, employers, higher education and assessment experts whenever new qualifications are developed. The extent to which this delicate balance is achieved can be assisted or inhibited by the involvement of governments and their agencies.
The UK 14-19 qualification system - a success story 4. It is important at the outset to emphasise the many strengths of the existing system. The system is well established and widely, if not perfectly, understood. In our experience it takes at least ten years for a new qualification to become accepted and take root, so 'heritage' is an essential feature of a qualifications system. It is no coincidence that GCSEs, GCEs and their equivalent forbears are the most widely recognised qualifications in the UK - they have been around for generations of learners. Nor is their success limited to the UK; in an increasingly knowledge-based global economy, it is no trivial mater that the international GCSEs and GCEs are offered in excess of 150 other countries[1].
5. But the system is not stuck in the past. Over the years it has proven remarkably adaptable to social and economic change. At A level, it has moved from a 1950s model of providing a service to a tiny minority of aspiring university entrants, to a mainstream, backbone of a comprehensive education system where a university place is a realistic aspiration for all. It has adapted to accommodate the increase in school leaving age from 14 to 16, it has reflected every change in teaching styles and pedagogy, and most remarkable of all, it has accommodated massive changes to the curriculum.
6. The system remains highly adaptable, constantly introducing incremental change, such as the revisions to new A Levels, changes to GCSE coursework, the introduction of cutting edge technologies. At the same time, exam boards have quietly introduced new choices and flexibility within the curriculum, leading to highly successful new qualifications such as OCR Nationals.
7. Despite the difficulties associated with the introduction of Curriculum 2000 and a growing culture of general mistrust in public services, trust in the exams system remains remarkably robust: "The level of support for the A level qualification remains high and unchanged since March 2003... Indeed among A level students, there has been a significant decline in the proportion who believe A levels should be abolished (falling from 14% in 2004 to just three percent now)...There has been an increase in confidence in the GCSE..."[2].
8. The regulatory bodies have played a significant role in providing a sound regulatory framework and responding to public concerns. It is still the case that, in the UK, the most critical question asked by employers and HE is not where did you learn, but what did you achieve? Nor is this just a matter of public perception. The independent review of standards commissioned by QCA concluded: "It is our considered judgement that QCA has done a commendable job in its effort to assure quality of the A level examinations, especially as QCA is a developing organisation. In addition, it must contend with a raft of notable changes: in curriculum, examination practices, consolidation of awarding bodies, policies seeking to expand upper secondary and university enrolment, and increased school accountability, among others."[3]
9. The system is supported by a range of independent exam boards, each with their distinctive strengths and heritage, competing to deliver efficiencies and to modernise infrastructures. The accuracy and precision of the examination system is one that would be the envy of almost any other industry:
10. The examination system has seen unprecedented investment in technological advances estimated at around £150 million over the last ten years with relatively little direct support from the taxpayer.
11. The exam boards are now well-placed to take on the challenges and opportunities that new technologies will provide. Many innovations are already in place, such as on-demand electronic tests; adaptive tests, e-portfolios, and industrial scale electronic script management. It is important that a measured and long term view is taken with technology which isn't just about relatively simple on-screen objective tests or replicating paper-based conventions in an electronic format. Eye-catching initiatives, such as the Adult Basic Skills tests, require greater scrutiny and analysis. The real opportunities for assessment lie in the creation of new virtual environments and interactive processes. A glimpse of what is possible is the virtual geography field trip created by Cambridge International Assessment, which makes it possible for students to interact with a simulated tropical rain forest, enabling their skills to be tracked as well as their answers logged - a significant enhancement of pen and paper work.
A far from perfect system 12. Despite its many clear benefits, there is growing unease from many stakeholders that there is something not quite right with the system. The sources of these tensions are not straight forward but we believe the system is most dysfunctional when policy makers and regulators begin to intervene too closely in assessment design, set prescriptive and unnecessary requirements, or actively participate in the development and implementation of qualifications.
This dissatisfaction tends to cluster around the following issues:
'the system is too burdensome' 13. The system is sometimes caricatured as being heavily bureaucratic and expensive and it cannot be denied that the administration of valid and reliable examinations comes with an associated amount of bureaucracy and cost. The submitting of accurate candidate data, the administering and timetabling of exams, the collection and distribution of results will never be entirely free from bureaucracy.
14. There have been huge advances in recent years such as: · on-line entries and results services, · the expansion of Exam board customer support services, · the NAA's programme to 'professionalise' exams officers, · direct capital spending on exams offices, · completion of the JCQ initiative 'Eight Pledges to reduce Bureaucracy' · and the centralisation and consolidation of exams offices within large institutions.
15. Yet there is still a feeling that more can be done to take pressure out of the system and the joint Council for Qualifications (JCQ) and the Federation of Awarding Bodies (FAB) remain committed to working together wherever possible to ensure that the systems and processes we require customers to use are aligned.
16. To some extent, the level of bureaucracy is determined by the assessment model used for any particular qualification, along with its structure and design. It should be noted that, as a rule, locally assessed, centrally moderated models (such as coursework) tend to be more bureaucratic than examinations. Yet often Government requirements or regulatory mandates have been at one remove from the impact on organisations delivering qualifications. It was the regulator, supporting government policy, that determined in its 2002 GCSE criteria that each GCSE must contain coursework. This was driven by a perfectly valid view that coursework supported the important practice of 'learning by doing' or 'applied learning', but the mandatory requirement across all subjects was disproportionate to the gain, made the volume of coursework unmanageable, and brought a perfectly valid form of assessment into disrepute.
17. Too often policy has driven solutions which take no account of their cost, manageability and impact. Key Skills, the first qualification to be designed and run by the regulator, became the only single unit qualification that had both an external test and a locally assessed portfolio, both designed to assess exactly the same thing. This belt and braces approach derived from a ministerial view that external tests were 'rigorous', but with an acceptance that local assessment was the best way of confirming that learners could actually apply their key skills to real tasks.
18. The blueprint for 'specialised' Diplomas, as laid down in the 14-19 Education and Skills White Paper, appears not to have taken into account cost and complexity or to have acknowledged at all the bureaucracy that would be needed to underpin it. A delegate at a recent OCR conference described the Diplomas as 'every exam officer's worst nightmare'. The requirement to provide an overall grade for the Diplomas has been an additional decision which does not seem to take into account the additional burden on schools and colleges.
19. Once initiatives have been put in place, the regulator and the system in general is capable of learning from, and correcting, mistakes. The reduction of A levels from six units to four and revisions to GCSE criteria are examples of this. But QCA has been less successful in trying to reduce the burden at the implementation end. The English regulator's project on common centre approval (widely believed to have cost £1 million pounds) seems unlikely, after more than two years, to deliver any tangible benefits. It is just one of a series of misguided attempts to find quick fixes and magic bullets where there are none.
20. Whilst much has been made of the costs of assessment in our system, and there are undoubtedly further efficiency savings to be made, more attention needs to be paid to evaluating the alternatives. A substantial case could be made to show that qualifications developed for large national uptake, in a competitive market, will always generate greater efficiencies than a more local, devolved model (i.e. teacher assessment). Although we will highlight the risks of over-dependence on exams as an assessment model, they are undoubtedly the most cost efficient way of assessing large numbers of people. Discussions with the Learning and Skills Council have led us to believe that the spend on exam board fees is a miniscule proportion of their overall spend. When looking at cost, we would recommend that some consideration is given to the costs of alternative systems operated in other countries.
'there is too much assessment' 21. This is intrinsically linked to the theme of burden and bureaucracy but refers to the view that learners themselves are subject to too much assessment. Again, much of this is a consequence of policy decisions or too inflexible, one-size-fits all, regulatory requirements. As already stated, many of the forthcoming reforms will address this issue. Whilst we are likely to see increased modularisation in the new GCSEs, which will lead to more ongoing assessment, this will be balanced against a reduction in coursework.
22. Moreover, we are developing a view that the spreading out of assessment over a longer period of time is less stressful than a concentrated period of assessment at the end of a two year period. In addition, our conversations with teachers lead us to believe that it is the sustained, unnecessary and inappropriate mass testing of very young people through the key stage national tests that is the single biggest cause of the view that there is too much assessment. (This is discussed at greater length in Cambridge Assessment's submission to the Committee.)
23. The amount of qualifications taken and achieved, and therefore the amount of assessment undertaken, has risen significantly in recent years[5]. A greater number of more qualified people must largely be a good thing, although there are some behaviours, such as the amount of re-sits taken, which need further analysis Also there is a tendency for some candidates to take increasing numbers of GCE and GCSEs, when higher level qualifications, requiring greater stretch and challenge, or different more skills- based qualifications, might be more appropriate.
The range of qualifications is too narrow, stifling innovation and choice 24. This phenomenon of learners being entered for increasingly large numbers of GCSE/GCEs may be a symptom of the view that the current curriculum is too narrow.
25. In truth, the curriculum has been getting broader and richer exponentially over the last ten years. With pathfinder projects to 'flex' the curriculum and the introduction of increasing numbers of alternative qualifications on section 96 (the list of qualifications approved for use in schools), we have seen a transformation of what many schools and colleges now offer. The 'Entitlement Framework' in Northern Ireland and the Welsh Baccalaureate are commendable examples of proportionate initiatives to broaden the curriculum offer.
26. However, many traditional institutions have yet to embrace these new opportunities; the dominance of General Qualifications in terms of recognition and the value placed on them, combined with poor levels of advice and guidance, means that some learners are still condemned to an inappropriate diet solely of General Qualifications. This may be why some employers and HE institutions still complain that current qualifications do not provide rounded, multi skilled, motivated young people.
27. Government policy on assessment has tended to reflect a nervousness of any form of assessment other than a formal examination. A belief that this is the only 'rigorous' way of assessing achievement has led to many alternative qualifications which assess the practical application of skills with suspicion. This was one of the key reasons for endless tinkering with the GNVQ and its final withdrawal and why we ended up with Vocational GCSEs were developed in such a way as to look the same as any other GCSE. Some have accused the current Diploma developments of showing signs of 'academic drift'.
28. Tomlinson recommended the withdrawal of A Levels and GCSEs as the most certain way of resolving the domination of these qualifications over the rest. OCR's view has always been that this solution is too drastic, and that it is possible to work towards a fuller curriculum by gradual and careful enhancement of the provision. Single, one hit initiatives, such as GNVQ, or possibly even Diplomas, are not easy or guaranteed solutions. There are other ways of implementing policy, as Cambridge International Examination's submission highlights.
29. Nor should we assume that general qualifications need to be wholly knowledge based, purely academic qualifications. Once again, the system continues to adapt and improve in response to change and to learn from its mistakes. The last round of GCSE developments took place against fairly prescriptive regulatory criteria which, narrowed the opportunity to develop stimulating and imaginative qualifications that would engage learners and allow teachers to bring a wide range of teaching styles to the classroom. We believe the new criteria will allow us to develop far more engaging new GCSEs and that their introduction in 2009 will be something of a watershed. A precursor has been the new GCSE in Science suite 'Twenty First Century Science', introduced in September 2006, ahead of the other new GCSEs. This highly practical suite, rooted in contemporary, relevant contexts has proven a runaway success with teachers and learners alike at a time when interest in Science appeared to be in terminal decline.
Using assessment to measure the wrong thing 30. The purpose of qualifications has already been set out. Problems arise when they are used as a proxy measure for completely different things. The use of qualifications in school performance tables, national targets, OECD comparisons etc leads to misinformation and drives undesirable behaviours. The use of performance tables in particular, leads to cynicism and a devaluing of qualifications and learning. Assessment is designed around that which it is intended to assess - it is no doubt wholly possible to devise mechanisms (many of which already exist) to assess the performance of schools and colleges, but qualifications are the wrong instrument. Cambridge Assessment's submission deals with this area in more detail.
2. Section 2
Questions Is the testing and assessment in 'summative' tests (for example, GCSE, AS, A2) fit for purpose? 31. By and large the assessments used in GCSE, AS and A2 are wholly fit for purpose and are adapted to suit the different nature of what is being assessed, albeit within a framework of regulatory criteria. Where wider skills and different learning styles come into play, other assessment models and qualifications are available outside of those offered by general qualifications.
Are the changes to GCSE coursework due to come into effect in 2009 reasonable? What alternative forms of assessment might be used? 32. It is a shame that coursework has been dropped across many subjects. All assessment requires trade-offs between validity (its ability to reflect the genuine level of achievement of a learner) and its reliability (its ability to produce the same outcome for learners who reach the same level of performance). Well designed coursework can sometimes deliver validity which sit-down examinations do not because of the inherent artificiality of the examination environment. Much of the criticism of coursework relates to scope for plagiarism; this is only a risk under the increasingly prescriptive regulation applied to coursework which means that all candidates undertake similar tasks, rather than the highly personalised work which characterised the original introduction of coursework. It is also disappointing that coursework has been scaled back at a time when technology, in the form of e-portfolios which OCR has already trialled in several subjects, offers the opportunity for learners' work to be monitored and audited in ways which were not previously possible.
What are the benefits of exams and coursework? How should they work together? What should the balance between them be? 33. In reality, there are a great many assessment models - examinations and tests come in many forms and can involve controlled assignments, practical activities, orals etc. Coursework might include the creation of artefacts, musical compositions, major project management activities and research projects. Often assessment takes place whilst a learner is performing a task, say in a dramatic performance, conducting a live experiment during a field trip, dealing with customers during work experience and so forth. It is necessary to design assessment around what is to be assessed.
Will the ways in which the new 14-19 diplomas are to be assessed impact on other qualifications, such as GCSE? 34. So far it has been the other way round. In seeking parity with GCSE and GCE, the main parts of the Diplomas have increasingly adopted models which mirror the models for GCSE/GCE laid out in the regulatory codes of practice. The grading structures have also been adopted to mirror GCSE/GCE scales. The diplomas already have a minimum of four different assessment models within them. The increased modularity of GCSEs from 2009 should encourage greater take up of components of GCSEs within Diplomas. With careful management and a vision that goes beyond 2013, it should be possible to see components of GCSEs, Diplomas and other qualifications being combined to create new choices which are not as narrow as the prescribed Diploma routes, and not as unvaried as a diet entirely of General qualifications.
Is holding formal summative tests at ages 16, 17 and 18 imposing too great a burden on students? If so, what changes should be made? 35. Evidence suggests that the majority of learners prefer assessment to be spread out over time and closer to the learning experience. OCR has supported QCA in developing guidance to teachers on synopticity at GCE to ensure that candidates are able to demonstrate a full grasp of a subject across and between topics.
How does the national assessment system interact with university entrance? What does it mean for a national system of testing and assessment that universities are setting entrance tests as individual institutions? 36. It has never been claimed that A levels should be the only tool used to determine university entrance (or recruitment to employment for that matter). The Cambridge Assessment response provides information on how university tests provide additional information about potential undergraduates, and where they seem to merely replicate existing 14-19 assessments (as with SAT 1)
General questions Who is the QCA accountable to and is this accountability effective? 37. We understand that QCA is accountable to the Secretary of State, who is accountable to Parliament. What is less clear is precisely what QCA is accountable for. This is a consequence its somewhat open-ended statutory remit as defined in the 1998 Education Act, a tendency for ministers to add to QCA's remit over time through ad hoc remits and funding, and a tendency over time for QCA to shed its independence from government and to act as the government's implementation arm for qualifications policy.
What role should exam boards have in testing and assessment? 38. We believe our evidence sets out clearly the key role of examination boards in developing and delivering qualifications for 14-19 year olds. We would like to emphasise again the delicate balance of stakeholder interests that must be obtained to ensure a valued qualification system, and the unique position of examination board, which possess deep understanding of assessment and qualifications, and sit independently of government and between the many stakeholders enables them to play the role of consensus builder, provided that regulation does not prevent them from doing so.
Footnotes
1 Source: Cambridge International Examinations 2 GCSEs and A levels: the experiences of teachers, students, parents and the general public, QCA February 2006 (results of an independent survey commissioned with MORI) [1] Source: Cambridge International Assessment Submission to the Committee [2] GCSEs and A levels: the experiences of teachers, students, parents and the general public, QCA February 2006 (results of an independent survey commissioned with MORI)
[3] Maintaining GCE A Level Standards, QCA October 2001 [4] Report on the performance of awarding bodies for general qualifications in 2006, QCA, 2007 [5] 'The success rate for 16-18 year olds taking full level 2 qualifications has improved by 9 percentage points over a two year period to 67% in 2005/06, with the success rate for adults following the same programmes increasing by 11 percentage points to 66% over the same 2 year period.' Further Education and work-based learning for young people - learner outcomes in England 2005/06, Learning and Skills Council, April 2007
|