Education CommitteeWritten evidence submitted by Council for Awards in Care, Health and Education (CACHE)
The arguments in favour of and against having a range of awarding bodies for academic and applied qualifications (including A Levels, GCSEs, Diplomas, BTECs and OCR Nationals), and the merits of alternative arrangements, such as having one national body or examination boards franchised to offer qualifications in particular subjects or fields.
How to ensure accuracy in setting papers, marking scripts, and awarding grades.
The commercial activities of awarding bodies, including examination fees and textbooks, and their impact on schools and pupils.
This Paper provides a description of the work of Council for Awards in Care, Health and Education, a niche awarding organisation offering qualifications in care, health and education. It includes a rationale for having a range of awarding bodies, a description of how we ensure accuracy in awarding, and brief comments on our commercial activities.
1. The Work of the Council for Awards in Care, Health and Education, CACHE
1.1 The Council for Awards in Care, Health and Education, CACHE, is a charity and a not-for-profit company limited by guarantee. It was originally called the National Nursery Examination Board, NNEB, and was set up in 1945 to provide high quality childcare qualifications. It merged with the Council for Early Years Awards in 1994 and under its new name of CACHE currently offers vocational qualifications for the childcare and education sector, and from September 2009 people caring for adults either as domiciliary care, or in residential care or as auxiliary heath workers.
1.2 CACHE’s charitable activities are:
to improve the skills level of the childcare, adult care and education workforce and its allied disciplines through the provision of qualifications; and
to influence the professional standards of the workforce.
1.3 To accomplish this, we are working with employers, trade unions, professional associations, Sector Skills Councils, (SSCs), and government departments to offer up-to-date qualifications for a wide variety of professions in the Children’s Services Sector including childminders, crèche staff, nannies, nursery nurses, nursery staff and managers, playgroup leaders and assistants, play workers, school support staff and teaching assistants, and the adult care sector.
1.4 Our core business is as an awarding organisation of vocational qualifications. We strive to be the provider of the best qualifications in our niche and to be highly regarded by our regulators. CACHE either writes the syllabus based on the workforce development strategy for the sector, or develops it jointly with employers, or it uses shared units developed by the SSC. CACHE then develops the assessments which are used to determine whether the candidate has gained sufficient knowledge and skills from the course to be awarded the qualification. For most qualifications at least one of the assessments is marked by professional examiners appointed by CACHE. Assessments by tutors are verified by a CACHE team of subject experts. Qualification certificates are then awarded.
1.5 CACHE is the market leader for the Childcare, Teaching Support and Playwork sector, but has a lower market share for Health and Social Care because it only entered the market in 2009 at the request of its customers who were looking for an alternative to the larger awarding organisations.
1.6 CACHE has customers in 11 countries although our core market is England, Wales and Northern Ireland. We have approximately 1,000 active customers including FE Colleges, independent training providers including employers, and schools. Some of our vocational qualifications are available in the Welsh language. We registered more than 100,000 learners in 2009-10.
2. Developments in Awarding
2.1 The role of awarding organisations is continuing to evolve from being organisations which developed and marketed their own qualifications to being organisations that award qualifications containing units developed by Governments and Sector Skills Councils. The awarding bodies compete on brand, customer support and on assessment methodology. This change will not be fully implemented till 2012.
2.2 CACHE has prepared for this by continually improving its processes so as to increase value to customers. On-going development of improved processes and new IT developments will allow CACHE to offer a better service to all its customers.
2.3 The aim of the new Qualifications and Credit Framework (QCF) is to assist candidates to build on learning already achieved and gives them more choice and flexibility to put together a collection of units which match their own particular learning needs. CACHE has 56 qualifications accredited for the QCF. These are high quality, which are inexpensive for training providers to implement, which offer maximum benefit to learners and which are sufficiently flexible to match the breadth of the different learning and assessment styles of candidates. They combine competencies assessed in the workplace and knowledge assessed by a variety of assessment methods including assignments prepared by CACHE.
2.4 A new Regulator, Ofqual, has replaced the Qualifications and Curriculum Authority. Ofqual is proposing a lighter touch, market-driven system of regulation based on the measurement of risk and the provision of best value to learners and learning providers. We welcome this initiative.
3. Value added by CACHE
3.1 Evidence that our qualifications raise the quality of care and education for children and their families comes from successive Ofsted reports into the quality of daycare provision, from the high levels of employer demand for CACHE qualifications and from the Effective Practice in Preschool Education (EPPE) research programme. CACHE is the major provider of qualifications in the children’s sector. CACHE has only just entered the adult health and social care market and it is too soon to assess how its presence is affecting beneficiaries in this new sector.
3.2 Our customers are telling us that they value us. CACHE commissions regular customer satisfaction surveys. Last year’s survey undertaken by an independent agency shows that 94% of customers were satisfied with our service. This is an encouraging indicator of the customer-focused improvements that have been made throughout business operations during the last 12 months. CACHE will be working to increase our customer satisfaction during the next 12 months.
3.3 We also ask our customers why they contract with us rather than our competitors.
3.4 In our 2010 survey, training providers told us that they reviewed their selection of Awarding Organisations regularly. More than half of providers were likely to change Awarding Organisation for at least one qualification in the next three years.
3.5 Main considerations when providers decide which awarding organisation to use are:
Suitability of the qualification for learners including progression to higher levels and employment.
Ease of using the awarding organisation’s online system for registering candidates and recording assessment results.
Speed of response of awarding organisations to customer queries.
Type of assessment.
Reputation of the awarding organisation and the qualification with employers and learners.
3.6 Price is very important but quality will always be the deciding factor. Least important was the breadth of offering.
3.7 Providers associate the following attributes with a “good” awarding organisation (AO):
Trustworthy (with learners coursework, to get information correct).
Efficient (quick and timely customer service).
Learner-centred.
3.8 They also want awarding organisations to embrace innovation, particularly for assessment, though this is slightly less important than the above.
3.9 When deciding whether to purchase from a particular awarding organisation, providers draw on their experience/perceptions of:
“Functional” factors such as overall efficiency including ease of administration; and
“Emotional” factors such as reputation and track record.
3.10 Compared to the larger awarding organisations, CACHE is rated more positively for having a friendly face and offering a personalised service, for being trustworthy, quick and timely and for leading in development of qualifications and assessment, more “consultative” and “collaborative”.
3.11 Training providers want choice and competition because it reduces risk and it enables them to meet learner and employer needs. A single provider would not provide the variety of qualifications and pathways required by employers and larger numbers of learners would be affected if a service failure occurred.
3.12 In summary, a niche awarding organisation is small enough to:
Tailor qualifications for learners needs.
Respond quickly to customer suggestions for improvements.
Employ up-to-date subject experts who can lead on qualification development.
Be close to employers and respond quickly to their changing requirements.
Be more “consultative” and “collaborative” with customers and employers.
4. How to Ensure Accuracy in Setting Papers, Marking Scripts and Awarding Grades
4.1 Senior staff at CACHE continually reviews the risks CACHE faces, prioritising these in terms of potential impact and likelihood of occurrence and identifying means of mitigating the risks. A Risk Management Strategy has been implemented and the Risk Register is reviewed annually by the Board. In addition, CACHE’s Scrutiny Panel monitors CACHE’s current internal controls and quality assurance systems.
4.2 Any events due to failure or variation to meet CACHE’s Standards are reported to the Scrutiny Panel. CACHE has put in place quality controls to minimise these and to ensure a fast response when they do occur. It has improved the way that it records and responds to complaints by customers. The integrity of CACHE’s qualifications is paramount.
4.3 Every centre undertakes a self-assessment and this is checked and risk rated by the CACHE’s customer quality advisers. High risk centres are given more support. CACHE’s customer quality advisers investigate all infringements of regulations and provide support and training to Centres as appropriate. CACHE’s staffs verify a sample of assessments made by centres. Another team of Principal Examiners monitors assessments made by CACHE’s team of examiners.
4.4 The key to high standard awarding is to:
Have rigorous processes which must be followed by all staff.
Have high quality risk management which predicts what can go wrong and then put into place preventative actions.
4.5 CACHE believes that the interlinked factors for accuracy in awarding are:
Subject specialist knowledge in relation to the qualifications offered.
Rigorous application of the assessment and awarding processes.
Training, professional development and management of writers, examiners and scrutinisers.
Quality assurance processes.
4.6 Although governed by regulatory requirements and recognised assessment principles, translating these critical factors into accurate assessment practice is the skill of the individual awarding organisation.
5. Accuracy in Setting Papers
5.1 CACHE has:
Processes which provide auditable evidence at every stage.
Sources of subject specialist knowledge to ensure both that the dissemination of the syllabus requirements/learning outcomes into the requirements for the candidates (questions/assessment criteria) and that the responses expected of candidates (answers/evidence) are realistic, current and correct.
Consistent test/assessment specifications, built on learning outcomes and assessment outcomes appropriate to the assessment level.
Development of assessment writers, for each qualification, in relation to the precise subject content, assessment outcomes and levels, requirements of the test specification and assessment writing techniques appropriate to the type of papers to be set.
5.2 Accuracy is dependent upon:
The individual Awarding Organisation’s commitment to the detail of the assessment item writing process undertaken by specialist assessment writers (as above), together with valid systems for editing, revision and scrutiny of individual assessment items.
Compilation of papers through adherence to the test specification.
Paper revision and scrutiny by independent subject specialist revisers and scrutinisers.
Internal review and audit at all stages of compilation.
Consistent monitoring by a subject specialist/senior examiner/decision maker at each stage from assessment item writing to test paper publication.
5.3 CACHE offers its assessments in English and in Welsh. Accuracy in setting bilingual papers, following formal translation, involves a bilingual subject specialist who ensures the language and context is correct in both languages.
6. Accuracy in Marking Scripts
6.1 CACHE:
Maintains robust pre-standardisation and standardisation processes which exemplify precisely the requirements of the assessment to all examiners.
Provides detailed support materials for examiners through realistic and accurate marking guidelines or indicative content.
Provides a high level of subject specialist expertise into the marking process.
Provides training and development opportunities for examiners to maintain specialist knowledge and to sustain skills in methods of marking appropriate to particular assessments.
Maintains on-going standardisation and monitoring procedures of all examiners.
Maintains an audit of examiners’ performance to identify their adherence to the requirements of their role and to assess their reliability, in order to ensure consistent standards of marking.
7. Accuracy in Awarding Grades
7.1 Accuracy in awarding grades is dependent upon parallel systems of:
Grade review—comparison of current scripts with archived materials to ensure comparability of grade boundaries, (or for a new assessment, comparison with archived scripts in a related area at the same level in order to define the expected standards and therefore the marks which will delineate the grade boundaries). This should provide comparability and accuracy in grades awarded between assessment sessions and year on year.
Examiner review—detailed monitoring of marks returned by examiners to ensure that marks awarded will enable scripts to be located within the appropriate grade boundaries.
7.2 To achieve a high degree of accuracy in awarding grades, the individual awarding organisation will need to have in place awarding processes which include:
Investigation, by the use of specified sampling procedures, of marks returned by examiners to confirm accurate application of authorised marking guidance to ensure the accurate mark for all scripts.
Moderation of marked scripts and/or possible re-marking to ensure consistency across all examiners and scripts, before recommended grade boundaries are applied.
Provision for the review against archived scripts from previous years before agreeing the appropriate marks for grade boundaries.
Application of agreed grade boundaries.
Borderlining procedures to be applied for candidates at one mark below the grade boundary.
Checking-out processes which ensure that manual and electronic systems have recorded marks and allocated grades accurately.
Application of nationally recognised adjustments for candidates with special considerations that apply at the time of the assessment.
Results enquiry processes.
7.3 CACHE believes that the systems which it has in place provide the fairest outcomes for learners undertaking assessments and gaining qualifications which will equip them to progress into the specific areas of care, health and education.
8. Commercial Activities of CACHE
8.1 Approximately 92% of our income is derived from learner and customer registration fees, the remainder coming from the sale of publications and badges.
8.2 Customers are asking awarding organisations to provide specialist support materials to increase learner achievement rates and to reduce the costs of teaching. The materials requested include text books tailored to individual awarding organisation’s qualifications, teaching resources which can be put on a training provider’s intranet, often as part of a learner management system such as Moodle, e-learning materials to support face-to-face training by a lecturer and e-portfolio products. These materials are often offered by awarding organisations at a loss to provide a more competitive offer than other awarding organisations.
There is however a risk with an awarding organisation offering these services—the awarding organisation must be able to demonstrate that these services do not give learners unfair advantage. This is very difficult to do, and so CACHE provides its support services through third parties so that there can be no accusation that CACHE gives preference to certain groups of learners.
8.3 CACHE publishes its fees openly (as required by the Regulator) but not all awarding organisations do this. Some awarding organisations offer large discounts which make their fees lists misleading. Others offer hidden discounts if a training provider transfer all their business to them.
8.4 Some awarding organisations are also selling e-learning directly to learners. This has potential for conflict of interest as awarding organisations must be able to demonstrate that they treat all candidates equally regardless of whether they were training by the awarding organisation. Some awarding organisations have set up arm’s length companies to provide the e-learning but the potential conflict of interest remains.
9. Summary
9.1 We believe strongly that there is a strong market need for a variety of specialist awarding organisations who experts in their vocational area(s), are trusted by the employers in their sector, and who can quickly adapt a qualification to meet employer or learner needs.
9.2 We believe that accuracy of awarding depends on the strength of the risk management throughout the awarding organisation, and the rigid adherence to high quality processes.
9.3 All awarding organisations are operating in an increasingly commercial market, and this can lead to conflicts of interest which need to be actively managed to ensure that standards are maintained and all learners are treated equally.
November 2011