Education CommitteeWritten evidence submitted by the Chartered Institute of Educational Assessors (CIEA)
Summary
1. The Chartered Institute of Educational Assessors (CIEA) is pleased to provide evidence to the Committee’s inquiry.
2. We believe that improvements to the management and practice of assessment can bring about greater reliability.
3. The introduction of pre-testing at item level would overcome many of the shortcomings of the current system.
4. Any changes should be accompanied by appropriate external professional training and accreditation.
5. Successive governments have accepted expert recommendations in respect of assessment (2009) and the inclusion of assessment in teacher standards (2011).
6. The development and support by the CIEA of a national network of Chartered Educational Assessors provides access to assessment experts who can support the changes we recommend.
The Chartered Institute of Educational Assessors
7. The Chartered Institute of Educational Assessors is a professional body dedicated to supporting the needs of everyone involved in educational assessment.
8. Our primary concern is with the quality, validity and reliability of assessments and examinations.
9. Our members include senior examiners, moderators and markers and individuals with an interest in or responsibility for assessment in primary schools, secondary schools, colleges, universities, training centres and other educational organisations. We provide training and support at each stage in the assessment process and individual accreditation against the national CIEA Professional Framework.
10. To date the CIEA has awarded Chartered status to 160 of the country’s most experienced educational assessors. We continue to develop this network as a national resource to support improvements in assessment policy and practice.
The Number and Range of Awarding Bodies
11. The CIEA’s view is that the number of awarding organisations is less important than the quality of the processes they apply. The robustness of assessment should be consistently high and not in itself a market differentiator between different organisations. It is difficult to see how a move to a single national body would stimulate the improvements that are needed in assessment practice.
Ensuring Accuracy
12. A number of changes to practice would improve the accuracy of papers and thus the reliability of student outcomes.
13. The CIEA has recommended to Ofqual that Awarding Organisations be required to pre-test all general qualifications, including GCSEs, A levels and Diplomas. This can be done at item level, which would reduce the time commitment for schools and the 16 and 18 year olds attempting each item, and allow each Awarding Organisation to develop a bank of items in any specific subject for use in future papers.
14. The data obtained by pre-testing will allow the reliability of each item to be established before the assessment goes live. It will allow the Awarding Organisations to determine the particular items that best discriminate amongst the cohort being assessed and establish the validity of each item.
15. Pre-testing will also allow the Awarding Organisations to pre-determine grade boundary scores and to calculate the risk of error before the examination is taken.
16. The introduction of pre-testing at item level would require a change in the way that examinations are set. At present the chief or principal examiner sets an entire paper. Under the arrangements proposed by the CIEA teams of people would write a variety of items for one paper, thereby developing a bank of items that can be used for any one qualification. Awarding Organisations will need to engage appropriately trained and qualified staff to undertake this work.
17. There is a need for greater transparency and support for the role of scrutineers. Currently it is not clear who carries out such scrutinies, only that a scrutineer has been used. The CIEA recommends that scrutineers are named and their assessment qualifications published.
18. The CIEA’s view is that training and accreditation should be provided by an external organisation.
The Role of the Chartered Educational Assessor (CEA)
19. The importance of accurate assessment has been recognised in the new teacher standards that will come into force from September 2012.
20. In 2009 the then Secretary of State accepted the recommendation from the Expert Group on Assessment that all schools should have access to a Chartered Assessor by 2020. While that deadline is still some time away, the continued expansion of a network of Chartered Educational Assessors(CEA) will improve reliability.
21. The CIEA has collected a number of case studies describing the interventions and impact of CEAs. These are available on the CIEA website.
22. The role of the CEA is to quality assure the assessment processes that a school, college or workplace use. The CEA will look to improve the processes of assessment common to all institutions, across multiple disciplines or subject contexts. This will involve ensuring that the preparation for assessment is effective, auditing assessment policies and procedures that are in place and how the strategy for assessment is delivered. The CEA will offer support in designing and developing effective assessments that are reliable, valid, fit-for purpose, manageable and fair.
23. The CEA will also look to improve the way that assessments are conducted. This will involve developing effective assessment criteria and setting suitable assessment objectives as well as arriving at consistent and accurate judgements. This includes developing effective standardisation and moderation procedures, as well as collecting and analysing assessment information from the outputs of these assessments.
24. Further, the CEA will look at ways of using this information to inform future teaching, training and learning programmes, as well as reviewing the effectiveness of the assessment itself. The evidence will be used to provide feedback to learners as well as members of the assessment teams, their line managers and stakeholders and to review the impact of the assessment.
25. The information will also form the basis of the next teaching or training programme as well as providing data which can be used to write reports for parents or carers, for institution managers and for other stakeholders such as local or national government as well as local and national industry.
26. The CEA will also encourage teams of assessors to reflect upon their individual performances, use assessment information to help individuals to plan their own professional development, by addressing any areas for development highlighted by the evidence, and evaluate the impact on their own learning in these key areas.
Commercial Activities
27. The CIEA does not feel qualified to offer opinions on this element of the inquiry.
November 2011