Annex 3:

Testing and assessment - the use of electronic media

This annex focuses on the national curriculum assessment system, looking at current developments and how they might shape future practice, based on the key stage 3 ICT on-line test and the Making Good Progress pilot.

1. Current developments in national curriculum assessment using electronic media

1.1 To date, the most common use of e-assessment in England occurs in the marking of general qualifications through the use of scanning technology and on-screen marking by markers. However, since 2004, the QCA has pioneered an innovative test designed to assess capability in key stage 3 ICT.

1.2 This test allows pupils to take tests within a one month test window thereby allowing secondary schools the flexibility to timetable the tests to school needs. In order to make this possible, each test is slightly different to minimise the chance of pupils passing on information. Tests are automatically marked and results, with diagnostic feedback, are sent directly to schools.

1.3 The test is delivered securely to schools through a specifically designed infrastructure that ensures that the test can run on all school operating systems. Small trials have demonstrated that the infrastructure can be utilised for the delivery of other tests.

1.4 As part of the Making Good Progress single level tests trial where tests are available every six months, the NAA will be looking to utilise the infrastructure provided by the ICT tests as means of delivering tests and test results and the application of on-screen marking. This will require extending the infrastructure and associated support into primary schools.

2. Ways ahead

2.1 Pupils growing up in the 21st century are familiar with a wide range of technological solutions and expect to see their application into every day life. A test system designed in a different epoch is becoming more and more at odds with the every day experiences of pupils.

2.2 Besides the provision of a more authentic experience for pupils, the wider use of electronic technology has the potential to benefit each element of the test system ranging from how we construct tests through to the delivery of results. Further, these developments should be seen as a model for all national curriculum subjects and broader curriculum areas such as functional skills and problem solving skills. By harnessing and developing technology, we have the potential to take the current system of census assessment to the level of the individual learner.

2.3 Tests will be available at a time when pupils are ready, rather than when the system is ready. The harnessing and development of technology offers a more varied assessment environment with reduced processing time leading to faster return of results and much improved data on the performance of individual pupils, schools and national performance.

2.4 Test development

· Tests will be carefully designed to suit their purpose and use a variety of presentations from multiple choice, extended open tasks, modelling and simulations.

· The tests will be targeted at specific national curriculum levels drawn down from a national test bank using items specifically designed for the task. Items will all be pre-tested to ensure their applicability to the test design.

· E-assessment will permit the use of mixed media including text, animation, video, sound and pictures, thereby presenting engaging examinations in formats more familiar to today's young people.

2.5 Test delivery

· Medium to longer-term targets include the provision of when-ready confirmatory tests whereby entry is determined by secure teacher assessment of pupil progress.

· Tests will be taken on screen where possible and delivered to where assessment is most relevant providing timely, and flexible assessments - for example in a school, workshop or at a fieldwork or work placement site using remote access technology.

· The development and introduction of the infrastructure currently in use form the key stage 3 ICT tests into all primary and secondary schools will provide a secure and manageable assessment system capable of delivering a variety of assessments.

2.6 Marking

· Marking will be undertaken by a mixture of human and/or automated marking regimes determined by the nature of the assessment.

· Electronic tests (and e-portfolios) permit rapid transportation of work for marking and moderation purposes, reducing logistics and time costs.

· Markers will be trained and monitored on-line ensuring quality assurance and quality control.

· Electronic capture and transportation provides the means for improved quality of marking through the introduction of simultaneous second marking, the placing of pre-assessed items into markers allocations thereby checking the application of marking standards as well as re-sending work already marked by markers as a check on reliability.

· Assessments will be divided into sections or single questions to support marker reliability through focused marking.

· Markers will be allocated a random sample of questions to mark rather than a whole school cohort, as is currently the case.

· Senior markers will monitor marking teams in real time, ensuring quality of marking and monitoring of progress and fatigue.

· The system will generate real-time data on marker performance allowing for early identification and correction of marking error.

2.7 Issue of results

· The return of results to pupils, schools and parents will be accelerated to facilitate more time to interpret and act on results.

· Results will be available through a range of technological devices including websites and mobile phones.

· Results will contain more diagnostic information allowing for formative assessment, self-assessment and continuous improvement.

2.8 Wider application

· Other assessment modes will be introduced either to complement tests or as stand-alone through the development of e-portfolios where pupils can gather and present their work using a variety of media including video, sound clips, documents and examples of work within and beyond the school environment.

· Through the use of voice profiling for example, tests can be taken in a variety of locations with the opportunity to take sample or practice tests designed to give formative feedback directly to the learner.

 

 

June 2007