Education CommitteeWritten evidence submitted by Ian Thow

Summary

The key points of this submission relate to the Qualifications and Awards procedures in Scotland for 15 to 18 year olds) which can then be compared to the situation in England. It describes how the Scottish Qualifications Authority (SQA) organises assessments and outlines two different methods of marking which the SQA uses. It also identifies the advantages and disadvantages of these marking processes. Finally, the submission explains, with an example, the issues which arise in Scotland with respect to complaints to the SQA in relation to questions raised (by schools and individuals) concerning the validity and reliability of assessment questions and the accuracy of the marking instructions in the absence of an organisation such as OfQual.

My Personal Details, Background and Experience

I have recently retired after 30 years experience as Principal Teacher (Head of Department) of Religious, Moral and Philosophical Studies (RMPS) in a large comprehensive secondary school in Scotland. During my time of employment, national qualifications in Religious Studies (as it was initially named) were established and the first presentations of candidates occurred in 1984. Over the following 25 years various developments in national qualifications have taken place in Religious Studies (later to be called Religious, Moral and Philosophical Studies ,RMPS, from 2000 onward). As Principal Teacher of RMPS I was involved in teaching these courses and presenting candidates for national examinations.

I have also been an examiner with the Scottish Qualifications Authority (SQA) for 15 years at different levels of RMPS (Higher, Intermediate 2 and Intermediate 1). In addition, I have personally written complete teaching Units and assessments in relation to these developments in the Scottish Examination system. I therefore have very wide experience in both teaching the subject and setting questions and marking (as an examiner) with the SQA.

1. I am writing this submission in response to the inquiry that the House of Commons Education Committee is currently holding into the administration of examinations for 15 to 19 year olds. While I note that the current inquiry relates specifically to England, I thought it might be useful to the committee if I were to present information of our situation in Scotland for comparison purposes.

2. As you may be aware, the examination system in Scotland differs from that in England in that there is only one examination and qualifications awarding body, the Scottish Qualifications Authority (SQA) whose annual turnover of candidates sitting national examination is around 160,000. This is obviously a significant contrast to the situation in England in relation to the number of awarding bodies, but it may be worth noting some of the advantages and disadvantages of a unitary rather than multiple awarding bodies.

3. Advantages

(i)Every candidate sitting an examination in each subject and at each level in that subject faces the same question papers. Currently in Scotland there are Standard Grade examinations at 3 levels (Foundation, General and Credit) Intermediate (levels 1 and 2) examinations, Higher (one level) and Advanced Higher (one level) examinations. The Scottish “Higher”, which is normally sat by 16 and 17 year olds is the standard for university entrance in Scotland.

(ii)The distinct advantage of the Scottish system is that examinations at all levels are standardised for all candidates who are sitting an examination in a particular subject at a particular level. This can be a useful comparison guide for both individual pupils, schools, further Education establishments and potential employers.

(iii)Another advantage of this system is that when the marking of scripts takes place, often using central marking (which means that, over a relatively short period of time, all scripts are marked by the same markers on the same premises at the same time) so the quality of marking is more consistent between the different markers. At central marking ,there are also regular quality assurance checks of all markers by experienced examiners and any marking issues eg severity or leniency or lack of consistency in marking, can be identified and remedied quickly.

(iv)A third advantage is that new markers can be given individual training “on the spot” and have the benefit of being able to discuss uncertainties and issues while marking with other, more experienced, markers.

(v)It is also the case that the marking of all scripts for a particular subject can be completed over a few days of intensive marking but this can also have disadvantages.

4. Disadvantages

(i)As all subjects do not operate a central marking system (mainly on grounds of cost and logistics – subjects with many thousands of candidates and hundreds of markers cannot be accommodated in a central marking situation) then there is a two-tier system of marking in operation by the SQA. The non central marking system involves all markers marking sample scripts sent to them and then taking them to a meeting of all markers for that subject or level and being given indications by the Principal Assessor for that subject about what answers are acceptable or not. This is a major undertaking and is very formal. Markers who attend such meetings complain about their brevity and impersonal nature and about the lack of opportunity (sometimes due to speaking out in such a large meeting – which is intimidating for some) to ask individual questions before they return home to mark scripts over the following three weeks or so. Further, as markers are more isolated in this situation in comparison with a central marking event, there is more emphasis on these markers using their own judgment when faced with issues on scripts or in the interpretation of marking instructions. This marker judgement of particular issues may be valid or not, depending on the experience and expertise of the individual marker. While these markers do have the possibility of contacts with the examiners should any problems and issues arise during marking, it is more than likely that, when markers send back their completed scripts, consistency and reliability may be issues not only across the scripts marked by an individual marker, but also when different markers have marked their scripts. This can raise significant issues of standardisation and validity in script marking.

(ii)A further disadvantage of non central marking is marker training. The nature of such marker meetings outlined in the previous paragraph raises the issue of the relevant training of markers which is especially important when new markers are recruited, especially when new markers are recruited. One of the key factors that differentiates the two different approaches to marking is that the central marking approach has important benefits and advantages over the more traditional approach. This is because, in central marking, training is much more immediate, structured and “hands on” with immediate feedback and gives such markers, especially new and inexperienced ones, the advantage of immediate access to experienced markers and examiners compared to the alternative.

(iii)One disadvantage of central marking is marker fatigue. Such events take place at a central location and involve 3x three-hour sessions per day over a period of three or four days. The sheer intensity of this can be an issue for markers. On the other hand, those markers who work from home have their normal teaching responsibilities during the day and then may spend several hours in the evening marking scripts, The situation is better at weekends when normal work is not a factor. There is probably no perfect solution to this situation although, having been involved in both approaches to marking, there is no doubt in my mind that the central marking approach is superior. I also realise that, given the much greater numbers of scripts to be marked in England, then the logistical problems are multiplied many times.

I have explained my current situation in detail as a distinct disadvantage of having a single national awarding body if that body is not directly responsible to an education Ombudsman. We clearly need a Scottish OfQual equivalent to deal with such issues. While there may be issues in England in relation to multiple awarding bodies for qualifications at least there is recourse to OfQual if circumstances merit this.

November 2011