House of Commons - The administration of examinations for 15-19 year olds in England

The administration of examinations for 15-19 year olds in England - Education Committee Contents

8 Support: training and textbooks

142. Professor Stephen Ball of the British Academy told us that "the market in examining is part of a much larger market in educational services".[231] Teacher support is a key area of competition between the exam boards and a growth area in terms of their activity (though not necessarily revenue). In this chapter we focus on two key areas of support: training and endorsed textbooks.

Training

143. In December 2011 the Daily Telegraph conducted an investigation that raised questions about the nature of training offered to teachers by exam boards, in particular whether examiners were giving too much information to teachers on likely question areas or topics that might compromise the security and standards of exams. Similar concerns featured in evidence to our inquiry. ACME told us that training sessions "risk being focused on coaching participants on how to pass the examination" and Dr Tony Gardner of the University of Birmingham warned that training "can too easily slip into explaining to teachers how to train students to maximise marks".[232] Others were more cautious. Professor Sir John of the Wellcome Trust told us that "I don't think we should say blanket-fashion that dialogue between examiners and teachers is wrong because, at its best, it can be constructive".[233] However, he acknowledged that "there is an enormous amount of sailing close to the wind in the way that those with inside information about the examinations use that knowledge, which, frankly, is in an unprofessional way".[234]

144. In its initial report into the Daily Telegraph allegations, Ofqual concluded that "teacher seminars have their place", but acknowledged that "there is a fine line" and that exam boards "need to make sure that in their seminars, lines are not crossed".[235] In its final report, Ofqual reached a rather different conclusion: that "there is a real risk that inappropriate information about the future content of secure exams is disclosed. And there is a risk of narrowing the curriculum through sessions on how to teach the specification." Ofqual concluded that "it is not possible to reduce the risks to an acceptable level, and so seminars relating to particular qualifications should stop".[236] This decision will take effect from August 2013. Ofqual's report suggests that information needed by teachers about the structure of a qualification and its assessment should be made readily available to all schools and colleges via the internet.

145. All three exam boards have stated, both in evidence to us and more widely, their commitment to supporting schools in preparing students for examinations. Exam board chief executives emphasized to us that almost all the information given at training sessions is freely available on their websites, but that some schools prefer "face-to-face" interaction. OCR described Ofqual's action to end face-to-face seminars as "a rushed decision" and expressed disappointment that Ofqual had not consulted more widely with teachers.[237] AQA and Pearson emphasized that much of their training is already offered as web-based support and that they will expand this approach in future.[238]

146. It is important that schools have access to high quality information about qualifications that enables them to prepare their students for assessments. We agree with Ofqual that there are "fairer and safer ways in which information can be shared with teachers".[239] Indeed, it was pointed out to us that "we have one of the most transparent systems in the world in terms of what we publish"[240] and a great deal of information about GCSEs and A levels is already disseminated through examiners' reports, question papers, mark schemes, syllabuses and associated guidance materials.

147. We had particular concerns about exam boards offering enhanced or tailored support to individual schools or groups of schools in exchange for loyalty to a particular qualification over a specified period of time. We consider that such bespoke training could easily lead to "lines being crossed" and inappropriate information being given to schools. We were pleased to note Ofqual's finding that "there are particularly high risks where bespoke qualifications training seminars are offered to individual schools".[241] Ofqual's decision to end qualification-specific training rightly extends to these events for individual schools.

148. There is a common perception that, as exam boards often charge for attendance at seminars (between £100 and £200 per delegate according to Ofqual),[242] they make a profit on training offered to schools.[243] Income data provided by the exam boards confirmed what senior exam board officials had told us: that exam boards make a loss on training. It is, however, very much part of the package of support used to market qualifications to schools. Clearly, some aspects of this support will need to change in the light of Ofqual's decision to end exam board seminars relating to specific qualifications.

149. We believe that the exam system has placed examiners involved in training in a difficult position, with fine lines to navigate. While some were able to do this appropriately and successfully, others were not, and neither the exam boards nor Ofqual were monitoring closely enough. We welcome Ofqual's decision to end exam board training on specific qualifications. Ofqual needs to monitor the impact of its decision and the activities and materials produced by exam boards to replace their seminars. We also recommend that Ofqual monitor other training offered by exam boards, such as marketing events to promote new syllabuses, and more general training, for example on improving results, taking further action if needed. Ofqual must ensure that a school's loyalty to a particular exam board cannot be rewarded with access to information not available to others.

Textbooks

150. Many of the concerns about training also exist with regard to exam board endorsed textbooks, frequently written by senior examiners. Textbooks emerged as an area of particular contention in our inquiry. The Government has said that it has "serious concerns" about the links between exam boards and textbooks.[244] The DfE itself identified the key issues raised in submissions to us. These are:

examiner authorship
the impact on competition in the publishing market (exclusive endorsements and the close alignment of exams and publishing at Pearson)
the branding and marketing of endorsed textbooks
the variable quality of endorsed textbooks and their narrowing impact on teaching and learning

151. As with training, income data provided by the exam boards show that exam boards make very little money from endorsed textbooks, but such books are used to help market qualifications to schools. Pearson told us that "we publish teaching resources as part of an integrated support package which is closely linked to our GCSE and GCE specification from Edexcel".[245] AQA stated that "endorsement acts as a kite-mark; quality assurance ensures the text accurately interprets the specification and assessment arrangements" and OCR aims for "a strong and varied range of support materials" for all its qualifications.[246]According to AQA, having endorsed resources "helps restrain the most misleading market provision".[247]

EXAMINER AUTHORSHIP

152. The predominant pattern in recent years has been for exam board endorsed textbooks to be written by senior examiners. AQA told us that "examiners are leading subject experts and usually practising teachers, so are well-placed to act as authors".[248] On the other hand, others argued that "the skills involved in being an effective examiner are not the same as those needed to devise good curriculum materials".[249] Many have also suggested that examiner authorship leads to a potential or actual conflict of interest for examiners when writing textbooks in a subject where they are responsible for setting and maintaining national standards in exams. One examiner told the Committee that "a conflict of interest is not inevitable, but if the boundaries are ignored, then it would be".[250] Others have taken a stronger line, describing examiner authorship as "insider dealing"[251] and have suggested that examiners "should not be compromised by gaining financially from endorsing textbooks".[252]

153. All three exam boards place contractual restrictions on their examiners, which prohibit them from disclosing their exam board affiliation when writing textbooks. In 2011 Pearson introduced a new code of conduct for examiners, preventing them from being the main author of textbooks linked to syllabuses for which they set question papers. Rod Bristow told us that Pearson had introduced the change because the situation "was putting examiners in quite a difficult position".[253] For the same reason, Edexcel examiners are no longer allowed to be directly involved in the training of students (but may still train teachers). Pearson is also "piloting approaches to prevent examiners from having any involvement in writing any resources for courses they examine".[254] Mark Dawe of OCR told us that if trust in examiners and in the system as a whole is evaporating then

it may be that we are reaching the point where anyone who has seen a question relating to the future cannot be involved in seminars or books, because they have that question in their head [...] but if the sacrifice we have to make is to put some of those things in place to regain the public's trust, that is what we are going to do.[255]

154. While we can see that there are strong arguments for senior examiners being well placed to write textbooks, we are concerned that there is a potential conflict of interest for examiners involved in question paper setting also writing textbooks that are linked closely to the same syllabus. We welcome indications that exam boards may place tighter restrictions on the role of examiners in textbook authorship. We recommend that Ofqual make clear the expected future role of examiners in textbook authorship, in order to ensure a consistent industry-wide approach.

ENDORSEMENT ARRANGEMENTS AND THEIR IMPACT ON COMPETITION

155. Links between publishers and the three English exam boards take several forms. Textbooks linked to Edexcel exams are either published by Pearson (the Edexcel Own range) or are produced by other publishers, such as Oxford University Press (OUP) or Hodder Education, and then endorsed by Edexcel. AQA had until recently an exclusive partnership with Nelson Thornes, and OCR has formal partnerships with three publishers as well as endorsing books by other publishers.

156. Two of these arrangements were brought to our attention in particular, by representatives in both education and publishing, with the suggestion that they had a negative impact on competition in the publishing market and may ultimately risk reducing the range of resources available to schools. One was the exclusive endorsement arrangement between AQA and Nelson Thornes; the other was the close alignment of publishing and exams at Pearson. AQA's arrangement with Nelson Thornes has now ended.[256] Although we appreciate that there were sound reasons for AQA's exclusive endorsement arrangement with Nelson Thornes when it was agreed (securing resources to support a wide range of syllabuses), we agree that exclusive endorsement arrangements risk limiting the range of resources available to schools. We recommend that Ofqual consider restricting exclusive endorsement arrangements between exam boards and publishers in future.

157. Kate Harris, Managing Director of the Education and Children's Division at OUP was critical of the close alignment of publishing and exams at Pearson and called for "guaranteed actual or perceived independence between those setting and marking exams and defining specification, and those who are publishing resources for them".[257] Pearson told us that "we have very strict firewalls between the people involved in textbooks and in the actual examinations" and that "there is no chance of anything that is going on in any publishing activity leaking back or influencing what is in the question papers".[258] However, OUP pointed out that "in terms of the spec development, there are very close links [...] the specification development bit is very integrated with the publishing".[259] Glenys Stacey of Ofqual told us that "for the moment, awarding bodies are required to have conflict-of-interest procedures in place. They need to confirm with us by May 2012 that they have those [...] we will be crawling all over them." However, she acknowledged that "there is more work for us to do to get to the detail of this".[260] Ofqual needs to be satisfied that Pearson has sufficient firewalls in place to ensure that its publishing and examining activities are separate, including syllabus development, and to say so publicly.

BRANDING AND MARKETING OF ENDORSED TEXTBOOKS

158. Exam boards and publishers supplied us with a sample of endorsed textbooks, as well as further examples of textbook covers. We supplemented this with a further sample of textbook covers drawn from our own research. We are concerned that describing a book as "all you need for your course" or saying that a textbook ensures students study topics "to the appropriate level of depth required by the specification" and helps students "to obtain the best grade they can" may encourage a narrow approach to teaching and learning.[261] We saw examples of all these practices and they serve to reinforce the point made to us by the Wellcome Trust that "textbooks have increasingly become 'examination guides' instead of providing broad and deep knowledge. The endorsement of textbooks by awarding bodies exacerbates this problem by promoting teaching to the test".[262] We have serious misgivings about the language used to market some endorsed textbooks and would urge exam boards and publishers to move away from marketing textbooks in this way.

159. We were shown and also found examples of very similar branding of Edexcel syllabuses and Edexcel Own textbooks published by Pearson, with the same logos and cover designs. Competitors in the publishing industry suggested that this gives the impression of "official" resources.[263] Pearson told us that it works "very hard to ensure a choice of books for teachers", through endorsing textbooks from other publishers.[264] Kate Harris of OUP also suggested to us that Pearson is not even-handed in the presentation of its own and other endorsed resources on the Edexcel website.[265] We found that the links to resources from the GCSE and A level qualification pages lead to Edexcel Own resources only. We were able to locate information about endorsed resources from other publishers using a different route, but not from the qualification pages where teachers will tend to look to find information about the syllabus they are using.[266] We welcome Pearson's statement that it is moving away from a shared design between Edexcel syllabus materials and Pearson textbooks, as we agree that this can unhelpfully overstate the link between the two. Pearson should give even-handed treatment to Edexcel Own and endorsed resources from other publishers on the Edexcel website.

QUALITY OF ENDORSED TEXTBOOKS AND THEIR IMPACT ON TEACHING AND LEARNING

160. Many submissions suggested that the quality of endorsed textbooks is variable and that they lead to a narrowing of teaching and learning by focusing on the exam syllabus. One examiner told us "as general texts these books are woeful: as guides to passing the exam they are very useful".[267] Headteacher Martin Collier (himself a textbook author) observed that textbooks "are rushed into production and can be of indifferent quality," while university lecturer Dr Tony Gardner stated that endorsed textbooks "give students little incentive or opportunity to engage with broader and richer material, and fail to foster an appreciation of the subject's subtleties".[268] Examiner authors told us of pressure from publishers to limit coverage to what is in the syllabus.[269]

161. The issues relating to textbooks have led some, such as the Wellcome Trust and several mathematics subject associations, to call for an end to exam board endorsement arrangements, as recommended in the 2010 Walport report.[270] School and college leaders also recommended to us a separation of textbooks from the examining function.[271] Evidence from examiners (many of whom were practising teachers) was more mixed, with some arguing that endorsed textbooks are a useful resource and that endorsement acts as a quality kite-mark.

162. By contrast, Pearson and Cambridge Assessment's Tim Oates drew attention to international research, which demonstrates that systems which have radically improved their performance use high quality resources (often approved textbooks) which are closely aligned to the curriculum and to assessments. Tim Oates warned that in the light of this evidence, "it would be a terrible error, in England, to diminish the linkage between textbooks, curriculum and assessment without ensuring that the form of the linkage is optimized".[272] He acknowledged that "we currently have the wrong sort of linkage between textbooks and examinations" but concludes that this "does not mean that there should no link".[273] In common with other submissions, Tim Oates pointed to School Mathematics Project (SMP) and Nuffield Science curriculum projects, which "were predicated on a very close link between learning materials and examinations", but crucially the linked materials "did not encourage restrictive teaching".[274] ACME told us that the "high quality is due to the books growing out of a vision of a course which is itself the result of a deeply considered curriculum and qualifications development programme".[275]

163. Tim Oates suggested that evidence presented to us highlights "the extent to which narrow instrumentalism has pervaded the whole education system—textbooks and exams have not been immune to this insidious tendency".[276] Mr Oates believes that the "linkage which we now have between textbooks and examinations is most likely a symptom of a deeper structural trend in the system." He argues that "publishers are highly sensitive to market demands. The narrow 'guide to the examination' is produced by them because this is precisely what an accountability-trammelled profession asks for". [277]

164. We agree with Tim Oates and others[278] that the criticisms of endorsed textbooks must be seen in the context of the accountability system and the pressures on teachers and schools to focus on exam preparation, in order to achieve the best possible results in exams. Clearly the situation will vary: some endorsed textbooks will provide enrichment and extension beyond the related syllabus and confident teachers with secure subject knowledge will not over-rely on a textbook, endorsed or otherwise. We believe that exam board endorsement is not necessarily the major factor driving the production of narrow text books.

CONCLUSION

165. Overall the evidence suggests that the current system of exam board endorsed textbooks written by senior examiners is not leading to high quality textbooks that enhance teaching and learning. We suspect that this situation results from a complex interaction between endorsement arrangements, examiner authorship and competition between the exam boards, laced heavily with pressures exerted by the accountability system—a heady cocktail indeed. As a member of the Society of Authors put it:

the specification is written, the textbook written by the examiners meets only the specification, and the teaching is restricted to the textbook. It is a circle that is difficult to break and destroys innovation, creativity and good teaching and learning.[279]

166. We feel that this circle does need to be broken. It is vital that schools are provided with a choice of high quality textbooks, which are not merely "examination guides", and that the system should work to provide this. Few would disagree with Tim Oates that "textbooks should help the delivery of a high quality curriculum." The link to the curriculum is, we believe, crucial. The examples cited by Tim Oates and some learned bodies were of textbooks linked to curriculum as well as assessment. The evidence presented to us suggests that problems have occurred because endorsed textbooks have been linked too heavily to assessment, becoming 'examination guides', instead of extending and enriching young people's curriculum experience: a classic case of the assessment tail wagging the curriculum dog. Amanda Spielman, Chair of Ofqual, described the "symbiotic relationship" between curriculum and assessment.[280] We would suggest that in the case of endorsed textbooks the relationship is no longer symbiotic as these textbooks are working to the advantage of assessment but to the detriment of curriculum.

167. We have considered whether to recommend a restriction on exam board endorsement of textbooks. We believe, however, that such action would not solve the problem, as it is related to a wider issue, namely the pervasive impact of the accountability system on teaching and learning. Under pressure to achieve results, teachers will continue to want textbooks closely linked to assessment in order to prepare their students as effectively as possible for their exams. Publishers will continue to respond to demand and produce such textbooks, with or without exam board endorsements. In order to strengthen the links between textbooks and the curriculum, as well as assessment, we recommend that in future A level textbooks be endorsed by the universities involved in developing a particular syllabus rather than by the exam board. At GCSE much will depend on the outcomes of the National Curriculum review and the ensuing reforms to GCSE, but a possible way forward might involve learned bodies endorsing textbooks instead of exam boards.

Ofqual's regulation of exam board support

168. Our system of multiple exam boards competing for market share may have made the issues concerning textbooks and training more acute. That said, many of the issues would remain, however the exam system were organised. Pressures from the accountability system would continue to exert an influence over the type of training and textbooks that teachers feel they need to prepare their students to succeed. Decisions would still need to be taken on the role examiners should play in training and how much they should be involved in writing textbooks.; and on whether and how exam boards should endorse textbooks and how such books should be marketed to schools. We believe that the answer to much of this lies in robust regulation. It is an area where a stronger Ofqual could and should make a significant impact.

169. Ofqual announced a focus on training and textbooks in late 2011 as part of its work on healthy markets. It has acknowledged that "in the past the market has not been regulated tightly enough".[281] We agree with Ofqual that the market has not been regulated tightly enough with regard to training and textbooks and we believe that this has allowed conflicts of interest to arise. Ofqual's healthy markets work is welcome, if overdue, as it is clear that many of the issues raised with us have gone unchecked for some time. We welcome Ofqual's recent report on exam board seminars and look forward to its publication of an action plan relating to textbooks and study aids in September 2012. Proper regulatory control and scrutiny of these issues will help to increase public confidence in the exam system.

170. There is an underlying assumption behind much of the support offered by exam boards to teachers that they have a role to play in helping schools to improve their performance and to raise standards. Pearson described in its written evidence how it helps teachers to "use data to enhance attainment" through its free ResultsPlus service, thereby "driving systematic improvements to teaching and learning".[282] The Welsh exam board, WJEC, has said that its training "is designed to provide feedback on previous exams and advice to teachers on best practice, with the aim of raising teaching standards and allowing all students to achieve their full potential".[283] We believe that there is a legitimate question about the appropriate role of exam boards and how much this should extend beyond the impartial assessment of attainment. We recommend that Ofqual, as part of its healthy markets work, take a clear view on the broader question about how much exam boards should be involved in helping to improve results as well as in the impartial assessment of attainment.

231 Q460 Back

232 Ev w83, paragraph 41 and Ev w47 paragraph C8 Back

233 Q415 Back

234 Q414 Back

235 Awarding Organisation Seminars for Teachers, Ofqual, December 2011 Back

236 Exam Board Seminars: Final Report, Ofqual, 2012 Back

237 OCR responds to Ofqual announcement on changes to teacher seminars, http://www.ocr.org.uk/news/2012 /item_025.aspx Back

238 AQA responds to Ofqual's announcement on changes to teacher seminars http://web.aqa.org.uk/media-centre/news-270412.php and Pearson's response to the Ofqual report on exam board seminars http://www.edexcel.com/Aboutus/press-room/press-releases/Pages/ViewItem.aspx?filter=2582# Back

239 Exam Board Seminars: Final Report, Ofqual, 2012 Back

240 Q230 Mark Dawe Back

241 Exam Board Seminars: Final Report, Ofqual, 2012 Back

242 Exam Boards Seminars: Final Report, Ofqual, 2012 Back