Education CommitteeFurther written evidence submitted by Cambridge Assessment (Annex A)

The role of high quality textbooks in raising educational standards—how we need to link textbooks to curriculum and to assessment—the evidence from transnational analysis.

Background

Discussions in Select Committee have confronted key issues relating to the link between textbooks and examinations; the quality of textbooks; and the factors influencing quality. This is the right territory, but it is vital for our system that the Select Committee influences the system in a manner which is consistent with what we know from international research.

Preamble

This paper was prompted by my recognition of the importance of making available to the Committee the evidence from the international research which has been undertaken as part of the current review of the National Curriculum.

I am writing the paper from my perspective as Chair of the Expert Panel advising the Secretary of State, on curriculum reform. However, I also lead research activity at Cambridge Assessment, and a natural question is: does that not mean there is an interest in a specific form of relationship between examinations, textbooks and other resources? The answer is simple: the Cambridge Assessment Group contains three awarding bodies, each of which maintains very different sorts of relationship with providers of materials—Cambridge ESOL has a tight linkage with CUP for some of its provision; CIE has very specific commercial relations with outside publishers, but also operates in jurisdictions where there are state-approved textbooks; while OCR commissions from a very wide range of providers depending on the strengths of the provider in meeting specific requirement. This highlights a key issue for this paper: it is not the form of the relationship or the legal relationship between publisher and examination body which determines quality. Quality is dependent on other crucial factors, and it is this which is the main focus of this paper.

The Nature of the Problem—and there is indeed a Problem

The Committee has received and reviewed evidence which suggests that a narrow instrumentalism has crept pervasively into the textbooks which are linked to specific examinations in the English system. Concerns have been expressed that this has tipped into having the effect of restricting and narrowing learning programmes. At worst, the close ties between certain books and examinations constitute a form of “teaching to the test” which undermines the values of fair and accurate assessment. These have been described as “unhealthy” even “incestuous” relations. I believe that this evidence does indeed indicate that something is seriously wrong with current relations.

But international studies make clear that introducing policy which breaks the link between textbooks (or key learning materials of any form) and examinations—and indeed the various producers of these different things—is simply contrary to what is happening in systems which have radically improved their performance. At its most extreme, a reaction against the current forms of linkage in England might be “there should be no link between text books and examinations”. This is patently absurd. Textbooks should help the delivery of a high quality curriculum. Examinations should provide fair and accurate assessment of the outcomes which are attained through the learning to which this curriculum gives rise. The fact that we currently have the wrong sort of linkage between textbooks and examinations—and we are thus unhappy with the current set of texts and examinations—does not mean that there should be no link.

This immediately gives rise to the question “so what sort of linkage should we encourage?”. I believe the history of our system and evidence from other education systems holds the answer. Some of the most important innovations in secondary school programmes of the 1970’s and 1980’s—such as SMP maths, Nuffield science—were predicated on a very close link between learning materials and examinations. The educational integrity of these programmes derived very much from this linkage; the combined materials conveyed with greater precision than previously the desired learning outcomes and the concepts which pupils should develop as a result of teaching. But there was no outcry regarding “narrowness”, or “reductivism”, or “cheating”. No departments complained that they felt “forced” or “blackmailed” into adopting a textbook linked to a specific examination. Far from it, the linked materials and examinations charted a clear course through the concepts and knowledge at the heart of the programme, gave a clear structure around which teachers could design engaging lessons, and did not encourage restrictive teaching.

What has changed since this form of linkage was used to improve maths and science teaching? Comments to the Select Committee highlight the extent to which narrow instrumentalism has pervaded the whole education system –textbooks and exams have not been immune to this insidious tendency. Rather than being the cause of the problem, the linkage which we now have between textbooks and examinations is most likely a symptom of a deeper structural trend in the system. Failure to recognise this may mean that new policy in this area attends to surface features of the textbook-exam relation and at best has no impact; at worst, a inappropriate policy response could reduce our capacity to use the linkage to raise system performance.

Textbook Quality as a Key Instrument of System Improvement

Singapore has lists of state-approved textbooks. So does Hong Kong, Alberta, Massachusetts, Japan and France. Finland, now characterized as a highly autonomous and non-centralist system, used approved textbooks as a key instrument during its period of deliberate and sustained improvement, when it moved from moribund to superlative performance in the 1970s and 1980s. The precise status of “approved” texts and the processes which are used for approval vary, but these and other systems all used the textbook-curriculum-assessment linkage as a key policy instrument in system improvement. I am not arguing that we should move, in England, to a formal process of State approval for textbooks. There are different approaches to achieving the same relations as are used in other high-performing jurisdictions, as I shall outline later. I believe that, on the basis of understanding the role which the linkage has had in these other system, it would be a terrible error, in England, to diminish the linkage between textbooks, curriculum and assessment without ensuring that the FORM of the linkage is optimized. We need high quality resources coupled to the curriculum and to assessments.

“Curriculum coherence is important; it is a key principle of the current National Curriculum Review. The following extracts makes clear the role of the linkage between curriculum content, assessment, materials (including textbooks), teacher and teaching quality, accountability etc (see annex 1 for full list of ‘control factors’. Curriculum coherence results when these factors are be in alignment—the insight derives from compelling work by Schmidt and Prawat on high performing system, investigated through the results of the transnational TIMMS data. High performing systems possess curriculum coherence, moribund systems do not):

… on the concept of control: Schmidt’s work suggests that a level of control must be exercised in a system in order to promote a necessary level of curriculum coherence. Once again, it is vital to recognise that the National Curriculum cannot, by itself, guarantee curriculum coherence in the system. A system is regarded as ‘coherent’ when the national curriculum content, textbooks, teaching content, pedagogy, assessment and drivers and incentives all are aligned and reinforce one another. For this to be the case, a certain level of control is necessary. Crucially, Schmidt and Prawat’s comparative work suggests that this level of control need not necessarily derive from top-down measures. It is more that the system must exercise control, not that individual agencies should take control:

Their analysis suggests that the existence of curriculum coherence through curriculum control is essential; the precise institutional and system form to achieve this can vary. The list of 13 policy control-factors should be interpreted in the light of this. An effective approach to improving education in England will not be associated with slavishly adopting isolated aspects of other systems. Rather, we should be concerned with scrutinising other high-performing systems in order to understand how different aspects of education policy in England can be adjusted to deliver curriculum coherence, using a pattern of control and governance which is both suitable and effective in the English setting. As with the problems with using a single overarching structure for different subjects within the formal statement of the National Curriculum, it may well be that, at the current time, different mechanisms for curriculum control are necessary in different subjects. A greater supply of specialist teachers may be essential in some subjects. Just as Finland’s current success can be traced to highly centralised control in previous decades, including control of textbooks, provision of highly specific learning materials and staff development may be required in certain subject areas and phases—without any necessary pre-judgment about which agencies may be best placed to manage such provision. Critically, if the National Curriculum is to be refined, in order to facilitate high-quality approaches to contextualisation, task design etc, then other measures (across the list of 13 factors) will be required to ensure that such developments are realised in the system—perhaps associated with initial teaching training, or with continuing professional development, or with a new generation of textbooks—or all three of these. Without this, curriculum coherence and entitlement will not be attained….”

Oates T 2010 Could do better—Using international comparisons to refine the National Curriculum in England pp5–16

Policy Response and Committee Recommendations

One policy response which has been suggested by others is to ensure separation of producers of textbooks and producers of examinations. This would fail to produce a desired result, and would encourage dysfunctional “evasion” of restriction—eg by setting up subsidiary companies and complex deal structures. Firstly, it is important to note that some of the problematic linkages have been created by tie-in between separate organizations. Thus, administrative separation does not address the quality problem. Secondly, existing joint bodies could simply “hive off” the different operations, but retain the existing linkages between textbooks and examinations—again, the quality issue would not be addressed.

There are naïve assumptions by different groups as to how and why we have got into the position we are now in. Teachers frequently have said “publishers love to make money out of constantly changing the texts”. But publishers do not share this view. The kind of textbook genuinely liked by publishers is a key textbook which is used by large numbers of people and is in print for thirty years. Likewise, publishers are highly sensitive to market demands. The narrow “guide to the examination” is produced by them because this is precisely what an accountability-trammelled profession asks for (please note that this paper is not arguing for a system in which there is no school accountability, but there are questions about the impact of the current form of arrangements—see Cambridge Assessment 2007). Publishers investigate in detail what teachers are asking for, and then produce exactly the things for which the market is calling. And as for “constant change”—this is due to the constant change in examinations; a pace of change driven principally by changes in State requirement for examinations (Curriculum 2000, modularized GCSE, controlled assessment, Diplomas, etc).

In Conclusion

In working through the potential policy associated with raising the quality of textbooks the following has emerged in discussions with researchers and publishers:

1.textbooks are of depressed quality due to narrow instrumentalism across the education system—emanating principally from the behaviours stimulated by high stakes accountability arrangements;

2.publishers are very aware of the narrow instrumentalism of many materials—but they are very efficient at providing the kinds of materials for which teachers are asking;

3.publishers do NOT seek to constantly change textbooks nor do they profit from this. Constant change in state regulation regarding the form and content of curriculum and qualifications is considered, by publishers and researchers, to be a principal threat to quality;

4.researchers and publishers recognise the importance of the international evidence on “curriculum coherence” and the latter actively want to increase the quality of textbooks and materials . There are examples, from the recent past, of very high quality materials which linked closely to examinations—SMP maths, Nuffield science—but we have moved away from this;

5.very high quality materials exist in high performing jurisdictions—eg maths text books in Singapore and Hong Kong—and these are “close coupled” to the curriculum and to examinations—but lack the instrumentalism present in narrowly-framed UK materials. We should use these as reference points for development of high quality materials in our own system;

6.teaching beyond the syllabus in respect of qualifications enhances examination grades. Textbooks which support this further exploration of, and reflection on, subjects should be encouraged; and

7.many high performing jurisdictions use “approved” lists and State approval of textbooks and materials to ensure curriculum coherence. This is not currently a structural feature of the English system, and approval systems are not free of complications and problems. Commercial restriction should be approached with caution. But approaches such as this should be considered and discussed, as should “kite-mark” and lighter-touch approval systems. It is important to note that approval of a likely future “tidal wave” of electronic materials would present any formal system of approval with huge, if not insurmountable, challenge. In the face of these realities, a culture of high quality—using the highest quality domestic and international materials as benchmarks—would be far more effective in ensuring that poor quality materials (including internet-based materials) are not used, and are given short shrift by education professionals.

In line with the analysis in this paper, I would contend that the key issues are: recognizing the importance of textbooks in improving the performance of the education system; and securing the right relation between textbooks and learning—rather than diminishing the relationship.

This last point is crucial. Improving the relationship almost certainly involves tightening the linkage between textbooks and the aims and content of the curriculum and qualifications rather than artificially separating them. Attempting to administratively or constitutionally separate the design, production and operation of textbooks from the aims and content of curriculum and qualification (eg by banning certain organizations from producing textbooks) would only work if it went hand in with an effort to ensure greater curriculum coherence—ie greater linkage of higher quality—in the form and content of textbooks, curriculum and qualifications. And this therefore feels like policy pulling in two different directions at the same time. Far better to concentrate on the need for linkage and the quality of the materials.

Tim Oates
Cambridge

March 2012

References

Cambridge Assessment 2007 Alternative approaches to National Assessment at KS1, KS2 and KS3

Oates T 2010 Could do better—Using international comparisons to refine the National Curriculum in England pp5–16

Annex 1

CONTROL FACTORS

A critical approach to transnational analysis suggests that we should use international comparisons to understand how different aspects of the system are subject to control and development, rather than engage in crude “policy borrowing”. These “control factors” exist in complex relations and balances:

1.curriculum content (national curriculum specifications, textbooks, support materials, etc);

2.assessment and qualifications;

3.national framework—system shape (eg routes, classes of qualifications);

4.inspection;

5.pedagogy;

6.professional development (levels and nature of teacher expertise);

7.institutional development;

8.institutional forms and structures (eg size of schools, education phases);

9.allied social measures (such as that which links social care, health care and education);

10.funding;

11.governance (autonomy versus direct control);

12.accountability arrangements; and

13.selection and gatekeeping (eg university admissions requirements).

These are very useful categories for looking at other nations’ policy arrangements. Studying the relation between them in different countries allows us to understand the operation of our own system. It is important to understand that, despite comparatively low rates of pay (OECD 2010) Finnish teachers enjoy high social status, and all have a high level of formal qualification (to Masters level).The importance of teaching quality, approaches to learning and task design is strongly reinforced in the work of Hattie (Hattie J 2003), Wiliam (Black P & Wiliam D op cit), Watson (Watson A undated; Watson A & Ollerton M 2005) Andrews (Andrews P 2007; Andrews P 2010) and Stigler & Stevenson (Stigler & Stevenson op cit). This is an important factor in national success, amongst others. A country’s national curriculum—both its form and content—cannot be considered in isolation from the state of development of these vital factors. They interact. Adjust one without considering development of the others, and the system may be in line for trouble (Green A 1997). Of equal importance, transnational analyses can provide evidence-based design principles which were absent from the 2007 revisions to the National Curriculum. Key amongst such work is Schmidt and Prawat’s analysis of “curriculum coherence” (Schmidt W & Prawat R 2006). This is strongly grounded in evidence from TIMSS, and argues that “curriculum coherence” is vital, and is associated with high performing systems. This is not just a trivial, common-language use of the term “coherence”. A system is regarded as “coherent” when the national curriculum content, textbooks, teaching content, pedagogy, assessment and drivers and incentives all are aligned and reinforce one another. “…Curricular materials in high-performing nations focus on fewer topics, but also communicate the expectation that those topics will be taught in a deeper, more profound way...” (Schmidt W & Prawat R 2006 p1). Their analysis of mathematics emphasises that “curriculum coherence” should also be demonstrated through arranging concepts in an appropriate age-related hierarchy. Their more extended analysis of the nature of national control suggests that there is no rigid association between a system possessing curriculum coherence and being subject to tight, “top down” control, nor it being devolved—the group of countries which exhibit curriculum coherence includes examples of both.

April 2012