Select Committee on Science and Technology Written Evidence


Memorandum from William Solesbury, Senior Visiting Research Fellow, ESRC UK Centre for Evidence Based Policy and Practice, Kings College London


  1.  This memorandum draws on the work of the ESRC UK Centre for Evidence Based Policy and Practice at Kings College London as it relates to the questions being addressed by the Committee's inquiry. The Centre's remit is to analyse the diverse ways in which evidence—including scientific, social scientific and other research, enquiry and debate—is used in public policy. To that end it has undertaken, over the last five years, a programme of research, training, consultancy and resource provision (in part funded by the Economic and Social Research Council—

    —  Research projects have been concerned with the public debate about GM crops and foods; the nature and uses of evidence in the audit, inspection and scrutiny functions of government; the conduct of research reviews; the development of a new "realist" approach to the synthesis of evidence; the nature and quality of knowledge within social care; the assessment of research quality; the contribution to the work of government departments of officials and board members from outside the civil service; and the role of strategic thinking in government.

    —  Training has been provided for clients including the National School of Government, the National Audit Office, and the Social Care Institute of Excellence and for researchers and postgraduate students supported by the Economic and Social Research Council.

    —  Consultancy clients have included the Parliamentary Office of Science and Technology, the Learning and Skills Development Agency, and the National Centre for Social Research.

    —  The Centre functions as a resource centre through its website which includes inter alia a guide to many databases of published research and other resources for evidence based policy and practice and an extensive searchable bibliography of publication on evidence based policy.

    —  The Centre also edits the journal Evidence and Policy, published by Policy Press.

  More details of most of this work can be found on the Centre's website: This memorandum now addresses the questions in the Committee's inquiry brief.


  2.  Government departments secure scientific (including social scientific) advice through a number of modes—

        Through standing or ad hoc scientific advisory bodies.

        Through commissioning new research externally from academic or other centres.

        Through searching existing published research.

  There has been a notable increase in departmental research budgets for commissioned research over the last decade. There has also been a growing interest in the practice of "systematic reviews", that is, searching published research in a rigorous way and seeking to establish the present state of scientific knowledge relevant to a particular policy question. Some believe that this has the potential to offer more comprehensive, speedier and cost-effective evidence than the other two modes. However, our research on systematic reviewing[105] and our experience in contributing to reviews suggests that the practice of commissioning, undertaking and using such reviews often underestimates the time, money and skills needed for such reviews. It also shows that policy makers can find the results difficult to interpret and apply.

  3.  All these ways of securing scientific advice depend on an in-house capability to handle it—identifying when science can contribute to policy, seeking it out from a wide range of sources and interpreting its relevance to policy. This should be a prime role of departmental scientific and social research staffs who can act as "brokers" between science and policy. But our training and consultancy experience suggests that this role is underdeveloped in departments. It is also important that mainstream civil servants develop an understanding of what science can contribute, and crucially when that contribution is needed. The recent work on Professional Skills for Government has recognised this in the new competency frameworks that have been developed.[106] Among the core skills for staff at Grade 7 and above are Analysis and Use of Evidence (the others are Financial Management, People Management and Programme and Project Management). This is laudable, but insufficient. Skills alone are not the problem. Attitudes need to change too. Our research on the experiences of outsiders recruited into Whitehall, mostly in quite senior positions, revealed that many of them struggled to gain recognition from their insider colleagues of the expertise they brought with them; and some left quite quickly.[107]


  4.  Although the term "evidence based policy" has gained currency in recent years (and is reflected in the title given to our Centre by the ESRC in 2000), our experience suggests that it misrepresents the relationships between evidence and policy. "Evidence informed policy" is nearer the reality. Our current research on the role of strategy in government[108] reveals the continuing interplay between "facts" and "values" that characterises policy activities. In our training work we find that the "Four Is" framework (originated by the US evaluator Professor Carol Weiss)[109] helps practitioners to place evidence appropriately. It states that policy is the outcome of the interaction of Ideologies, Interests and Information (that is, evidence) within the context of Institutions where:

    —  Information is "the range of knowledge and ideas that help people make sense of the current state of affairs, why things happen as they do, and which new initiatives will help or hinder";

    —  Interests are essentially "self-interests";

    —  Ideologies are "philosophies, principles, values and political orientation";

    —  Institutions are doubly important: "first the institutional environment shapes the way in which participants interpret their own interests, ideologies, and information. [ . . . ] Second, organisational arrangements affect the decision process itself, such as who is empowered to make decisions."

  This framework also proved fruitful in our analysis of policy making on GM foods.[110] We found that, because the Four Is are in dynamic interaction creating mutual influences that repeatedly interweave, decisions and outcomes become progressively reshaped. This requires stakeholders to reposition themselves in relation to each other, the evidence, the decision process and the policy and practice outcomes.

  5.  Government must be concerned about the quality of scientific evidence for policy work. Its commitment to quality assurance is one of the distinct strengths of scientific research as a source of evidence for policy. But for this purpose it is important that methodological considerations of validity and reliability are complemented by utilitarian considerations of relevance and accessibility. For example, our work on the practice of research reviewing has revealed a far too casual approach to the use of evidence (particularly social science findings) from other countries without adequate regard to contextual differences. More generally, from our work we would argue for the concept of "fitness for purpose" as the appropriate measure of quality for scientific evidence for policy, meaning that the science should be methodologically good enough for the weight to be attached to it in informing policy.[111], [112]

  6.  Our training and consultancy work has revealed tensions between scientists and government about the interpretation and application of research findings. The details of each case differ. But overall there is a need to pay adequate prices for commissioned research, to allow researchers adequate time to undertake the work rigorously, and to expect them to report "as they find". If government decides—for whatever reasons—to act in ways that contradict or are unsupported by scientific advice, then it should do so without seeking to misrepresent the evidence. We note with interest the proposals from HM Treasury to safeguard the objectivity and impartiality of National Statistics.[113]


  7.  Our research on strategy in politics confirms the observation that policies often fail for one of two reasons: unforeseen changes in context or unintended consequences of the policy itself. Effective policy work requires an initial assessment of the risks of both as part of policy development and constant alertness to either occurring through policy monitoring. This suggests to us that assuming research on "what works" will deliver "evidence based policy" is dangerously naive. Research is needed not just to show "what works" but also why policies work (or not), and what we understand of current phenomena and likely future trends in shaping policies and outcomes. Horizon scanning research can help the latter, but such research must be concerned as much with societal as technological trends.


  8.  Most government departments have a commitment to publish the research they commission—in print and/or electronically. That degree of openness does not apply to the other two modes in paragraph 2 above (scientific advice, and internal reviews of existing research). What is not, understandably, transparent is how scientific research has been interpreted and applied in policy work. In these circumstances it is all too frequently the case that suspicions arise about the misuse of scientific evidence, maybe fuelled by the media. Only openness by government about how it has weighed such evidence (the Information I in the "Four Is" framework in paragraph 4 above) against other considerations will allay such suspicions. For example, for the decision on GM crops, the specially constructed evidence base our research examined was put into the public domain. It was not all exactly fit for the government's purpose. The government had a priori decided to give "science" pre-eminence and greater weight over other types of evidence. In the event the science was interpreted to give the government insufficient grounds for supporting immediate commercialisation of the three test crops. On the wider decision about how to sustain competing choices (GM/conventional/organic) in the face of competing interests and ideologies, the special evidence base was mostly of little use. It arrived too late to add anything crucial, and well after stakeholders' views had already shifted strategically.[114]

  9.  Part of our training and consultancy portfolio concerns the communication of research to lay audiences—policy makers, parliamentarians, professionals, and the public. Scientists are generally not very good at this and traditionally not interested in it. (The POSTnotes produced by the Parliamentary Office of Science and Technology are an important and deliberate attempt to redress this.) Our experience suggests that this is largely a consequence of inadequate skills training (not just in writing accessibly but in making presentations, running interactive workshop and seminars, and exploiting the potential of the Web) and of incentives that favour communicating research to fellow scientists more than to practitioners (notably the use of research publications as performance measures in the Research Assessment Exercise). Increasing emphasis is being placed on dissemination and "knowledge transfer" by the Research Councils, the Wellcome Trust, the Royal Society and others, but there is a long way to go.


  10.  From our research and consultancy work on the relationship between evidence and policy, we consider that the need is not so much (as the Inquiry brief suggests) "to re-evaluate the evidence base after the implementation of policy." Rather the need is to maintain, for all policy fields, a continuous updating of the evidence base as new scientific research—commissioned by government or by others—yields results that can inform policy development and delivery in a timely way. In our experience most policy needs for scientific evidence could be met—at a standard of reliability that is "fit for purpose"—from knowledge that already exists and is available. Managing those stocks of policy-relevant knowledge—keeping them objective and impartial, up-to-date, accessible—is the big scientific challenge for more evidence-informed policy.

May 2006

105   Annette Boaz, William Solesbury and Fay Sullivan (2004): The practice of research reviewing 1. An assessment of 28 review reports ESRC UK Centre for Evidence Based Policy and Practice, Working Paper No 22; available via; also Annette Boaz, William Solesbury and Fay Sullivan (forthcoming): The practice of research reviewing 2. 10 case studies of reviews, ESRC UK Centre for Evidence Based Policy and Practice, Working Paper No 24. Back

106   See Back

107   Ruth Levitt and William Solesbury (2005), Evidence-informed policy: what difference do outsiders make in Whitehall, , ESRC UK Centre for Evidence Based Policy and Practice, Working Paper No 23; available via Back

108   This is part of an international comparative study of Strategy and Politics funded by the German Bertelsmann Foundation; it will report later in 2006. Back

109   Carol Weiss (1995), "The four `I's' of school reform: how interests, ideology, information and institution affect teachers and principals", Harvard Educational Review 65(4) pp 571-592. Back

110   Ruth Levitt (2003): GM Crops and Foods: Evidence, Policy and Practice in the UK; a case study, ESRC UK Centre for Evidence Based Policy and Practice, Working Paper No 20; available via Back

111   Annette Boaz and Deborah Ashby (2003), Fit for purpose? Assessing research quality for evidence based policy and practice, Working Paper 11, ESRC UK Centre for Evidence Based Policy and Practice, Working Paper No 22; available via Back

112   This argument is similar to that used by the National Audit Office in its VFM work where it adopts the criteria that evidence should be relevant, reliable and sufficient. See National Audit Office, Value for Money Handbook-a guide for building quality into VFM examinations, page 16. Back

113   H M Treasury (2006), Independence for Statistics: a consultation document, available via Back

114   The then Secretary of State, Margaret Beckett, said "The UK is neither pro nor anti GM. We are pro consumer safety and choice and pro protection of the environment. [ . . . ] The challenge for any government is to regulate the use of this new technology in a way that safeguards the public and our planet, commands public confidence, but also ensures that our society does not unnecessarily throw away the benefits science can provide. This is no easy task . . . " DEFRA News Release 174/03, 15 May 2003; via Back

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2006
Prepared 8 November 2006