House of Lords - Behaviour Change - Science and Technology Committee

Behaviour Change - Science and Technology Committee Contents

CHAPTER 4: EVIDENCE-BASED POLICY

Are Government policies evidence-based?

4.1. Behaviour change interventions based on evidence about what works are more likely to be successful than those which are not. We have concluded that there is marked lack of research at a population level (see paragraph 3.11 above). This leads us to two further conclusions: first, the Government should take steps to ensure that this sort of research is undertaken; and, secondly, policies should, insofar as is feasible, reflect the evidence that is, or becomes, available.

EXAMPLES OF EVIDENCE-BASED POLICIES

4.2. We were given a number of examples of policies which were, to a greater or lesser extent, designed to reflect the available evidence. These included smoking cessation services (see Box 1, page 20), the Health Trainers Intervention (see Box 15, page 49) and energy efficient products policy (see Box 6, page 27).[48] A common feature of these examples is that they were all developed by, or in consultation with, academics with expertise in changing behaviour.

BOX 1

NHS Centre for Smoking Cessation and Training

The NHS Centre for Smoking Cessation and Training (NCSCT) was set up after implementation of Stop Smoking Services (SSSs) across the United Kingdom. The Centre was established in 2009 "to assess training needs, develop training standards, pilot and evaluate training programmes, develop a certification system for smoking cessation practitioners, deliver the training across England and continuously evaluate it, develop an accreditation system for trainers and courses and contribute to national policy development".[49] Professor Michie noted that the team at the NCSCT undertook systematic reviews and looked at the Cochrane evidence reviews, alongside analysis of DH data on smoking cessation. That evidence was then used to form the basis of outcome measures and interventions which have been put into practice. Assessment and training have been continually monitored and revised in order to learn from experience and to take account of scientific advances, new evidence and contextual changes. The training is evaluated by its impact on stop smoking success rates, using comparisons of success rates of practitioners against controls. Participants' feedback on the training as well as their self-reported confidence in their competences are also assessed and used to evaluate training.[50]

EXAMPLES OF POLICIES WHICH WERE NOT EVIDENCE-BASED

4.3. We were also given examples of (previous and current) Government policies which were not based on evidence. Two of these, minimum alcohol pricing and the Act on CO₂ campaign, are described below (see Boxes 2 and 3, page 21). The case studies provided further examples: witnesses observed that those commissioning weight management interventions at a local level were often insufficiently knowledgeable to make evidence-based decisions (see paragraph 7.24 below), and the DfT policies in relation to sustainable transport were said not to reflect the evidence about the effectiveness of disincentives to car use (see paragraph 7.36 below).

BOX 2

Act on CO₂

'Act on CO₂' was a cross-Government brand launched in 2007 with the aim of getting people to reduce their carbon footprint. It included a range of communications activities relating to home energy usage, smarter driving and car purchasing. The Sustainable Development Commission noted that the campaign 'Act on CO₂' had been criticised for "failing to communicate effectively with the public, for being too negative in its messages, and for not including any supporting interventions to address the barriers to adopting low carbon behaviours".[51] The campaign is an example of how policy was not based on the available evidence because:

It involved only the provision of information. The Green Alliance note that it is now widely known "that information deficit models in practice rarely work: information alone is insufficient to lead to action".[52]
It did not include a range of interventions within a multi-component package to tackle a number of causes of behaviour and barriers to change.

BOX 3

Alcohol pricing

Professor Kelly and Professor Michie told us in November 2010 that there was good evidence about the effectiveness of alcohol pricing on reducing alcohol related harm but that it had not fed through to Government alcohol policy. Subsequently, in January 2011, the Home Office announced a ban on the sale of alcohol below the rate of duty plus VAT.[53]

This policy has been criticised however for not reflecting the evidence about the level at which pricing affects behaviour. Requiring alcohol to be sold for no less than the rate of duty plus VAT means that minimum price for a unit of beer would be around 21p and for spirits around 28p. The NICE guidance on preventing harmful drinking published in 2010 shows that at the minimum price level proposed by the Government, a reduction in consumption of between 0.1% and 0.4% could be expected. However, a minimum price of 40p per unit would reduce consumption by 2.4%, while minimum prices of 50p and 60p would reduce consumption by 6.7% and 11.9% respectively.[54]

REASONS WHY POLICIES MAY NOT BE EVIDENCE-BASED

4.4. There are two reasons, in addition to a lack of applied research at a population level (see paragraphs 4.9-4.16 below), why policies are not always based on the best available evidence: ministers are unaware of relevant evidence, or they are aware of the evidence but choose not to reflect it in policy decisions.

4.5. Where ministers are unaware of relevant evidence, this is a failure of the process by which the Government are informed about research findings—a process often described as the translation of research.[55] We were given a number of reasons for the breakdown of this process: a lack of significant involvement of Government social scientists and economists; an absence of adequate mechanisms for communication between policy makers and external researchers; an inadequate understanding of behavioural research by policy makers; and an absence of adequate mechanisms for sharing knowledge within Government.[56] We consider how to address these problems in paragraphs 4.17 to 4.42 below.

4.6. Even where ministers are aware of relevant evidence, other factors may lead them to disregard it. This appears to have been the case with current alcohol pricing policy (see Box 3, page 21). Norman Baker MP, a Minister at the DfT, explained: "evidence is best used to inform policy … but not to drive it in an unreconstituted way"; the Government "have to make choices based not just on the evidence-base … but also on the political objectives of the Government at a particular time, and to ensure fairness across the country".[57] Other considerations might include immediate reaction to events, judgements about ethical acceptability, cost and cost-effectiveness. These considerations might justifiably affect the extent to which a policy is based on the available evidence.

4.7. We acknowledge that there will be occasions when it is legitimate for a government not to implement behaviour change interventions for which there is good evidence of effectiveness. In these circumstances, however, we believe that ministers have a responsibility to explain why they have decided not to do so.

4.8. We agree with the principle, stated in the Government's Principles of Scientific Advice, that ministers should explain publicly their reasons for policy decisions, particularly when a decision is not consistent with scientific advice and, in doing so, should accurately represent the evidence. This places a responsibility on scientists and social scientists within government to ensure that ministers are provided with accurate and up-to-date advice on the available evidence about how to change behaviour so that they can identify where and why they are not basing policies on that evidence.

Addressing the barriers to evidence-based policy

APPLIED RESEARCH AT A POPULATION LEVEL

4.9. In paragraph 3.8, we conclude that there is a lack of applied research at a population level. Mr Bartholomew of the GSR told us about the "frustration" of policy makers at the fact that, although there is very good academic research, researchers often do not take the final step and answer the question "what would you do about it?"[58] We were provided with a number of reasons why this sort of research is lacking.

POOR EVALUATION OF GOVERNMENT BEHAVIOUR CHANGE INTERVENTIONS

4.10. It is clear that if the Government's attempts to change behaviour at a population level were rigorously evaluated, this would provide evidence about effective interventions. Evaluation is discussed in Chapter 7.

FUNDING AND RESEARCH CAPACITY

4.11. There was disagreement in the evidence we received about whether, on the one hand, there is research capacity to conduct research at a population level but insufficient funding available to support it or whether, on the other hand, there is simply insufficient research capacity, so that further funding would make little difference. The majority of the evidence we received on this issue related to public health research.

4.12. The BPS suggested that the problem was one of funding: there would be more research at a population level if there were more "funding [of] evidence-based translational research to develop cost-effective strategies that can be adopted and utilised in practice".[59] Many other witnesses agreed. NICE argued, for example, that "the UK's capacity for this kind of research is good with much potential" (albeit "disparate and often highly individualistic") but "there is not enough funding available for behaviour change evaluation ..."[60] Professor Karen Lucas, Department for Transport Studies at the University of Oxford, agreed, stating: "... there is sufficient research expertise, but insufficient research funding and not enough interdisciplinary interaction on this subject ..."[61]

4.13. In relation to funding, Dr Halpern and Professor Michie both highlighted the findings of a 2006 report by the UK Clinical Research Collaboration which estimated that 2.5% of health research funding is spend on prevention and just 0.5% on primary behavioural factors, despite the fact that understanding how to change behaviour is of significant potential benefit.[62] DH responded: "research with relevance to behavioural factors is supported through most of our funding streams" but "spend on this cannot be disaggregated from total spend across the portfolio".[63]

4.14. Professor Dame Sally Davies, Director General of Research and Development and Chief Scientific Adviser for DH and the NHS, suggested that the block to behavioral research was not a lack of funding but rather a lack of research capacity. Other witnesses agreed that research capacity was the more important issue.[64] Professor Marteau and Dr Haynes, for example, noted in relation to health research that:

Professor Erik Millstone, Professor in Science and Technology Policy at the University of Sussex, said, more broadly, that "... there is very little capacity in the UK to conduct research that is of practical relevance".[66] Dr Tim Chatterton, University of the West of England, agreed that there was scope to build greater capacity in academia for this kind of work.[67]

4.15. We were, nonetheless, provided with a number of examples of how the Government are providing funding for the development of research at a population level in collaboration with the research councils.[68] We set out two of them below (see Boxes 4 and 5, page 24).

BOX 4

The Sustainable Behaviours Research Groups: Defra and the ESRC

The Sustainable Behaviours Research Groups were begun and funded by Defra, the Economic and Social Research Council (ESRC) and the Scottish Government in order to enhance the evidence-base in this field and specifically to develop research in a form that could be used by policy makers. The groups are researching issues including the rebound effects of behavioural changes, the role of routine and habit, and circumstances which facilitate or constrain sustainable behaviour. Both research groups are involved in evidence synthesis and investigating issues relating to the use of evidence by policy officials.

An advisory group has been established, made up of the funders, leading academics from the sustainability field and two independent members (one representing businesses and the other the third sector). This group provides a challenge function from the different fields of expertise and will optimise investments by broadening the reach of findings. Defra has also undertaken a policy timeline mapping exercise so that findings feed into policy objectives.

BOX 5

National Prevention Research Initiative (NPRI): DH and the MRC

DH Policy Research Programme spent £34 million in 2009-10. As part of this the department provides funding to the NPRI alongside the devolved governments and third sector organisations. The aim of the NPRI is to develop and implement successful, cost-effective interventions that reduce people's risk of developing major diseases by influencing their health behaviours. The NPRI has so far committed £33 million to research projects looking at the use of alcohol and tobacco, and diet and physical activity. The most recent calls for research proposal focus funding on cross-disciplinary research that has a large potential influence on population health.

In February 2011, Anne Milton MP, Minister for Public Health, established an National Institute for Health Research School for Public Health Research in order to increase the evidence-base for effective public health practice. DH has also established a Policy Research Unit on Behaviour and Health which will focus on behaviour such as diet and physical activity.

CONCLUSION

4.16. Whilst we welcome efforts by some departments to work collaboratively with academics to develop behavioural research, further capacity to conduct research at a population level needs to be developed. Funding for research of this kind is a necessary in order to build this capacity. The long-term evaluation of interventions using population-representative samples will be expensive and it will be necessary for funding to be made available before this work can be carried out. We urge ministers to consult their departmental Chief Scientific Advisers (CSAs) about whether the amount of money spent on applied behaviour change research at a population level is sufficient to meet their policy needs.

Translation of research

ROLE OF GOVERNMENT SCIENTISTS

4.17. Government scientists have an important role in ensuring that academic research is used to inform policy decisions. Professor Kelly noted the need for "a specialist way of making that link"[69] between researchers and policy makers, in order to bring the two very different cultures together. This should be core work for Government scientists.

4.18. A cross-departmental social science resource is provided by the Government Economic Service (GES) and the Government Social Research service (GSR) which are responsible for giving "evidence-based advice to support the rationale, objectives, appraisal, monitoring, evaluation and feedback to support effective policy making and delivery".[70] Government economists and social scientists are civil servants who work within particular Government departments in order to ensure that policies formulated within their department are "guided by the best available analysis and evidence".[71] Mr Bartholomew, joint head of the GSR, described his role within his department as ensuring that policy makers were made aware of the most up-to-date scientific findings, including those about behaviour change, in order to enable effective evidence-based policy.[72] The Government Social Research Unit and Government Economic Service Team in HM Treasury (now the combined Government Economic and Social Research Team) provide the professional support and leadership for social researchers and economists across all government departments.[73]

THE GOVERNMENT CHIEF SOCIAL SCIENTIST

4.19. The Government's Chief Social Scientist (CSS), the head of the social research profession in government, has in the past been an independent expert in the social sciences. The post has, however, most recently been filled by two government social scientists from the Department of Education and the Department for Work and Pensions.[74] Professor Sir John Beddington, Chief Government Scientific Adviser (GCSA), confirmed that the CSS is a job currently "divided between two civil servants".[75] The CSS sits on the Heads of Analysis (HoA) group alongside the heads of the other analytical professions, including the GCSA in his capacity as Head of Science and Engineering Profession, and heads of profession for economics, statistics and operational research. The role of the HoA group is to "provide leadership to all analysts in government and champion first rate analysis across government".[76]

4.20. The majority of government departments also have a departmental CSA who works within their department to "ensure that science and engineering are at the core of decisions within departments and across government".[77] The network of departmental CSAs works closely with the GCSA through the Chief Scientific Advisers Committee, one of the functions of which is to facilitate communication on high profile science issues and those posing new challenges for government.[78] We note however that departmental CSAs belong to a different profession from that of the social scientists working within their departments and it is not clear what responsibility CSAs have for the performance and development of social scientists. Sir John Beddington confirmed, when giving evidence to this Committee on science spending in May 2011, that the Government "do not have anybody as a Chief Scientific Adviser who has a background in social research at the moment".[79]

4.21. Professor Marteau was critical of the GES and GSR and suggested they were not doing enough to drive forward an agenda of evidence-based policy making.[80] Academics from the Faculty of Humanities and Social Sciences at the University of Bath also noted that "the profile, status and consequent influence of Government Social Research staff in informing the policy strategy and delivery ... varies across different departments".[81] We were also concerned to hear that current capacity within Government with regard to behaviour change expertise was "variable" and that though a few departments have some notable expertise, others have less or none at all.[82]

4.22. The limited reference to the GES and GSR in the evidence we received suggests to us that they are not as effective as they might be. We also note that evidence submitted to us by the GES and GSR provides little detail about how they intend to promote evidence-based policy.

4.23. Government scientists and social scientists have an important role to play in facilitating the translation of behaviour change research and in remedying the problems with translation which we have identified. We recommend therefore that, at the earliest opportunity, the Government appoint a Chief Social Scientist (CSS) who reports to the GCSA and is an independent expert in social science research to ensure the provision of robust and independent social scientific advice.

4.24. We further recommend that the Government consider whether existing mechanisms for the provision of social scientific advice, in particular advice on behavioural science, are fit for purpose. This should include consideration of how departmental CSAs and social scientists within departments can best work together to provide up to date social scientific advice to support evidence-based behaviour change interventions.

BETTER LINKS BETWEEN THE ACADEMIC AND POLICY MAKING COMMUNITIES

4.25. Many witnesses highlighted the need for closer working between behaviour change researchers and policy makers to ensure that policies are properly informed by relevant evidence. Some suggested that, given the complexity of the area, researchers should be involved in intervention design from the beginning.[83] As we have already observed, the examples of interventions which were properly evidence-based were notable for their use of external expertise. The BPS were brought into DH to design the Health Trainers programme (see Box 15, page 49), and Defra's energy efficient products policy was developed in collaboration with a broad range of experts (see Box 6, page 27).

BOX 6

Energy efficient products

Defra's energy efficient products policy was developed by a team of behavioural economists, social researchers and communications experts.[84] The policy engaged the manufacturers and retailers to change the context in which the products were sold, using evidence about the importance of the environment in influencing people's decisions. Drawing on research into consumer purchasing patterns, interventions were also designed to raise the salience of energy efficiency at the point of decision making through labelling and communications campaigns. Between 1996 and 2007 the percentage of fridges and freezers purchased by consumers which were A-rated for efficiency increased from 5% to over 70%.[85]

The Sustainable Development Commission described the policy on energy efficient products as evidence-based and, as a result, multi-faceted, making use of a number of different types of interventions, including communication and information provision alongside requirements on industry.[86]

4.26. Other efforts to involve behavioural scientists in the development of policy are also being made. We were told that one of the functions of BIT was to bring in external experts on an ad hoc basis and invite the relevant departments to meet them.[87] The departments also provided some examples. DH had developed an arrangement with BPS for the provision of health psychologists to advise on evidence about behaviour change.[88] Though that arrangement has come to an end, Dr Sunjai Gupta, Head of Public Health Strategy and Social Marketing at DH, said that his team continued to include a health psychologist.

4.27. Gemma Harper, Chief Social Scientist at Defra, told us that her department work closely with their departmental scientific advisory council and expert committees. They were also working with the Department of Energy and Climate Change (DECC) on proposals for a social science orientated expert committee in order to ensure that they had the best advice, external to government, on behaviour change.[89] Liz Owen, Head of Customer Insight at DECC, said that, in relation to development of energy efficiency policy, her department was engaging with external experts. She admitted, however, that this was an area in which DECC had "more to do".[90]

4.28. Links are also being made with the research community through internships and research placements. Dr Rachel McCloy, a psychologist from the University of Reading, was an ESRC-funded Public Sector Research Fellow based in HM Treasury. She has been involved in developing a Behavioural Science in Government Network and compiling a database on work across Government (see paragraph 4.38 below).[91] Dr McCloy told us that arrangements such as her fellowship were "very useful in bringing academics in" so that they could see "what it's like on the other side of the table".[92] Dr Chatterton, an ESRC-funded placement fellowship at DECC,[93] told us that placements were "a great way forward". His placement had "opened [his] eyes to how big the gulfs are between the world of government policy making and the world of academia".[94]

4.29. Though many witnesses supported these mechanisms for linking behavioural scientists and policy makers more closely, some suggested that more needed to be done.[95] The Green Alliance, for example, said that placements and ad hoc consultation would not be enough for "government to keep on top of the wealth of academic progress, and to ensure the latest research is impacting on decision-making" but that a "greater dedicated resource" was required.[96]

4.30. Departmental CSAs, whether or not they have experience of the sciences of human behaviour, should be responsible for establishing and maintaining contacts with leading behavioural scientists with expertise relevant to their policy areas and for consulting them as necessary.

BEHAVIOURAL INSIGHTS FOR POLICY MAKERS

4.31. The final link in the translation of research is that between scientists and policy makers. A number of witnesses suggested that policy makers themselves should become more familiar with behavioural insights and their potential importance for improving policy.[97] Dr Halpern said that a mark of success for BIT would be widespread expertise in behavioural approaches within five years and that BIT was working with the head of profession for policy making to embed insights about behaviour change across Whitehall. We received, however, no indication of how this would be achieved.

4.32. We recommend that the Cabinet Secretary, in consultation with the GCSA and CSS, once appointed, should take steps to ensure that civil servants with responsibility for policy making have the necessary understanding of the importance of changing behaviour and can identify the most appropriate people to consult in their own departments about the development of behaviour change interventions.

GUIDANCE TO POLICY MAKERS

4.33. There are a number of resources available to help policy makers understand behaviour change and design policies which take on board behavioural insights. These include the MINDSPACE report, produced by the Cabinet Office and the Institute for Government;[98] a review by the GSR which discusses models of behaviour change and provides a framework for designing interventions based on their models;[99] a report by the Central Office of Information which also summarises models of behaviour change;[100] NICE public health guidance on behaviour change at population, community and individual levels;[101] and Defra's framework for pro-environmental behaviours, the four Es[102] (adapted by the MINDSPACE report into the 6 Es).[103]

4.34. Despite this wealth of material, witnesses observed that none of the guidance provided an accessible, multi-disciplinary framework for designing behaviour change interventions. Furthermore, the sheer quantity of guidance, none of which covered everything and much of which was too detailed, was potentially confusing and unhelpful.[104]

4.35. We recommend that the Cabinet Office, in consultation with the CSS, once appointed, consider how to consolidate the available guidance in a form which is evidence-based and accessible to policy makers.

4.36. We further recommend that NICE updates its 2007 Behaviour Change Guidance and considers whether accessible, multi-disciplinary guidance could be provided in relation to health-related behaviour change policies, particularly to offer more explicit advice on how behaviour change techniques could be applied to reduce obesity, alcohol abuse and smoking.

SHARING KNOWLEDGE ACROSS GOVERNMENT

4.37. Several witnesses identified a need for better coordination of what is known about behaviour change within and across government.[105] GES and GSR, for example, said that there should be:

more shared practice in terms of what works and what does not work in influencing behaviour; and
mechanisms for bringing people working in this area across government together to promote good practice as well as cost-effectiveness.[106]

4.38. The primary mechanism for achieving better co-ordination appears to have been Dr McCloy's work.[107] The Behavioural Science in Government Network is to be supported by a "Civil Pages community" which will act as a "forum for the sharing of relevant information on work in this area across government ... [incorporating] an inventory of work on behaviours across Government ... [and] extant reports on behaviour change, and information about relevant events [and] research developments".[108] Dr McCloy said that the Network had been successful in bringing people together.[109] This view was supported by Ms Harper.[110] Dr McCloy's appointment as Research Fellow is however time-limited and it is unclear how her work will be continued. When we asked Mr Bartholomew, he said that GES and GSR were "looking at other options for fellowships".[111] Several witnesses were concerned about detrimental consequences arising from a lack of continuity of Dr McCloy's work.[112]

4.39. A function of BIT is to "foster more inter-departmental discussion about the effectiveness of different means of changing behaviour".[113] The Government told us that a number of departments, including Defra, DECC and DH, have already contributed to Government understanding and knowledge about behaviour change and that heads of professions had a role in disseminating that knowledge.[114] We were also told that government officials, particularly members of the GES and GSR, sometimes went into other departments to help disseminate knowledge.[115]

4.40. It is not clear how, or how well, these different resources work together—how, for example, the work of BIT relates to the Behavioural Science in Government Network, or how departmental scientists and policy makers participate in either. Andrew Lee, Director of the Sustainable Development Commission, also argued that "there is not nearly enough connecting up between the Cabinet Office and Defra, which have now developed quite a lot of expertise in this area. We had a lot of feedback in our work about officials not knowing where the evidence was, or what other people were doing".[116] The Central Office of Information agreed that the structures to join up the different silos of expertise were not yet in place, though also noted that many of these areas of expertise were focused on particular disciplines. A mechanism to join up experts across disciplines was needed to ensure that interventions were strategically planned, and they suggested the creation of a cross-government network.[117]

4.41. There are a number of different mechanisms in place for sharing knowledge but too much activity can make sharing knowledge more difficult rather than easier. We recommend that the Cabinet Office, together with the GCSA and CSS, once appointed, review the current mechanisms for sharing knowledge about behaviour change among Government departments with a view to introducing a more streamlined structure.

4.42. We recommend further that this revised structure should involve the continuation of work begun on the "inventory of behaviours" in order to establish an archive of behaviour change interventions. This archive should provide accounts of the evaluation of the interventions and include unsuccessful as well as successful interventions.

48 We note that the Health Trainers programme had substantial flaws in its evaluation and so must be considered poorly designed in that regard; however, its design did reflect the available evidence. Back

49 BC 27. Back

50 Ibid. Back

51 BC 83. Back

52 BC 107. Back

53 HC Deb 18 January 2011 col 34WS. Back

54 Alcohol use disorders-preventing harmful drinking, NICE (2010). Back

55 BC 83, Q141. Back

56 BC 52, BC 83, BC 99, BC 100, BC 110, BC 113, Q 131. Back

57 Q 733. Back

58 Q 33. Back

59 BC 105. Back