Chapter 2: The policy development process
Why legislation?
7.Legislation is an essential tool for Government to achieve its policy aims, but it is not the only route by which policies can be pursued and the Government’s ends achieved. David Halpern, Chief Executive of the Behavioural Insights Team, told us that “You do not need to go to legislation. [As] head of the Behavioural Insights Team … [we] spend a lot of time avoiding legislation. The point is that there are so many other things that you can do.”3
8.The Government recognises that legislation should not necessarily be the first port of call for departments seeking to advance a policy objective. The Cabinet Office Guide to Making Legislation states that “before seeking a slot in the Government’s legislative programme, departments should consider whether primary legislation is necessary … [D]epartments should consider whether the ends they wish to achieve could be reached by purely administrative means.”4 David Lidington MP, the then Leader of the House of Commons, told us that “probably any Government of any political stripe would argue that all their legislation is necessary”, but added:
“One of the points that I and other business managers make to any Secretary of State who is pressing for a Bill is the question: is this absolutely necessary, or can you deliver your policy objectives through non-legislative means or through existing secondary legislative powers rather than resorting to a new statute? That is one of the elements of the internal government process of scrutiny, preparation and testing that goes on in the preparation of legislation before it is brought forward.”5
9.There are, therefore, formal hurdles that need to be cleared before a department can bring forward a legislative proposal. In addition, there are clearly other reasons why departments might try to avoid legislation unless absolutely necessary: taking a bill through Parliament is resource-intensive, slow, and policies can become politicised during the process.
10.Despite these procedural hurdles, and the relative attractiveness of pursuing non-legislative change where possible, Dr Ruth Fox, Director and Head of Research at the Hansard Society, told us that “the reality is that there is no standard test of why legislation is required that is applied upstream in Whitehall.” She elaborated:
“Sometimes there will be a process in place for consideration of the mischief that they want to remedy and, therefore, why legislation is required; sometimes it will emerge from a rather ad hoc political process in which it is more about Ministers being seen to act, show initiative, respond to media issues and so on. In those circumstances, what emerges is legislation as a means of demonstrating that something is happening and communicating a message, rather than a legal or legislative test about whether the policy you want to achieve is required in legislative terms.”6
11.Other witnesses agreed that factors other than legislative necessity could play a role in the introduction of legislation. Sir Richard Mottram, Chair of the Better Government Initiative, noted that one way for a minister to “shine” is by taking a bill through Parliament.7 Meanwhile, the Institute of Chartered Accountants in England and Wales (ICAEW) told us that “in some recent examples we have observed ‘tough new legislation’ being announced to fix a problem for which there is already legislation in place (just not being used effectively, or not brought into force). The motives for this approach are unclear.”8 Daniel Greenberg, a former parliamentary counsel, gave us a recent example of legislation that might, it could be argued, fail a necessity test:
“the National Citizen Service Bill … What does it do? It sets up a body corporate with the following primary functions ‘to provide or arrange for the provision of programmes for young people … for the purpose of … enabling participants of different [backgrounds]’ … If you want to do it, just do it. You do not need a Bill. I cannot find a legislative proposition in that Bill. I cannot see anything that needs to be done by legislation that cannot be done in other ways.”9
12.Lord Lisvane, giving the Statute Law Society’s Annual Lecture in February 2017, also suggested that the purpose of the National Citizen Service Bill could be achieved without legislation: it was “practically unnecessary but politically desirable.” Arguing against the idea of legislation being used to “send a message”, he stated that “the sole purpose of primary legislation” should be “to change the law only to the extent required to achieve the precise changes” sought.10
13.Mark Ryan, Senior Lecturer in Constitutional and Administrative Law at Coventry University, was among those11 who suggested that the Government should have “to formally justify the need for its proposed legislation and also how it would sit in relation to other existing legislation (including any law which it would displace).” He suggested that “such a requirement would focus the mind of Government policy-makers on the necessity of the law proposed as it is well documented that one of the major problems with modern legislation is that of its volume and detail.”12 We consider issues related to formal legislative standards later in this report (see paragraphs 172–183).
14.We recognise that the initiative for new policy may come from empirical evidence, political pressure or principles; and often a mixture of all three. This is an inevitable result of our democratic system. While we recognise that there are occasions on which it will be necessary for ministers to initiate legislation in the absence of a detailed evidence base (see paragraphs 31–37 later in this chapter), we do not believe that such decisions should be taken lightly, or be viewed as an easy option when the Government feels it must be seen to do something. We welcome the then Leader of the House of Commons’ assurance that he “tests” the necessity of legislation when considering the Government’s forward legislative programme. We consider later in this report whether there is a need for a legislative standards committee which would, among other things, require civil servants and ministers to justify, and be shown to justify, whether legislation is necessary to achieve a particular policy goal (see paragraphs 172–183).
The use of evidence
Policy development
15.The introduction of legislation to Parliament is one of the final steps of a long process of policy development. While this report is focused on the process by which laws are made, the quality of legislation depends heavily on the appropriateness of the process by which the underlying policy was developed. As Sir Richard Mottram noted: “You can have what is apparently a well-drafted piece of legislation, in that it is comprehensible, coherent and so forth, but it has no possibility of achieving the objective … [I]f it is to be a good piece of law it has to be founded on a good piece of policy-making, and that involves a prior set of questions.”13
16.The Committee heard evidence that, overall, evidence has become a more important and integral part of the Government’s policy-making process. Sir Richard Mottram felt that, in general, there was a “strong drive towards more evidence-based policies.”14 Jill Rutter, Programme Director at the Institute for Government, agreed. She told us that “since 2010, with the coalition, there has been a concerted attempt to produce more evidence.”15
17.Others felt there was still significant room for improvement. Jonathan Breckon, Director of the Alliance for Useful Information, told us bluntly that “The use of evidence by government in policy and legislation is not good enough.”16 His organisation’s written evidence argued in particular that tools such as systematic reviews and Rapid Evidence Assessments should be used more frequently, to ensure that the evidence on which policy decisions are taken is trustworthy.17
18.Government departments can face challenges in trying to ensure policies are evidence-based. One difficulty can be the timeframes involved. Emran Mian, Director of the Social Market Foundation, told us that there is “a mismatch between the timeframes needed to do rigorous, robust research and the timeframes that we usually allow for policy-making. On the occasions when those timetables can be aligned, we make effective use of policy-making in government … That is often not the case because policy-making is often done in a much more abbreviated timescale.”18 Jill Rutter agreed:
“At one of our [Institute for Government] seminars, a Minister said, ‘The problem is, if there is no evidence that is immediately out there and I want to make a decision, I am up against the political cycle. A reasonable forecast is that I am probably in this job for 18 months or two years. I want to get things done. I need the evidence now. If I ask you to get the evidence for me, it will take you six months to develop a proposal and a bit longer to get it funded, and you will come back with the results in a couple of years’ time. By that time, I will be Fisheries Minister’ … [T]here is a real conflict between what is there now and what Ministers need now. Ministers are always going to have to make decisions with only partial information.”19
19.We welcome the fact that the process of policy development within Government now includes embedded mechanisms that place an emphasis on gathering and evaluating evidence.
Scrutinising the evidence base for Government policies
20.While the Government’s use of evidence was generally perceived to have improved, a number of witnesses expressed concern that it was often hard for organisations outside Government to scrutinise the evidence base for policies; they were not always published or made available in a form that permitted proper external scrutiny.20 Witnesses generally felt that, absent certain special factors such as national security or foreign policy concerns, the evidence base for government policies ought to be published.21 Jill Rutter told us that “you are committing public resources, so you ought to tell people the basis on which you have decided to commit those public or private resources. There are relatively few exceptions.”22 David Halpern agreed, telling us that he would “struggle to find many examples of why you would not publish” the evidence base for legislative proposals.23 Jonathan Breckon suggested learning from the experience of other nations:
“In South Africa, they now have White Papers that have a section that shows the evidence … It can be empty, which is fine; you are being transparent about the fact that there is no evidence or it is of very dubious quality—it is a think tank with some wishy-washy survey—but at least you have something to get your teeth into.”24
21.In addition to publishing the evidence for policies in a more methodical manner, it was suggested that it needed to be made more accessible. A joint submission from the Institute for Government, the Alliance for Useful Evidence and Sense about Science stated that “We do not expect the public or parliament to have to ‘dig’ for evidence—it is the responsibility of government to make it readily accessible through inclusion, discussion and clear links and references.”25 Jonathan Breckon also suggested that, at present, evidence could be hidden away from public view:
“it is so hard to find evidence behind particular policies and legislation when you are on the outside … If we take something like the Department for Education’s Educational Excellence Everywhere White Paper and look at even really simple things such as the referencing, we find that it goes nowhere. It goes to a splurge of different, random bits of evidence—so we cannot find the evidence behind the particular policy.”26
22.We recommend that the Government should routinely publish the evidence base for legislative proposals. If a robust evidence base is not available, the Government should explain why it is nevertheless appropriate to proceed.
Piloting policy
23.The evidence we heard drew a distinction between piloting new policies before they are introduced more widely, and piloting different policy variations to build an evidence base for subsequent policy-making. Dr Halpern promoted the value of the latter. He suggested that it was helpful to “put into legislation the capacity to carry on experimenting”, including testing variations in legislation. He explained:
“I will give you a concrete example … It concerns licensing arrangements for small outlets; the example often given is a community centre or a church hall. Do they need to get a big licence to serve a couple of bottles of wine? The question arose, why could we not test it? Why could we not test variations in the legislation?
We rapidly get pushback, especially from the legal establishment, which finds it slightly abhorrent that we would deliberately vary practice, of law in this case. In principle, it applies to law or, indeed, any other practice. We might want to do that in order to establish efficacy. My own view is that we should absolutely do that. When people have objections, we should remember that that is how we made progress in medicine. When it is literally life or death, we have been prepared to vary practice to find out whether something is marginally less or more effective.”27
24.Dr Halpern’s written evidence specifically drew attention to the opportunities afforded by Brexit for trialling new approaches in areas such as agriculture. He argued that when legislating for Brexit, “the more that we can include the power to trial new approaches the better.”28 Dr Halpern’s call for more ‘experimentation’ echoes the views of organisations such as the Institute for Government, which in its 2011 report Making Policy Better argued that “In a complex and decentralised environment, the perception of policy success needs to change. A more trial and error approach to policy is likely to yield better results than policies which require wholesale system change.” The report concluded, however, that “one of the big barriers to experimentation is the perception that a failed experiment is a political failure, and a waste of public funds (rather than saving larger sums by preventing full-scale implementation of a flawed concept). Politicians and civil servants need to be more confident in defending such approaches.”29
25.Emran Mian commented on the more traditional form of piloting—trialling policies in preparation for their introduction more widely—and suggested to us that while pilots had their place, they were not appropriate in every case. This might be for practical reasons, or where piloting is “inappropriate to the structure of the policy problem.” He added, however, that in some cases piloting was not used simply because there was “a desire to just get on with it and enact the political decision … [I]t is often read either by the political decision-maker—the Minister—or by the Opposition as foot-dragging. That is the reason why a pilot is not pursued, and that feels to me like a much less sound reason for failing to use piloting.”30 To complicate matters further, he noted that there were no criteria used within Government to determine whether or not a pilot was appropriate for a particular policy: “[i]t is much more an impressionistic judgment.”31
26.Chris Walker, former Head of Housing, Planning and Urban Policy at Policy Exchange, agreed that piloting policies could be helpful in gathering an evidence base, but noted “There is always the time tension. Even if you can do a pilot, and all the right conditions are met to do a pilot, it will not always be the case that it necessarily adds to the evidence base … There are ways of getting at things other than pilots.”32
27.We recognise the value of piloting, both as a means of experimenting with different policy variations in order to create an evidence base for policy-making and as a way to trial the impact of new policies before they are rolled out nationwide.
28.In some cases, piloting new policies will require the law to be applied differently in different areas—to take a high-profile example, the piloting of universal credit means that individuals will have different entitlements to benefits depending on whether they live in an area piloting universal credit. There are also, of course, variations in the law in different parts of the country as a result of devolution from which lessons could be learned.
29.There are, however, areas of the law where such experimentation should only be undertaken after thorough consideration of the constitutional implications of introducing variation. For example, the equal and consistent application of the criminal law is a fundamental tenet of the constitution and an important principle of the rule of law. We would not wish to see that principle diminished.
30.We recognise that Parliament may authorise the Government to pilot any policy it chooses, but it may help ensure a more consistent approach if the Government were to set out guidance as to when piloting is appropriate or desirable.
The political dimension
31.One of the key issues which repeatedly arose in our evidence was, if not the tension, then the balance that must sometimes be struck between political imperatives and the requirements of evidence-based policy-making. Several witnesses noted that, while a research base for policies was important, it was only one element upon which policy-making was based. Jonathon Breckon said that “There is often a misunderstanding that evidence will trump everything else. That was never the model … [It is a] triangulation between research, professional judgment and the stakeholders.”33 Jill Rutter agreed, stating that while an evidence base was important, the political judgments of ministers were also a valuable source of information: “we should not be too narrow about the sorts of evidence. Ministers, as constituency MPs, are a vital source of a different sort of evidence.”34 We recognise that MPs and ministers receive valuable information and insight on policy issues through communications with constituents, businesses, interest groups and others. The political judgement they bring to bear on policy on the basis of that knowledge is beneficial to the legislative process.
32.Chris Walker told us that:
“At the end of the day, it is that tension between having an elected Government and a technocratic arrangement. At the end of the day, you cannot just have adherence to an economic model that says yes or no; you have to take the other considerations into account. One of the key roles of evidence in policy-making is to get the best possible outcome for Ministers and the Government, subject to the political constraints and the realpolitik.”35
33.We heard that, in situations where politics and empirical research were pulling policy development in different directions, transparency was key to ensuring that the outcome could be appropriately scrutinised. In that context, several witnesses noted that ministerial directions might be a useful tool for promoting transparency. It was argued that civil servants should be able to request ministerial directions when decisions were taken that were contrary to the balance of evidence (at present they can only be requested if a permanent secretary, as accounting officer for a department, feels that a ministerial decision to spend money does not meet one of the Treasury criteria of regularity, propriety, value for money, and feasibility). Jill Rutter, for example, suggested that:
“You do not want unelected civil servants saying to Ministers, ‘You cannot do that’. Ministers will always want to do innovative things. We do not want to constrain policies only to things that have been done somewhere else before. We want policy innovation, but when people think that there is not a good enough basis for proceeding, they should be able to seek a policy direction and put on the record that they did not think there was an adequate evidence base.”36
34.Emran Mian agreed, and suggested a rather different concern:
“I completely understand that there will be cases when Ministers choose not to follow where the balance of evidence might be. What is inexcusable is when the Civil Service fails to inform Ministers about where the balance of evidence is, and that happens all too often. All too often, a political decision is allowed to take place without Ministers even knowing what the evidence base is, and that does not feel right to me. It is the Civil Service’s obligation to ensure that, when Ministers are making that decision, they are doing it at least with sight of all available evidence. I am not sure that we always have the mechanisms in place to make sure that that is happening.”37
35.These concerns echo a National Audit Office report, published in 2016, which found that departmental accounting officers “appear to lack confidence to challenge ministers where they have concerns about the feasibility or value for money of new policies or decisions, not least because standing up to ministers is seen as damaging to a civil servant’s career prospects.”38
36.Francis Maude MP, then Minister for the Cabinet Office, suggested in a 2013 speech that ministers should be less wary of taking decisions that required a direction, noting that “any minister should be confident enough in the judgment he or she has made to be willing to justify it in public.”39 Notwithstanding this statement, it seems to remain the case that ministerial directions are seen as a ‘nuclear’ option.
37.Ministers are responsible for the policies they oversee, and they are accountable to Parliament and to the electorate for their decisions. Empirical evidence, where available, is a valuable input in the policy-making process, but it is not the only factor that politicians may take into account. Civil servants have a duty to ensure that ministers are aware of any relevant evidence—even when that might be politically inconvenient. We recommend that the Government should routinely publish the evidence base for legislative or policy proposals, as recommended in paragraph 22, as this should help ensure that ministers have an accurate picture of relevant evidence before decisions are taken.
Green Papers and White Papers
38.The Prime Minister has recently indicated that “she would normally expect a Minister, before having legislation, to have gone through a Green Paper stage for discussion and then a White Paper stage to set out policy.”40 Our witnesses were universally in favour of this stance.41
39.The Bingham Centre for the Rule of Law stated that “There is a strong case for the use of standardised or default positions in a number of areas, so that there is predictability about what process will be used (eg, timing, green paper, white paper) and then any proposed departure from the default should be published and explained.”42 The Liberal Democrat Constitutional Affairs Team agreed that Green Papers and White Papers ought to be “a routine pre-requisite for bills coming before Parliament.”43 That view was echoed by Valerie Vaz MP, Shadow Leader of the House of Commons, who noted that “I consider publishing a green paper and white paper together with impact assessments built on evidence based policy would be a route to good legislation.”44
40.The then Leader of the House of Commons confirmed that the “rather old-fashioned idea of Green Papers and White Papers” was “welcome and right in principle for making for good policy”; indeed, he added that “it is actually essential for taking legislation successfully through a Parliament in which the Government cannot simply snap their fingers and assume that a majority will be there.”45
41.We welcome the Prime Minister’s commitment to a greater use of Green Papers and White Papers. A more structured approach to policy development and consultation can only improve the quality and consistency of legislation. In addition, we draw particular attention to the recommendation in our 2011 report, The Process of Constitutional Change, in which we concluded that “We regard it as essential that, prior to the introduction of a bill which provides for significant constitutional change, the government … publish green and white papers.”46 This has not always been the case in the recent past, and we hope that in the future a more rigorous approach will be taken towards the process of legislating for constitutional change.
42.The then Leader of the House of Commons commented that proper policy development processes are essential for taking legislation through a Parliament in which the “parliamentary arithmetic” is such that the Government cannot be assured of pushing legislation through. While a Government with a large majority will always have an easier time passing its legislation through Parliament, the quality of legislation should not be dependent on the composition of the House of Commons. It is in the interest of all parties that there are procedural mechanisms to protect the quality of legislation, regardless of the political ‘strength’ of any particular government.
4 Cabinet Office, Guide to Making Legislation, April 2017: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/616341/Guide_to_Making_Legislation_Apr_2017_v2.0_final_May_update.pdf [accessed 19 October 2017]
10 Lord Lisvane, ‘Why is there so much bad legislation?’, Statute Law Society Annual Lecture, 28 February 2017
25 Written evidence from the Institute for Government, the Alliance for Useful Evidence and Sense about Science (LEG0008)
29 Institute for Government, Making Policy Better, April 2011: https://www.instituteforgovernment.org.uk/sites/default/files/publications/Making%20Policy%20Better.pdf [accessed 19 October 2017]
38 National Audit Office, Accountability to Parliament for taxpayers’ money, 23 February 2016: https://www.nao.org.uk/wp-content/uploads/2016/02/Accountability-for-Taxpayers-money.pdf [accessed 19 October 2017]
39 Francis Maude MP, Ministers and Mandarins: speaking truth unto power, 5 June 2013: https://www.gov.uk/government/speeches/ministers-and-mandarins-speaking-truth-unto-power [accessed 19 October 2017]
41 See, for example, Q 8 (Sir Richard Mottram), written evidence from the Liberal Democrat Constitutional Affairs Team (LEG0021)
46 Constitution Committee, The Process of Constitutional Change, (15th Report, Session 2010–12, HL Paper 177), summary