• Accessibility
• Email alerts
• RSS feeds
• Contact us

Losing Impact: why the Government’s impact assessment system is failing Parliament and the public Contents

---

Chapter 3: What are the problems? Principal concerns with Impact Assessments

21.In this chapter we set out our principal concerns based on our scrutiny
of hundreds of SIs each year. We make reference throughout to the five principles of Better Regulation to illustrate where we think the current system is falling short of those precepts.

Missing IAs

22.We have found that an increased number of instruments which require an IA have been laid before Parliament without one. In a report in October 2021, we published correspondence in which we drew to the attention of the responsible Minister, Paul Scully MP, the following examples:18

• HM Treasury: Money Laundering and Terrorist Financing (Amendment) Regulations 2019 (IA was awaiting sign-off by the RPC at the time).
• Department for Digital, Culture, Media and Sport: draft Code of Practice for Online Services on Age Appropriate Design (IA was promised before the Code had passed through Parliament).
• BEIS: draft Product Safety and Metrology etc. (Amendment etc.) (UK(NI) Indication) (EU Exit) Regulations 2020 (promise of publication of regulatory triage assessment “later this year”).

23.Mr Scully replied that it “remains the responsibility of individual departments to produce a proportionate assessment of the impacts of their policy proposals” but said that the Government “will always strive to ensure that there is sufficiently robust analysis to support the decision-making that underpins regulation”. He continued: “I will ask my officials to take additional steps to reinforce this message, by writing to departments to remind them of the requirements and asking them to commit to meeting them”.19

24.More than six months later, we have found no discernible improvement:

• Department for Levelling Up, Housing and Communities (DLUHC): failed to complete an IA for the draft Private Parking Code of Practice (because the department had incorrectly assumed that only statutory instruments required an IA).20
• Department for Transport: published the IA on the Motor Vehicle (Driving Licences) (Amendment) (No. 5) Regulations 2021 six months after they had been laid and four months after the Regulations had come into effect (because of a lack of data).21
• BEIS: laid the Draft Building Safety (Leaseholder Protections) (Information etc.) (England) Regulations 2022 without an IA (“not yet finalised” to be published “shortly, ahead of debates”).22

25.IAs provided months after the instrument has come into effect are no use for scrutiny purposes. Other examples include:

• DLUHC: the Town and Country Planning (General Permitted Development) (England) (Amendment) (No. 3) Order 2020 came into effect on 31 August 2020 and the IA was published in March 2021.
• Department of Health and Social Care:
  • Regulations imposing the compulsory vaccination of care home staff23 were laid on 22 June 2021, we took oral evidence from the Minister on 13 July 2021 about the lack of an IA, which was finally published on 9 November 2021.
  • Further legislation extending the vaccination requirement to all NHS staff24 was laid on 9 November 2021, its IA was submitted for independent validation to the RPC on 15 November, which subsequently rated several sections as unfit for purpose.25

26.This is not just about ‘paperwork’—late laying of an IA raises important concerns:

• Parliamentary scrutiny: we endeavour to report on an instrument about two weeks after laying. We often have questions about what other policy options were considered or costs, which is information that should be included in an IA. Without this information, the Committee and Parliament are unable to scrutinise an instrument effectively. Furthermore, given that IAs tend to be substantial documents, laying an IA shortly before a debate may be of limited value.
• Opportunity for wider challenge: publishing an IA at the same time as the instrument gives those affected an opportunity to challenge the Government’s assumptions. For example, the Home Office recently revoked legislation to licence a chemical that could also be used as a drug: it had believed only 65 firms used the substance, but an industry body subsequently told them it was closer to 7,500 firms and the system envisaged would, therefore, not work.26
• Policy development: most important of all, the IA should inform policy development and evolve with it (see paragraphs 10–11 above). If an IA is not available when an instrument is laid, then we are led to the conclusion that the information was not available or used when the policy was being formulated.

27.All supporting information, including IAs, must be laid before Parliament at the same time as the instrument in order to ensure effective Parliamentary scrutiny, transparency and accountability.

Delays due to the RPC?

28.On occasion, a department may explain their failure to provide an IA alongside an instrument on the grounds that it has not yet been cleared by the RPC. We raised this with Stephen Gibson, Chair of the RPC. He told us that RPC scrutiny periods were well advertised:

“We have a target of a 30-day turnaround for the RPC process for the production of an opinion on an impact assessment. Currently the average timescale for that production is 22 days.”27

29.Given this, we queried whether the delay is not within the RPC but rather a failure by the departments to factor in the RPC turnaround time. Andrew Williams-Fry of RPC commented:

“In 2019, 35% of impact assessments received a request for an expedited process, that increased in 2020–21 to 40%. Where RPC have to expedite a process for a particular IA, that leads to other IAs potentially being delayed.”28

30.Departments should ensure that they plan realistically, including time to address any problems identified by the RPC, and only ask for expedited consideration in exceptional circumstances. We urge the BRE to take steps to ensure departments understand this important point and to support departments with appropriate training.

31.Conversely the publication of the RPC’s opinion on an IA can be delayed because the department itself has not published the IA. For example, when the draft Money Laundering and Terrorist Financing (Amendment) (No. 2) Regulations 2022 were laid before Parliament on 15 June, the EM stated that a full IA would be published “in due course”. After we had completed our scrutiny of the draft Regulations, it was brought to our attention that the IA had been assessed as not fit for purpose (“red-rated”) by the RPC. The IA and the RPC’s rating were published together on 14 July, a month after the instrument had been laid.29

32.We find the current approach unsatisfactory: if the RPC feels constrained not to pre-empt a department’s publication of an IA, material on the quality of an IA that could influence our view of the instrument’s policy is not available to us. We suggest that the RPC should demonstrate its independence by publishing its view of a final-stage IA as soon as it is ready: when an SI has been laid for scrutiny Parliament should always have access to the RPC’s assessment, whether the department publishes the IA or not. We also suggest that the RPC could do more to communicate to Parliament when it has ‘red-rated’ any IAs.

Delays due to the BIT target?

33.The 2015 Act introduced the Business Impact Target (BIT) to monitor the financial impact of government legislation on business. The Act requires IAs to be produced within the BIT reporting year so that the net cost of government legislation in that year can be calculated. Laying an IA six months after an instrument has come into effect can still meet the BIT requirement, as long as the IA arrives before the end of the reporting year. We are concerned that this looser deadline may distract officials from Parliament’s need for an IA to be available as soon as an instrument has been laid. In addition, the BIT requires highly technical calculations about costs that may significantly lengthen the time it takes to produce an IA. Both BRE and RPC agreed that, at present, the BIT system was not achieving its intended purpose.30 We urge the BRE to ensure that officials are aware that, irrespective of the BIT reporting year, parliamentary scrutiny starts as soon as the instrument is laid before Parliament and an IA needs to be published at the same time as the instrument it supports.

‘No IA required’

34.For instruments that do not require an IA (because they do not exceed the £5 million per annum threshold), the Impact Section of the EM often just states that ‘no IA is required’. This is, in our view, unhelpful in terms of both transparency and accountability.

35.We raised the problem of ‘no IA required’ with the BRE and RPC.31 Although neither routinely see impact information for SIs below the threshold, they told us that departments have to prepare a simplified de minimis IA to be able
to demonstrate when ‘called-in’ by the RPC that the £5 million threshold has not been reached. For instruments below the threshold, departments should always include basic impact information in the EM and, we suggest, attach the de minimis assessment where available.

36.That need not be an onerous request—often a couple of sentences in the EM would be adequate, for example:

Pneumoconiosis (Workers’ Compensation) (Amendment) Regulations 202132

There will be an estimated £218,000 increase to the DWP Departmental Expenditure Limit for 21/22. These estimates are at the time of the submission but delayed assessments due to Covid-19 may impact on final costings.

This example also helpfully indicates the degree of uncertainty in the estimate. For pilot projects and new initiatives accurate data may not be available, but a well-explained ‘best guess’ of the likely costs offers a reassurance that the policy’s likely effects have been thought through and analysed.

37.We have remarked in several recent reports that, despite being made at speed, many of the Foreign, Commonwealth and Development Office (FCDO) instruments imposing sanctions against Russia have been accompanied by a well-developed draft of the IA. In these cases, there is good reason why the RPC has not had time to consider the IA but the FCDO has nonetheless provided the best information available alongside the instrument. We encourage other departments introducing emergency legislation to follow the same practice and avoid the ‘all or nothing’ approach we frequently see.

Pandemic exemption

38.During the pandemic, temporary measures lasting less than 12 months were exempted from the IA requirement. In consequence, the EMs to most coronavirus SIs simply stated ‘no IA required’. Mr Gibson of RPC thought that this was a missed opportunity:

“We think we could have added a lot of value, perhaps not at the first lockdown stage but thinking about what we learnt from the first lockdown for the second and third lockdowns: was it right to close gyms, hairdressers, restaurants or whatever? Doing that monitoring and seeing how it worked the first time around would have informed better regulatory policy-making at a later stage.”33

39.When the Government are reviewing their handling of the pandemic legislation, we recommend that they consider whether these IA exemptions prevented useful information from being gathered that could have made subsequent measures more proportionate and better targeted.

40.We wrote to Mr Scully, the previous BEIS Minister with responsibility for the provision of IAs, when he announced that exemption and he reassured us that:

“We will also continue to monitor the position of emergency legislation to ensure departments produce retrospective impact assessments where Covid-19 measures are to be made permanent.”34

We would welcome information about how many departments have actually done this and what proportion of the total due that represents.

Post-Implementation Review

41.Post-Implementation Review (PIR) is a checking process to see whether estimates were accurate, predictions were fulfilled, and the policy has achieved its intended outcome.

42.Sections 28–32 of the 2015 Act require any “regulatory provision” that passes the IA threshold to be reviewed five years after commencement and every five years after that. Section 30 requires the minister to publish a report on the conclusion of each review that must be validated by the RPC. It is required to:

(a)set out the objectives intended to be achieved by the regulatory provision,

(b)assess the extent to which those objectives are achieved,

(c)assess whether those objectives remain appropriate, and

(d)if those objectives remain appropriate, assess the extent to which they could be achieved in another way which involves less onerous regulatory provision.

43.There is extensive guidance on how to carry out a PIR from BRE,35 the RPC,36 the Treasury’s Magenta Book37 and the National Audit Office (NAO).38

44.We find it surprising that despite the statutory requirement to carry out a PIR and the range of guidance available, we very rarely see any mention in EMs that the changes being made by an instrument are the result of a PIR. Mr Gibson made a similar point:

“Often we see post-implementation reviews that simply have a default assumption that everything is working well, whereas in my experience Governments do not always get it right first time.”39

45.In 2008, this Committee conducted an inquiry into the use of PIR in collaboration with the NAO.40 Key issues identified were the lack of any common review methodology or consistent tracking of SIs. The NAO followed up 229 SIs which required an IA and were at least three years old (which was the standard review date at that time). The outcome was:

• 29% of them had completed a PIR,
• 54% had done some sort of evaluation (though mainly informal and in-house), and
• 46% had not been subject to any sort of review.

46.Regrettably, there appears to have been little progress since then. Mr Carr of the BRE estimated that only 25–40% of instruments completed their PIR despite it often being a statutory obligation.41 Lord Callanan gave a higher figure, stating that before the pandemic 72% of instruments received a review on time, and suggested that the recent slump was understandable due to the pandemic.42 We have found that a few PIR reports on secondary legislation may be found on the Gov.uk website43 but they are hard to find, not systematically published and do not appear to use a common format or methodology.

47.The BRE guidance says that each PIR report should be reviewed by the Cabinet Office Domestic and Economy Implementation Committee and both statutory and non-statutory reviews should be published on the Legislation.gov.uk website, under the “More resources” tab of the instrument to which it relates.44 Our search of that website identified five such PIR documents45—the most recent of which was published in 2013. However, they are inconsistently flagged and hard to find.46

48.Lord Callanan told us that, as part of the review of the Better Regulation Framework, he was looking to set up a scrutiny board that would keep track of when PIRs are required, would remind departments of the timescale perhaps a year in advance, and support them through the process of implementing and coming up with the outcome of the review.47 We welcome this initiative to ensure that PIRs are monitored but the carrot of BRE support needs to be balanced with an effective stick to ensure compliance.

49.We also recommend that the review should consider how PIRs are published to make them more easily accessible and how officials can be encouraged to make use of them when formulating subsequent legislation.

Operational difficulties

50.Despite the availability of guidance and 80 BRE staff advising departmental officials on the process, between a quarter and a third of IAs sent to the RPC get an initial review notice (IRN) to say they are not fit for purpose.48 Mr Gibson said that at the initial stage IAs tended to be weakest on the policy objectives and consideration of options, and those submitted for the final stage review tended to be weakest on the wider impacts (such as the legislation’s effects on competition, innovation, trade, or the environment) and on the monitoring and evaluation plan.49

51.Due to the RPC’s interventions very few final-stage IAs are judged not fit for purpose (“red-rated”). We query why there is such a high percentage of IRN’s issued, when there is such substantial support available. We think that part of the explanation may be the complexity of IAs, which are often 60–80 pages long, and their focus on the technical financial calculations set out in the Treasury Green Book.

52.The Better Regulation principles define well-targeted regulation as law that focuses on the problem identified, minimises side effects and focuses enforcement primarily on the most serious risks. It is difficult to know the extent to which a thorough analysis of risks and options has been undertaken from looking at final-stage IAs because they often only offer the alternatives ‘do nothing’ or ‘do this’. We acknowledge that an IA produced at the consultation stage may have included a more extensive consideration of options, but we do not see it and we wonder why this useful information is removed.

53.Mr Gibson made a similar point:

“… often the options we see at the final stage are to do nothing or to take the preferred option … whereas in fact there may be quite a lot of choices: different options for how the regulation may be introduced, mitigation for smaller micro-businesses or different ways in which it can be as effective but have much lower costs to businesses and to society.”50

54.To improve the targeting of regulation, the BRE make a proposal in their consultation on Reforming the framework for better regulation51 that departments should be required to submit an options appraisal for independent scrutiny at an earlier stage in the process. This proposal appears to us to have merit because it would demonstrate that departments have properly considered alternatives to regulation or different lighter-touch regulation, leaving the more complex analysis involved in a full IA for “the right option”.52

55.This approach has the potential for several benefits:

• easier for non-specialist civil servants to operate, whereas by contrast a full IA often needs the services of an economist;
• broadening the range of factors considered at an early stage
  • because a policy team may lack experience of the business environment, and
  • because teams can unintentionally develop ‘group think’ that exposure to an external body should challenge;
• improving the quality of policy targeting because the IA process is better integrated into the policy process; and
• wider in scope because at the initial stage it will not be so evident which proposals will exceed the £5 million threshold.

Public sector projects

56.At present, measures that only impact the public sector are not required to provide an IA, although they are sometimes provided on a voluntary basis for major public policy changes and we find that extremely helpful.53 Properly used the impact assessment process encourages officials to think of the wider consequences of their legislation and might have avoided the sort of ‘tunnel vision’ we recently saw in a Department for Education instrument on Student Loans.54 The EM included a single sentence stating that the changes were expected to generate £3.7 billion of savings for the Government, but failed to mention that this money would come from around a million students each paying an extra £113.40 per year. We recommend that IAs should be published alongside all instruments which implement significant policy changes irrespective of the sector impacted.

Lack of a central point of authority

57.Both the Rt Hon. Jacob Rees-Mogg MP, former Leader of the House of Commons, and the current Leader, told us that they took seriously the importance of assisting parliamentary scrutiny by providing supporting explanatory material.55 But the responsibility for the provision of that material rests with departments without any apparent point of authority within government to ensure the IA procedure is followed.

58.Lord Callanan said he was very keen to promote the use of IAs but admitted that:

“because we have no statutory means of enforcing the writ of impact assessments, we are relying on peer pressure to encourage and cajole departments to do it”.56

We find this disheartening because there appears to be little substance behind that peer pressure. As Baroness Vere of Norbiton, a Department for Transport Minister, told us during an oral evidence session about the missing IA for the Draft Motor Vehicles (Driving Licences) (Amendment) (No.5) Regulations 2021, the absence of an IA “did not cause delay because the regulations went through without the impact assessment.”57

59.Mr Gibson was clear that the RPC could not insist on an IA:

“We simply assess the quality of the evidence and analysis. If the Minister wishes to bring forward a measure where the costs and benefits have not been properly assessed, that is up to him or her.”58

60.The BRE guidance says that “All correspondence seeking collective agreement to a regulatory measure should continue to include an IA rated “fit for purpose” by the RPC, except where the impact on business is below the threshold for independent scrutiny.” Lord Callanan told us that “the provision of impact assessments is enforced by the Cabinet Office under the application of collective agreement rules”.59 Yet no one appears to be enforcing those rules.

61.We reasonably assumed that enforcement was a function of the Parliamentary Business and Legislation (PBL) Committee as the Cabinet Committee responsible for giving clearance for secondary legislation to be laid before Parliament. In his written evidence, however, Mr Spencer told us that the PBL triage process “does not consider the policy within secondary legislation, or supporting documents such as Impact Assessments. This is a matter for the department responsible … Departments are responsible for their own internal clearances and quality assurance including the text of the SI, vires and the content of the Explanatory Memorandum”.60

62.In an oral evidence session on 20 July 2022 with the Leaders of both Houses we raised PBL’s hands-off approach again.61 On this occasion the responses were slightly more promising:

• Mr Spencer agreed that “in principle, wherever possible, the impact assessment should be produced at the same point as the legislation. We are fairly robust in PBL in making sure that departments understand that.”
• Baroness Evans of Bowes Park added that “we often go back expressing unhappiness at the way in which some departments have been running the process.”

However, the Committee concluded that some departments are serial offenders and PBL’s “expressions of unhappiness” do not appear to have sufficient influence to persuade them to change their ways.

So who within Departments is responsible for quality control?

63.In the run up to Brexit, and from time to time thereafter, we have asked the lead civil servants with the responsibility for legislation, the Treasury Solicitor, First Parliamentary Counsel and the Permanent Secretary with responsibility for the Policy Profession, to respond in an oral evidence session to our concerns about SI production. In 2017, we were told by Sir Chris Wormald, the Head of the Policy Profession:

“Of the other big changes that we have made since the last time we were here, one of the biggest is that every department has now appointed a senior responsible owner for the SI process, so there is somebody senior in every department who has a responsibility for the SI process within their departments …

… Of course, the overall responsibility for anything that happens in the department goes to the Secretary of State and the permanent secretary. If the department is underperforming on anything, it is ultimately those two people’s responsibility …

… those are quite busy people. That is why we wanted a clear lead Minister, which is normally at junior Minister level, and a clear SRO who was part of the senior leadership who were responsible for the system; and then for an individual SI, the quality control would be the lead Minister and normally the deputy director. Those would be the two people who would sign it off. Effectively, those are the three levels of accountability. The bit we have added is that clear lead Minister and clear lead SRO to make sure that somebody, both at political level and at Civil Service level, was looking across the whole performance on SIs. That is the bit of the accountability that we felt was missing and we therefore dealt with.”62

64.In a further evidence session in 2021, Sir Chris’ successor as head of the Civil Service Policy Profession, Tamara Finkelstein, said:

“Some of the improvements that have already been put in place have helped, but there is more to do and in a number of areas… identifying accountable SROs—senior responsible owners—in departments who are accountable on SIs, and having a Minister for SIs, has made a considerable difference. I see that in Defra. There is proper accountability. We need to build on that so that those SROs across departments are co-ordinated by the PBL secretariat to meet and to share ways in which to improve, and that there is a genuine sense of accountability to make improvements.”63

65.In contrast, in his written evidence, Mr Spencer told us that SI Ministers “are responsible for their departments’ secondary legislation programme as a whole, with policy ministers and officials responsible for delivering each individual SI within that programme”.64 We have struggled to obtain information on how the SI Ministers carry out that role as it seems to vary widely between departments. The BRE and the RPC both confirmed that they have no contact with any of the appointed SI Ministers.65

66.We acknowledge that departments may have to deal with political imperatives that would make it difficult for an SI Minister to ask for a delay so that an IA can be completed. We note, for example, that even Lord Callanan could not prevent his department from laying the draft Conduct of Employment Agencies and Employment Businesses (Amendment) Regulations 2022 without one (IA “to be published shortly, ahead of debates, as it has not yet been finalised.”)66 We are concerned however that insufficient weight is being placed on the role of the SI Minister. We would welcome further information about the role, their common objectives, what training they receive about the standards that SIs are expected to meet and what influence they are able to exert within a department in order to ensure that those standards are maintained.

Is departmental ‘self-discipline’ working?

67.Both the BRE and the RPC commented that there is a wide variation in the quality of IAs produced within departments. Mr Carr of the BRE said:

“The difference between the good and the bad is within departments and not between them. Even the smallest department comprises hundreds of civil servants, and some of them are very good at this and some of them lack the skills and experience.”67

68.Mr Gibson of the RPC explained:

“There is a range of reasons for differences. In some cases there are tight timescales or resource constraints. It may be very difficult to obtain good evidence because of the nature of the proposal. We occasionally see that impact assessments have been developed as an afterthought; once the policy has basically been made there is a realisation that they have to go through this hurdle. That undermines much of the purpose of the impact assessment, which is to support the decision-making process …There are also issues with the experience of the civil servants involved. Many of them have very limited experience of business or of life outside Whitehall.”68

69.As well as frustrating the purpose of the IA as a policy-making tool, Mr Carr of BRE commented that: “The faster you wish to implement a policy, the poorer the analysis and evidence base will be”.69 Such last-minute assessments are likely to be based on poor data or cannot provide a clear baseline, which can in turn create problems for PIR.

70.We infer from this that some departments are failing to carry out adequate consultation with the relevant external groups that might improve their data, or they do not allow enough time for it in their project plan.70 The high percentage of IRNs that RPC are issuing at initial review stage may also indicate that departments are relying too much on RPC’s experts to tell them how to fix some of their deficiencies.

71.Given the large number of BRE staff, mainly based within departments, we are surprised that the quality of IAs within a department varies so much since we would assume that all policy officials have access to that resource. We are forced to conclude that departments, whether through their BRE Unit or through their internal clearance procedures, are failing to impose adequate quality control themselves.

72.To be effective any requirement under the IA procedure needs to be robustly enforced, and we will be considering the role of the PBL Committee in providing that enforcement below. Of course, an exemption for emergency legislation needs to be in place, but its conditions should be clearly defined and adhered to. Poor planning by a department should no longer be treated as an acceptable reason to bypass important elements of the policy formulation process.

---

• A-Z index
• Glossary
• Contact us
• Freedom of Information
• Jobs
• Using this website
• Copyright