3: Policy objectives and post-implementation
review
Summary: We welcome the new format for Impact
Assessments (IAs), which should now be provided for key measures
which affect the public and voluntary sectors, as well as for
those which impact on businesses. This has been Government policy
for the last 9 months, but we have seen few of them. Whether or
not an IA is provided, every significant instrument (or its explanatory
memorandum) should clearly express its policy objective, preferably
in measurable terms, indicating how and when its success is to
be measured and evaluated.
32. Our report in 2005-06 recommended that:
"departments, when they draw up their plans
for secondary legislation, should include against each instrument
a target date for post-implementation review and the outcomes
of such reviews should be reported when completed"
This requirement has now been happily included in
the revised Impact Assessment system but we cannot claim the sole
credit for the change. The NAO has recommended in its reports
that Departments and regulators should set out clearly when and
how they intend to measure a policy's impact[20].
The Law Commission's report on Post Legislative Scrutiny
regards the clarification of policy objectives as critical and
invites Parliament to review both primary and secondary legislation[21].
The Select Committee on Regulators, in their report on UK Economic
Regulators, stated that post-implementation reviews should be
conducted more frequently, and that "meaningful evaluation
is not possible without clear targets and objectives. The original
IA should set such targets in anticipation of a post-implementation
evaluation"[22].
We agree. Indeed, they went a step further by suggesting that
the quality of a random sample of the evaluation exercises should
be independently audited to monitor the objectivity of the analysis.
33. Our remit is to scrutinise secondary legislation
laid before the House to draw particular instruments to the special
attention of the House on the grounds identified for us in our
terms of reference. Post-implementation review thus comes long
after our work is done, but a system for proper evaluation is
important to our scrutiny. It is difficult for us to assess whether
a policy will achieve its objective if that aim is stated in vague,
unquantified terms. We would like to see greater precision in
the phrasing of an instrument's policy objective. Those who gave
evidence seemed defensive about this suggestion, particularly
that the policy objective should be articulated in quantifiable
and measurable terms (Q123). DCSF have the intention to implement
this, but they explained that setting the appropriate criteria
was not always simple: for example, in determining whether school
admissions criteria are operating "fairly" (QQ4-9).
DWP were reluctant to be bound by success criteria, even self-determined
ones (Qs 73-77). Yet the BRE Impact Assessment Toolkit states:
"Policy-makers should be mindful of Post-implementation
review from the outset of policy development, so that information
that will inform the review can be collected from the beginning
of policy development… You should use Impact Assessment as
the starting point for the review. The review should establish
a baseline and include the success criteria against which you
will assess the effectiveness of the policy in delivering the
objective."
34. Secondly, the Impact Assessment is a key
tool in helping us to judge whether the proposed outcome is achievable.
An illustration of poor practice in this area is seen in the four
successive sets of Regulations relating to Home Information Packs
(HIPS) which we drew to the attention of the House: in June 2006,
in May 2007, in June 2007 and in December 2007. Each set of Regulations
laid in 2007 reduced the content of HIPs (and, until December
2007, the Government limited their application to only parts of
the housing market). The stated objective of the policy was originally
presented as making the process of buying and selling houses more
transparent and quicker. The Government had supported tests of
HIPs in a number of trial areas, which must have shed light on
the achievement of this objective. However, the decision to apply
"cut-down" HIPs to all houses from December 2007 was
taken before the results of the trials had been evaluated and
published. Observers were left wondering whether this was a policy
which the Government wished to subject to any degree of objective
assessment.
35. The HIPs initiative demonstrated weaknesses
in DCLG's estimates of the numbers of surveyors who could be trained
in the time available. Similarly the Home Office underestimated
the time it would take to set up a system for processing wheel-clamping
licences and had to revoke the legislation and reapply it a couple
of months later[23].
This is poor management and demonstrates a tendency by Departments
to use IAs as a tool "to justify policy rather than a policy-making
tool"[24].
36. If post-legislative scrutiny were conducted
more systematically and used to refine policy on the basis of
real world experience, it would give the Committee greater faith
in the cost/benefit analysis offered in EMs and IAs. When making
policy in a new area there will always be uncertainties, but these
can be refined over time to make more accurate estimates and give
a more practical view of how effective is the law. The Control
of Asbestos Regulations 2006 (SI 2006/2739) [25]
are an example of good practice from the last session.
37. Although the criteria for post-implementation
review are set out in the BRE's Impact Assessment Toolkit,[26]
awareness amongst those who gave evidence was not high. The Toolkit
states:
"A Post implementation review establishes whether
implemented regulations are having the intended effect and whether
they are implementing policy objectives efficiently. It is not
intended to review the effects of the policy itself
or to determine whether the intended policy is still desirable."
DWP were clear that, although they conducted some
evaluation exercises, they were rolling reviews of policy and
did not relate to specific instruments (QQ76-77). The Minister
seemed even more equivocal (QQ123-6), saying that he did not think
that evaluation exercises should be the norm (although it was
he who had originally launched the Toolkit).
38. While we accept that the changes to Impact
Assessments remain comparatively new, we suggest that the Minister
for Better Regulation could do more to publicise and encourage
the setting of (measurable) policy objectives and success criteria,
and the regular adoption of post-implementation review.
20 See Evaluation of Regulatory Impact Assessments
2006-07 published 11 July 2007 (HC paper 606) for example,
paragraph 3. Back
21
Post Legislative Scrutiny - Law Commission No 302, October
2006. Back
22
House of Lords Select Committee on Regulators, UK Economic
Regulators, First Report, 2005-06 (HL Paper 189), page 46. Back
23
Private Security Industry Act 2001 (Designated Activities)(Revocation)
Order 2005 SI 2005/361 - see also Undersized Bass Order SI 2007/809
and Undersized Bass (Revocation) Order SI 2007/857 both laid 15
March 2007 or Immigration, Asylum and Nationality Act 2006 (Data
Sharing Code of Practice) Order 2007 SI 2007/3427 and Immigration,
Asylum and Nationality Act 2006 (Data Sharing Code of Practice)
(Revocation) Order 2007 SI 2007/3447 both laid 10 December 2007. Back
24
House of Lords Select Committee Regulators, UK Economic Regulators,
First Report, 2005-06 (HL Paper 189), paragraph 4.71. Back
25
47th Report (2005-06). Back
26
http://bre.berr.gov.uk/regulation/ria/toolkit/post_implementation.asp
Back
| 
