House of Commons - Environmental Audit

Select Committee on Environmental Audit Third Report

Measuring Environmental Impacts

39. It is not enough merely to ensure that environmental impacts are simply considered. Where environmental impacts are assessed in RIAs the accuracy of the assessment remains in question due to the particular challenge of calculating and presenting environmental impacts on an equal footing with economic concerns.

40. BRE guidance for officials producing RIAs currently advises the following approach for presenting impacts:

"Costs and benefits must be quantified wherever possible.
You must show costs and benefits as monetary values where this is possible. This will allow easier comparison between the costs and the benefits of each option. The direct costs of each section must be expressed as monetary values.

[…]

Record costs and benefits in qualitative terms only when the above are not possible. Without quantification it will be less clear what the differences between the options are and whether the benefits of an option justify the costs."[40]

41. Environmental assessment is often seemingly incompatible with the convention within RIAs for quantified, and above all monetised, analysis of impacts. Many environmental issues and impacts cannot easily be assigned a monetary value. Environmental impacts are also frequently unpredictable in their extent and scale, and their true consequences, positive or negative, may only become apparent over a lengthy period of time. Even the Treasury Green Book, in its examination of possibilities for evaluating a number of environmental impacts, acknowledges the complexities and uncertainties involved in calculating economic values for environmental damage.[41]

42. In spite of the new emphasis on the economic cost of climate change in the wake of the Stern Review, we do not consider monetary totals alone as appropriate for adequately conveying the extent and coverage of all adverse environmental impacts. It is difficult to see how an appropriate monetary total could be arrived at which would adequately communicate the impact of species loss, or the consequences of climate change on a global scale, and it is unrealistic to expect officials carrying out RIAs to be able to achieve this. The BRE guidance for RIAs suggests a number of methods for analysing difficult-to-quantify impacts. The particular example provided for environmental benefits is:

"The environment - you could use surveys that show people's willingness to pay (how much people would pay for a clean river, fresh air or a national park) or their willingness to accept (how much people would be prepared to accept in compensation for suffering from pollution)." [42]

43. However, Steve Smith of Ofgem made clear to us the difficulties of such approaches in certain situations, and why a qualitative assessment can be invaluable:

"Possibly one of the most difficult ones we deal with is the whole issue of visual amenity with transmission lines and things like that. That still exercises a lot of thought within Ofgem, because you can go and do surveys of people who say to you, 'I'd be willing to pay X not to have a line near my house', but you are always a bit sceptical unless they are actually paying it! That is an area where we have had to think about qualitative as well as quantitative assessment."[43]

44. These methods risk reducing environmental impacts to a level where the ultimate, wide-reaching consequences are lost, and a multi-faceted problem is expressed as a one-dimensional issue. Such trivialisation of environmental impacts can only undermine attempts to achieve a fair consideration of environmental issues against economic concerns which are both more adequately assessed in monetary terms and have greater resonance in this format. Our predecessor Committee's report of April 2005 expressed these same concerns, describing how the emphasis on monetisation in RIAs either leads to environmental impacts being omitted, or to an inadequate assessment of environmental impacts versus other, more easily monetised, factors:

"we believe that the emphasis throughout the RIA guidance […] on quantifying in monetary terms environmental impacts is fundamentally mistaken and fails to take account of the overriding need to meet environmental policy objectives."[44]

45. We continue to support this view. Although some witnesses to the inquiry, notably Anglian Water,[45] supported monetisation wherever possible on the grounds that—within the current RIA structure and guidance—this was the only way to ensure that environmental concerns were considered on an equal footing with other factors, we believe that the RIA system as it stands must be reconsidered to ensure that non-monetised assessment can be successfully included, and that it also must be recognised as a standard and suitable form of assessment, rather than an undesirable last resort. We recognise that the field of quantifying and assessing environmental impacts is an expanding one, and that considerable effort is being directed into research towards developing new techniques. We do not, however, consider blanket monetisation to be a desirable or feasible objective of impact assessment.

46. The NAO annual evaluation of RIAs for 2005-06 identified at least one RIA from its sample (the RIA for the Department for Transport's Railways Bill) where "the Department considered that a sensible quantification of costs and benefits was not possible. There were, however, no qualitative descriptions where quantification may have been difficult."[46] Although this example did not relate to directly environmental impacts, it does illustrate the dependence on monetised assessment of impacts and the inability for many RIAs to successfully incorporate and even accept qualitative analysis. Environmental impacts are particularly at risk from such behaviour, and it is deeply unsatisfactory for environmental impacts to be omitted from an RIA purely because the structure does not easily permit the forms of assessment which may most suitably convey their importance, or because there is a reluctance among officials to approach impact assessment in a non-monetised manner which is seen to contradict the conventions for RIAs.

47. We recognise that certain areas of environmental assessment such as greenhouse gas emissions and consequences to health lend themselves more easily to monetisation. In these instances it would be both desirable and appropriate to accompany a qualified form of analysis with a monetised assessment of the impact, in order to facilitate the comparison of different impacts on these grounds. We do not, however, believe that monetisation should ever constitute the sole method of illustrating an environmental impact, and call for any monetised impact to also be accompanied by a wider description of the consequences.

48. We continue to support the recommendation made by our predecessor Committee that, on the RIA summary sheet, environmental impacts should be broken down and categorised in non-monetary terms on a plus/minus 7 point scale.[47] We believe that this will help to break the reliance on monetisation and will lead to a more effective communication of environmental impacts. Where possible and appropriate a quantified or monetised assessment should be included alongside this rating, and all impacts should be suitably supported by background analysis and a descriptive assessment to ensure that the true consequences of the impact—beyond numbers—is understood. Monetisation should not be viewed as the only suitable form of assessment, or indeed as appropriate in all situations. Crucially, monetised impacts should not be viewed by decision makers as somehow more valid or significant than non-monetised values. We also urge the BRE to work more closely with academic groups and the private sector to investigate and consider techniques used successfully elsewhere for environmental assessment.

49. This difficulty in accurately assessing environmental impacts also goes some way to accounting for the neglect of environmental issues in RIAs. Environmental impacts may be identified but then not included in the RIA due to uncertainty over how accurately to assess and present them. In their oral evidence, both Ofgem and Anglian were keen to stress that inadequate analysis and assessment of environmental issues could well be attributed above all to the difficult and discouraging nature of such work, rather than a lack of awareness or a wilful refusal to consider environmental issues. Steve Smith of Ofgem related the difficulties experienced in carrying out their impact assessments:

"I do not think people are unwilling but it is the section they come to and think 'Gosh, this is actually really difficult', and it does not lend itself as well to the financial analysis you do in the other sections of the IA. I think it is all about the commitment you get from whoever is producing the document to do what would be the hardest part. […] [I]n many of the policy issues we deal with it is the most difficult part to actually work out what the sustainability aspects of the proposal are."[48]

50. Difficulties encountered when assessing impacts may also lead to assessments based on incorrect assumptions, the omission of crucial aspects of a particular impact, or poor presentation which does not adequately express the impact's significance. The accuracy of the assumptions behind figures is a particularly crucial factor in determining the value of an impact assessment. Any value for an impact assessment will necessarily be based on various assumptions, assigned values, estimations and predictions. Impact assessments are unavoidably a "best guess" rather than indicators of certainty, and current guidance quite rightly encourages officials to acknowledge this by highlighting any uncertainty, spelling out assumptions and presenting a range of values where appropriate.[49]

51. However, it is vital that officials are provided with the expertise and support to ensure that any assumption acting as the basis for an impact assessment is as well-informed as possible. For this to happen officials must have clear and helpful guidance; the appropriate training and knowledge, in addition to access to more specialised expertise within the department; a thorough consultation programme; and a clear process of review, scrutiny and evaluation.[50]

52. As we have seen, RIAs remain undeniably focused on cost-benefit analysis. This not only marginalises issues which do not easily lend themselves to monetisation, but also risks reducing the full worth of a policy to a narrow balancing of out-of-context figures. Simon Bullock, Economic Adviser at Friends of the Earth, expressed particular concern about the need to move from a purely cost-benefit approach to a more integrated style of assessment:

"one of the difficulties with cost benefit analysis generally is that it does reduce what should be political complex decisions to technical bureaucratic ones. […]. By reducing everything to one figure, and the costs exceed the benefits, that hides that complexity."[51]

53. It is precisely for this reason that any monetised figures need to be supported by qualitative or descriptive analysis, so that policy-makers can recognise the issues behind the figures. Similarly, it is essential that any assumptions used as the basis for figures are also clearly laid out in the document, so that the figures can be valued appropriately, and effectively challenged if necessary. BRE guidance currently recommends that policy makers assess impacts on three levels—a full description, quantification and monetisation. The NAO's briefing on sustainable development in RIAs praised the impact assessment for the Electricity and Gas (Energy Efficiency Obligations) Order 2004, which set out environmental impacts under all of these categories.[52] However, it noted that this was the only RIA in its sample of ten which demonstrated good practice in this respect.[53] Many RIAs are therefore failing to describe environmental impacts adequately, leaving certain aspects ambiguous or entirely unexplained.

54. Similarly, although we are in favour of a short summary sheet for the RIA document as recommended in our last report,[54] we are concerned that an increasing reliance on short, monetisation-based summary sheets might lead to decisions being made without considering the full conflicts, assumptions and difficulties behind the figures, or indeed what the impacts actually mean. This is a particular problem for environmental impacts, which we believe are currently neglected on the summary sheet due to the emphasis on monetised figures and trading costs against benefits.

55. We believe that one way to counteract this, and to reinforce the role of RIAs as a mechanism for implementing the overarching policy objective of sustainable development, would be to raise the profile of assessment against sustainable development on the summary sheet and throughout the document. After all, it must be remembered that the concept of sustainable development encompasses far more than simply environmental issues, even though the term is most often used in that context. As a set of principles embracing environmental, social and economic development, sustainable development represents the umbrella concept beneath which truly integrated policy decisions can be formed, and it must therefore be placed at the centre of impact assessment.

56. The consultation document produced by the BRE provides one mention of sustainable development on the summary sheet, whereby a simple "yes/no" answer is required for the question "Does the policy comply with sustainable development principles?"[55] Although we were encouraged to see a requirement to consider sustainable development, we were disappointed that it is so superficial. We believe that an answer of this type fails to acknowledge the different aspects of sustainable development, provide the Minister signing off the RIA with any concept of the nuances involved in reaching this conclusion, or present officials completing RIAs with any strict criteria for making such a claim.

57. We recommend that the RIA summary sheet should require a yes/no answer for each of the principles of sustainable development,[56] and that a separate section prominent within the main body of the document (referenced from the summary sheet) covers the assessment behind each of these responses in more detail. Sustainable development can only be fully integrated into policy making if it is recognised as the definitive component for developing and reviewing impact assessments.[57]

58. We are encouraged by the development of evaluation tools such as DEFRA's 'Stretching the Web' tool which facilitate the comparison of different sectors of impact through pictorial means, shifting the emphasis from trading off monetised figures in a table to a consideration of the overall worth of the policy across all sectors.[58] We would ask the BRE to examine and develop these models with a view to introducing such tools in the RIA structure in the future.

40 Better Regulation Executive Guidance for RIAs : Quantification http://www.cabinetoffice.gov.uk/regulation/ria/ria_guidance/costs_and_benefits/quantification.asp Back

41 The Treasury Green Book, Annex 2, Valuing Environmental Impactsgives advice on covering many environmental impacts, and frequently begins each section by commenting on the 'difficulty' or 'complexity' of placing a value on these impacts - see http://greenbook.treasury.gov.uk/annex02.htm#valuing Back

42 Better Regulation Executive Guidance for RIAs, Analysing the Benefits http://www.cabinetoffice.gov.uk/regulation/ria/ria_guidance/costs_and_benefits/analysing_benefits.asp Back

43 Q 94 Back

44 Environmental Audit Committee, Pre-Budget 2004 and Budget 2005: Tax, Appraisal and the Environment, paragraph 53. Back

45 See, for example, Ev 10 and Ev 12. Back

46 National Audit Office, Evaluation of Regulatory Impact Assessments 2005-06, HC 1305 of session 2005-2006 Text Box 15, page 19. Back

47 Environmental Audit Committee, Pre-Budget 2004 and Budget 2005: Tax, Appraisal and the Environment, paragraph 55. Back

48 Q 77 [Mr Smith] Back

49 Better Regulation Executive Guidance for RIAs, Testing your Assumptions http://www.cabinetoffice.gov.uk/regulation/ria/ria_guidance/costs_and_benefits/testing_assumptions.asp Back

50 These aspects are dealt with in greater detail in Chapters 6 and 7 of this report. Back

51 Q 113 Back