2Measuring and reporting performance
What regulators want to achieve for consumers
19.The National Audit Office’s report concluded that the regulators have not been specific enough in defining the overall outcomes they want to achieve for consumers.34 We agree that it is important for any public body to be clear about and accountable for what it is trying to achieve, and what criteria it will use to judge whether it is being successful. Each regulator sets overall intended consumer outcomes at a high level, such as affordability, high quality services or value for money. But they have not translated these aims into measurable criteria, such as targets, that they could use to assess progress.35
20.The regulators all highlighted the difficulty in setting meaningful targets and outcome measures when they do not have direct control over all of the factors that impact on consumer outcomes.36 We also recognise that, in some areas, setting specific targets can create perverse incentives and unintended consequences.37
21.However, the regulators acknowledged that the outcomes they have set can be too high-level and vague, and that there would be value in being more specific about what they are trying to achieve. Ofgem noted that from 2020 it will need to report on the impact of the energy price cap, and whether the market is working well enough for consumers for the cap to be removed. In order to do this, Ofgem will need to be define what conditions need to be met in order to judge whether the market is working well enough for consumers.38 Ofcom told us that it has work under way to develop a set of metrics that feel real to consumers, for example relating to better access and more affordable prices, and which Parliament and the public could use to track progress.39 Ofwat also noted how it was working with other water regulators—the Environment Agency and Drinking Water Inspectorate - to set longer-term targets for the water sector.40
Regulators’ understanding of their own impact
22.Regulators need to measure their performance and assess their impact to make sure that they are achieving their objectives and take corrective action where necessary. The National Audit Office’s report found that the performance information that regulators use is focused on data that tracks the progress of individual projects and programmes. The regulators do not typically use information on consumer outcomes to monitor and assess their own overall performance.41
23.The regulators highlighted the difficulty in isolating the effects of their own actions from other factors that affect consumer outcomes, such as economic conditions or the effects of wider government policy. But they also recognised that they can do more to measure their effectiveness.42 The FCA has made the most progress in starting work to better understand its impact and influence, and has committed to a programme of evaluations to examine whether specific interventions worked out as expected.43 The other regulators recognised the benefits of these exercises and have similarly committed to undertaking more evaluations. Ofcom told us that it is planning to do serious evaluations of two or three of its interventions on an annual basis.44 Similarly, Ofgem told us that it had recently published its first review and it plans to more of these in the future.45 Ofwat also noted that it currently reviews its five-yearly price control process, but is committed to expanding its approach to include reviews across the different levers and interventions that it uses.46
24.The regulators also told us about some of the ways they were trying to improve their regular performance information. Ofwat told us that while it currently monitors data on individual companies, it is developing sector-wide dashboards with information on consumer outcomes.47 Ofcom also explained that it plans to turn the information it uses on the progress of specific interventions into more effective tools for holding itself to account for protecting consumers.48
Regulators’ publicly available information
25.The information that independent regulators make publicly available should enable consumers, Parliament and wider stakeholders to hold them accountable for protecting consumer’s interests. The regulators told us that they could do more to help the public understand what they are doing to protect consumers and why. Ofcom told us it hasn’t done enough to publicly explain how it has developed its priorities based on its consumer research and the changing issues in the market.49 Ofwat similarly noted its intention to communicate more clearly its understanding of the challenges consumers face and its own decisions on how to address them.50 Ofgem told us that it has made progress by introducing its annual ‘Consumer Impact Report’ which attempts to quantify the total expected benefit to consumers of Ofgem’s interventions over the past year.51
26.We also challenged the regulators about whether their public reporting helps consumers understand what difference they are making in practice. The FCA told us that it could do more to report on the effectiveness of its actions, and noted that it is working to improve this, particularly in its annual report.52 Ofwat acknowledged that, to date, it has not been clear or consistent in reporting on consumer outcomes, and noted that it planned to report annually on progress against a new set of high-level consumer outcomes.53 Ofcom also explained that it planned to use its annual plans and annual reports to provide more transparency about what they have and have not achieved, and a clearer link between what they set out to achieve and what progress they are making.54
27.We asked the regulators about their websites, and the type and consistency of information provided online to the general public. While there is some good practice, for example Ofcom’s website is generally accessible, there are several areas where regulators’ websites and the documents they contain could be improved. In particular, the regulators could do more to improve the use of plain English in their online material and minimise technical jargon. The layout of the webpages for consumers could also be improved, to make it as easy as possible for consumers to access the information they need.55
34 C&AG’s Report, paras 12, 22
35 C&AG’s Report, paras 2.13–2.14
36 Q 69, 74–75
37 C&AG’s Report, para 2.14
38 Q 75
39 Q 69, 83
40 Q 74
41 C&AG Report, paras 3.6–3.10
42 Qq 69–70, 74–75; C&AG’s report, para
43 Qq 69, 74; C&AG Report, para 3.14
44 Q 69
45 Q 76
46 Q 74
47 Qq 74–79
48 Q 80
49 Q 83
50 Q 84
51 Qq 75–76
52 Q 74
53 Q 74
54 Qq 71–73, 83
55 Qq 96–100
Published: 12 July 2019