5Evaluation
31.During the course of our inquiry, it became clear to us that the Government has devoted considerable effort to improving the way in which it delivers its major projects. We saw in chapter four the extensive appraisal process which prospective projects go through before commencement. Once underway, the IPA regularly reviews major projects’ progress until they are sufficiently advanced to exit the GMPP or, in fewer instances, deemed beyond salvation and terminated. However, it also became apparent that the focus is on the delivery of projects on time and on budget. This is, to some extent, understandable. The difficulty of forecasting benefits was stressed to us.42 It is hard to accurately predict cost and completion dates for large, complex projects which might take several years to complete. and harder still to predict benefits accurately. These could, for instance, be significantly impacted by the wider policy environment.
32.It was suggested to us that, despite the scrutiny of business cases we discussed in the previous chapter, the link between prospective projects and the policy goals they are supposed to achieve is not as strong as it should be. Projects can, for example, be developed before the needs they are designed to address have been established: Professor Lewis, of the University of Bath and a theme leader for Project X, referred to this as “benefit hunting”.43 Professor Williams, University of Hull and a theme leader for Project X, said that:
projects should be there to address a need and, having decided to address a need, you should then think about how to address it. There is quite an emphasis on, “Here is a solution and this is a solution we will use” and then thinking about how it addresses a need.44
33.The case for HS2 seems to highlight this. The IfG note that it has variously been presented as a means to reduce journey time, of increasing capacity to meet growing demand, and a vehicle for regenerating the West Midlands.45 It may, of course, be all of those things but it hints at retrospective justification for a controversial project.
34.Similarly, once projects commence, the focus is on successful completion rather than on the extent of any delays and cost overruns. Much less attention is given to assessing whether projects deliver the benefits that they promised. Professor Williams told us:
We also have the impression now where the emphasis perhaps of the IPA is on the delivery of the project as defined: getting it on time, on cost, as defined at the beginning, and the actual benefits that the project is there to deliver sometimes get—I will not say lost but there is less priority put on that than the actual delivery of the project.46
35.The IPA’s oversight of projects ends when they leave the GMPP and there is no centralised, systematic post hoc evaluation. Both the NAO and the IfG have been critical of the insufficiently robust evaluation of whether projects actually deliver their anticipated benefits. This, in turn, limits Government’s ability to learn from previous projects when planning new ones.
36.The NAO found that practice varied between departments and projects. Many were either monitoring projects to see whether they were achieving the benefits or conducted a post hoc review. But this was not always the case and, in some instances, project objectives and benefits were not specified in a way that made it possible to judge whether they had been realised.47 The IfG concluded that:
Evaluation is failing to capture whether or not infrastructure projects are actually achieving the outcomes they promised, despite evidence that by doing so they could improve future forecasting and project selection.48
37.The difficulty of accurately forecasting the benefits of a project is clear. It can also be difficult to assess whether they have actually achieved what they were designed to achieve. Nonetheless, whilst there is considerable attention paid to improving project management, major projects are not an end in themselves but exist to achieve public policy goals. Robust ex ante scrutiny of the link between projects and the needs that they are supposed to address, and proper post hoc evaluation of whether they have done so, cannot be overlooked in the focus on improving project delivery.
45 G. Atkins et al What’s Wrong with Infrastructure Decision Making Institute for Government 2017 p.3
47 NAO Projects Leaving the Government Major Projects Portfolio HC1620 Session 2017–19, p.24–27; G Atkins et al What’s Wrong With Infrastructure Decision-Making: Conclusions from Six UK Case Studies, Institute for Government, 2018, p.28
48 G Atkins et al What’s Wrong With Infrastructure Decision-Making: Conclusions from Six UK Case Studies, Institute for Government, 2018, p.33
Published: 5 November 2019