House of Commons - Environment, Transport and Regional Affairs

Select Committee on Environment, Transport and Regional Affairs Appendices to the Minutes of Evidence

Supplementary memorandum by Declan Gaffney and Dr Jean Shaoul (FLU 06A)

1. Since submitting our evidence, the government has just published a document[5] setting out the methodology for evaluating the PPP proposal for London Underground. The PPP will be compared against publicly funded procurement (PSC) and bond finance. This raises a number of points that need to be considered in the present inquiry.

2. LUL states that it will test the PPP against a bond-financed option (paragraph 4.5). this should remove a distorting factor that has been present in the economic appraisal of earlier PFI/PPP developments. Most Public Sector Comparators (PSCs) have been based on the convention that the costs of public sector investment have to be met in the year in which they occur: in other words, the option for the public sector of spreading costs over time through financing has been ignored. The discounted cashflow approach to measuring "value for money" (VFM), which the Treasury insists on for all investment programmes including London Underground, favours options which defer expenditure over those which have high costs in early years. Taken together with the assumption that the public sector cannot spread expenditure over time through financing, this approach has created an artificial advantage for PFI/PPP options, where costs are spread over a period of 20 to 35 years. Testing the PPP against a public sector option to be financed through borrowing would go some way towards making the comparative appraisal more robust, subject to the reservations set out below. The adoption of this new approach for a scheme that will attract unprecedented levels of public scrutiny suggests a recognition that the previous approach to PSCs have been flawed. The claims that PFI "cost savings" of 17 per cent frequently cited[6] in recent weeks are based on comparisons using this earlier approach, and should be retired in the light of the proposed methodology.

3. The document raises several additional issues relating to the value for money comparisons. The government has said that a comparison will be made between the PPP, bond financing and conventional public procurement. This needs clarification in relation to "conventional public procurement" since presently LU receives grants not loans. The methodology will use discounting cash flow techniques that take account of the time value of money and produce for each scheme, a single measure—the net present value of the difference between the stream of payments and receipts. The implication, but this is not stated explicitly, is that the government will choose the scheme that provides the highest NPV or lowest NPC (net present cost). However, the final paragraphs of the document (paragraph 9.1 and 9.2) suggest that the decision may be taken on other grounds as well, but this is not explained and needs clarification. We have examined other PFI business cases where it was unclear how the VFM comparison based on NPV could have resulted in the choice of a PFI option.[7] While there are several criteria for making investment decisions, of which the VFM comparison is only one, these should be clearly stated.

4. While the government has undertaken to publish more material to demonstrate LUL's commitment to making a rigorous comparison, the emphasis appears to be on showing the bidders, not the public, that the comparison will be made in an open and appropriate way (paragraph 1.2). the results of the value for money comparison will only be made available after contracts have been signed. In other words, neither the Mayor of London who will ultimately be responsible for ensuring that the Infrastructure Service Charge is paid to the PPPs or the public will be able to make any input into the decision making process.

5. The comparison will be made between a PPP, and a publicly run LU with conventional procurement and bond finance. However, LU has now been restructured into four units explicitly for the PPP. This is not necessarily the ideal or cheapest form of organisation if LU's operations are to remain in public ownership. It also means that LU's organisation is being driven by the PPP before it has been established that the PPP is value for money or even affordable.

6. The choice will depend upon which provides the best value for money. However, this is a moving feast. Estimates of value for money depend upon a number of factors where judgement rather than evidence plays a determining role. VFM depends upon: (i) what is included or excluded, (ii) how it is counted and (iii) over what period. We discuss each of these in turn. The VFM methodology used is known as options appraisal based on NPV of the cash flows over the life of the project.

(i) What is included or excluded

7. The document released by PwC,[8] LU's financial advisors, and government press releases assume that the private sector will be able to make cost "savings" on a £15 billion investment programme of about £2.5 billion, so the PPP will spend about £12.53 billion over 15 years. It does not explain where these savings will come from and why LU could not make similar savings. It bases this assumption on "private sector efficiencies." However, there is a lack of clarity as to the meaning of efficiency, as opposed to economy and effectiveness. Efficiency measures the relationship between inputs and outputs. Greater efficiency means that there are more outputs for the same or fewer inputs. Economy relates to lower costs. The research literature as it relates to comparisons between public and private sector efficiency is not characterised by carefully controlled studies that explain the cause of any observed differences, which may lie in a different regulatory and pricing regime, wage structure, social and legal obligations, etc.

8. Some commentators point to the "efficiency savings" made by other privatised industries, particularly rail and water. However, these are "own goals." Services, however defined, have declined in the national railways, as infrastructure investment by Railtrack has not risen commensurate with need, expectation or commitments, according to the Booz Allen and Hamilton report.[9] The much vaunted "efficiency savings" that were assumed to follow water privatisation came not so much from lower operating costs but from a lower level of capital expenditure on both renewals and investment than that predicted at privatisation.[10] Indeed, the level of renewals meant that the industry was in fact running down the asset base in ways that threatened future delivery. These "savings" were in part at least the result of inflated cost estimates that provided the basis for price increases. More importantly from the perspective of the PPP proposals while some performance targets were set for the water companies, many of the targets were not achieved[11] as the failure of the public water supply to West Yorkshire in 1995 and leakages running at a higher level than pre-privatisation testify. Thus, the "efficiency savings" were made at the expense of consumers and the public at large. None of this can be described as efficiency, much less effectiveness, as opposed to economy. Furthermore, both made more improvements in productivity and efficiency before privatisation, which suggests that the private sector does not have a monopoly on efficient management techniques. Yet, proponents of the PPP believe that private sector operators can make similar savings in investment and maintenance expenditure while still delivering the necessary standards of performance.

9. The evidence of efficiency savings from contracting out is also inconclusive.[12] While there are studies showing savings from contracting out manual services, much of this is due to the reappraisal, redesign and re-specification of the services when they were put out to tender rather than superior private sector work practices per se. The rest of the savings were achieved by a reduction in jobs, wages and working conditions,[13][14] economy rather than more efficient work practices. Indeed, several commentators, including the Arthur Andersen report on PFI[15] make the point that with the increase in efficiency achieved by the public sector over the years, there are few operational savings that can be made by the private sector.

(www,treasury-projects-taskforce.gov.uk/series1/anderson).

10. Other research,[16] using the national accounts, does not show any savings due to contracting out. Indeed, it suggests that contracting out has proved more expensive. Recalculating public expenditure for 1996, assuming the external purchasing/wages split of 1986 and up to 12 per cent wage increase on 1991 levels for those workers who left the public sector, indicates that public expenditure would have been lower without the increased external procurement. While clearly not definitive, this does mean that the extent of the cost savings is therefore unclear as other previously cited and more detailed studies have shown.

11. However, the outcome of the appraisal will also depend on LUL's estimation of the risks it will retain under any public sector option and which will be transferred to the private sector under the PPP contracts,. Unfortunately the LUL paper does not specify the extent to which risk transfer will be assumed to take place under a public sector contract. for example, it would seem reasonable to assume that any construction contracts let under a bond financed option would transfer the risk of delays and cost overruns to the contractors in accordance with best practice in the public sector, with strong penalty clauses. This, after all, is what the PPP companies will do when they sub-contract construction work. We consider it unfortunate that the methodology does not explicitly take account of the possibilities for risk transfer under public sector procurement.

12. LUL says that "the data source for quantifying [risks] is principally LUL experience from previous projects, for example on costs diverging from expectations." There is the possibility of serious distortion if overruns on major construction projects such as the JLE are cited as typical. The risk of cost and time overruns on large complex schemes that involve tunnelling is infinitely greater than for the sort of largely routine tasks the PPP will be engaged in. Any estimate of average cost overruns using evidence from projects such as the JLE would be quite spurious in this context.

13. It is worth pointing out that the risk assessment methodology traditionally carried out for PFI projects does not address the question of the additional risks that are created by PFI/PPP. In this context, it should be noted that the document admits that the new organisational structure created for the PPP has already generated and will generate extra costs, although the document acknowledges that these will be excluded from the PSC. (The PPP must be consuming a tremendous amount of human and financial resources. There are the additional manpower costs incurred through creating three separate organisations, rather than one organisation with some administrative economies of scale. There will also be the extra contract management costs needed to supervise the contracts with the three PPP consortia rather than a relationship with different departments in the same organisation.) This fragmentation will raise problems of planning, co-ordination and safety—the same issue that lay at the heart of the Paddington disaster. The publicly owned company will retain responsibility for safety, which has been declining continuously over the last five years as a result of the removal of the operating subsidy and reduction in governments grants. These additional risks are unlikely to be dealt with explicitly in the appraisal. In practice, this simply means that LU will carry the can for any slipshod work by the infracos. It is far from clear how performance standards can be set, monitored and legally enforced, if necessary. If this is indeed the case, this will have implications for the performance and safety of the Underground that are not reflected in the comparisons between the different methods of financing the investment.

(ii) How it is counted

14. Secondly, the method of counting is the discounted cash flow (DCF) approach which looks at whole life cash flows and discounts those made in the future. According to the Net Present Value (NPV) rule, the project with the largest positive NPV of the discounted cash flows is chosen. But as many researchers have noted, the usefulness of the NPV is severely limited.[17] Firstly, it is one of a number of methods of project appraisal used by the private sector because it is assumed to maximise shareholder or owner wealth. It is therefore inappropriate in most public sector organisations, which are not wealth maximisers in that sense. Indeed, LU has been in public ownership since the 1930s because it could not maximise shareholder wealth. Since then it has been run as a public service. Secondly, the NPV is not appropriate when capital is rationed as it is in this case. Yet the ostensible reason for the PPP is that the government has not or will not provide the cash that the Underground needs. Thirdly, it applies only where in those cases where the investment opportunity instantly disappears if it is not immediately undertaken. But in the majority of cases there is usually a time period over which the investment may be undertaken. Lastly, many of the investment for LU are in fact interdependent.

15. As well as these formal assumptions of the NPV rule, which are not met in practice, there are a number of practical problems. A crucial aspect of the methodology is the choice of discount rate. Following Treasury rules[18] a discount rate of 6 per cent is used because it does not preclude the private sector. In other words, the rate has been set to favour the private sector. As we have shown on our BMJ article, previously cited, VFM comparisons are very sensitive to changes in the discount rate from 6 per cent to 5.7 per cent.

16. A further point is that the low interest rates expected for a bond issue will have a crucial impact on the value for money judgement. However, the methodology paper suggests that the PSC will not be costed on the basis of a bond interest rate, but rather using the Treasury Test Discount rate, which is higher. The difference in the two rates is estimated at up to 2.5 per cent by LUL (6.0-6.5 per cent nominal for a bond issue, 8.65 per cent for the Treasury rate). The use of the Treasury discount rate to model the PSC could therefore make a material difference to the outcome of the appraisal, as is recognised by LUL who note "using the standard discount rate . . . Would not reflect the simple cash cost of a bond . . . and could be said to disadvantaged the public sector option." LUL states that it will look at both rates, but unfortunately appears to qualify this by saying it will "use the bond interest rate as a sensitivity." We would suggest that this gets things precisely the wrong way around. The PSC modelled using a bond interest rate is the object of interest here, not a "sensitivity", and any appraisal using a different rate would deservedly be taken as evading the central issue.

17. The argument for using the Treasury rate to model the PSC is that the financing costs of a public sector bond issue do not represent the full costs, which include the (non-cash) costs of risks and guarantees. A similar line of argument is also used in Lord Currie's paper (see footnote 29). The argument, which is by no means uncontroversial, has been extensively canvassed by a number of economists in recent years. The important point to recognise is that it is quite irrelevant in this context. The cost of project risks such as cost overruns is already dealt with separately in the appraisal, by adjusting the PSC values upwards to take account of retained risks. This is made perfectly clear in section 5 of the LU methodology paper. There is no cost which we are aware of which is not already dealt with elsewhere that would justify adjusting the interest rate upwards as well. LUL states "[a bond rate] would not reflect the cost to the public sector of the TfL guarantee that would be necessary to enhance the credit of the debt to a high investment grade rating" (paragraph 4.5.5). the cost of that guarantee is the risk that LUL would not have sufficient revenue to meet debt service costs. To the extent that this reflects project risks, this is accounted for elsewhere. To the extent that it reflects non-project risks (such as economic recession) there is no reason why it should affect a public sector option more than the PPP (paragraph 2.2).

(iii) The period counted

18. Thirdly , the NPV approach looks at whole life cash flows and discounts those made in the future. While this appears to have merit, it necessarily favours annual payments spread over the whole life of the project over up front payments. Therefore, the methodology is loaded against conventional public procurement. While this is generally true for new builds under PFI/PPP, because the investment will be spread out over a long period, this factor has slightly less impact on the LU PPP, which to some extent is a continuous stream of small projects.

19. Despite the fact that the NPV approach is used by some private sector organisations in some circumstances, it is by no means used in the majority of cases or as the sole criterion. One of the reasons that relatively few private sector organisations use this approach is the difficulty in predicting cash flows for very long into the future due to uncertainty.

20. The usefulness of NPV techniques, in the context of a 30 or 15 year concession under the conditions of the LU PPP, is limited by a further factor. The PPP scheme relates to maintenance and upgrades to the infrastructure (the output specification has not been made public). But there is incomplete information about the status and condition of much of the assets. Whereas nearly all PFI projects are based on new builds, the PPP is based on old assets whose condition is unknown. This makes it even more difficult to be precise about what needs to be done, and whether and how it can be done, to ensure the outputs, for example, the number of trains per hour on a given line. If the infracos were to bear all the risks of, say, a tunnel collapse, this would make the scheme inordinately expensive. Hence the public sector must and will bear the risk. But that in turn means that it makes little sense to estimate costs over a 30 year period. Indeed, the methodology document implicitly recognises this since it says in paragraph 5.2.3 that there will be Periodic Reviews every seven to eight years to "give LU the opportunity to restate its performance requirements and allow the Infracos to put forward revised charges". Several points follow from this. Firstly, the VFM comparisons will be based on a methodology of 30 year NPV calculations that favour the PPP, but in reality, the agreements will run for seven to eight years and hence the actual costs and benefits will be different from those estimated in the business case. Only the first seven to eight years data will be very meaningful. Hence the VFM comparison will have little practical validity. Secondly, since the infrastructure service charge will be determined by negotiation not regulation, the infracos will have LU over a barrel.

21. All this means that the VFM is of limited relevance to the public debate and little reliance can in fact be placed upon the VFM comparisons, which are necessarily ex ante. The use of a bond financed PSC will make it considerably more difficult for the PPP to demonstrate value for money. However, the use of a high interest rate for the PSC tips the balance back towards the PPP. The aim of carrying out the appraisal in a manner that will withstand criticism would be undermined by any inflation of the PSC values.

22. These are a priori problems with the VFM methodology. It is worth considering how useful these appraisals have been in practice or ex post facto. The recent Arthur Andersen report evaluating PFI has been widely cited. It examined 30 business cases, out of 90 possible projects where there were PSC comparisons, without explicitly identifying the projects. It found that there were savings but these came from just a few projects. But the report failed to state, although it could be inferred from other data in the report, that the project that accounted for the most savings, labelled "S" in their tables was the IT project, NIRS2, carried out by Andersen Consulting for the Benefits agency. This one project's estimates of NPV accounted for 55 per cent of the "savings" and 80 per cent of the risk transfer supposedly ascribed to PFI in the business cases. Andersen Consulting, Arthur Andersen's sister consulting arm, won the contract as it priced the deal considerably less than the other bidders. However, even more importantly, the project has been a spectacular failure that has gone massively over budget and years overdue. As Andersen's paid a fine of only £3.9 million, the public sector and the public has borne the costs. Other IT PFI projects have also failed dismally. While IT projects are notoriously difficult to get right, the relevance here is that their business cases showed that PFI was preferable to public procurement, in part if not wholly because of the risk transfer. Sadly, this was not the case. The NAO noted that other projects turned out to be more costly than their business cases suggested.

23. There are also pragmatic pressures that undermine the VFM comparisons. As PFI is usually the only way of obtaining the much needed new school or hospital, there is an incentive to rig the comparison in favour of PFI. Sarah Wood, Director of finance and performance review at Birmingham City Council, said she had "reservations" about the figures supporting a local PFI project to rebuild seven schools and refurbish a further three. "Of course publicly I've got to say that it is value for money. Privately, I am not sure," she told delegates at the CIPFA Scottish branch conference, "But it is the only game in town. It is the way you do get money into your services.[19] That is a sentiment we have heard expressed many times. Under these conditions, PFI cases are prepared and constructed so as to demonstrate VFM that subsequently prove to be less than expected.[20]

24. As a result of all these factors, it is difficult to place much reliance on the VFM comparison. In any event it is only one criterion among several. These include: affordability, the implications of the PPP structure for planning, co-ordination and safety, future investment to expand the network, democratic accountability and stakeholder consent including the wishes of those who will have to fund it—the passengers, tax payers both local and national, and the people of London. All of these criteria give rise to problems with the PPP, as we argued in our earlier submission.

Declan Gaffney

School of Public Policy, University College London

Dr Jean Shaoul

School of Accounting and Finance, Manchester University

April 2000

5 "London Underground Public-Private Partnership: Methodology for preparing the Public Sector Comparator," March 2000. Back

6 See for example D Currie, "Regulation Initiative: Funding the London Underground," London Business School, March 2000. Back

7 For example, the full business case of the Royal Infirmary at Edinburgh, as explained in Shaoul J, " Looking Glass World of PFI", Public Finance, 29 January 1999, the PFI case for Pimlico School as explained in "The Lessons of Pimlico," Public Finance, October 29 1999, PFI in the NHS: Is there an economic case?" British Medical Journal, 10 July 1999, and the summary business case for the UK Passport Agency PFI. Back

8 PriceWaterhouseCooper, "London Underground PPP-Briefing Document," December 1999. Back

9 Booz Allen and Hamilton, "Railtrack's performance in the period 1999-2001, Report to the Office of the Rail Regulator, 1999. Back

10 Shaoul, J, "A Critical Financial Analysis of the Post-Privatisation Performance of the Water industry in England-Wales," Critical Perspectives on Accounting, 1997, Vol 8, pp 479-505. Back

11 Schofield, R, and Shaoul, "Regulating the Water Industry: By Any Standards?" Utilities Law review, Vol 8, Issue 2, Mar-April 1997, pp 56-70. Back