Supplementary memorandum by Declan Gaffney
and Dr Jean Shaoul (FLU 06A)
1. Since submitting our evidence, the government
has just published a document[5]
setting out the methodology for evaluating the PPP proposal for
London Underground. The PPP will be compared against publicly
funded procurement (PSC) and bond finance. This raises a number
of points that need to be considered in the present inquiry.
2. LUL states that it will test the PPP
against a bond-financed option (paragraph 4.5). this should remove
a distorting factor that has been present in the economic appraisal
of earlier PFI/PPP developments. Most Public Sector Comparators
(PSCs) have been based on the convention that the costs of public
sector investment have to be met in the year in which they occur:
in other words, the option for the public sector of spreading
costs over time through financing has been ignored. The discounted
cashflow approach to measuring "value for money" (VFM),
which the Treasury insists on for all investment programmes including
London Underground, favours options which defer expenditure over
those which have high costs in early years. Taken together with
the assumption that the public sector cannot spread expenditure
over time through financing, this approach has created an artificial
advantage for PFI/PPP options, where costs are spread over a period
of 20 to 35 years. Testing the PPP against a public sector option
to be financed through borrowing would go some way towards making
the comparative appraisal more robust, subject to the reservations
set out below. The adoption of this new approach for a scheme
that will attract unprecedented levels of public scrutiny suggests
a recognition that the previous approach to PSCs have been flawed.
The claims that PFI "cost savings" of 17 per cent frequently
cited[6]
in recent weeks are based on comparisons using this earlier approach,
and should be retired in the light of the proposed methodology.
3. The document raises several additional
issues relating to the value for money comparisons. The government
has said that a comparison will be made between the PPP, bond
financing and conventional public procurement. This needs clarification
in relation to "conventional public procurement" since
presently LU receives grants not loans. The methodology will use
discounting cash flow techniques that take account of the time
value of money and produce for each scheme, a single measurethe
net present value of the difference between the stream of payments
and receipts. The implication, but this is not stated explicitly,
is that the government will choose the scheme that provides the
highest NPV or lowest NPC (net present cost). However, the final
paragraphs of the document (paragraph 9.1 and 9.2) suggest that
the decision may be taken on other grounds as well, but this is
not explained and needs clarification. We have examined other
PFI business cases where it was unclear how the VFM comparison
based on NPV could have resulted in the choice of a PFI option.[7]
While there are several criteria for making investment decisions,
of which the VFM comparison is only one, these should be clearly
stated.
4. While the government has undertaken to
publish more material to demonstrate LUL's commitment to making
a rigorous comparison, the emphasis appears to be on showing the
bidders, not the public, that the comparison will be made in an
open and appropriate way (paragraph 1.2). the results of the value
for money comparison will only be made available after contracts
have been signed. In other words, neither the Mayor of London
who will ultimately be responsible for ensuring that the Infrastructure
Service Charge is paid to the PPPs or the public will be able
to make any input into the decision making process.
5. The comparison will be made between a
PPP, and a publicly run LU with conventional procurement and bond
finance. However, LU has now been restructured into four units
explicitly for the PPP. This is not necessarily the ideal or cheapest
form of organisation if LU's operations are to remain in public
ownership. It also means that LU's organisation is being driven
by the PPP before it has been established that the PPP is value
for money or even affordable.
6. The choice will depend upon which provides
the best value for money. However, this is a moving feast. Estimates
of value for money depend upon a number of factors where judgement
rather than evidence plays a determining role. VFM depends upon:
(i) what is included or excluded, (ii) how it is counted and (iii)
over what period. We discuss each of these in turn. The VFM methodology
used is known as options appraisal based on NPV of the cash flows
over the life of the project.
(i) What is included or excluded
7. The document released by PwC,[8]
LU's financial advisors, and government press releases assume
that the private sector will be able to make cost "savings"
on a £15 billion investment programme of about £2.5
billion, so the PPP will spend about £12.53 billion over
15 years. It does not explain where these savings will come from
and why LU could not make similar savings. It bases this assumption
on "private sector efficiencies." However, there is
a lack of clarity as to the meaning of efficiency, as opposed
to economy and effectiveness. Efficiency measures the relationship
between inputs and outputs. Greater efficiency means that there
are more outputs for the same or fewer inputs. Economy relates
to lower costs. The research literature as it relates to comparisons
between public and private sector efficiency is not characterised
by carefully controlled studies that explain the cause of any
observed differences, which may lie in a different regulatory
and pricing regime, wage structure, social and legal obligations,
etc.
8. Some commentators point to the "efficiency
savings" made by other privatised industries, particularly
rail and water. However, these are "own goals." Services,
however defined, have declined in the national railways, as infrastructure
investment by Railtrack has not risen commensurate with need,
expectation or commitments, according to the Booz Allen and Hamilton
report.[9]
The much vaunted "efficiency savings" that were assumed
to follow water privatisation came not so much from lower operating
costs but from a lower level of capital expenditure on both renewals
and investment than that predicted at privatisation.[10]
Indeed, the level of renewals meant that the industry was in fact
running down the asset base in ways that threatened future delivery.
These "savings" were in part at least the result of
inflated cost estimates that provided the basis for price increases.
More importantly from the perspective of the PPP proposals while
some performance targets were set for the water companies, many
of the targets were not achieved[11]
as the failure of the public water supply to West Yorkshire in
1995 and leakages running at a higher level than pre-privatisation
testify. Thus, the "efficiency savings" were made at
the expense of consumers and the public at large. None of this
can be described as efficiency, much less effectiveness, as opposed
to economy. Furthermore, both made more improvements in productivity
and efficiency before privatisation, which suggests that the private
sector does not have a monopoly on efficient management techniques.
Yet, proponents of the PPP believe that private sector operators
can make similar savings in investment and maintenance expenditure
while still delivering the necessary standards of performance.
9. The evidence of efficiency savings from
contracting out is also inconclusive.[12]
While there are studies showing savings from contracting out manual
services, much of this is due to the reappraisal, redesign and
re-specification of the services when they were put out to tender
rather than superior private sector work practices per se.
The rest of the savings were achieved by a reduction in jobs,
wages and working conditions,[13][14]
economy rather than more efficient work practices. Indeed, several
commentators, including the Arthur Andersen report on PFI[15]
make the point that with the increase in efficiency achieved by
the public sector over the years, there are few operational savings
that can be made by the private sector.
(www,treasury-projects-taskforce.gov.uk/series1/anderson).
10. Other research,[16]
using the national accounts, does not show any savings due to
contracting out. Indeed, it suggests that contracting out has
proved more expensive. Recalculating public expenditure for 1996,
assuming the external purchasing/wages split of 1986 and up to
12 per cent wage increase on 1991 levels for those workers who
left the public sector, indicates that public expenditure would
have been lower without the increased external procurement. While
clearly not definitive, this does mean that the extent of the
cost savings is therefore unclear as other previously cited and
more detailed studies have shown.
11. However, the outcome of the appraisal
will also depend on LUL's estimation of the risks it will retain
under any public sector option and which will be transferred to
the private sector under the PPP contracts,. Unfortunately the
LUL paper does not specify the extent to which risk transfer will
be assumed to take place under a public sector contract. for example,
it would seem reasonable to assume that any construction contracts
let under a bond financed option would transfer the risk of delays
and cost overruns to the contractors in accordance with best practice
in the public sector, with strong penalty clauses. This, after
all, is what the PPP companies will do when they sub-contract
construction work. We consider it unfortunate that the methodology
does not explicitly take account of the possibilities for risk
transfer under public sector procurement.
12. LUL says that "the data source
for quantifying [risks] is principally LUL experience from previous
projects, for example on costs diverging from expectations."
There is the possibility of serious distortion if overruns on
major construction projects such as the JLE are cited as typical.
The risk of cost and time overruns on large complex schemes that
involve tunnelling is infinitely greater than for the sort of
largely routine tasks the PPP will be engaged in. Any estimate
of average cost overruns using evidence from projects such as
the JLE would be quite spurious in this context.
13. It is worth pointing out that the risk
assessment methodology traditionally carried out for PFI projects
does not address the question of the additional risks that are
created by PFI/PPP. In this context, it should be noted that the
document admits that the new organisational structure created
for the PPP has already generated and will generate extra costs,
although the document acknowledges that these will be excluded
from the PSC. (The PPP must be consuming a tremendous amount of
human and financial resources. There are the additional manpower
costs incurred through creating three separate organisations,
rather than one organisation with some administrative economies
of scale. There will also be the extra contract management costs
needed to supervise the contracts with the three PPP consortia
rather than a relationship with different departments in the same
organisation.) This fragmentation will raise problems of planning,
co-ordination and safetythe same issue that lay at the
heart of the Paddington disaster. The publicly owned company will
retain responsibility for safety, which has been declining continuously
over the last five years as a result of the removal of the operating
subsidy and reduction in governments grants. These additional
risks are unlikely to be dealt with explicitly in the appraisal.
In practice, this simply means that LU will carry the can for
any slipshod work by the infracos. It is far from clear how performance
standards can be set, monitored and legally enforced, if necessary.
If this is indeed the case, this will have implications for the
performance and safety of the Underground that are not reflected
in the comparisons between the different methods of financing
the investment.
(ii) How it is counted
14. Secondly, the method of counting is
the discounted cash flow (DCF) approach which looks at whole life
cash flows and discounts those made in the future. According to
the Net Present Value (NPV) rule, the project with the largest
positive NPV of the discounted cash flows is chosen. But as many
researchers have noted, the usefulness of the NPV is severely
limited.[17]
Firstly, it is one of a number of methods of project appraisal
used by the private sector because it is assumed to maximise shareholder
or owner wealth. It is therefore inappropriate in most public
sector organisations, which are not wealth maximisers in that
sense. Indeed, LU has been in public ownership since the 1930s
because it could not maximise shareholder wealth. Since then it
has been run as a public service. Secondly, the NPV is not appropriate
when capital is rationed as it is in this case. Yet the ostensible
reason for the PPP is that the government has not or will not
provide the cash that the Underground needs. Thirdly, it applies
only where in those cases where the investment opportunity instantly
disappears if it is not immediately undertaken. But in the majority
of cases there is usually a time period over which the investment
may be undertaken. Lastly, many of the investment for LU are in
fact interdependent.
15. As well as these formal assumptions
of the NPV rule, which are not met in practice, there are a number
of practical problems. A crucial aspect of the methodology is
the choice of discount rate. Following Treasury rules[18]
a discount rate of 6 per cent is used because it does not preclude
the private sector. In other words, the rate has been set to favour
the private sector. As we have shown on our BMJ article, previously
cited, VFM comparisons are very sensitive to changes in the discount
rate from 6 per cent to 5.7 per cent.
16. A further point is that the low interest
rates expected for a bond issue will have a crucial impact on
the value for money judgement. However, the methodology paper
suggests that the PSC will not be costed on the basis of a bond
interest rate, but rather using the Treasury Test Discount rate,
which is higher. The difference in the two rates is estimated
at up to 2.5 per cent by LUL (6.0-6.5 per cent nominal for a bond
issue, 8.65 per cent for the Treasury rate). The use of the Treasury
discount rate to model the PSC could therefore make a material
difference to the outcome of the appraisal, as is recognised by
LUL who note "using the standard discount rate . . . Would
not reflect the simple cash cost of a bond . . . and could be
said to disadvantaged the public sector option." LUL states
that it will look at both rates, but unfortunately appears to
qualify this by saying it will "use the bond interest rate
as a sensitivity." We would suggest that this gets things
precisely the wrong way around. The PSC modelled using a bond
interest rate is the object of interest here, not a "sensitivity",
and any appraisal using a different rate would deservedly be taken
as evading the central issue.
17. The argument for using the Treasury
rate to model the PSC is that the financing costs of a public
sector bond issue do not represent the full costs, which include
the (non-cash) costs of risks and guarantees. A similar line of
argument is also used in Lord Currie's paper (see footnote 29).
The argument, which is by no means uncontroversial, has been extensively
canvassed by a number of economists in recent years. The important
point to recognise is that it is quite irrelevant in this context.
The cost of project risks such as cost overruns is already dealt
with separately in the appraisal, by adjusting the PSC values
upwards to take account of retained risks. This is made perfectly
clear in section 5 of the LU methodology paper. There is no cost
which we are aware of which is not already dealt with elsewhere
that would justify adjusting the interest rate upwards as well.
LUL states "[a bond rate] would not reflect the cost to the
public sector of the TfL guarantee that would be necessary to
enhance the credit of the debt to a high investment grade rating"
(paragraph 4.5.5). the cost of that guarantee is the risk that
LUL would not have sufficient revenue to meet debt service costs.
To the extent that this reflects project risks, this is accounted
for elsewhere. To the extent that it reflects non-project risks
(such as economic recession) there is no reason why it should
affect a public sector option more than the PPP (paragraph 2.2).
(iii) The period counted
18. Thirdly , the NPV approach looks at
whole life cash flows and discounts those made in the future.
While this appears to have merit, it necessarily favours annual
payments spread over the whole life of the project over up front
payments. Therefore, the methodology is loaded against conventional
public procurement. While this is generally true for new builds
under PFI/PPP, because the investment will be spread out over
a long period, this factor has slightly less impact on the LU
PPP, which to some extent is a continuous stream of small projects.
19. Despite the fact that the NPV approach
is used by some private sector organisations in some circumstances,
it is by no means used in the majority of cases or as the sole
criterion. One of the reasons that relatively few private sector
organisations use this approach is the difficulty in predicting
cash flows for very long into the future due to uncertainty.
20. The usefulness of NPV techniques, in
the context of a 30 or 15 year concession under the conditions
of the LU PPP, is limited by a further factor. The PPP scheme
relates to maintenance and upgrades to the infrastructure (the
output specification has not been made public). But there is incomplete
information about the status and condition of much of the assets.
Whereas nearly all PFI projects are based on new builds, the PPP
is based on old assets whose condition is unknown. This makes
it even more difficult to be precise about what needs to be done,
and whether and how it can be done, to ensure the outputs, for
example, the number of trains per hour on a given line. If the
infracos were to bear all the risks of, say, a tunnel collapse,
this would make the scheme inordinately expensive. Hence the public
sector must and will bear the risk. But that in turn means that
it makes little sense to estimate costs over a 30 year period.
Indeed, the methodology document implicitly recognises this since
it says in paragraph 5.2.3 that there will be Periodic Reviews
every seven to eight years to "give LU the opportunity to
restate its performance requirements and allow the Infracos to
put forward revised charges". Several points follow from
this. Firstly, the VFM comparisons will be based on a methodology
of 30 year NPV calculations that favour the PPP, but in reality,
the agreements will run for seven to eight years and hence the
actual costs and benefits will be different from those estimated
in the business case. Only the first seven to eight years data
will be very meaningful. Hence the VFM comparison will have little
practical validity. Secondly, since the infrastructure service
charge will be determined by negotiation not regulation, the infracos
will have LU over a barrel.
21. All this means that the VFM is of limited
relevance to the public debate and little reliance can in fact
be placed upon the VFM comparisons, which are necessarily ex
ante. The use of a bond financed PSC will make it considerably
more difficult for the PPP to demonstrate value for money. However,
the use of a high interest rate for the PSC tips the balance back
towards the PPP. The aim of carrying out the appraisal in a manner
that will withstand criticism would be undermined by any inflation
of the PSC values.
22. These are a priori problems with
the VFM methodology. It is worth considering how useful these
appraisals have been in practice or ex post facto. The
recent Arthur Andersen report evaluating PFI has been widely cited.
It examined 30 business cases, out of 90 possible projects where
there were PSC comparisons, without explicitly identifying the
projects. It found that there were savings but these came from
just a few projects. But the report failed to state, although
it could be inferred from other data in the report, that the project
that accounted for the most savings, labelled "S" in
their tables was the IT project, NIRS2, carried out by Andersen
Consulting for the Benefits agency. This one project's estimates
of NPV accounted for 55 per cent of the "savings" and
80 per cent of the risk transfer supposedly ascribed to PFI in
the business cases. Andersen Consulting, Arthur Andersen's sister
consulting arm, won the contract as it priced the deal considerably
less than the other bidders. However, even more importantly, the
project has been a spectacular failure that has gone massively
over budget and years overdue. As Andersen's paid a fine of only
£3.9 million, the public sector and the public has borne
the costs. Other IT PFI projects have also failed dismally. While
IT projects are notoriously difficult to get right, the relevance
here is that their business cases showed that PFI was preferable
to public procurement, in part if not wholly because of the risk
transfer. Sadly, this was not the case. The NAO noted that other
projects turned out to be more costly than their business cases
suggested.
23. There are also pragmatic pressures that
undermine the VFM comparisons. As PFI is usually the only way
of obtaining the much needed new school or hospital, there is
an incentive to rig the comparison in favour of PFI. Sarah Wood,
Director of finance and performance review at Birmingham City
Council, said she had "reservations" about the figures
supporting a local PFI project to rebuild seven schools and refurbish
a further three. "Of course publicly I've got to say that
it is value for money. Privately, I am not sure," she told
delegates at the CIPFA Scottish branch conference, "But it
is the only game in town. It is the way you do get money into
your services.[19]
That is a sentiment we have heard expressed many times. Under
these conditions, PFI cases are prepared and constructed so as
to demonstrate VFM that subsequently prove to be less than expected.[20]
24. As a result of all these factors, it
is difficult to place much reliance on the VFM comparison. In
any event it is only one criterion among several. These include:
affordability, the implications of the PPP structure for planning,
co-ordination and safety, future investment to expand the network,
democratic accountability and stakeholder consent including the
wishes of those who will have to fund itthe passengers,
tax payers both local and national, and the people of London.
All of these criteria give rise to problems with the PPP, as we
argued in our earlier submission.
Declan Gaffney
School of Public Policy, University College London
Dr Jean Shaoul
School of Accounting and Finance, Manchester University
April 2000
5 "London Underground Public-Private Partnership:
Methodology for preparing the Public Sector Comparator,"
March 2000. Back
6 See
for example D Currie, "Regulation Initiative: Funding the
London Underground," London Business School, March 2000. Back
7
For example, the full business case of the Royal Infirmary at
Edinburgh, as explained in Shaoul J, " Looking Glass World
of PFI", Public Finance, 29 January 1999, the PFI case for
Pimlico School as explained in "The Lessons of Pimlico,"
Public Finance, October 29 1999, PFI in the NHS: Is there an economic
case?" British Medical Journal, 10 July 1999, and
the summary business case for the UK Passport Agency PFI. Back
8
PriceWaterhouseCooper, "London Underground PPP-Briefing
Document," December 1999. Back
9
Booz Allen and Hamilton, "Railtrack's performance in the
period 1999-2001, Report to the Office of the Rail Regulator,
1999. Back
10
Shaoul, J, "A Critical Financial Analysis of the Post-Privatisation
Performance of the Water industry in England-Wales," Critical
Perspectives on Accounting, 1997, Vol 8, pp 479-505. Back
11
Schofield, R, and Shaoul, "Regulating the Water Industry:
By Any Standards?" Utilities Law review, Vol 8, Issue 2,
Mar-April 1997, pp 56-70. Back
12
Cutler, T, and Waine, B, "Managing the Welfare State: The
politics of public sector management", Berg, Providence and
Oxford, 1994.
Berg, Providence and Oxford, 1994. Back
13
Bargaining Report "Compulsory Competitive tendering-the
effect on wages and conditions," May 1990. pp5-11 Back
14
Treasury "Using Private Enterprise in Government: Report
of multi-departmental review of competitive tendering and contracting
for services in government departments," HMSO, London, 1986. Back
15
Arthur Anderson and Enterprises LSE, "Value for Money drivers
in the Private Finance Initiative," report commissioned by
The Treasury Taskforce, January 2000. Back
16
Shaoul, I "the Economic and Financial Context: The Shrinking
State?" in Corby S and White G, "Employee relations
in the Public Services-themes and issues," Routledge, 1999. Back
17
Ross, S A "Uses, Abuses and Alternatives to the Net Present
Value Rule." Financial management, Vol 24, No 3, Autumn 1995,
pp 96-102. Back
18
The Treasury, "Appraisal and Evaluation in Central Government"
(The Green Book), Treasury Taskforce Policy Statement 2 (Public
Sector Comparators and Value for Money), and Treasury Taskforce
technical Note 5 (How to prepare a Public Sector Comparator). Back
19
Cited in Public Finance (7/4/00), p 6. Back
20
Note that the Public Accounts Committee reported (7/4/00) that
the new PFI build for Dartford and Gravesham NHS Trust will only
deliver savings of £5 million not £17 million in comparison
with conventional procurement because it had failed to detect
"significant errors" in its calculation. Back
|