III. THE VALUE FOR MONEY TEST
Public Sector Comparator
25. The value for money test will examine whether
the bidders will offer better value for money than an alternative,
publicly funded infrastructure option. To assess value for money,
a Public Sector Comparator has been created. A Public Sector Comparator
is an estimate of the likely public cost of funding the infrastructure
improvement proposed by the bidders under alternative financing
scenarios. The PPP bids will be compared against a traditional
funding model of a combination of public subsidy and fares revenue
and a second option of public subsidy and bond financing with
the bonds raised against future fares revenue.
Each of the three Infraco contracts will be assessed independently
of the others for value for money. Should one or two of the Infraco
bids fail the value for money test that will not preclude the
other bids from being approved.
26. At the time of our predecessor Committee's inquiry,
there were concerns about how the Public Sector Comparator had
been created. That
Committee recommended that the National Audit Office examined
the financial analysis of the PPP at an early stage to ensure
that the process was well guided and transparent.
The National Audit Office Report
PUBLIC SECTOR COMPARATOR DEVELOPMENT
27. The National Audit Office reported on its findings
in December 2000, and noted that this is the largest Public Sector
Comparator, by value, developed by any public entity.
The National Audit Office found that the Public Sector Comparator
had been developed using a clearly documented methodology to estimate
the costs of conducting the work in the public sector. It also
concluded that the evaluation of the bids submitted by the private
sector was thorough and comprehensive. London Underground Limited's
auditors, KPMG, found that the methodology for the comparators
complied with the relevant Treasury guidance.
28. Although concluding that the methodology used
in the development of the Public Sector Comparator was sound,
the National Audit Office identified several concerns about the
- "There is considerable uncertainty in modelling
the costs of conducting such a major programme of work over a
30 year period.
- The financial models alone would provide only
limited guidance as to the most likely cost of a public infrastructure
- There is uncertainty over the most appropriate
financing approach for the cost of public infrastructure operation."
Those technical concerns are being considered by
London Underground Limited. The National Audit Office told us
during this inquiry that " these technical problems will
not, in the end, be decisive".
Subsequent to the National Audit Office report, Transport for
London commissioned work which highlighted "questionable
judgemental factors" used in the PSC and claim that the advantages
of bond financing have been largely dismissed.
29. The National Audit Office emphasised that too
much weight had been given to the financial analysis and that
in fact it only had "limited use".
Its report stated that:
"There is a wide range of other factors, which
are either difficult or impossible to quantify in financial terms,
and which could impact directly on the value for money of the
Those wider factors include:
- the risks and benefits of dividing responsibility
for the network;
- loss of flexibility for managing network upgrade
decisions compared to the extra efficiency of private sector management;
- the risk of a highly contractual and litigious
approach to managing the contracts;
- the ability of the performance specification
to avoid perverse incentives and act in the public's interest;
- the transfer of risks between the parties.
30. The allocation of risk between parties is a major
part of the PPP process. For example, asking a private sector
company to price for the risk of the Thames flood barrier failing
would make the contracts too expensive
and the "success lies with allocating risks to those best
able to carry them".
London Underground Limited told the Committee that the public
sector will retain the risk responsibility relating to revenues,
changes to health and safety law, legislative changes and some
31. The Government believes risk can successfully
be transferred, particularly by giving the private sector consortia
responsibility for the condition of the assets over a long period.
In contrast, Mr Bob Kiley, Commissioner of Transport for
London, told us that less and less risk has been handed to the
private sector as the contract has progressed and that "the
amount of risk that is really being transferred is barely perceptible".
Mr Martin Blaiklock, a specialist in infrastructure private finance
deals, described underground metro systems as "second only
to nuclear power stations in technical, commercial and financial
He, like other witnesses, doubted whether "London Underground
Limited is on top of the risk allocation and mitigation program".
The balance of risk transfer is critical to the success of the
PPP. If little risk can be transferred to the private sector,
due to the cost attached to it, then the rationale for the PPP
is seriously undermined.
Subjectivity of value for money
32. All parties accepted that the wider factors described
above are subjective in nature. For example, the determination
of risk, the uncertainty that an event might occur, is subjective.
Values can be assigned to some risks through assumptions about
the probability of such events occurring. London Underground Limited
has developed a good understanding of the operational risks to
their business through historical performance data. However, whilst
estimates can be made of some risks, many of the factors identified
by the National Audit Office, such as the risk of significant
litigation in managing the contracts, the loss of cohesion from
splitting the operation from the maintenance of the network and
the likelihood of continued poor public sector management performance,
are both highly subjective and critical to determining whether
the bids submitted offer better value for money.
33. London Underground Limited accept that many of
the factors are subjective and has invested considerable effort
in understanding the nature and importance of these wider factors
since the National Audit Office report. London Underground Limited
told us that it was confident that it could provide an assessment
of all of those factors but acknowledged that "it is not
a solution to an equation, where you can say is this figure bigger
than that figure".
The uncertainty over whether the PPP will offer value for money,
acknowledged by both London Underground Limited and the Government,
shows that "the cost savings [of £4.5 billion] originally
promised to Parliament when it endorsed the PPP have proven illusory".
ERNST AND YOUNG'S REVIEW
34. In October 2001, the Government appointed Ernst
and Young to carry out an independent review of the value for
money assessment being undertaken by London Underground. That
review, which will cost £400,000, is to include an assessment
of both the financial and non-financial factors involved in the
decision. Ernst and
Young, in its terms of reference for the study, points out that:
"Due to the subjective nature of the value
for money assessment, it is not always possible to conclude decisively
on the position of proposed contractual negotiations. The conclusions
of the Review are likely to be based on an assessment of a range
of factors and hence the overall assessment may be subjective
and based on stated assumptions."
The Secretary of State told the Sub-Committee that
he would only approve the PPP contracts if they could conclusively
demonstrate that they offered value for money over the Public
The Secretary of State also told us however that "it will
not be one of those recommendations which says clearly, yes it
is value for money or no it is not".
ROLE OF CONSULTANTS
35. The National Audit Office told us that the decision
on whether the PPP contracts would offer value for money depended
significantly on the subjective, wider factors. In assessing those
wider factors it did not believe that London Underground or the
Government "should rely on consultants; those [decisions
on the wider factors] are absolutely decisions for the executive,
in this case the Board of London Transport and the Department.
It is their responsibility to decide what they want to do and
how to justify it."
Value for money over seven and a half and 30 years
36. Assets on the Underground network can be in service
for 20 to 40 years. The PPP contracts are for a 30-year period
to encourage the Infracos to take a "whole life" approach
to upgrading the infrastructure. London Underground Limited told
us that a contract that was fixed for 30 years would not be practicable
and no bidder could offer fixed prices for 30 years in a way that
would offer value for money.
One of the unsuccessful consortia told us that it would have been
willing to sign up to a 30-year price to improve the infrastructure
and provide the finance to implement it.
London Underground Limited has inserted periodic reviews every
seven and a half years to allow itself the flexibility to vary
the terms of the PPP so that it can adapt to changes in demand
on the network over time.
If the two parties cannot agree on the price for the work to be
completed at the Periodic Review then an arbiter is appointed
by the Secretary of State under the Greater London Authority Act,
Section 229. One of the arbiter's roles is to determine the price
paid to the Infraco to achieve, if the Infraco acts in an economic
and efficient manner, the agreed rate of return.
The bids submitted for evaluation under the PPP contain fixed
prices for the first seven and a half year period and estimates
of 30-year costs. Value for money is being assessed over the seven
and a half and 30- year periods.
Risk of cost inflation
37. A number of questions were raised as to how value
for money could be assessed over 30 years when the bidders were
not committed to the price submitted for that period. The financial
advisers to Transport for London, Deloitte and Touche,
found "no evidence that there is a valid basis for establishing
that the PPP will achieve value for money using commonly accepted
techniques for projects of this nature".
It believed that assessing value for money over 30 years "is
flawed because bidders were not required to submit firm prices
for the whole period".
Mr Blaiklock also questioned whether it was possible to determine
whether the bids are more expensive over the period from seven
and a half years to 30 years when prices were unknown.
London Underground's advisor, Pricewaterhouse Coopers, strongly
disagreed. It pointed out that assessing value for money over
30 years was essential to ensure efficient long term investment
and that the seven and a half year year comparison was used to
compare committed prices and to protect against bidders attempting
to make artificially low bids for the period from seven and a
half years to 30 years.
Deloitte and Touche told us that subsequent to its report, Pricewaterhouse
Coopers had admitted that the seven and a half year comparison
was not valid.
38. The National Audit Office told us that the decision
to have review periods was a matter for the management of the
Underground but that inserting such reviews produced uncertainty
about the level of costs for those periods without fixed prices.
The National Audit Office also noted that calculating the impact
of that uncertainty was "very difficult".
Parsons Brinckerhoff stated that the first seven and a half year
period would be "a learning curve" for the Infracos
and that they would not be able to make a clearly-defined forecast
of costs beyond that point.
The Arbiter and cost inflation
39. There was considerable uncertainty about
the likely price of PPP contracts after the first review period.
The Sub-Committee received a number of submissions about the role
that the arbiter will have in preventing cost escalations. Transport
for London questioned the capability of the arbiter to
set prices in the public interest. It did not believe that there
are sufficient comparable businesses ,except the other Infracos,
for the arbiter to assess whether the Infracos are acting in an
economic and efficient manner and that the Infracos will control
the information required for any such assessment. It concluded
that "London Underground's ability to make any informed submissions
to the Arbitrator will be limited and the Infracos will be free
to exploit their monopoly positions."
Both London Underground Limited and the Government are confident
that the arbiter will be able to resolve pricing disputes in a
manner that protects both the public interest and the interests
of the Infracos.
Capital spend profile
40. Mr Kinglsey Manning, an advisor to the National
Audit Office, argued that one of the key differences between the
Underground PPP and a number of other Private Finance Initiative
deals was that the capital spending profile for the PPP continues
well beyond the first review period. Mr Manning said that:
"In most PFIs, at least you are able to look
at the capital spend over relatively short periods, so your major
uncertainty is often to do with the revenue stream beyond the
original capital spend; in this case, you have got uncertainties
both in the revenue streams and also in the capital spend into
He believed the deal to be "on the very edge
of the complex and difficult to analyse".
41. The inclusion of review periods within the contracts
has added considerable uncertainty to the final costs of the PPP
project. Opinion is divided as to whether it is possible to assess
value for money over 30 years when prices are fixed for the first
seven and a half years only. That consideration is particularly
important as the capital spend for the PPP is not concentrated
at the start of the contract period as in many Private Finance
Initiatives. There was, however, agreement that any analysis of
value for money of the PPP is extremely difficult and highly subjective.
42. Experience from the mainline railway has shown
that the failure to hold and maintain a complete register of assets
and their condition inevitably leads to cost inflation resulting
from unforseen problems and insufficient planning information.
In his oral evidence to the Transport Sub-Committee on the future
of rail franchising and Railtrack, the Secretary of State, acknowledged
that "for a company like Railtrack not to know the conditions
of its assets is deeply worrying",
and that the lack of a well defined asset register was "a
In its review of the Office of the Rail Regulator, for the mainline
railway network, the National Audit Office noted that deficiencies
in the information of the condition of Railtrack's assets meant
that the Office of the Rail Regulator could not "assess satisfactorily
on customers' behalf what is being achieved with this money."
43. London Underground Limited told us that an up
to date asset register exists for most of the Underground's assets
including track, tunnels, signals and trains because they are
all accessible for inspection. There was a classification of assets,
known as 'grey assets', about which less information was available
because they cannot be inspected easily.
However, the National Audit Office report of December 2000 found
that "past knowledge about asset conditions means that margins
for uncertainty (in estimating the capital spend programme) are
a finding that London Underground Limited did not dispute. Transport
for London estimates that the bidding consortia have a
"solid understanding of the nature and cost of the capital
expenditures over the first two years of the PPP. Thereafter the
priorities are not as well defined."
Mr Kiley told us that the first priority, if Transport for
London were awarded control of the system, would be a "stem
to stern" assessment of the physical condition of the assets.
44. The Infracos take the risk for cost inflations
due to poor knowledge about asset conditions over the first seven
and a half years. Both of the preferred bidders have undertaken
significant investigations to check on London Underground Limited's
assessment of asset conditions. Tube Lines have conducted over
one million man hours of 'due diligence' assessments.
Transport for London claim that the Infracos liability
for cost overruns is limited to £30 million per year over
the first seven and a half years
although this was denied by London Underground Limited.
45. If the Infracos faced increased costs over the
first seven and a half years, they would be allowed to request
an increase in the Infrastructure Service Charge for the following
seven and a half years. If the Infracos and London Underground
Limited do not agree, the arbiter will have to determine whether
any price rises are due to Infraco inefficiency or due to circumstances
beyond their control, such as worse than anticipated asset conditions.
The arbiter has a duty to ensure that the rate of return specified
in the contracts with the Infracos would be earned if the Infraco
acts economically and efficiently. Transport for London
argued that a lack of knowledge about asset condition would contribute
significantly to the inability of the arbiter to prevent price
46. A register of assets exists and is maintained
by London Underground. However, the register is incomplete and
worse than anticipated conditions in the "grey assets"
could have very significant cost implications for the PPP. Even
with the asset register, London Underground Limited has acknowledged
that there is considerable uncertainty in estimating the costs
of the investment programme over 30 years.
58 HC (2001-02) 373-IV, Q766. Back
Funding of London Underground. Back
The financial analysis for the London Underground Public Private
Partnerships (2000-01) National Audit Office, HC54. Back
The financial analysis for the London Underground Public Private
Partnerships (2000-01) National Audit Office, HC54. Back
London Underground Public Private Partnership - Emerging Findings,
17 July 2001, Deloitte and Touche Back
Commentary on Deloitte and Touche Report, (redacted Version),
24 August 2001.Pricewaterhouse Coopers. Back
LU11A, LU12B. Back
First Report of the Transport, Local Government and the Regions
Select Committee, HC (2001-02) 239-II, Q915. Back
Ibid, Q913. Back
Ensuring that Railtrack maintain and renew the railway network.
(1999-2000) National Audit Office Report, HC397, p9. Back
Grey assets include tunnel surrounds, bridges and station structures
behind cladding. Back
The financial analysis for the London Underground Public Private
Partnerships (2000-01) National Audit Office, HC54, p5. Back
LU3 Annex E. Back