Examination of Witnesses (Questions 296-319)
WEDNESDAY 28 NOVEMBER 2001
MR JEREMY
COLMAN, PATRICIA
LEAHY AND
MR KINGSLEY
MANNING
Chairman
296. Good afternoon, madam and gentlemen. Thank
you for coming to see us, and I apologise for the fact that we
have kept you waiting. May I ask you to identify yourselves, for
the record?
(Mr Colman) Yes. I am Jeremy Colman. I am Assistant
Auditor General in the National Audit Office, responsible for
the work on Public Private Partnerships.
(Ms Leahy) I am Patricia Leahy, Director of Studies
of Public Private Partnerships at the National Audit Office.
(Mr Manning) I am Kingsley Manning. I am Chief Executive
of Newchurch Limited. I acted as an independent adviser to the
National Audit Office last year on the matter of the (inaudible).
297. May I put on record the Committee's thanks
to you for coming to give us evidence this afternoon. It is extremely
helpful, and we know that there is always a slight frisson at
the National Audit Office when we ask these awkward questions;
but you are most welcome, and I think we understand the terms
of engagement. Did you, Mr Colman, want to say anything, to begin
with?
(Mr Colman) I want to say two things, if I may. The
first, just to clarify why the three of us are here. Kingsley
Manning is a distinguished expert in this field and was an adviser
to us, but this afternoon he speaks for himself. The second point
I would like to make, just briefly to summarise our report of
last year; that report, to my mind, made three key points. The
first was, the financial analysis, which was the subject of the
report, can only ever be part of the story on value for money.
Secondly, the financial analysis, only part of the story, was
necessarily subject to huge inherent uncertainty. And, thirdly,
at the time of writing, last year, the financial analysis was
subject to some problems, so, in addition to its inherent uncertainties,
there were problems created by the way the analysis was being
done. I hope that will be helpful.
Chairman: Thank you very much indeed.
Mr O'Brien
298. You heard the previous evidence from Mr
Kiley, and one of the concerns that I have is the fact that he
considers that PPP will not survive. What is the view of the National
Audit Office on that particular matter? In actual fact, there
are plenty of examples of PFI and PPP; why should London Underground
be different, do you think?
(Mr Colman) I think that the strict answer to your
question, Mr O'Brien, is that the NAO does not have an opinion
about whether the PPP for London Underground is a good idea or
a bad idea, that would be presuming on the role of the executive,
and we are the independent auditors of the executive, on behalf
of Parliament. It is, of course, quite true, as you say, that
many PFIs and PPPs in existence do appear to be working pretty
well, there are also some that are in difficulties. I do not see,
therefore, that the PPP is necessarily a bad idea. I do not think
I would like to go further than that into this area of policy.
299. Can I put it in another context. We are
being advised that it is because of the fact that the Underground
is inefficient and it does not meet the travelling public's desires
and has been like that for many years; is it reasonable to assume
that Underground, if retained in the public sector, would continue
to be inefficient, as it has been for the last 30 years?
(Mr Colman) We commented on this point in our report
last year. The financial analysis appeared, at that time, to be
based on an assumption about future efficiency improvements in
the public sector, and we noted that it might be unrealistic to
assume that London Underground would continue for as long as 30
years in a highly inefficient state. So the force of this point
is that, in considering a financial comparison, you need to forecast
what you think really would happen, on the public sector side
or on the PPP, and it struck us as unrealistic to assume that
continued inefficiency really would be allowed to continue for
30 years.
300. In your report that you published, was
there any record of the fact that there was credible evidence
of the efficiencies that the private sector management of Underground
would achieve, and was that presented in the report?
(Mr Colman) Patricia Leahy might want to say something
in detail about that. My recollection is that the basis for the
comparison is the bid against the comparator, so it is up to the
companies bidding to bid on the basis of the efficiency savings
they expect to meet; and, from the point of view of London Underground,
the question is, is that bid credible? So they do not necessarily
have to say `do we agree that those efficiency savings will be
achieved in the private sector,' they have to test whether the
bid is robust in the event that the savings are not achieved.
(Ms Leahy) At the time we carried out our work last
year, we had very little information on the bids that came in,
so we had some discussion about the implications of the bids,
but we did not look at them in any great detail. But we did notice,
at the time, and reported on this, that we attached importance
to an understanding of why it was (hinting?) the private sector
could perform in the way that was suggested, which is, it performed
well, and why the public sector might not be able to do that,
and I think at that time very little work had been done; but,
since then, we understand that there has been quite a bit of work
into looking at why it is that the private sector is expected
to be able to perform pretty efficiently.
301. And do you intend to update your report?
(Mr Colman) What we certainly intend to do, as we
would do for any major PPP in central government, is to report
to Parliament, in the usual way, some time after the deal has
been done; that will include, obviously, looking at how far our
report of last year was followed.
Chairman
302. Is it difficult, really, is it very difficult
to assess the sort of wider factors that will affect the value
for money?
(Mr Colman) It is not only difficult, Madam Chairman,
but it is, in a sense, not our job. It is the job of those deciding
to do this deal, or not, to form their opinion of the wider factors,
and the wider factors are subjective. There is nothing wrong with
that, and they are factors to be considered by the people responsible
for managing this enterprise.
303. We understand that, but, given that, do
you think there is any clarity in the assessments, any clarity
in the assumptions that had been made, if I can put it in that
sense? What we are really saying is that a certain number of subjective
assessments have been made, were you able to isolate a really
genuine statistical basis on which those assumptions had been
made?
(Mr Colman) Not last year, because when we were doing
our work last year, I think there had been, frankly, very little
progress outside the financial analysis.
304. But you now think that they have, this
is what you are saying, but that would only be included in your
report after the event and not in a report that could be used
to assess the accuracy of the assumptions?
(Mr Colman) That is correct.
Mr Stevenson
305. This is the only question I want to ask,
really, because, if I may suggest that one of the factors that
led to your report last year, which was extremely useful, was
the insistence of this Committee that that should be done by your
Office, as a well-respected, independent body. So you are now
saying two things, I think. One, `when we did the report last
year, there were a number of important issues we were not able
to assess in any credible sense, because they just were not there
for us to do it;' now those issues have been fleshed out somewhat,
but you are not intending to report, other than in the normal
way, that is, after the deal has been done. Is that what you are
saying?
(Mr Colman) That is what I have said so far, but I
would like to add to it, if I may, that the reason, last year,
that our report covered the financial analysis only was not just
because that was all there was to look at a year ago. We are very,
very jealous of our independence, and we feel that there is a
serious risk of undermining our independence if we cross the line
and start advising the executive on what it should do. So, therefore,
this Committee's request that we should do a report last year
caused us to think very, very carefully about what we might do
before the deal was signed; we were very anxious to be helpful,
but, on the other hand, very, very keen to protect our independence.
And so we decided, after a lot of thought, that we could, just
about, produce a report on the financial analysis, as a technical
issue, without stepping into the wider factors that we thought
were essentially a matter for the executive.
306. And do you believe, Mr Colman, that the
wider factors that you wanted to avoid, because, quite rightly,
jealously guarding your independence, are in play today that would
mean that your organisation would not respond positively to a
request for a further look at this issue before the deal is signed?
(Mr Colman) I think we would be in great difficulty
reporting on the wider factors before the deal is signed.
307. I did not ask that. I am simply asking
you, I know I shall be stopped in a minute, but I am simply asking
whether, if the circumstances were acceptable, your organisation
would be prepared to embark on a further report, perhaps updating
what you did last year, or considering factors that you feel you
can consider, as a well-respected, independent organisation, before
the deal is signed?
(Mr Colman) We have thought very, very carefully about
that as well, and, of course, we have consulted the Comptroller
and Auditor General, with whom the decision would rest, and his
view is that, at present, there is nothing that we could usefully
add to what we said last year. The position, as we understand
it, is that the points in our report last year were accepted by
London Underground's advisers, and we know that they are working
very hard to take them into account, to take them on board, in
the assessment that they will be preparing for the Board of London
Underground. Now, given that, there really is not anything new
for us to say.
308. But there might be, after the deal is done?
(Mr Colman) I suspect there may not be very much new
on this point of the financial analysis after the deal is done.
In our report after the deal is done, we will range much wider
on the value for money of the deal. We will be examining and reporting
on what the wider factors were.
Mr Stevenson: A very interesting distinction.
Thank you.
Miss McIntosh
309. Chairman, could I ask Mr Colman, you said
that the report that you undertook was against a background of
huge uncertainty and huge problems; could you just specify those?
(Mr Colman) The inherent uncertainty is that the financial
analysis seeks to compare two alternative ways of achieving the
same result. The alternative ways are the PPP, on the one hand,
and a conventionally public-financed alternative, on the other;
but those projects are major undertakings, a huge capital programme,
over a 30-year period. So the comparison must take account of,
must be based upon, forecasts of what a capital programme might
be over 30 years. Now just to say that shows that the inherent
uncertainties are enormous. Who can say what it will cost to replace
the escalators at King's Cross in ten years' time, but an estimate
has had to be made for the purpose of this analysis, it must be
an uncertain estimate, it cannot be any better than that. So that
is the inherent uncertainties. The problems we found last year
with the comparator, we thought that it was overcomplex, and therefore
there was a risk of errors creeping in; we thought that the adjustment
for risks on the public sector side, a crucial adjustment, whilst
logical and very, very thorough, might not have completely avoided
double counting of risks, and that would overstate the cost of
the comparator. We found unresolved technical issues to do with
the discount rate and reputational externalities, which Mr Kiley
referred to. I actually think that these technical problems will
not, in the end, be decisive. But there were a number of other
technical problems with the model, and our understanding is that
London Underground and their advisers are dealing with these technical
issues as best they can.
310. Can I just ask you, would you not accept
that after September 11 and October 5 there are even more uncertainties
now?
(Mr Colman) What I think is the result of those events
is to heighten everyone's awareness of risks, but the risks were
always there.
311. Can I just ask, did you hear Mr Kiley's
evidence?
(Mr Colman) Yes, we did.
312. He said that it is very poor public policy
not to have inserted the potential for a contractor to go bust
during the life of the contract; do you have a view on that?
(Mr Colman) This is one of the issues that is somewhere
between something you put into the financial analysis and a wider
issue. In any PFI deal or PPP, we would expect to see a thoroughly
thought-through plan for managing risks, and one of the prime
risks, obviously, dealing with a private sector company, is that
that company may go bust, possibly for reasons unconnected with
your contract; so we would expect to see a thoroughly worked-out
risk-management plan.
Mr Donohoe
313. Do you see any conflict between the use
of consultants and the use of the National Audit Office?
(Mr Colman) In the first place, we in the NAO are
not really consultants, and our prime function is to serve Parliament
and to report to Parliament on value for money, and, of course,
on the financial audit of Government Departments. So it really
is not something that we generally do, to act as consultants.
But as we are the external auditors of central government, it
is part of the job of external auditors to give advice, from time
to time, so we do give advice to our audit clients, where we can
properly do so without compromising our independence.
314. But, on the basis of what we see, given
that it is all public money at this stage, is there not duplication
of resources that you are partly applying, as with the consultants;
you must see that as conflict, surely?
(Mr Colman) I am really not sure I see that as conflict,
Mr Donohoe. We are, at present, doing two things. The first is
preparing ourselves to report in due course, which means that
we try to keep in touch with developments, so that we are mentally
prepared to do our work if and when a deal is signed. The second
thing we are doing is, we are responding to requests for comments,
or advice, as London Underground and the Department and their
advisers do their work; that is not something that involves a
huge effort on our part, but we do think it is consistent with
our aim of promoting beneficial change in the provision of public
services, as our mission statement has it, to be helpful in these
circumstances, to be as helpful as we can.
315. Should the National Audit Office review
the assessment of value for money prior to the signing of these
contracts?
(Mr Colman) We have never done that before. The report
we did last year was absolutely as far as we had ever gone before,
absolutely unprecedented, and it made us—
Mr Donohoe: Why was that; why did you go in
that direction?
Chairman: We asked them to, for a start.
Mr Donohoe
316. Yes, I know we did.
(Mr Colman) Not only, I may say, has this Committee
asked us but all the parties to the deal asked us, and it is our
nature—
Chairman
317. But we were the most important, were we
not, Mr Colman?
(Mr Colman) You certainly were, Madam Chairman, and
it is our aim always to be as helpful as we possibly can be, consistent
with preserving our independence. So, in last year's report, we
were only trying to be helpful, and we went as far as we could.
Mrs Ellman
318. Have you been involved in assessing any
other PPP or PFI which has equivalence in terms of uncertainties?
(Mr Colman) Yes, indeed, we have. I think the inherent
uncertainties are a common feature of PFIs, they are very commonly
for 20 or 30 years, and so the uncertainty I have referred to
arises. Sometimes the capital programme is concentrated at the
beginning of the period, which does reduce the uncertainty, in
this case, the capital programme is spread through it, but we
are used to dealing with very uncertain situations.
319. And has enough time passed for you to be
able to assess whether your judgements were correct, in any of
these others; could you give me some examples?
(Mr Colman) I think, actually, the answer to that
question is no, but we do not try to reach an independent view
of our own on what the assessment should be, we examine the reasons
given by the Government Department, if that is what it is, for
proceeding the way they have. In some cases, we have found good
reasons that we could recognise, in other cases, we have found
questionable arguments, or questionable calculations, but we have
not sought to substitute our own view, or substitute our own opinion.
So, in that sense, we can never be found wrong.
|
