Select Committee on Transport, Local Government and the Regions Minutes of Evidence

Examination of Witnesses (Questions 296-319)




  296. Good afternoon, madam and gentlemen. Thank you for coming to see us, and I apologise for the fact that we have kept you waiting. May I ask you to identify yourselves, for the record?
  (Mr Colman) Yes. I am Jeremy Colman. I am Assistant Auditor General in the National Audit Office, responsible for the work on Public Private Partnerships.
  (Ms Leahy) I am Patricia Leahy, Director of Studies of Public Private Partnerships at the National Audit Office.
  (Mr Manning) I am Kingsley Manning. I am Chief Executive of Newchurch Limited. I acted as an independent adviser to the National Audit Office last year on the matter of the (inaudible).

  297. May I put on record the Committee's thanks to you for coming to give us evidence this afternoon. It is extremely helpful, and we know that there is always a slight frisson at the National Audit Office when we ask these awkward questions; but you are most welcome, and I think we understand the terms of engagement. Did you, Mr Colman, want to say anything, to begin with?
  (Mr Colman) I want to say two things, if I may. The first, just to clarify why the three of us are here. Kingsley Manning is a distinguished expert in this field and was an adviser to us, but this afternoon he speaks for himself. The second point I would like to make, just briefly to summarise our report of last year; that report, to my mind, made three key points. The first was, the financial analysis, which was the subject of the report, can only ever be part of the story on value for money. Secondly, the financial analysis, only part of the story, was necessarily subject to huge inherent uncertainty. And, thirdly, at the time of writing, last year, the financial analysis was subject to some problems, so, in addition to its inherent uncertainties, there were problems created by the way the analysis was being done. I hope that will be helpful.

  Chairman: Thank you very much indeed.

Mr O'Brien

  298. You heard the previous evidence from Mr Kiley, and one of the concerns that I have is the fact that he considers that PPP will not survive. What is the view of the National Audit Office on that particular matter? In actual fact, there are plenty of examples of PFI and PPP; why should London Underground be different, do you think?
  (Mr Colman) I think that the strict answer to your question, Mr O'Brien, is that the NAO does not have an opinion about whether the PPP for London Underground is a good idea or a bad idea, that would be presuming on the role of the executive, and we are the independent auditors of the executive, on behalf of Parliament. It is, of course, quite true, as you say, that many PFIs and PPPs in existence do appear to be working pretty well, there are also some that are in difficulties. I do not see, therefore, that the PPP is necessarily a bad idea. I do not think I would like to go further than that into this area of policy.

  299. Can I put it in another context. We are being advised that it is because of the fact that the Underground is inefficient and it does not meet the travelling public's desires and has been like that for many years; is it reasonable to assume that Underground, if retained in the public sector, would continue to be inefficient, as it has been for the last 30 years?
  (Mr Colman) We commented on this point in our report last year. The financial analysis appeared, at that time, to be based on an assumption about future efficiency improvements in the public sector, and we noted that it might be unrealistic to assume that London Underground would continue for as long as 30 years in a highly inefficient state. So the force of this point is that, in considering a financial comparison, you need to forecast what you think really would happen, on the public sector side or on the PPP, and it struck us as unrealistic to assume that continued inefficiency really would be allowed to continue for 30 years.

  300. In your report that you published, was there any record of the fact that there was credible evidence of the efficiencies that the private sector management of Underground would achieve, and was that presented in the report?
  (Mr Colman) Patricia Leahy might want to say something in detail about that. My recollection is that the basis for the comparison is the bid against the comparator, so it is up to the companies bidding to bid on the basis of the efficiency savings they expect to meet; and, from the point of view of London Underground, the question is, is that bid credible? So they do not necessarily have to say `do we agree that those efficiency savings will be achieved in the private sector,' they have to test whether the bid is robust in the event that the savings are not achieved.
  (Ms Leahy) At the time we carried out our work last year, we had very little information on the bids that came in, so we had some discussion about the implications of the bids, but we did not look at them in any great detail. But we did notice, at the time, and reported on this, that we attached importance to an understanding of why it was (hinting?) the private sector could perform in the way that was suggested, which is, it performed well, and why the public sector might not be able to do that, and I think at that time very little work had been done; but, since then, we understand that there has been quite a bit of work into looking at why it is that the private sector is expected to be able to perform pretty efficiently.

  301. And do you intend to update your report?
  (Mr Colman) What we certainly intend to do, as we would do for any major PPP in central government, is to report to Parliament, in the usual way, some time after the deal has been done; that will include, obviously, looking at how far our report of last year was followed.


  302. Is it difficult, really, is it very difficult to assess the sort of wider factors that will affect the value for money?
  (Mr Colman) It is not only difficult, Madam Chairman, but it is, in a sense, not our job. It is the job of those deciding to do this deal, or not, to form their opinion of the wider factors, and the wider factors are subjective. There is nothing wrong with that, and they are factors to be considered by the people responsible for managing this enterprise.

  303. We understand that, but, given that, do you think there is any clarity in the assessments, any clarity in the assumptions that had been made, if I can put it in that sense? What we are really saying is that a certain number of subjective assessments have been made, were you able to isolate a really genuine statistical basis on which those assumptions had been made?
  (Mr Colman) Not last year, because when we were doing our work last year, I think there had been, frankly, very little progress outside the financial analysis.

  304. But you now think that they have, this is what you are saying, but that would only be included in your report after the event and not in a report that could be used to assess the accuracy of the assumptions?
  (Mr Colman) That is correct.

Mr Stevenson

  305. This is the only question I want to ask, really, because, if I may suggest that one of the factors that led to your report last year, which was extremely useful, was the insistence of this Committee that that should be done by your Office, as a well-respected, independent body. So you are now saying two things, I think. One, `when we did the report last year, there were a number of important issues we were not able to assess in any credible sense, because they just were not there for us to do it;' now those issues have been fleshed out somewhat, but you are not intending to report, other than in the normal way, that is, after the deal has been done. Is that what you are saying?
  (Mr Colman) That is what I have said so far, but I would like to add to it, if I may, that the reason, last year, that our report covered the financial analysis only was not just because that was all there was to look at a year ago. We are very, very jealous of our independence, and we feel that there is a serious risk of undermining our independence if we cross the line and start advising the executive on what it should do. So, therefore, this Committee's request that we should do a report last year caused us to think very, very carefully about what we might do before the deal was signed; we were very anxious to be helpful, but, on the other hand, very, very keen to protect our independence. And so we decided, after a lot of thought, that we could, just about, produce a report on the financial analysis, as a technical issue, without stepping into the wider factors that we thought were essentially a matter for the executive.

  306. And do you believe, Mr Colman, that the wider factors that you wanted to avoid, because, quite rightly, jealously guarding your independence, are in play today that would mean that your organisation would not respond positively to a request for a further look at this issue before the deal is signed?
  (Mr Colman) I think we would be in great difficulty reporting on the wider factors before the deal is signed.

  307. I did not ask that. I am simply asking you, I know I shall be stopped in a minute, but I am simply asking whether, if the circumstances were acceptable, your organisation would be prepared to embark on a further report, perhaps updating what you did last year, or considering factors that you feel you can consider, as a well-respected, independent organisation, before the deal is signed?
  (Mr Colman) We have thought very, very carefully about that as well, and, of course, we have consulted the Comptroller and Auditor General, with whom the decision would rest, and his view is that, at present, there is nothing that we could usefully add to what we said last year. The position, as we understand it, is that the points in our report last year were accepted by London Underground's advisers, and we know that they are working very hard to take them into account, to take them on board, in the assessment that they will be preparing for the Board of London Underground. Now, given that, there really is not anything new for us to say.

  308. But there might be, after the deal is done?
  (Mr Colman) I suspect there may not be very much new on this point of the financial analysis after the deal is done. In our report after the deal is done, we will range much wider on the value for money of the deal. We will be examining and reporting on what the wider factors were.

  Mr Stevenson: A very interesting distinction. Thank you.

Miss McIntosh

  309. Chairman, could I ask Mr Colman, you said that the report that you undertook was against a background of huge uncertainty and huge problems; could you just specify those?
  (Mr Colman) The inherent uncertainty is that the financial analysis seeks to compare two alternative ways of achieving the same result. The alternative ways are the PPP, on the one hand, and a conventionally public-financed alternative, on the other; but those projects are major undertakings, a huge capital programme, over a 30-year period. So the comparison must take account of, must be based upon, forecasts of what a capital programme might be over 30 years. Now just to say that shows that the inherent uncertainties are enormous. Who can say what it will cost to replace the escalators at King's Cross in ten years' time, but an estimate has had to be made for the purpose of this analysis, it must be an uncertain estimate, it cannot be any better than that. So that is the inherent uncertainties. The problems we found last year with the comparator, we thought that it was overcomplex, and therefore there was a risk of errors creeping in; we thought that the adjustment for risks on the public sector side, a crucial adjustment, whilst logical and very, very thorough, might not have completely avoided double counting of risks, and that would overstate the cost of the comparator. We found unresolved technical issues to do with the discount rate and reputational externalities, which Mr Kiley referred to. I actually think that these technical problems will not, in the end, be decisive. But there were a number of other technical problems with the model, and our understanding is that London Underground and their advisers are dealing with these technical issues as best they can.

  310. Can I just ask you, would you not accept that after September 11 and October 5 there are even more uncertainties now?
  (Mr Colman) What I think is the result of those events is to heighten everyone's awareness of risks, but the risks were always there.

  311. Can I just ask, did you hear Mr Kiley's evidence?
  (Mr Colman) Yes, we did.

  312. He said that it is very poor public policy not to have inserted the potential for a contractor to go bust during the life of the contract; do you have a view on that?
  (Mr Colman) This is one of the issues that is somewhere between something you put into the financial analysis and a wider issue. In any PFI deal or PPP, we would expect to see a thoroughly thought-through plan for managing risks, and one of the prime risks, obviously, dealing with a private sector company, is that that company may go bust, possibly for reasons unconnected with your contract; so we would expect to see a thoroughly worked-out risk-management plan.

Mr Donohoe

  313. Do you see any conflict between the use of consultants and the use of the National Audit Office?
  (Mr Colman) In the first place, we in the NAO are not really consultants, and our prime function is to serve Parliament and to report to Parliament on value for money, and, of course, on the financial audit of Government Departments. So it really is not something that we generally do, to act as consultants. But as we are the external auditors of central government, it is part of the job of external auditors to give advice, from time to time, so we do give advice to our audit clients, where we can properly do so without compromising our independence.

  314. But, on the basis of what we see, given that it is all public money at this stage, is there not duplication of resources that you are partly applying, as with the consultants; you must see that as conflict, surely?
  (Mr Colman) I am really not sure I see that as conflict, Mr Donohoe. We are, at present, doing two things. The first is preparing ourselves to report in due course, which means that we try to keep in touch with developments, so that we are mentally prepared to do our work if and when a deal is signed. The second thing we are doing is, we are responding to requests for comments, or advice, as London Underground and the Department and their advisers do their work; that is not something that involves a huge effort on our part, but we do think it is consistent with our aim of promoting beneficial change in the provision of public services, as our mission statement has it, to be helpful in these circumstances, to be as helpful as we can.

  315. Should the National Audit Office review the assessment of value for money prior to the signing of these contracts?
  (Mr Colman) We have never done that before. The report we did last year was absolutely as far as we had ever gone before, absolutely unprecedented, and it made us—

  Mr Donohoe: Why was that; why did you go in that direction?

  Chairman: We asked them to, for a start.

Mr Donohoe

  316. Yes, I know we did.
  (Mr Colman) Not only, I may say, has this Committee asked us but all the parties to the deal asked us, and it is our nature—


  317. But we were the most important, were we not, Mr Colman?
  (Mr Colman) You certainly were, Madam Chairman, and it is our aim always to be as helpful as we possibly can be, consistent with preserving our independence. So, in last year's report, we were only trying to be helpful, and we went as far as we could.

Mrs Ellman

  318. Have you been involved in assessing any other PPP or PFI which has equivalence in terms of uncertainties?
  (Mr Colman) Yes, indeed, we have. I think the inherent uncertainties are a common feature of PFIs, they are very commonly for 20 or 30 years, and so the uncertainty I have referred to arises. Sometimes the capital programme is concentrated at the beginning of the period, which does reduce the uncertainty, in this case, the capital programme is spread through it, but we are used to dealing with very uncertain situations.

  319. And has enough time passed for you to be able to assess whether your judgements were correct, in any of these others; could you give me some examples?
  (Mr Colman) I think, actually, the answer to that question is no, but we do not try to reach an independent view of our own on what the assessment should be, we examine the reasons given by the Government Department, if that is what it is, for proceeding the way they have. In some cases, we have found good reasons that we could recognise, in other cases, we have found questionable arguments, or questionable calculations, but we have not sought to substitute our own view, or substitute our own opinion. So, in that sense, we can never be found wrong.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2001
Prepared 21 December 2001