Select Committee on Public Accounts Minutes of Evidence

National Audit Office's response to the memorandum submitted by the Centre for International Public Health Policy


  1.  The briefing note dated 18 October 2005 from the Centre for International Public Health Policy (CIPHP) on the National Audit Office Report "Innovation in the NHS: Local Improvement Finance Trusts" criticises the methodology used by the National Audit Office. The note suggests that the methodology used is based on surveys of stakeholders with a vested interest in LIFT and that, other than these interviews, there is no evaluation of value for money or the factors that underpin it. The note says the National Audit Office Report marks a new phase in a shift away from quantitative towards qualitative analysis. The note also says that potential delivery problems arising from new governance structures are identified in the Report but not evaluated.

  2.  The National Audit Office has considered these points carefully and responds to the methodological issues made below. This note provides clarification on appropriate elements of the National Audit Office's fieldwork. It also refers to the results of further analysis carried out as a result of the hearing of the Committee of Public Accounts on 17 October 2005.


  3.  The National Audit Office attaches importance to both quantitative and qualitative analysis. In reports on subjects such as LIFT, where an initiative is at an early stage, there is room for judgement on how it will develop and whether its ambitions will be fulfilled. The report carried out both qualitative and quantitative analysis as set out in paragraphs 14-17 below.


  4.  The following methodological points were made:

    (i)  There is no comparison of LIFT proposals against other current or potential financing methods.

    (ii)  There is no examination of risk transfer.

    (iii)  There is some evidence that affordability may be a problem but no analysis.

Comparisons against other methods

  5.  The National Audit Office Report compared LIFT to both traditional public capital investment and third party development (paragraphs 1.6-1.9). The reasons why the final Report did not quantify the cost of the alternatives and compare them to the cost of LIFT are expanded on below.

  6.  There was a lack of information on suitable comparators procured using alternative mechanisms. The National Audit Office considered comparing the cost of LIFT to third party developments and noted the different financing and contract structures in such developments, together with different whole life cost profiles. Moreover, analysis of individual leases including rent review clauses and the extent of further charges such as utility and insurance services would have been necessary. The National Audit Office met with several third party developers who were unwilling to share detailed data.

  7.  The National Audit Office asked each PCT and LIFTCo to provide examples of suitable primary care developments by way of comparison. It found that public finance in the form of central funding was rarely available and usually only for small scale refurbishment and redevelopment. At present improvement grants of up to £100,000 are available so long as there is a commitment that the development will remain in the NHS for 10 years.

  8.  The CIPHP briefing paper refers to funding under the London Improvement Zone (LIZ) which was available until 1999. It cites the example of the Fairfield Grove Health Centre, Greenwich which was completed in 1996 for a contract value of £1.5 million. Fairfield Health Centre offers integrated primary health care facilities for general medical practice, community services and other primary care practitioners. Its size and the fact that it was completed nearly 10 years ago suggest that it is not a good comparator for LIFT.

  9.  In the light of the interest shown by the Committee of Public Accounts in such comparisons the National Audit Office approached the PCTs involved in the case studies in its report to try again to find meaningful comparators. Information on a comparable third party development—the Lyng Centre for Health and Social Care is provided in the supplementary memorandum to the Committee. The PCTs could not find any comparable centrally funded developments.

Risk transfer

  10.  The National Audit Office Report did not set out details of the risk transfer arrangements in LIFT as it identified no value for money concerns about the allocation of risk under the LIFT model.


  11.  The National Audit Office Report discussed affordability—some tenants were concerned about the rental costs they faced (paragraph 2.14). Representatives from the National Pharmaceutical Association, the British Dental Association and Local Authorities told the National Audit Office that they had concerns over rental costs. The National Audit Office asked for relevant background and received qualitative feedback rather than quantitative data.

  12.  Wide consultation at the time the NAO Report was carried out (paragraphs 16 and 17 below) did not reveal any general affordability issues for PCTs. The business cases reviewed for the case studies had considered affordability for the local health economy. After the Report was published, in the light of experience as developments came on stream, affordability questions were starting to be raised. During a visit by members of the Committee of Public Accounts to a LIFT development, arranged by the National Audit Office after the Report was completed, this issue was raised by Dr Kohli, a GP in the LIFT development. He also raised the issue in the 17 October 2005 Committee Session. As a result, the National Audit Office gathered relevant information, which it has included in its supplementary memorandum to the Committee.


  13.  The National Audit Office Report found some potential governance problems which it exposed in the Report (paragraphs 3.11-3.15). It set out the potential implications of these but it found no tangible delivery issues at that stage—possibly because the LIFT schemes were at an early stage.


  14.  The National Audit Office's methodology was set out in detail in the Report to allow scrutiny. The CIPHP main criticisms are that the Report relies on surveys of interested parties, it looks at only the first six projects completed and that the Report is not structured around the four main high level questions it identified. The independent quality review of the Report commissioned by the National Audit Office concluded that a range of appropriate methodologies had been used and gave the Report a good mark—an average of 4.1 out of five. Further information on methodology is set out below.

  15.  Questionnaires were issued to three main interest groups, principally to gain an understanding of the adequacy of the standardised procurement process devised by Partnerships for Health. These were: all LIFT Project Directors; shortlisted bidders across the 42 LIFT schemes extant at that time; and the Local Pharmaceutical Committees in each case study areas.

  16.  The National Audit Office aimed to obtain a balanced view of the scheme by consulting widely across parties with differing views on LIFT, although not all agreed to speak to the National Audit Office team. As is normal in National Audit Office studies, the team had access to a panel of experts who were invited to comment on the audit plan and the structure and draft of the Report, thereby providing quality assurance. The panel for the LIFT Report included amongst others a dental practitioner, a general practitioner and the Chief Executive of a PCT.

  17.  A key element of the evidence was provided from analysis of six case studies (Appendix 1, paragraph 8). The first six schemes to have completed the procurement process were chosen for practical reasons and because they represented all three waves of LIFT. These case studies were used to identify best practice and lessons for future LIFT developments. A wide range of stakeholders were interviewed in the local health economy for each case study and key documents were reviewed. The National Audit Office was aware of concern in some areas about LIFT schemes. It flagged this up in the Report (paragraph 2.4) but the projects were not advanced enough to be analysed as case studies.

  18.  The CIPHP note suggests that the National Audit Office team was unaware of Department of Health guidance "Learning from Post Project Evaluation" although this guidance is referred to in paragraph 3.8 of the Report. This paragraph of the National Audit Office Report went on to say "there is no clear guidance recommending either its nature or its timing" in relation to the post project evaluation of LIFT. This latter reference relates to the National Audit Office's concern that the Department and Partnerships for Health should issue detailed guidance—as referred to in recommendation 9 of the Report.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2006
Prepared 4 July 2006