some default text...
Select Committee on Public Accounts Minutes of Evidence


Supplementary memorandum submitted by the National Audit Office

  After the hearing, Mr Richard Bacon submitted a series of supplementary questions to the National Audit Office, what follows is their response.

Q1.   How many gateway reviews did the National Audit Office see in relation to the National Programme for IT in the NHS?

  A1.  The National Audit Office saw just under 30 Gateway reviews in relation to the National Programme for IT in the NHS.

Q3.   Why were the contents and some results of gateway reviews mentioned in some detail in recent National Audit Office reports on the Child Support Agency and on the ASPIRE contract for HM Revenue and Customs, but not for the National Audit Office's report on the National Programme for IT in the NHS?

  A3.  The National Audit Office's aim in preparing the report was to provide a reasonably succinct overview of the National Programme overall. As with other reports, we have to consider the competing merits of the information at our disposal. In this instance, we did not consider it appropriate to bring into the report the recommendations made through such a large number of Gateway reviews. As the question indicates, we do sometimes cover detailed results from Gateway reviews in our reports, but this is by no means always the case.

Q4.   Before producing its report on the NPfIT (pursuant to Caroline Flint's parliamentary answer of 24 May 2006):

    (i)   Did the National Audit Office see the results of internal audits on the National Programme for IT in the NHS?;

  A4.  (i)  Yes.

Q5.   How many independent reviews have been commissioned under contract (pursuant to Caroline Flint's parliamentary answer of 24 May 2006) such as:

    (i)   The study by McKinsey to inform the approach before the start of the programme;

    (ii)   from other suppliers to establish the value for the National Health Service and taxpayer achieved through the contracts; and

    (iii)   any other studies? And

    (iv)   before producing its report on the NPfIT did the National Audit Office see the results of these independent reviews? And if so,

    (v)   Which ones? And

    (vi)  what were the key results of each study?

  A5.  National Audit Office saw a number of independent reviews commissioned by the Department as it conducted the study, including the McKinsey review. Many of these are referred to in the C&AG's report:

    —  Paragraph 3.2 refers to a report commissioned from IT industry analyst Ovum, which compared the prices achieved by NHS Connecting for Health with estimates of the prices that could have been achieved by individual NHS organisations purchasing the same services separately.

    —  Paragraph 3.20 refers to a review commissioned in February 2005 jointly by NHS Connecting for Health and BT which confirmed that the overall technical architecture deployed by BT was sound and capable of supporting NHS Connecting for Health's requirements.

    —  Paragraph 4.10 refers to the research commissioned by NHS Connecting for Health from Ipsos MORI to track awareness and understanding of the Programme across the NHS.

  (Transferred to the Department of Health the request for key results from each of these studies)

Q6.   What is the most recent figure for total expenditure on central administration of the NPfIT since 2002, and does the National Audit Office consider this value for money so far?

  A6.  (Transfer the request for the most recent figure for total expenditure on central administration of NPfIT to the Department of Health). Paragraph 4 on page 1 of the C&AG's report states that "it will be some time before it is possible fully to assess the value for money of the Programme, as this will depend on the progress made in developing and using the systems it is intended to provide". In the same way, the National Audit Office is not in a position to conclude whether or not central administration expenditure is value for money so far. The National Audit Office can only judge the value for money of the National Programme when there is a sufficient body of evidence of benefits being achieved for patients.

Q8.   Please provide a breakdown of the figures referred to in paragraphs 1.20 to 1.25 of the National Audit Office report showing how one arrives at a total of £9.2 billion for forecast national expenditure

  A8.  The breakdown of forecast national expenditure is shown in paragraph 1.20-1.25 of the C&AG's report. The sum of the figures in paragraphs 1.20 to 1.25 is actually £9.078 billion, which rounded is £9.1 billion. (There is a typographical error in the heading above paragraph 1.20, which give the figure of £9.2 billion).

Q9.   What was the total cost of the National Audit Office report on the National Programme for IT in the NHS?

  A9.  The total cost of the National Audit Office study was £578,000 (including £72,000 consultancy). This is higher than the average cost of a National Audit Office VFM study, but this reflects the scale and complexity of the subject. For example, each of the ten major contracts is comparable in size to a major PFI deal. The investigation also required interviews and visits with more than 30 organisations across England, including the programme regional clusters, suppliers, NHS Trusts, professional, regulatory and academic bodies, and trade unions. The body of knowledge built up on this study also leaves the National Audit Office well placed to monitor further developments on the National Programme.

Q10.   Paragraph 1.26 of the National Audit Office report refers to forecasts of local expenditure (made in the investment appraisals carried out at the time of the award of the main LSP contracts in late 2003 and early 2004) which total £2.6 billion excluding PACS. Did the National Audit Office subject the figure of £2.6 billion referred to in paragraph 1.26 to independent verification?

Q11.   Is the National Audit Office satisfied that the figure of £2.6 billion represents a realistic estimate of total forecast local expenditure?

  A10-A11. The National Audit Office did not subject the figure of £2.6 billion to independent verification. As is described in the report, this was the figure used in the Department's investment appraisals carried out at the time of the award of the main LSP contracts in late 2003 and early 2004. That is why the National Audit Office report made the recommendation (recommendation (e) page 7) that: "the Department, NHS Connecting for Health and the NHS should commission a study to measure the impact of the Programme on local NHS IT expenditure—both costs and savings—where systems are now being deployed, and, together with its quantification of financial and non-financial benefits (recommendation (d)), use this to provide an up to date assessment of the overall investment case for the Programme."

Q12.   Given the significant changes in the contracts reflected in the increase in forecast national expenditure from £6.2 billion to £9.2 billion, does the National Audit Office believe that the figure of £2.6 billion for forecast local expenditure is still correct?

  A12.  Paragraph 1.26 of the National Audit Office notes that in addition to the £2.6 billion a further £775 million in local expenditure was estimated as being needed as a result of the extension of the central contracts to include PACS. This is the basis of the total forecast local expenditure of £3.4 billion referred to in the heading to paragraph 1.26. The National Audit Office recommended that the Department, NHS Connecting for Health and the NHS should commission a study to measure the impact of the Programme on local NHS IT expenditure (recommendation (e) on page 7 of the C&AG's report).

Q14.   What are the national costs of PACS?

  A13-A14.  The contracted central expenditure on PACS is £245 million for the central data store over 10 years to 2013-14 (paragraph 1.20 page 25 and Figure 3 Page 17). These contracts would involve local IT spending of an estimated £775 million, to be paid for by local NHS Trusts (paragraph 1.26 and Figure 3 page 17).

  (Otherwise, transfer to the Department of Health)

Q16.   What assessment has the National Audit Office made of the statement by Connecting for Health that: "It is generally accepted in the IT industry that implementation costs are some 3-5 times the cost of procurements. That is reflected in the business case that was made for the National Programme"

  A16.  This is a statement attributed to NHS Connecting for Health in response to media enquiries in October 2004. It was not considered as part of the National Audit Office analysis.

Q17.   What is the source for the statement in paragraph 4.8 of the National Audit Office report that: "The programme is the only major public sector IT project in the UK where the responsible body has a dedicated website to provide information on the progress of the project"?

  A17.  We are aware from our research that there are other sites providing information on aspects of public sector IT projects, such as, the Criminal Justice IT Programme. However such websites are not a common feature of UK public sector IT initiatives and so far as we are aware none provides as much information as the NPfIT website.

Q18.   In the National Audit Office report, Patient Choice at the point of GP Referral, the National Audit Office expressed concerns about the costs of interim systems that will have to be replaced by full systems at a later date. Does the National Audit Office have similar concerns about the costs of interim systems being deployed in the NPfIT because of the late delivery of core software?

  A18.  National Audit Office recommendation 9a on Page 6 of the National Audit Office NPfIT report highlights the importance of ensuring a robust, engineering-based timetable for delivery which NHS Connecting for Health is confident its suppliers are capable of achieving.

  Recommendation 9c on Page 6 recognises that whilst some adjustment of suppliers' delivery milestones may be a necessary response to suppliers' difficulties in delivering, NHS Connecting for Health should not allow this to compromise the eventual achievement of the vision of the fully integrated care record service that was the objective of the Programme at its inception.

  Assessing the net impact of deploying interim systems is complicated because both costs and benefits are affected. We would expect the assessment of local costs and benefits we recommend (recommendations 9e) to cover these issues.

Q19.  What concerns does the National Audit Office have about the long timeframe for the NPfIT, given that the NHS and its demands for IT could change substantially between now, when systems are being defined, and the time they are deployed?

  A19.  The NPfIT is a ten year programme. Just as the last ten years have seen large changes in the NHS, for example in organisation, medical techniques and information technology, so further changes should be expected in the future, even if the precise form of these changes cannot now be fully foreseen. Paragraph 3.10 identifies the mechanisms put in place by NHS Connecting for Health to help ensure continuing value for money over the life of the contracts, such as ensuring technology is continuously improved and refreshed so that systems continue to meet the changing needs of the NHS throughout the contract periods. The contracts with suppliers also include mechanisms for incorporating changes in the NHS's requirements. It will be important that NHS Connecting for Health uses such mechanisms to make sure that the Programme adapts to the changing nature and needs of the NHS.

Q20.  Given the apparently long timeframes, at what point will the National Audit Office judge whether the programme has been value for money?

  A20.  Paragraph 4 of the C&AG's report explains that "the main implementation phase of the Programme and realisation of benefits is mainly a matter for the future and it will therefore be some time before it is possible fully to assess the value for money of the Programme, as this will depend on the progress made in developing and using the systems it is intended to provide." The National Audit Office will be in a position to judge whether the Programme has been value for money when there is a sufficient body of evidence on patient benefits from the Programme.

Q21.   In paragraph 3.5 of the National Audit Office report the National Audit Office praises the work done on requiring suppliers to produce a "Proof of Solution". Why does the National Audit Office believe this good work has not led to core systems being delivered on time?

  A21.  As reported in paragraph 3.5, "Proof of Solution" tests were carried out in a simulated environment to show whether potential suppliers' systems could meet a number of scenarios devised by NHS Connecting for Health. They are helpful because they provided a degree of confidence that the potential contractors understood NHS Connecting for Health's requirements. They therefore help to reduce risk but of course cannot on their own eliminate it.

  The main factors that have contributed to delays in delivery are (i) additions and changes to the scope of the Programme (paragraph 1.7), (ii) difficulties by some suppliers in meeting agreed delivery dates (paragraph 1.11), (iii) a need to pilot more extensively the clinical record before full scale implementation (paragraph 1.11), (iv) system integration being highly complex and taking longer than suppliers initially planned (paragraph 1.9), and (v) a move away from the original structure of phases and releases, with new functionality being delivered in a different order from that originally envisaged (paragraph 1.11 and Figure 2 on page 12).

Q22-23.   Before producing its report on the NPfIT, did the National Audit Office see the document Delivering 21st Century IT Support for the NHS: National Strategic Programme? If so, did the National Audit Office see:

    (i)  the version with four appendices including Appendix 3 with the Project Profile Model completed by Sir John Pattison showing a risk score of 53 out of a maximum possible 72 and estimated whole life costs of £5 billion; or

    (ii)  the version with only three appendices from which the Project Profile Model with its risk scoring and estimated whole life costs had been removed?

Q23.   If the National Audit Office did see the version including the Project Profile Model

    (i)  Why was no reference made to the PPM scoring in the National Audit Office report?

    (ii)  What is the National Audit Office's assessment of the PPM scoring broken down by category; and in total?

  A22-23.  The National Audit Office saw the published version of Delivering 21st Century IT Support for the NHS: National Strategic Programme with three Appendices. Whilst there may have been earlier drafts of this document, this is the only substantive version definitively representing the Department's views.

Q28.   Why are there no detailed references in the report to the numerous technical difficulties encountered by users of systems deployed under the contracts?

  A28.  Paragraph 4 of the Summary to the C&AG's report sets out the scope of the National Audit Office examination: "we examined the progress being made in delivering the systems against the original plans and the costs of the Programme; the steps taken by the Department, NHS Connecting for Health and the NHS to deliver the Programme; how the IT systems have been procured; and how the NHS is preparing to use the systems delivered." This National Audit Office study did not, therefore, include within in its scope an examination of the technical difficulties that users of deployed systems may have encountered.

  The reason why we drew the scope of the report in this way was that we considered it was too early to examine whether, for each project in the Programme, these were signs of insoluble problems from which the National Audit Office could draw far reaching conclusions.

Q29.   Why did the National Audit Office report the July 2005 Mori poll and not the February 2006 Mori poll?

  A29.  At the time of finalising our report, the analysis of the February 2006 Mori poll was not complete.





 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2007
Prepared 17 April 2007