Select Committee on Public Accounts Eleventh Report


LSE PUBLIC POLICY GROUP

Review of National Audit Office Report HC 730 Session 2001-02

Public Private Partnerships: Airwave

PUBLIC PRIVATE PARTNERSHIPS: AIRWAVE

  1.  In February 2000, the Police IT Organisation (a non-departmental public body, hereafter PITO) signed a deal with British Telecommunications plc (now the separate company O2) for provision and management of a national radio communications system across all police forces in the UK. The project is still in its early pilot stages and is expected to cost nearly £1.5 billion over 19 years' use. This report examines value for money of the project to date.

Administrative and management context

  2.  There is a clear overall explanation of the Airwave public-private partnership. The project involves a diverse range of public and private stakeholders at various levels of jurisdiction from local police forces up to Home Office. A good explanation is given of the TETRA technology, the three-stage service model, and the PFI charging mechanism and coverage pay-off. Issues relating to the potential problems of sharing the new network with other emergency services are effectively set out. Deficiencies with the current mobile communications systems used by police forces are effectively described. The powers of the Home Office to secure central procurement of a national replacement for current systems, which are currently operated on a force-by-force basis, are made clear, as is the strong Home Office preference for centralisation.

Structure and presentation

  3.  The report is well structured and clear. The three main Parts follow on logically through the procurement and implementation process to date. The Executive Summary gives a reasonably concise précis of the main points. However, a couple of points seemed to be more clearly communicated in the Executive Summary than in the main body of the report. For example, a substantial portion of the income stream flows to O2 at an early stage, thus reducing the incentive for them to take action to meet outstanding contractual conditions. Using narrative headings for each of the three Parts is helpful for readers, and they are not too evaluative. The Part 1 heading is: "The procurement strategy was well thought through but was not delivered in full". It rather vaguely relates to the actual contents of Part 1. The Part 2 heading is: "In difficult circumstances the procurement was well managed by the project team". This sounds rather melodramatic, and seems to imply a rather loose definition of "well managed" given the actual contents of Part 2. Presentation was reasonably good with a few outstanding points, such as Figure 2 and its pullout section. In general, this is a focused and well-written report. It deals with a complicated and topical area of PFI and new technology in an accessible way.

Graphics and statistics

  4.  The limited use of charts and other forms of statistical presentation reflect the limited use of data per se in the report. Two useful graphs "breakdown of police spending" (Figure 3) and "time spent in the police station" (Figure 17). There is a table setting out PITO advisers' costs, tabulating the total of £1.9 million over the procurement period (Figure 7). A very basic table shows the public sector comparator costs against the Airwave should-cost model (Figure 11). Surprisingly, there are no Appendices giving more detailed breakdown of the two predictions or elucidating the difficulties of constructing both cost models. Figure 19 on the benefits of Airwave also seems vague.

Scope

  5.  The report focuses narrowly on a single procurement, but also draws out potentially wider implications for other procurement processes involving new or uncertain technologies. Important issues relating to PFI procurement are addressed, such as charging mechanisms, the problems of specifying services requirements by contract in advance, contractor service delivery standards, public sector resource and expertise atrophy, single bidder competitions, and risk transfer. The scope seems to be defined slightly differently at two points. In Appendix 2, the report's focus is defined as whether Home Office and PITO "achieved objectives in procuring and implementing" Airwave. In the Executive Summary paragraph 4, however, the scope is defined as "[examining] the prospects for achieving value for money from negotiation and early implementation of the core service". By specifying scope as future prospects for VFM rather than actual performance to date, the study team acquire an exit clause which lets them skirt over any hard-edged critical comments about the procurement of the relationship with O2. For example, PITO's lack of success in any gain for the police if potential sharers take-up Airwave passes without evaluative comment (paragraph 11). By contrast previous NAO studies have highlighted the importance of using innovative mechanisms to guarantee that public agencies can share in mutual benefit incentives (for example, HC 21, The Radio-communications Agency's Joint Venture with CMG).

Methods

  6.  The methodology used is set out in Appendix 2 and combines interviews and documentation review. However, there is very little data collection and "number crunching" analysis on either the "Should-cost" model or on the public sector comparator. The study team point out that information was limited, even for PITO. Nevertheless it would have been good if the study team had set out to deconstruct independently the data available, perhaps commissioning some accounting analysis from specialists in this field. The study team did address the business model analysis which PITO constructed, but only in a limited way (paragraphs 3.26 to 3.28). for example, PITO estimated a reduction of at least 10 per cent in time spent in the police station and efficiency gains of up 30 per cent. Assuming that police officers are likely to generate more paperwork by being outside the police station an extra 10 per cent of the time, these reductions seem rather optimistic and under-analysed. We were unclear how much data systems would allow functions to be fully automated, for example, creating an automatic log of police officers' movements and timings which they would not then need to redo in reports? Also, the report suggests that increases in the cost of services is likely to be compensated by a reduction in staff-costs and/or services. It is not clear whether these are net estimates which include increased cost factors deriving from the new contracted services. Elsewhere in the report the potential for savings or service enhancements arising out of new police working methods were given less prominence and there appears to be no overall assessment of such projections, even though these are relevant to assessing the value for money of Airwave. Although Figure 2 shows police officers receiving photographs and text on handheld computers, for the rest of the time the report is incredibly unspecific on what "data services" will mean. For example, will police radios give continuous GPS (positioning) information to officers on where they are, and to their managers? Will police services get GPS services such as mapping and direction-giving? Assuming that these presumably high priority services are indeed included in the contract, where do they sit between the core, the "market exclusive" services which are only O2 can supply, and the "market competitive" services which other companies can supply? These issues should be crystal clear for readers, not left foggy.

Conclusions and recommendations

  7.  The major aspects of PFI procurement are covered well, given the obvious limitations in data available and in co-operation from O2 as a private sector company. The "lessons learned" section on pages 6 and 7 is relevant to PFI in general as well as to this particular project. Some interesting incentive issues are highlighted which have a considerable impact on national procurement schemes—for example, the imperative that all police authorities sign up to Airwave regardless of increased cost and resulting potential reduction in local services; and the incentive for O2 to increase the scale of the project (extending it to other services) despite potential adverse impacts on service quality. On other occasions the study team seem to not dig too deeply into what seem to be obvious barriers to effective implementation, and bland-out phrasing are used to cover up problems: "lessons learnt from the pilot and earlier cohorts will b disseminated in order to make migration of later forces smoother" (paragraph 3.6). It is clear from the report that the roll-out schedule is very tight given the 2004 spectrum expiry and events such as the 2002 Commonwealth Games. It does not yet seem clear that police authorities will have secure and reliable means of radio communications by then, given that the first pilot is only just underway. And the question of contingency plans if systems are not ready on time is not covered much. The problem of coverage specification dealt with in paragraph 3.7 was not clearly enough explained, despite its critical nature in relation to the success of the system. How can it be that areas of "incidental coverage" have emerged? How can the study team be so sure that the "procurement was well managed by the project team" (as the Part 2 heading says) if they have left contractual room for "incidental coverage"? Occasionally the report also seems to reiterate familiar hype around PFI without saying specifically what the main advantages of PFI would be for this procurement. Paragraph 1.1, for example, is quite peculiar in its rather abstract, specifically "context-less" assessment of PFI. The study team write that:

      "the PFI can offer the prospect of better value for money than conventional procurement because it adds a wider range of private sector capabilities to those previously available to the public sector [...]. This means that, in specifying what they want from PFI deals, procuring departments need to avoid imposing unnecessary constraints on how the private sector carries out projects".

  The validity of these two sweeping statements, especially the latter, are by no means borne out by the issues raised in the report. It seems odd that they should appear in the opening paragraph as a kind of "universal truth" without more specific reference to the advantages of PFI in this particular case.

Overall

  8.  This is a valuable report, at the higher end of the three scores, which deals with some important issues of PFI applied to new technology. There are some data weaknesses and our readers were concerned that at times the report is a bit vague and the study team withholds analysis from some potentially disconcerting issues.

Minor Points

  The key to Figure 2 contained some interesting points on the "payment-to-coverage" pay-off, one of the more interesting issues of the report. Unfortunately, the rather toy-town portrayal of Airway in the pullout section was far too simple to add any real value to the analysis. We assume that officers featured would have to be well within designated coverage areas, portrayed rather confusingly in Figure 16. The text states that "Figure 16 shows an example of areas where coverage is guaranteed as well as areas where coverage is obtained but not paid for and therefore not guaranteed". In fact all that the Figure is to shade two areas, point arrows towards them designating both types of coverage, and then add towns in one area type or the other.

LSE Public Policy Group Pro Forma Scores for HC 730: Public Private Partnerships—Airwave

  1.  To what extent does the report clearly describe the administrative and managerial context within which the area examined is carried out? = 4

  2.  To what extent is the report well-structured and well-written, and does it include an effective executive summary? = 3

  3.  To what extent were graphics and statistics appropriately used, and how well presented were any that appeared? = 3

  4.  To what extent is the rationale for the report's scope clearly set out (within NAO's remit, which excludes questioning the merits of policy objectives)? = 3

  5.  How far is the methodology clearly set out? And does it include an appropriate range of evaluative criteria and techniques to answer the question posed in the specification? = 3

  6.  To what extent did the conclusions provide a balanced view of successes and shortfalls in performance, the underlying causes, and make reasoned and cost-effective recommendations for remedies? = 3

  7.  To what extent has the study been successful in meeting its objectives, and maintaining or enhancing NAO's reputation? = 3




 
previous page contents

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 7 November 2002