Select Committee on Public Accounts Fifth Report


FIFTH REPORT


The Committee of Public Accounts has agreed to the following Report:—[1]

MINISTRY OF DEFENCE: MAJOR PROJECTS REPORT 2000

INTRODUCTION AND SUMMARY OF CONCLUSIONS AND RECOMMENDATIONS

1. Since 1984 the Ministry of Defence (the Department) has reported each year to Parliament on progress in procuring major defence equipments.[2] The Comptroller and Auditor General's Major Projects Report 2000 covers the period to 31 March 2000 and is the first to be produced in a new format reflecting the changes in project accounting and approvals brought about by the introduction of Smart Acquisition and Resource Accounting and Budgeting. Under the Smart Acquisition lifecycle (Figure 1), there are two key approval points, Initial Gate, at which parameters for the assessment phase are set, and Main Gate, at which performance, time and cost targets for the demonstration and manufacture phase are set. The Major Projects Report 2000 details progress on the top 30 equipment procurement programmes split, in accordance with Smart Acquisition principles, between the 20 largest projects on which the main gate decision has been taken and the 10 largest projects yet to reach that point. The Report includes data on the cost and time performance of projects against the Department's estimates and, for the first time, data on the technical performance of projects against the Customer's key requirements.[3]


2. Our predecessors held two hearings on the Major Projects Report 2000. At the first session on 15th January 2001 they examined the impact which project slippage had had on the operational capability and costs of four case study projects - the Air-Launched Anti-Armour Weapon (Brimstone); the Type 45 Destroyer; the Medium Range TRIGAT anti-tank weapon system and the BOWMAN communications system.[4] At the second session, on 17th January 2001, they examined the progress of defence projects against the cost, timescale and technical performance parameters set by the Department.

3. At both hearings our predecessors took evidence from the Chief of Defence Procurement and from the Deputy Chief of Defence Staff (Equipment Capability) about the role of the Equipment Capability Customer organisation responsible for deciding what capability is required and how much funding to allocate to it. We have decided to produce two separate reports based on that evidence. This report examines whether there are signs of improvement in the Department's performance in acquiring projects within approved cost, time and technical parameters and the Department's progress in implementing Smart Acquisition. Since schedule performance has been worsening, the report also examines the adverse cost and operational impact of slippage. The second report[5] considers the role of the new Equipment Capability Customer Organisation in improving the provision of defence capability; and how the Equipment Capability Customer organisation prioritises capability needs and funding.

4. Our main conclusions are as follows:

  • There are tentative signs of improvement in the technical and cost performance of projects but schedule performance has deteriorated with adverse cost and operational impacts. Delays to two thirds of the projects in the Major Projects Report 2000 which have passed Main Gate approval have led to capability shortfalls, with the Armed Forces not receiving the equipments they need when they need them.

  • The Department is expecting Smart Acquisition to improve acquisition performance through, among other things, better relationships with industry, use of positive incentives and more realistic attitudes to risk. Progress is hard to judge at this stage, since projects which are still over time and budget were started before Smart Acquisition. Now that Smart Acquisition is well underway, the Department could soon be expected to provide concrete evidence of its success.

  • Some three years after its inception, the Department has not set appropriate performance measures for Smart Acquisition. The Department needs to address this shortcoming and set robust and challenging performance measures against which to monitor the achievement of the principles underlying Smart Acquisition.

5. Our detailed conclusions and recommendations are as follows:

Project technical and cost performance: tentative signs of improvement

      (i)  The 20 projects in the Major Projects Report 2000 are expected to meet 98 percent of their Key Requirements, with three projects forecast to miss one Key Requirement due to technical difficulties. Following Smart Acquisition principles, there may in future be other reasons why a Key User Requirement may not be met. For example, the scenario in which the equipment is intended to operate may change, or the customer may choose to trade off a requirement against an alternative higher priority capability. The Key User Requirement should be changed in such cases and the rationale explained, instead of just registering a failure to meet the Requirement (paragraph 8).

      (ii)  The 20 post-Main Gate projects in Major Projects Report 2000 are currently forecast to cost 5.7 per cent more than at approval. The Department suggested that this cost performance would compare well with other countries, but the Department does not systematically evaluate its performance against other organisations procuring similar equipments. The Department should undertake a more systematic evaluation of its performance against both overseas defence acquisitions and commercial comparators (paragraph 9).

Schedule performance: deterioration (adverse cost and operational impacts)

      (iii)  Seventeen of the 20 projects in the Major Projects Report 2000 have slipped against the in-service dates set at the approval equivalent to Main Gate and seven have slipped by a total of 63 months during the last year. This is an unsatisfactory situation which has negative cost and performance implications. As new projects approved under Smart Acquisition principles begin to enter the Major Projects Report we expect to see the rate of slippage reduce (paragraph 18).

      (iv)  The Department appears on occasion to excuse delays by arguing that they may allow improvements to be included that were not available at the planned in service date. For example, the Department argued that Brimstone's image recognition capability would not have been available if it had entered service on time some 10 years earlier, and that the missile would now have needed upgrading; but it admitted that this was an inadvertent consequence of the delay rather than a planned strategic decision. The Department needs to be clear about the time it is prepared to wait for new equipment in order to secure the latest improvements (paragraph 19).

      (v)  The Department suggested that it acted quickly when it decided to withdraw from programmes, for example replacing the Common New Generation Frigate programme with the Type 45 Destroyer, and deciding to re-open the BOWMAN competition. In both of these cases the Department moved fast once the decision was made, but the time taken to reach the decision itself was too long given the track record of problems on both programmes. The Armed Services will not receive, when originally planned, the capabilities which these equipments would have provided. The Department should make decisions on when to withdraw from programmes in a more timely manner in future to prevent such capability gaps occurring (paragraph 20).

      (vi)  The adverse operational impacts of the delays in entry into service of all four case study equipments is unacceptable. Without the Type 45 Destroyer the capability of the Navy will be less between 2002 and 2007. In the meantime the Sea Dart missile, supplemented by other systems such as SeaWolf and Phalanx, will have only limited effectiveness against modern sea-skimming missiles. Troops still do not have a secure communications system because of the delays on BOWMAN, the MILAN missile has limited ability to defeat modern armour and the RBL755 cluster bomb, which Brimstone will replace, is only effective against soft-skinned targets (paragraph 21).

      (vii)  Over a number of years our predecessors have highlighted numerous problems caused by equipments being unable to operate effectively in the required environment. To add to this list, the Department has admitted that the upgrade of the Sea Dart system has been delayed because the software could not discriminate targets against the sun. The Department should develop processes to enable it to better anticipate such environmental difficulties by drawing on its own past experience and the expertise available to it in DERA and industry (paragraph 22).

      (viii)  The Sea Dart system was designed in the 1960's and contains components which are obsolete and which the Department is having to go to a great deal of effort to replace. Obsolescence is another example of the Department having to spend more money because replacement equipment is not available on time. The Department should ensure that the costs and resource effort required to manage obsolescence are properly managed and minimised (paragraph 23).

      (ix)  It is unacceptable that the Department has wasted between £35 million and £102 million in pursuing an unworkable solution to the BOWMAN requirement. The precise scale of the write-off is unclear but, even at the lower end, it would probably cover the cost of the sonar which the Department now intends to fit on the Type 45 Destroyer. We expect the Department to minimise the amount written-off and to be able to demonstrate that it has utilised the results of the abortive BOWMAN expenditure to good effect (paragraph 24).

      (x)  The Department has written off £115 million of Medium Range TRIGAT development costs. It argued that, if this money had not been spent on Medium Range TRIGAT, it would have been spent on an alternative solution. The fact remains that the Department failed to procure an effective weapons system, so the money has, in effect, been wasted (paragraph 25).

Smart Acquisition: efforts to improve procurement performance

      (xi)  Our predecessor's report on the 1994 Major Projects Report recorded that the Department had stated that it used three-point estimates on all major projects. It is therefore surprising that only two of the ten assessment phase projects in the Major Projects Report 2000 had three-point estimates for time and cost at 31 March 2000. The Department could not justify this failure, although it did make progress quickly in generating three-point estimates after the National Audit Office had highlighted the problem. We also note the Department's assurance that three-point estimates for Bowman will be available soon and expect all projects in the Major Projects Report 2001 to have robustly generated three-point estimates (paragraph 32).

      (xii)  The Department highlighted the Tomahawk Land Attack Missile and Trident missiles as successful projects. One obvious common factor is that both of these projects were largely off-the-shelf purchases from the United States, but the Department should look to identify other common features which may enable it to emulate these successes in future (paragraph 33).

      (xiii)  The Type 45 destroyer is an anti-air warship being procured in two parts comprising the warship and the Principal Anti-Air Missile System (PAAMS). The Department recognised the risks but assured us that it was confident there would be no interface problems between the warship and PAAMS. That would be a significant achievement, and we look forward to hearing whether the Department has achieved it (paragraph 34).

      (xiv)  The Department said that it was now more cautious about accepting over optimistic bids from industry. For example, it had not continued with the ARCHER industrial prime collaboration contractor on BOWMAN because it believed that the company was making extravagant promises that would not be deliverable. Such scepticism is a welcome development (paragraph 35).

      (xv)  The Department seeks to transfer the cost risk of resolving technical difficulties and the delays it causes to industry by using performance based contracts. For example, the Department received substantial liquidated damages from Lockheed Martin in respect of the delays to the Hercules C-130J. Whilst liquidated damages may offset the costs of delays the Department still bears the operational implications of the delays, a fact which re-emphasises the importance of planning to and agreeing realistic timescales at the outset (paragraph 36).

      (xvi)  The Department regards control of costs through contracts as straightforward, but believes that incentivising contractors to deliver on time is more difficult. The principle of incentivising contractors to deliver on time is a good one, but it is important that incentives are based on a thorough understanding of what is achievable and are effective in securing added performance. The Department needs to ensure that it does not give contractors extra benefits for simply doing what they should be doing anyway under the contract (paragraph 37).

Smart Acquisition: little progress in measuring success

      (xvii)  The Department is not clear what level of risk it expects will remain in projects at Main Gate and how this will be measured. A much clearer definition of the acceptable level of risk at Main Gate is needed and the Department must give priority to defining this and how to measure it (paragraph 43).

      (xviii)  Under Smart Acquisition, the Department bases its Main Gate Approval on the cost and in-service date of an equipment which it is 90 percent confident of achieving. We note that the SeaWolf Mid-Life Update project has already exceeded the in-service date which the Department was 90 percent certain of achieving by some three months. This does not bode well for the future (paragraph 44).

      (xix)  The Department does not expect there to be big improvements in the timescale performance reported in the Major Projects Report until 2002, some four years after Smart Acquisition was first introduced. Whilst the timescale performance reported in the Major Projects Report will continue to reflect pre-Smart Acquisition projects for some years, we will expect to see signs of improvement as new "smart" projects come through (paragraph 45).

PROJECT TECHNICAL AND COST PERFORMANCE: TENTATIVE SIGNS OF IMPROVEMENT

6. Under Smart Procurement, Key User Requirements are agreed at Main Gate and form a contract between the Integrated Project Team and the equipment capability customer. The Major Projects Report includes, for the first time, a section on technical performance which reports whether projects are currently expected to meet the Key User Requirements specified by the customer.[6] Each project has ten Key User Requirements attributed to it and, with three exceptions, all twenty of the major projects are expected to meet all of their Key User Requirements.[7] Asked what the point of projects was if they did not meet the military requirement,[8] the Chief of Defence Procurement said that it might be quite sensible for a project not to achieve all of its requirements if, in fact, one turned out to be unachievable within the state of technology at the time. It was not self-evident that every requirement (for example, there are 14,000 associated with the Astute Class submarine) had to be met, which is why the Department has identified ten Key User Requirements.[9]

7. There are signs that the Department is getting better at controlling costs during the demonstration and manufacture of equipment with forecast costs down 0.2 percent since last year and now standing 5.7 percent higher than at approval.[10] The Department accepted that 5 percent was 5 percent too many but suggested that, compared with the high technology defence programmes undertaken in other countries, it was a record to be proud of.[11]

Conclusions

8. The 20 projects in the Major Projects Report 2000 are expected to meet 98 percent of their Key Requirements, with three projects forecast to miss one Key Requirement due to technical difficulties. Following Smart Acquisition principles, there may in future be other reasons why a Key User Requirement may not be met. For example, the scenario in which the equipment is intended to operate may change, or the customer may choose to trade off a requirement against an alternative higher priority capability. The Key User Requirement should be changed in such cases and the rationale explained, instead of just registering a failure to meet the Requirement

9. The 20 post-Main Gate projects in Major Projects Report 2000 are currently forecast to cost 5.7 per cent more than at approval. The Department suggested that this cost performance would compare well with other countries, but the Department does not systematically evaluate its performance against other organisations procuring similar equipments. The Department should undertake a more systematic evaluation of its performance against both overseas defence acquisitions and commercial comparators.

SCHEDULE PERFORMANCE: DETERIORATION (ADVERSE COST AND OPERATIONAL IMPACTS)

10. Although there is evidence that the Department has begun to control costs better, control of time remains a problem. 17 out of 20 post-Main Gate projects have slipped with the average project delay increasing by three months in the last year. [12] The reasons for these new delays break down into three roughly equal parts: aligning missile projects with aircraft projects (24 months), technical difficulties (19 months) and budgetary deferral (18 months).[13]

11. The Air-launched Anti-Armour Weapon (Brimstone) will enter service eleven years later than originally planned and 13 months later than approved at the equivalent of Main Gate.[14] At an earlier hearing the Department had suggested that the equipment now expected to enter service would be better than if had there been no delay.[15] Pressed on the extent to which improvements in capability which arose as an accidental outcome of delays could compensate for the operational and cost penalties associated with the delayed introduction of equipment, the Department said that the image recognition which allows the Brimstone missile to discriminate between various different targets would not have been available if it had been introduced when originally planned.[16] The Department conceded that this justification was no basis for a strategy as it meant that the Armed Services did not have the right equipment in the frontline[17] but emphasised that it was important to keep pace with the technological change. For example, the twelfth Type 45 destroyer would be of much greater capability than the first.[18]

12. In July 2000, the Department announced it was not confident that Archer Communications Systems Ltd (ACSL) could deliver a solution to the BOWMAN requirement and that the competition would be re-opened, some 12 years after the requirement was first raised. The Department wished it had stopped working with ACSL sooner, but insisted that it acted quickly once it received the final bid on 23 June 2000, announcing the termination with ACSL on 25 July.[19]

13. In April 1999, the Defence Ministers of the United Kingdom, France and Italy decided not to proceed with the collaborative Common New Generation Frigate programme. The Department is now procuring the Type 45 destroyer as a national solution to the United Kingdom requirement to provide air defence to the fleet. The new ships will be equipped with the Principal Anti-Air Missile System (PAAMS) which is being procured collaboratively with France and Italy.[20] The Type 45 is expected to enter service almost 5 years later than the date forecast for the Common New Generation Frigate and with some capability shortfalls which will be addressed through an Incremental Acquisition Programme.[21] The Department admitted that the collapse of the Common New Generation Frigate programme left it with a gap in replacing the existing Type 42 destroyers. It was nevertheless noteworthy that within not much more than a year of the collapse it actually had on order a programme of ships to deploy PAAMS.[22]

14. Figure 2 lists some of the capability shortfalls which have been caused by delays on the case study projects examined by the National Audit Office. This huge gap in capability has been going on for many years. The Department re-assured our predecessors that it always regretted the delay in capability. It was not complacent, and recognised that the task was to ensure that the situation in future was very much improved.[23]

Figure 2: Examples of the operational impact of in-service date delay
The Type 45 FrigateThe anti-air missile system deployed on the Type 42 destroyers which the Type 45 will replace—Sea Dart—was not designed as an anti-missile missile. The modifications which have been undertaken throughout its life are designed to enhance its capability against sea skimming missiles and the Department is confident it remains a very effective long range weapon against aircraft.[24] However, the Department conceded that our ships would only be able to defend themselves if the United Kingdom were involved in an operational scenario similar to the Falklands conflict because Sea Dart forms part of a mixed fleet and will be in service with Sea Wolf, which has been designed specifically as an anti-missile missile.[25]
The Air-launched Anti-Armour Weapon (Brimstone) The weapon Brimstone is to replace, the BL755 cannot defeat modern armour and only half of one percent that went to the Gulf was actually used. The Department said that this version of the BL755 had to be dropped from a height of around 500 feet, limiting its use in that conflict. The weapon has since been updated so that it can be dropped from a much greater height.[26] The Department accepted that BL755 was not as effective against modern armour as it was against other targets but said that it was extremely effective against soft skinned targets.[27]
BOWMANAnalysis conducted by the Centre for Defence Analysis predicted that deploying digitised battlefield command and control systems may reduce the time taken to seize an objective by up to 75 percent. BOWMAN will not enter service until 2003 at the earliest and, until this time, forces conducting and supporting land operations cannot reap the significant benefits expected from digitisation.[28]
Medium Range TRIGATThe requirement for a medium-range anti-tank weapon has not been cancelled but following the United Kingdom's withdrawal from Medium Range TRIGAT the Army is conducting an Anti-Armour Balance of Investment study. Until this concludes and a solution is procured the Army must rely on MILAN which first entered service in 1979 and represents an increasingly limited threat against modern armour.[29]

15. The Sea Dart anti-air warfare system fitted on the Type 42 destroyers is undergoing a programme of modifications so that it can deal more effectively with modern threats such as sea-skimming and high-diving missiles. The upgrade to equip Sea Dart with infrared fuses is currently running eight years late.[30] Asked what had gone wrong and why the delay was allowed to happen, the Department said that a number of test firings had shown false triggers of the fuse. It had been extremely difficult to establish that these false triggers were caused, among other things, by the effect of the sun on the upward looking component of the infrared fuse. The Department was, however, confident that it would soon prove software arrangements to discriminate successfully against the sun if it suddenly emerged from behind a cloud at the critical moment in the engagement.[31]

16. The Sea Dart missile system contains old-fashioned obsolete components and will be harder to maintain as the missiles become older. The Department re-assured us that it was taking enormous efforts to defeat obsolescence on the missile and assured us that, although it was expensive to make small production runs of the components, there were no technological barriers to doing so.[32]

17. Our predecessors asked the Department how the £800 million of write-offs and additional expenditure incurred on the Medium Range TRIGAT, BOWMAN, Brimstone and Type 45 destroyer programmes would affect the provision of defence capability more widely and sought assurance that they would not result in vital funding being cut elsewhere in the budget.[33] The Department's responses are summarised in Figure 3.


Figure 3: The effects of the write-offs and additional costs incurred on four projects
ProjectWrite-off / Additional costs Comment
The Type 45 Frigate£565 million additional support costs The additional support costs were for running on the ageing Type 42 frigates.[34] The Department suggested that the additional costs had not yet been incurred and had been accompanied by a deferral of the Type 45 procurement expenditure. Thus it argued that over the period of slippage the Department would not have to spend more than it planned and so there was no impact on the defence budget during the period.[35]
BOWMANWrite-off of between £35 million and £102 million The Department regretted that the money spent on procuring BOWMAN had been wasted and said that the consequences in terms of the impact on the defence budget would depend on the size of the write-off, and on which contractor won the new competition.[36]
Medium Range TRIGAT£115 million write-off The Department suggested that, if the programme had not been conceived, there would have been an alternative anti-tank guided weapon system, although the Chief of Defence Procurement did not believe that the effect on the budget would have been so substantial. The problem was rather that the front line of the armed forces was today relying on MILAN when they had a reasonable expectation of having a modern anti-tank guided weapon system.[37]


Conclusions

18. Seventeen of the 20 projects in the Major Projects Report 2000 have slipped against the in-service dates set at the approval equivalent to Main Gate and seven have slipped by a total of 63 months during the last year. This is an unsatisfactory situation which has negative cost and performance implications. As new projects approved under Smart Acquisition principles begin to enter the Major Projects Report we expect to see the rate of slippage reduce.

19. The Department appears on occasion to excuse delays by arguing that they may allow improvements to be included that were not available at the planned in service date. For example, the Department argued that Brimstone's image recognition capability would not have been available if it had entered service on time some 10 years earlier, and that the missile would now have needed upgrading; but it admitted that this was an inadvertent consequence of the delay rather than a planned strategic decision. The Department needs to be clear about the time it is prepared to wait for new equipment in order to secure the latest improvements.

20. The Department suggested that it acted quickly when it decided to withdraw from programmes, for example replacing the Common New Generation Frigate programme with the Type 45 Destroyer, and deciding to re-open the BOWMAN competition. In both of these cases the Department moved fast once the decision was made, but the time taken to reach the decision itself was too long given the track record of problems on both programmes. The Armed Services will not receive, when originally planned, the capabilities which these equipments would have provided. The Department should make decisions on when to withdraw from programmes in a more timely manner in future to prevent such capability gaps occurring.

21. The adverse operational impacts of the delays in entry into service of all four case study equipments is unacceptable. Without the Type 45 Destroyer the capability of the Navy will be less between 2002 and 2007. In the meantime the Sea Dart missile, supplemented by other systems such as SeaWolf and Phalanx, will have only limited effectiveness against modern sea-skimming missiles. Troops still do not have a secure communications system because of the delays on BOWMAN, the MILAN missile has limited ability to defeat modern armour and the RBL755 cluster bomb, which Brimstone will replace, is only effective against soft-skinned targets.

22. Over a number of years our predecessors have highlighted numerous problems caused by equipments being unable to operate effectively in the required environment. To add to this list, the Department has admitted that the upgrade of the Sea Dart system has been delayed because the software could not discriminate targets against the sun. The Department should develop processes to enable it to better anticipate such environmental difficulties by drawing on its own past experience and the expertise available to it in DERA and industry.

23. The Sea Dart system was designed in the 1960's and contains components which are obsolete and which the Department is having to go to a great deal of effort to replace. Obsolescence is another example of the Department having to spend more money because replacement equipment is not available on time. The Department should ensure that the costs and resource effort required to manage obsolescence are properly managed and minimised.

24. It is unacceptable that the Department has wasted between £35 million and £102 million in pursuing an unworkable solution to the BOWMAN requirement. The precise scale of the write-off is unclear but, even at the lower end, it would probably cover the cost of the sonar which the Department now intends to fit on the Type 45 Destroyer.[38] We expect the Department to minimise the amount written-off and to be able to demonstrate that it has utilised the results of the abortive BOWMAN expenditure to good effect.

25. The Department has written off £115 million of Medium Range TRIGAT development costs. It argued that, if this money had not been spent on Medium Range TRIGAT, it would have been spent on an alternative solution. The fact remains that the Department failed to procure an effective weapons system, so the money has, in effect, been wasted.

SMART ACQUISITION: EFFORTS TO IMPROVE PROCUREMENT PERFORMANCE

26. Smart Acquisition requires projects to produce three-point cost and timescale estimates at 30 percent, 50 percent and 90 percent confidence levels. In giving evidence to our predecessors on the Major Projects Report 1994, the Department stated that three-point estimating was used routinely on all projects and undertaken thoroughly. Yet the National Audit Office found that only two of the ten assessment phase projects had reliable three-point cost and time estimates. Asked why this was so, the Department could not justify its record and agreed that it was not satisfactory that it had failed to fulfil its assurance to the Committee. But it did point out that since 31st March 2000 (the datum of the National Audit Office report) there were only three projects without three-point estimates. As part of the changed procurement strategy the Department had asked bidders for enough data to support three point estimates and promised the Committee would see these as soon as they were available.[39]

27. In response to questioning on whether there was a major project that it was proud of, the Department listed Trident, Tomahawk and the recovery of the Challenger 2 programme. When it was suggested that Trident and Tomahawk were largely off-the-shelf projects where the Department did not have a great deal of scope to alter the specification the Chief of Defence Procurement responded that the nuclear propulsion plant did not feel off-the-shelf to him.[40]

28. The contract for the Type 45 Destroyer is arbitrated under British law, whilst the contract for its PAAMS missile system has been placed under French law. The Department was asked what would happen if the two systems did not work together and it was not clear which half was the problem. The Department admitted that it carried the risk, but was confident that PAAMS was required to conform to all of the interface specifications which had been agreed with the ship project.[41]

29. In giving evidence on the Major Projects Report 1998 the Chief of Defence Procurement stated that there were "the best industry people in the world" working on BOWMAN and that "they were close to a cost-effective solution". Since the competition has subsequently been re-opened, some 12 years after the requirement was first raised the Department was asked why it had not seen the difficulties with the ARCHER industrial collaboration earlier and why it had taken so long to decide what to do. The Department said that ARCHER was completely re-energised in 1999 as a result of the consortium being managed properly by BAE SYSTEMS. They had brought in new staff and were supported by their consortium members both from the United States and from France, with the best people that could be found. The Department had, however, been very keen to make sure that a decision on whether or not to continue with ARCHER was based on their final bid and, right until the last few days of the bid, it was not clear whether it was going to be satisfactory or not. The Department said that it was very careful about contractors' extravagant promises, particularly when they were made near the deadline on the basis that it had become a must-win project for them. The fact that ARCHER put so many changes into their bid in the last few days had caused the Department to look closely at whether they would be able to live up to their promises.[42]

30. The Department was asked why technical delays had emerged on the Hercules C-130J programme, since at the time the contract was awarded there were categorical assurances from Lockheed-Martin and also the RAF that there would be no delays. The Department said that the Hercules C-130J was a new aircraft with new engines, propellers and avionics and that Lockheed Martin had underestimated the development content. The Department did not understand why the company made an under-estimate because they built its predecessor, the C-130K, and knew more about this aircraft than anybody else in the world.[43] The Department also said it recovered substantial liquidated damages from Lockheed-Martin as they fell due, which had at least compensated it for the running costs of the existing United Kingdom C-130K's.[44]

31. The Chief of Defence Procurement believed that the Department needed to find better ways of incentivising contractors to deliver on time. The Department had plenty of 'sticks' but needed to balance them with 'carrots'. For example, the Eurofighter programme included some incentives for doing things on time by using the retentions on the progress payments on the contract, which were planned to be withheld to the end. Now, if industry delivered the first, eighth and final Tranche 1 aircraft on time they would receive bonuses.[45]

Conclusions

32. Our predecessor's report on the 1994 Major Projects Report recorded that the Department had stated that it used three-point estimates on all major projects. It is therefore surprising that only two of the ten assessment phase projects in the Major Projects Report 2000 had three-point estimates for time and cost at 31 March 2000. The Department could not justify this failure, although it did make progress quickly in generating three-point estimates after the National Audit Office had highlighted the problem. We also note the Department's assurance that three-point estimates for Bowman will be available soon and expect all projects in the Major Projects Report 2001 to have robustly generated three-point estimates.

33. The Department highlighted the Tomahawk Land Attack Missile and Trident missiles as successful projects. One obvious common factor is that both of these projects were largely off-the-shelf purchases from the United States, but the Department should look to identify other common features which may enable it to emulate these successes in future.

34. The Type 45 destroyer is an anti-air warship being procured in two parts comprising the warship and the Principal Anti-Air Missile System (PAAMS). The Department recognised the risks but assured us that it was confident there would be no interface problems between the warship and PAAMS. That would be a significant achievement, and we look forward to hearing whether the Department has achieved it.

35. The Department said that it was now more cautious about accepting over optimistic bids from industry. For example, it had not continued with the ARCHER industrial prime collaboration contractor on BOWMAN because it believed that the company was making extravagant promises that would not be deliverable. Such scepticism is a welcome development.

36. The Department seeks to transfer the cost risk of resolving technical difficulties and the delays it causes to industry by using performance based contracts. For example, the Department received substantial liquidated damages from Lockheed Martin in respect of the delays to the Hercules C-130J. Whilst liquidated damages may offset the costs of delays the Department still bears the operational implications of the delays, a fact which re-emphasises the importance of planning to and agreeing realistic timescales at the outset.

37. The Department regards control of costs through contracts as straightforward, but believes that incentivising contractors to deliver on time is more difficult. The principle of incentivising contractors to deliver on time is a good one, but it is important that incentives are based on a thorough understanding of what is achievable and are effective in securing added performance. The Department needs to ensure that it does not give contractors extra benefits for simply doing what they should be doing anyway under the contract.

SMART ACQUISITION: LITTLE PROGRESS IN MEASURING SUCCESS

38. The Department has not yet set targets for its performance measures for the assessment phase and does not have a measure to show whether risk is reduced sufficiently to enable a sound decision to be taken at Main Gate. Asked why it had taken so long to set these performance measures and when they would be fully in place, the Department said that setting the measures was difficult, and that the measures agreed would have to be used for all assessment phase projects, not just those in the Major Projects Report. However, the Department promised it would have the measures in place by 1st April 2001.[46]

39. Following the changes to the approval process associated with Smart Acquisition approvals the Department approves the cost and in-service date of an equipment which it is 90 percent confident of achieving, while basing its plans on a lower/earlier figure which it is 50 per cent confident of achieving. Timescale and cost performance is measured in the Major Projects Report by comparing the current forecast in-service date or cost underpinning the Department's plans (the 50 percent estimate) with the latest acceptable in-service date or highest cost set at approval (the 90 percent estimate). This means that individual projects approved under 'Smart' procedures will be forecast to enter service earlier or cost less than their approved in-service date or procurement cost unless all of the risks allowed for in the 90 per cent approval materialise.[47]

40. The differences between the 50 percent and 90 percent figures can be very large, for example, the Multi-Role Armoured Vehicle is reporting a 31-month negative in-service date variation. In this case, the Department said it was unfortunate that one of the first projects to appear under the new regime was a two-nation collaborative programme which might soon become a three nation programme if the Netherlands joined. This uncertainty had increased risk, and that there were a number of other international factors which would bedevil the programme through development. If these uncertainties were added up, the 31 months did not look like a generous allowance of time.[48]

41. Asked how large, on average, it expected the risk provisions for time and cost to be at Main Gate, the Department suggested that the absolute differences would be variable between projects. The fact that one project approved under "Smart" procedures, the Seawolf Midlife Update, had already breached its 90 percent in-service date approval by 3 months showed it was not being institutionally generous to itself.[49]

42. All of the projects covered in the Report were conceived before Smart Procurement, and the Department suggested that it was still early to expect to see improvements in timescale performance from the application of Smart Procurement principles reflected in the Major Projects Report. Only when projects with significant delays such as the C-130J begin to pass out of the Report, and new 'Smart' projects entered the Report, would the average delay decrease. The Department was confident that this effect would start to become apparent from the Major Projects Report 2002.[50]

Conclusions

43. The Department is not clear what level of risk it expects will remain in projects at Main Gate and how this will be measured. A much clearer definition of the acceptable level of risk at Main Gate is needed and the Department must give priority to defining this and how to measure it.

44. Under Smart Acquisition, the Department bases its Main Gate Approval on the cost and in-service date of an equipment which it is 90 percent confident of achieving. We note that the SeaWolf Mid-Life Update project has already exceeded the in-service date which the Department was 90 percent certain of achieving by some three months. This does not bode well for the future.

45. The Department does not expect there to be big improvements in the timescale performance reported in the Major Projects Report until 2002, some four years after Smart Acquisition was first introduced. Whilst the timescale performance reported in the Major Projects Report will continue to reflect pre-Smart Acquisition projects for some years, we will expect to see signs of improvement as new "smart" projects come through.


1  This report is based on two days of evidence and is complemented by another (6th Report of the Committee of Public Accounts: MOD Major Projects Report 2000-The Role of the Equipment Capability Customer (HC 369 (2001-02)) Back

2  C&AG's Report (HC 970 Session 1999-2000), Executive Summary para. 1 Back

3  ibid, Executive Summary para. 2 Back

4  At this session the Committee also drew on evidence on the four case studies from the C&AG's Report on the Major Projects Report 1999 (HC 613 Session (1998-1999))  Back

5  6th Report from the Committee of Public Accounts: MOD Major Projects Report 2000-The Role of the Equipment Capability Customer (HC 369 (2001-02)) Back

6  C&AG's Report (HC 970 (1999-2000)) para 1.21 Back

7  Q266 Back

8  Q265 Back

9  Q267 Back

10  C&AG's Report (HC 970 (1999-2000)) para 1.4 Back

11  Q278 Back

12  C&AG's Report (HC 970 (1999-2000)) para 1.16 Back

13  Q189 Back

14  C&AG's Reports (HC 970 (1999-2000)) p31, and HC 613 (1999-2000)) Figure 20 Back

15  33rd Report from the Committee of Public Accounts (HC 247 (1999-2000)) Back

16  Qs 9-10 Back

17  Q51 Back

18  Q140 Back

19  Q8  Back

20  C&AG's Report (HC 970 (1999-2000)) para 3.10 Back

21  ibid, paras 3.11 and 3.14 Back

22  Q18 Back

23  Q269 Back

24  Qs 28-31 Back

25  Q32 Back

26  Q54 Back

27  Q56 Back

28  C&AG's Report (HC 970 (1999-2000)) para 3.31 Back

29  ibid, paras 3.18 and 3.21 Back

30  ibid, para 3.13 Back

31  Q38 Back

32  Q40 Back

33  Q2 Back

34  Q41 Back

35  Q3 Back

36  Q7 Back

37  Q5 Back

38  For the detail of the fitting of the sonar to the Type 45 Destroyer see the 6th Report of the Committee - Ministry of Defence: Major Projects Report 2000-The Role of the Equipment Capability Customer (HC 369 (2001-02)) Back

39  Qs 207-209 Back

40  Q209 Back

41  Qs 185-186  Back

42  Q8 Back

43  Q285 Back

44  Q286 Back

45  Q278 Back

46  Qs 202-203 Back

47  C&AG's Report (HC 970 (1999-2000)) para 1.13 Back

48  Q204 Back

49  Q205 Back

50  Qs 280-281 Back


 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2001
Prepared 28 November 2001