Sixth memorandum from the Ministry of
Defence
PERFORMANCE AGAINST KEY TARGETS 2005-06
1. For Key Target 2, actual average new time
slippage was 0.7 months which was below the target of 1 month.
Performance against this target used to be calculated on the basis
of the 20 largest projects but, from April 2004, has been calculated
on all projects over £20 million. Is the good performance
on a large number of smaller value projects compensating for poorer
performance on the largest projects?
Analysis of the data shows that category C projects
(ie those valued between £20 million and £100 million)
encounter similar levels of delay as those with higher values.
For example in 2005-06, there were 13 Cat Cs in a Key Target population
of 46. On average the in year delay to those Cat Cs was one month
per project so they scored slightly worse than the overall average.
It is relevant to recall the reasons for introducing
the changes in setting this KT in 2004. It was considered that
a larger project population gave a more representative measure
of DPA performance than just the top 20 projects by value. It
is also more appropriate because timely delivery of lower value
projects may be just as important operationally as those in the
higher cost bracket. From this time, performance has also been
measured by comparing at the 50% confidence level. Previously
comparisons had been made against the 90% confidence level which
included contingency. The new regime is a tougher and more relevant
measure of performance in delivering against the originally planned
timescales.
2. Delivering projects to time may be assisted
by setting realistic in-service dates, procuring equipment off-the-shelf,
and providing incentives for the contractor. To what extent have
each of these played a role in the DPA's improved performance
relating to time slippage?
We see each of these factors as being relevant.
We are focusing more closely on due diligence and in factoring
in more appropriate incentive for industry to deliver on time.
Off-the-shelf products, where available, reduce risk. The JAVELIN
medium range anti-tank guided weapon is good example of mitigating
some schedule risk by buying off-the-shelf. Although it has been
in-service with the United States since 1996, JAVELIN did require
an element of modification to meet UK specific requirements and
to address inter-operability issues. A consistent and robust approach
to this procurement, and close co-operation between the DPA and
DLO, the US Army programme office, the Javelin Joint Venture,
enabled it to be delivered 13 months in advance of the date approved
at Main Gate.
3. In its Major Projects Report 2005, the
NAO reported that the decreases in forecast costs in 2004-05 were
"primarily due to reductions in the numbers or capability
of the equipment". To what extent did reducing the numbers
of equipment ordered or reducing the capabilities of equipment
contribute to the DPA achieving Key Target 3 relating to cost
growth in 2005-06?
Re-examining the numbers of equipment ordered
and equipment capabilities led us to identify opportunities to
reduce cost growth by some £145 million in 2005-06.
4. What was the actual in-year cost growth
for the defence equipment projects covered by Key Target 3?
The actual cost growth was £65.674 million.
5. Key Target 4 required the DPA to deliver
greater than 90%, by value, of its forecast asset deliveries and
the DPA delivered 107%. Does the DPA's achievement represent a
catching up of previous slippages, or getting ahead of schedule
for future years? Are Urgent Operational Requirements (UORs) included
in the delivery figure?
The Agency has achieved its asset delivery targets
for the last two years, delivering 100% in FY 2004-05 and 107%
in 2005-06. The target is based on forecasts set at the beginning
of each financial year. There is therefore there is no catching
up.
UORs contribute to the over-achievement seen
in FY 2005-06 as these were not part of deliveries planned at
the outset of the year. At £99 million they represented some
3% of the total deliveries of some £3.3 billion.
UORs that emerge in year are excluded from the
baseline calculation but included in the overall delivery figure.
This is on the basis that project resources could be refocused
on delivering the UOR rather than the planned programme.
6. The DPA delivered new equipment valued
at £3.3 billion in 2005-06, but in 2004-05 the DPA delivered
new equipment valued at £8.3 billion. Why was the value of
new equipment delivered in 2005-06 so much lower compared with
2004-05?
When equipment is delivered to the front line
the value of the equipment itself is transferred and, at an appropriate
point, so is the value of the investment made in developing the
capability. There were significant development costs delivered
in 2004-05 (£4.7 billion) for Typhoon which distorted the
delivery picture. If these were excluded, the figures are more
consistent (2004-05: £3.6 billion; 2005-06: £3.3 billion).
7. In 2005-06 the DPA met, for the
first time, all its Key Targets and delivered against the MoD's
Public Service Agreement targets. What are the main factors that
have contributed to the DPA's improved performance?
The corporate performance improvement programme,
"DPA Forward" underpins the improvements we are seeing.
This has introduced new and rigorous processes right across the
Agency's project activity, to ensure that estimates of time, cost
and performance that form the basis of major decisions are sound
and take account of risk. It has included improvements to the
Agency's governance and links with industry. DPA Forward accelerated
a process of continuous improvement that has in turn been built
on with the changes proposed as part of the implementation of
the Defence Acquisition Change programme.
8. We have just passed the half way mark
in the financial year 2006-07. How is the DPA doing against its
2006-07 Key Targets midway through the year?
The Agency's Key Targets remain challenging,
as they should be to remain relevant and appropriate in demanding
year on year improvements. KT3 is under pressure because of further
potential cost growth from the long standing legacy projects ASTUTE
and T45. KT4 is at risk because of the re-timing of deliveries
compared with the schedules set at the beginning of the financial
year. Performance against those targets is being actively managed
by the DPA Executive Board to mitigate the corporate impact of
the project specific problems.
DPA FORWARD PROGRAMME/INTERNAL
CONTROL WEAKNESSES
9. Many of the changes in the "DPA Forward"
programme are due to be implemented by late 2006. Will all the
changes be implemented by the end of the year? If not, which planned
changes will not have been implemented?
In order to maintain focus on the implementation
of the Defence Acquisition Change Programme, DPA Forward
was brought to a controlled close in July 2006. Many of the changes
introduced by the programme—such as Key Supplier Management,
Performance Review & Assurance and the Corporate Management
Information System—are now embedded in the normal business
practices of the DPA and will be more widely adopted through the
Defence Acquisition Change programme; a clear endorsement of the
success for DPA Forward. Ongoing work is also being subsumed with
the Defence Acquisition Change programme; this includes work on
the Operating Framework, Effective Project Control, Performance,
Time & Cost Trade Offs, future development of Key Supplier
Management and Workforce Development and Incentivization. CDP
and CDL's Joint Working initiative begun in 2003 has laid the
foundations for combining DPA and DLO into the single DE&S
organisation from 1 April 2007 as recommended by the McKane report
on Enabling Acquisition Change.
10. In the DPA's Annual Report and Accounts
2005-06, internal control weaknesses in the application of Earned
Value Management, project cost estimating and risk management
were identified. How are these weaknesses being addressed and
in what timeframe?
(a) Earned Value Management (EVM)
From January 2006 the DPA has mandated the use
of Earned Value Management (EVM) on all projects valued at £20
million or above that are entering the Demonstration Phase. To
ensure consistency of application of the technique across the
Agency we have created two dedicated teams. One team is responsible
for policy, information and advice, training sponsorship and the
maintenance of industrial links through participation in the Defence
EVM Implementation Group and the Association of Project Management's
EVM Specific Interest Group. The other team provides "hands
on" support to IPTs in their implementation of EVM and in
reviews and audits of their contractors. We have also jointly
developed with industry a template for contract Terms and Conditions
which ensures that all implementations of EVM are in accordance
with the EVM User Guide.
(b) Risk Management
In 2003 a team was established to assist IPTs
to improve their risk management procedures. This team has developed
detailed guidance and advice, has reconfigured our approach to
risk management training, and has sponsored the introduction of
computer-based risk management tools. Benchmarking exercises consistently
identify the DPA's risk management as "Best in Class"
for the approach, although we are working to further improve our
deployment of process and procedure across the Agency. The risk
team has been able to demonstrate a significant increase in awareness
of the need for and benefits of effective risk management in the
IPTs, and has measured real improvements in IPT processes and
outcomes as a result. Risk management is a key part of the governance
we apply to projects, and is a major element of the Project Review
and Assurance process, also introduced as part of the DPA Forward
corporate performance improvement programme.
A series of "Risk Master classes"
have been initiated for IPT Leaders and other Senior Staff. These
emphasise the importance of risk management to the overall business
of the Agency, the linkages between project risk and corporate
risk, and the cultural changes that we need to ensure a seamless
and integrated approach to risk management at all levels of the
organisation.
(c) Project Cost Estimating
The DPA has improved project estimating by taking
into account trends observed in actual costs incurred for previous
procurements. This recognises that it is challenging to improve
on the historic trend, unless there is a step change in technology
or manufacturing process. The DPA is now using this historic trend
analysis to help validate project cost estimates prior to major
decision points (such as CVF). This approach will be further strengthened
by detailed, non-advocate advice and departmental initiatives
as part of Enabling Acquisition Change to improve due diligence
for major investments in the future, prior to the approval.
The DPA also uses the Treasury's Optimism Bias
(OB) model as part of the risk analysis process for cost estimating.
This enables a comparison to be made between the detailed (bottom
up) assessment of risk and the (top down) OB indication. Any discrepancies
are thoroughly reviewed and causes identified. The scrutiny of
the business case reviews the outcomes of the process. The risk
maturity assessments provide assurance that the risk management
processes are being correctly applied in the derivation of project
cost estimating.
Further key improvements in process have been
identified for this year to provide improved project cost estimates.
This includes gathering more detailed company data on competitive
and non competitive contracts to inform future estimates, track
progress, and analyse options. This will provide a more robust
basis for trend analysis and decision support. We are working
more closely with some major players in industry to assess their
End To End estimating process and to be involved in the "Estimate
Challenge" phase. This will provide improved confidence in
industry figures and an opportunity to influence manufacturing
methods.
Key Targets for the New Organisation
11. One of the aims of the DPA/DLO merger
is to improve Through Life Capability Management. How will the
expected improvements be tracked? Will there be new Key Targets
covering this? Following the merger, will the DPA Key Targets
covering equipment acquisition still remain, so that improvements
in equipment acquisition can continue to be tracked?
Current Departmental Public Service Agreement
(PSA) targets that are equivalent to current DPA Key Targets 1-3,
will remain in force throughout the new organisation's first year.
Similarly we have to deliver against Customer/Supplier Agreements
(CSAs) that are already in advanced negotiation between the DLO
and its customers, which will run to end of 2007-08. But the clear
thrust of the Defence Acquisition Change programme and Defence
Industrial Strategy towards improving the management of acquisition
on a through-life basis will require the introduction of targets
focused on the delivery of the equipment and support component
of Departmental through-life acquisition targets as well as the
effectiveness and efficiency of the business.
A single target cannot capture performance appropriately
across the range of roles detailed above and we recognise the
need for a basket of metrics to drive improvement. MoD Head Office
is leading on the definition of new targets consulting with single
Service users and the NAO.
SMALLER SIZE
PROJECTS
12. As part of its inquiry the Committee
selected four smaller size equipment projects to examine. Can
you update us on how each of these projects is currently doing:
specifically, how is each doing against their approved costs and
in-service dates? Have there been any major changes to these four
projects since our oral evidence session at the end of June?
The table below provides detail on changes since
the project information was provided to the Committee in June
2006.
| Change in
Forecast Cost (£ million)
|
Explanation
|
Change in
Forecast ISD
|
Explanation
|
| Anti Surface Warfare (ASUW)[2]
| -3.0 | This is the combined effect of a reduction in the VAT rating for Phalanx 1B, a change in the risk provision and programming and contractual adjustments
| No Change | |
| General Service Respirator (GSR)[3]
| +0.5 | Additional funding required for User Trials
| + 10 months (now Oct '07) | Additional Performance and User Trials required by the Customer
|
| Maritime Training System (MTS)[4]
| No Change | | No Change
| |
| Thermal Sighting System (TSS)[5]
| No Change | | No Change
| |
13. At the evidence session on 27 June the Committee asked why the four smaller projects we selected appeared to be performing better against approved costs and in-service dates than some of the larger defence equipment projects. Dr Watson told the Committee that "short projects do tend to be slightly more successful, but we do not have that kind of separation" [Q7]. Is the DPA planning to undertake such an analysis to identify relevant lessons for larger equipment projects?
| | | |
|
Yes. Learning From Experience is required for all projects,
regardless of their size and this is shared across the organisation
so that previous pitfalls can be avoided. It is through this medium
that lessons from all projects are shared. We are working closely
with the NAO in implementing their Gold Standard for effective
project control in order to standardise good practice in controlling
all projects. The NAO Gold Standard defines the drivers for successful
project delivery which are applicable to projects of all sizes.
In the Defence Industrial Strategy (DIS), the Department recognised
specifically the benefit of adopting more flexible and agile approaches
to procurement. Such approaches have been successful in delivering
Urgent Operational Requirements and are currently more typical
of smaller projects. DIS also recognised the importance of more
agile strategies in procurement, such as better use of off-the-shelf
solutions and commercial arrangements that provide more effective
incentives to suppliers to deliver innovative solutions and better
value for money. A particular facet of small projects is the ability
for the project stakeholders, including the front-line users to
work as a small, effective team in delivering the project. DIS
recognised the importance of better team behaviours and relationship
management as part of the core business of all our acquisition.
The new DE&S organisation is being constructed in a way that
will better enable stakeholder relationship management both within
the Department and between the Department and suppliers.
The Agency has already introduced new and rigorous processes
to ensure that estimates of time, cost and performance are fully
informed before major decisions are taken.
The Department is also developing an Acquisition Operating
Framework, based on work that emerged from the DPA Forward initiative,
which will create a consistent and clearly defined framework for
how we conduct, govern and control our projects. As part of the
pilot phase for the Acquisition Operating Framework, small project
teams will be assessed alongside larger teams and good practice
pulled through into the framework that will be applied to all
projects in future. The Programme and Project Management section
of the AOF will be foundation for project management and control
in the new Defence Equipment & Support organisation.
14. What are the main reasons why smaller size equipment
projects tend to be more successful than some of the larger equipment
projects?
There are a variety of reasons why smaller projects could
be deemed more successful than some of the larger projects, particularly
from an in year cost variation position. Smaller projects, that
is, the Category C projects between £20 million-£100
million may involve less risk, be less technically demanding,
be of shorter duration, have less integration issues, are unlikely
to involve collaboration and have fewer stakeholders. This is
not to say that some smaller value projects do not involve some
or all of these. By contrast the level of technical difficulty
is often greater in the larger projects because there is a much
greater element of development to meet more demanding performance
criteria. The Department has recognised that the degree of complexity
in a project is an important factor in assessing how it should
be managed. We are working closely with the Australian DMO in
defining project complexity and pulling together a skills competence
framework that will improve the Department's ability to manage
large, complex projects. Larger projects, for example warships,
also tend to involve higher degrees of integration and dependencies
with other programmes which also adds to their risk and complexity.
FUTURE CARRIER
15. Defense News of 2 October 2006 reported that "French
industry has slashed the chances of building a second aircraft
carrier, reducing the prospective value of French shipbuilder
DCN and the amount Thales will pay for its 25% stake". There
have been reports that there is uncertainty as to whether the
French will proceed with building a second aircraft carrier. Are
these reports of concern and, if true, are they likely to impact
on the UK's carrier programme?
It is for France to comment on their programme. The reports
have no direct impact on the UK programme. Co-operation was only
agreed on the basis that it must deliver savings without delaying
the UK CVF (or French PA2) programmes, so the UK programme is
not directly affected by any changes to the PA2 programme. We
continue to co-operate during the demonstration phase to produce
a Common Baseline Design that will meet requirements for CVF and
PA2 and to explore areas of commonality which will bring savings
in design costs, procurement, in-service support and through life
technology insertion.
URGENT OPERATIONAL
REQUIREMENTS
16. What impact will the recent UORs for Armoured Fighting
Vehicles (AFV) have on the FRES programme? In order to be consistent
with the Defence Industrial Strategy should the AFV fleet be addressed
as a whole rather than for FRES to be treated in isolation?
Operational experience, especially in Iraq, has illustrated
the rapidly developing threat to our troops on the ground; particularly
the threat posed by mines and improvised explosive devices. The
majority of UORs that have been applied to in-service armoured
vehicles have been aimed at providing improved protection against
these and other threats. This experience has reinforced the need
for adequately protected armoured vehicles, a factor that is being
taken fully into account in taking FRES forward. The way forward
on FRES is being developed within an overall AFV strategy that
is consistent with the Defence Industrial Strategy, at the same
time ensuring that the unique challenges presented by FRES are
properly addressed.
FUTURE RAPID
EFFECTS SYSTEM
(FRES)
17. When is the new acquisition strategy for FRES expected
to be finalised?
Confirmation of the optimum acquisition strategy for FRES
is one of the key objectives of the initial Assessment Phase.
Senior stakeholders are fully engaged and we expect to put proposals
to HMT and OGDs this autumn with a view to making a public announcement
before the end of the year.
18. How will the MoD ensure that the new acquisition strategy
for FRES will be stable?
The stability of the programme will be achieved by agreeing
an incremental delivery of capability with the Army sponsor; by
careful selection of the companies best suited to deliver this
challenging capability through life, and by high quality due diligence
before each investment decision.
19. How will the new acquisition strategy for FRES take
account of the Defence Industrial Strategy?
The Defence Industrial Strategy sets the context within which
the FRES acquisition strategy is being developed. We will secure
the appropriate level of operational sovereignty through life
by proactive management of IPR matters. We will secure value for
money in a strategic alliance which will benefit from a stable
programme of incremental development which will sustain an efficient
production programme and a rolling programme of technology insertions
through life.
25 October 2006
2
Defensive weapons for use against fast attack craft. Back
3
Replacement of the in service respirator to improve levels of
protection, equipment compatibility and to be more user friendly. Back
4
Shore based Warfare Operator Training capability for legacy platforms
& the T45 Destroyer. Back
5
A Thermal Sighting System for the Self Propelled High Velocity
Missile Weapon System. Back
| 
