6 Agile Development
76. Another problem with government IT identified
during our inquiry lies in the way in which it develops new systems.
The Institute for Government (IfG) report System Error
discussed this issue at length, contrasting how the Government
had traditionally run projects using "waterfall development"-
with their preferred approach "agile development".
Waterfall versus Agile Development
77. In the waterfall model the design goes through
a series of sequential phases with each phase being completed
before moving onto the next. The main argument advanced in favour
of this model is that time spent up front making sure requirements
and design are correct saves time and effort later. Therefore
each phase should be completed before proceeding to the next phase.
78. The IfG argues that this model would be successful
"as long as the plan will not need to change mid-way through".[109]
However, in the vast majority of cases, the complex nature of
the problems that Government is faced with means that this criterion
is rarely met. New priorities, policies and initiatives, as well
as new technological options, often lead to changes being required
both during development and after implementation.
79. The waterfall approach encourages users to try
and define all their requirements in as much detail as possible
during the initial specification stage, due to the high costs
of making changes at later stages in the process. This can therefore
result in unnecessary 'gold-plating' of requirements as the initial
specifications are seen as the only opportunity to request functionality.
80. In contrast agile development is based on an
iterative and incremental process, where requirements and solutions
continuously evolve, this is the way many businesses acquire IT.
Rather than specifying rigid system requirements and then attempting
to design the whole programme at once, the design team will produce
a partially complete design which goes through a number of iterations.
The design is also broken up into a number of parts (modules)
which can be worked on and changed independently. The design,
often in the form of a working although incomplete prototype,
is shown to the customer at the end of each iteration allowing
them to experience, comment and feedback on the product; thereby
ensuring much greater integration of the business into the design
of the systems.
81. The figure below, from the IfG Report, summarises
the main difference between the two approaches:
Figure 1: Waterfall v. Agile development approach.
Source: IfG: System Error, p32
82. The IfG recently facilitated a project using
agile development techniques that developed software to help the
DWP and the Metropolitan Police Service share data on suspected
fraudsters. Their report identifies the following benefits:
- Reducing time-consuming decision-making
processes;
- Implementing those parts of the system that
are ready, without having to wait for the complete system to be
finished;
- Encouraging SME involvement, by breaking programmes
up into smaller modules; and
- Greater flexibility by being better able to deal
with changes in system requirements.[110]
Challenges to using Agile
83. The Government has responded positively to the
IfG Report, and its ICT Strategy contains a commitment to apply
agile methods to ICT procurement and delivery.[111]
Mark Holden, HMRC, told us that there was not much in the way
of barriers to the greater use of agile development. However,
he did say that some types of projects - particularly those which
aimed to develop or enhance legacy systems - might not be suitable
environments for using this methodology.[112]
Sir Ian Magee, IfG, made a similar point saying that the Institute's
report was not claiming that Agile would be suitable for every
project.[113]
84. The IfG identified three
main barriers to using agile working methods in Government:
- Governance issues. The current
approval process looks for a detailed specification as a sign
that a project has been properly thought out, but such specifications
are not normally produced for Agile development as the specification
is expected to change as the project develops;
- Commercial processes. A preference for fixed
price contracts to deliver a particular solution reinforces the
tendency for both sides to demand a high level of detail up-front;
- Cultural issues. A reluctance to delegate and
assign high levels of responsibility at lower levels of the organisation,
in addition to a more general wariness of change from inside Government.[114]
In addition, greater use of agile development is
likely to necessitate behaviour changes within Government. As
agile methodology requires increased participation from the business
to provide feedback on different iterations of the solution, departments
will need to release their staff, particularly senior staff with
overall responsibility of the project, to allow them to participate
in these exercises.
85. Alistair Maughan, partner and head of Technology
Transactions at an international law firm, has argued that the
fact that agile projects do not have a fixed cost at the outset
combined with the hierarchical nature of civil service management
structures meant that it would not able to be successfully used
in government.[115]
86. We do not think that any of these challenges
are insurmountable. For example the theoretical ability to acquire
a "fixed price" under the current model has not prevented
projects being delivered late or additional costs being incurred.
However, it is clear that moving towards a greater use of agile
development will require changes in the way government commissions
and manages projects. For example, the "System Error"
report identified the need to develop different governance and
oversight mechanisms for agile projects; arguing that the current
Gateway Reviews are unsuitable both due to the length of the review
process and the fact that agile projects lack obvious Gateway
points at which to conduct these reviews.[116]
87. Agile development is a powerful tool to enhance
the effectiveness and improve the outcomes of Government change
programmes. We welcome the Government's enthusiasm and willingness
to experiment with this method. The Government should be careful
not to dismiss the very real barriers in the existing system that
could prevent the wider use of agile development. We therefore
invite the Government to outline in its response how it will adapt
its existing programme model to enable agile development to work
as envisaged and how new flagship programmes will utilise improved
approaches to help ensure their successful delivery.
AGILE AND PROCUREMENT
88. We also heard specific concerns about the compatibility
of agile development with current procurement guidelines. Alistair
Maughan argued that under EU procurement law the Government is
required to compare bidders "on a like-for-like basis,
and deciding on best value for money" but that "Agile
can't give you a clear specification of outputs up-front. Nor
can it give a definitive up-font price" that is required
to make Government to make a like-for like comparison.[117]
Similarly the IfG noted that:
The procurement process can undermine agile projects
in two ways. First, when detailed specifications are used it can
restrict the freedom to innovate. Second, the long timelines associated
with the procurement process can restrict the ability to deliver
production ready solutions rapidly.[118]
89. Kelvin Prescott, Newbury Management Consultants
and Susan Atkinson, Legal Director, Gallenalliance Solicitors
suggested a number of changes which they believed would help overcome
these problems including:
- A different contract structure
which breaks projects down in several phases, with parties only
committed initially to the first stages. This would allow the
customer to adapt the process as development continues.
- The creation of an agile procurement strategy
based on a "fail early, fail fast" principle
that provides the ability to terminate a project when failure
looks likely and that provides access to a number of suppliers
than can be rapidly appointed to projects, or parts of project.
- A number of specific changes to the current Procurement
Contract Regulations.[119]
90. The Government should examine how it can remove
barriers to agile development as an integrated part of its wider
efforts to reform the procurement process and increase the role
of SMEs. The Government will have to bear in mind the need to
facilitate agile development as it renegotiates the EU procurement
directive and revises the associated guidance.
109 IfG, System Error,p28 Back
110
IfG, System Error, p 47 Back
111
Cabinet Office, Government ICT Strategy, para 27 Back
112
Q 334 Back
113
Q 97 Back
114
IfG, System Error, p 53-54 Back
115
"Agile will fail Gov IT, says corporate lawyer", Computer
Weekly, April 26 2011 Back
116
IfG, System Error, p 72 Back
117
"Agile will fail GovIT, says corporate lawyer", Computer
Weekly, April 26 2011 Back
118
IfG, System Error: Fixing the Flaws in Government IT, March
2011 p 68 Back
119
Ev w46-154 Back
| 
