Memorandum submitted by Computer Weekly
1. We are not computer experts. Computer
Weekly has, however, investigated innumerable project failures
in the public and private sectors, in the UK and elsewhere, over
a period of more than 10 years. Seeking to help as wide an audience
of specialists as possible to learn from the mistakes of others,
we report almost monthly on the lessons learned from major projects
that have successfully met their original objectives or have disappointed.
1.1 In 1997 two reporters on Computer
Weekly published a book, based on case studies in the magazine,
which sought to disseminate the lessons learned from major project
failures (and successes). In researching the case studies for
the book, the reporters found that each project failed or succeeded
for reasons that seemed unique to that project; and rarely, if
ever, did the supplier and the customer agree on the major factors
which had caused or contributed to the initiative's outcome. However,
with the benefit of hindsight, talking to those involved in the
project, obtaining documents such as memos and steering committee
minutes, and looking at projects in the round, the reporters were
able to see that similar problems were recurring, particularly
in the UK and US public sectors. Copies of the book are available
to members of the committee. Its case studies include projects
at the CSA and the Department for Work and Pensions.
1.2 In writing this paper we have, for brevity's
sake, made a number of generalised observations but we can supply
detail and examples if asked.
The UK needs best practice to be
enshrined in law, as in the US.
We recommend also that the results
of Gateway Reviews are available to Parliament, the public and
Taking a very high level view, many
successful projects can be characterised by three things: openness,
honesty and humility, by which we mean a willingness of the organisation
to admit that is capable of making mistakes.
The CSA has not demonstrated openness
and honesty in its dealings with us. If it is open and honest
in its dealings with staff and Parliament its approach to the
media is of little importance. In our experience, however, organisations
that are open and honest when answering questions from the media
are likely to be open and honest in their dealings with staff
and in discussing serious problems as they arise on computer projects.
If a department has a "win at
all costs" approach, discourages open discussion of problems,
and categorises potentially serious problems as teething, it may
fail to ask itself critical questions at the start of a project,
and during the design, test and implementation phases. Some of
these questions are: "Is this project feasible? Should we
cancel it? Should we scale it back and deliver half of the software's
original features which will still give us 80% of the benefits?
Should we halt work or delay implementation while we extend testing
or commission an independent analysis? In our experience these
questions may sometimes not be asked in earnest until it is too
late and serious problems have become all too evident.
In our view there has been too much
of a focus at the CSA on the technology rather than the concomitant
organisational, managerial and cultural changes. This is a common
factor in projects that fail to meet expectations.
In its statements to Parliament and
the media, the department has emphasised its problems with new
technology when referring to the delays in the introduction of
new systems and the delays in transferring one million cases from
old to new rules. Indeed the CSA's staff have seen their screens
go blank at times, and at other times some calculations have not
been possible. The software has had bugs. On the other hand we
understand that the CSA has made thousands of changes of the design
of the system partly to reflect changes in the CSA's strategy.
It also appears to have underestimated the task of filling gaps
in its records so that comprehensive and verified data on clients
can be loaded into the new system. In contrast to its willingness
to blame computer systems, the CSA has not been overly forthcoming
in answering our questions on its internal administrative problems,
in particular the poor quality of its data.
We believe that the phrase "commercial
confidentiality" can, on occasions, be used by government
departments to categorise documents as confidential when they
do not need to be.
3.1 US administrations still suffer IT disasters
but they have gone much further than the UK in seeking to prevent
them, namely by enshrining important elements of best practice
3.2 The US Clinger Cohen Act was enacted
after Senator William S Cohen (later Secretary of State for Defense)
published a report in 1994 "Computer Chaos" which highlighted
some long-standing, systematic problems such as those identified
Insufficient attention to (a) the
way business processes were conducted, and (b) the opportunities
to improve these processes before investing in the IT that supported
Investments in new systems for which
agencies had not adequately planned, and
which did not work as intended and
did little to improve the performance of the department;
Implementation of ineffective information
systems resulting in waste, fraud, and abuse; and
Outdated approaches to buying IT
that did not adequately take into account the competitive and
fast pace nature of the IT industry.
3.3 On February 10, 1996, the President
signed the Information Technology Management Reform Act which,
with the Federal Acquisition Reform Act, became known as the Clinger-Cohen
Act. Together with other reform legislation, the Clinger-Cohen
Act provided the statutory foundation for correcting the above
3.4 It requires that major federal agencies
establish the position of chief information officer who has the
clear authority, responsibility and accountability for an agency's
information resources management activities. The CIO's job is
to ensure that the mandates of the Act are implemented.
3.5 In particular the CIO must ensure that
IT investments support work processes that have been redesigned
or otherwise improved, and that overly ambitious, high-risk, difficult
to manage schemes are discouraged. A download of the Clinger-Cohen
Act is available on the following website:
3.6 In the UK we have the benefit of volumes
of guidance on best practice. It is also the case that ministers
and accounting officers have some mandatory responsibilities in
respect of mission-critical projects, but in our view this is
not as effective as introducing legislation. The UK ministerial
responsibilities are not, as far as we can see, as clearly defined
as they could be; and there appears to be no clear mechanisms
for allowing independent scrutiny by Parliament of whether the
responsibilities have been met.
3.7 There are some signs that the gateway
reviews and an increased awareness in government of the risks
associated with overly ambitious projects have made departments
wary of "Big Bang" implementations and grandiose computer
3.8 For example we note that DWP is adopting
a more cautious approach to IT and wishes, for example, to rely
more on off the shelf software. However, as the case studies in
the book referred to above pointed out, large-scale tailoring
of tried and tested packages can prove as risky to implement as
custom-built software, particularly if the original software has
been written for mainly overseas clients. It appears to be more
sensible to simplify working practices to suit the software, rather
than change the software to suit working practices.
3.9 One of the biggest challenges facing
government as it seeks to modernise by exploiting new technology
is the needbeforehandto simplify working practices
that may date back decades. Departments find it much easier to
introduce new systems and then requireor hopethat
end-users will adapt to them, as with the Libra project at the
Department of Constitutional Affairs.
3.10 It can be difficult or impossible to
force departments to change working practices to suit new systems.
This is one of the characteristics that divides private and public
sector computing. A major bank or oil company will introduce a
new system and instruct its staff on the concomitant changes in
their working practices. If staff do not like the new ways in
working their choices may be limited to accepting the new systems
or leaving. The Singaporean government had notable successes with
IT to help handle exports but it was able to impose simplified
forms and working practices on end-users before it embarked on
major computerisation. UK departments cannot always force their
staff to accept new systems or change their working practices,
as the Lord Chancellor's Department found with its "Libra"
project, and the Department of Health may also discover when it
seeks to change the working practices of doctors and nurses to
exploit new systems that are due to be delivered under the National
Programme for IT in the NHS.
3.11 There is an argument for saying that
departments should simplify their paper arrangements and working
practices before introducing new systems. This will make the subsequent
implementation much easier and reduce the risks of failure. The
Immigration and Nationality Directorate, when it encountered problems
with the introduction of complex caseworking systems, found that
it could obtain many of the benefits of its proposed new systems
from introducing changes in working practices, supported by office
3.12 In our experience, much of the work
early in a project's life will tend to comply with practice; but
rarely, in our experience, does anyone who has the power to cancel
the project question earnestly at the design, test or implementation
stages whether the project has a right to continued existence:
should the project be cancelled now before more money is wasted,
irrespective of the public relations implications? Are the proposed
timescales too tight? Is the scope too ambitious? Should the project
be scaled back so that half of the software's original features
are delivered but those which deliver 80% of the benefits? Should
the supplier receive more money because the customer has underestimated
the complexity and/or the changes, and/or how much it will all
cost? These are all tough political decisions which few people
will want to make, or have the authority to make, and therefore
senior executives may tip-toe around the most difficult questions,
or acts too hastily by cancelling a sound projectfor example
the ASSIST project at the DSS (before it became the Department
for Work and Pensions). The system that supported tax credits
went live when a number of people knew it was high risk but nobody,
it appears, had the audacity to say in the midst of the TV campaign:
we should allow more time for testing and not go live when we
said we would.
3.13 If computer projects are about problem
solving, then success will depend largely on how those problems
are tackled. In our view some central departments are culturally
unsuited to dealing with serious problems. Departments will rarely
wish to concede that mistakes have been made, unless it is to
blame the supplier. In our experience, when projects fail, it
is usually because of weaknesses on both sides.
3.14 It is right that organisations should
be accountable, attend to detail and be success oriented, but
to strive for success at all costs, underestimating the risks,
and adopting an attitude in senior and middle management that
brooks no serious criticism can impede problem resolution. Indeed
it can lead to a doomed project clinging to life and wasting money
and time long after the need for a major re-evaluation was demanded
by the circumstances, the MASS project at Department for Constitutional
Affairs, for example. In an excellent briefing paper "Getting
IT Right for GovernmentA Review of Public Sector IT Projects,"
Intellect observed that a "hostility to bad news" and
a "must succeed at all costs" approach were failure
factors" in major projects.
3.15 If we had to distil the lessons from
major projects into three things that were likely to improve the
success rate of government projects, we would have to oversimplify
the complex causes of failures project and could therefore be
legitimately accused of making sweeping generalisations. Still,
at the risk of laying ourselves to these charges, we would say
that, taking a very high level view, many successful projects
can be characterised by three things: openness, honesty and humility,
by which we mean a willingness of the organisation to admit that
is capable of making mistakes. If the committee members wish to
see how cultural stagnation, particularly a hostility to bad news,
can damage an IT project we would recommend that they look at
a report by the consultancy Arthur D Little, which was published
by Parliament in July 1999. Rarely does such a document come into
the public domain. The consultancy's researchers investigated
the management of a £337 million IT project to introduce
a new air traffic control centre at the Swanwick centre in Hampshire.
The costs of the initial fixed-price system more than doubled
and the system was delayed by five years. We have read no better
or more thorough report on the cultural, managerial and organisational
factors that can cause or contribute to an IT project's failing
to meet expectations. Arthur D Little's report, which highlights
mistakes that are made in the run up to many if not most project
failures in the public sector, is available online.
4.1 In its statements to the public, Parliament
and the media, the CSA has implied that the delay in transferring
one million cases from the old rules and the old system to the
new system and to the new rules is mainly because of system-related
4.2 For example a supplementary memorandum
submitted to the committee by the Child Support Agency in August
2003 said: "The Department's Ministers have always said that
no cases would be moved on to the new scheme from the old scheme
until they were satisfied that it is working well. Accordingly
no date has been set for conversion, and clients have been told
they will transfer to the new scheme when the Government is sure
it is working well. The Agency continues to work towards achieving
successful conversion for the majority of old scheme cases as
soon as possible. The software required to facilitate bulk migration
of cases from the old computer system to the new is not expected
to be available until the end of the year. The software required
to facilitate bulk conversion of the migrated cases from the old
child support scheme to the new is not expected to be available
until next spring."
4.3 Computer Weekly understands that
the CSA does have system-related problems but also that it has
internal administrative difficulties which prevent any rapid transfer
of cases from the old to new rules. In particular, we understand
that the poor quality of the data on the old system requires a
large amount of manual effort to rectify. Experienced staff need
to be allocated to regularise and verify the "old" data
such it will be accepted by the new system. We are told that there
are millions of items of data that need to be verified, a process
that can be partly automated but which will also involve much
manual effort. The point, perhaps, is that even if the systems
were working perfectly today, and the software to handle bulk
conversions available now, the transfer of the one million cases
from old to new rules could not happen immediately.
4.4 There appears to have been a recognition
within the CSA that system and cultural factors would impede the
introduction of new systems, but initially little action was taken
to pre-empt such problems. It appears that the difficulties were
put to one side, like a large credit card debt that cannot be
paid back. The department has been unable to tell us when the
cases will be transferred from the old to the new rules, or indeed
how long it will take to make the data on one million cases fit
to transfer to the new system.
4.5 We believe it was disingenuous of the
department to imply that its problems lay mainly with the difficulties
it has experienced with its main IT supplier and the quality of
the company's systems. There is little to be gained in one side
seeking to put most of the blame on the other.
4.6 In essence, computer projects are about
problem solving. How you solve those problems, and the openness
and honesty you demonstrate, may determine your success in solving
those problems. If serious problems are dismissed as teething
because nobody wants to countenance the possibility that the project
faces serious problems, those problems may not be tackled with
the rigor and objectivity that is required to solve them. For
example, the Passport Service saw evidence of a slowing down in
the issuing of passports when it introduced more secure systems,
but it appears that nobody was willing to recognise the serious
impact this was having on the business. As a result, the Passport
Service continued with the roll out until the delays in issuing
passports became so severe that some holiday-makers were unable
to get new or replacement passports before their travel dates.
It is our experience that some departments are likely to regard
anyone within the department who raises serious criticisms as
a Luddite, or as a trouble-maker who is out of step with the rest
of the staff because he or she is unenthusiastic about the project.
They less likely to perceive a knowledgeable critic as a concerned
realistic who wants to ensure that the project does not fail.
4.7 In our experience, the CSA may at times
lack openness and honesty, at least in its dealings with us. We
can provide evidence of our emailed correspondence with the CSA
on this subject if requested. That is not to say that the department
is dishonest. Indeed we have no evidence of dishonesty. In answering
our questions, however, the department has omitted to give us
direct questions to direct questions, has refused to answer questions
on topics that are in the public domain, citing commercial confidentiality,
and in our view has at times obfuscated. If the department is
open and honest in its dealings with staff and Parliament it may
be of little importance if it lacks openness when dealing with
only the media. In our experience, however, organisations that
are open and honest when answering questions from the media are
likely to be open and honest in their dealings with staff and
in discussing problems as they arise on computer projects.
4.8 One of the problems experienced by the
CSA when it introduced CSR2 was an underestimation of the impact
of the change in working practices. The new system is workflow-based:
so designed that users, in seeking to settle a claim, cannot progress
from one step to another without completing each in a logical
and disciplined sequence. Under the old system this was not always
necessary. Calculations could sometimes be made even if all the
information had not been verified as correct at that time. The
new system imposes rigours that the old one does not. It is not
so easy for calculations to be made and payments dispatched while
cases are "set aside" for want of verified data. One
advantage of the new system, as we understand it, is that if it
contains all the necessary correct informationand significant
bugs in the software are removed or fully understoodcalculations
can be made swiftly, within a strong administrative and auditing
framework, and more accurately than has been the case in the past.
The drawbackif it is a drawbackis that staff do
not always have as much flexibility as in the past to settle cases
and make calculations without all the necessary information being
available. An internal data "cleansing" exercise is
therefore critical to the success of the new system.
5.1 Although there is a greater awareness
of the risks of failure when new IT systems and fundamental changes
in working practices are introduced, it has not stopped some departments
from embarking on high risk projects.
5.2 The Department of Health, under its
National Programme for IT in the NHS, has signed contracts worth
more than £3 billion for a series of projects that have been
categorised in internal assessments as high risk. The potential
benefits for patients and doctors are enormous; the project is
run by committed and highly professional specialists; the objectives
of the various initiatives have widespread support, and the timetable
avoids a "Big Bang" implementation. The specific implementation
plans, however, do not as yet have the widespread buy-in of doctors
and nurses, the timescales for phasing in new systems are extremely
tight, the possible impact on hospitals and doctors of the transition
from existing to new systems and ways of working have yet to be
fully assessed or published, and the full business processes to
support the new systems have yet to be fully designed and costed.
More importantly still, redesigned processes have yet to be agreed
by doctors and nurses; yet the first systems are due to go live
this summer. Although the risks seem obvious, there appears to
be a "win at all costs" approach. We would welcome a
NASA-like mindset of planning at every stage for a possible catastrophic
5.3 It is often said that the private sector
suffers as many disasters as the public sector. This is probably
true; but in our experience the private sector has a greater incentive
to learn from failure. In the private sector people may lose their
jobs when a project is cancelled. For example project executives
at Fujitsu were replaced when problems arose on the "Libra"
project, according to a report by the National Audit Office. Nobody
suffered any adverse consequences within the department, according
to the same report. Libra represented the department's fourth
attempt to introduce a single caseworking system in magistrates
5.4 As we mentioned earlier, there are volumes
of books on best practice, but in our view it is lack of adherence
to best practice that is a common factor in failed projects. One
of the reasons for appointing a Single Responsible Owner is that
the same civil servant should be responsible for a project from
start to finish. However the SRO for the National Programme for
IT in the NHS is due to retire shortly, before the benefits of
the systems can be appraised; and the minister who was originally
responsible for the project resigned over the Gulf War.
5.5 We believe that Parliament should consider
commissioning independent audit reports when it believes that
projects are failing. We urged the Transport Committee to conduct
an investigation into the feasibility of a new air traffic control
system in Hampshire (as mentioned earlier). National Air Traffic
Services was strongly opposed to the study, but the government
acceded to the committee's request. The reports of the auditors
contained many recommendations, some of which required fundamental
changes to the structure of the management and the organisation.
The Transport Committee took the decision to publish the reports
of the auditors; and the recommendations were acted on.
5.6 Gateway Reviews are invaluable but departments
do not have to act on them and the reports are not published.
5.7 Weaknesses in the process of accountability
over project failures were summarised by Richard Bacon MP, a member
of the Public Accounts Committee, in a speech he gave to an IT
conference. "Judging by what I have seen in terms of witnesses
appearing before the Public Accounts Committee, one might be forgiven
for thinking that the main skill required to get to the top rank
of the civil service is an ability to explain fluently why something
which looks for all the world like a total Horlicks is in fact
nothing of the kind but the best which could reasonably have been
hoped for in the exceptionally difficult set of circumstances
with which the Department was confronted".
5.8 Citing a project at the National Probation
Service which had seven programme directors in seven years Bacon
said "there is something about the way the civil service
moves people around, which is inherent in its culture, that is
fundamentally at odds with successful project management".
As mentioned earlier, the Cabinet Office now urges departments
to have a single responsible owner on major projects but its guidance
cannot be enforced.
6.1 Better guidelines and advice will not
prevent IT disasters. What are needed are tough decisions, direct
language, and the ability to listen to warnings that are "off
6.2 Departments will sometimes not take
tough decisions when serious problems emerge because they fear
bad publicity. This provides the perfect breeding ground for IT
6.3 Openness is the first casualty of a
project that is in serious trouble. Sometimes, it seems, ministers
are not always aware of the seriousness of IT problems within
their departments. But if ministers cannot always discover the
breadth and depth of problems on major projects within their departments,
6.4 Serious problems are sometimes categorised
as teething troubles. Yet computer projects are largely about
solving problems, sometimes serious ones. If problems cannot be
faced up to unless circumstances force recognition of them, options
for resolution may be limited to disaster avoidance.
6.5 Do not expect suppliers to always tell
the whole truth. Those suppliers that do suspect they will not
win the contract. In the UK civil service, and particularly among
IT suppliers, criticism is associated wrongly with disaffection.
Until optimism is checked by realism and scepticism, and constructive
criticism is encouraged, we should expect project failures to
6.6 Split projects into phases that can
be delivered in a maximum of six months, with each phase able
to work on a stand-alone basis. So if integration fails there
is still much value in the project. To insist that everything
is fully integrated is to court disaster.
6.7 As suggested above, accountability or
rather the lack of it is the major challenge facing the UK public
sector. Whereas a fear of failure often drives success in the
private sector, there is not such a fear of failure in the public
sector. When heads do roll, it is usually the wrong ones: those
who have been critical of the project.
6.8 End-users must buy into the project.
If a system is imposed on end-users the risk of failure is greatly
increased. Departments sometimes think they have buy-in of end-users
whereas they may have the support of groups of end-users who are
so familiar with the project that they have emotional equity in
its success and cease to be objective.
7.1 There will always be a genuine need
for commercial confidentiality, but we believe that the protection
from disclosure afforded by the phrase is over-used, and not always
justifiable. For example the National Audit Office has reported
in considerable detail on 12 February 2004 about how much the
computer company Capita had been paid for work at the Criminal
Records Bureau, where it is the main IT supplier. The report listed
the cost to Capita of a number of service credits, and the extra
sums paid to the company for additional work. Yet when we asked
the CSA about payments to its main supplier, EDS, and about other
matters that were already in the public domain, it declined to
answer citing the need to maintain commercial confidentiality.
When, in 2003, Computer Weekly published
details of internal documents issued by the Department of Health,
which were marked "restricted commercial" and which
listed the output-based specifications for new national systems
to be commissioned by the department, its officials issued a denial
that the documents were confidential and, for the first time,
published them on a official government website, still marked
"restricted commercial" on each page. Prior to this,
various journalists had requested that the documents be made public,
without success. This shows, we believe, that the phrase "commercial
confidentiality" can on occasions be used by government departments
to categorise documents as confidential when they do not need
to be. Indeed we believe that the use of the phrase "commercially
confidential" is sometimes used by departments like a tap:
turned on or off at will.
8.1 We can see good reasons why departments
and gateway reviewers would not wish the results of their assessments
to be documents that are available to parliament, the public and
the media. We believe, however, the advantages in publishing them
outweigh the desire in officialdom not to publish them. As referred
to earlier, the report by consultancy Arthur D Little report was
published at the request of the Transport Committee. Written during
a major project's pre-implementation stages, the report disclosed
important organisational, managerial and cultural weaknesses and
recommended some fundamental reforms at National Air Traffic Services.
The recommendations in the report were acted on and it was possible
for outsiders, including the media and staff, to ask the organisation
what actions it had taken. This was public sector accountability
at its best.
8.2 Initially, however, National Air Traffic
Services had strongly opposed the independent audit although it
was commissioned anyway at the request of the Transport Committee.
One reason that gateway reviews are not published is that some
officials believe it would compromise the advice given if reviewers
thought their concerns would be made public. This is shown to
be nonsense by the Arthur D Little report which was full and frank
in its disclosures. We have no reason to believe that if Arthur
D Little were asked again to conduct a thorough analysis of the
management of a major IT project, it would act with the same professionalism
and courage as it did when assessing the New En Route project
at National Air Traffic Services.
8.3 If gateway reviewers believe the quality
and rigour of their advice and work would suffer if their reviews
were published, we would question whether they are too culturally
close to those they are reviewing and therefore perhaps not be
sufficiently independent and objective to reach the tough conclusions
that gateway reviews sometimes demand.
8.4 Gateway reviews have done much good
in making departments more aware of the risks of complex software-led
projects. There have been many specific achievements. For example
a Gateway 4 Review in July 2001 on a greenfield IT project at
the Criminal Records Bureau noted that there was no solid plan
for a model office and pilot before the system went live. The
bureau agreed that the time allowed in the contract for System
Acceptance Tests was too short. It therefore decided to double
the time and to add model office and pilot tests.
8.5 On the other hand, when there was a
subsequent Gateway 4A review on the same Bureau project in February
2002, a series of potentially serious problems were identified.
This time, according to a report of the National Audit Office
which was published on 12 February 2004, the gateway review accepted
that "there was `now no turning back'", and recognised
that, "on balance, the March 2002 operation launch would
go ahead, given the confusion and bad publicity that would result
from delay." Soon after going live with the new system, there
were a series of operational difficulties which occurred for a
complexity of reasons; at one point there was a backlog of more
than 250,000 applications for checks on people who were to work
with children or vulnerable adults. Although the situation has
now stabilised, the National Audit Office reports that customers
have been dissatisfied with the service provided by the bureau.
In our view a fear of bad publicity should not be a material consideration
in any decision to recommend whether a key government system should
to go live or not.
8.6 We have seen other weaknesses in gateway
reviews: a pre-implementation gateway review of the tax credits
gave it the green light.
8.7 It may be that the reviewers did not
have the information with which to reach any other finding. But
it is surely the job of independent reviewers, if they are endorsing
a project by giving it a green light, to be satisfied they have
all the necessary information on which to base such a key decision?
If they are not confident, perhaps they should withhold a decision
on whether to endorse a project, even if this means delaying the
implementation of a system to allow for further testing and analysis.
One assumes there will always be departmental pressure on gateway
reviewers to approve the work on a major project. At the Criminal
Records Bureau, for example, the report of the National Audit
Office acknowledges that there were considerable pressures to
go live with the system. Knowledge that the gateway reviews would
be made public and open to independent scrutiny would be a counter-balancing
force to that departmental pressure.
8.8 With Gateway reviews operating in an
environment of secrecy, and without any Parliamentary scrutiny
until the NAO publishes limited details of a review, sometimes
months or years later, we believe there may be insufficient incentive
on reviewers to always reach truly the courageous recommendations,
perhaps to halt a project indefinitely while fundamental assumptions
are challenged, that may be required to avoid project failures.
8.9 There are many concerns about the assumptions
that underpin the Department of Health's £2.3 billion National
Programme for IT in the NHS. The first national systems under
the programme are due to go live this year and there are concerns
about the extent to which the department has obtained the buy-in
of doctors and nurses, and how it will measure the success of
implementations. The Department has declined, however, to publish
any part of the gateway reviews.
8.10 We recommend that gateway reviews are
made public, or at least that there is an independent assessment
commissioned on the extent to which gateway reviews are capable
of pre-empting IT disasters without discouraging innovation or
without greatly impeding the implementation of policy initiatives.
CSA, DWP (AND OTHER
9.1 Drawing on the report of Arthur D Little,
which we mentioned earlier, we ask some questions of the DWP/CSA.
We quote from the Arthur D Little report because, in our view,
it has not been bettered as a thorough and genuinely independent
analysis of what has gone wrong on a major project; and some of
the mistakes made on the project are common to a number of UK
9.2 Arthur D Little's report said: "We
recommend . . . that National Air Traffic Services avoid `witch-hunts'
and individual blame for unwanted events where possible, and focus
instead on management process issues. . . We believe that the
whole culture and environment within which reviews were held was
a major contributor to weakness in picking up on major issues.
. . The approach adopted was non-investigative, there was an unwillingness
to face up to and discuss bad news, and a style which inhibited
an open and frank discussion of difficult problems."
Does the department believe that it has a willingness to face
up to and discuss any bad news related to its IT projects, and
a style of management that encourages open and frank discussion
of difficult problems?
9.3 Arthur D Little's report said: "A
further important observation is that the underlying reason for
the New En Route Centre delays was not `weak' project managementon
the contrary, the senior NERC personnel who ran the project were
experienced, strong project managers hired in from outside. It
could be argued that their `firmness' in isolating the project
from the influence of National Air Traffic Services' wider operations,
was itself one of the major contributors to the problems . . .
What is needed for NERC is not greater project management firmness,
but rather better strategic management, team working, common understanding
and shared expertise between the projects and operational sides."
Is the department aware of the dangers of overly firm management?
9.4 "The New En Route Centre progress
reports did not provide sufficient information for adequate oversight
by the department, especially with regard to early-warning of
delays . . . There is no section in the report (from NATS to the
department) intended to highlight concerns and key issues to be
resolved. The only narrative section is the Commentary section,
which in most of the reports only contained information on work
completed and successes achieved. . . There is no mention in the
report of major changes to either the project concept or the commercial
arrangements of the project. For example, there is no mention
in any of the reports of the effective termination of the parallel
US AAS project . . . There is also no mention of any of the major
systems contract amendments, most notably Amendment 8 which had
significant commercial consequences . . . As a further comment,
there is no mention of revenue costs versus budget in the reports.
Revenue costs are excluded because they are not the direct concern
or responsibility of the Department, and fall outside the project
capital approval mechanisms. However, omission of these costs
has some implications: without revenue costs, the Department does
not get a complete picture of the total cost of the project to
NATS. Does the department believe that ministers always have a
full picture of the total possible costs of IT projects and the
extent and seriousness of any problems?
9.5 "One of the most striking, and
perhaps in retrospect ironic, features of the New En Route Centre
Project is the very strong emphasis on meeting the tight timescales,
promulgated particularly by the then CEO and passed down to the
Project Director. Meeting completion dates is important for any
project, but in the case of the New En Route Centre this was elevated
to a level of importance that almost seemed to outweigh other
key considerations . . . The CEO in particular, as a consequence
of his high personal commitment to the success of the project,
maintained significant first-hand involvement, and had a strong
sense of personal investment in achievement of the target Operational
date. Can the department be confident that it has never elevated
the importance of meeting timescales to the point where they outweigh
over key considerations?
9.6 "Finally, the Project Director
and CEO were aware that even the hint of a possible delay to the
[project], would immediately initiate a flood of questions, requests
for clarification, and presentations that would divert their own
efforts and attention away from solving the problems at hand,
. . . The following examples illustrate the point:
The Quarterly Progress Report . .
. states that NERC will still be `on-line to meet the traffic
demands of Summer 1997', yet it was already well-known to the
NATS project team and Loral in August 1995 that there were potential
problems in the schedule, and a full bottom-up review had already
been initiated . . . There was no specific mention of this review
in the report.
It has been reported that Area Control
Services maintained an outward commitment to the Operational date
. . . even though they were sceptical that the date was achievable.
NATS Board Paper . . . includes a
confidential note to the effect that it was recognised that the
Operational date was looking `increasingly unlikely', . . . and
that announcement of further delay should be `carefully managed'.
A number of interviewees in [our]
evaluation confirmed that open debate of project problems was
in general not encouraged outside the project team.
Has the department been deterred from delaying
the go-live of a major system by the possibility that a delay
might immediately initiate a flood of questions, requests for
clarification, and presentations that would divert their own efforts
and attention away from solving the problems at hand?
9.7 "We also acknowledge that changes
have been made by the current CEO since 1997 to promulgate a more
open management style . . . Nevertheless, it was reported to us
in the evaluation that `fiefdoms' may be said still to exist within
NATS which are again indicative of cultural barriers. Are there
fiefdoms within the department that are indicative of cultural
9.8 "NATS' senior management other
than the CEO had very little stake in decisions regarding scope,
timing, and commercial issues on New En Route Centre, and were
therefore not able to influence developments significantly. Do
the department's senior management have an influential stake in
decisions regarding the scope, timing and commercial issues on
9.9 "NATS Board meetings are held monthly.
It is evident from the meeting reports that a lot of material,
often in great detail, was presented at these meetings. It is
also evident from interviews that Board members felt they received
little information enabling them to make informed decisions and
carry out their Governance duties. This situation points to a
number of causes:
There was a lack of distillation
of the key messages and their ramifications for the New En Route
Centre and for NATS business as a whole in the information as
presented. . . From documents available, the messages presented
seem very mild when compared with the reality of the true risks
for the New En Route Centre Systems Project and its impact on
NATS business. Almost without exception, any negative statement
about the NERC Systems Project is balanced by a positive one without
an objective measure being provided of the severity of the risks
or their potential impact. Does the department believe that its
high-level reports always convey enough information about the
true risks of failure on major projects, and that the impact of
any negative statements is not cushioned or nullified by a positive
message which lacks objectivity?
9.10 "These [past audits] will have
provided a good level of assurance that the specified procedures
and systems were being complied with during the NERC Systems Project,
although it does not necessarily indicate whether the procedures
and systems themselves were effective. Do the department's internal
audit reports question not only whether specified procedures and
systems are complied with during major projects but question also
whether the procedures and supervisory systems themselves are
9.11 "The National Air Traffic Services
quality audit programme has included some 104 audits on the NERC
Project to date since 1990, aimed at verifying compliance with
the NATS Management System and the Work Instructions defined for
the NERC Project. The CAA also conducts financial and management
audits . . . Project management audits and routine
project quality audits failed to identify the critical weaknesses
in the New En Route Centre Systems Project". Is the department
confident that internal audit is sufficiently proactive to identify
any critical weaknesses in major projects?
Tony Collins and Ian