Select Committee on Work and Pensions Written Evidence


Memorandum submitted by Computer Weekly (SC 13)

INTRODUCTION:

  1.  We are not computer experts. Computer Weekly has, however, investigated innumerable project failures in the public and private sectors, in the UK and elsewhere, over a period of more than 10 years. Seeking to help as wide an audience of specialists as possible to learn from the mistakes of others, we report almost monthly on the lessons learned from major projects that have successfully met their original objectives or have disappointed.

  1.1  In 1997 two reporters on Computer Weekly published a book, based on case studies in the magazine, which sought to disseminate the lessons learned from major project failures (and successes). In researching the case studies for the book, the reporters found that each project failed or succeeded for reasons that seemed unique to that project; and rarely, if ever, did the supplier and the customer agree on the major factors which had caused or contributed to the initiative's outcome. However, with the benefit of hindsight, talking to those involved in the project, obtaining documents such as memos and steering committee minutes, and looking at projects in the round, the reporters were able to see that similar problems were recurring, particularly in the UK and US public sectors. Copies of the book are available to members of the committee. Its case studies include projects at the CSA and the Department for Work and Pensions.

  1.2  In writing this paper we have, for brevity's sake, made a number of generalised observations but we can supply detail and examples if asked.

SUMMARY

    —  The UK needs best practice to be enshrined in law, as in the US.

    —  We recommend also that the results of Gateway Reviews are available to Parliament, the public and the media.

    —  Taking a very high level view, many successful projects can be characterised by three things: openness, honesty and humility, by which we mean a willingness of the organisation to admit that is capable of making mistakes.

    —  The CSA has not demonstrated openness and honesty in its dealings with us. If it is open and honest in its dealings with staff and Parliament its approach to the media is of little importance. In our experience, however, organisations that are open and honest when answering questions from the media are likely to be open and honest in their dealings with staff and in discussing serious problems as they arise on computer projects.

    —  If a department has a "win at all costs" approach, discourages open discussion of problems, and categorises potentially serious problems as teething, it may fail to ask itself critical questions at the start of a project, and during the design, test and implementation phases. Some of these questions are: "Is this project feasible? Should we cancel it? Should we scale it back and deliver half of the software's original features which will still give us 80% of the benefits? Should we halt work or delay implementation while we extend testing or commission an independent analysis? In our experience these questions may sometimes not be asked in earnest until it is too late and serious problems have become all too evident.

    —  In our view there has been too much of a focus at the CSA on the technology rather than the concomitant organisational, managerial and cultural changes. This is a common factor in projects that fail to meet expectations.

    —  In its statements to Parliament and the media, the department has emphasised its problems with new technology when referring to the delays in the introduction of new systems and the delays in transferring one million cases from old to new rules. Indeed the CSA's staff have seen their screens go blank at times, and at other times some calculations have not been possible. The software has had bugs. On the other hand we understand that the CSA has made thousands of changes of the design of the system partly to reflect changes in the CSA's strategy. It also appears to have underestimated the task of filling gaps in its records so that comprehensive and verified data on clients can be loaded into the new system. In contrast to its willingness to blame computer systems, the CSA has not been overly forthcoming in answering our questions on its internal administrative problems, in particular the poor quality of its data.

    —  We believe that the phrase "commercial confidentiality" can, on occasions, be used by government departments to categorise documents as confidential when they do not need to be.

BACKGROUND:

  3.1  US administrations still suffer IT disasters but they have gone much further than the UK in seeking to prevent them, namely by enshrining important elements of best practice into legislation.

  3.2  The US Clinger Cohen Act was enacted after Senator William S Cohen (later Secretary of State for Defense) published a report in 1994 "Computer Chaos" which highlighted some long-standing, systematic problems such as those identified below:

    —  Insufficient attention to (a) the way business processes were conducted, and (b) the opportunities to improve these processes before investing in the IT that supported them;

    —  Investments in new systems for which agencies had not adequately planned, and

    —  which did not work as intended and did little to improve the performance of the department;

    —  Implementation of ineffective information systems resulting in waste, fraud, and abuse; and

    —  Outdated approaches to buying IT that did not adequately take into account the competitive and fast pace nature of the IT industry.

  3.3  On February 10, 1996, the President signed the Information Technology Management Reform Act which, with the Federal Acquisition Reform Act, became known as the Clinger-Cohen Act. Together with other reform legislation, the Clinger-Cohen Act provided the statutory foundation for correcting the above deficiencies.

  3.4  It requires that major federal agencies establish the position of chief information officer who has the clear authority, responsibility and accountability for an agency's information resources management activities. The CIO's job is to ensure that the mandates of the Act are implemented.

  3.5  In particular the CIO must ensure that IT investments support work processes that have been redesigned or otherwise improved, and that overly ambitious, high-risk, difficult to manage schemes are discouraged. A download of the Clinger-Cohen Act is available on the following website:

    www.c3i.osd.mil/org/cio/doc/CCA-Book-Final.pdf

  3.6  In the UK we have the benefit of volumes of guidance on best practice. It is also the case that ministers and accounting officers have some mandatory responsibilities in respect of mission-critical projects, but in our view this is not as effective as introducing legislation. The UK ministerial responsibilities are not, as far as we can see, as clearly defined as they could be; and there appears to be no clear mechanisms for allowing independent scrutiny by Parliament of whether the responsibilities have been met.

  3.7  There are some signs that the gateway reviews and an increased awareness in government of the risks associated with overly ambitious projects have made departments wary of "Big Bang" implementations and grandiose computer schemes.

  3.8  For example we note that DWP is adopting a more cautious approach to IT and wishes, for example, to rely more on off the shelf software. However, as the case studies in the book referred to above pointed out, large-scale tailoring of tried and tested packages can prove as risky to implement as custom-built software, particularly if the original software has been written for mainly overseas clients. It appears to be more sensible to simplify working practices to suit the software, rather than change the software to suit working practices.

  3.9  One of the biggest challenges facing government as it seeks to modernise by exploiting new technology is the need—beforehand—to simplify working practices that may date back decades. Departments find it much easier to introduce new systems and then require—or hope—that end-users will adapt to them, as with the Libra project at the Department of Constitutional Affairs.

  3.10  It can be difficult or impossible to force departments to change working practices to suit new systems. This is one of the characteristics that divides private and public sector computing. A major bank or oil company will introduce a new system and instruct its staff on the concomitant changes in their working practices. If staff do not like the new ways in working their choices may be limited to accepting the new systems or leaving. The Singaporean government had notable successes with IT to help handle exports but it was able to impose simplified forms and working practices on end-users before it embarked on major computerisation. UK departments cannot always force their staff to accept new systems or change their working practices, as the Lord Chancellor's Department found with its "Libra" project, and the Department of Health may also discover when it seeks to change the working practices of doctors and nurses to exploit new systems that are due to be delivered under the National Programme for IT in the NHS.

  3.11  There is an argument for saying that departments should simplify their paper arrangements and working practices before introducing new systems. This will make the subsequent implementation much easier and reduce the risks of failure. The Immigration and Nationality Directorate, when it encountered problems with the introduction of complex caseworking systems, found that it could obtain many of the benefits of its proposed new systems from introducing changes in working practices, supported by office automation.

  3.12  In our experience, much of the work early in a project's life will tend to comply with practice; but rarely, in our experience, does anyone who has the power to cancel the project question earnestly at the design, test or implementation stages whether the project has a right to continued existence: should the project be cancelled now before more money is wasted, irrespective of the public relations implications? Are the proposed timescales too tight? Is the scope too ambitious? Should the project be scaled back so that half of the software's original features are delivered but those which deliver 80% of the benefits? Should the supplier receive more money because the customer has underestimated the complexity and/or the changes, and/or how much it will all cost? These are all tough political decisions which few people will want to make, or have the authority to make, and therefore senior executives may tip-toe around the most difficult questions, or acts too hastily by cancelling a sound project—for example the ASSIST project at the DSS (before it became the Department for Work and Pensions). The system that supported tax credits went live when a number of people knew it was high risk but nobody, it appears, had the audacity to say in the midst of the TV campaign: we should allow more time for testing and not go live when we said we would.

  3.13  If computer projects are about problem solving, then success will depend largely on how those problems are tackled. In our view some central departments are culturally unsuited to dealing with serious problems. Departments will rarely wish to concede that mistakes have been made, unless it is to blame the supplier. In our experience, when projects fail, it is usually because of weaknesses on both sides.

  3.14  It is right that organisations should be accountable, attend to detail and be success oriented, but to strive for success at all costs, underestimating the risks, and adopting an attitude in senior and middle management that brooks no serious criticism can impede problem resolution. Indeed it can lead to a doomed project clinging to life and wasting money and time long after the need for a major re-evaluation was demanded by the circumstances, the MASS project at Department for Constitutional Affairs, for example. In an excellent briefing paper "Getting IT Right for Government—A Review of Public Sector IT Projects," Intellect observed that a "hostility to bad news" and a "must succeed at all costs" approach were failure factors" in major projects.

  3.15  If we had to distil the lessons from major projects into three things that were likely to improve the success rate of government projects, we would have to oversimplify the complex causes of failures project and could therefore be legitimately accused of making sweeping generalisations. Still, at the risk of laying ourselves to these charges, we would say that, taking a very high level view, many successful projects can be characterised by three things: openness, honesty and humility, by which we mean a willingness of the organisation to admit that is capable of making mistakes. If the committee members wish to see how cultural stagnation, particularly a hostility to bad news, can damage an IT project we would recommend that they look at a report by the consultancy Arthur D Little, which was published by Parliament in July 1999. Rarely does such a document come into the public domain. The consultancy's researchers investigated the management of a £337 million IT project to introduce a new air traffic control centre at the Swanwick centre in Hampshire. The costs of the initial fixed-price system more than doubled and the system was delayed by five years. We have read no better or more thorough report on the cultural, managerial and organisational factors that can cause or contribute to an IT project's failing to meet expectations. Arthur D Little's report, which highlights mistakes that are made in the run up to many if not most project failures in the public sector, is available online.

THE CSA

  4.1  In its statements to the public, Parliament and the media, the CSA has implied that the delay in transferring one million cases from the old rules and the old system to the new system and to the new rules is mainly because of system-related issues.

  4.2  For example a supplementary memorandum submitted to the committee by the Child Support Agency in August 2003 said: "The Department's Ministers have always said that no cases would be moved on to the new scheme from the old scheme until they were satisfied that it is working well. Accordingly no date has been set for conversion, and clients have been told they will transfer to the new scheme when the Government is sure it is working well. The Agency continues to work towards achieving successful conversion for the majority of old scheme cases as soon as possible. The software required to facilitate bulk migration of cases from the old computer system to the new is not expected to be available until the end of the year. The software required to facilitate bulk conversion of the migrated cases from the old child support scheme to the new is not expected to be available until next spring."

  4.3  Computer Weekly understands that the CSA does have system-related problems but also that it has internal administrative difficulties which prevent any rapid transfer of cases from the old to new rules. In particular, we understand that the poor quality of the data on the old system requires a large amount of manual effort to rectify. Experienced staff need to be allocated to regularise and verify the "old" data such it will be accepted by the new system. We are told that there are millions of items of data that need to be verified, a process that can be partly automated but which will also involve much manual effort. The point, perhaps, is that even if the systems were working perfectly today, and the software to handle bulk conversions available now, the transfer of the one million cases from old to new rules could not happen immediately.

  4.4  There appears to have been a recognition within the CSA that system and cultural factors would impede the introduction of new systems, but initially little action was taken to pre-empt such problems. It appears that the difficulties were put to one side, like a large credit card debt that cannot be paid back. The department has been unable to tell us when the cases will be transferred from the old to the new rules, or indeed how long it will take to make the data on one million cases fit to transfer to the new system.

  4.5  We believe it was disingenuous of the department to imply that its problems lay mainly with the difficulties it has experienced with its main IT supplier and the quality of the company's systems. There is little to be gained in one side seeking to put most of the blame on the other.

  4.6  In essence, computer projects are about problem solving. How you solve those problems, and the openness and honesty you demonstrate, may determine your success in solving those problems. If serious problems are dismissed as teething because nobody wants to countenance the possibility that the project faces serious problems, those problems may not be tackled with the rigor and objectivity that is required to solve them. For example, the Passport Service saw evidence of a slowing down in the issuing of passports when it introduced more secure systems, but it appears that nobody was willing to recognise the serious impact this was having on the business. As a result, the Passport Service continued with the roll out until the delays in issuing passports became so severe that some holiday-makers were unable to get new or replacement passports before their travel dates. It is our experience that some departments are likely to regard anyone within the department who raises serious criticisms as a Luddite, or as a trouble-maker who is out of step with the rest of the staff because he or she is unenthusiastic about the project. They less likely to perceive a knowledgeable critic as a concerned realistic who wants to ensure that the project does not fail.

  4.7  In our experience, the CSA may at times lack openness and honesty, at least in its dealings with us. We can provide evidence of our emailed correspondence with the CSA on this subject if requested. That is not to say that the department is dishonest. Indeed we have no evidence of dishonesty. In answering our questions, however, the department has omitted to give us direct questions to direct questions, has refused to answer questions on topics that are in the public domain, citing commercial confidentiality, and in our view has at times obfuscated. If the department is open and honest in its dealings with staff and Parliament it may be of little importance if it lacks openness when dealing with only the media. In our experience, however, organisations that are open and honest when answering questions from the media are likely to be open and honest in their dealings with staff and in discussing problems as they arise on computer projects.

  4.8  One of the problems experienced by the CSA when it introduced CSR2 was an underestimation of the impact of the change in working practices. The new system is workflow-based: so designed that users, in seeking to settle a claim, cannot progress from one step to another without completing each in a logical and disciplined sequence. Under the old system this was not always necessary. Calculations could sometimes be made even if all the information had not been verified as correct at that time. The new system imposes rigours that the old one does not. It is not so easy for calculations to be made and payments dispatched while cases are "set aside" for want of verified data. One advantage of the new system, as we understand it, is that if it contains all the necessary correct information—and significant bugs in the software are removed or fully understood—calculations can be made swiftly, within a strong administrative and auditing framework, and more accurately than has been the case in the past. The drawback—if it is a drawback—is that staff do not always have as much flexibility as in the past to settle cases and make calculations without all the necessary information being available. An internal data "cleansing" exercise is therefore critical to the success of the new system.

RECENT LESSONS

  5.1  Although there is a greater awareness of the risks of failure when new IT systems and fundamental changes in working practices are introduced, it has not stopped some departments from embarking on high risk projects.

  5.2  The Department of Health, under its National Programme for IT in the NHS, has signed contracts worth more than £3 billion for a series of projects that have been categorised in internal assessments as high risk. The potential benefits for patients and doctors are enormous; the project is run by committed and highly professional specialists; the objectives of the various initiatives have widespread support, and the timetable avoids a "Big Bang" implementation. The specific implementation plans, however, do not as yet have the widespread buy-in of doctors and nurses, the timescales for phasing in new systems are extremely tight, the possible impact on hospitals and doctors of the transition from existing to new systems and ways of working have yet to be fully assessed or published, and the full business processes to support the new systems have yet to be fully designed and costed. More importantly still, redesigned processes have yet to be agreed by doctors and nurses; yet the first systems are due to go live this summer. Although the risks seem obvious, there appears to be a "win at all costs" approach. We would welcome a NASA-like mindset of planning at every stage for a possible catastrophic failure.

  5.3  It is often said that the private sector suffers as many disasters as the public sector. This is probably true; but in our experience the private sector has a greater incentive to learn from failure. In the private sector people may lose their jobs when a project is cancelled. For example project executives at Fujitsu were replaced when problems arose on the "Libra" project, according to a report by the National Audit Office. Nobody suffered any adverse consequences within the department, according to the same report. Libra represented the department's fourth attempt to introduce a single caseworking system in magistrates courts.

  5.4  As we mentioned earlier, there are volumes of books on best practice, but in our view it is lack of adherence to best practice that is a common factor in failed projects. One of the reasons for appointing a Single Responsible Owner is that the same civil servant should be responsible for a project from start to finish. However the SRO for the National Programme for IT in the NHS is due to retire shortly, before the benefits of the systems can be appraised; and the minister who was originally responsible for the project resigned over the Gulf War.

  5.5  We believe that Parliament should consider commissioning independent audit reports when it believes that projects are failing. We urged the Transport Committee to conduct an investigation into the feasibility of a new air traffic control system in Hampshire (as mentioned earlier). National Air Traffic Services was strongly opposed to the study, but the government acceded to the committee's request. The reports of the auditors contained many recommendations, some of which required fundamental changes to the structure of the management and the organisation. The Transport Committee took the decision to publish the reports of the auditors; and the recommendations were acted on.

  5.6  Gateway Reviews are invaluable but departments do not have to act on them and the reports are not published.

  5.7  Weaknesses in the process of accountability over project failures were summarised by Richard Bacon MP, a member of the Public Accounts Committee, in a speech he gave to an IT conference. "Judging by what I have seen in terms of witnesses appearing before the Public Accounts Committee, one might be forgiven for thinking that the main skill required to get to the top rank of the civil service is an ability to explain fluently why something which looks for all the world like a total Horlicks is in fact nothing of the kind but the best which could reasonably have been hoped for in the exceptionally difficult set of circumstances with which the Department was confronted".

  5.8  Citing a project at the National Probation Service which had seven programme directors in seven years Bacon said "there is something about the way the civil service moves people around, which is inherent in its culture, that is fundamentally at odds with successful project management". As mentioned earlier, the Cabinet Office now urges departments to have a single responsible owner on major projects but its guidance cannot be enforced.

EIGHT GENERALISATIONS:

  6.1  Better guidelines and advice will not prevent IT disasters. What are needed are tough decisions, direct language, and the ability to listen to warnings that are "off message".

  6.2  Departments will sometimes not take tough decisions when serious problems emerge because they fear bad publicity. This provides the perfect breeding ground for IT disasters.

  6.3  Openness is the first casualty of a project that is in serious trouble. Sometimes, it seems, ministers are not always aware of the seriousness of IT problems within their departments. But if ministers cannot always discover the breadth and depth of problems on major projects within their departments, who can?

  6.4  Serious problems are sometimes categorised as teething troubles. Yet computer projects are largely about solving problems, sometimes serious ones. If problems cannot be faced up to unless circumstances force recognition of them, options for resolution may be limited to disaster avoidance.

  6.5  Do not expect suppliers to always tell the whole truth. Those suppliers that do suspect they will not win the contract. In the UK civil service, and particularly among IT suppliers, criticism is associated wrongly with disaffection. Until optimism is checked by realism and scepticism, and constructive criticism is encouraged, we should expect project failures to continue.

  6.6  Split projects into phases that can be delivered in a maximum of six months, with each phase able to work on a stand-alone basis. So if integration fails there is still much value in the project. To insist that everything is fully integrated is to court disaster.

  6.7  As suggested above, accountability or rather the lack of it is the major challenge facing the UK public sector. Whereas a fear of failure often drives success in the private sector, there is not such a fear of failure in the public sector. When heads do roll, it is usually the wrong ones: those who have been critical of the project.

  6.8  End-users must buy into the project. If a system is imposed on end-users the risk of failure is greatly increased. Departments sometimes think they have buy-in of end-users whereas they may have the support of groups of end-users who are so familiar with the project that they have emotional equity in its success and cease to be objective.

COMMERCIAL CONFIDENTIALITY

  7.1  There will always be a genuine need for commercial confidentiality, but we believe that the protection from disclosure afforded by the phrase is over-used, and not always justifiable. For example the National Audit Office has reported in considerable detail on 12 February 2004 about how much the computer company Capita had been paid for work at the Criminal Records Bureau, where it is the main IT supplier. The report listed the cost to Capita of a number of service credits, and the extra sums paid to the company for additional work. Yet when we asked the CSA about payments to its main supplier, EDS, and about other matters that were already in the public domain, it declined to answer citing the need to maintain commercial confidentiality.

  When, in 2003, Computer Weekly published details of internal documents issued by the Department of Health, which were marked "restricted commercial" and which listed the output-based specifications for new national systems to be commissioned by the department, its officials issued a denial that the documents were confidential and, for the first time, published them on a official government website, still marked "restricted commercial" on each page. Prior to this, various journalists had requested that the documents be made public, without success. This shows, we believe, that the phrase "commercial confidentiality" can on occasions be used by government departments to categorise documents as confidential when they do not need to be. Indeed we believe that the use of the phrase "commercially confidential" is sometimes used by departments like a tap: turned on or off at will.

GATEWAY REVIEWS

  8.1  We can see good reasons why departments and gateway reviewers would not wish the results of their assessments to be documents that are available to parliament, the public and the media. We believe, however, the advantages in publishing them outweigh the desire in officialdom not to publish them. As referred to earlier, the report by consultancy Arthur D Little report was published at the request of the Transport Committee. Written during a major project's pre-implementation stages, the report disclosed important organisational, managerial and cultural weaknesses and recommended some fundamental reforms at National Air Traffic Services. The recommendations in the report were acted on and it was possible for outsiders, including the media and staff, to ask the organisation what actions it had taken. This was public sector accountability at its best.

  8.2  Initially, however, National Air Traffic Services had strongly opposed the independent audit although it was commissioned anyway at the request of the Transport Committee. One reason that gateway reviews are not published is that some officials believe it would compromise the advice given if reviewers thought their concerns would be made public. This is shown to be nonsense by the Arthur D Little report which was full and frank in its disclosures. We have no reason to believe that if Arthur D Little were asked again to conduct a thorough analysis of the management of a major IT project, it would act with the same professionalism and courage as it did when assessing the New En Route project at National Air Traffic Services.

  8.3  If gateway reviewers believe the quality and rigour of their advice and work would suffer if their reviews were published, we would question whether they are too culturally close to those they are reviewing and therefore perhaps not be sufficiently independent and objective to reach the tough conclusions that gateway reviews sometimes demand.

  8.4  Gateway reviews have done much good in making departments more aware of the risks of complex software-led projects. There have been many specific achievements. For example a Gateway 4 Review in July 2001 on a greenfield IT project at the Criminal Records Bureau noted that there was no solid plan for a model office and pilot before the system went live. The bureau agreed that the time allowed in the contract for System Acceptance Tests was too short. It therefore decided to double the time and to add model office and pilot tests.

  8.5  On the other hand, when there was a subsequent Gateway 4A review on the same Bureau project in February 2002, a series of potentially serious problems were identified. This time, according to a report of the National Audit Office which was published on 12 February 2004, the gateway review accepted that "there was `now no turning back'", and recognised that, "on balance, the March 2002 operation launch would go ahead, given the confusion and bad publicity that would result from delay." Soon after going live with the new system, there were a series of operational difficulties which occurred for a complexity of reasons; at one point there was a backlog of more than 250,000 applications for checks on people who were to work with children or vulnerable adults. Although the situation has now stabilised, the National Audit Office reports that customers have been dissatisfied with the service provided by the bureau. In our view a fear of bad publicity should not be a material consideration in any decision to recommend whether a key government system should to go live or not.

  8.6  We have seen other weaknesses in gateway reviews: a pre-implementation gateway review of the tax credits gave it the green light.

  8.7  It may be that the reviewers did not have the information with which to reach any other finding. But it is surely the job of independent reviewers, if they are endorsing a project by giving it a green light, to be satisfied they have all the necessary information on which to base such a key decision? If they are not confident, perhaps they should withhold a decision on whether to endorse a project, even if this means delaying the implementation of a system to allow for further testing and analysis. One assumes there will always be departmental pressure on gateway reviewers to approve the work on a major project. At the Criminal Records Bureau, for example, the report of the National Audit Office acknowledges that there were considerable pressures to go live with the system. Knowledge that the gateway reviews would be made public and open to independent scrutiny would be a counter-balancing force to that departmental pressure.

  8.8  With Gateway reviews operating in an environment of secrecy, and without any Parliamentary scrutiny until the NAO publishes limited details of a review, sometimes months or years later, we believe there may be insufficient incentive on reviewers to always reach truly the courageous recommendations, perhaps to halt a project indefinitely while fundamental assumptions are challenged, that may be required to avoid project failures.

  8.9  There are many concerns about the assumptions that underpin the Department of Health's £2.3 billion National Programme for IT in the NHS. The first national systems under the programme are due to go live this year and there are concerns about the extent to which the department has obtained the buy-in of doctors and nurses, and how it will measure the success of implementations. The Department has declined, however, to publish any part of the gateway reviews.

  8.10  We recommend that gateway reviews are made public, or at least that there is an independent assessment commissioned on the extent to which gateway reviews are capable of pre-empting IT disasters without discouraging innovation or without greatly impeding the implementation of policy initiatives.

SOME QUESTIONS FOR THE CSA, DWP (AND OTHER DEPARTMENTS):

  9.1  Drawing on the report of Arthur D Little, which we mentioned earlier, we ask some questions of the DWP/CSA. We quote from the Arthur D Little report because, in our view, it has not been bettered as a thorough and genuinely independent analysis of what has gone wrong on a major project; and some of the mistakes made on the project are common to a number of UK government projects.

  9.2  Arthur D Little's report said: "We recommend . . . that National Air Traffic Services avoid `witch-hunts' and individual blame for unwanted events where possible, and focus instead on management process issues. . . We believe that the whole culture and environment within which reviews were held was a major contributor to weakness in picking up on major issues. . . The approach adopted was non-investigative, there was an unwillingness to face up to and discuss bad news, and a style which inhibited an open and frank discussion of difficult problems." Does the department believe that it has a willingness to face up to and discuss any bad news related to its IT projects, and a style of management that encourages open and frank discussion of difficult problems?

  9.3  Arthur D Little's report said: "A further important observation is that the underlying reason for the New En Route Centre delays was not `weak' project management—on the contrary, the senior NERC personnel who ran the project were experienced, strong project managers hired in from outside. It could be argued that their `firmness' in isolating the project from the influence of National Air Traffic Services' wider operations, was itself one of the major contributors to the problems . . . What is needed for NERC is not greater project management firmness, but rather better strategic management, team working, common understanding and shared expertise between the projects and operational sides." Is the department aware of the dangers of overly firm management?

  9.4  "The New En Route Centre progress reports did not provide sufficient information for adequate oversight by the department, especially with regard to early-warning of delays . . . There is no section in the report (from NATS to the department) intended to highlight concerns and key issues to be resolved. The only narrative section is the Commentary section, which in most of the reports only contained information on work completed and successes achieved. . . There is no mention in the report of major changes to either the project concept or the commercial arrangements of the project. For example, there is no mention in any of the reports of the effective termination of the parallel US AAS project . . . There is also no mention of any of the major systems contract amendments, most notably Amendment 8 which had significant commercial consequences . . . As a further comment, there is no mention of revenue costs versus budget in the reports. Revenue costs are excluded because they are not the direct concern or responsibility of the Department, and fall outside the project capital approval mechanisms. However, omission of these costs has some implications: without revenue costs, the Department does not get a complete picture of the total cost of the project to NATS. Does the department believe that ministers always have a full picture of the total possible costs of IT projects and the extent and seriousness of any problems?

  9.5  "One of the most striking, and perhaps in retrospect ironic, features of the New En Route Centre Project is the very strong emphasis on meeting the tight timescales, promulgated particularly by the then CEO and passed down to the Project Director. Meeting completion dates is important for any project, but in the case of the New En Route Centre this was elevated to a level of importance that almost seemed to outweigh other key considerations . . . The CEO in particular, as a consequence of his high personal commitment to the success of the project, maintained significant first-hand involvement, and had a strong sense of personal investment in achievement of the target Operational date. Can the department be confident that it has never elevated the importance of meeting timescales to the point where they outweigh over key considerations?

  9.6  "Finally, the Project Director and CEO were aware that even the hint of a possible delay to the [project], would immediately initiate a flood of questions, requests for clarification, and presentations that would divert their own efforts and attention away from solving the problems at hand, . . . The following examples illustrate the point:

    —  The Quarterly Progress Report . . . states that NERC will still be `on-line to meet the traffic demands of Summer 1997', yet it was already well-known to the NATS project team and Loral in August 1995 that there were potential problems in the schedule, and a full bottom-up review had already been initiated . . . There was no specific mention of this review in the report.

    —  It has been reported that Area Control Services maintained an outward commitment to the Operational date . . . even though they were sceptical that the date was achievable.

    —  NATS Board Paper . . . includes a confidential note to the effect that it was recognised that the Operational date was looking `increasingly unlikely', . . . and that announcement of further delay should be `carefully managed'.

    —  A number of interviewees in [our] evaluation confirmed that open debate of project problems was in general not encouraged outside the project team.

  Has the department been deterred from delaying the go-live of a major system by the possibility that a delay might immediately initiate a flood of questions, requests for clarification, and presentations that would divert their own efforts and attention away from solving the problems at hand?

  9.7  "We also acknowledge that changes have been made by the current CEO since 1997 to promulgate a more open management style . . . Nevertheless, it was reported to us in the evaluation that `fiefdoms' may be said still to exist within NATS which are again indicative of cultural barriers. Are there fiefdoms within the department that are indicative of cultural problems?

  9.8  "NATS' senior management other than the CEO had very little stake in decisions regarding scope, timing, and commercial issues on New En Route Centre, and were therefore not able to influence developments significantly. Do the department's senior management have an influential stake in decisions regarding the scope, timing and commercial issues on major projects?

  9.9  "NATS Board meetings are held monthly. It is evident from the meeting reports that a lot of material, often in great detail, was presented at these meetings. It is also evident from interviews that Board members felt they received little information enabling them to make informed decisions and carry out their Governance duties. This situation points to a number of causes:

    —  There was a lack of distillation of the key messages and their ramifications for the New En Route Centre and for NATS business as a whole in the information as presented. . . From documents available, the messages presented seem very mild when compared with the reality of the true risks for the New En Route Centre Systems Project and its impact on NATS business. Almost without exception, any negative statement about the NERC Systems Project is balanced by a positive one without an objective measure being provided of the severity of the risks or their potential impact. Does the department believe that its high-level reports always convey enough information about the true risks of failure on major projects, and that the impact of any negative statements is not cushioned or nullified by a positive message which lacks objectivity?

  9.10  "These [past audits] will have provided a good level of assurance that the specified procedures and systems were being complied with during the NERC Systems Project, although it does not necessarily indicate whether the procedures and systems themselves were effective. Do the department's internal audit reports question not only whether specified procedures and systems are complied with during major projects but question also whether the procedures and supervisory systems themselves are effective?

  9.11  "The National Air Traffic Services quality audit programme has included some 104 audits on the NERC Project to date since 1990, aimed at verifying compliance with the NATS Management System and the Work Instructions defined for the NERC Project. The CAA also conducts financial and management audits  .  .  .  Project management audits and routine project quality audits failed to identify the critical weaknesses in the New En Route Centre Systems Project". Is the department confident that internal audit is sufficiently proactive to identify any critical weaknesses in major projects?

Tony Collins and Ian Mitchell

February 2004





 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2004
Prepared 19 July 2004