||Impact of problems
The United Kingdom Passport Agency
In 1997 the Passport Agency let contracts to the private sector to undertake some of its activities and to introduce a new computer system. The objective was to improve the efficiency of the passport issuing process and improve the security of the United Kingdom passport. The Agency had planned to roll-out the new processing system to all its offices within a tight timetable before the busy season, but this was postponed following difficulties at the first two offices. In the spring and summer of 1999 there were serious delays in processing passport applications by the United Kingdom Passport Agency, partly as a result of problems with the implementation of new IT driven passport issuing arrangements in the Liverpool and Newport passport offices.
Although the specification for the new computer system broadly mirrored the processes and functions of the existing system, it did incorporate more sophisticated software and technology. Agency staff found the system more complex to use and it took some time for output to return to previous levels.
Most elements of the system development had been completed successfully prior to launch, but project delays meant that the productivity of the new system was not thoroughly tested by the Agency prior to going live.
The Agency's roll out timetable was short and allowed little room for manoeuvre should problems arise. The second office went live despite the first office failing to meet the criterion of output for continuing the roll-out.
When the Agency took its decision to halt roll out it had no contingency plan. Despite considerable effort, at no point during early 1999 did the Agency process sufficient output to catch up on the rising backlog.
From early 1999 the Passport Agency encountered increasing difficulties meeting demand for passports.
The unit cost of producing a passport in 1999-2000 is likely to be £15.50, compared to the Agency's £12 target.
Processing times reached 50 days in July 1999. The problems received widespread publicity and caused much anxiety for members of the public. The Agency's telephone service became overloaded, and members of the public had to visit and queue at one of Agency's offices.
The Agency employed additional staff, and optimised the efficiency of its examination processes, consistent with the need to maintain the integrity and security of its issuing procedures. Only the introduction of emergency measures enabled the Agency to reduce its backlog.
Departments must examine carefully the full implications of the introduction of changes to IT systems, including the impact of any policy changes that may affect demand for services.
It is essential that bodies draw up contingency plans to cover the risk that the system will not be delivered on time. Such plans should include an assessment of potential compensation payments to customers.
The National Insurance Recording System (NIRS)
NIRS supports the work of the Contributions Agency by maintaining details on over 65 million National Insurance accounts. The system impacts on almost every adult in the country as it maintains details of contributions paid and reflects their entitlement to contributory benefits. In May 1995 the Agency awarded a contract under the Private Finance Initiative to Andersen Consulting to develop a new system - NIRS2 - by February 1997.
The contract was to replace NIRS in one go, but by January 1996, it became clear that Andersen Consulting had realised the system size and scope were bigger and more complex than originally thought. They had therefore concluded that delivery of NIRS2 in accordance with the contract timetable was not the best approach. The company believed a phased approach would bring less development risk to themselves and less business risk to the Agency.
Following a full evaluation, the Agency accepted a proposal to vary the timetable. In 1998 and 1999 there were serious delays and problems with the delivery of the replacement National Insurance Fund Recording System during the extended transition and pilot period.
In our first report on this matter, we registered our concern about delays in implementing NIRS2, weaknesses in contingency planning to minimise the impact on the customers and business of the Agency; and the adequacy of compensation arrangements should Andersen be unable to deliver.
We expressed surprise at the delay in the Contributions Agency deciding that the replacement of NIRS1 would be carried out using a PFI approach. This delay meant that the timescale allowed for the PFI completion was very short. It also meant that the winning bidder had only 22 months to deliver this large and complex system on which important new pensions arrangements depended. This proved to be inadequate.
In our second report we reported that the situation was much worse. The system was still not fully operational, and the backlog of items relating to benefit awards and age related rebates which were waiting to be input into the system could take years to clear. There were over 1,500 unresolved system problems, many of which were crucial to full implementation.
Delays in implementing NIRS2 in full have led to the calculation of thousands of short term and long term benefits on an interim or emergency basis, and that payments to personal pension holders continue to be delayed. By January 1999, for example, it was estimated that potentially some 172,000 pensioners could be being underpaid in respect of their SERPS by between £0.01 to £100 a week.
In December 1998 there were more than 1 million amendments to the records of the self-employed requiring input to the Contributions Agency National Insurance Recording System (NIRS2). The Department was also unable to register some individuals to pay Class 2 self-employed contributions either by the Direct Debit or quarterly payment method as a consequence of the delays in introducing the NIRS2 on-line facilities.
Compensation has been paid out to both personal pension providers and benefit recipients. We noted that the Contributions Agency's failure to make timely rebate payments to personal pension providers had led to a loss of investment income to customers of personal pension schemes. We reported that the Agency had agreed to compensate these schemes, and through them those receiving personal and occupational pensions, by an estimated £38 million.
On 1 February 1999 the Secretary of State for Social Security announced that those receiving Retirement Pension, Widows Benefit or Incapacity Benefit who would not qualify for compensation under the existing rules, and who had experienced unreasonable delays, would receive a flat-rate compensation payment of £10.
Departments should ensure that they understand fully the potential impact of delay on their business and customers. The Agency were not able to transfer the business risk of not having the system of having in operation when required.
It is essential that bodies draw up contingency plans to cover the risk that the system will not be delivered on time. Such plans should include an assessment of potential compensation payments to customers.
It is essential that there are clearly defined roles and responsibilities for the parties to the contract.
When a PFI contract purports to have transferred the risk of late or non-delivery to the contractor, Departments should ensure that the business implications of late delivery are reflected in contractual penalties.
Inadequate levels of compensation have the effect of transferring risk back to the public sector.
The Immigration and Nationality Directorate Casework Programme
The Immigration and Nationality Directorate of the Home Office operate immigration controls at ports and process applications from people who have already entered the United Kingdom who wish to apply for citizenship, seek asylum, or extend their stay. The number and complexity of the cases, combined with limitations in the Directorate's business systems, had led to large backlogs in cases. In April 1996 the Directorate let a contract for a privately financed business change initiative which depended heavily on new IT. The contract is part of an integrated strategy to achieve a fairer, faster and firmer approach to immigration control.
The Casework Programme should have been fully implemented by October 1998. However modernisation of working practices has had to proceed in the absence of the new IT system. This has now been delivered and is being tested with aim of full availability by October 1999, a year later than expected.
A major factor in the delay of the project had been that Siemens had not ensured that the design of the new IT would be consistent with the organisational design and shape of the Directorate emerging from the business process re-engineering exercise.
The original proposal to use existing software packages as the basis of the solution was abandoned and Siemens decided to develop a bespoke system - a large IT development task - within the same timescale. In July 1998 Siemens decided to complete the programme without their main sub-contractor, who had been mainly responsible for the development of the new IT.
As a result of the delay in full implementation of the Casework Programme, including the IT element, there has been severe disruption to the Directorate's services for recent months, and a large increase in the backlog of cases. There are backlogs of 76,000 asylum cases and 100,000 nationality cases.
There has been considerable anxiety and hardship caused to thousands of applicants due to the failure to resolve their cases.
The Casework Programme delays and the uncertainty over the timing of the roll out of the Programme made it difficult for the Directorate to budget for casework expenditure in the short-term, and to take decisions on staffing levels.
Departments should consider whether programmes of this kind might be too ambitious to be attempted in one go.
It is crucial that departments think through properly the structure of their deals. Departments should always take a close interest in arrangements with sub-contractors, and that the terms under which sub-contractors are employed should align the incentives and penalties placed on all service providers to ensure the project objectives are achieved.
18th Report 1998-99
Ministry of Defence: Project Trawlerman
In the mid-1980s, the Defence Intelligence Staff required a computer system to handle large quantities of classified information. A £32 million firm price contract was awarded in July 1988. The planned date for site acceptance testing was October 1991. The project was abandoned in November 1996.
By 1991 it had become clear that the programme was well behind schedule, and in April 1992 the contractor entered a claim for additional costs. Following negotiations with the contractor, the Department agreed to amend the completion date to 31 January 1993.
Site acceptance testing of the Trawlerman system was finally completed in November 1993, more than two years later than originally planned.
The Department did not accept the system until 1995, and by then it could not meet the developing needs of the Defence Intelligence Staff. The original specification had not included a requirement for the system to be able to link to other computer systems, but by 1995 this was viewed as essential. The limitations of the system also meant that users could not view on screen several files at once, or merge and edit easily the information contained in separate computerised documents. In addition, the word processing facility included as part of the system was incompatible with others in use in the Department.
Following a review, the Department concluded that the system was obsolete, and in November 1996 abandoned the project without it ever being used.
The Department wrote off costs of £41million, and acquired a replacement system in 1997 at a cost of £6 million.
It is essential that departments recognise the complexity of the IT required at the outset.
There are considerable risks in relying too much on assurances provided by contractors that the system can be delivered, and not having sufficient involvement with them when a project runs into difficulties.
There are benefits in introducing systems in phases and designing them to be sufficiently flexible to allow for technological advances.
Delays in implementing projects place them at risk of being overtaken by technological developments.
The 1992 and 1998 Information Management & Technology Strategies of the NHS Executive
The NHS uses IT in order to provide appropriate care for patients. In 1992 the NHS Management Executive launched a package of projects, programmes and aspirations to ensure that IT and information were managed as significant resources. The projects ranged from an intranet to reviews of activities of General Practitioners. In 1998, the Executive launched a package of new and existing projects and service aspirations. Key aims of this package are sharing information about patients between doctors and other health professionals in Primary Care Groups, NHS Trusts and Health Authorities, and providing information about healthcare to both health professionals and public.
The Comptroller and Auditor General reported that, in launching its 1992 Strategy, the NHS Executive had communicated effectively a useful vision and set of principles for the development of NHS information management and technology.
However, the design of the 1992 strategy was unsatisfactory and its impact was undermined by certain shortcomings in implementation. A lack of overall objectives in the 1992 Strategy contributed to a lack of direction in its implementation.
While the Executive set objectives for individual projects, their business cases were not always complete in terms of specific, measurable and time related objectives, financial analysis and proposals for monitoring and evaluation.
The Executive did not consider how all the projects related to each other, and overall the Strategy lacked coherence.
The 1998 Strategy represented an improvement in design, although the Comptroller and Auditor General stated that its objectives and targets should be made specific and measurable if they are to be useful in monitoring expenditure and achievements. As with the 1992 Strategy, there was no overall business case, and there should be clearer plans for evaluation of the Strategy.
The lack of clear and coherent objectives in the 1992 strategy led to a lack of coherence between projects and programmes.
There was slow progress in implementation of the projects.
The impact of the projects and programmes in the 1992 strategy has been reduced or limited by poor project design, communication problems and, in particular, inadequate co-ordination with each other. They have not been fully evaluated.
In preparing a strategy it is important to have clear objectives and milestones.
Care should be taken when preparing business cases for individual components of the Strategy to ensure that interdependencies are taken into account and fully integrated, to avoid problems such as the double counting of benefits.
It is important that there is clarity as to how projects fit together which is understood by those implementing the projects.
There should be clear plans for the evaluation of the Strategy.
20th Report 1998-99
Home Office - Handgun Surrender and Compensation
In 1997 the Home Office needed a computer system tailored to meet the specific and complex requirements of the processing of claims for compensation after the surrender of handguns. Planning of the new system started in March 1997, with the aim of having it fully operational by the anticipated commencement of the large-calibre handgun surrender period, then thought to be June 1997.
The system had to be introduced very quickly.
A prototype version of the system was delivered to the Home Office in early June 1997, by which time the Government had decided on a start date of 1 July for large-calibre surrender.
A number of problems were identified during the system's testing stage, but because of the tight timescale, the system was put into operation while faults were still occurring.
Throughout the first month, processing productivity was significantly hampered by numerous and unpredictable system failures, such that staff confidence in it fell and the contractors were asked to provide amended software.
The system was not finally signed off as fully satisfactory until December 1997. Problems continued as the system struggled to cope with the additional users brought in to speed up the processing of claims.
Processing productivity was hampered.
Arising from initial concerns as to the reliability of the computer system, and as a safeguard against internal fraud, the Home Office instituted a final manual check of payments, which had slowed processing further.
There are significant risks to successful implementation in introducing a system within a very tight timescale.
Post implementation review.
25th Report 1998-99
Ministry of Agriculture Fisheries and Food: Arable Area Payments Scheme
Under European Union requirements, introduced in late 1992, the Ministry of Agriculture was required to implement an arable area scheme from early 1993 and an Integrated Administration and Control System for it and other schemes over the following three years.
The Ministry adopted a phased approach but had to implement the basic system very quickly.
Software was developed and updated on an annual basis, but released during the year in several stages to reflect when particular functions are likely to be required by users. This significantly affected the manner in which claims were processed. If the software requirements had been available at the outset, many claims could have processed in one go, rather than in three or four stages.
Subsequently, problems with the central computer led to delays in the validation of claims against the Integrated Administration and Control System database to check for duplicate claims, and restrictions on the number of claims that regional offices could send electronically at any one time for validation. In 1996 staff in regional offices had to re-submit each claim for local validation approximately four times on average and for central validation approximately 3.5 times on average.
There appeared to be a significant number of occasions when the system had been either partly or completely unavailable to users.
The validation difficulties caused further interruptions in the processing of individual claims in 1996 as staff had to wait at least a day for each central validation request to be processed before they could investigate claims that were rejected.
The absence of a database based on a geographical information system, the need to process each claim in a number of stages, and the increasing demands placed on the system by complex checks and periods of peak demands, lead to inefficiencies
The annual costs of the systems have escalated due to the difficulty of developing and maintaining the programming software
Introducing systems in a phased approach to provide flexibility and meet urgent requirements can lead to problems in processing and frustration for staff carrying out complex tasks.
HC 155 1998-99
Council for the Central Laboratory of the Research Councils (CCLRC) - Integrated accounting system
The CCLRC decided to replace their old and outdated cash-based accounting system with an integrated accruals based system to record all financial transactions, produce their accounts and provide meaningful internal management information.
The integrated accounting system was due to be introduced on 1 April 1997 for full operation by June 1997. The Council ceased to operate their old system on 31 March 1997, but did not follow the normally accepted practice of running the two systems in parallel. CCLRC estimated this would have incurred costs of up to £2.5 million in 1997-98, although the Comptroller and Auditor General reported that a more reasonable estimate was between £0.75 million and £1million.
The Council encountered problems with the operation of the new system, together with delays and errors in the migration of data from the previous system,
The project management structure failed to comply with best practice, particularly in terms of ensuring clarity of roles, responsibilities and accountabilities. There had also been a failure to ensure clear reporting lines to senior management consistent with their responsibilities.
The project team was not equipped to manage a situation where the contractor progressively withdrew before the project was completed and demonstrated a lack of effective control over the contractor.
Scrutiny of the project was compromised because the Audit Committee was presented with insufficient and over-optimistic information on the problems being encountered.
The system was still not fully operational in early 1999, two years after the planned implementation date and costs had overrun by 84%.
The original contract value was £544,000, but the Council estimated that they incurred further, unanticipated, direct costs of some £458,000 on additional hardware, training, legal and audit fees and temporary staff.
Inadequate controls over manual payments made to ease the backlog in payment processing on the new system led to nearly 500 overpayments to staff and suppliers, totalling some £270,000, most of which were subsequently recovered.
A fixed asset module did not account correctly for the Council's large capital asset base, forcing them to resort to a spreadsheet register to support their 1997-98 accounts.
The Council were also unable to reconcile their general ledger to their bank statements. There was an original difference of some £1m, which was eventually reduced to £48,000, which was written off at the end of the year.
The Comptroller and Auditor General reported that the new Chief Executive had taken prompt action to address the problems he had inherited and various parties to the contract were continuing to work together to ensure the system became fully operational in 1999-2000.
There are very significant risks in not introducing back-up procedures, such as the continuation of the outgoing system for a limited period. Full consideration of the risks and a realistic analysis of the case for parallel running must be undertaken.
It is not acceptable for the body concerned to rely on the contractor to manage the project.
Organisations must recognise that introduction of a bespoke financial and management accounting system to a demanding timetable is a business critical operation, requiring good project management skills.
If an organisation has reservations about the technical viability of a proposal, they should seek expert advice before proceeding.
Sponsor departments should be notified directly of significant problems impacting on their IT systems.
62nd Report 1997-98
The Purchase of the Read Codes and the Management of the NHS Centre for Coding and Classification
In March 1990 the NHS Executive purchased the copyright to a clinical coding system from Dr Read - the Read Codes - for £1.25m, and in April 1990 they established the NHS Centre of Coding and Classification to develop the codes for use across the NHS. The system is designed to enable clinicians and other data users to record the details of clinical care on a computerised record, share and exchange computerised clinical information and retrieve and analyse clinical information held in clinical information systems.
We considered it was a serious failing that the NHS Executive did not prepare a full business case, including a cost benefit analysis or investment appraisal, to assess the benefits and risks of different options to justify investment in the clinical terms projects.
In addition, we noted that the Executive had set the objectives for the project in 1992, but did not set out criteria for measuring whether the key objective had been achieved until July 1994.
The NHS Executive belatedly agreed to a full evaluation of the Read Codes before implementation.
By March 1998 the Executive had spent £32 million, and we concluded that it would cost much more to implement the project in full.
We expressed concern that eight years after the Codes were purchased and three years after the clinical terms projects ended, Version 3 of the Codes was being tested and used in only 12 NHS hospital sites. We also commented that if the benefits of the Codes were as good as the Executive suggested, progress had been very disappointing.
The Executive told us that the version for use in hospitals would require continuous development, and that it would only function effectively with the appropriate clinical systems in place. Provided this was achieved it would take five years to implement, because implementation was linked to wider developments in information technology. The Executive noted that a clear management and accountability framework had been put in place at the Centre, and internal auditors had endorsed the new controls.
Sound project appraisal must include a rigorous assessment of costs and benefits, and a realistic assessment of any risks. These should be contained in a properly structured business case.
If departments are to secure the confidence of users in such projects and overcome any scepticism about new ideas and technology, independent evaluations should be a core requirement of any major development project.
It is important that departments do not feel locked into using projects because they have already spent a lot of money on them.
58th Report 1997-98
Benefits Agency: Jobseeker's Allowance
During our hearing on the Appropriation Accounts 1996-97 (Administered Social Security Benefits and Other Payments) in 1998, the Benefits Agency explained that when they introduced the Jobseeker's Allowance in October 1996, there were a number of difficulties with the new computer system.
The system experienced a number of teething problems. For example, the interface with other benefits failed to work properly in the first months of operation.
The computer problems led to the initial high number of clerical cases25,000 with a greater risk of error in the awards processed.
High levels of error on Jobseeker's Allowance contributed to the Comptroller and Auditor General qualifying his opinion on this account.
There were also mistakes because 79,000 staff had to be trained over a long period. Implementing the new benefit was a huge challenge involving staff in two Agencies across two Departments.
Although the Agency advised us they were initiating action to assess and measure the current scale of error in the accuracy of clerical awards, the Comptroller and Auditor General had reported that around a third of the estimated value of error in the account (£46 million) was explained by errors in awards calculated manually, which had been necessary as a result of computer problems.
Introduction of new systems may have an adverse effect on performance in the initial stages following implementation.
58th Report 1997-98
Benefits Agency/Post Office Counters Limited : Benefit Payment Card
In 1996 the Benefits Agency, in partnership with Post Office Counters Ltd and a private sector supplier (ICL Pathway), began the initial phase of implementing benefit payment by payment card at post offices, with a view to full implementation within the next few years. This was a Private Finance Initiative project aimed at delivering a fraud free method of paying benefits, developing a system that met recognised accountancy practices, enabled the Post Office Counters Ltd to improve competitiveness and increase efficiency through automation, and provided an improved level of service to customers. The project had an original target date for national roll out starting in April 1999, subject to successful completion of trials.
Trials of the card started at the end of 1996 and were extended to 205 post offices.
The project suffered considerable delays and setbacks, and a major review was commissioned to decide the way forward.
In May 1999, the Department of Trade and Industry announced a revised agreement with ICL to remove the magnetic strip payment card from the project. Given the delays, this was now an outdated concept, with clearing banks already moving towards the use of smart cards.
Building on banking technology, POCL and DSS/BA will be working on plans to introduce the new arrangements in 2003, with computerisation of the Post Office network by the end of 2001.
The problems have resulted in delays to the implementation of one of the Benefits Agency's major projects to tackle fraud.
Estimated fraud savings from the automation of benefit payments at post offices, if the project had been completed on schedule, would have been about £190 million a year (including Order Book Control Service savings that would gradually have been superseded by Payment Card savings during roll out).
Delays in implementing projects place them at risk of being overtaken by technological developments.
52nd Report 1997-98
Intervention Board Executive Agency: Integrated Accounts project
The Intervention Board implemented a new integrated accounting system in May 1996. This followed previous Committee of Public Accounts concerns about the Board's management and accounting systems in 1990.
In implementing all three phases of its new integrated accounting system, the Board experienced delays and significant cost overruns.
The final phase was implemented in a single step without parallel running as the Board judged that it was not technically feasible to do so without significant costs. In addition, there was only a limited fallback facility to enable the Board to revert to its old system in the event of problems. The Board advised us that it was on the best advice then available that they did not adopt parallel running.
We were very concerned that an off-the-shelf software product should have developed faults so soon after implementation.
We were also very doubtful as to the value provided by the consultants who advised the Board and were surprised that they were reappointed to assist with resolving the problems. The Board felt that criticism of the consultants was not wholly justified since they completed their work in accordance with their contract.
The Board encountered a number of problems including under-performance of the system which led to discrepancies between the accounts modules; non-closure of accounting periods which led to incorrect postings; software problems which resulted in postings to incorrect accounting periods and codes; unreliable accounting reports which were produced as a result of mispostings and the incorrect linking of account codes; and cash received which could not be matches to invoices issued.
There were substantial cost over-runs as a result of extra consultancy time arising from the delays. The outturn was roughly double the estimated cost.
Because of the problems encountered in the implementation of the system, they were unable to complete essential reconciliations between certain account balances recorded in the general ledger and subsidiary accounting records at the year end.
In the absence of reconciliations, the Comptroller and Auditor General was not able to obtain all the information and explanations considered necessary, and therefore qualified his opinion on the accounts for 1996-97. In addition, the implementation of the project cost roughly twice the estimated cost.
There are significant risks in failing to undertake parallel running or having insufficient backup. A fallback position should be maintained for a sufficiently long time to establish that the new system is working satisfactorily.
The introduction of a major new accounting system must be accompanied by provision of proper desk instructions and training for staff.
27th Report 1997-98
Department of Social Security: Measures to Combat Housing Benefit Fraud
In 1997-98 the effective delivery of Housing Benefit still depended on the exchange of over 22 million pieces of paper between local authorities and the BA. In 1993-94 the Department of Social Security piloted the use of remote access terminals which gave local authorities direct access to the Income Support computer system.
The introduction of terminals improved liaison and provided local authorities with instant access to information, but only a few terminals had been installed at the time of our hearing.
We considered it unacceptable that more than four years since first piloting these terminals, further piloting was now only underway, with the aim of making terminals available nationally from April 1998.
In May 1998 the Department offered terminals to all 409 local authorities and over 350 applied for them to be installed. The Department plan to install them by the end of April 1999.
The Department told the Committee that they were determined to address the difficulty in exchanging information between the Benefits Agency and local authorities by a series of measures. Within the context of the ACCORD project they are considering an information technology strategy for Housing Benefit.
The effective delivery of Housing Benefit has remained dependent on the exchange of millions of pieces of paper.
Opportunities for sharing information have been missed.
Delays in the introduction of IT systems can have an adverse impact on the development of business.
33rd Report 1997-98
Crown Prosecution Service
In 1989 the Crown Prosecution Service told our predecessors that a new case tracking computer system, which was planned to replace a variety of outdated systems, would be its top information technology priority and would be implemented by 1993-94. The new system was intended to improve efficiency by capturing and manipulating information on cases more efficiently and by generating management and performance information.
In 1997, the Comptroller and Auditor General reported that, by the end of that year, the system had been implemented in just over half of CPS branches, and a substantial amount of performance information still has to be produced manually.
The slippage was caused partly by management effort being directed towards other priorities such as the introduction of teamworking and the need to reschedule implementation to take account of changes in the organisation of CPS.
The delays in implementing a comprehensive case tracking system to the planned timescale hampered the efficient and effective operation of the Crown Prosecution Service.
The original specification was overtaken by changes in working practices and new technology and, mainly as a result of these, in 1997 the system was expected to cost £15.9 million, twice as much as the original estimate.
In 1997 the CPS ceased implementation of the project on the grounds that the technology was outdated. The Committee welcomed the overdue decision to do this and develop a new system via a PFI project.
A PFI contract for a new system is now expected to be finalised in autumn 2000, with implementation to all CPS branches then expected to take two years.
Long delays in implementation can lead to projects being overtaken by technological developments.
The CPS must set key milestones for the new project and a clear timetable for delivering the system to all local staff.
Sufficient flexibility needs to be built into the new system to allow for the development of data exchange and communication links with other criminal justice agencies and for future changes in criminal justice legislation.
7th Report 1996-97
The Hospital Information Support Systems Initiative
By the late 1980s, many acute hospitals had developed their own computer systems. They were not linked together and as a result the recording of information was slow and inefficient. The NHS Executive launched the Hospital Information Support Systems Initiative (HISS) in 1988 to explore the costs and issues involved in implementing integrated systems in NHS hospitals in England. The Comptroller and Auditor General examined 6 of the sixteen projects funded under the Initiative.
In 1996 our predecessors were concerned about the progress of the initiative. They noted the gap between the Executive's plans and achievements in implementing integrated systems, and looked to the NHS Executive to convince those trusts not already involved of the value of the systems.
The Executive went ahead with one project despite high costs and the significant risks involved. The fact that two major suppliers did not believe it was feasible did not act as a warning.
Our predecessors noted that the NHS Executive had selected the three pilot projects under the Initiative within the very short period of two months. Subsequently, each of the projects suffered problems and delays.
Our predecessors considered these delays may have stemmed, at least in part, from the Executive's failure to prepare the hospitals to run their projects, and that they might have obtained better value for money had they taken more time to ensure hospitals were fully prepared. The Executive argued that in view of the innovative and complex nature of these projects all reasonable steps had been taken to manage them effectively.
Our predecessors were concerned that projects proceeded without sound business cases being established. Although the aim was to learn lessons to inform the development of integrated systems in other hospitals, our predecessors were surprised that the development of formal evaluation mechanisms was only started four years after the start of the initiative. The Executive replied that full business cases for early pilot sites, as defined by today's standards were not realistic given the decision to invest in order to understand the issues.
They noted that all six projects examined by the National Audit Office had experienced delays, which they considered were likely to have had an adverse effect on the quality of care those hospitals could provide.
It is essential that departments learn from the lessons of previous projects.
The commitment of users such as clinicians is crucial to the success of such projects. All users need to be involved in the development of these systems.
There are considerable dangers in trying to implement projects too quickly and not preparing sites properly.
Projects should not proceed without first establishing that there are sound business cases for doing so, and without undertaking full investment appraisals and risk analyses.
The need to consider evaluation mechanisms at an early stage is integral to all initiatives of this kind.
HC 11-v 1996-97
Northern Ireland Vehicle System Replacement Project
In the early1990s the Driver and Vehicle Licensing Agency (DVLA) decided to replace the Northern Ireland vehicle system at a cost of £3.93 million, for implementation by March 1994. In 1997 the Comptroller and Auditor General reported on the write off of £3.7million of fruitless expenditure incurred on the project.
Work began in October 1992 on a project definition document to develop a detailed system definition. The contractor, DVOIT, requested only a high-level user requirement as the replacement system was intended to be a conversion of the existing system plus fixes of existing faults.
Over time, amendments to the user requirement were needed to correct misunderstandings, and changes were cited as reasons for increasing costs.
In 1993 DVOIT increased their cost estimate for development staff time from £2.3million to £3.4 million, citing changing user specifications and previous mis-estimates as reasons. The project cost rose and DVLA repeatedly sought details of costs incurred to date from DVOIT. Information was not forthcoming, partly because the DVOIT project manager was changed three times.
In December 1993, the contractor, DVOIT, was sold to EDS. In February 1994, EDS informed the project board that the additional work required to complete the project would put back the implementation date from March 1994 to January 1995. They also revised the estimate of the total project cost to £5.95 million.
Following a meeting with EDS in February 1995, the project board asked CCTA to review the EDS proposal and cost estimates. CCTA identified project management as a significant problem. It was cumbersome and inefficient. In addition, project managers had to adapt to the changing nature of the relationship with the contractor from in-house supplier, to executive agency, to external commercial supplier. Lack of continuity of key project staff was also a problem.
In March 1996 the project was abandoned, and fruitless expenditure totalling some £3.7 million written off.
In November 1995 CCTA reported costs totalling £3.5 million had been incurred on the project to November 1995. The completion date had slipped from March 1994 to November 1996, and total project costs were forecast to exceed by 71 per cent (£2.78 million) the original business case approved by Treasury.
A new system was being developed at the time the Comptroller and Auditor General reported.
Business cases must be based on a clear understanding of user requirements and identify clearly and realistically the cost of individual components of the project.
Contracts should, wherever possible, be awarded on a fixed price basis and should not be open-ended. This transfers risk to the supplier and would prevent cost overruns.
It is essential that there are clearly defined roles and responsibilities for the parties to the contract.
24th Report 1996-97
Inland RevenuePay and File
In October 1993 the Department introduced new procedures for assessing and collecting corporation tax known as Pay and File. These required companies to pay over corporation tax based on their own calculation of liability, and to file the tax return on fixed dates. The new arrangements were designed to streamline the administration of tax by reducing the need for estimated requirements and consequent appeals. The Department introduced a new computer system to support Pay and File.
Our predecessors reported that the new computer system to support Pay and File had been introduced to time and budget.
The system was described by the Department as perhaps the most complex they had so far introduced.
The Department advised our predecessors that in the early days there were teething troubles and in the availability of the system to staff was in the low nineties, as against 99% they had hoped to achieve. This improved subsequently and reached 100% at times.
The training for the new system did not go as well as they had hoped. The Department told our predecessors that they misjudged the speed and facility with which some inspectors would take to computer training, and had relied too much on distance learning, and not enough on tutor-led training.
In retrospect, they acknowledged that it had taken some time to develop the right blend of training. Lessons learnt here had been applied to the introduction of Income Tax Self-Assessment.
The availability of the system was less than expected for a while after introduction.
It is important to keep an expert team in being some time after the system becomes operational so that any teething problems can be put right as quickly as possible.
Careful consideration needs to be given to he type of training available to staff using new systems.
15th Report 1995-96
Department for Education and Employment - Teachers' Superannuation scheme
Teachers in both the state and independent sectors can belong to the Teachers Superannuation Scheme, administered by the Teachers' Pension Agency, an Executive Agency of the Department for Education and Employment. In 1991 the Department decided to invest in a major new computer system for the scheme.
The Government Actuary reviews the scheme every five years and makes recommendations on contribution rates.
The Department had expected that the new IT system would enable them to provide the information required straight from the new database. However, the system had failed and was the subject of litigation at the time of our predecessors' hearing.
The Agency had had to go back to the original database and apply interrogation programmes to it. However, these had not been ready and had needed upgrading.
The review by the Government Actuary covering the period to 1991, was delayed principally by difficulties experienced by the Department and Agency in providing him with information. At the time of our predecessors' hearing the review had not been completed.
The Agency advised our predecessors that they now collected information on a year by year basis so that at the time of the next review they would not have to go through five year's information from scratch.
7th Report 1993-94
Department of Education and Employment: Computer systems for training and enterprise councilsIn 1988 the Department of Employment decided to introduce an integrated computer system - the Field System - into their regional and area offices responsible for delivering training and enterprise programmes. Later that year the Government created Training and Enterprise Councils (TECs). The Department decided to press on with implementation in order to meet the IT needs of TECs when they started in 1990.
The Department adjusted the design of the Field System to meet the core needs of TECs, and they expected TECs to use the system well into the 1990s.
The Department experienced significant problems in developing and implementing the system, which in the view of many TECs, did not adequately meet their needs.
Following a review of TECs information technology needs, the Department ceased to provide information technology systems and services directly to TECs, who assumed this responsibility from April 1993. The Department argued that the system was never intended to meet all TECs' needs and it was designed to enable them to develop their own systems as their operating requirements became clearer.
The cost of the redesigned system was estimated in the business case as £71 million. By 1993 the Department had spent £48 million, of which about £19 million was on hardware.
The Department spent £48 million on the Field System but most TECs considered that it had not enhanced their performance. The Department disagreed, believing that the system considerably enhanced TECs' performance. It provided them with their core accounting and management support systems, without which, for example, their ability to handle money from the Department would have been weakened.
The Department did not share the Committee's conclusions about The Field System, which they stated was delivered within budget and substantially to time. They stated that planned savings had been secured and most of the expected benefits secured.
Full involvement of prospective users is essential if projects are to meet fully user needs.
Systematic risk analysis is essential before embarking on a high risk project.
Project management staff must have the necessary background and experience.
Departments should not engage consultants in a haphazard manner.
27th Report 1993-94
Ministry of Defence: Disposal by Sale of Defence Surplus Equipment and Stores
In April 1991 the Department installed a new system to aid the efficiency of the Sales Directorate.
The Department installed a new computer system in April 1991 to aid the efficiency of the Directorate, but the costs of the system rose substantially and nearly three years later it was not functioning properly.
Problems experienced included hardware lock-ups, a lack of proper trunking and cabling, inadequate training and insufficient professional support.
One problem with installation of the computer system had been a lack of appreciation of the Directorate's requirements. There was an internal 'disconnect' between the Directorate and the Department's computer organisation; they had not got it right between them, so what had been implemented had been inadequate.
The Department accepted that there were substantial problems associated with the implementation and that they had underestimated the scale of the task.
As a result the Directorate did not have an adequate computerised information system, and were relying on a largely manual database, which hindered the timely extraction of information for management purposes.
The Department acknowledged to our predecessors that this was not a large, complex system, but it was absolutely essential to the provision of adequate management information on the disposals operation.
The cost of the project had overrun by 43% .
Departments must ensure that their computer requirements are clearly understood. Clear definition of requirements is a basic tenet of good practice and there should be good co-ordination between IT experts and users.
Departments must assess carefully the size of the task and ensure that they have an adequate system of project management.
63rd Report 1992-93
Wessex Regional Health Authority - Regional Information Systems Plan
In 1984 the Authority launched their regional information systems plan. This was intended to provide computer systems which would optimise the use of information to improve clinical and other health services. In April 1990 the plan was abandoned with at least £43 million having been spent.
In July 1992 the appointed auditor reported on serious failures in the management of the project.
These included a failure to control the activities of consultants upon whom the region was heavily reliant.
There were serious failings in contract, financial and management controls, such as the awarding of a contract following a tendering process which did not enable meaningful comparisons to be made.
There were serious conflicts of interest at a senior level within the authority.
Given the ambitious nature of the project and the risks that money would be wasted if it did not succeed, appropriate targets and penalty provisions would have encouraged the company to work for its success and secure a measure of protection for the taxpayer.
Our predecessors stated that these represented a serious failure on the part of the Regional Health Authority to secure accountability from the then Regional General Manager and failure on the part of the NHS Management Executive to act with sufficient urgency.
After expenditure of some £43 million over a period of six years, the project was abandoned in April 1990 without any significant benefit having accrued to the region.
The NHS Management Executive considered £20 million was wasted between 1984 and 1990.
Bodies should avoid undue reliance on consultants.
57th Report 1992-93
West Midlands Regional Health Authority
Our predecessors reported that in April 1990 the Regional Supplies Division of the West Midlands Regional Health Authority entered into an arrangement with a major computing firm to carry out preparatory work for the development of an electronic trading system. The business case anticipated development costs over the first five years of £5.3 million, offset by royalties paid by the contractor, estimated at £3.9 million.
Development started in May 1990 and continued until January 1992, when work was halted, following the setting up of the National Health Service Supplies Authority. By January 1992 the contractor had billed the Regional Health Authority for £7.3 million.
Our predecessors were advised that the contractor produced a working system in line with the Authority's changing requirements, and that it was in partial use at four sites until December 1992, when the Authority decided not to proceed with the development.
The project was not viable because information contained in the business plan was speculative and unrealistic.
Proper market research was not carried out and suppliers were not consulted. Estimates of supplier take up were significantly overstated. Potential customers were not consulted and the royalty projections were unrealistic.
Losses made here were part of the waste of least £10 million as a result of serious shortcomings in the management, control and accountability within the Health Authority.
Business plans must be realistic and soundly prepared.
50th Report 1992-93
Northern Ireland Education and Library Boards Information Strategy
In 1984 the Department of Education for Northern Ireland began to develop a common IT strategy for administrative functions within the five Education and Library Boards. Expenditure on development and implementation costs £15 million by 1991.
In the absence of any expertise in IT on the strategy Steering Group, they were wholly dependent on consultants. The Group decided not to seek CCTA's services but rely on consultants' own methodology for procurement.
Because the Group did not adequately specify the Board's requirements for three priority financial systems, the supplier had not provided hardware sizing calculations or an undertaking that its proposed hardware solution would satisfy the Board's needs and the Steering Group had not challenged the supplier's proposals, the computers purchased were not capable of meeting the Board's needs.
The supplier was not able to provide a fully integrated system and offered separate payroll and software packages which were to be linked to allow for the transfer of data. The linking of the two packages proved to be a critical and persistent problem.
There were deficiencies in the project management methodology, including the absence of detailed cost monitoring.
Only a limited number of project management evaluations were undertaken and they were not regarded as adequate.
The Committee were concerned at poor value for money from a strategy that between April 1985 and March 1991 cost more than £15 million.
Care should be taken to specify user requirements in as much detail as possible.
Purchasers should avoid being too dependent on consultants.
There must be a clear statement of objectives and the establishment of effective project management at the start of any project or programme.
There must a clear statement of what has actually been achieved against objectives at the end of the project.
Evaluations of projects are very important in enabling lessons to be learned for the management and development of future projects.
13th Report 1991-92
Ministry of Defence: Support Information Technology
The Ministry of Defence use IT for management and administration, and for planning and conducting military operations. The National Audit Office examined their performance in delivering IT systems and improving the planning and management of information technology.
Our predecessors recognised the complexities involved in implementing IT in a large organisation such as the Ministry of Defence, but considered the cost of the learning process was unacceptably high.
There were a number of problems with the LANDSCAPE project. It suffered from successive delays due to changing user requirements, the contractor's achievement on the software was imperfect and the hardware was unsuitable for the project.
Fundamental misjudgements were made on the SEMA project which led to a nine-fold increase in the development effort required.
User involvement in IT had been insufficient to ensure the systems were capable of meeting business needs.
The 1988 IT strategy identified project management weaknesses as a factor in the failure to fully realise the benefits of investment in IT.
All nine systems examined had suffered delays varying from five months to two years, postponing the achievement of predicted benefits.
Of four systems subjected to post implementation review, only one had achieved all intended financial and operational benefits.
The LANDSCAPE project had resulted in a loss of some £6 million.
In the mid-1980s the Department recognised the need for co-ordinated planning and in 1988 approved a strategy for support IT.
Clear definitions of user requirements were important before going ahead with projects.
The Department recognised the need to break projects down into shorter tranches and apply strict control over requirement changes during development.
Post-implementation reviews are crucial in ensuring that expected benefits have been achieved and in identifying lessons for the future.
HC 163-II 1990-91
Foreign and Commonwealth Office - Appropriation Accounts
The Foreign and Commonwealth Office implemented a new computerised accounting system.
In 1987 the Foreign and Commonwealth Office decided to replace its computerised payment and accounts system.
The Department employed consultants to study the Department's book-keeping system and define future requirements. In 1988 they made a firm recommendation for a package, which the department accepted.
The Department aimed to start parallel running from autumn 1988. As their IT staff did not have the time and Finance Department did not have the experience, the Department asked the consultants to undertake the testing. The software could not be delivered to time and the company ran into financial difficulties. Eventually they delivered the final part of the software but went into liquidation and could not deal with problems arising.
The Department began parallel running in November 1989 but identified discrepancies in the accounting data. Shortly afterwards the old computer broke down and could not be repaired. The Finance Department had no option but to run the new system even though it had not been fully tested and was known to have fault.
Problems persisted including hardware difficulties and technical problems. By May 1990 it was apparent that the discrepancies could not be resolved. Staff were drafted in to undertake a major reconciliation.
Because of difficulties in implementing a new computerised accounting system, the Comptroller and Auditor General reported that the Foreign and Commonwealth Office were unable to produce their four Appropriation Accounts for 1989-90 to the agreed audit and publication timetables.
The accounts were signed by the statutory date, but the Department had been unable to balance their books across the combined accounting records produced by the new computerised system.
Expenditure on the new system to the end of September 1990 amounted to some £937,000, two-thirds more than the original estimate. The cost over-run was due almost entirely to the extra cost of consultants' fees arising from problems experienced during the extended implementation period.
It is essential that bodies draw up contingency plans to cover the risk that the system will not be delivered on time.