Evidence submitted by Professor Brian
Randell (EPR 20)
The submission is on behalf of the group
of 23 senior academics in computing and systems, listed at the
end of this document
EXECUTIVE SUMMARY
This submission addresses the issue: "Current
progress on the development of the NHS Care Records Service and
the National Data Spine and why delivery of the new systems is
up to 2 years behind schedule". It draws on the Dossier of
Concerns regarding NPfIT that we have assembled from a variety
of sources, and recently made available to Members of the Select
Committee. Despite the difficulty of assessing NPfIT's plans and
progress, caused by the Programme's size and complexity, the secrecy
regarding detailed system specifications, and the atmosphere of
fear that prevents many NHS staff from expressing criticisms,
our Dossier contains extensive evidence, some but by no means
all anecdotal, that supports our assessment that the Programme
is in serious danger. The huge range of problems, covering technical
matters, methods of procurement, the lack of buy-in from stakeholders,
privacy and security questions, delivery delays and spiralling
costs, greatly complicate the task of correctly identifying the
fundamental causes and most effective remedies. Hence our recommendation
that a detailed technical review of the Programme be commissioned,
a review that must be open and manifestly independent if public
confidence in NPfIT is to be regained.
Brian Randell was with IBM Research in the States
1964-69, before being appointed to a Chair at Newcastle University.
He has published nearly two hundred technical papers and reports,
and is co-author or editor of seven books. He is now Emeritus
Professor of Computing Science, and Senior Research Investigator,
at Newcastle University. He received the 2002 IEEE Emanuel R.
Piore Award for his research on system dependability.
INTRODUCTION
1. We, a group of concerned senior academics
in computing and systems, first wrote to the Select Committee
last April, saying that we believed that NPfIT was showing many
of the symptoms we have seen in major IT systems that had eventually
been cancelled, overrun massively in terms of time and anticipated
budget, failed to deliver an acceptable service to users or to
reach business benefit targets for their organisations. We are
pleased that the Committee has now agreed to hold an inquiry into
the critical aspect of NPfIT, its support for and use of Electronic
Patient Records (EPRs).
2. Since we first wrote we have compiled,
and made available in printed form to all Members of the Committee,
an extensive Dossier of Concerns. This has been assembled from
nearly 600 published and unpublished sources, ranging from scholarly
research papers to what might (unwisely) be dismissed as mere
media rhetoric. Our submission makes numerous references to this
Dossier— for convenience simply by indicating the relevant
section number, eg "3.8.8", in the printed version of
the Dossier, dated 18 January 2007. (This version can be found
online, along with the current further-extended version, via our
website at http://nhs-it.info.)
3. As one of the MPs we sent our Dossier
to remarked to us, it is very difficult to obtain an adequate
picture of NPfIT free of the "entanglements of political
axe-grinding, professional jealousies and commercial interests".
An even greater difficulty is the climate of fear in the NHS that
has prevented many from expressing publicly the views that they
have been willing to share with us in private. Nevertheless, even
though much— though by no means all— of the evidence
in our Dossier is anecdotal, and from secondary sources, we believe
that it provides strong support for our assessment that NPfIT
is in serious danger. What is significant about our Dossier is
not just its size, but the range of problems noted, covering technical
matters, methods of procurement, the lack of buy-in from a range
of stakeholders, questions of privacy and security, delays in
delivery and spiralling costs.
4. We have also attempted, as outsiders,
to assess the technical merits of NPfIT, particularly those of
direct relevance to the safety and reliability of the overall
system. However many details of NPfIT (even the contractual integrity
and availability requirements and specifications) are regarded
as commercially highly confidential, making this task virtually
impossible. (Such difficult-to-justify secrecy has, we are certain,
also contributed to the lack of confidence that many working in
the NHS have in the Programme [3.8.8].)
5. Our hope is that the outcome of the Committee's
Inquiry will be the setting up of an open independent constructive
technical review, ideally of NPfIT as a whole, but at least of
the centrally-provided NHS Care Records Service (NHS CRS), and
of the various systems being provided by the Local Service Providers
(such as Patient Administrative Systems, Clinical Systems and
Departmental Systems), that together support the creation, maintenance
and utilisation of EPRs. Dr Granger and his senior colleagues
publicly expressed support for such a review following our meeting
with them last April. It is we believe the best way of arriving
at a disinterested expert assessment of NPfIT and of significantly
improving the chances of a successful, timely and well-accepted
outcome of this major investment by the NHS. (The current refusal
by NHS to contemplate such a review is worryingly reminiscent
of management attitudes during London Stock Exchange's disastrous
Taurus project [Drummond 1996].)
THE ELECTRONIC
PATIENT RECORD
6. Virtually all the claimed clinical advantages
for patients of centralised EPRs (at cluster or national level)
could be achieved by replacing paper records with electronic ones
at the local (ie trust) level [2.6, 3.5.21]. The claimed importance
of being able to access a central EPR directly when a patient
requires treatment far from home is not supported by evidence
[2.7]. Making what could have been local record keeping part of
a cluster-level, leave alone an immense national-level, "system-of-systems"
introduces system interdependencies that, because of their effect
on system complexity, pose risks to system reliability and availability
that in our judgement are likely to prove out of all proportion
to any potential benefits [3.8.8]. Also the integration of EHR
files at cluster, and certainly national, level greatly exacerbates
the problem of maintaining patient confidentiality [Javitt 2005,
2.8, 3.5.3, 3.8.25].
7. Electronic records need to be generated
as a by-product of relevant medical activity, and at the time
of that activity, if they are to be of direct support to health
care (for example in preventing possible clinical errors, such
as related to drug dosage) [2.1]. If in contrast their generation
has (perhaps because of usability or system performance limitations)
to be undertaken as a supplementary time-consuming after-the-fact
task, especially if it is one of little evident direct benefit
to patients or clinicians, there will be much less incentive on
the part of staff to undertake detailed record generation or to
maintain quality control [Brennan 2005, 7.1.1].
8. Moreover, such are the differing circumstances
from hospital to hospital that centrally-imposed standard EPR
systems will often prove ineffective, and will not be used as
intended, as Professor Eason, for example, has convincingly demonstrated
in his study of mental health trusts [Eason 2007]. The clear implication
is that it would be better to let local experts decide how best
to satisfy local needs and circumstances, to identify minimum
standards needed for interlinking local systems, and to defer
such interlinking until after local systems have been successfully
implemented and gained general support.
THE CAUSES
OF DELAYED
DELIVERY OF
THE NEW
SYSTEMS
9. Many studies have reported significant
time delays, cost overruns and, all too frequently, complete failures
of projects that involve extensive software development or customisation.
For example, a 1995 study [Jones 1995] of 164 software projects
concluded that over 24% of projects were cancelled and that two-thirds
experienced significant cost and time overruns. A 2001 survey
[Taylor 2001] reported that of more than 500 development projects,
only three met the survey's criteria for success. A 2002 survey
of 13,552 IT projects [Standish Group 2002] reported that only
34% were completed on time and to budget and that 51% though completed
and operational were over-budget, over the time estimate, and
offering fewer features and functions than originally specified.
10. The ever-growing levels of ambition
on the part of system builders and system commissioners are such
that the situation is not improving: in 2007 a US National Institute
of Standards and Technology report stated that: "By most
estimates, over half of all large application development projects
. . . end in failure— after all the time and money is spent,
the product still cannot be used operationally".
11. Hence it is clear that the utmost care
has to be taken to (i) avoid undue ambition, (ii) make sure that
the most circumspect software acquisition processes are employed,
(iii) minimise undue dependence on the continuous correct functioning
of the complete system, (iv) evolve towards the intended overall
system via a sequence of practical and cost-effective intermediate
systems, and (v) avoid at each stage the trap of specifications
that are vague, changing or in conflict [Curtis 1988, 3.8.25].
12. Many healthcare systems are necessarily
large and complex, but the NHS is huge and organisationally highly
intricate [Beynon-Davies 1994; Wyatt 1995]. A fully-integrated
NHS-wide healthcare system is vastly larger and more complex than
any previous healthcare system [Javitt 2004]. Indeed it is admitted
to be the world's largest civil IT project ever. Thus NPfIT is
by definition an undertaking that is inherently "at risk".
Moreover, given the crucial and pervasive role played by EPRs,
we believe much of this risk arises from the complex systems supporting
cluster-level and national-level EPRs.
13. Adverse outcomes from large IT projects
rarely can be linked to a single cause (Lyytinen and Hirschheim
1987). Indeed analyses have identified many different causes.
Some we list here, with representative references to relevant
sections of our Dossier, as a contribution to identifying problematic
aspects of NPfIT.
— cryptic or concealed agendas [3.3.2,
3.8.31]
— not involving users or not incorporating
organisational needs [2.1, 2.3-4, 3.4.1-3, 3.4.9-10, 3.4.12, 3.4.16-17]
— treating a project as an IT project
rather than as a business change project [3.8.5, 3.8.8-9, 4.2]
— ill-defined, unrealistic or conflicting
objectives [2.1, 3.1.18, 3.7.33, 3.8.29]
— suppression or mismanagement of risks
and uncertainties [2.8, 3.1.21, 5.3.7]
— ineffective project or resource planning
[2.2, 3.1.10-12, 3.1.14, 3.4.6, 3.7.8]
— planning for "big-bang"
instead of evolutionary delivery and failure to plan for longer-term
evolution [2.2, 3.8.13-14, 3.8.28]
— fixing objectives, timeframes or
costs when excessive uncertainty still exists [1.2.5, 3.1.11,
3.7.28]
— ignoring costs of business change
[2.2, 3.8.1, 3.8.8]
— insufficient political support [2.3,
3.4.18-19]
— over-dependency on, and/or failure
to control, suppliers [3.1.14, 3.1.26, 3.3.10-13, 3.3.20, 3.7.27]
— excessive project size and complexity
[3.1.12, 3.2.11, 3.3.17, 3.7.32, 3.8.15]
— unproven technology or approach,
especially exceeding the limits of proven performance [3.1.25,
3.1.30]
— inadequate provision for data transfer
and quality [2.6, 3.3.14, 3.8.9]
— lack of provision for information
governance [3.8.26, 3.8.47-48]
— failure to identify or achieve relevant
standards [2.6, 3.7.25, 3.8.26]
— failure to plan for necessary levels
of safety, security or recovery [2.8, 3.3.19, 3.5.1-47, 3.6.1-23]
— lack of project or technical skills
[1.6.4, 2.2, 3.1.14, 3.7.32, 3.8.21]
— undefined exit conditions [1.2.5]
— inflexibility, inappropriate aggression
and machismo [3.3.13, 3.5.34-35, 3.8.34]
— loss of political support [1.6.5,
3.4.11, 3.8.33]
— resources not properly allocated
[3.3.6-7, 3.8.44]
— failure to create effective partnership
with contractors [3.2.2, 3.2.9, 3.8.22, 3.8.46]
— unreliable progress reporting [3.1.14,
3.3.9, 3.8.16]
— unrealistic timeframes or budgets
[1.2.5, 3.1.11, 3.7.30, 3.8.21, 3.8.40]
— inattention to quality [3.1.14, 3.3.14]
— barriers to communication [3.8.17,
3.8.31]
— fear of failure [3.6.8, 3.8.14]
14. We find it quite remarkable, and extremely
worrying, that our Dossier shows that all of the above
lengthy list of generic system problems would appear to exist
in NPfIT.
15. Our Dossier also provides extensive
evidence of a number of further problems that are specific to,
or particularly challenging in, NPfIT including:
— inadequate clinical engagement during
system requirements analysis and specification [2.3-4, 3.4.3,
3.4.9, 3.4.16-17, 3.8.22, 5.3.9]
— replacement of existing successful
well-trusted systems by standard systems that are perceived to
be inferior [3.6.8, 3.8.9, 3.8.39]
— employing Patient Administration
Systems that were designed for the very different US healthcare
market [3.8.10]
— inadequate response to the medical
profession's concerns regarding issues of patient confidentiality
[3.4.5, 3.5.1, 3.5.11, 3.5.30, 3.5.34]
— excessive reliance on IT consultants
and suppliers with little knowledge of UK healthcare and the NHS
[2.4, 3.8.7]
— implementing a complex centralised
system in a situation in which the NHS is constantly faced by
changes in organisation, medical practice and even the law [2.6,
3.8.23, 9.3]
— identifying and managing the changes
in NHS working practices needed to complement, shape, and exploit
proposed new IT facilities [3.8.5, 3.8.8, 3.8.9]
16. The above exercise of identifying relevant
references in a large collection of evidence, gathered mainly
from the public (albeit mainly specialised) media, is a far from
satisfactory way of assessing the real extent and seriousness
of the issues which should be addressed if NPfIT is ever to succeed.
We argue however that our analysis illustrates very dramatically
the number, variety and complexity of the concerns surrounding
NPfIT, and thus provides a compelling argument for commissioning
a detailed review of the project, carried out by evidently-independent
experts with full access to all relevant information and personnel.
17. It will, we believe, take such a review
to pinpoint the most important causes of the present delayed delivery
of the Programme, in particular those aspects related to EHRs.
More importantly, such a review is needed to determine whether
there are, as we suspect, strong reasons to assume that the Programme
will actually fail, not just continue to over-run its schedule
and budget, unless appropriate remedial action is identified and
undertaken urgently.
CONCLUDING REMARKS
18. A well-known aphorism in the IT industry
is: "A complex system that works has evolved from a simple
system that worked" [Gall, 1975]. Another hard-won insight
is that the most successful highly-critical large IT systems,
such as the worldwide VISA payments system [Stearns 2006], achieved
their success through ruthless control of their complexity, and
minimisation of the level of dependence that needed to be placed
on them, as well as through high levels of hardware/software reliability.
A further crucial insight concerns the criticality of employing
evolutionary procurement methods, and socio-technical expertise,
in order to determine precise specifications that meet the various
stakeholders' requirements [2.2]. A detailed constructive review
of NPfIT in the light of these insights could, we argue, greatly
increase the likelihood of the project's eventual success.
19. The NHS has many good working systems,
and NPfIT is planning and delivering others, but our and others'
research and experience suggests that NPfIT's problems, especially
those centred on EHRs, are daunting; our expert opinion therefore
is that there is a high (but probably avoidable) risk that the
Programme will fail. Hence our recommendation that an open independent
technical review is an essential first step toward managing that
risk in a professional manner. (In making this recommendation
we are not seeking to review NPfIT ourselves—being entirely
independent of NPfIT, we are simply acting out of strong professional
concern and in the public interest.)
REFERENCES1. Beynon-Davies,
P. Information management in the British National Health Service.
Int J Information Management (1994;14): pp 84-94.
2. Brennan, S. The NHS IT Project. Radcliffe
Publishing Ltd (2005).
3. Curtis, B, et al. A field study of the software
design process for large systems. Communications of the ACM
31,11 (Nov 1988) pp 1268-1287.
4. Drummond, P.H. Escalation in Decision-Making:
The Tragedy of Taurus. Oxford University Press (1996).
5. Eason, K. Local socio-technical system development
in the NHS National Programme for Information Technology (To appear
in the J. Information Technology).
6. Gall, J. Systemantics: how systems work
and especially how they fail. New York Times Book Co. 1975).
7. Javitt, J. C. How to succeed in health information
technology. Health Affairs (25 May 2004) pp 321-324.
8. Jones, C. Patterns of large software systems:
failure and success. IEEE Computer (March 1995) pp 86-87.
9. Lyytinen, K., Hirschheim R. Information systems
failures. Oxford Surveys in Information Technology (1987;4)
pp 257-309.
10. Standish Group, The CHAOS Report (2002).
11. Stearns, D. In plastic we trust: dependability
and the Visa payment system. DIRC Conference, Newcastle
(April 2006). http://www.sociology.ed.ac.uk/finance/Papers/StearnsDIRC06.pdf
12. Taylor, A. IT projects sink or swim. BCS
Annual Review (2001) pp 61-64.
13. US National Institute of Standards and Technology.
Advanced technology program on component-based software
(2007). http://www.atp.nist.gov/focus/cbs.htm.
14. Wyatt, J. C. Hospital information management.
BMJ (1995;311) pp 175-178.
SUBMISSION FROM
Ross Anderson
Professor of Security Engineering
Cambridge University
James Backhouse
Director, Information System Integrity Group
London School of Economics
David Bustard
Professor and Head of Computing and Information Engineering
University of Ulster
Ewart Carson
Professor of Systems Science
Centre for Health Informatics
City University
Patrik O'Brian Holt
Professor
School of Computing
The Robert Gordon University
Roland Ibbett
Professor
School of Informatics
University of Edinburgh
Ray Ison
Professor of Systems
The Open University
Achim Jung
Professor
School of Computer Science
University of Birmingham
Frank Land
Emeritus Professor
Information Systems Group
Department of Management
London School of Economics
Bev Littlewood
Professor of Software Engineering
City University
John A McDermid
Professor of Software Engineering
University of York
Julian Newman
Professor of Computing
Glasgow Caledonian University
| Brian Randell
Emeritus Professor of Computing Science
School of Computing Science
Newcastle University
Uday Reddy
Professor
School of Computer Science
University of Birmingham
Peter Ryan
Professor of Computing Science
Newcastle University
Geoffrey Sampson
Professor
Department of Informatics
University of Sussex
Martin Shepperd
Professor of Software Technologies
Brunel University
Michael Smith
Visiting Professor
Department of Computer Science
University College London
Tony Solomonides
Reader in Computer Science and Medical Informatics
University of the West of England
Ian Sommerville
Professor
Computing Department
Lancaster University
Harold Thimbleby
Professor of Computer Science
Swansea University
Martyn Thomas
Visiting Professor of Software Engineering
Computing Laboratory
Oxford University
Colin Tully
Professor Emeritus of Software Practice
School of Computing Science
Middlesex University
|
| |
|
