APPENDIX 4
MEMORANDUM SUBMITTED BY COMPUTER WEEKLY
(PAC 1999-2000/158)
We write to the Public Accounts Committee in relation
to its consideration of the Chinook's FADEC computerised engine
control system.
Computer Weekly has made a particular study
of the Chinook's FADEC and aviation accidents in which computers
or the man-machine interface have been a suspected factor.
In the case of the Chinook's FADEC there has
never, to Computer Weekly's knowledge, been a procurement or an
implementation of safety-critical software that has had such a
dense history of significant problems.
Therefore the issue of whether the Chinook's
FADEC was poorly procured, whether it was ready for operational
use when it went into service and whether flaws in the system
could have caused the RAF's worst peacetime accident, are issues
that give the Public Accounts Committee a unique opportunity to
examine some of the wider matters that are of great significance.
The key question, in our view, is whether the
pilots of Chinook ZD576 that crashed on the Mull were at fault,
or whether, given the FADEC's history of causing engine surges,
spurious cockpit warnings and engine run-downs, sometimes without
leaving physical evidence of any software problems, the FADEC's
software could have caused the accident on the Mull of Kintyre
by again malfunctioning without leaving any physical evidence.
The far wider issues that have been raised by
the National Audit Office's research into the Chinook's FADEC,
and have not so far been considered by Parliament include:
— the feasibility of independent testing
of safety critical and mission critical software;
— what checks exist if any to stop
departments sidelining any independent advice that goes against
the grain;
— the conflicts of interest that arise
when only manufacturers understand their software sufficiently
to identify any of its flaws after a major incident;
— the accountability of civil servants
to Parliament after a major incident;
— the difficulties of establishing
physical evidence of software problems after a major incident;
and
— whether, in the light of a major
incident, a department will seek to protect a supplier from criticism
rather than allow some of the opprobrium to spill onto the department's
lap.
These are not problems that are confined to
aviation. Software now controls everything from City transportation
systems to missiles, command and control systems, critical telecommunications
equipment and the systems on which the UK's reputation as a major
financial centre depend.
This last point is topical at the present point
in time. It is currently more than a week after a computer failure
brought down the London Stock Exchange for nearly an entire day,
and the bug that brought down the system has not been identified
because it has not proved possible as yet to replicate the exact
problem. The inability to identify the bug is despite the forensic
skills of some of the most expert software specialists in the
USA and the UK.
Having studied all the available evidence and
more—including hundreds of pages of documents that have
never been published by the Ministry of Defence—we believe
that we can show that there is insufficient evidence to blame
the pilots, and good grounds for believing that the software may
have played a critical role.
The evidence we have studied supports the following
conclusions (if the committee so requests, we will supply documentary
evidence for any of the following conclusions):
— the FADEC was procured (without open
competition) without sufficient controls by the prime contractor
(Textron Lycoming) on the subcontractors;
— the RAF was kept at a distance from
the development process rather than working in tandem, which is
usually necessary in major software-intensive projects. Studies
into the lessons from IT problem projects, such as the Stock Exchange's
Taurus project and the "Croeso" system undertaken by
South Western Electricity and its neighbouring utility South Wales
Electricity underline the desirability of joint working to ensure
that the final product fulfils the customer's objectives and also
because the developers can be held accountable for the quality
of their work by an independent organisation (the customer) on
an ongoing basis;
— also for reasons of accountability,
and in the light of the failure of the Post Office/Benefit Agency
"Pathway" project, HM Treasury have stressed to the
DTI select committee the need for major software developments
to be financed by an organisation that is entirely independent
of the developers. In the case of the Chinook's FADEC, the project
was financed initially by the subcontractors who were also the
developers;
— the original proposal for a Chinook
FADEC contained an undertaking that the project would be low-risk.
This low-risk concept was not carried through to implementation,
however. A change was made. Instead of a mechanical backup the
FADEC's main (primary) and backup (reversionary) lanes were controlled
by software. So pilots were to have no direct mechanical control
of engine acceleration or deceleration. They had to delegate "full
authority" to the software. Two years ago, however, Textron
Lycoming announced a joint project to develop a successor to FADEC
that "unlike FADEC . . . will feature in independent mechanical
backup subsystems for all critical control functions";
— the UK version of the FADEC software
design was inadequately tested and flawed, as evidenced by the
software's ongoing (flight critical) problems and the number of
changes the software underwent after it was accepted by the MoD
and the RAF;
— the system was brought into service
to meet operational needs (the number of Chinooks available for
operations in 1994 being at a record low), despite the fact that
the MoD was at the time taking legal action against the FADEC
suppliers over claims that the FADEC software had not been designed
to international military or civil avionics standards;
— the RAF and the MoD did not take
the advice of its airworthiness assessors at Boscombe Down that
the software should be re-written. This was because of the disruption
this would have caused to the timetable for the Mid-Life Update,
and also because there was a resistance to giving in to Boscombe
Down whose concerns over the FADEC were perceived at the highest
level in the RAF as being exaggerated (memo reference ADD/308/04
dated 6 June 1994);
— the concerns about FADEC that were
expressed in 1993 by an independent contractor EDS-Scicon, which
was commissioned to analyse the software, were not addressed by
the time the FADEC came into operational service. This was partly
because the MoD was assured in a Textron Lycoming "White
Paper" that the concerns of EDS-Scicon and Boscombe Down
were misplaced. However, the trust that the MoD placed on Textron's
assurances about FADEC were in contradiction to the MoD's distrust
of Textron's assurances that the Ministry expressed in its arbitration
proceedings against Textron, proceedings which were at that time
secret.
The above points suggest strongly that the FADEC
was implemented against the conventions of best practice. The
Committee may therefore wish to consider whether the procurement
of the FADEC and the poor relationship in 1993 and 1994 between
the MoD and its appointed independent arbiter of aircraft software
quality at Boscombe Down is a matter of some concern.
It may be asked how it is possible for a government
department to procure, accept and implement a safety-critical
software product that has not been benchmarked and approved by
qualified arbiters, to the satisfaction of those expert assessors,
using methodologies and tools laid down in the MoD's main standard
00-55 which sets out procedures for designing, verifying and validating
software in safety-related applications?
Another question the Committee could ask is:
how can the MoD persist with its argument that the software was
ready to be put into operational use when the software's performance
and reliability was questioned by its own assessors and by independent
private contractors?
So far, the MoD response has been to say that
Boscombe Down was using an inappropriate methodology to validate
the software. The Ministry's Permanent Under Secretary of State,
Mr Kevin Tebbit, at the Public Accounts Committee's hearing on
8 March 2000 suggested that Boscombe Down's chosen methodology,
static code analysis, was more appropriate for the nuclear industry.
"I would not like to comment on Static Code
Analysis procedure on the general side. I do know that it was
used in the nuclear power industry and was applied in this context".
However, Martyn Thomas, of the UK's most respected
independent safety-critical software specialists, has written
to Computer Weekly pointing out that a branch of the MoD helped
to develop static code analysis. Indeed it has emerged that static
code analysis was, in 1994, and is today, a recommended methodology
in 00-55, the MoD's benchmark standard for the design, verification
and validation of safety-related software.
Thomas says in his letter that it was the Royal
Signals and Radar Establishment, now part of the Defence Evaluation
Research Agency, an agency of the Ministry of Defence, that helped
to develop static code analysis.
"The RSRE developed the secret technology
so that they could verify security-critical software. Work on
static analysis was declassified as a matter of public policy,
precisely so that it could be used on safety-critical software,
such as the Chinook FADEC".
Other letters to Computer Weekly say that the
Lockheed C130J Hercules aircraft is undergoing static code analysis
for the purposes of UK flight certification; and separately, Bath-based
Praxis Critical Systems, which develops software for the defence,
banking and other industries, says that static code analysis has
been used to validate safety-critical software in aircraft such
as the Tornado F3 and the Eurofighter.
It is also used in safety-critical aircraft
support functions, such as the Sholis system that helps helicopters
to land safely on ships. In addition, it has been used by the
Government's communications centre GCHQ to spot viruses in software.
Praxis said the importance of static code analysis,
which involves testing code without executing it, lies in its
ability to highlight anomalies and faults that could remain undetected
by dynamic testing.
It said dynamic testing, which involves executing
the code, can highlight only a small number of potential problems.
This is because it does not check paths through the software that
can be taken by executable code.
What all this shows is that Boscombe Down's
preferred methodology, that has been much-denigrated by the MoD,
was and is the MoD's own preferred methodology.
The ministry's incorrect statements on static
code analysis may leave it open to the accusation that it has
misled the National Audit Office, and Parliament.
On the basis of its MoD briefings, the National
Audit Office (NAO) reported on the anomalies found by EDS-Scicon,
but added that the contractor had used static code analysis, which
it said was an "internal Boscombe Down policy, not supported
by defence standards".
In 1998, MPs on the Commons' Defence Committee
were told by the Ministry of Defence that, "static code analysis
is . . . a requirement placed by British Nuclear Fuels on the
safety of a nuclear system".
In July last year, a senior civil servant at
the Secretariat (Air Staff) of the Ministry of Defence wrote in
a letter that, "Static code analysis does not validate the
performance of the software and the department therefore had no
requirement for it".
In August last year, the Ministry wrote to Defence
Committee MP Michael Hancock saying that "Boscombe Down's
preferred method of examination is static code analysis, a system
of verification not widely in use but employed in the nuclear
industry."
Also last year, the House of Lords was told
by the Ministry that, "Boscombe Down indicated a wish to
assess the design of the FADEC software using static code analysis—a
methodology used by the nuclear industry".
None of this gives a true impression of the
importance to the MoD of static code analysis. It should be pointed
out that static code analysis is not the only method that should
be used to validate software. Specialists who have written to
us say that a combination of static and dynamic will help to spot
flaws and potential causes of failure.
So, if Boscombe Down was dissatisfied with the
software having used the Ministry's preferred methodology to test
the code, why is the MoD persisting in its claims that the FADEC
was not flawed?
Nobody is certain why, but it is evident that
the issues surrounding the crash on the Mull and the FADEC have
become mired in half-truths, doublespeak, and falsehoods. There
are a number of examples of these, which the Public Accounts Committee
has encountered first hand (see later comments relating to the
MoD evidence to the committee).
If the MoD cannot argue rationally and logically
on why their actions were justifiable in putting the FADEC into
service despite expert reservations, can it argue with credibility
that the pilots of Chinook ZD576 were to blame for the accident
on the Mull of Kintyre?
The FADEC is an unusually complex piece of equipment,
with nearly two million lines of software code. Indeed we note
that when the Public Accounts Committee asked Mr Tebbit, Sir Robert
Walmsley, Chief of Defence Procurement, or Vice Admiral Sir Jeremy
Blackham, Deputy Chief of Defence Staff (Equipment Capability),
about how FADEC can cause engines surges, none knew sufficient
about the way the FADEC impacts on the helicopter's flying capabilities
to give the Committee an answer.
It took Computer Weekly researchers many
months of studying the design documents, and dozens of conversations
with pilots and technicians, to understand how FADEC interacts
with the controls on a Chinook, what can happen when the system
malfunctions, how pilots should react, and what faults would not
leave any trace.
We were able to conclude that no evidence of
technical malfunction does not mean no technical malfunction.
If, however, the MoD's contention that no evidence
of technical malfunction is analogous to no technical malfunction,
this sets a dangerous precedent.
The "Rand" report for the National
Transportation Safety Board which investigates aviation accidents
in the United States points to the fact that accidents caused
by software may not be traceable to software because it tends
to leave no physical trace of its behaviour in the wreckage.
Indeed it was a conclusion of the RAF Board
of Inquiry into the crash on the Mull of Kintyre that: "an
unforeseen technical malfunction of the type being experienced
on the Chinook HC2, which would not necessarily have left an physical
evidence, remained a possibility and could not be discounted".
This issue, of manufacturers being held accountable
for the software they produce, is of concern to Computer Weekly
readers. If the MoD's view of the crash on the Mull of Kintyre
is accepted—that no evidence of technical malfunction means
that there was no technical malfunction—this in our view
provides a cushion of comfort for manufacturers who supply poor
quality software.
It means that if software causes a fatal accident
or fails in a mission-critical system, and the fault cannot be
traced afterwards because software has left no physical evidence
of its behaviour, then the manufacturers cannot be blamed. This
would leave computer users, Parliament and safety regulators unable
to hold software manufacturers to account if their products caused
critical systems to fail.
In the case of the crash on the Mull of Kintyre
the pilots were found posthumously to have been grossly negligent
in the absence of any concrete evidence of technical malfunction.
If this verdict is accepted, irrespective of whether this represents
an injustice or not to the families of the pilots, we believe
this would send the wrong signals to manufacturers.
Acceptance of the verdict would also, in effect,
condone what we believe is MoD doublespeak. The MoD says it will
examine with compassion any new evidence but it knows no evidence
of software problems can be produced. Therefore the Ministry is
relying on its detractors to produce evidence that it knows cannot
physically be produced. Is this not evidence of the MoD's doublespeak?
This brings us to another of the wider issues.
In its anxiety to defend the FADEC, and therefore the reputation
and integrity of the MoD and RAF, statements have been made to
Parliament that have been incorrect and/or misleading, sometimes
patently so. This may raise questions that go beyond the Chinook
and FADEC, and touch on matters related to accountability of the
department to Parliament. To avoid making this letter too long
we have not always given examples but can do so if requested.
The Ministry has:
— withheld relevant information and,
when this information has leaked out, has made incorrect statements
about that information. For example, as attention focused on the
FADEC in the light of the crash on the Mull of Kintyre, the MoD
and the RAF made no mention of its litigation against Textron
Lycoming. Therefore the families of the dead pilots or passengers
were unaware that FADEC was capable of causing a Chinook to crash,
and indeed had caused a serious accident in 1989, after which
the Ministry had issued a writ against the FADEC supplier. When
evidence of the arbitration proceedings leaked out in 1997, the
MoD issued incorrect and contradictory statements about the matter.
It made statements, for example, saying that the RAF Board of
Inquiry and the Fatal Accident Inquiry was aware of the litigation.
Then it conceded that neither inquiry was made aware was the litigation.
Then it issued statements that the arbitration proceedings arose
from faulty testing and had "nothing to do with the software".
In fact the opening page of the government's writ against Textron
was that the 1989 accident was "caused by respondent Textron's
faulty design of a computerised engine fuel control device, FADEC".
Indeed the legal papers denied that the accident was due to faulty
testing. The MoD's case was that "Boeing's test procedures
were reasonable and adequate";
— continued to issue incorrect statements
even after these have been shown to be incorrect. For example
the Ministry last year apologised for stating incorrectly to MPs
on the Defence Committee that the FADEC was not safety-critical.
However the Ministry last month repeated the original incorrect
statement to the Public Accounts Committee;
— used selective quotations from the
report of accident investigators. The effect of this has been
to give an impression of certainty when investigators, in sentences
immediately before or after the one selected by the MoD, have
expressed uncertainty or a caveat. For example the MoD, in several
letters to MPs and in one to the British Airline Pilots Association
sought to show that the FADEC's main computer component (called
the DECU—Digital Electronic Control Unit) was not at fault
in the accident on the Mull. All of these letters quoted one particular
part of the accident report which said: "Strip examination
of the engines . . . revealed no signs of pre-impact failure or
malfunction that could have affected the operation of either engine".
However none of the letters quoted the very next sentence in the
accident report which said: "Fire damage prevented assessment
of the functionality of the No 1 (engine's) DECU and had destroyed
its memories of the operating program and exceedance fault history";
— omitted facts or parts of official
reports that have not been consistent with the MoD's stated position
that FADEC has never caused a Chinook accident, could not cause
a Chinook accident, and has never, in its production versions,
had any serious flaws. For example the MoD did not mention, in
its dozens of letters to MPs and in Parliamentary Answers that
one conclusion of the RAF Board of Inquiry was that: ". .
. an unforeseen technical malfunction of the type being experienced
on the Chinook HC2, which would not necessarily have left any
physical evidence, remained a possibility and could not be discounted".
— Mixed undisputed facts with disputed
MoD speculation, without making the distinction clear, in such
a way as to convey certainty when none exists. It has also made
statements without context that have had the effect of giving
an incorrect impression. For example, after the fatal crash of
a US Army Chinook equipped with FADEC in 1996, the deceased pilots
were blamed because no fault was found of a technical malfunction.
Later it was found that the original verdict was wrong and that,
contrary to the initial investigation report which found no evidence
of any electrical problems, there had in fact been an electrical
failure. When the MoD was asked about possible electrical problems
on the aircraft that crashed on the Mull, the MoD replied that
the findings on this were "unequivocal" and it quoted
part of a sentence in the investigator's report which said that
a "major pre-impact loss of electrical suppliers had not
occurred". The MoD omitted to mention the first part of the
sentence which said "While none of the direct indications
of electrical system behaviour was conclusive . . ."
To cite another example, the US Army issued a warning to its Chinook
community in 1999 that hydraulic contamination was a suspected
cause of sudden, unexpected and potentially fatal manoeuvres of
the aircraft. When it was put to the MoD that investigators of
the crash on the Mull had found a "considerable quantity"
of particles in hydraulic fluid and had concluded that there was
"pre-impact hydraulic system contamination", the MoD
quoted from other parts of the accident report in which hydraulic
components had been found without abnormalities. The Ministry
also, in a letter to the MP Michael Hancock, dated August 1999,
said that hydraulic contamination had been "ruled out"
as a result of an investigation by the Air Accidents Investigation
Branch. The letter added that the investigators found hydraulic
contamination that was "consistent only with what would have
been expected as a result of normal wear and tear". The MoD
omitted to quote the reference in the investigation report to
"pre-impact hydraulic system contamination and "high"
wear rates. Also in its letter, the MoD omitted to mention that
the accident investigators, among their final conclusion in the
accident report, had remarked that there were "possible utility
hydraulic system abnormalities".
— criticised all specialists who have
not agreed with the MoD over the FADEC. The criticism has extended
to Malcolm Perks, the MoD's own expert witness in its arbitration
proceedings against Textron Lycoming, Malcolm Perks, who has expressed
the view that FADEC could have caused the crash on the Mull. The
MoD has also questioned the professionalism of Squadron Leader
Robert Burke, a pilot with more test hours on Chinooks than any
other at the time who expressed the view that at an engine surge
could have contributed to the crash on the Mull. The Ministry
has also criticised the A&AEE at Boscombe Down for using inappropriate
methods to test the FADEC, when in fact those methods were those
recommended by the MoD. In addition the Ministry belittled the
evidence of its independent contractor EDS-Scicon by saying that
it raised issues on FADEC that were related mainly to documentation.
In fact EDS-Scicon had raised concerns about the safety of the
software. The Ministry has also criticised the media, sometimes
by denigrating statements that the media did not make;
— issued a letter warning a respected
aviation magazine that if it went ahead and published an article
that was critical of the decision to blame the pilots for the
crash on the Mull of Kintyre, it may face an action for defamation;
— defended FADEC by quoting the assurances
of the Design Authority (Textron Lycoming) without drawing attention
to the fact that the Ministry had undermined those same assurances
during the arbitration proceedings, which the Ministry won.
But why, since the crash on the Mull, has the
MoD made so many misleading statements (many more than have been
mentioned above) with the effect, apparently, of turning attention
away from the FADEC and onto the pilots?
One possible explanation is that the Ministry
does not want to give any ground to those who are critical of
its decision to put the Chinook into operational service before
the problems with the FADEC's software had been eliminated.
It could also be said that the MoD does not
wish to lend weight to the concerns shared by the families of
the dead pilots and the families of the dead VIP passengers on
that last flight of Chinook ZD576, that FADEC may have been involved
in the accident.
We are not certain, however, that these are
the reasons for the MoD's position.
What we have seen is that, the more information
that comes to light, from the US Army and elsewhere, which throws
doubt on the original decision to blame the pilots, the more entrenched
has become the MoD's defence of FADEC's implementation on the
Chinook.
We cannot help but conclude that the reason
for the MoD's position is that it has always in public defended
the decision to blame the pilots and that it must, for the sake
of consistency, continue to defend this decision, whatever new
evidence or information arises.
It appears to us that the matter of whether
the FADEC was fundamentally flawed, or whether it caused the crash
on the Mull, is in some ways subordinate to the need to sustain
departmental price and not admit that a mistake may have been
made.
Indeed there appears to be, within the MoD,
an institutional machismo that will admit no imposition on its
decision making by those whom it regards as outsiders: particularly
politicians and the media.
If this attitude continues to prevail—and
we suspect it will—then no matter what information comes
to light, or whatever political pressure is applied, the Ministry
and particularly air marshals in the RAF will not allow the verdict
against the pilots to be stood down.
Therefore we believe that the matter needs to
be investigated by a body that is independent of the MoD. For
if no action is taken to reinvestigate, or to question the MoD
statements in relation to the FADEC, this will leave unanswered
the question of whether the Ministry has inadvertently misled
Parliament or has done so systematically rather than countenance
the possibility that it made mistakes, firstly by rushing the
FADEC into service and then defending the decision to blame two
dead Special Forces pilots for a crash that has no identifiable
cause but which could have involved FADEC.
CONCLUSION
(a) It is not possible to look at the procurement
of the FADEC and say whether it was safe, procured properly and
in a timely fashion without looking at its performance after it
came into operational service. The greatest possible example of
its malfunctioning could have been the crash on the Mull of Kintyre.
The question of whether the FADEC worked properly could be related
directly to that accident.
(b) The MoD is failing its own legal defence
team in the arbitration hearing. The Ministry used hundreds of
pages of evidence to prosecute its case against the FADEC supplier
Textron Lycoming but failed to make any of this material available
to any of the inquiries into the performance of the FADEC including
the Public Accounts Committee. Once details of the MoD's successful
legal case against Textron were put into the public domain by
the media, the MoD attacked its own evidence as irrelevant and
peripheral to the performance of the FADEC.
(c) Ultimately there are two issues: Did
the FADEC pass all the tests set for it by the procurement team
which rightly included Boscombe Down and EDS-Scicon, or for a
host of operational reasons, was FADEC rushed into service, amid
a hasty dismissal of expert concerns, with possible tragic consequences?
If the Committee wishes to see any or all of
the documents on which this letter is based, we will be happy
to oblige.
Kark Schneldir
Tony Collins
Hooman Bassirian
Computer Weekly
18 April 2000
|
