Select Committee on Public Accounts Minutes of Evidence


SUPPLEMENTARY MEMORANDUM SUBMITTED BY THE MINISTRY OF DEFENCE (PAC 1999-2000/196)

SECTION 1—SUMMARY

Chinook Mk2 Mid Life Update

1.  The Chinook Mid Life update (MLU) Programme was reported on by the NAO as a Case Study in Box 5 of Part 3 of the Report. The purpose of the programme was to upgrade 32 Chinook HC Mk1 helicopters to the Mk2 standard. The conversion involved the replacement of the transmission, hydraulic and electrical systems and various structural modifications. The opportunity presented by the MLU allowed the fitting of converted modified Textron-Lycoming T-55 engines and of the Full Authority Digital Electronic Control (FADEC) system, developed under a separate programme.

FADEC

  2.  FADEC was developed to replace the hydro-dynamic fuel control system of the Mk1 Chinook. Reliability of this hydro-dynamic system was very poor and support costs were high as was the workload on pilots. The purpose of FADEC is to control the engines by regulation of the fuel flow to automatically meet the power demands of the aircraft rotor system, in maintaining a constant rotor speed in flight. The FADEC system fitted to each of the Chinook's two engines was designed under a fail safe philosophy with independent Primary and Reversionary channels, each with dissimilar software, hardware, and control algorithms. FADEC continually monitors its performance and if a fault cannot be managed by the levels of redundancy within the Primary system, it will automatically change to Reversionary at which point the aircrew receive an audible and visual warning. Should the Reversionary system also fail, the affected engine will still be supplied with a constant fuel supply, set at the level of the pre-failure rate. In the highly unlikely event of a total FADEC system failure, the aircraft can still fly safely within certain operational parameters on one engine. The probability of both Primary and Reversionary channels on both engines failing simultaneously is infinitesimal; and even if that were to happen the Chinook is designed to descend in a controlled manner.

  3.  FADEC started life in 1979 as a Technology Demonstrator Programme. By the time the development programme contract for Chinook FADEC was placed in 1985, the feasibility of design had been successfully demonstrated on the Rolls Royce military Gem engine. Under the Chinook development programme, successive risk reduction measures were carried out in the form of hazard analyses, computer simulation, engine bench testing and eventually flight testing, before the production contract was placed in 1991.

Wilmington Incident

  4.  An incident occurred in January 1989 at Boeing's flight test facility at Wilmington, Pennsylvania, USA on an HC Mk1 aircraft during development testing of FADEC. This test used pre-production software and was aimed at demonstrating FADEC's performance under simulated battle damage conditions. A software fault was incorrectly determined by the contractors as one that the design should have accommodated without hazarding the aircraft. The contractors, having missed the significance of the fault, elected to proceed with the test and consequently an engine and rotor overspeed incident occurred. The damage that occurred would have been minimised had adequate test procedures been in place. The claim against Boeing was settled. However, the claim against Textron-Lycoming proceeded to Arbitration. The fault was quickly identified and the software design changed to allow safe engine operation in the event of a complete loss of signals as required by the original contract. The whole FADEC system was then re-qualified (which included further testing in degraded conditions) before the aircraft entered service in January 1994.

In service experience

  5.  As reported to the Committee at the hearing, the FADEC system had accumulated some 119,000 hours in RAF service and demonstrated a Mean Time Between Failures (MTBF) of some 5,000 hours. By comparison the overall MTBF of the pre FADEC fuel control system on the Chinook Mk1 was in the region of 500-800 hours, as Dr John Reid, MP, then Minister of State for the Armed Forces, reported to the House of Commons Defence Committee2[38]. In the last three years of the pre FADEC system this deteriorated to an MTBF of 168 hours. There were far more engine control failures experienced on the Mk1 aircraft than have ever been experienced on the FADEC equipped Mk2 aircraft, including engine run ups, run downs and control freezes.

  6.  There were a number of nuisance faults associated with FADEC on its introduction into service, none of which were serious software faults. There have been no instances of a total FADEC system failure in flight, although minor sub-system faults have occurred and FADEC has safely accommodated these—exactly as it was designed to.

Mull of Kintyre Accident

  7.  The aircraft which crashed on the Mull of Kintyre, (ZD576) on 2 June 1994 was flown by an experienced crew. An RAF Board of Inquiry (BOI) was convened following the accident in accordance with standard RAF procedures, to examine the evidence and to determine its cause. The Board was assisted on the technical part of the investigation by, amongst others, civilian specialists from the Air Accidents Investigation Branch.

  8.  There has been much speculation about the possibility of a catastrophic engine malfunction as a result of a FADEC failure. This was one possibility considered by the Air Accident Investigation Branch (AAIB) in their report to the BOI. The AAIB said that "the engine change units and FADEC were examined in some detail". The report goes on to say that "Strip examination [of the engines] indicated that both were running at high speed with the turbines hot at the time of impact, and revealed no signs of pre-impact failure or malfunction that could have affected the operation of either engine". Furthermore, post accident evidence was consistent with both powerplants operating at similar, intermediate power levels. At the time of the accident, the aircraft was operating within the safe single engine performance envelope.

  9.  The accident investigation established that, at 1.75km (0.95 nautical miles, or about 20 seconds) before impact, the crew released their navigational computer from its fix on the Mull of Kintyre and set it to indicate the bearing and distance to the next way point at Corran. At that point the pilots therefore had a clear indication of how close they were to the Mull. In view of the deteriorating weather and the strict visibility requirements of visual flight rules, under which they were flying, they should by that time already have chosen an alternative course. Given that they had not done so they could and should immediately have either turned away from the Mull or slowed down and climbed to a safe altitude.

  10.  The re-setting of their navigational computer about 20 seconds before the crash is a telling indicator that at that time the pilots were not grappling with an in-flight emergency. Yet they were flying on, contrary to both instrument and visual flight rules. The evidence is that they were already too close to the cloud-covered mass of the Mull, travelling too low and too fast.

  11.  The Board conducted a very thorough investigation, and considered all the factors which may conceivably have had a bearing on the accident, including a major technical failure of the aircraft prior to impact. It was, however, the overall finding of the inquiry that in continuing to fly their aircraft in the way that they did, toward the high ground of the Mull of Kintyre, the pilots did not exercise the skill, care or judgement they were known to possess. As such, they were deemed negligent.


SECTION 2—MOD COMMENT ON THREE ITEMS SUBMITTED TO THE COMMITTEE BY COMPUTER WEEKLY UNDER COVER OF A LETTER DATED 3 APRIL 2000 (NOT REPORTED)

XXXXXX

SECTION 3—ANSWERS TO QUESTIONS ON CHINOOK PUT BY THE COMMITTEE AT THE HEARING ON 8 MARCH 2000

Question 14: Predicted improvement in safety to be offered by the Chinook Mk2 against the Mk1. Questions 102-103 and Questions 112-113: Faults and problems found with the FADEC software, highlighting those that are not related to safety, from October 1994 to October 1999 and listing the 23 confirmed FADEC faults in that period

1.  The reliability of the old hydro-dynamic fuel control system was very poor. The overall MTBF of the pre FADEC fuel control system on the Chinook Mk1 was in the region of 500-800 hours, as Dr John Reid, MP, then Minister of State for the Armed Forces reported to the House of Commons Defence Committee2[39]. In the last three years of the pre FADEC system this deteriorated to an MTBF of 168 hours. There were far more failures experienced on the Mk1 aircraft than have ever been experienced on the FADEC equipped Mk3 aircraft, including engine run ups, run downs and control freezes.

  2.  By comparison, the FADEC system had demonstrated some 5,000 hours MTBF. In April 1988 MoD noted that in the previous three years nearly 40 incidents had been caused by engine run ups, freezing or run downs of the then (pre FADEC) electro-mechanical engine control system, some resulting in rotor overspeeds and subsequent rotor head rejection for overhaul.

  3.  The purpose of FADEC is to control the engines by regulation of the fuel flow to automatically meet the power demands of the aircraft rotor system, in maintaining a constant rotor speed in flight. The FADEC system fitted to each of the Chinook's two engines was designed under a fail safe philosophy with independent Primary and Reversionary channels, each with dissimilar software, hardware, and control algorithms. FADEC continually monitors its performance and is designed to accommodate the loss of all sensor signals. If a fault cannot be managed by the levels of redundancy within the Primary system, it automatically changes to Reversionary at which point the aircrew receives an audible and visual warning. Should the Reversionary system also fail, the affected engine will still be supplied with a constant fuel supply, set at the pre-failure rate. Furthermore, in the highly unlikely event of a total FADEC system failure, the aircraft can still fly safely within certain operational parameters on one engine. At the time of the accident the Mull of Kintyre aircraft was within those parameters. The probability of both Primary and Reversionary channels on both engines failing simultaneously is infinitesimal; and even if that were to happen the Chinook is designed to descend in a controlled manner.

  4.  Since its introduction in November 1993, the Mk2 has experienced six run downs and one run up. All these incidents occurred on the ground. Most were caused by interruptions to electrical power during engine start up procedures or by switching into Reversionary mode, as part of a precautionary pre-flight check. The Mull of Kintyre aircraft crew was required to complete satisfactorily these pre-flight Reversionary checks prior to take off on its last flight. All incidents record "Fault Codes" within the Digital Electronic Control Unit (DECU) part of the FADEC system (the Fault Codes are hexadecimal codes that indicate system faults to support further diagnostics). There have been no instances of a total FADEC system failure in flight. Minor sub-system faults have occurred and FADEC has safely accommodated these—exactly as it was designed to.

  5.  MoD has never denied that there were a number of faults associated with FADEC on its introduction into service. None was a serious software fault.

  6.  During the period October 1994 and October 1999 there were 23 reported FADEC component faults dealt with under the FADEC warranty procedure. None of these faults, detailed below, was considered by MoD or the Design Authorities to represent flight safety critical concerns, although some had safety implications.

FADEC COMPONENT FAULTS DEALT WITH UNDER WARRANTY—OCTOBER 1994-OCTOBER 1999

Serial No Fault ReportedFault Found
Ser No: 2AHW50Fault Code F9 during engine bench testing Low resistance found on the alternator winding, within the Hydro Mechanical Assembly. System stayed in Primary mode.
Ser No: 2AHW27Fault Code F9 during engine bench testing Low resistance found on the alternator winding, within the Hydro Mechanical Assembly. System stayed in Primary mode.
Ser No: 2AHW54External fuel leak Hydro Mechanical Assembly—chipped "O" ring on the windmill bypass valve, assembly method revised to ensure no re-occurence. Build quality problem. Fuel leak could have represented fire risk.
Ser No: 2AHW16Fuel leak stepper motor drain Hydro Mechanical Assembly—repaired under warranty. Fuel leak could have represented fire risk.
Ser No: 2AHW27In Reversionary mode No 1 engine froze whilst aircraft on the ground during pre flight checks Stepper motor froze. Service Bulletin raised to inspect within next 500Hrs and replace as necessary. Primary Mode stepper motor remained fully functional.
Ser No: 2AHW09Engine failed start in Reversionary mode Hydro Mechanical Assembly adjusted and returned to service.
Ser No: 2AHW34High Power Turbine Inlet Temperature (PTIT) during engine Reversionary start with Fault Code F5 Hydro Mechanical Assembly fault caused by binding Reversionary stepper motor. Primary stepper motor remained fully functional.
Ser No: 2AHW07AAir Bleed Actuator remained open in Reversionary Mode Seal Assembly replaced on Bleed Band piston and returned to service.
Ser No: 2AHW84AIntermittent FADEC Fault Codes in flight Aircraft switched to Reversionary mode and recovered to RAF Odiham, Hydro Mechanical Assembly repaired and returned to service.
Ser No: 2AHW64Engine Failed to attain ground idle during start Hydro Mechanical Assembly repaired and returned to service. Fault occurred during engine start procedure.
Ser No: 2AHW31Fault Codes (B7, DA, F8 & 6) on shut down FADEC system operated as designed by switching into Reversionary mode. Hydro Mechanical Assembly repaired, adjusted and returned to service.
Ser No: 2AHW61AAir Bleed Actuator remained open in Reversionary mode Seal assembly replaced on Bleed Band piston and returned to service.
Ser No: 2AHW40AAir Bleed Actuator remained open in Reversionary mode Replaced seal assembly on Bleed Band piston and returned to service.
Ser No: 2AHW45AEngine ran down to zero during pre-flight Reversionary checks Hydro Mechanical Assembly stripped and flushed through to remove build debris, and returned to service. Contractor's manufacturing processes improved.
Ser No: F158Whilst moving Engine Condition Lever to flight, Fault Code A4 appeared DECU returned for repair and returned to service. Fault was unserviceable rotor speed sensor.
Ser No: F106No 1 engine would not start Fault Codes BC, B6 Engine Condition Lever position resolver problem. Repaired by contractor.
Ser No. F120Unable to clear D2 Fault Code DECU had pressure transducer fault, repaired and returned to service.
Ser No: F143Fault Code C9 DECU could not read engine compressor speed signal. Item repaired and returned to service. This fault occurred at Boeing prior to delivery.
Ser No: F163No 1 engine, unrecognised "C" Fault Code Returned to contractor, after fault confirmed on Engine Test Bench at RAF Odiham. Item repaired and returned to service.
Ser No: F114DECU failed functional check "LL, OO" Fault Codes Faults caused by component failure. Found as part of functional checks during scheduled Minor Maintenance task.
Ser No: F142On shut down, Fault Codes DA, DB, DD, D6, E4, F6, B2, B3, B5 & B7 displayed Circuit boards found burnt out, caused by shorting aircraft loom (classed as user inflicted damage) during aircraft servicing.
Ser No: F175Intermittent FADEC 1 engine fail caption Returned to contractor for repair and investigation, Intermittent +5 volt failure found in the Primary lane. Reason for engine fail caption illumination was due to Primary/Reversionary solenoid being de-activated, as the system reverted to Reversionary mode. FADEC system operating correctly.
Ser No: F105Fault code F5 DECU returned to contractor. Unserviceable Diode on Power Supply Unit circuit board. System correctly reverted to Reversionary Mode.


Question 34: How many complaints were made by flight staff about the way in which Chinook Mk2s were performing?

  1.  In written evidence given to the House of Commons Defence Committee Hearing against Question 174[40], it was explained that there were a number of ways available to aircrew to express any concerns that they may have about the safety of their aircraft in addition to the Incident Reporting system for them to report routine failures. At squadron and station level, commanders invariably fly the same aircraft type as their junior personnel and are therefore well aware of any difficulties that they may be experiencing. Within the station's flight safety organisation concerns can be raised at regular flight safety meetings for consideration by both commanders and specialist flight safety staff. In addition there are confidential reporting systems (CONDOR for aircrew and MURPHY for ground crew) where personnel may raise specific incidents or concerns confidentially with the Inspector of Flight Safety. There is no evidence that any concerns relating to the Chinook or FADEC were ever raised through these channels. This remains the case today.

Question 17

  1.  At Question 17, the Chairman read statements from a letter to the Committee from the NAO reporting preliminary comments of Mr Frederick Bullock, who was at the time of that letter the Managing Director of EDS Defence Ltd, formerly EDS-Scicon. Mr Bullock was not with the Company when it worked on the software. We understand that Mr Bullock was not afforded the opportunity to review and comment on the content of the NAO's letter and is concerned that the Committee is not misled. Mr Bullock has since advised the NAO that it was inaccurate to state that the Company stopped work because of safety implications. He explained that the presence of anomalies does not necessarily have safety implications, or affect the overall operation of the system.

  2.  We understand that the statement read at the hearing that "HSDE dismissed the concerns as being trivial" attributed to EDS-Scicon was not repeated in Mr Bullock's subsequent explanation of the Company's position.5[41]

  3.  Mr Bullock's earlier comments, which we understand have now been corrected, are also quoted against Question 88 and its contents may have influenced some of the Questions at 182-189. In addition to this correction, the Committee may wish to note a later EDS-Scicon report, of June 1994. EDS-Scicon were tasked to undertake a Feasibility Study into the Options for Verification of FADEC, and in summary, they were confident that whilst there was always a possibility that changes to the object code may exist, none were required to mitigate the Category 1 anomalies found by them from the earlier verification study in 1993.


Question 95

1.  Regarding Question 95, when MoD sought EDS-Scicon support in the FADEC Arbitration case, the Company declined. Amongst the reasons cited was that the validity of the SCA methodology used previously by EDS in their analysis of the FADEC software (which analysis had been unsuccessful) could be called into question, and EDS-Scicon considered that because SCA is not generally used by US Aerospace and defence companies, and was not specifically mandated by the specifications to which FADEC was developed, their evidence would be unhelpful to MoD in pursuing its Arbitration claim about the Wilmington incident during FADEC development.

Question 97

  Mr David Rendel, MP, asked why Boscombe Down started to use SCA. The following is offered as background.

  2.  Boscombe Down are cognisant of, but not constrained by, regulations or specifications when assessing equipment in order to provide advice to the airworthiness authority (see also the answer to Question 253). In the case of the FADEC for Chinook they accept that SCA was not mandated. Nevertheless, this was their preferred choice of analysis. The fact that the software was not amenable to SCA made it impossible for Boscombe Down to comment either way on the safety aspects of the software, other than to recommend a complete rewrite to make it compatible with SCA process. This would have allowed them to verify the software and provide further supporting evidence for their recommendations for the CA Release.

Questions 20-21: Comparison between the UK and US Chinook Fleets in terms of actual numbers and compatibility of software

Question 210: Clarification of the compatibility of software between the UK and US Chinook Fleets

1.  The US Army have 25 and the RAF have 34 in-service Chinooks fitted with FADEC. A further six Chinook Mk2As with FADEC will come into service with the RAF shortly. The US Army are about to increase their FADEC equipped fleet by installing it in 300 of their CH47D aircraft fleet as part of a major upgrade programme.

  2.  Upon its introduction into service in 1994, the RAF FADEC system was, according to the Design Authority, 99 per cent common with the US Army's FADEC software. The differences were associated with the US Army's engines which were slightly more powerful and as a result had different temperature and performance limits registered within the software. The control functionality is the same between the two systems.

  3.  In 1995 the RAF introduced some minor modifications with no safety significance which reduced commonality to approximately 95 per cent. With respect to Question 210, which sought clarification of the phrases "95 per cent the same" and "95 per cent compatible", no difference was implied, the systems now have 95 per cent commonality.

Question 31: How long Defence Standard 00-56 has been in place

Questions 202-208: Whether Static Code Analysis is supported by Defence Standard 00-55

    —  The chronology of Standards

    —  Is the statement in paragraph 3 (page 36) of the NAO report incorrect?

  1.  Defence Standard 00-55 supports the requirement for Static Code Analysis of flight safety critical software systems but was inappropriate in the context of FADEC as explained below.

  2.  Defence Standards 00-55 and 00-56 were issued as interim versions on 5 April 1991. They post date, and consequently did not apply, to the Chinook FADEC development contract which started in December 1985 and concluded in November 1990.

  3.  The interim versions of both Defence Standards were replaced in August 1997 with the final standards. MoD does not generally seek application of later Defence Standards or Specifications to equipment which is already developed, and which has met standards applying at the time of development. However, should a system fail to achieve its design goals or demonstrate unacceptable performance (including safety related aspects of performance), the retrospective application of later standards may be appropriate. This is in line with Civil Aviation Authority philosophy. In FADEC's case, no such reasons existed to apply these later Standards. Neither was SCA mandated by specification RTCA DO 178A, against which the FADEC software was developed.

  4.  Therefore, the statement made in the NAO report that "the requirement for SCA was an internal Boscombe Down policy not supported by Defence Standards" is correct in the context of the assessment of Chinook FADEC software, but in retrospect it would have been clearer if the Report had explained the rationale.

  5.  When the Mk2 flight trials were held in 1992-93, these Defence Standards existed as interim versions. However, because Boscombe Down are not constrained by any design regulations or specifications, their preferred method of verifying software, using SCA was pursued even though this requirement was neither mandated nor included in the development contract placed in 1985. The software structure subsequently proved to be unamenable to this form of analysis.

  6.  With respect to other relevant Defence Standards at the time when Boscombe Down were assessing FADEC, Defence Standard 00-17 was issued in October 1985. This provided advice on the Modular Approach to Software Construction, Operation and Testing (MASCOT). The Chinook FADEC development contract quoted RTCA DO 178A (an internationally accepted standard) because this was regarded as having a higher standing than a UK Defence Standard. RTCA DO 178A was also the standard in the FADEC development contract when the programme began as a Technology Demonstrator in the early 1980s. As such, when the Chinook FADEC development programme was contracted in 1985 much of the legacy software had already been based on the RTCA DO 178A standard.

  7.  Nevertheless, in view of the continuing concerns expressed by Boscombe Down over the software, the MoD Project Director commissioned a different form of static analysis, as part of the Block One software upgrade programme in 1995. This consisted of an analysis of the software using a technique called Sneak and Traceability Analysis (SATA). This analysis is used by major US Aerospace companies, by the Federal Aviation Authority (FAA), NASA and by the Department of Defence, and included a static audit of 100 per cent of the software source code against the design specification to ensure traceability, and a Sneak analysis to look for flow paths that would lead to anomalous behaviour. The results of this analysis identified some 125 anomalies, mainly associated with the documentation of the software, all of which were assessed by the Design Authority. None was found to affect safety. The re-documentation work was independently overseen. This subsequent analysis reinforced the decision to introduce the aircraft into Service, as does the system's performance demonstrated to date.

Question 33

  1.  There is no evidence to support a view that the FADEC warning lights were in the "on" condition, prior to impact. The Central Warning Panel was examined as part of the accident investigation, and whilst there was clear evidence that number 2 engine's FADEC warning captions were not illuminated, the evidence in respect to the number 1 engine was not so categorical. Nevertheless, physical evidence exists to corroborate that both engines were operating satisfactorily up to point of impact.

Question 182: Whether the Court documents used by the Arbitration case involving Textron-Lycoming claimed that FADEC software was flight safety critical

Questions 198-199: Whether MoD had stated in the Textron-Lycoming Arbitration case that "FADEC was truly critical in maintaining safe flight"

Questions 200: MoD evidence concerning this phrase to the House of Commons Defence Committee

1.  MoD's view is that the Chinoock FADEC system is not flight safety critical when judged by the standards to which MoD authorities work; namely that its failure "would" lead to catastrophe, as opposed to the US definition "could". This view is based upon the design of the system (which has a number of back up features as described in the answer to Question 14 above) and the design of the Chinook aircraft itself. The probability of both Primary and Reversionary channels on both engines failing simultaneously is infinitesimal; and even if that were to happen the aircraft is designed to descend in a controlled manner.

  2.  Textron-Lycoming, designed and qualified FADEC as a flight critical system. Boeing also regard it as such, although they assess that the risk of catastrophic failure is mitigated by design—a view shared by MoD.

  3.  However, the Arbitration was not about flight safety criticality. It arose out of faulty test procedures in the early days of the system's development programme. Mr Perks, who was contracted by MoD's US lawyers under a duty of confidentiality to provide them with expert assistance in the preparation and presentation of the FADEC Arbitration, against Textron-Lycoming, made statements to the effect that the system was flight safety critical. These statements were Mr Perks' opinion, offered in a US Arbitration Hearing against a Company which had designed and qualified FADEC in line with their definition of a flight safety critical system. It is, therefore, not surprising that this opinion was used against that Company, argued in front of an American tribunal, reflecting US definitions and Textron-Lycoming's own view.

  4.  The phrase "The software was truly critical in maintaining safe flight" was used by Mr Perks in the context described above.

  5.  The memorandum submitted to the House of Commons Defence Committee on 24 April 1998[42] which responded to the Committee's questions about the Mull of Kintyre accident reported, amongst other things, the opinion of Boeing on the flight safety criticality of the FADEC system. The advice provided then was that Boeing regarded a failure of the FADEC as a level B risk, ie not catastrophic and therefore not flight safety critical.

  6.  That was the most up-to-date information available at the time the evidence was submitted. But we learnt subsequently that Boeing did regard FADEC as flight safety critical, albeit that the hazards were adequately controlled by design (see above). This evidence was corrected by a letter dated 21 April 1999 from Mr John Spellar, MP, then Parliamentary Under Secretary of State for Defence, to the Chairman of House of Commons Defence Committee, who went on to advise that "Our Judgement remains that FADEC is not flight safety critical by the standards to which MoD authorities work, namely that failure "would" lead to catastrophe, as opposed to the US definition "could". It also remains the position that the engines, including the FADEC system were carefully examined by the crash investigators. All the evidence indicated that the engines were working normally up to the point of impact and that FADEC could not have been a factor in the accident".

Questions 193 and 195: If the word "unacceptable" was used by Boscombe Down in the context of FADEC Software

1.  Boscombe Down did not use the word "unacceptable" in their CA Release Recommendations which is their formal airworthiness assessment report to the Project Director. However, in working level correspondence between officials the word was used. This in turn ensured that all perceived safety concerns were adequately investigated and judgements firmly based. The extract from an MoD document supplied to the Committee by Computer Weekly uses that phrase in that context.

  2.  Boscombe Down had continuing concerns about FADEC because they had been unable to verify the software independently. However, Boscombe Down provided specific instructions should it be deemed necessary to operate the aircraft. These instructions were followed.

Question 229: The side effect of controlling an aircraft which has an engine overspeed

  1.  In an engine overspeed whilst airborne the rotor speed is contained by coarsening the pitch of the blades, and fuel flow to the engine is then reduced through use of the Engine Control Lever. During this process the aircraft may climb. If the rotor speed cannot be contained the engine is shut down. In either event the aircrew would seek to land the aircraft at the nearest appropriate point. The aircraft can still be flown safely on one engine within certain operational parameters. At the time of the accident, the Mull of Kintyre aircraft was within those parameters.

  2.  An overspeed can stress the aircraft and very rapidly cause damage to the rotor system as a result of the huge physical forces involved. For instance, the rotor head tie bars could physically increase in length. Rotor overspeeds were not an unusual occurrence on the pre FADEC Chinook Mk1 and often this lead to rejection of the rotor head for overhaul as components were damaged beyond limits by the forces generated. As part of the BOI and AAIB investigation into the Mull of Kintyre accident, the rotor system was carefully examined and no signs of pre impact damage were found.

Question 233: Can a software system failure lead to mechanical fault?

  1.  Yes, but in the hypothetical event that a software fault had caused a mechanical fault, evidence of such a consequential fault would be available from a physical inspection of the mechanical parts concerned. For example, in the hypothetical case that an engine overspeed had occurred, the rotor system would show signs of distress, such as distortion of mechanical parts as a result of the increasing forces imparted into the rotor system.

Questions 240-241: Whether RAF aircraft would have been fitted with black boxes in 1994 at the time of the Chinook crash

  1.  The Chinook was first introduced into RAF service in 1980 when it was not MoD's policy to install Flight Data Recorders into military aircraft. Although there have been a number of accidents to aircraft fitted with Flight Data Recorders where the cause has not been positively determined, MoD's policy is now to install them on all new aircraft which is consistent with CAA policy for civil aircraft. At the time of the Chinook crash in 1994, however, Flight Data Recorders were only installed in the newer types of aircraft such as Tornado F3.

  2.  The decision on retrofitting existing aircraft is considered carefully against a number of factors, such as the type's remaining in-service life, flight safety record, cost and operational considerations. As a result of such considerations, a decision was taken to install a Flight Data Recorder and cockpit voice recorder as part of the helicopter Health and Usage Monitoring System (HUMS) programme for Chinook. The Trial Installation of HUMS into Chinook Mk2 and subsequent flight trials have been completed successfully. Work is now in progress to fit the system to all aircraft in the Mk2/2A fleet and the planned completion date is August 2001.

Question 253: Whether the statement that he had quoted in Q251 from the memorandum by Mr Malcolm Perks is well founded

  1.  In answering this question we have also addressed a number of the points made by Mr Perks in the memorandum he volunteered to the Committee so that it has a fuller appreciation of the issues he seeks to raise. This response also addresses issues raised by the Committee in later questions.

  2.  Mr Perks' memorandum to the PAC is based on his knowledge of the material provided to him during MoD's Arbitration case against Textron-Lycoming. As Mr Perks noted in a letter to MoD lawyers in 10 January 1995, he was not familiar with the production standard of software. Understandably he is, therefore, not well placed to comment on the functionality, qualification and performance of the in-service aircraft.

  3.  The following comments are offered to illustrate some of the more significant misconceptions contained in his memorandum, although there are also other errors, to which a response has not been included in the interests of brevity. The headings are those in Mr Perks' memorandum.

  4.  Executive Summary—Mr Perks claims that criticisms created a "fortress mentality" which still exists today. MoD has sought advice from a wide variety of internal and external parties to ensure that all FADEC issues were properly staffed. It appears that Mr Perks' criticism is that the Project did not accept all the advice offered and that it continues to present the Chinook Mk2 Programme as a success.

  5.  In managing the project, the Project Director had to weigh and balance advice from a variety of sources, not all of which was in accord. It therefore was inevitable that some of this advice would not be accepted.

  6.  Mr Perks' view concerning the final conclusion in the NAO Report case study (Box 5, Part 3), is based on a misunderstanding of that Report. The case study was about the Chinook Mid Life Update (MLU) programme and the acceptance of the Mk2 aircraft into service. It was not about the FADEC development programme, which was an entirely separate requirement. Mr Perks has presented them as one programme which they are not.

  7.  The MLU programme was delivered on time and within budget, as reported by the NAO.

  8.  FADEC was essentially a software project—Mr Perks reviews the operation of FADEC and concludes that it is "Flight Safety Critical" and that FADEC was a "unique design". The issue of criticality is addressed in the answer to Questions 182, 198-199 and 200. MoD paid great attention to Boscombe Down's concerns over software, albeit that these concerns were not substantiated. In addition, MoD undertook SATA (another form of static analysis which was amenable to the software) which reinforced the decision to introduce the aircraft into Service, as does the system's performance demonstrated to data. SATA is used by major USA aerospace companies, by the Federal Aviation Authority, NASA and by the US Department of Defense. Full authority digital electronic control systems, such as the Chinook FADEC are not unique and are employed on other aircraft in service today.

  9.  Software projects are difficult—Mr Perks implies that the lack of automated tools makes software unsafe. The absence of such tools generally means that it takes longer to produce, and the structuring of the software may be less amenable to verification processes, such as SCA. Tools are a valuable aid in software production, particularly for very large computer programs (which may have over a hundred thousand lines of code). However, FADEC software had only 22,000 lines of code. The use of automatic tools is no guarantee of safety. The use of such tools should improve the quality of the source code on large programs, but it is possible to have well written code which will still not meet the required functionality.

  10.  The project management approach—Mr Perks says that MoD had no direct control over the software. As CDP made clear in his evidence to the Committee at the Hearing, it is not MoD's policy to control design but rather to ensure Contractors comply with the contract requirements. In the case of FADEC, MoD procurement staff had access to all Contractors during the development programme and raised concerns over the poor standard of documentation. The contractors failed to correct this deficiency completely by the end of the development phase, hence the reason why the software was re-documented. However, none of the documentation changes represented safety concerns.

  11.  The early signs of trouble—Mr Perks refers to the FADEC programme being "in trouble" three years after launch. He says that "far from being in production, development FADECs were only just starting their test programme". The contract at that stage was for development only: that is why FADEC was not in production.

  12.  Any delay in a development programme is unwelcome, and the Wilmington incident in January 1989 delayed the programme by eight months. Following a redesign, the FADEC system recommenced testing on a Mk1 aircraft in September 1989 and successfully completed flight acceptance in October 1989, ending the development programme. Mr Perks claims that testing had "restarted" in 1990. In 1990 the development programme had already completed, and CA Release Flight Testing commenced on schedule. Mr Perks notes that the RAF Board of Inquiry (sic) into the Wilmington incident recommended a review of the software design: this was carried out as part of the normal mitigation action following the incident.

  13.  Software quality was a real issue—Mr Perks accepted at the time of his employment by MoD's US lawyers that he had no direct involvement with software engineering. He has not reviewed the software code. He was provided with information from MoD's own quality audits of the 1988 version of the software code. His assertion that the role of Boscombe Down is to approve is based on a fundamental misunderstanding. Their role is to provide advice to the Project Director on airworthiness matters. The fact that Boscombe Down pronounced the software as unverifiable using a particular method does not mean that the software was unsafe. Boscombe Down wanted to use SCA, but the software was not amenable to this process, neither was it mandated by the standards in force at the time of its development. Rewriting the software would only serve to increase the risk, by negating the many thousands of hours of testing and operating experience. Subsequent analysis and in-service experience has supported this decision. The EDS-Scicon review highlighted that documentation anomalies earlier identified to the Contractors during development had still not been sufficiently addressed in the production version. None of the documentation anomalies represented any safety concerns.

  14.  Mr Perks' claim that MoD overruled Boscombe Down is again based on a misunderstanding of the role played by Boscombe Down and the role and responsibilities of the Project Director. Boscombe Down's advice was properly considered along with advice from other experts and a balanced decision was made with the full consent of stakeholders. Boscombe Down have no executive role in making airworthiness decisions. Their job is to provide advice. Given their choice of software verification tool, they could not comment either way on the integrity of the software other than to observe that it was functionally acceptable. To enable them to comment any further, they recommended that the software would have to be rewritten to make it compatible with their chosen verification process.

  15.  So why was it accepted into service?—Mr Perks claims there was little choice but to accept FADEC equipped aircraft into service. This is untrue. There was always the option of retaining the existing hydro-dynamic engine control system, particularly as this was available and installed on the aircraft being inducted into the MLU programme. The decision to fit FADEC was made following successful completion of its flight test qualification programme in December 1989. It had originally been intended to introduce FADEC as a stand-alone modification on the Mk1 aircraft. However, when the MLU programme was approved in 1990, it was considered more logical to harmonise its embodiment within that programme, rather than incur a significant disruption to the RAF fleet.

  16.  Was FADEC in fact "acceptable"—Mr Perks' assertions appear to be based on the lack of software verification. He seems unaware that the production software was verified using SATA, another static technique. This consisted of a static audit of 100 per cent of the software source code against the design specification to ensure traceability (in much the same way as recommended by Mr Perks in his footnote), and a Sneak analysis to look for flow paths that would lead to anomalous behaviour. The results of this analysis identified some 125 anomalies, mainly associated with the documentation of the software, all of which were assessed by the Design Authority. As in the case of the aborted attempt by EDS-Scicon to apply SCA, none was found to affect safety. The re-documentation work was independently overseen.

  17.  However, in light of Boscombe Down's continuing but unsubstantiated concerns, the MoD accepted their advice in November 1993 to restrict the aircraft weight to the safe single engine performance envelope. Whilst Mr Perks continues to raise questions over the "unpredictability aspects of software", he appears to take no account of the fact that many of the earlier "teething" problems experienced were caused by mechanical problems and were not related to software. All of the "teething" problems were controlled in service by Standard Operating Procedures (SOPs). Neither does he mention the reliability of the system, which has been demonstrated in-service, and during its re-qualification programme following Wilmington. Based on the evidence available from software analyses, and the system's significant in-service reliability record within the RAF, the aircraft weight restriction was lifted in September 1998.

  18.  At the hearing the Chairman read out Mr Perks' assertion that FADEC's alleged unpredictability remained a possible factor in the Mull of Kintyre accident (Question 265). This speculation is not borne out by the evidence. MoD has never denied that there were problems with FADEC development; that is the nature of a development programme, but the problems found during development were resolved before the aircraft with FADEC fitted was introduced into service.

  19.  In-service experience of FADEC is that it is much more reliable and capable than its predecessor system. Its design philosophy is such that it has built in serviceability test and monitoring and a separate Reversionary system providing a back up capability. While the Chinook Mk2 has experienced six run downs caused by interruptions to electrical power during engine start up procedures, and one run up, all these happened on the ground as part of precautionary pre-flight checks. In addition, there were also nuisance type faults such as spurious engine fail captions. None raised safety concerns and where necessary, SOPs were developed to manage them.

  20.  Mr Perks says that "although not provable" it remains a possibility that FADEC unpredictability was a factor in the Mull of Kintyre Accident. The BOI conducted a very thorough investigation, and considered all the factors which may conceivably have had a bearing on the accident, including a major technical failure of the aircraft prior to impact. The AAIB report to the BOI found no signs of pre impact failure or malfunction that could have affected the operation of either engine. Furthermore, post accident evidence was consistent with both powerplants operating at similar, intermediate power levels indicating that neither engine had suffered an overspeed or a run down.

  21.  Why was it done at all?—Mr Perks asserts that the MoD fleet was too small to justify the effort. MoD saw an opportunity to reduce the cost of ownership of its Chinook Fleet which would reduce the sums spent at the US Contractors concerned in supporting the old system. Mr Perks implies that this benefit should have been offered by the US Contractors as a private sector funded programme: this does not reflect the realities of the market.

  22.  Boeing now offer FADEC for all new Chinook customers. The US Army already have 25 Chinooks fitted with FADEC and are about to upgrade a further 300 of their fleet of CH47D Chinooks to FADEC standard starting this year, as part of a major remanufacture programme.

  23.  Risk—Mr Perks suggests that FADEC was a high risk programme. FADEC started life in 1979 as a Technology Demonstrator Programme. By the time the development programme was placed for Chinook in 1985, the feasibility of the design had been successfully demonstrated on the Rolls Royce military Gem engine and the technical risks had been reduced to an acceptable level. Under the Chinook development programme successive risk reduction measures were carried out in the form of hazard analyses, computer simulation, engine bench testing and eventually flight testing, before the production contract was placed in 1991. Altogether, over 70,000 hours of testing were undertaken in qualifying the FADEC system.

  24.  Use of experts—Mr Perks says that MoD should take the views of its experts. MoD took advice on FADEC from a number of experts, but as noted above, because of differing views, the advice received often has to be balanced. The Project Director worked to relevant military or industrial standards and regulations in discharging those elements of airworthiness for which he had delegated responsibility from Secretary of State for Defence. Boscombe Down's role is as an advisor to the Project Director providing independent airworthiness advice. In so doing they are cognisant of, but not constrained by the regulations. Boscombe Down is one of many advisers that Project Directors use on airworthiness matters. Boscombe Down's role is not to approve or to re-qualify, but to provide advice on whether the equipment meets the contracted requirements and recommend a safe operating envelope for its intended use in-service.

  25.  Has anyone been blamed?—Mr Perks implies that MoD should publicly blame its Contractors for what he believes are FADEC issues. MoD has not taken an active role in commenting publicly on the 1989 pre-production software test and design failure, because the Arbitration against Textron-Lycoming required that all parties recognise and respect the confidentiality of those proceedings. All involved parties, including Mr Perks, are bound by this confidentiality of the Arbitration process. Blame was not a feature of the Arbitration and no liability was accepted by either party.

  26.  In some respects Mr Perks appears to have taken the sometimes robust and critical discussion during the Arbitration as a sign of weakness in the eventual decision to certify the production standard of FADEC. In reality, the opposite was true. The additional design reviews, analyses and testing actions undertaken to ensure that the pre-production development problem which emerged in 1989 was fully resolved and that the system's functionality was compliant with contract requirements, only served to reinforce MoD confidence in the airworthiness of the design.

Questions 264-265: Notes on Mr Malcom Perks' memorandum including the footnote7[43]

  1.  In Mr Perks' memorandum to the Committee, he offered his opinion that the 1988 version (pre-production) of software had not been designed and developed in accordance with industry accepted practices. This comment appears to relate to the performance of the contractor. (Textron-Lycoming) against the contracted specification. Mr Perks did not query the contracted specifications with MoD during his employment on behalf of the Department.

  2.  In January 1989 the FADEC design, as evidenced by the Wilmington incident, was clearly not in compliance with the contracted specification. The functional design errors that lead to this incident were remedied before its introduction into service in November 1993. As the contract specification did not mandate SCA and the software was not written with the verification process in mind, it is hardly surprising that it proved to be unamenable to that technique, and therefore that it was unverifiable using that technique. This is not the same as saying that the software was unverifiable. As noted above (Questions 31 and 202-208), in view of the continuing concern expressed by Boscombe Down, the Project Director arranged to verify the software using another method of static technique (SATA) as part of the Block One software upgrade programme in 1995. All safety issues having previously been mitigated, the changes between the Baseline Software version and Block One addressed the nuisance faults; those issues addressed procedurally; and anomalies addressed by EDS-Scicon. SATA consisted of a static audit of 100 per cent of the software source code against the design specification to ensure traceability (in much the same way as recommended by Mr Perks in his footnote), and a Sneak analysis to look for flow paths that would lead to anomalous behaviour. SATA identified 125 anomalies, mainly associated with the documentation of the software, all of which were assessed by the Design Authority: none was found to affect safety. The re-documentation work was overseen independently. The results reinforced MoD's confidence over the integrity of the production standard of software.

  3.  With respect to remarks over doubtful quality, MoD accepts that the software documentation was not compliant with contract requirements, that is why Textron-Lycoming re-documented the code in 1995. Despite the concerns raised at the time over the spurious nuisance faults, none of them was proved to question the safety of the system.

  4.  The assertion that MoD overruled its experts shows an incorrect understanding of the process and the role of Boscombe Down, as noted above (Question 253). With respect to the MoD commitment to continuing the FADEC programme, the response to Question 253 also answers this allegation.

  5.  The aircraft which crashed on the Mull of Kintyre, (ZD576) on 2 June 1994 was flown by an experienced crew. An RAF BOI was convened following the accident in accordance with standard RAF procedures, to examine the evidence and to determine its cause. The Board was assisted on the technical part of the investigation by, amongst others, civilian specialists from the Air Accidents Investigation Branch.

  6.  There has been much speculation about the possibility of a catastrophic engine malfunction as a result of a FADEC failure. This was one possibility considered by the Air Accidents Investigation Branch (AAIB) in their report to the BOI. The AAIB said that "the engine change units and FADEC were examined in some detail". The report goes on to say that "Strip examination [of the engines] indicated that both were running at high speed with the turbines hot at the time of impact, and revealed no signs of pre-impact failure or malfunction that could have affected the operation of either engine". Furthermore, post accident evidence was consistent with both powerplants operating at similar, intermediate power levels indicating that neither engine had suffered an overspeed or a run down. Physical damage from an engine overspeed happens very quickly. No signs of pre-impact malfunction of the transmission or rotor systems were found. Had FADEC been a factor, investigators would expect to have found evidence of the following; physical evidence of pre impact failure or malfunction in either engine; physical evidence pointing to the engines running at dissimilar power levels; physical evidence of pre impact damage to the transmission and the rotor system; and physical evidence that the DECU warning lights were illuminated. At the time of the accident, the aircraft was operating within the safe single engine performance envelope.

  7.  The accident investigation established that, at 1.75 km (0.95 nautical miles, or about 20 seconds) before impact the crew released their navigational computer from its fix on the Mull of Kintyre and set it to indicate the bearing and distance to the next way point at Corran. At that point the pilots therefore had a clear indication of how close they were to the Mull. In view of the deteriorating weather and the strict visibility requirements of visual flight rules, under which they were flying, they should by that time already have chosen an alternative course. Given that they had not done so they could and should immediately have either turned away from the Mull or slowed down and climbed to a safe altitude.

  8.  The re-setting of their navigational computer about 20 seconds before the crash is a telling indicator that at that time the pilots were not grappling with an in-flight emergency. Yet they were flying on, contrary to both instrument and visual flight rules. The evidence is that they were already too close to the cloud-covered mass of the Mull, travelling too low and too fast.

  9.  The Board conducted a very through investigation, and considered all the factors which may conceivably have had a bearing on the accident, including a major technical failure of the aircraft prior to impact. It was, however, the overall finding of the inquiry that in continuing to fly their aircraft in the way that they did, toward the high ground of the Mull of Kintyre, the pilots did not exercise the skill, care or judgement they were known to possess. As such, they were deemed negligent.

Ministry of Defence

25 May 2000


38   House of Commons Defence Committee Fourth Report dated 13 May 1998 (HC 611). Back

39   House of Commons Defence Committee Fourth Report dated 13 May 1998 (HC 611). Back

40   House of Commons Defence Committee Fourth Report dated 13 May 1998 (HC 611), memorandum from the Ministry of Defence responding to the Committee's Questions. Back

41   Ibid. Back

42   House of Commons Defence Committee Fourth Report dated 13 May 1998, (HC 611) Memorandum from the Ministry of Defence responding to the Committee's Questions. Back

43   See Evidence page 2. Back


 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2000
Prepared 30 November 2000