Select Committee on Health Minutes of Evidence

Memorandum by the Consumers' Association (NC 30)


  1.1   Consumers' Association (CA) is pleased to contribute evidence to the Health Committee inquiry into the progress that NICE has made towards achieving its key goals as set out in the NHS White Paper—A First Class Service. CA fully supports these goals, particularly those that relate to the provision of clear and credible guidance; the provision of a single national focus for guidance on treatments and the promotion of interventions with good evidence of clinical effectiveness and cost-effectiveness. However, in relation to these goals CA research has found several areas where NICE could significantly improve its performance.

  1.2  Specifically, CA has identified serious shortcomings in NICE appraisals of health technologies that raise questions about the Institute's role, the credibility of its guidance and how useful the guidance is to healthcare professionals and patients. In some cases, these problems include weaknesses in the way that evidence about treatments have been collated, interpreted, extrapolated and ultimately used in NICE guidance. In other instances, NICE has inappropriately considered individual treatments largely in isolation from alternative treatment options or indeed previous NICE guidance. This could confuse healthcare professionals as to whether or how various potential treatments should be used in relation to each other. Other problems identified include contradictory recommendations within guidance documents and a failure to acknowledge or address discrepancies between NICE guidance and other standard sources of advice for healthcare professionals and patients (such as summaries of product characteristics and patient information leaflets). The Institute has also appeared reluctant to respond to concerns raised about its guidance.

  1.3  CA policy research into how well NICE performs from the perspective of patients and patient organisations has identified similar issues around lack of clarity about the relationship between evidence considered and final NICE guidance. Patient groups have particular concerns about the extent to which NICE considers what is important from a patient point of view when assessing the impact of treatments. Confusion and lack of transparency surrounding the NICE appraisal process makes it especially difficult for patient groups to contribute effectively to the appraisal process—and to measure the impact of their contribution on the outcome of the appraisal.

  1.4  It is CA's view that the following actions would significantly improve the performance of NICE from the perspective of healthcare professionals and patients:


2.1  Context of guidance

  2.1.1  NICE needs to make greater efforts to ensure that its guidance relates directly to the clinical situations in which it might be used. This means, for example, making certain that where NICE is recommending a single health technology, it defines how this intervention should be used in relation to other treatment options available in the NHS. This may require indicating whether the appraised treatment should be used before, after, instead of, or together with the other treatments. If the place of an individual health technology under review cannot be defined without undertaking simultaneous appraisals of other treatments, then NICE should consider making appropriate representations (for example, to the Secretary of State for Health or the National Assembly for Wales), with a view to broadening the brief of the appraisal. At a minimum, NICE guidance should state clearly whether or not is has addressed how the appraised health technology should be used in relation to other treatment options.

2.2  Evidence

  2.2.1  There needs to be a full-scale review of the appraisal process. Part of this must consider how NICE can ensure that the measures used to evaluate clinical effectiveness and cost-effectiveness are appropriate. Calculations of costs and savings for specific health technologies should be based on how those treatments might actually be used in practice. For example, cost should not be based on assumptions about continuous long-term use of treatments when in reality patients may be prescribed and use such treatments on an intermittent basis.

  2.2.2  NICE must also ensure that the evidence that relates to clinical effectiveness is relevant to the experience of and priorities for healthcare professionals and patients. For example, information about the performance of a treatment from existing clinical trials may not be significant from a patient/professional point of view. Where there is considerable anecdotal evidence that treatments may have benefits that are not formally substantiated in existing trial data, NICE should consider commissioning independent research to assess this evidence.

2.3  Updating and communication systems

  2.3.1  NICE must make greater efforts to ensure closer integration of its guidance on separate health technologies for the same condition. This means, for example, being clear about the implications that new guidance has for other current NICE guidance. This information must be detailed in the new guidance and, ideally, included as an addendum to the other guidance.

  2.3.2  NICE must put systems in place to, where necessary, amend guidance before the planned expiry date and to alert healthcare professionals and patients on such changes - for example, to revise the text of guidance in the light of new or reconsidered evidence or other information, or where errors or omissions have been identified.

  2.3.3  NICE must also ensure that it has systems in place to respond to issues and questions raised by healthcare professionals and patients about its guidance.

2.4  Transparency

  2.4.1  There is an urgent need for NICE to be more transparent about the appraisal process. Lack of clarity and openness about how decisions are taken about clinical effectiveness and cost-effectiveness and the weight that the Institute gives to different kinds of evidence seriously undermine the credibility and authority of NICE. This should be dealt with as part of the recommended review of the NICE appraisal process.

2.5  Resources

  2.5.1  Patient groups make considerable efforts to contribute to and influence the way that health technology appraisals are undertaken. Resources should be made available to support this contribution—either resources provided directly from NICE or from other government allocations.


  3.1  Consumers' Association (CA) is an independent organisation that works on behalf of consumers to achieve improvements in the quality of goods and services. CA's key health objectives are to ensure that the public has access to information to make informed decisions about health services and policy and to campaign for positive change in those areas where we have concerns. These objectives are supported through health policy research, reports published in consumer magazines Which? and Health Which? and also, more recently, through the publication of Treatment Notes, a bulletin designed to give patients access to independent and rigorous reviews of the effectiveness of drugs and other treatments. Treatment Notes is produced in conjunction with Drug and Therapeutics Bulletin (DTB).

  3.2  Drug and Therapeutics Bulletin (DTB) is an independent monthly journal circulated to over 125,000 doctors and pharmacists throughout the UK. It aims to provide such healthcare professionals with impartial information on treatment choices and the management of disease. Its advice is intended to be clear, unambiguous and usable. To this end, such guidance is based, wherever possible, on well-designed, appropriately conducted randomised controlled clinical trials published in full, is informed by the views of professional organisations, consumers and patients, and takes account of UK law and NHS policy. In producing its reviews, DTB scrutinises and assesses published source material, paying particular attention to the reliability of such information.


  4.1  Consumers' Association (CA) has had a long-standing interest in medicines policy in relation to questions of effectiveness, access, and information. Drug and Therapeutics Bulletin (DTB) plays an important role in helping prescribers by providing information about the effectiveness of treatments. As part of its role in informing healthcare professionals, DTB has produced reviews of various treatment areas on which the National Institute for Clinical Excellence (NICE) has previously or subsequently issued advice in the form of Technology Appraisal Guidance. It is with reference to this material, that DTB offers evidence relating to the terms of reference of the Health Committee Inquiry.

  4.2  CA also deals with broader policy issues about medicines from a patient perspective. We have a particular interest in user involvement at all levels of decision-making, from decisions about individual treatment through to involvement in wider policy decisions about which specific treatments are provided on the NHS and why. CA has researched and published reports on the Patient Information Leaflets (PILs) that are included with all prescription drugs and on promotion of prescription drugs directly to patients. More recently we published a report on the outcome of a Consumers' Association inquiry into how well NICE works from the patient perspective. This submission to the Health Committee also includes evidence relating to the outcome of the CA inquiry.

  4.3  This following evidence is presented in two parts. The first part draws on the expertise and experience of DTB in relation to NICE. The second part of this evidence deals specifically with the outcome of the CA inquiry into how well NICE performs from a patient and patient group perspective.

Part 1



  The evidence presented below focuses on nine of the Technology Appraisal Guidance documents that NICE has issued so far. These collectively cover six separate treatment areas:

    —  Influenza1,2

    —  Type 2 diabetes mellitus3,4

    —  Attention deficit hyperactivity disorder (ADHD) in children5

    —  Alzheimer's disease6

    —  Motor neurone disease7

    —  Obesity in adults.8,9

  DTB has published advice relating to each of these treatment areas.10-19


  6.1  In 1999, NICE concluded that the drug zanamivir should not be used in the NHS to manage patients with influenza,1 a conclusion independently reached by DTB.10 After reviewing further evidence, NICE revised its position in November 2000, when it recommended that, under specific circumstances, zanamivir should be available for treating a defined group of individuals whose age or underlying medical problems puts them at particular risk of experiencing more prolonged and/or severe illness or developing complications as a result of influenza virus infection (referred to in the guidance as "at-risk patients").2 DTB disagreed with this revised NICE guidance having reviewed the evidence on which it was based.11 Furthermore, DTB's assessment of the revised guidance (and the associated Health Technology Assessment report commissioned by NICE,20 the Supplement to this report21 and the guidance's information for patients2) raised other serious concerns about the credibility and usability of NICE's recommendations on the use of zanamivir.22

  6.2  The revised NICE guidance appears to be based primarily on an analysis of data pooled from six randomised controlled clinical trials that tested zanamivir against a placebo treatment. In all, these trials involved around 800 at-risk patients. The guidance makes three key statements about the effects of zanamivir:2

    —  "In the overall pooled analysis of at-risk individuals, the duration of symptoms of influenza is reduced by 1.2 days (95 per cent CI 0.1 to 2.2) from about 6 to 5 days in the ITTP [intention-to-treat population]."

    —  "Overall zanamivir reduced the absolute risk of complications requiring antibiotics in the ITTP by 6 per cent (95 per cent CI: 0 to 11 per cent)."

    —  "No reliable data are available as to the impact of the use of zanamivir on hospitalisation rates or mortality."

  However, crucial weaknesses in the way the evidence was collated, interpreted, extrapolated and ultimately used undermine the credibility of the revised guidance.

  6.2.1  There are several important limitations in the trials chosen for inclusion in the pooled-data analysis, which make the robustness of the analysis highly questionable. First, most of these trials were not designed specifically to assess the effects of zanamivir in "at-risk" patients. Indeed, such individuals formed only a minority of those recruited into these trials. Also, since these patients were from separate studies, there are dangers in assuming they constitute a representative or typical group of "at-risk" people. Only one of the trials included in the pooled analysis specifically recruited "at-risk" patients (in this case, individuals with asthma or chronic obstructive pulmonary disease). There is considerable doubt as to the usefulness of the data from this trial for assessing whether use of zanamivir in the NHS will prevent serious complications of influenza in "at-risk" patients.22 A key problem is that the participants of the trial are unrepresentative of "at-risk" groups generally in the UK. For instance, the study included only patients with respiratory conditions (primarily asthma), generally of mild to moderate severity. In addition, relatively few of these people (under 10 per cent) were aged over 65 years, and overall only 23 per cent had been vaccinated against influenza.22 (At the time of publication of the revised NICE guidance, over 60 per cent of people aged over 65 years had been vaccinated against influenza.) Finally, the data-pooling process combined results from this trial with the data from the (also potentially unrepresentative) subgroups of at-risk patients in other trials.

  6.2.2  The pooled-data analysis reported in the revised NICE guidance also has other weaknesses. It considers zanamivir's effects on "complications requiring antibiotics" as key evidence of the drug's benefits.2 However, it is questionable whether an effect on such complications is a reliable guide as to whether the drug helps reduce the risk of being hospitalised or dying as a result of influenza infection.11,22 This is partly because "complications requiring antibiotics" could be relatively minor problems (eg sinusitis or tracheitis) and it is not clear from the revised guidance (or its supporting literature) what proportion of the patients treated with zanamivir and control groups had more severe complications (eg pneumonia). Also, it is not clear exactly what criteria were used for starting antibiotic therapy in the trials from which the evidence was collated. So, in some cases, such treatment might have been started for relatively minor illnesses. Also, none of the trials in the pooled analysis were designed to look primarily at zanamivir's effects on "complications requiring antibiotics". In addition, zanamivir's effect on "complications requiring antibiotics" varied considerably from trial to trial (as acknowledged by the authors of the NICE Health Technology Assessment report.21) This variation further limits the robustness of the pooled-data analysis. Furthermore, the data analysis shows that the measured effect of zanamivir on complications requiring antibiotic use (an apparent 6 per cent reduction in the rate of such complications) did not reach conventional levels of statistical significance. The analysis is therefore not only flawed, but provides no evidence that zanamivir prevents the serious complications of influenza in at-risk patients.

  6.2.3  The pooled analysis also suggested that, in at-risk patients, zanamivir reduced the duration of symptoms of influenza by around one day, a benefit similar to that reported for patients in general.1,10 NICE does not see this limited effect as a compelling reason for prescribing zanamivir for "otherwise healthy" people.2 So, it is questionable whether this benefit alone would be a convincing basis for recommending the drug for at-risk patients.

  6.2.4  The revised NICE guidance2 (and the associated Health Technology Assessment report20and its Supplement21) reached conclusions and offered advice that are hard to justify given the inherent weaknesses in the underlying evidence. First, the claim in the revised guidance that zanamivir reduces the risk of "complications requiring antibiotics" is not justified by the data presented. The various flaws in the data also undermine the Supplement's suggestion that "it seems reasonable to suppose that, if we can prevent 6 per cent of the at risk population from needing antibiotics, more probably than not, we may also be preventing some deaths." 21,22 Furthermore, the revised guidance cites an economic evaluation2 (published in the Supplement21) that estimates the cost-effectiveness of zanamivir by assuming, inappropriately, that the drug would produce a 6 per cent fall in death rates from influenza among at-risk patients; the evaluation is therefore flawed.

  6.3  The revised NICE guidance does not mention how it conflicts with other current NHS advice and policy. The guidance raises such conflicts by, for example, not including any advice on use of symptom-relieving medication (eg paracetamol) in at-risk adults with symptoms of influenza; omitting patients with chronic renal failure from its definition of "at-risk patients" (ie those for whom the Department of Health recommends annual influenza vaccination); and ignoring relevant concerns about microbial resistance detailed in advice from the NHS Executive on the use of Patient Group Directions.2,22 The guidance's failure to acknowledge, explain or address such conflicts could lead to confusion among healthcare professionals and patients, seriously limit the credibility of the guidance and hinder its ability to become locally owned and trusted.

  6.4  The revised NICE guidance also omits other important treatment information relating to the use of zanamivir. For instance, it does not mention possible unwanted effects of zanamivir, in particular, serious respiratory symptoms and how these might be prevented (as detailed in the drug's summary of product characteristics). 2,22,23 In addition, the guidance is at odds with the current patient information leaflet supplied with zanamivir. 2,24 This leaflet recommends that before using the drug, patients should talk to their doctor if they have "been told that your immunity to infections is poor" or have "severe or persistent asthma, or another long-standing lung disease". By contrast, the advice to patients included in the revised guidance suggests that these relative contraindications are the very reasons for giving zanamivir and that the treatment could be provided by a nurse or pharmacist without first contacting the patient's doctor.2 These unexplained discrepancies could confuse both healthcare professionals and patients and further reduce the credibility and local ownership of the guidance.

  6.5  It is worrying that the group commissioned by NICE to prepare the Health Technology Assessment report on zanamivir indicated that they did not have access to all the information needed to fulfil this brief and suggested that the assessment of zanamivir could not be done reliably in the time available.20 These criticisms of the procedure serve to undermine the validity of the Assessment's conclusions and, potentially, the overall appraisal of zanamivir. There were also weaknesses in the associated information for patients issued by NICE.2,22 In particular, the language in the text is likely to be inaccessible to many lay readers and the advice was inaccurate in stating that nurses or pharmacists might "prescribe" zanamivir to patients. It therefore seems reasonable to question whether the text of the patient information was checked for accuracy and field-tested for reader accessibility.

  6.6  Another concern is how NICE seems to have exaggerated publicly the strength of evidence underlying its revised guidance. When confronted with the suggestion that some GPs might refuse to prescribe zanamivir, the Chairman of the Institute was quoted as saying "I would like to remind them that a group of highly expert individuals have looked at this, which includes GPs. Just how will they feel when one or two patients in the primary care group dies of flu?"22,25 As detailed above, such a statement would not be supported by any reliable evidence gathered by NICE that zanamivir had an effect on mortality rates.2,22

  6.7  DTB detailed its various concerns about the revised NICE guidance on zanamivir in an unpublished report.22 A copy of this critique was sent to NICE, along with an invitation to the Institute to discuss the issues raised. To date, DTB has not received any formal reply to the points made nor has NICE taken up the offer to discuss these issues.


  7.1  NICE has recommended that the drugs rosiglitazone and pioglitzone should be used in the treatment of certain patients with type 2 diabetes mellitus.3,4 The relevant guidance documents clearly state that each drug "should be used in accordance with the manufacturer's recommendations".3,4 However, each document contradicts itself by suggesting that the drugs be used in ways not specified by the manufacturer, even though there seems to be no direct evidence to justify such use.

  7.1.1  The manufacturers' recommendations for pioglitazone and for rosiglitazone are that these drugs can be given in combination with the drug metformin only in obese patients, and in combination with a sulphonylurea drug in patients who are intolerant of metformin or for whom this drug is contraindicated.3,4 Contrary to these recommendations, the NICE guidance also suggests that patients whose blood glucose remains high despite an adequate trial of metformin plus sulphonylurea combination therapy "may be offered pioglitazone in combination with metformin or sulphonylurea as an alternative to injected insulin."4 The guidance on rosiglitazone offers a much stronger recommendation, suggesting that such patients "should be offered rosiglitazone combination therapy as an alternative to injected insulin."3 The guidance documents neither cite any direct supporting evidence nor offer any convincing rationale for suggesting these unlicensed uses of pioglitazone and rosiglitazone. (DTB could find no published evidence to support these wider, unlicensed recommendations, when producing its own review on pioglitazone and rosiglitazone.12) The NICE recommendations therefore lack credibility.


  8.1  NICE has recommended the drug methylphenidate as part of a comprehensive treatment programme for children with severe ADHD.5 However, the NICE guidance is of limited value because it makes no attempt to advise on use of the other licensed drug treatment for this condition—dexamfetamine [previously known as dexamphetamine]. As a result, healthcare professionals have no way of knowing from the guidance whether NICE intends that dexamfetamine should be used as an alternative to methylphenidate, reserved for only certain (undefined) groups of children or not used at all. These concerns are particularly pertinent given that the available evidence suggests that there is little difference in effectiveness between methylphenidate and dexamfetamine13 (as noted in the associated Health Technology Assessment report commissioned by NICE26). The two drugs also have similar unwanted effects. In addition, it is worth noting that the drug costs to the NHS of dexamfetamine are much lower than those of methylphenidate. All these factors make the omission of any advice on dexamphetamine particularly puzzling. They also suggest that by limiting itself to a review of just one treatment, the NICE guidance fails to answer key questions or to provide comprehensive and complete advice on whether, when and how drug therapy should be used in the NHS to manage children with ADHD.

  8.2  The NICE guidance also makes no reference to the fact that for some children with severe ADHD use of appropriate non-drug therapy might be a suitable way of avoiding exposure to drug therapy (for example, where parents are particularly keen to avoid using such medication). Nor does the guidance mention the evidence suggesting that combining medication with non-drug treatments might help limit the drug doses needed to treat a child.13 These practical issues merit discussion in any guidance that attempts to offer credible, practical information on optimal use of drug treatment for children with ADHD.


  9.1  NICE has recommended that the drugs donepezil, rivastigmine and galantamine should be available in the NHS as one component of the management of certain people with Alzheimer's disease.6 There are reasons to question the evidential basis for, and therefore the credibility of, this judgement.

  9.1.1  The main available evidence on donepezil, rivastigmine and galantine comes from randomised placebo-controlled trials that have assessed the effects of these drugs on patients' cognitive-function and global measures. However, there is considerable doubt about how well such indicators reflect meaningful benefits for patients and their carers, for example, improvements in mood, behaviour and quality of life, and delaying the need for long-term residential care. As stated in the Health Technology Assessment report (commissioned by NICE and reviewing the clinical effectiveness and cost-effectiveness of donepezil, rivastigmine and galantamine27), "It is difficult to quantify benefits from the evidence available in the literature. Statistically significant improvements in tests such as ADAS.cog [a measure commonly used in trials] may not be reflected in changes in daily life." The report also suggests that "there is great uncertainty surrounding the cost-effectiveness of these drug therapies" and goes on to conclude "The implications of the use of donepezil, rivastigmine and galantamine are unclear. The main issue is whether the modest benefits seen in the outcome measures used in the trials would translate into benefits significant to patients."27

  9.1.2  The uncertainties about the evidence on the effects of donepezil, rivastigmine and galantamine are reflected in the NICE guidance. For example, in relation to the cost-effectiveness of donepezil, rivastigmine and galantamine, the guidance states that "From a health economics perspective, the main benefit of these drugs is the improvement in patients' cognitive and other functioning, and the main potential cost-saving results from possible delayed progression to the requirement for nursing home care. Neither can be reliably or easily estimated from the existing trial evidence".6 Also in commenting on economic evaluations submitted by the manufacturers of the three drugs, the guidance states "The same caveats regarding the quality of evidence apply to these estimates as they do to the published studies: cost savings on later institutionalisation are not well established, and quality of life is not easily measured."6

  9.2  The guidance offers no other convincing rationale for recommending the use of donepezil, rivastigmine and galantamine in the NHS.


  10.1  NICE has recommended the use of the drug riluzole for the treatment of patients with the amyotrophic lateral sclerosis (ALS) form of motor neurone disease.7 There are reasons to question the evidential basis for, and so the validity of, this advice.

  10.1.1  It is worth noting the conclusion of the Health Technology Assessment report28,29 (commissioned by NICE as part of the drug's appraisal) which reviewed the clinical effectiveness and cost-effectiveness of riluzole: "There is limited evidence of a modest benefit in tracheostomy-free survival for patients taking riluzole. However, the evidence base is restricted and there remains uncertainty as to the true benefit of riluzole; the confidence interval is wide and compatible with little or no difference between riluzole and placebo. When costs and health economic impact extrapolating survival beyond that observed in trials are considered, the uncertainty about whether the benefits are worth the costs is magnified. Even under the most optimistic assumptions, riluzole at best only postpones death for a few months, and does not preclude the need for supportive care and practical help." and "Consequently, existing evidence on effectiveness and cost-effectiveness does not unequivocally indicate the best policy concerning use of riluzole in ALS for the NHS."

  10.1.2  The NICE guidance appears to make errors in stating that "The assessment report reviewed the results from all four of the trials identified and reported riluzole to be associated with a relative reduction in hazard ratio for tracheostomy-free survival at 18 months of 17 per cent (ie hazard ratio of 0.88, 95 per cent CI: 0.77-1.02".7 In reality, the statistical analysis of these four trials28,29 suggests that riluzole therapy was associated with a 12 per cent (not 17 per cent) lower risk of dying or requiring tracheostomy (hazard ratio 0.88, 95 per cent CI: 0.75-1.02). Also, there is considerable statistical uncertainty about this effect, since the associated confidence interval (95 per cent CI: 0.75-1.02) is compatible with riluzole being more effective, no different from or slightly less effective than is placebo, in prolonging a patient's life or reducing the need for tracheostomy. This point could be crucial for prescribers, patients and carers considering use of the drug, and should have been spelt out clearly in the guidance. By omitting such an explanation, the guidance gives the impression that riluzole definitely extends the period patients survive without needing a tracheostomy. In reality, the available evidence is much more equivocal.


  11.1  In March 2001, NICE issued guidance recommending the drug orlistat as part of the management for adult patients who are obese.8 Subsequently, in October 2001, NICE issued guidance recommending another drug—sibutramine—also for use in managing obesity in adults.9 The Technology Appraisal Guidance document for sibutramine mentions that NICE has previously issued guidance on orlistat.9 However, this document gives no advice on how the two drugs should be used in relation to each other. For example, the guidance offers no views on whether NICE is recommending that the drugs should be regarded as interchangeable treatment choices, whether the use of each drug should be restricted to certain (undefined) groups of patients or whether they can (or should) be used sequentially in some individuals. The guidance therefore does not seem to offer clear, integrated advice on the use of these two treatments in the overall management of patients who are obese. As a result, it is possible that individual healthcare professionals and service providers will have to formulate their own policies on selecting between and using the two treatments: this could lead to marked variation in practice throughout the NHS. This would be counter to NICE's remit to end confusion by providing a single national focus of advice.

  11.2  The NICE guidance on sibutramine does not mention some of the practical limitations that could make the drug difficult to use or which limit its value. Examples of these are most of the drug's important contraindications, which include conditions that commonly accompany obesity, such as coronary heart disease, peripheral artery disease, congestive heart failure and cerebrovascular disease. Similarly, the guidance does not refer to how various interactions with other drugs could complicate use of sibutramine. (DTB recently recommended against using sibutramine, having considered not only these practical factors, but also the limited efficacy of the drug, concerns about some of the pivotal evidence supporting its use, the risk of treatment-induced rises in blood pressure and heart rate, and the stringent requirements for monitoring during treatment.19 )


  12.1  There are significant weaknesses in current NICE guidance on various health technologies. In some cases, the brief of such guidance appears too narrowly drawn, for example, when it covers only one of several treatment options for a given condition. There is also evidence of failure to integrate new NICE guidance with other current advice issued by the Institute. In some cases, guidance has appeared counter to other NHS advice or policy or other standard advice available to patients and healthcare professionals (in summaries of product characteristics and patient information leaflets), without acknowledging, explaining or justifying such conflicts. This suggests that guidance is not always sensitive to the clinical context in which it is to be used. The evidential basis for some of the Institute's recommendations appears weak and, in these cases, no convincing alternative rationale for the advice is offered. Another concern is the evidence of inaccurate interpretation or inappropriate extrapolation of data underlying some of the Institute's recommendations. When directly challenged on such issues, NICE has seemed either unwilling or unable to debate the points raised, to offer a clear explanation for some of its assessments or, where appropriate, to revise the text of its guidance.

  12.2  The various problems with current NICE guidance on health technologies raise concerns about the clarity and credibility of such advice, and about whether, in general, the Institute is promoting health technologies for which there are convincing grounds for recommending use in the NHS. Such deficiencies may well discourage local ownership of NICE guidance and increase, rather than decrease, confusion among healthcare professionals and patients about the appropriate use of different treatments and the optimal management of different conditions. All of the issues raised need addressing, through a thorough review of the NICE appraisal process, if the Institute is to be regarded as a trusted source of sound and usable healthcare advice.

Part 2


  13.  The idea to hold a public inquiry developed from CA's work on the promotion of prescription drugs, through which we became aware that the Department of Health was planning to review NICE. Concerned that the interests of patients would not be fully considered in such a review, CA decided to hold a patient-centred inquiry of its own. The CA inquiry set out to consider the perspective and experience of patient groups in relation to how well NICE performs.

  13.1  The CA inquiry was set up along the lines of a select committee. Patient groups with an interest in NICE were invited to submit written evidence. Issues raised in the written evidence were used to develop lines of questioning for each of the four evidence sessions held on the day of the inquiry. The following paragraphs highlight the key findings and conclusions that CA derived from the day. Copies of the full inquiry report and briefing are available from CA.


  14.1  One crucial question is how quality-of-life measures—measures that relate to how a treatment impacts on a patient's life in ways that are important to them—are identified, evaluated and taken into account in the NICE appraisal process. Evidence provided to the inquiry from patient organisations as well as from NICE itself highlighted deficiencies in the availability of this kind of information in relation to specific treatments. (Another more difficult question is how quality of life measures can be evaluated for treatments not yet on the market or widely used.)

  14.2  Evidence provided on the day of the inquiry would suggest that in the absence of formal data, patient organisations do their best to fill the gap with information about the views and experiences of the patients that they represent. This information is valid in its own right and can provide valuable insight into what is important to those who might benefit from treatments. In addition, some patient organisations, like the Alzheimer's Society, have conducted extensive research into quality of life issues with their members. However, few patient organisations have the resources or expertise to undertake full and proper research in this area—and none that took part in the inquiry had been set up with this role in mind.


  15.1  Linked to this is the issue of information about how treatments perform in routine day-to-day clinical practice - circumstances that often differ from the parameters of clinical trials. The evidence presented at CA's inquiry indicates a need to ensure consideration of evidence about how treatments are actually prescribed and used—in ways that may be different from the circumstances considered in clinical trials. For example, patients with long-term medical conditions like multiple sclerosis may use treatments as their condition worsens—but not all of the time. This may be crucial when it comes to assessing the cost implications of treatments.

  15.2  There was general agreement that information that relates to both quality of life and how treatments work in day-to-day practice is essential to enable proper and meaningful appraisals of treatments. Evidence from NICE suggests that manufacturers are starting to take this information need into account in their approach to undertaking pre-market research and clinical trials.

  15.3  However, this still raises a number of questions in relation to identifying the outcome measures relevant to the NICE appraisal process and who should be responsible for ensuring that the necessary information is collected and assessed.


  16.1  For many patient groups, there is an underlying question about the weight accorded to different kinds of evidence in the appraisal process. Almost every patient organisation that took part in CA's inquiry reported issues about the hierarchy of evidence considered by NICE and lack of clarity about the weight given to different kinds of evidence. Most groups believe that evidence from patient groups is viewed as less important than other kinds of evidence. Mostly, patient groups have concerns about the lack of feedback about how their evidence is used—and specifically what difference their evidence made in relation to the outcome of the appraisal.

  16.2  In part, this may be the result of confusion and lack of clarity about exactly what question NICE is trying to answer or the precise nature of the outcome that it is trying to assess in the course of an appraisal. Overall, there is a strong case to be made for NICE being much clearer at the outset of an appraisal about what exactly it is trying to evaluate and, following that, a great deal more open and explicit about the methodology that it is using. This could include, for example, being clear about the questions that need to be answered and the relative importance of those answers in relation to the outcome of the appraisal.


  17.1  This leads directly to the question of transparency. As well as the need for NICE to be more explicit about the kind of evidence that is required, and how different evidence is measured and weighed, our inquiry also highlighted a need for more openness and transparency about how effectiveness is defined and decisions about cost are dealt with. It was not within the remit of the inquiry to challenge the issue of cost of treatments with NICE, but it clearly emerged as an area where there is much confusion and debate. For example, during the inquiry it was not possible to establish how decisions are taken about where or how NICE draws the line on whether a treatment is cost-effective. Nor was it possible to pin NICE down on costs per qualy.


  18.1  Many of the above issues deal with the question of openness and transparency, and these are linked to the way that NICE works and the process by which appraisals are undertaken. Patient organisations taking part in CA's inquiry were almost unanimous in their view that, although there is more to be done, there has been a continuous improvement in the way that NICE works with patient groups. Research commissioned by NICE from the King's Fund on improving the involvement processes should also go some way to improve the way that NICE engages with patients and patient groups.

  18.2  On the other hand, there remain a number of very real problems with the appraisal process that make it difficult for patient organisations to take part. Issues about confidentiality seem to present particular problems for organisations. For example, a number of organisations reported that confidentiality clauses made it difficult for them to consult with their patient or patient group members on appraisals. Many organisations also reported that the timeframes that NICE works within during an appraisal process are often too short to allow them to contribute in a meaningful way.


  19.1  Patient organisation clearly have an important role to play in the NICE appraisal process, both in providing specific information about effectiveness from the user perspective and ensuring that the patient and public interest is properly represented in the overall workings of NICE. However, the patient organisations that took part in the inquiry all reported that contributing to the NICE appraisal process consumed significant resources. In one case the estimated cost was £40,000, while in another, the resource required was a dedicated member of staff working over a three-month period. None of these groups receive financial support for this work.

  19.2  This highlights a particularly difficult issue for patient groups in relation to ensuring that the interests of their stakeholders are taken into account in the NICE process, while at the same time meeting their other commitments and obligations. For many patient groups, there is a very real issue about diversion of funds and other resources from core charitable activities. It is especially difficult where organisations are making concerted efforts to gather evidence from patients about their experiences and priorities that would not otherwise be captured or taken into account in the appraisal process. There is a need for serious consideration to be given to ensuring that, where organisations are collecting information pertinent to the appraisal process, this work is properly supported and funded. However, there is also a need for NICE to be prepared to commit resources to commission formal research in those areas where patient organisations have anecdotal evidence about the impact of treatments from the user perspective.


  20.1  From the outset of this inquiry, it was clear that there are issues linked to NICE but that are important to patients. In particular, issues that fall on either side of the NICE appraisal process—for example, the selection of technologies for appraisal and implementation of guidance across the health service. This inquiry did not tackle the question of selection for topics. However, we did consider the question of patient access to treatments.

  20.2  The discussion about outcomes highlighted two key concerns. First, anecdotal evidence would suggest that not all NICE guidance is implemented consistently across the health service. For example, some health authorities make treatments available but will use different criteria or higher thresholds for access to those treatments than those set out in NICE guidance. Secondly, some groups expressed concern that while appraisals are underway patients are sometimes denied access to the treatment under review, something that is not supported by formal government policy.

  20.3  It defeats the object of NICE if patients are being denied access to treatments that have been found to be effective—or which are under review. In this way, we welcome the requirement that patients be given access to treatments that have been found by NICE to be clinically and cost effective. However, there is a very important issue about ensuring that resources are not diverted from providing treatments that have not been subject to NICE guidance in order to meet such requirements.


  21.1  CA's patient-centred inquiry raised a number of key questions for NICE. Many of these centre around the need for clarity, in terms of how effectiveness is defined, the value of different kinds of evidence and where that evidence fits into the overall appraisal.

  21.2  There is also a need for clarity, or at a minimum an open debate, about what information is needed to ensure a meaningful appraisal of a treatment. Currently, it remains unclear as to whether NICE attempts to base appraisals on the best available information or on the information that is needed to make the best decision. The failure of NICE to be as open and transparent as it should be about how judgements about treatments are actually made makes these problems worse.

  21.3  Patient organisations clearly have an important role to play in the NICE appraisal process—both in providing specific information about effectiveness from a patient perspective and to ensure that the patient and public interest is properly served. There is an urgent need to consider how patient groups are supported and funded to undertake this important work.


  22.1  A review of NICE has been on the agenda for some time, with much debate about how and by whom the review should be undertaken. Both industry and government have a number of key issues that they want the review to consider. CA also believes that the time is right for a review of NICE. We believe that it is critical that the patient and patient group perspective is taken into account in this review. The terms of reference must be made public and widely consulted on—and the review must be carried out by an independent agency.

  22.2  There are a number of immediate steps that NICE could take to eliminate some of the confusion and misunderstanding that currently surrounds the appraisal process. Appraisal methodologies should be explicit and made public, along with details about the basis upon which decisions about specific treatments have been taken. There is also a need for a thorough and open discussion about the kind of information that is required to ensure a meaningful appraisal process. Processes for identifying gaps in the availability of information should be introduced—and, where possible, measures taken to fill such gaps.

  22.3  Patient groups spend considerable time and resources to ensure that the patient voice is heard in the NICE appraisal process. However, not all patient groups have the resources to support this work and, for many, contributing to NICE means diverting resources away from charitable activities. It is therefore essential that patient groups are properly funded and supported to enable them to effectively contribute to NICE. It is also essential, where there are no patient groups to represent the interests of a particular patient constituency, that measures are taken to obtain the patient perspective.

  22.4  At the same time, it is important that patient groups themselves are clear about why they are contributing to the NICE appraisal process. For example, there is a difference between working to ensure that NICE operates in a way that is open and transparent and which takes into account what is important from the patient perspective, and participating in the collection and analysis of that data. In some cases, patient organisations are lobbying to ensure access to treatments in a general sense; in other cases, groups may have access to clinical information that is relevant to the appraisal. All of these functions and roles are legitimate, but it is important that and patient groups themselves are clear about the nature of the contribution. This should help to eliminate the need for patient groups to try to fill information gaps when they may not be in a position to do so. It should also help NICE to be clearer about identifying missing information and how such information can be best collected.


  1.  National Institute for Clinical Excellence. Guidance to the NHS on zanamivir (Relenza) in the treatment of influenza. Technology Appraisal Guidance. October 1999. (Subsequently published as Old Guidance—Zanamivir. Available: [February 2000]).

  2.  National Institute for Clinical Excellence. Guidance on the use of zanamivir (Relenza) in the treatment of influenza. Technology Appraisal Guidance—No 15. Available: [21 November 2000].

  3.  National Institute for Clinical Excellence. Guidance on the use of rosiglitazone for type 2 diabetes mellitus. Technology Appraisal Guidance—No 9. Available: [August 2000].

  4.  National Institute for Clinical Excellence. Guidance on the use of pioglitazone for type 2 diabetes mellitus. Technology Appraisal Guidance—No 21. Available: [March 2001].

  5.  National Institute for Clinical Excellence. Guidance on the use of methylphenidate (Ritalin, Equasym) for attention deficit/hyperactivity disorder. Technology Appraisal Guidance—No 13. Available: [October 2000].

  6.  National Institute for Clinical Evidence. Guidance on the use of donepezil, rivastigmine and galantamine for the treatment of Alzheimer's disease. Technology Appraisal Guidance—No 19. Available: [January 2001].

  7.   National Institute for Clinical Excellence. Guidance on the use of riluzole (Rilutek) for the treatment of motor neurone disease. Technology Appraisal Guidance—No 20. Available: [January 2001].

  8.  National Institute for Clinical Excellence. Guidance on the use of orlistat for the treatment of obesity in adults. Technology Appraisal Guidance—No 22. Available: [March 2001].

  9.  National Institute for Clinical Excellence. Guidance on the use of sibutramine for the treatment of obesity in adults. Technology Appraisal Guidance—No 31. Available: [October 2001].

  10.  Zanamivir for influenza. DTB 1999; 37: 81-4.

  11.  Why not zanamivir? DTB 2001; 39: 9-10.

  12.  Pioglitazone and rosiglitazone for diabetes. DTB 2001; 39: 65-8.

  13.  Stimulant drugs for severe hyperactivity in childhood. DTB 2001; 39: 52-4.

  14.  Donepezil for Alzheimer's disease? DTB 1997; 35: 75-6.

  15.  Donepezil update. DTB 1998; 36: 60-1.

  16.  Rivastigmine for Alzheimer's disease. DTB 2000; 38: 15-6.

  17.  Riluzole for amyotrophic lateral sclerosis. DTB 1997; 35: 11-2.

  18.  Why and how should adults lose weight? DTB 1998; 36: 89-92.

  19.  Is sibutramine more than a slim hope? DTB 2001; 39: 89-91.

  20.  Burls A, Clark W, Stewart T et al. Zanamivir for the treatment of influenza in adults. Health Technology Assessment Report. Available: [12 December 2000].

  21.  Burls A, Clark W, Stewart T et al. Zanamivir for the treatment of influenza in adults. Supplement to the Assessment Report. Available: [12 December 2000].

  22.  Drug and Therapeutics Bulletin. Report detailing issues of concern raised by the November 2000 Guidance on the use of zanamivir (Relenza) from the National Institute for Clinical Excellence. 12 February 2001.

  23.  Relenza. Summary of product characteristics, UK. GlaxoWellcome UK Limited. [First authorisation, June 1999; partial revision of text, May 2001].

  24.  Relenza. Patient information leaflet. GlaxoWellcome UK Limited. May 2001.

  25.  Browne A. GPs warned as they snub advice on anti-flu drug. The Observer; 10 December 2000.

  26.  Lord J, Paisley S. The clinical effectiveness and cost effectiveness of methylphenidate for hyperactivity in childhood. London: National Institute for Clinical Excellence, Version 2, August 2000.

  27.  Clegg A, Bryant J, Nicholson T et al. Clinical and cost effectiveness of donepezil, rivastigmine and galantamine for Alzheimer's disease s. Health Technology Assessment Report. Available: [January 2001].

  28.  Stewart A, Sandercock J, Bryan S, Hyde C, Fry-Smith A, Burls A. The clinical effectiveness and cost effectiveness of riluzole for motor neurone disease. Health Technology Assessment Report. Available: [January 2001].

  29.  Bryan S, Barton P, Burls A. The clinical effectiveness and cost effectiveness of riluzole for motor neurone disease - An update. Health Technology Assessment Report. Available: [January 2001].

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 8 July 2002