Memorandum by the Consumers' Association
(NC 30)
1. SUMMARY
1.1 Consumers' Association (CA) is pleased
to contribute evidence to the Health Committee inquiry into the
progress that NICE has made towards achieving its key goals as
set out in the NHS White Paper—A First Class Service.
CA fully supports these goals, particularly those that relate
to the provision of clear and credible guidance; the provision
of a single national focus for guidance on treatments and the
promotion of interventions with good evidence of clinical effectiveness
and cost-effectiveness. However, in relation to these goals CA
research has found several areas where NICE could significantly
improve its performance.
1.2 Specifically, CA has identified serious
shortcomings in NICE appraisals of health technologies that raise
questions about the Institute's role, the credibility of its guidance
and how useful the guidance is to healthcare professionals and
patients. In some cases, these problems include weaknesses in
the way that evidence about treatments have been collated, interpreted,
extrapolated and ultimately used in NICE guidance. In other instances,
NICE has inappropriately considered individual treatments largely
in isolation from alternative treatment options or indeed previous
NICE guidance. This could confuse healthcare professionals as
to whether or how various potential treatments should be used
in relation to each other. Other problems identified include contradictory
recommendations within guidance documents and a failure to acknowledge
or address discrepancies between NICE guidance and other standard
sources of advice for healthcare professionals and patients (such
as summaries of product characteristics and patient information
leaflets). The Institute has also appeared reluctant to respond
to concerns raised about its guidance.
1.3 CA policy research into how well NICE
performs from the perspective of patients and patient organisations
has identified similar issues around lack of clarity about the
relationship between evidence considered and final NICE guidance.
Patient groups have particular concerns about the extent to which
NICE considers what is important from a patient point of view
when assessing the impact of treatments. Confusion and lack of
transparency surrounding the NICE appraisal process makes it especially
difficult for patient groups to contribute effectively to the
appraisal process—and to measure the impact of their contribution
on the outcome of the appraisal.
1.4 It is CA's view that the following actions
would significantly improve the performance of NICE from the perspective
of healthcare professionals and patients:
2. RECOMMENDATIONS
2.1 Context of guidance
2.1.1 NICE needs to make greater efforts
to ensure that its guidance relates directly to the clinical situations
in which it might be used. This means, for example, making certain
that where NICE is recommending a single health technology, it
defines how this intervention should be used in relation to other
treatment options available in the NHS. This may require indicating
whether the appraised treatment should be used before, after,
instead of, or together with the other treatments. If the place
of an individual health technology under review cannot be defined
without undertaking simultaneous appraisals of other treatments,
then NICE should consider making appropriate representations (for
example, to the Secretary of State for Health or the National
Assembly for Wales), with a view to broadening the brief of the
appraisal. At a minimum, NICE guidance should state clearly whether
or not is has addressed how the appraised health technology should
be used in relation to other treatment options.
2.2 Evidence
2.2.1 There needs to be a full-scale review
of the appraisal process. Part of this must consider how NICE
can ensure that the measures used to evaluate clinical effectiveness
and cost-effectiveness are appropriate. Calculations of costs
and savings for specific health technologies should be based on
how those treatments might actually be used in practice. For example,
cost should not be based on assumptions about continuous long-term
use of treatments when in reality patients may be prescribed and
use such treatments on an intermittent basis.
2.2.2 NICE must also ensure that the evidence
that relates to clinical effectiveness is relevant to the experience
of and priorities for healthcare professionals and patients. For
example, information about the performance of a treatment from
existing clinical trials may not be significant from a patient/professional
point of view. Where there is considerable anecdotal evidence
that treatments may have benefits that are not formally substantiated
in existing trial data, NICE should consider commissioning independent
research to assess this evidence.
2.3 Updating and communication systems
2.3.1 NICE must make greater efforts to
ensure closer integration of its guidance on separate health technologies
for the same condition. This means, for example, being clear about
the implications that new guidance has for other current NICE
guidance. This information must be detailed in the new guidance
and, ideally, included as an addendum to the other guidance.
2.3.2 NICE must put systems in place to,
where necessary, amend guidance before the planned expiry date
and to alert healthcare professionals and patients on such changes
- for example, to revise the text of guidance in the light of
new or reconsidered evidence or other information, or where errors
or omissions have been identified.
2.3.3 NICE must also ensure that it has
systems in place to respond to issues and questions raised by
healthcare professionals and patients about its guidance.
2.4 Transparency
2.4.1 There is an urgent need for NICE to
be more transparent about the appraisal process. Lack of clarity
and openness about how decisions are taken about clinical effectiveness
and cost-effectiveness and the weight that the Institute gives
to different kinds of evidence seriously undermine the credibility
and authority of NICE. This should be dealt with as part of the
recommended review of the NICE appraisal process.
2.5 Resources
2.5.1 Patient groups make considerable efforts
to contribute to and influence the way that health technology
appraisals are undertaken. Resources should be made available
to support this contribution—either resources provided directly
from NICE or from other government allocations.
3. INTRODUCTION
3.1 Consumers' Association (CA) is an independent
organisation that works on behalf of consumers to achieve improvements
in the quality of goods and services. CA's key health objectives
are to ensure that the public has access to information to make
informed decisions about health services and policy and to campaign
for positive change in those areas where we have concerns. These
objectives are supported through health policy research, reports
published in consumer magazines Which? and Health Which?
and also, more recently, through the publication of Treatment
Notes, a bulletin designed to give patients access to independent
and rigorous reviews of the effectiveness of drugs and other treatments.
Treatment Notes is produced in conjunction with Drug
and Therapeutics Bulletin (DTB).
3.2 Drug and Therapeutics Bulletin
(DTB) is an independent monthly journal circulated to over
125,000 doctors and pharmacists throughout the UK. It aims to
provide such healthcare professionals with impartial information
on treatment choices and the management of disease. Its advice
is intended to be clear, unambiguous and usable. To this end,
such guidance is based, wherever possible, on well-designed, appropriately
conducted randomised controlled clinical trials published in full,
is informed by the views of professional organisations, consumers
and patients, and takes account of UK law and NHS policy. In producing
its reviews, DTB scrutinises and assesses published source
material, paying particular attention to the reliability of such
information.
4. CONSUMERS'
ASSOCIATION INTEREST
IN NICE
4.1 Consumers' Association (CA) has had
a long-standing interest in medicines policy in relation to questions
of effectiveness, access, and information. Drug and Therapeutics
Bulletin (DTB) plays an important role in helping prescribers
by providing information about the effectiveness of treatments.
As part of its role in informing healthcare professionals, DTB
has produced reviews of various treatment areas on which the National
Institute for Clinical Excellence (NICE) has previously or subsequently
issued advice in the form of Technology Appraisal Guidance. It
is with reference to this material, that DTB offers evidence
relating to the terms of reference of the Health Committee Inquiry.
4.2 CA also deals with broader policy issues
about medicines from a patient perspective. We have a particular
interest in user involvement at all levels of decision-making,
from decisions about individual treatment through to involvement
in wider policy decisions about which specific treatments are
provided on the NHS and why. CA has researched and published reports
on the Patient Information Leaflets (PILs) that are included with
all prescription drugs and on promotion of prescription drugs
directly to patients. More recently we published a report on the
outcome of a Consumers' Association inquiry into how well NICE
works from the patient perspective. This submission to the Health
Committee also includes evidence relating to the outcome of the
CA inquiry.
4.3 This following evidence is presented
in two parts. The first part draws on the expertise and experience
of DTB in relation to NICE. The second part of this evidence
deals specifically with the outcome of the CA inquiry into how
well NICE performs from a patient and patient group perspective.
Part 1
DTB EVIDENCE
5. NICE TECHNOLOGY
APPRAISAL GUIDANCE
TOPICS REVIEWED
The evidence presented below focuses on nine
of the Technology Appraisal Guidance documents that NICE has issued
so far. These collectively cover six separate treatment areas:
— Type 2 diabetes mellitus3,4
— Attention deficit hyperactivity disorder
(ADHD) in children5
DTB has published advice relating to
each of these treatment areas.10-19
6. ISSUES RELATING
TO NICE GUIDANCE
ON TREATMENT
FOR PATIENTS
WITH INFLUENZA
6.1 In 1999, NICE concluded that the drug
zanamivir should not be used in the NHS to manage patients with
influenza,1 a conclusion independently reached by DTB.10
After reviewing further evidence, NICE revised its position in
November 2000, when it recommended that, under specific circumstances,
zanamivir should be available for treating a defined group of
individuals whose age or underlying medical problems puts them
at particular risk of experiencing more prolonged and/or severe
illness or developing complications as a result of influenza virus
infection (referred to in the guidance as "at-risk patients").2
DTB disagreed with this revised NICE guidance having reviewed
the evidence on which it was based.11 Furthermore, DTB's
assessment of the revised guidance (and the associated Health
Technology Assessment report commissioned by NICE,20 the Supplement
to this report21 and the guidance's information for patients2)
raised other serious concerns about the credibility and usability
of NICE's recommendations on the use of zanamivir.22
6.2 The revised NICE guidance appears to
be based primarily on an analysis of data pooled from six randomised
controlled clinical trials that tested zanamivir against a placebo
treatment. In all, these trials involved around 800 at-risk patients.
The guidance makes three key statements about the effects of zanamivir:2
— "In the overall pooled analysis
of at-risk individuals, the duration of symptoms of influenza
is reduced by 1.2 days (95 per cent CI 0.1 to 2.2) from about
6 to 5 days in the ITTP [intention-to-treat population]."
— "Overall zanamivir reduced the
absolute risk of complications requiring antibiotics in the ITTP
by 6 per cent (95 per cent CI: 0 to 11 per cent)."
— "No reliable data are available
as to the impact of the use of zanamivir on hospitalisation rates
or mortality."
However, crucial weaknesses in the way the evidence
was collated, interpreted, extrapolated and ultimately used undermine
the credibility of the revised guidance.
6.2.1 There are several important limitations
in the trials chosen for inclusion in the pooled-data analysis,
which make the robustness of the analysis highly questionable.
First, most of these trials were not designed specifically to
assess the effects of zanamivir in "at-risk" patients.
Indeed, such individuals formed only a minority of those recruited
into these trials. Also, since these patients were from separate
studies, there are dangers in assuming they constitute a representative
or typical group of "at-risk" people. Only one of the
trials included in the pooled analysis specifically recruited
"at-risk" patients (in this case, individuals with asthma
or chronic obstructive pulmonary disease). There is considerable
doubt as to the usefulness of the data from this trial for assessing
whether use of zanamivir in the NHS will prevent serious complications
of influenza in "at-risk" patients.22 A key problem
is that the participants of the trial are unrepresentative of
"at-risk" groups generally in the UK. For instance,
the study included only patients with respiratory conditions (primarily
asthma), generally of mild to moderate severity. In addition,
relatively few of these people (under 10 per cent) were aged over
65 years, and overall only 23 per cent had been vaccinated against
influenza.22 (At the time of publication of the revised NICE guidance,
over 60 per cent of people aged over 65 years had been vaccinated
against influenza.) Finally, the data-pooling process combined
results from this trial with the data from the (also potentially
unrepresentative) subgroups of at-risk patients in other trials.
6.2.2 The pooled-data analysis reported
in the revised NICE guidance also has other weaknesses. It considers
zanamivir's effects on "complications requiring antibiotics"
as key evidence of the drug's benefits.2 However, it is questionable
whether an effect on such complications is a reliable guide as
to whether the drug helps reduce the risk of being hospitalised
or dying as a result of influenza infection.11,22 This is partly
because "complications requiring antibiotics" could
be relatively minor problems (eg sinusitis or tracheitis) and
it is not clear from the revised guidance (or its supporting literature)
what proportion of the patients treated with zanamivir and control
groups had more severe complications (eg pneumonia). Also, it
is not clear exactly what criteria were used for starting antibiotic
therapy in the trials from which the evidence was collated. So,
in some cases, such treatment might have been started for relatively
minor illnesses. Also, none of the trials in the pooled analysis
were designed to look primarily at zanamivir's effects on "complications
requiring antibiotics". In addition, zanamivir's effect on
"complications requiring antibiotics" varied considerably
from trial to trial (as acknowledged by the authors of the NICE
Health Technology Assessment report.21) This variation further
limits the robustness of the pooled-data analysis. Furthermore,
the data analysis shows that the measured effect of zanamivir
on complications requiring antibiotic use (an apparent 6 per cent
reduction in the rate of such complications) did not reach conventional
levels of statistical significance. The analysis is therefore
not only flawed, but provides no evidence that zanamivir prevents
the serious complications of influenza in at-risk patients.
6.2.3 The pooled analysis also suggested
that, in at-risk patients, zanamivir reduced the duration of symptoms
of influenza by around one day, a benefit similar to that reported
for patients in general.1,10 NICE does not see this limited effect
as a compelling reason for prescribing zanamivir for "otherwise
healthy" people.2 So, it is questionable whether this benefit
alone would be a convincing basis for recommending the drug for
at-risk patients.
6.2.4 The revised NICE guidance2 (and the
associated Health Technology Assessment report20and its Supplement21)
reached conclusions and offered advice that are hard to justify
given the inherent weaknesses in the underlying evidence. First,
the claim in the revised guidance that zanamivir reduces the risk
of "complications requiring antibiotics" is not justified
by the data presented. The various flaws in the data also undermine
the Supplement's suggestion that "it seems reasonable to
suppose that, if we can prevent 6 per cent of the at risk population
from needing antibiotics, more probably than not, we may also
be preventing some deaths." 21,22 Furthermore, the revised
guidance cites an economic evaluation2 (published in the Supplement21)
that estimates the cost-effectiveness of zanamivir by assuming,
inappropriately, that the drug would produce a 6 per cent fall
in death rates from influenza among at-risk patients; the evaluation
is therefore flawed.
6.3 The revised NICE guidance does not mention
how it conflicts with other current NHS advice and policy. The
guidance raises such conflicts by, for example, not including
any advice on use of symptom-relieving medication (eg paracetamol)
in at-risk adults with symptoms of influenza; omitting patients
with chronic renal failure from its definition of "at-risk
patients" (ie those for whom the Department of Health recommends
annual influenza vaccination); and ignoring relevant concerns
about microbial resistance detailed in advice from the NHS Executive
on the use of Patient Group Directions.2,22 The guidance's failure
to acknowledge, explain or address such conflicts could lead to
confusion among healthcare professionals and patients, seriously
limit the credibility of the guidance and hinder its ability to
become locally owned and trusted.
6.4 The revised NICE guidance also omits
other important treatment information relating to the use of zanamivir.
For instance, it does not mention possible unwanted effects of
zanamivir, in particular, serious respiratory symptoms and how
these might be prevented (as detailed in the drug's summary of
product characteristics). 2,22,23 In addition, the guidance is
at odds with the current patient information leaflet supplied
with zanamivir. 2,24 This leaflet recommends that before using
the drug, patients should talk to their doctor if they have "been
told that your immunity to infections is poor" or have "severe
or persistent asthma, or another long-standing lung disease".
By contrast, the advice to patients included in the revised guidance
suggests that these relative contraindications are the very reasons
for giving zanamivir and that the treatment could be provided
by a nurse or pharmacist without first contacting the patient's
doctor.2 These unexplained discrepancies could confuse both healthcare
professionals and patients and further reduce the credibility
and local ownership of the guidance.
6.5 It is worrying that the group commissioned
by NICE to prepare the Health Technology Assessment report on
zanamivir indicated that they did not have access to all the information
needed to fulfil this brief and suggested that the assessment
of zanamivir could not be done reliably in the time available.20
These criticisms of the procedure serve to undermine the validity
of the Assessment's conclusions and, potentially, the overall
appraisal of zanamivir. There were also weaknesses in the associated
information for patients issued by NICE.2,22 In particular, the
language in the text is likely to be inaccessible to many lay
readers and the advice was inaccurate in stating that nurses or
pharmacists might "prescribe" zanamivir to patients.
It therefore seems reasonable to question whether the text of
the patient information was checked for accuracy and field-tested
for reader accessibility.
6.6 Another concern is how NICE seems to
have exaggerated publicly the strength of evidence underlying
its revised guidance. When confronted with the suggestion that
some GPs might refuse to prescribe zanamivir, the Chairman of
the Institute was quoted as saying "I would like to remind
them that a group of highly expert individuals have looked at
this, which includes GPs. Just how will they feel when one or
two patients in the primary care group dies of flu?"22,25
As detailed above, such a statement would not be supported by
any reliable evidence gathered by NICE that zanamivir had an effect
on mortality rates.2,22
6.7 DTB detailed its various concerns
about the revised NICE guidance on zanamivir in an unpublished
report.22 A copy of this critique was sent to NICE, along with
an invitation to the Institute to discuss the issues raised. To
date, DTB has not received any formal reply to the points
made nor has NICE taken up the offer to discuss these issues.
7. ISSUES RELATING
TO NICE GUIDANCE
ON TREATMENT
FOR PATIENTS
WITH TYPE
2 DIABETES MELLITUS
7.1 NICE has recommended that the drugs
rosiglitazone and pioglitzone should be used in the treatment
of certain patients with type 2 diabetes mellitus.3,4 The relevant
guidance documents clearly state that each drug "should be
used in accordance with the manufacturer's recommendations".3,4
However, each document contradicts itself by suggesting that the
drugs be used in ways not specified by the manufacturer, even
though there seems to be no direct evidence to justify such use.
7.1.1 The manufacturers' recommendations
for pioglitazone and for rosiglitazone are that these drugs can
be given in combination with the drug metformin only in obese
patients, and in combination with a sulphonylurea drug in patients
who are intolerant of metformin or for whom this drug is contraindicated.3,4
Contrary to these recommendations, the NICE guidance also suggests
that patients whose blood glucose remains high despite an adequate
trial of metformin plus sulphonylurea combination therapy "may
be offered pioglitazone in combination with metformin or sulphonylurea
as an alternative to injected insulin."4 The guidance on
rosiglitazone offers a much stronger recommendation, suggesting
that such patients "should be offered rosiglitazone combination
therapy as an alternative to injected insulin."3 The guidance
documents neither cite any direct supporting evidence nor offer
any convincing rationale for suggesting these unlicensed uses
of pioglitazone and rosiglitazone. (DTB could find no published
evidence to support these wider, unlicensed recommendations, when
producing its own review on pioglitazone and rosiglitazone.12)
The NICE recommendations therefore lack credibility.
8. ISSUES RELATING
TO NICE GUIDANCE
ON TREATMENT
FOR ATTENTION
DEFICIT HYPERACTIVITY
DISORDER (ADHD) IN
CHILDREN
8.1 NICE has recommended the drug methylphenidate
as part of a comprehensive treatment programme for children with
severe ADHD.5 However, the NICE guidance is of limited value because
it makes no attempt to advise on use of the other licensed drug
treatment for this condition—dexamfetamine [previously known
as dexamphetamine]. As a result, healthcare professionals have
no way of knowing from the guidance whether NICE intends that
dexamfetamine should be used as an alternative to methylphenidate,
reserved for only certain (undefined) groups of children or not
used at all. These concerns are particularly pertinent given that
the available evidence suggests that there is little difference
in effectiveness between methylphenidate and dexamfetamine13 (as
noted in the associated Health Technology Assessment report commissioned
by NICE26). The two drugs also have similar unwanted effects.
In addition, it is worth noting that the drug costs to the NHS
of dexamfetamine are much lower than those of methylphenidate.
All these factors make the omission of any advice on dexamphetamine
particularly puzzling. They also suggest that by limiting itself
to a review of just one treatment, the NICE guidance fails to
answer key questions or to provide comprehensive and complete
advice on whether, when and how drug therapy should be used in
the NHS to manage children with ADHD.
8.2 The NICE guidance also makes no reference
to the fact that for some children with severe ADHD use of appropriate
non-drug therapy might be a suitable way of avoiding exposure
to drug therapy (for example, where parents are particularly keen
to avoid using such medication). Nor does the guidance mention
the evidence suggesting that combining medication with non-drug
treatments might help limit the drug doses needed to treat a child.13
These practical issues merit discussion in any guidance that attempts
to offer credible, practical information on optimal use of drug
treatment for children with ADHD.
9. ISSUES RELATING
TO NICE GUIDANCE
ON TREATMENT
FOR PATIENTS
WITH ALZHEIMER'S
DISEASE
9.1 NICE has recommended that the drugs
donepezil, rivastigmine and galantamine should be available in
the NHS as one component of the management of certain people with
Alzheimer's disease.6 There are reasons to question the evidential
basis for, and therefore the credibility of, this judgement.
9.1.1 The main available evidence on donepezil,
rivastigmine and galantine comes from randomised placebo-controlled
trials that have assessed the effects of these drugs on patients'
cognitive-function and global measures. However, there is considerable
doubt about how well such indicators reflect meaningful benefits
for patients and their carers, for example, improvements in mood,
behaviour and quality of life, and delaying the need for long-term
residential care. As stated in the Health Technology Assessment
report (commissioned by NICE and reviewing the clinical effectiveness
and cost-effectiveness of donepezil, rivastigmine and galantamine27),
"It is difficult to quantify benefits from the evidence available
in the literature. Statistically significant improvements in tests
such as ADAS.cog [a measure commonly used in trials] may not be
reflected in changes in daily life." The report also suggests
that "there is great uncertainty surrounding the cost-effectiveness
of these drug therapies" and goes on to conclude "The
implications of the use of donepezil, rivastigmine and galantamine
are unclear. The main issue is whether the modest benefits seen
in the outcome measures used in the trials would translate into
benefits significant to patients."27
9.1.2 The uncertainties about the evidence
on the effects of donepezil, rivastigmine and galantamine are
reflected in the NICE guidance. For example, in relation to the
cost-effectiveness of donepezil, rivastigmine and galantamine,
the guidance states that "From a health economics perspective,
the main benefit of these drugs is the improvement in patients'
cognitive and other functioning, and the main potential cost-saving
results from possible delayed progression to the requirement for
nursing home care. Neither can be reliably or easily estimated
from the existing trial evidence".6 Also in commenting on
economic evaluations submitted by the manufacturers of the three
drugs, the guidance states "The same caveats regarding the
quality of evidence apply to these estimates as they do to the
published studies: cost savings on later institutionalisation
are not well established, and quality of life is not easily measured."6
9.2 The guidance offers no other convincing
rationale for recommending the use of donepezil, rivastigmine
and galantamine in the NHS.
10. ISSUES RELATING
TO NICE GUIDANCE
ON TREATMENT
FOR PATIENTS
WITH MOTOR
NEURONE DISEASE
10.1 NICE has recommended the use of the
drug riluzole for the treatment of patients with the amyotrophic
lateral sclerosis (ALS) form of motor neurone disease.7 There
are reasons to question the evidential basis for, and so the validity
of, this advice.
10.1.1 It is worth noting the conclusion
of the Health Technology Assessment report28,29 (commissioned
by NICE as part of the drug's appraisal) which reviewed the clinical
effectiveness and cost-effectiveness of riluzole: "There
is limited evidence of a modest benefit in tracheostomy-free survival
for patients taking riluzole. However, the evidence base is restricted
and there remains uncertainty as to the true benefit of riluzole;
the confidence interval is wide and compatible with little or
no difference between riluzole and placebo. When costs and health
economic impact extrapolating survival beyond that observed in
trials are considered, the uncertainty about whether the benefits
are worth the costs is magnified. Even under the most optimistic
assumptions, riluzole at best only postpones death for a few months,
and does not preclude the need for supportive care and practical
help." and "Consequently, existing evidence on effectiveness
and cost-effectiveness does not unequivocally indicate the best
policy concerning use of riluzole in ALS for the NHS."
10.1.2 The NICE guidance appears to make
errors in stating that "The assessment report reviewed the
results from all four of the trials identified and reported riluzole
to be associated with a relative reduction in hazard ratio for
tracheostomy-free survival at 18 months of 17 per cent (ie hazard
ratio of 0.88, 95 per cent CI: 0.77-1.02".7 In reality, the
statistical analysis of these four trials28,29 suggests that riluzole
therapy was associated with a 12 per cent (not 17 per cent) lower
risk of dying or requiring tracheostomy (hazard ratio 0.88, 95
per cent CI: 0.75-1.02). Also, there is considerable statistical
uncertainty about this effect, since the associated confidence
interval (95 per cent CI: 0.75-1.02) is compatible with riluzole
being more effective, no different from or slightly less effective
than is placebo, in prolonging a patient's life or reducing the
need for tracheostomy. This point could be crucial for prescribers,
patients and carers considering use of the drug, and should have
been spelt out clearly in the guidance. By omitting such an explanation,
the guidance gives the impression that riluzole definitely extends
the period patients survive without needing a tracheostomy. In
reality, the available evidence is much more equivocal.
11. ISSUES RELATING
TO NICE GUIDANCE
ON TREATMENT
FOR OBESITY
IN ADULTS
11.1 In March 2001, NICE issued guidance
recommending the drug orlistat as part of the management for adult
patients who are obese.8 Subsequently, in October 2001, NICE issued
guidance recommending another drug—sibutramine—also
for use in managing obesity in adults.9 The Technology Appraisal
Guidance document for sibutramine mentions that NICE has previously
issued guidance on orlistat.9 However, this document gives no
advice on how the two drugs should be used in relation to each
other. For example, the guidance offers no views on whether NICE
is recommending that the drugs should be regarded as interchangeable
treatment choices, whether the use of each drug should be restricted
to certain (undefined) groups of patients or whether they can
(or should) be used sequentially in some individuals. The guidance
therefore does not seem to offer clear, integrated advice on the
use of these two treatments in the overall management of patients
who are obese. As a result, it is possible that individual healthcare
professionals and service providers will have to formulate their
own policies on selecting between and using the two treatments:
this could lead to marked variation in practice throughout the
NHS. This would be counter to NICE's remit to end confusion by
providing a single national focus of advice.
11.2 The NICE guidance on sibutramine does
not mention some of the practical limitations that could make
the drug difficult to use or which limit its value. Examples of
these are most of the drug's important contraindications, which
include conditions that commonly accompany obesity, such as coronary
heart disease, peripheral artery disease, congestive heart failure
and cerebrovascular disease. Similarly, the guidance does not
refer to how various interactions with other drugs could complicate
use of sibutramine. (DTB recently recommended against using
sibutramine, having considered not only these practical factors,
but also the limited efficacy of the drug, concerns about some
of the pivotal evidence supporting its use, the risk of treatment-induced
rises in blood pressure and heart rate, and the stringent requirements
for monitoring during treatment.19 )
12. CONCLUSION
12.1 There are significant weaknesses in
current NICE guidance on various health technologies. In some
cases, the brief of such guidance appears too narrowly drawn,
for example, when it covers only one of several treatment options
for a given condition. There is also evidence of failure to integrate
new NICE guidance with other current advice issued by the Institute.
In some cases, guidance has appeared counter to other NHS advice
or policy or other standard advice available to patients and healthcare
professionals (in summaries of product characteristics and patient
information leaflets), without acknowledging, explaining or justifying
such conflicts. This suggests that guidance is not always sensitive
to the clinical context in which it is to be used. The evidential
basis for some of the Institute's recommendations appears weak
and, in these cases, no convincing alternative rationale for the
advice is offered. Another concern is the evidence of inaccurate
interpretation or inappropriate extrapolation of data underlying
some of the Institute's recommendations. When directly challenged
on such issues, NICE has seemed either unwilling or unable to
debate the points raised, to offer a clear explanation for some
of its assessments or, where appropriate, to revise the text of
its guidance.
12.2 The various problems with current NICE
guidance on health technologies raise concerns about the clarity
and credibility of such advice, and about whether, in general,
the Institute is promoting health technologies for which there
are convincing grounds for recommending use in the NHS. Such deficiencies
may well discourage local ownership of NICE guidance and increase,
rather than decrease, confusion among healthcare professionals
and patients about the appropriate use of different treatments
and the optimal management of different conditions. All of the
issues raised need addressing, through a thorough review of the
NICE appraisal process, if the Institute is to be regarded as
a trusted source of sound and usable healthcare advice.
Part 2
CA PATIENT-CENTRED
INQUIRY INTO
NICE EVIDENCE
13. The idea to hold a public inquiry developed
from CA's work on the promotion of prescription drugs, through
which we became aware that the Department of Health was planning
to review NICE. Concerned that the interests of patients would
not be fully considered in such a review, CA decided to hold a
patient-centred inquiry of its own. The CA inquiry set out to
consider the perspective and experience of patient groups in relation
to how well NICE performs.
13.1 The CA inquiry was set up along the
lines of a select committee. Patient groups with an interest in
NICE were invited to submit written evidence. Issues raised in
the written evidence were used to develop lines of questioning
for each of the four evidence sessions held on the day of the
inquiry. The following paragraphs highlight the key findings and
conclusions that CA derived from the day. Copies of the full inquiry
report and briefing are available from CA.
14. QUALITY OF
LIFE
14.1 One crucial question is how quality-of-life
measures—measures that relate to how a treatment impacts
on a patient's life in ways that are important to them—are
identified, evaluated and taken into account in the NICE appraisal
process. Evidence provided to the inquiry from patient organisations
as well as from NICE itself highlighted deficiencies in the availability
of this kind of information in relation to specific treatments.
(Another more difficult question is how quality of life measures
can be evaluated for treatments not yet on the market or widely
used.)
14.2 Evidence provided on the day of the
inquiry would suggest that in the absence of formal data, patient
organisations do their best to fill the gap with information about
the views and experiences of the patients that they represent.
This information is valid in its own right and can provide valuable
insight into what is important to those who might benefit from
treatments. In addition, some patient organisations, like the
Alzheimer's Society, have conducted extensive research into quality
of life issues with their members. However, few patient organisations
have the resources or expertise to undertake full and proper research
in this area—and none that took part in the inquiry had
been set up with this role in mind.
15. IMPACT OF
TREATMENTS IN
DAY-TO-DAY
USE
15.1 Linked to this is the issue of information
about how treatments perform in routine day-to-day clinical practice
- circumstances that often differ from the parameters of clinical
trials. The evidence presented at CA's inquiry indicates a need
to ensure consideration of evidence about how treatments are actually
prescribed and used—in ways that may be different from the
circumstances considered in clinical trials. For example, patients
with long-term medical conditions like multiple sclerosis may
use treatments as their condition worsens—but not all of
the time. This may be crucial when it comes to assessing the cost
implications of treatments.
15.2 There was general agreement that information
that relates to both quality of life and how treatments work in
day-to-day practice is essential to enable proper and meaningful
appraisals of treatments. Evidence from NICE suggests that manufacturers
are starting to take this information need into account in their
approach to undertaking pre-market research and clinical trials.
15.3 However, this still raises a number
of questions in relation to identifying the outcome measures relevant
to the NICE appraisal process and who should be responsible for
ensuring that the necessary information is collected and assessed.
16. WEIGHT OF
EVIDENCE
16.1 For many patient groups, there is an
underlying question about the weight accorded to different kinds
of evidence in the appraisal process. Almost every patient organisation
that took part in CA's inquiry reported issues about the hierarchy
of evidence considered by NICE and lack of clarity about the weight
given to different kinds of evidence. Most groups believe that
evidence from patient groups is viewed as less important than
other kinds of evidence. Mostly, patient groups have concerns
about the lack of feedback about how their evidence is used—and
specifically what difference their evidence made in relation to
the outcome of the appraisal.
16.2 In part, this may be the result of
confusion and lack of clarity about exactly what question NICE
is trying to answer or the precise nature of the outcome that
it is trying to assess in the course of an appraisal. Overall,
there is a strong case to be made for NICE being much clearer
at the outset of an appraisal about what exactly it is trying
to evaluate and, following that, a great deal more open and explicit
about the methodology that it is using. This could include, for
example, being clear about the questions that need to be answered
and the relative importance of those answers in relation to the
outcome of the appraisal.
17. TRANSPARENCY
17.1 This leads directly to the question
of transparency. As well as the need for NICE to be more explicit
about the kind of evidence that is required, and how different
evidence is measured and weighed, our inquiry also highlighted
a need for more openness and transparency about how effectiveness
is defined and decisions about cost are dealt with. It was not
within the remit of the inquiry to challenge the issue of cost
of treatments with NICE, but it clearly emerged as an area where
there is much confusion and debate. For example, during the inquiry
it was not possible to establish how decisions are taken about
where or how NICE draws the line on whether a treatment is cost-effective.
Nor was it possible to pin NICE down on costs per qualy.
18. PROCESSES
18.1 Many of the above issues deal with
the question of openness and transparency, and these are linked
to the way that NICE works and the process by which appraisals
are undertaken. Patient organisations taking part in CA's inquiry
were almost unanimous in their view that, although there is more
to be done, there has been a continuous improvement in the way
that NICE works with patient groups. Research commissioned by
NICE from the King's Fund on improving the involvement processes
should also go some way to improve the way that NICE engages with
patients and patient groups.
18.2 On the other hand, there remain a number
of very real problems with the appraisal process that make it
difficult for patient organisations to take part. Issues about
confidentiality seem to present particular problems for organisations.
For example, a number of organisations reported that confidentiality
clauses made it difficult for them to consult with their patient
or patient group members on appraisals. Many organisations also
reported that the timeframes that NICE works within during an
appraisal process are often too short to allow them to contribute
in a meaningful way.
19. RESOURCES
19.1 Patient organisation clearly have an
important role to play in the NICE appraisal process, both in
providing specific information about effectiveness from the user
perspective and ensuring that the patient and public interest
is properly represented in the overall workings of NICE. However,
the patient organisations that took part in the inquiry all reported
that contributing to the NICE appraisal process consumed significant
resources. In one case the estimated cost was £40,000, while
in another, the resource required was a dedicated member of staff
working over a three-month period. None of these groups receive
financial support for this work.
19.2 This highlights a particularly difficult
issue for patient groups in relation to ensuring that the interests
of their stakeholders are taken into account in the NICE process,
while at the same time meeting their other commitments and obligations.
For many patient groups, there is a very real issue about diversion
of funds and other resources from core charitable activities.
It is especially difficult where organisations are making concerted
efforts to gather evidence from patients about their experiences
and priorities that would not otherwise be captured or taken into
account in the appraisal process. There is a need for serious
consideration to be given to ensuring that, where organisations
are collecting information pertinent to the appraisal process,
this work is properly supported and funded. However, there is
also a need for NICE to be prepared to commit resources to commission
formal research in those areas where patient organisations have
anecdotal evidence about the impact of treatments from the user
perspective.
20. OUTCOMES
20.1 From the outset of this inquiry, it
was clear that there are issues linked to NICE but that are important
to patients. In particular, issues that fall on either side of
the NICE appraisal process—for example, the selection of
technologies for appraisal and implementation of guidance across
the health service. This inquiry did not tackle the question of
selection for topics. However, we did consider the question of
patient access to treatments.
20.2 The discussion about outcomes highlighted
two key concerns. First, anecdotal evidence would suggest that
not all NICE guidance is implemented consistently across the health
service. For example, some health authorities make treatments
available but will use different criteria or higher thresholds
for access to those treatments than those set out in NICE guidance.
Secondly, some groups expressed concern that while appraisals
are underway patients are sometimes denied access to the treatment
under review, something that is not supported by formal government
policy.
20.3 It defeats the object of NICE if patients
are being denied access to treatments that have been found to
be effective—or which are under review. In this way, we
welcome the requirement that patients be given access to treatments
that have been found by NICE to be clinically and cost effective.
However, there is a very important issue about ensuring that resources
are not diverted from providing treatments that have not been
subject to NICE guidance in order to meet such requirements.
21. CONCLUSIONS
AND RECOMMENDATIONS
21.1 CA's patient-centred inquiry raised
a number of key questions for NICE. Many of these centre around
the need for clarity, in terms of how effectiveness is defined,
the value of different kinds of evidence and where that evidence
fits into the overall appraisal.
21.2 There is also a need for clarity, or
at a minimum an open debate, about what information is needed
to ensure a meaningful appraisal of a treatment. Currently, it
remains unclear as to whether NICE attempts to base appraisals
on the best available information or on the information that is
needed to make the best decision. The failure of NICE to be as
open and transparent as it should be about how judgements about
treatments are actually made makes these problems worse.
21.3 Patient organisations clearly have
an important role to play in the NICE appraisal process—both
in providing specific information about effectiveness from a patient
perspective and to ensure that the patient and public interest
is properly served. There is an urgent need to consider how patient
groups are supported and funded to undertake this important work.
22. SUMMARY OF
FINDINGS OF
THE PATIENT-CENTRED
INQUIRY INTO
NICE AND CA RECOMMENDATIONS
FOR CHANGE
IN THIS
AREA
22.1 A review of NICE has been on the agenda
for some time, with much debate about how and by whom the review
should be undertaken. Both industry and government have a number
of key issues that they want the review to consider. CA also believes
that the time is right for a review of NICE. We believe that it
is critical that the patient and patient group perspective is
taken into account in this review. The terms of reference must
be made public and widely consulted on—and the review must
be carried out by an independent agency.
22.2 There are a number of immediate steps
that NICE could take to eliminate some of the confusion and misunderstanding
that currently surrounds the appraisal process. Appraisal methodologies
should be explicit and made public, along with details about the
basis upon which decisions about specific treatments have been
taken. There is also a need for a thorough and open discussion
about the kind of information that is required to ensure a meaningful
appraisal process. Processes for identifying gaps in the availability
of information should be introduced—and, where possible,
measures taken to fill such gaps.
22.3 Patient groups spend considerable time
and resources to ensure that the patient voice is heard in the
NICE appraisal process. However, not all patient groups have the
resources to support this work and, for many, contributing to
NICE means diverting resources away from charitable activities.
It is therefore essential that patient groups are properly funded
and supported to enable them to effectively contribute to NICE.
It is also essential, where there are no patient groups to represent
the interests of a particular patient constituency, that measures
are taken to obtain the patient perspective.
22.4 At the same time, it is important that
patient groups themselves are clear about why they are contributing
to the NICE appraisal process. For example, there is a difference
between working to ensure that NICE operates in a way that is
open and transparent and which takes into account what is important
from the patient perspective, and participating in the collection
and analysis of that data. In some cases, patient organisations
are lobbying to ensure access to treatments in a general sense;
in other cases, groups may have access to clinical information
that is relevant to the appraisal. All of these functions and
roles are legitimate, but it is important that and patient groups
themselves are clear about the nature of the contribution. This
should help to eliminate the need for patient groups to try to
fill information gaps when they may not be in a position to do
so. It should also help NICE to be clearer about identifying missing
information and how such information can be best collected.
REFERENCES
1. National Institute for Clinical Excellence.
Guidance to the NHS on zanamivir (Relenza) in the treatment of
influenza. Technology Appraisal Guidance. October 1999. (Subsequently
published as Old Guidance—Zanamivir. Available: http://www.nice.org.uk
[February 2000]).
2. National Institute for Clinical Excellence.
Guidance on the use of zanamivir (Relenza) in the treatment of
influenza. Technology Appraisal Guidance—No 15. Available:
http://www.nice.org.uk [21 November 2000].
3. National Institute for Clinical Excellence.
Guidance on the use of rosiglitazone for type 2 diabetes mellitus.
Technology Appraisal Guidance—No 9. Available: http://www.nice.org.uk
[August 2000].
4. National Institute for Clinical Excellence.
Guidance on the use of pioglitazone for type 2 diabetes mellitus.
Technology Appraisal Guidance—No 21. Available: http://www.nice.org.uk
[March 2001].
5. National Institute for Clinical Excellence.
Guidance on the use of methylphenidate (Ritalin, Equasym) for
attention deficit/hyperactivity disorder. Technology Appraisal
Guidance—No 13. Available: http://www.nice.org.uk [October
2000].
6. National Institute for Clinical Evidence.
Guidance on the use of donepezil, rivastigmine and galantamine
for the treatment of Alzheimer's disease. Technology Appraisal
Guidance—No 19. Available: http://www.nice.org.uk [January
2001].
7. National Institute for Clinical Excellence.
Guidance on the use of riluzole (Rilutek) for the treatment of
motor neurone disease. Technology Appraisal Guidance—No
20. Available: http://www.nice.org.uk [January 2001].
8. National Institute for Clinical Excellence.
Guidance on the use of orlistat for the treatment of obesity in
adults. Technology Appraisal Guidance—No 22. Available:
http://www.nice.org.uk [March 2001].
9. National Institute for Clinical Excellence.
Guidance on the use of sibutramine for the treatment of obesity
in adults. Technology Appraisal Guidance—No 31. Available:
http://www.nice.org.uk [October 2001].
10. Zanamivir for influenza. DTB 1999;
37: 81-4.
11. Why not zanamivir? DTB 2001;
39: 9-10.
12. Pioglitazone and rosiglitazone for diabetes.
DTB 2001; 39: 65-8.
13. Stimulant drugs for severe hyperactivity
in childhood. DTB 2001; 39: 52-4.
14. Donepezil for Alzheimer's disease? DTB
1997; 35: 75-6.
15. Donepezil update. DTB 1998; 36:
60-1.
16. Rivastigmine for Alzheimer's disease.
DTB 2000; 38: 15-6.
17. Riluzole for amyotrophic lateral sclerosis.
DTB 1997; 35: 11-2.
18. Why and how should adults lose weight?
DTB 1998; 36: 89-92.
19. Is sibutramine more than a slim hope?
DTB 2001; 39: 89-91.
20. Burls A, Clark W, Stewart T et al. Zanamivir
for the treatment of influenza in adults. Health Technology
Assessment Report. Available: http://www.nice.org.uk [12 December
2000].
21. Burls A, Clark W, Stewart T et al. Zanamivir
for the treatment of influenza in adults. Supplement to the Assessment
Report. Available: http://www.nice.org.uk [12 December 2000].
22. Drug and Therapeutics Bulletin. Report
detailing issues of concern raised by the November 2000 Guidance
on the use of zanamivir (Relenza) from the National Institute
for Clinical Excellence. 12 February 2001.
23. Relenza. Summary of product characteristics,
UK. GlaxoWellcome UK Limited. [First authorisation, June 1999;
partial revision of text, May 2001].
24. Relenza. Patient information
leaflet. GlaxoWellcome UK Limited. May 2001.
25. Browne A. GPs warned as they snub
advice on anti-flu drug. The Observer; 10 December 2000.
26. Lord J, Paisley S. The clinical effectiveness
and cost effectiveness of methylphenidate for hyperactivity in
childhood. London: National Institute for Clinical Excellence,
Version 2, August 2000.
27. Clegg A, Bryant J, Nicholson T et al.
Clinical and cost effectiveness of donepezil, rivastigmine and
galantamine for Alzheimer's disease s. Health Technology
Assessment Report. Available: http://www.nice.org.uk [January
2001].
28. Stewart A, Sandercock J, Bryan S, Hyde
C, Fry-Smith A, Burls A. The clinical effectiveness and cost effectiveness
of riluzole for motor neurone disease. Health Technology Assessment
Report. Available: http://www.nice.org.uk [January 2001].
29. Bryan S, Barton P, Burls A. The clinical
effectiveness and cost effectiveness of riluzole for motor neurone
disease - An update. Health Technology Assessment Report. Available:
http://www.nice.org.uk [January 2001].
| 
