Recommendations and their Basis
1. CRITERIA FOR
ASSESSMENT
The key technology assessment dimension of NICE
is "the balance of clinical and cost effectiveness"
which is a composite of two elements.
Clinical effectiveness
Clinical effectiveness differs from clinical
efficacy as it incorporates the notion of how a drug works in
real life, rather than the constrained observational environment
of a clinical trial being run according to a strict protocol.
It also includes the expanded notion of tolerability, which includes
not only the classical side effect profile but also the broader,
specific and general impact on a patient's quality of life.
Consideration of clinical effectiveness rather
than clinical efficacy is sensible but new. To date clinical effectiveness
has not been the subject of pharmaceutical development in this
country or abroad. It is still the case that the development of
pharmaceutical products is driven by the needs of regulatory authorities
such as the Medicines Control Agency here in the UK and the FDA
in the USA and the EMEA in Europe and not the broader considerations
being evaluated by NICE. To improve clarity, pertinence and ease
of regulatory assessment these agencies publish detailed criteria
for the assessment of different types of therapy. These criteria
have been developed following careful consultation with all relevant
expert groups.
As expected, the introduction of a new assessment
discipline has meant that much of the detail relating to which
criteria to use, and which methodologies should be applied to
generate the Institute's preferred assessment criteria, have been
made up in isolation of the stakeholders as the Institute's work
progressed. This has resulted in several anomalies where clearly
superior treatments, as judged by a broad consensus of stakeholders,
have been the subject of negative assessments.
In very many cases, the type of data that the
Institute wants to assess has, hitherto, never been generated
before, or it has not been appropriate to generate the type of
data that the Institute wants to see. Moreover the lack of data
requested by the Institute may never become available for good
scientific and/or ethical reasons related to for example, but
not limited to:
The time course of the disease
Some treatments can be easily assessed within
the time course of the disease, for example influenza. However,
delaying the assessment of chronic relapsing conditions like multiple
sclerosis, until the final outcome of treatment intervention is
known, which may be many decades later, is unethical. In this
case assumption modelling is required and this is totally reliant
upon making the correct assumptions about the relevant parameters
and criteria and then modelling them in a representative manner.
The natural history of the disease
Establishing the correct endpoints for a technology
assessment is vital. For example, the reason for treating diabetes
is to reduce the morbidity and mortality effects that correlate
well with chronic high blood sugar measurements. The aim of therapy
is not to reduce the blood sugar levels per se. Fortunately,
due to the published results of the seminal UKPDS study, the correlation
of blood sugar and morbidity is well understood. However, this
is a very special case and is not the norm for the vast majority
of other diseases.
Appropriate endpoints for study
When assessing the effects of cancer treatments
the Institute has been inconsistent in its approach as to which
endpoints it has seen fit to use and what weight to give them.
The scientific ideal will always be to use hard endpoints like
survival, but this is not ethically justifiable, particularly
as treatments are getting better and patients now increasingly
receive early, adjuvant or neo adjuvant treatment, which takes
a considerable time to impact survival because the better the
treatment, the longer people live. Denial of positive guidance
based on trends and "soft" surrogate endpoints like
time to disease progression or tumour regression response rates,
etc, is wrong and will not improve the health of the Nation. It
is well recognised by all that the UK languishes unacceptably
below the cancer survival levels of other comparable countries
in Europe and the developed world.
To date, the Institute and all stakeholders
have not been working to the same robust and transparent set of
relevant clinical effectiveness assessment criteria due in part
to ignorance and in others to a lack of consensus on what the
correct measures should be.
Cost Effectiveness
Health economics is a new and still relatively
embryonic social science. For reasons that are similar in substance
to those made for clinical effectiveness, it is imperative that
the methodologies used in this assumption based discipline are
appropriate, consistent and well understood before assessment
is contemplated.
Recommendations
1.1 The Institute, in concert with interested
parties, should develop and publish a clear set of criteria for
both the clinical and the health economic dimensions of assessment
for each specific type of technology or therapy area prior to
its assessment.
1.2 There should be an open consultation
process between stakeholders and the Institute to agree the relevant
criteria to use for a specific technology assessment.
— Accelerate the development of appropriate
methodologies to collect relevant data and the rapid establishment
of this new medical assessment discipline—"clinical
effectiveness".
— Reduce the time and resources needlessly
wasted by the Institute and relevant stakeholders redressing inconsistencies
and poor decisions made by the Institute.
2. PROCESS DURATION
The process is frequently too long and as a
consequence squanders health improvement opportunities for patients
and the NHS. From our own experience the assessment of products
for the treatment of multiple sclerosis is still ongoing for almost
two years. Similarly the assessment of treatments for colorectal
cancer is still ongoing after more than 14 months of consideration
and following two appeals to the Institute.
While an assessment is being considered patients
are denied access to modern medicines that have been assessed
as being of proven efficacy, safety and quality by the Medicines
Control Agency, because Health Authorities will not fund new treatments
in the knowledge that official guidance is on its way.
Due to the effective blight put upon a new technology
by the uncertainty of knowing when guidance will be issued it
is not possible to effectively commercialise the product. The
consequence of this is that the necessary funds to continue the
development and research of the product by either extending our
knowledge and understanding further, or researching new patient
groups, such as children or the elderly, or new indications is
being lost. The failure of the Aventis UK to capture research
investment is becoming increasingly clear. Aventis is not alone
in this respect within the pharmaceutical industry. Indeed the
issue of the UK's lack of research competitiveness is subject
of an ongoing Prime Ministerial initiative.
The Institute does not undertake appraisals
itself; instead it commissions others to do the work for it. Moreover,
we appreciate that as the Institute is still in its infancy it
may not yet have access to all of the necessary resources and
expertise that it needs to conduct the technology appraisals required
of it by the Department of Health. This being said, we are dismayed
that the Institute does not use the same third-party appraisal
organisations to perform similar types of appraisal. The institute
has assessed Aventis products for the treatment of three types
of cancer. Lessons learned from one assessment have not been used
in other assessments and as a consequence, different approaches
and, we contend, decisions have been adopted.
By way of example from our own experience unreasonable
delay has, and continues to occur in the assessment of two technologies.
Multiple Sclerosis
This case relates to the assessment of technologies
for the treatment of multiple sclerosis when the Institute failed
to undertake the correct of the analysis of the clinical and health
economic data. Following a successful appeal the assessment work
and health economic modelling was re-commissioned and assessment
started all over again.
Colorectal Cancer
In the second case, the appraisal of colorectal
cancer the Institute was in error of its own process and failed
to consult with stakeholders following a successful first appeal
that resulted in the distribution of a flawed Final Assessment
Determination that became the subject of a second appeal.
The assessment of Marketing Authorisation Applications
for new pharmaceutical products is governed by a well-documented
timeline. If an agency has questions of the applicant the assessment
clock is stopped while the applicant answers the agency's questions.
Recommendations
2.1 A clear and short timeline is needed
for each assessment marked with clear milestone events. We contend
that this should be no longer than 180 days, with the provision
for "clock-stop" periods if necessary.
2.2 We recommend that in the long run, the
Institute should resource itself adequately in order to be able
to conduct its own appraisals rather than commission third-party
organisations of varying quality.
2.3 In the short run, and if it is also
necessary in the long run for the Institute to continue to use
third-party assessment organisations, they should use the same
organisations to assess similar technologies. We contend that
this will improve self-learning by NICE and its third-party contractors
with benefits accruing to the quality of assessment and hopefully
the speed as well.
2.4 A mechanism for the provision of interim
guidance needs to be established to ensure that patients are not
denied access to treatments that they and their doctors consider
of benefit when the Institute has been either tardy or negligent
or both.
3. TRANSPARENCY
OF APPRAISAL
COMMITTEE DATA
USE AND
DECISION MAKING
The members of the Appraisal Committees are
not selected as experts in their own right for the technologies
that they are assessing. The broad skill base present in the constitution
of the various committees needs access to reliable expert opinion
in order to form a well-considered view.
The stakeholders to the process have a valuable
contribution to make to the overall assessment but are denied
access to the Appraisal Committee other than by a single submission
document at the start of the process. To date there is no mechanism
for stakeholders to help the Appraisal Committee in its work by
appearing before the Committee to explain and/or help clarify
complex points.
We have had numerous cases where the Committee
appear to have misunderstood complex points. This has resulted
in needless time being wasted coupled with increased patient anxiety
as Preliminary and Final Appraisal Determinations have had to
be appealed successfully in order to avert poor guidance being
issued to the NHS.
The Appraisal Committee only needs to ensure
that it has considered all relevant information set before it.
The Committee does not have to provide an explanation, or even
an insight into how it arrived at its decision. This lack of transparency
is the source of much needless confusion to all who read the guidance
with anything but a passing interest. It does not permit challenge
of decisions that may be erroneous. In the event of an appeal
the only matter that the Appraisal Committee has to satisfy the
Appeal Board on is that it considered the matter not how it has
considered the evidence or the relative weights that it has given
to various forms of evidence.
The test used in an appeal pertaining to the
content of the guidance issued by the Institute is the severe
test of "perversity" which we interpret to be the strict
test of Wednesbury unreasonable.
Recommendations
3.1 Introduce a consultative mechanism for
stakeholders to be able to address the Appraisal Committee in
the early stage of their work and after the first draft of the
Appraisal Determination has been issued.
3.2 Introduce a mechanism for the Appraisal
Committee to explain its decisions by publishing the reasons for
decisions and inviting oral questions with answers, in public,
as part of the consultation process.
3.3 The standard of decision making should
be that of a competent, informed body and not one that can only
be challenged if it can be shown to be wholly (Wednesbury) unreasonable.
4. THE DEPTH
AND BREADTH
OF EXPERT
OPINION AVAILABLE
TO THE
APPRAISAL COMMITTEE
The mechanism by which the Appraisal Committees
obtain expert opinion has been seriously flawed. In the ongoing
case of the assessment of colorectal cancer technologies the Committee
only had the benefit of one expert in the field. The Chairman
of the Appraisal Committee as being unacceptable subsequently
acknowledged this.
Recommendations
4.1 Publish the list of expert opinion that
the Appraisal Committee will refer to prior to the start of the
appraisal of the data and submissions by the Committee; inviting
additions and challenges to the experts selected.
4.2 As the Institute clears its backlog
of appraisals it will increasingly appraise cutting edge technologies.
Expert opinion should not be limited to UK experts but include
experts irrespective of their place of abode or clinical or scientific
practice.
4.3 The breadth and depth of expert view
should not be arbitrarily limited by number of experts but only
be limited by predefined scope of opinion required to achieve
a sound decision.
5. INDEPENDENT
APPEALS PROCESS
Currently all appeals of the Final Appraisal
Determination, which if unchallenged, is accepted by the Institute
and issued as Guidance, are heard by the Institute. This is not
necessarily a bad thing. However, we contend that transparency
and trust in the Institute's processes and decisions would be
better served by and independent appeals process that is not allied
to NICE or any other special health authority or establishment
of the Department of Health.
Recommendations
5.1 Establish an appeals process for Final
Appraisal Determinations and NICE Guidance that is independent
of the NICE executive.
5.2 Appeals should be possible on the merits
and substance of the decision and not limited to Wednesbury unreasonable:
"perverse".
6. CLARIFY THAT
THE OUTPUT
OF NICE IS
GUIDANCE NOT
DICTUM
There is considerable confusion as to the authoritative
basis of NICE guidance. Health managers with responsibility for
the financial management of the NHS institutions that they manage
consider that treatments that are not given positive NICE guidance
should not be financed. Clinicians are similarly of the view that
if there is no budget for a treatment they cannot use it, even
if it is clearly in the best interest of their patient and, indeed,
the NICE recommended treatment is in some way contra-indicated.
Recommendations
6.1 A clear footnote must be written into
all published guidance setting out that the document is only guidance
and that no patient should be disadvantaged by this guidance if
sound clinical reasons for deviating from it can be made by their
attending physician.
6.2 The nature of the content and process
to establish reasonable justification for using a treatment that
is not the subject of positive NICE guidance should not be so
onerous as to act as an effective barrier to the patient receiving
the treatment.
January 2002
| 
