APPENDIX 14
Memorandum by The NHS Centre for Reviews
and Dissemination (NC 65)
1. SUMMARY
1.1 NICE has worked hard to provide an infra-structure
for the production of guidance for the NHS. A small organisation
with a huge remit, it is to be applauded for aiming to be consultative
and transparent in its workings, while maintaining academic rigour
and delivering guidance.
1.2 As a credible and independent organisation,
NICE guidance needs to be based on high quality evidence from
research and to be seen as unbiased. However, NICE guidance may
lack credibility on several counts. Time constraints may adversely
affect the quality of the technology assessment reports (TARs)
that inform NICE guidance. There are also concerns about the transparency
of the Technology Appraisal Committee's decision-making process.
Moreover, despite its best efforts, NICE is hampered by the pharmaceutical
industries' concerns about the confidentiality of commercial documents.
1.3 NICE aims to provide a single national
focus for appraisal and guidance. It also has a remit to co-ordinate
and disseminate high quality evidence based information within
the NHS. Under pressure to develop quickly an infra structure
to support guidance development, NICE has yet to fully develop
its dissemination strategy. Its dissemination strategy is currently
under review. NICE needs to collaborate with other key stakeholders
in co-ordinating the dissemination and distribution of good clinical
and cost effectiveness information to the NHS and its consumers.
2. INTRODUCTION
2.1 The NHS Centre for Reviews and Dissemination
(CRD) was set up in January 1994, and is core funded by the DH,
through its R & D Division, to identify and review the results
of good quality research and to disseminate the findings to decision
makers (policy makers, health professionals and patients) in the
NHS. By producing and disseminating high quality systematic reviews
of health care interventions, CRD promotes clinical effectiveness
and the use of cost effectiveness information to inform best practice.
2.2 CRD is one of the six review teams producing
technology assessment reports (TARs) for NICE. These reports contain
clinical and cost effectiveness data to inform the NICE technology
appraisal process, which results in NICE guidance on new and existing
health technologies.
2.3 NICE also funds CRD to produce Effective
Health Care bulletins (EHCBs) and Effectiveness Matters (EMs).
These publications summarise the findings of systematic reviews
and are widely distributed in the NHS, to inform clinical decision-making.
The publications form part of the NICE dissemination strategy.
3. DOES NICE
PROVIDE CLEAR
AND CREDIBLE
GUIDANCE?
3.1 NICE has worked hard to provide an infra-structure
for the production of guidance for the NHS. A small organisation
with a huge remit, it has set up six National Collaborating Centres,
to assist in the development of clinical guidelines and audit.
There are also two new advisory units (the National Guidelines
Support and Research Unit and the National Guidelines and Audit
Patient Involvement Unit) and a Guidelines Advisory Committee.
These bodies will advise on and support the development of clinical
guidelines and future national audits.
3.2 NICE has also developed and reviewed
its technology appraisal process and has produced guidance documents
to assist those who contribute to the process, and thus to the
development of NICE guidance. In developing systems to underpin
the guidance process NICE is to be applauded for committing itself
to using rigorous methodology to develop evidence based guidance
in processes that involve health professionals, patients and carers.
The Institute aims to be consultative and transparent in its workings,
while maintaining academic rigour and delivering guidance.
3.3 As a credible and independent organisation,
NICE guidance needs to be based on high quality evidence from
research and to be seen as unbiased. However, NICE guidance may
lack credibility on several counts. Time constraints may adversely
effect the quality of technology assessment reports (TARs) that
inform NICE guidance at a national level. There are also concerns
about the transparency of the Technology Appraisal Committee's
decision-making process. Moreover, despite its best efforts, NICE
is hampered by the pharmaceutical industries' concerns about the
confidentiality of commercial documents. These concerns are outlined
in more detail below.
3.4 Time constraints. Utilising rigorous
methodology (1) CRD estimates that a high quality systematic review
takes an average of 12-18 months to complete. NICE aims for a
six month turnaround from commission to submission. Such timeframes
cause huge problems for CRD staff throughout the review process.
3.5 One of the key elements of good quality
systematic review is its searches, where the aim is to identify
as much relevant research as possible, to minimise the many forms
of publication bias. The search for studies and data gathering
is inevitably narrowed by time constraints, which means that important
research information may be missed.
3.6 The sheer volume of work within tight
deadlines places high demands on research staff who are committed
to maintaining high academic standards. Staff are concerned that
quality could be jeopardised in meeting such stringent deadlines.
3.7 The six month rapid review process includes
assessment of industry submissions. While NICE has prepared guidance
for manufacturers and sponsors in preparing a submission, drug
companies typically supply vast quantities of data, from which
it is often difficult to extract relevant data. There are concerns
that staff may miss important information, which could affect
research findings and thus NICE guidance.
3.8 Peer review, an integral component of
external quality assurance, is also affected by time constraints.
Both protocols and the completed TAR are peer reviewed. The quick
turnaround time means that both the research question and its
findings may not be subjected to the detailed scrutiny that is
desirable.
3.9 Recommendation: NICE guidance
would be more credible, if founded on a more extensive review
of the evidence base. Such reviews demands longer timescales for
completion of the work.
3.10 The Appraisal Committee's decision
making process. CRD recognises and welcomes the fact that NICE
aims to involve many stakeholders in the development of guidance.
CRD also recognises that NICE aims to be a transparent organisation.
Nonetheless, it is not clear how the submissions of various stakeholders
are weighted. On several occasions, CRD has noticed a disparity
between the research findings submitted to the Appraisal Committee
and that body's subsequent guidance.
3.11 For example, orlistat, an anti-obesity
drug, was approved by NICE for use in certain types of obese patients.
Although most of the studies reviewed showed results which were
statistically significant in favour of orlistat, average results
indicated that an orlistat-treated patient weighing around 100kg
at baseline would be likely to lose around 3kg more in body weight
compared with a patient receiving placebo after one year of treatment.
In clinical terms, such a difference may be deemed to be small,
and possibly of limited clinical value. Although the issues of
the clinical relevance of the findings was mentioned in the NICE
guidance, it was not given much emphasis.
3.12 A further example is the NICE recommendation
that topotecan should be considered as one of the treatment options
for women with advanced ovarian cancer, if first line chemotherapy
has not been successful. The research evidence consists of only
two randomised trials (and more studies of weaker designs) which
compared topotecan with paclitaxel and caelyx respectively. All
three drugs have considerable side effects, and the clinical significance
of the findings has not been discussed in these studies. The systematic
review concluded that there is no convincing evidence for the
effectiveness of topotecan in the second line treatment of ovarian
cancer, and that further studies are indicated.
3.13 The research findings appear at times
to receive less weight than submissions from other stakeholders.
The lack of clarity and transparency in the decision making process
may lend itself to accusations of bias in favour of certain aspects
of the process (e.g, input by the pharmaceutical industry). The
independence of NICE is threatened by this lack of clarity. NICE
may be viewed as open to persuasion by various pressure groups—doctors,
patients, government and the drug companies, all of whom have
special interests to advocate or protect.
3.14 Recommendation: The decision
making process that concludes the work of the appraisal committee
needs to be more transparent, with due weight given to the scientific
review of research evidence.
3.15 Confidentiality of consultation documents.
NICE is aware that the concerns of the pharmaceutical industries
about maintaining the confidentiality of consultation documents
is matched by equal concern in the academic community about submissions
of partial evidence from the pharmaceutical industries. Unlike
submissions to the Medicines Control Agency, the pharmaceutical
industry is not bound by law to disclose all information about
a health technology in its submissions to NICE. This may render
some of the evidence underpinning NICE national guidance incomplete.
It may also threaten NICE guidance in that health professionals
and the public may doubt the credibility of NICE recommendations
if they are partly based on "commercial in confidence"
data which is not in the public domain. It also threatens the
independence of NICE, which may be seen as less powerful than
the pharmaceutical industry, or even open to accusations of bias
in favour of the industry. Moreover, it might be considered unethical
to base national guidance on partial disclosure of information
from an industry keen to protect and maintain its profits.
3.16 Recommendation: The pharmaceutical
industry should set up a voluntary agreement that all evidence
will be submitted as part of the technology appraisal process,
or the Government should introduce legislation compelling full
disclosure of all evidence.
4. HAS NICE ENDED
CONFUSION BY
PROVIDING A
SINGLE NATIONAL
FOCUS FOR
APPRAISAL AND
GUIDANCE?
4.1 NICE was set up to provide a single
national focus for appraisal and guidance and, with a high media
profile and government backing, there is evidence to suggest that
the NHS is setting up systems, through its clinical governance
function, to take note of NICE guidance.
4.2 In setting up six guidelines collaborating
centres, two advisory units and a Guidelines Advisory Committee,
NICE aims to manage the guidelines process and to maintain standards,
while increasing the sense of ownership of guidance and devolving
some of the work necessary to produce it. Through effective partnerships,
guidance should be seen as the responsibility of a range of professional
organisations, not just NICE.
4.3 While NICE aims to provide a single
national focus for appraisal and guidance, decision-makers in
the NHS need to sift and weigh evidence, from a range of sources,
to provide best practice. Other independent and well respected
sources, such as the Drugs and Therapeutic Bulletin, the Effective
Health Care publications, the Cochrane Library, and academic journals
contribute to informed debate about effective health care. The
National electronic Library for Health (NeLH) is developing as
a gateway to evidence-based resources. NICE needs to develop further
as an independent organisation working collaboratively with the
many others committed to providing information to enable high
standards of care.
4.4 Recommendation: Set up a working group
to facilitate effective partnership between information providers
to the NHS. NICE and other stakeholders need to collaborate on
horizon scanning, topic selection and coverage, targeted dissemination
and the distribution of information to the NHS and its consumers.
5. IS NICE PROVIDING
GUIDANCE THAT
IS LOCALLY
OWNED AND
ACTED ON
IN THE
RIGHT WAY?
5.1 Effective implementation of guidance
depends upon local ownership and available resources. The mandatory
status of NICE guidance, with funding to follow, will go some
way to making NICE guidance "everyone's business". However,
NHS practitioners need to be persuaded of the validity of the
guidance underpinning recommendations for best practice. The fact
that NICE guidance is based on systematic reviews and NICE consults
interested parties from commissioning through to publication will
help ensure wide ownership of guidance. However, the decision
making process which concludes the work of the appraisal committee,
needs to be more transparent, to ensure credibility and thus take
up. (See Recommendation, paragraph 3.14, above).
6. IS NICE ACTIVELY
PROMOTING INTERVENTIONS
WITH GOOD
EVIDENCE OF
CLINICAL AND
COST EFFECTIVENESS
SO THAT
PATIENTS HAVE
FASTER ACCESS
TO TREATMENTS
KNOWN TO
WORK?
6.1 NICE has a remit to co-ordinate and
disseminate high quality evidence based information within the
NHS. In doing this it actively promotes interventions with good
evidence of clinical and cost effectiveness. NICE supports National
Cancer Service Guidance, Prodigy, MeReC publications and Effective
Health Care publications as part of its dissemination strategy
(3).
6.2 CRD's Effective Health Care bulletins
(EHCBs) are succinct summaries of research findings from up to
date systematic reviews. They tend to focus on the clinical effectiveness,
cost effectiveness and acceptability of health service interventions,
such as the effectiveness of laxatives in adults (enclosure).
The bulletins also focus on the evidence base underpinning guidance
such as the National Cancer Service Guidance (enclosure). The
bulletins are disseminated to a target audience of some 60,000
health service decision-makers. They are well regarded, with high
levels of impact. A recent paper in the BMJ indicated that at
a cost of £25,000, the bulletin on glue ear may have saved
the NHS some £25 million (2).
6.3 Despite good relations between the two
organisations, the re-positioning of the bulletins within the
NICE range of publications has proved problematic. Due to problems
with the NICE contracting and commissioning process, the planning
and production of the bulletins programme has been difficult,
so that CRD has struggled to meet its contractual commitment to
publish six EHCBs and two Effectiveness Matters per financial
year. The lack of a regular publication schedule has impacted
on CRD's dissemination activity and on some Trusts' clinical audit
and effectiveness agendas.
6.4 Under pressure to develop quickly an
infra structure to support guidance development, NICE has yet
to fully develop its dissemination strategy, which has, in turn,
affected CRD's dissemination activity. The NICE dissemination
strategy is now under review, suggesting that the organisation
is aware of the need for its further development. As the national
Centre for Reviews and Dissemination, CRD staff have several years
experience of targeting information to policy makers, health professionals
and the public. CRD is, of course, willing to collaborate with
NICE in promoting clinical and cost effectiveness information
to the NHS and its consumers.
6.5 Recommendation: NICE and CRD
to formally review their contract and commissioning process.
6.6 Recommendation: NICE needs to
collaborate with CRD and other key stakeholders in co-ordinating
the dissemination and distribution of evidence based information
to the NHS and its consumers. (see also Recommendation paragraph
4.4 above)
6.7 REFERENCES
1. NHS Centre for Reviews and Dissemination.
Undertaking Systematic Reviews of Research on Effectiveness: CRD's
Guidance for Carrying Out or Commissioning Reviews. (CRD Report
Number 4) (2nd Edition). York: NHS Centre for Reviews and Dissemination,
2001.
2. Mason J, Freemantle N and Browning G.
Impact of Effective Health Care bulletin on treatment of persistent
glue ear in children: time series analysis. British Medical Journal
2001; 323:1096—97.
3. National Institute for Clinical Excellence.
Annual Report 2000-01 and Summary Financial Statement. National
Institute for Clinical Excellence, September 2001.
| 
