Select Committee on Health Appendices to the Minutes of Evidence

Recommendations and their Basis


  The key technology assessment dimension of NICE is "the balance of clinical and cost effectiveness" which is a composite of two elements.

Clinical effectiveness

  Clinical effectiveness differs from clinical efficacy as it incorporates the notion of how a drug works in real life, rather than the constrained observational environment of a clinical trial being run according to a strict protocol. It also includes the expanded notion of tolerability, which includes not only the classical side effect profile but also the broader, specific and general impact on a patient's quality of life.

  Consideration of clinical effectiveness rather than clinical efficacy is sensible but new. To date clinical effectiveness has not been the subject of pharmaceutical development in this country or abroad. It is still the case that the development of pharmaceutical products is driven by the needs of regulatory authorities such as the Medicines Control Agency here in the UK and the FDA in the USA and the EMEA in Europe and not the broader considerations being evaluated by NICE. To improve clarity, pertinence and ease of regulatory assessment these agencies publish detailed criteria for the assessment of different types of therapy. These criteria have been developed following careful consultation with all relevant expert groups.

  As expected, the introduction of a new assessment discipline has meant that much of the detail relating to which criteria to use, and which methodologies should be applied to generate the Institute's preferred assessment criteria, have been made up in isolation of the stakeholders as the Institute's work progressed. This has resulted in several anomalies where clearly superior treatments, as judged by a broad consensus of stakeholders, have been the subject of negative assessments.

  In very many cases, the type of data that the Institute wants to assess has, hitherto, never been generated before, or it has not been appropriate to generate the type of data that the Institute wants to see. Moreover the lack of data requested by the Institute may never become available for good scientific and/or ethical reasons related to for example, but not limited to:

The time course of the disease

  Some treatments can be easily assessed within the time course of the disease, for example influenza. However, delaying the assessment of chronic relapsing conditions like multiple sclerosis, until the final outcome of treatment intervention is known, which may be many decades later, is unethical. In this case assumption modelling is required and this is totally reliant upon making the correct assumptions about the relevant parameters and criteria and then modelling them in a representative manner.

The natural history of the disease

  Establishing the correct endpoints for a technology assessment is vital. For example, the reason for treating diabetes is to reduce the morbidity and mortality effects that correlate well with chronic high blood sugar measurements. The aim of therapy is not to reduce the blood sugar levels per se. Fortunately, due to the published results of the seminal UKPDS study, the correlation of blood sugar and morbidity is well understood. However, this is a very special case and is not the norm for the vast majority of other diseases.

Appropriate endpoints for study

  When assessing the effects of cancer treatments the Institute has been inconsistent in its approach as to which endpoints it has seen fit to use and what weight to give them. The scientific ideal will always be to use hard endpoints like survival, but this is not ethically justifiable, particularly as treatments are getting better and patients now increasingly receive early, adjuvant or neo adjuvant treatment, which takes a considerable time to impact survival because the better the treatment, the longer people live. Denial of positive guidance based on trends and "soft" surrogate endpoints like time to disease progression or tumour regression response rates, etc, is wrong and will not improve the health of the Nation. It is well recognised by all that the UK languishes unacceptably below the cancer survival levels of other comparable countries in Europe and the developed world.

  To date, the Institute and all stakeholders have not been working to the same robust and transparent set of relevant clinical effectiveness assessment criteria due in part to ignorance and in others to a lack of consensus on what the correct measures should be.

Cost Effectiveness

  Health economics is a new and still relatively embryonic social science. For reasons that are similar in substance to those made for clinical effectiveness, it is imperative that the methodologies used in this assumption based discipline are appropriate, consistent and well understood before assessment is contemplated.


  1.1  The Institute, in concert with interested parties, should develop and publish a clear set of criteria for both the clinical and the health economic dimensions of assessment for each specific type of technology or therapy area prior to its assessment.

  1.2  There should be an open consultation process between stakeholders and the Institute to agree the relevant criteria to use for a specific technology assessment.

    —  Accelerate the development of appropriate methodologies to collect relevant data and the rapid establishment of this new medical assessment discipline—"clinical effectiveness".

    —  Reduce the time and resources needlessly wasted by the Institute and relevant stakeholders redressing inconsistencies and poor decisions made by the Institute.


  The process is frequently too long and as a consequence squanders health improvement opportunities for patients and the NHS. From our own experience the assessment of products for the treatment of multiple sclerosis is still ongoing for almost two years. Similarly the assessment of treatments for colorectal cancer is still ongoing after more than 14 months of consideration and following two appeals to the Institute.

  While an assessment is being considered patients are denied access to modern medicines that have been assessed as being of proven efficacy, safety and quality by the Medicines Control Agency, because Health Authorities will not fund new treatments in the knowledge that official guidance is on its way.

  Due to the effective blight put upon a new technology by the uncertainty of knowing when guidance will be issued it is not possible to effectively commercialise the product. The consequence of this is that the necessary funds to continue the development and research of the product by either extending our knowledge and understanding further, or researching new patient groups, such as children or the elderly, or new indications is being lost. The failure of the Aventis UK to capture research investment is becoming increasingly clear. Aventis is not alone in this respect within the pharmaceutical industry. Indeed the issue of the UK's lack of research competitiveness is subject of an ongoing Prime Ministerial initiative.

  The Institute does not undertake appraisals itself; instead it commissions others to do the work for it. Moreover, we appreciate that as the Institute is still in its infancy it may not yet have access to all of the necessary resources and expertise that it needs to conduct the technology appraisals required of it by the Department of Health. This being said, we are dismayed that the Institute does not use the same third-party appraisal organisations to perform similar types of appraisal. The institute has assessed Aventis products for the treatment of three types of cancer. Lessons learned from one assessment have not been used in other assessments and as a consequence, different approaches and, we contend, decisions have been adopted.

  By way of example from our own experience unreasonable delay has, and continues to occur in the assessment of two technologies.

Multiple Sclerosis

  This case relates to the assessment of technologies for the treatment of multiple sclerosis when the Institute failed to undertake the correct of the analysis of the clinical and health economic data. Following a successful appeal the assessment work and health economic modelling was re-commissioned and assessment started all over again.

Colorectal Cancer

  In the second case, the appraisal of colorectal cancer the Institute was in error of its own process and failed to consult with stakeholders following a successful first appeal that resulted in the distribution of a flawed Final Assessment Determination that became the subject of a second appeal.

  The assessment of Marketing Authorisation Applications for new pharmaceutical products is governed by a well-documented timeline. If an agency has questions of the applicant the assessment clock is stopped while the applicant answers the agency's questions.


  2.1  A clear and short timeline is needed for each assessment marked with clear milestone events. We contend that this should be no longer than 180 days, with the provision for "clock-stop" periods if necessary.

  2.2  We recommend that in the long run, the Institute should resource itself adequately in order to be able to conduct its own appraisals rather than commission third-party organisations of varying quality.

  2.3  In the short run, and if it is also necessary in the long run for the Institute to continue to use third-party assessment organisations, they should use the same organisations to assess similar technologies. We contend that this will improve self-learning by NICE and its third-party contractors with benefits accruing to the quality of assessment and hopefully the speed as well.

  2.4  A mechanism for the provision of interim guidance needs to be established to ensure that patients are not denied access to treatments that they and their doctors consider of benefit when the Institute has been either tardy or negligent or both.


  The members of the Appraisal Committees are not selected as experts in their own right for the technologies that they are assessing. The broad skill base present in the constitution of the various committees needs access to reliable expert opinion in order to form a well-considered view.

  The stakeholders to the process have a valuable contribution to make to the overall assessment but are denied access to the Appraisal Committee other than by a single submission document at the start of the process. To date there is no mechanism for stakeholders to help the Appraisal Committee in its work by appearing before the Committee to explain and/or help clarify complex points.

  We have had numerous cases where the Committee appear to have misunderstood complex points. This has resulted in needless time being wasted coupled with increased patient anxiety as Preliminary and Final Appraisal Determinations have had to be appealed successfully in order to avert poor guidance being issued to the NHS.

  The Appraisal Committee only needs to ensure that it has considered all relevant information set before it. The Committee does not have to provide an explanation, or even an insight into how it arrived at its decision. This lack of transparency is the source of much needless confusion to all who read the guidance with anything but a passing interest. It does not permit challenge of decisions that may be erroneous. In the event of an appeal the only matter that the Appraisal Committee has to satisfy the Appeal Board on is that it considered the matter not how it has considered the evidence or the relative weights that it has given to various forms of evidence.

  The test used in an appeal pertaining to the content of the guidance issued by the Institute is the severe test of "perversity" which we interpret to be the strict test of Wednesbury unreasonable.


  3.1  Introduce a consultative mechanism for stakeholders to be able to address the Appraisal Committee in the early stage of their work and after the first draft of the Appraisal Determination has been issued.

  3.2  Introduce a mechanism for the Appraisal Committee to explain its decisions by publishing the reasons for decisions and inviting oral questions with answers, in public, as part of the consultation process.

  3.3  The standard of decision making should be that of a competent, informed body and not one that can only be challenged if it can be shown to be wholly (Wednesbury) unreasonable.


  The mechanism by which the Appraisal Committees obtain expert opinion has been seriously flawed. In the ongoing case of the assessment of colorectal cancer technologies the Committee only had the benefit of one expert in the field. The Chairman of the Appraisal Committee as being unacceptable subsequently acknowledged this.  


  4.1  Publish the list of expert opinion that the Appraisal Committee will refer to prior to the start of the appraisal of the data and submissions by the Committee; inviting additions and challenges to the experts selected.

  4.2  As the Institute clears its backlog of appraisals it will increasingly appraise cutting edge technologies. Expert opinion should not be limited to UK experts but include experts irrespective of their place of abode or clinical or scientific practice.

  4.3  The breadth and depth of expert view should not be arbitrarily limited by number of experts but only be limited by predefined scope of opinion required to achieve a sound decision.


  Currently all appeals of the Final Appraisal Determination, which if unchallenged, is accepted by the Institute and issued as Guidance, are heard by the Institute. This is not necessarily a bad thing. However, we contend that transparency and trust in the Institute's processes and decisions would be better served by and independent appeals process that is not allied to NICE or any other special health authority or establishment of the Department of Health.


  5.1  Establish an appeals process for Final Appraisal Determinations and NICE Guidance that is independent of the NICE executive.

  5.2  Appeals should be possible on the merits and substance of the decision and not limited to Wednesbury unreasonable: "perverse".


  There is considerable confusion as to the authoritative basis of NICE guidance. Health managers with responsibility for the financial management of the NHS institutions that they manage consider that treatments that are not given positive NICE guidance should not be financed. Clinicians are similarly of the view that if there is no budget for a treatment they cannot use it, even if it is clearly in the best interest of their patient and, indeed, the NICE recommended treatment is in some way contra-indicated.  


  6.1  A clear footnote must be written into all published guidance setting out that the document is only guidance and that no patient should be disadvantaged by this guidance if sound clinical reasons for deviating from it can be made by their attending physician.

  6.2  The nature of the content and process to establish reasonable justification for using a treatment that is not the subject of positive NICE guidance should not be so onerous as to act as an effective barrier to the patient receiving the treatment.

January 2002

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 8 July 2002