Health CommitteeWritten evidence from Karl Claxton, Steve Martin, Marta Soares, Nigel Rice, Eldon Spackman, Sebastian Hinde, Peter C Smith and Mark Sculpher (NICE 61)
Summary
Amongst others, the Health Select Committee has previously called for more research into the NICE cost effectiveness threshold.
We report on research funded by the National Institute for Health Research (NIHR) and Medical Research Council (MRC) Methodology Research Programme to develop methods to estimate the threshold.
We estimate the relationship between changes in overall NHS expenditure and changes in mortality and extend this to changes in broader health effects in the form of quality-adjusted life-years (QALYs).
Our central estimate of the threshold is £18,317. The probability that the overall threshold is less than £20,000 per QALY is 0.64 and the probability that it is less than £30,000 is 0.92.
Although other assumptions and judgments are possible that retain some level of plausibility, they do not all favour a higher threshold. Indeed, when considered together, they suggest that, on balance, the central estimate is, if anything, likely to be an overestimate.
1. Introduction
1.1 NICE’s comparison of the incremental cost effectiveness ratio (ICER) of a new technology, which is more costly than existing alternatives, with the cost-effectiveness threshold is important in assessing whether the health expected to be gained from its use exceeds the health expected to be forgone elsewhere as other NHS activities are displaced (ie whether the new technology is cost effective).
1.2 When NICE issues positive guidance for a new intervention which imposes additional costs on the NHS, the resources required to deliver it must be found by disinvesting from other interventions and services elsewhere. This displacement will result in health decrements for other types of individual. Thus the threshold represents the additional cost that has to be imposed on the system to forgo 1 QALY worth of health through displacement.
1.3 Currently NICE uses a threshold range of £20,000 to £30,000 QALY gained. Amongst others, the Health Select Committee has asked for further research on the value of the threshold.1
1.4 This submission provides a summary of a draft report on a two-year project funded by the NIHR and MRC Methodology Research Programme to develop methods to estimate the NICE cost effectiveness threshold.2
1.5 Given NICE’s remit, empirical research on the threshold requires some key characteristics:
Reflect the expected health effects (in terms of length and quality of life) of NICE guidance through the displacement decisions taken across the NHS rather than what specific services are (or could have been) displaced.
Facilitate regular updates, based on routinely available data, to reflect NHS changes such as real overall expenditure and productivity. This would encourage accountability through scrutiny by stakeholders and provide predictability for technology manufacturers’ investment decisions.
The nature of service displacement and the magnitude of the health forgone will depend on the scale of the budget impact which should, ideally, be reflected in the value of the threshold.
Methods should recognise the inevitable uncertainty relating to the evidence currently available for the threshold and reflect its implications for policy.
2. Study Methods
2.1 The aim was to develop methods to estimate the NICE cost-effectiveness threshold making use of routinely available data. Objectives were:
(i)
(ii)
(iii)
(iv)
2.2 Earlier econometric analysis estimated the relationship between differences in primary care trust (PCT) spending and associated disease-specific mortality.3 Expenditure came from programme budgeting data which allocates the entire volume of health care expenditure to broad programme budget categories (PBCs) according to primary diagnosis.
2.3 The recent research extended this in several ways including estimating the impact of marginal increases or decreases in overall NHS expenditure on spending in each of the 23 PBCs. These were linked to changes in mortality outcomes by PBC across 11 PBCs.
2.4 The results of the econometric analysis were translated into broader effects in terms of QALYs. The first stage linked estimated effects on mortality to life years taking into account the “counterfactual” deaths that would have occurred if the population in a given PBC faced the same mortality risks as the general population. The second stage accounted for the health (QALY) effects of changes in mortality due to changes in expenditure reflecting how QoL differs by age and gender. The third stage incorporated those effects on health not directly associated with mortality and life year effects (ie, the “pure” QoL effects) to estimate an overall cost per QALY threshold. This effectively used the estimates of mortality and life year effects as a “surrogate outcome” for a more complete measure of the health effects of a change in expenditure.
3. Central or “Best” Estimate of the Threshold
3.1 The most relevant threshold is estimated using the latest available data (2008 expenditure, 2008–10 mortality). The central or “best” threshold is estimated to be £18,317 per QALY.
4. Which PBCs have the Greatest Influence on the Overall Threshold?
4.1 Although the 11 PBCs where a mortality effect of changes in expenditure could be estimated only account for 36% of the change in overall expenditure, they account for 80% of the overall health effects. The other 12 PBCs, where mortality effects could not be estimated, account for the greater part of a change in overall expenditure (64%) but only 20% of the overall health effects, ie, the cost per QALYs estimates associated with a change in expenditure in these PBCs are, in general, much higher.
4.2 Insofar as investment and disinvestment opportunities in these PBCs might have been more valuable (offered greater improvement in QoL) than suggested by the implied PBC thresholds, the overall QALY effects will tend to be underestimated and the overall cost per QALY threshold will be overestimated.
4.3 The overall threshold of £18,317 may be especially conservative (ie, likely to be overestimated) with respect to health effects in PBC5 (Mental Health Disorders), which accounts for a large proportion of the change in overall expenditure (25%) and contributes most to the overall health effects (9%) compared to these other PBCs. The cost per QALY associated with this PBC is based on an extrapolation rather than observations of the direct impact of changes in expenditure on QoL. Available evidence suggests that the investment and disinvestment opportunities in mental health are likely to have been much more valuable than its implied cost per QALY.
5. How Uncertain are the Estimates and what are the Implications?
5.1 Simulation methods were used to reflect the combined uncertainty in the various estimates from the econometric analysis. This indicated that the probability that the overall threshold is less than £20,000 per QALY is 0.64 and the probability that it is less than £30,000 is 0.92.
5.2 As the consequences of overestimating the threshold are more serious than underestimating it in terms of population health, a policy threshold will be lower than the mean of the cost per QALY threshold (ie, lower than £18,317) to compensate for the more serious consequences of overestimating the “true” value.
5.3 There were other (“structural”) sources of uncertainty associated with the estimated threshold, specifically relating to the choice of econometric models and identification of causal effects. Although all the models passed the relevant tests of validity, there remained some uncertainty about the validity of the instruments. This structural uncertainty constituted a greater part of the overall uncertainty associated with the mortality effects of changes in expenditure, but the central estimate of the cost per QALY threshold was robust to this uncertainty.
5.4 The methods of analysis used to link the effects of changes in expenditure on mortality to a fuller measure of health expressed in QALYs was also subject to uncertainty. A preferred analysis (or scenario) was identified as making the best use of available information, with assumptions appearing more reasonable than the available alternatives and providing a more complete picture of the likely health effects of a change in expenditure.
5.5 A critical issue is whether, on balance, the central or best estimate is likely to be an underestimate or overestimate of the cost per QALY threshold. Although other assumptions and judgments are possible that retain some level of plausibility, they do not all favour a higher threshold. Indeed, when considered together, they suggest that, on balance, the central or best estimate of £18,317 is, if anything, likely to be an overestimate.
6. The Impact of Investment, Disinvestment and Non-Marginal Effects
6.1 The central estimate of the cost per QALY threshold is based on estimates of the health effects of changes in expenditure across all 152 PCTs, some of which will be making investments (where expenditure is increasing) and others making disinvestments (where expenditure is reduced or growing more slowly).
6.2 The threshold is, however, likely to differ across these different types of PCT. It would be expected that, other things equal, more expenditure would increase health but at a diminishing rate. Therefore, the amount of health displaced by disinvestment would be expected to be greater, and the associated threshold lower than the central estimate. Conversely, the health gained from investment would be expected to be lower, and the associated threshold higher.
6.3 This was examined by re-estimating the outcome and expenditure effects separately for those PCTs where their actual budget is under the target allocation from the Department of Health resource allocation formula (ie, those under greater financial pressure and more likely to be disinvesting than investing), and those that are over target (under less financial pressure and more likely to be investing than disinvesting).
6.4 The results confirm these expectations: the health effects of changes in expenditure are greater when PCTs are under more financial pressure and are more likely to be disinvesting then investing. The analysis suggests that budget impact not only displaces more valuable activities within each PBC but that overall expenditure tends to be reallocated to PBCs which can generate more health. Although further research might enable a quantitative assessment of how the relevant threshold should be adjusted for the scale of budget impacts, the qualitative assessment seems clear: the central estimate of the threshold is likely to be an overestimate for all technologies which impose net costs on the NHS (almost all technologies appraised by NICE); and the appropriate threshold to apply should be lower for technologies which have a greater impact on NHS costs.
7. How does the Threshold Change with Overall Expenditure?
7.1 The same methods are used to consider how the cost per QALY threshold is likely to have changed from 2007 to 2008 as overall expenditure has increased. This provides some insights into how the threshold might be expected to change over time as, for example, overall expenditure and NHS productivity changes.
7.2 This has implications for a judgement about the appropriate frequency of periodic reassessment of the cost per QALY threshold. Other things equal, the threshold would be expected to increase following a rise in overall expenditure, although this will depend on whether there is discretion over how additional resources can be spent. However, insofar as the productivity of those activities that are valuable to the NHS also improves through innovation, the threshold will tend to fall. So the net impact of these two countervailing effects on the threshold cannot be determined a priori.
7.3 Differences in the estimated thresholds between 2007 and 2008 are assessed. Although overall expenditure increased by 6% between 2007 and 2008 which represented real growth of 2% in 2007 prices, the overall threshold for all 23 PBCs fell by 2% in nominal terms and by 5% in real terms.
7.4 The reasons are complex but reflect changes in productivity, which differ across PBCs, but also a general reallocation of a change in overall expenditure towards those PBCs that appear more valuable in 2008. Given the uncertainty in estimation, subtle differences between 2007 and 2008 should not be over interpreted. This analysis does suggest, however, that the overall threshold will not necessary increase with growth in the real or even nominal NHS budget. This suggests that the threshold is more likely to fall at a time when real budget growth is flat or falling and PCTs find themselves under increasing financial pressure.
8. What Type of Health is Forgone by Approval of a New Technology?
8.1 The methods of analysis can identify not only how many QALYs are likely to be forgone across the NHS as a consequence of approving a technology which imposes additional costs on the NHS, but also where those QALYs are likely to be forgone and how they are made up, ie, the additional deaths, life years lost and the QoL impacts on those with disease.
8.2 As an example, based on the 2008 central estimate of the cost per QALY threshold (£18,317), the approval of Ranibizumab for the treatment of diabetic macular oedema (prior to the patient access scheme agreement) would have imposed additional annual costs of up to £80m on the NHS each year and been likely to displace 4,367 QALYs elsewhere in the NHS. This forgone health is likely to be made up of 295 additional deaths and 1,337 life years forgone, most of which are likely to occur in Circulatory, Respiratory and Cancer PBCs. However, much of the total health effect of these additional costs (3,509 QALYs) is associated with QoL forgone during disease which is most likely to occur in Respiratory, Neurological and Mental Health PBCs.
9. Conclusions
9.1 The methods of analysis presented here go some way to providing an empirically-based and explicit quantification of the scale of opportunity costs the NHS faces when considering whether the health benefits associated with new technologies are expected to offset the health that is likely to be forgone elsewhere in the NHS.
9.2 The study also starts to make the other NHS patients, who ultimately bear the opportunity costs of such decisions, less abstract and more “known” in social decisions. Since who happens to be known or unknown is only a matter of perspective, time and ignorance, ethical and coherent social decisions require that both should be treated in the same way. These methods contribute to removing some of the “ignorance” and making the unknown more real.
9.3 This work has implications for the Government’s proposals to move to a system of value-based pricing for new prescription pharmaceuticals. These proposals include a widening of the measure of benefit to be used in evaluating new products including wider social benefits such as carer time and a consideration of the unmet need. Implementation of an augmented measure of benefit will also need to be reflected in terms of benefits forgone through service displacement. In principle, this will be feasible given the methods used in this study.
October 2012
1 House of Commons Health Committee. National Institute for Health and Clinical Excellence: First report of the Health Committee 2007–2008. HC27-I. . London: Stationery Office, 2008.
2 Full draft report available at http://www.york.ac.uk/che/research/teehta/methodological-research/#tab-4
3 Martin S, Rice N, Smith P C. Does health care spending improve health outcomes? Evidence from English programme budgeting data. Journal of Health Economics 2008; 27:826–42.