Examination of Witnesses (Questions 285
- 299)
THURSDAY 16 MARCH 2006
PROFESSOR JOHN
APPLEBY, MR
JAMES JOHNSON,
DR PAUL
MILLER, DR
SALLY RUANE
AND MR
DANIEL EAYRES
Q285 Chairman: Could I welcome you
to the Committee. Thank you for coming along For the record, could
I ask you to introduce yourselves and the organisations which
you come from.
Mr Eayres: Daniel Eayres. I work
for the National Centre for Health Outcomes Development. We work
under contract to the Department of Health, analysing the KPI
data and the ISTCs.
Mr Johnson: I am James Johnson.
I am the Chairman of the British Medical Association and I am
a consultant vascular surgeon in Cheshire.
Dr Miller: I am Dr Paul Miller.
I am Chairman of the British Medical Association Consultants Committee
and I am a consultant psychiatrist in Sunderland.
Professor Appleby: I am John Appleby.
I am the Chief Economist at the King's Fund.
Dr Ruane: I am Sally Ruane from
the Health Policy Research Unit at De Montfort University in Leicester.
Q286 Chairman: Could I ask a question
of all of you: what research has been carried out into the effectiveness
of ISTCs? Have there been any problems with carrying out research
with this area?
Dr Miller: The Health Policy and
Economic Research Unit of the British Medical Association late
last year surveyed clinical directors in anaesthetics, ophthalmology
and orthopaedics, the three specialities far and away most likely
to be affected by treatment centres. They surveyed them on their
views and the impact on NHS treatment centres and the independent
sector treatment centres. I think the main conclusions or headlines
would be that the perception and the experience was that NHS treatment
centres were more beneficial for patients than the independent
sector ones, and this was overwhelmingly to do with integration
with the rest of the NHS: that the continuity of patients' care,
the availability of notes, the ability to talk to other doctors
and consultants involved were much easier with the NHS treatment
centres than they were with the independent sector treatment centres.
Though I should say from the start that it was widely found that
there were benefits to patients in terms of shortening waiting
times.
Q287 Chairman: Was there any clinical
indication in this research at all? Was there any thing different
there?
Dr Miller: We did not go in any
great depth into differences in clinical issues, though one of
the outcomes that was found was that these clinical directors,
in their experience, found there were more problems with readmissions
post-operatively from the independent sector centres, almost certainly
because they are not integrated with an NHS facility which would
have the ability to deal with post-operative complications. That
is not what the ISTCs are for.
Professor Appleby: As far as I
am aware, there has been very little systematic research into,
as you say, the effectiveness of ISTCs. There is, as I understand
it, an official Department of Health funded study of NHS TCs but
no equivalent on the independent sector side, which I think is
rather remiss. In part, it depends what you mean by research into
effectiveness. I suppose I would go back into research into achieving
the aims and objectives of the ISTC programme, and as far as I
am aware there is no research into that at all.
Dr Ruane: The Health Service
Journal conducted a survey of PCT and acute trust chief executives
which was published in January last year. That was not specifically
on effectiveness; it was more a question of how those chief executives
perceived the impact of ISTCs on them and certainly the acute
trusts. I think 79% of respondents of the acute trust chief executives
believed they had had a significant impact on forcing their trusts
either to reduce activity or to forego growth as a result, and
there seemed to be particular impact on orthopaedic work. I think
some of the more qualitative material that came out of the HSJ
survey is equally important though. One of the columnists commented
that there had been more strength of feeling, and more, what he
called, "alarmed and angry" communications to the Health
Service Journal over this policy than over any other policy
that had taken place over the last few years, and that this was
perceived as a fundamental contradiction of other health service
reforms.
Mr Johnson: The research that
has been referred to, both for the BMA and what you have just
heard, is essentially extremely soft research: it is asking people
who may well have pre-formed opinions about the general principles
involved here what they think about how it is going. Probably
that is all you can do at the moment, because even some of the
first wave ISTCs have not even started taking patients yet, let
alone the second wave, but we believe it is absolutely essential—and
in the three-quarters of an hour I have been listening to your
discussions, clearly so do you—that we have outcome data
published from the treatment centres and equivalent outcome data
from the NHS—which largely is absent—to compare it
with. If you just have one and not the other, it is meaningless.
If you have complications, is that bad, is it good? Who knows?
You need to know whether it is doing better or worse on average
than a similar basket of NHS hospitals across the board. You heard
in the last session that some outcome data is available. It is
very mechanistic sort of outcome data that you can get off a computer:
how many people returned to theatre? How many people were readmitted?
When you hear the criticisms of the treatment centres—"The
hips are not being done as well; they are having to be revised"—why
are we not collecting data about revision rates for hips and comparing
it with the NHS? It is absolutely essential, in our view. If you
are going to make the system work, you must have the data and
you must have the NHS data as well.
Q288 Mr Burstow: That last comment
is quite helpful. In some of the data we were supplied with in
our first session this morning from Care UK, they do provide just
that in the information—hip revisions, specifically. That
is why I wanted to ask Mr Eayres if you could perhaps tell us
a bit more about the research that you conducted, because, as
I understand it, that is based very much on the key performance
indicators collected from ISTCs. What did the research tell you
about the standards of ISTCs?
Mr Eayres: First of all, we are
contracted to analyse and report on the 26 KPIs that are collected
by the ISTCs. An aspect of that would also be to look at possible
benchmarking against NHS or other external sources. I think we
need some clarification about what the indicators are. We have
heard about clinical indicators, clinical outcome indicators,
performance indicators. Of the 26 performance indicators, some
are what I would consider clinical indicators, in that they reflect
some aspect of the clinical care of the patient. Some of these
might be clinical outcome indicators, in that they reflect some
outcome of the care. For example, one of the indicators is clinical
cancellations: patients who have got an appointment but they have
cancelled for some clinical reasons. You could argue that is a
clinical indicator but it is not a clinical outcome indicator.
Things like readmissions, transfers to another hospital, day cases
that end up becoming inpatients, you could argue are clinical
indicators and possibly clinical outcome indicators. There is
also another set of indicators on which we have currently not
received any data, which we would consider as pure clinical outcome
indicators, and these are things like, for example, complication
rates and wound infections, but also patient-reported outcome
measures, where the status of the patient is measured before the
operation and the status of the patient is measured again after
the operation and some sort of measure of improvement or change
or impact is made. At the moment, the key performance indicators
that we have are some clinical ones, such as cancellation, readmission.
Some are purely process ones, such as: Did referral lead to an
inpatient appointment? Some of them do not reflect the patient
pathway at all. For example, there is an indicator about additionality—you
know: Were any staff employed who should not have been?—and
that is in no way a clinical indicator or a patient noted indicator.
We have looked solely at these, and, as I have said, on what we
would call outcome indictors, KPI 15, no data has so far been
collected and given to us. We understand from the Department of
Health that that sort of information will be collected from April.
These are the types of outcome indicators in which you were particularly
interested.
Q289 Mr Burstow: That is right. Before
we go on to that, I just want to see if there is anything more
you can tell us about the research you have done to date on the
KPIs that have been published today and what they tell us about
things.
Mr Eayres: We have five key points
about them. First, we had quibbles with some indicators, and particularly
the way specification is related to the way in which they were
reported, which gave us some problems in creating robust comparable
indicators. There are a lot of issues around interpretation of
the indicators by the different ISTCs. They interpreted definitions
in different ways and supplied different data. There are issues
around completeness and quality of the data that was returned.
Although they were all supposed to be returning data on certain
same KPIs, there was very little guidance from Department of Health
in terms of in what format it should come. There was not a standard
template, so there was a lot of variation in the completeness
and quality that came in. Another issue we had was the lack of
a clinical outcomes data, which was KPI 15. The final point was
the way the data comes into us in terms of monthly aggregated
returns and there is very little we could do in terms of validation
of that data. We are basically accepting what the ISTC give us.
They say, "Oh, yes, we had 100 admissions, five of those
led to readmissions." They give us that; we cannot really
validate it at the moment.
Q290 Mr Burstow: That brings me back
to the point you were making just now about the non-availability
to you of this point about the clinical outcomes data. If you
were here earlier on with our first session, you would have heard
the exchanges we had with the various operators of the centre
at the moment. My observations on that were that there seemed
to be some confusion amongst operators as to what they were supplying
at this stage, and that is something our advisors need to unpick,
but one of the things which was also unclear was that at least
one of the providers was putting into the public domain considerably
more outcome data than the others at this stage. Do you believe
that all the providers are currently collecting more outcome data
but they are just not supplying it to the Department. What information
or knowledge do you have of what is being collected, even if you
are not being supplied it?
Mr Eayres: I do not know what
individual ISTCs are collecting internally or making available
to patients or the public internally. All I am aware of is what
Department of Health provides to us that they have collected from
the ISTCs.
Q291 Mr Burstow: Apart from this
point about the Department providing a clearer framework in which
data is collected so that the data is more comparable, are there
any other points of learning you could draw from what you have
done so far about how the system could be improved to make sure
the data is being collected better?
Mr Eayres: Yes. We have made a
number of recommendations to the Department of Health about how
the data ought to be collected. In particular, we recommended
a move away from monthly aggregate returns, to a system whereby
we build the indicators ourselves out of the patient level data
which they are obliged to submit in the same way that NHS hospitals
are obliged to submit. Some of the KPIs might require additional
information outside of the standard data set, but that way we
can then do all the aggregation of the data, and that would remove
all the possibilities of different interpretations of definitions,
etc, so we could standardise it a lot more.
Q292 Mr Burstow: Has the Department
responded to that?
Mr Eayres: They have agreed in
principle and they are in discussions along the lines of implementing
that at some point in the near future.
Q293 Mr Burstow: KPIs, you have outlined
to us in some detail now what each of them might be in terms of
the categories they broadly fall into, but what was the process
for choosing the KPIs? How was that arrived at?
Mr Eayres: We were not involved
at that stage, when the KPIs were chosen. My understanding is
that they were chosen to reflect in some way the patient pathway
through the ISTC. But, for example, they start off with referral,
so there is an indicator which says: How many patients were inappropriately
referred? At the next stage there is an indicator saying: Of those
referred, how many then led to an inpatient appointment? At the
inpatient appointments, how many did not attend? How many were
cancelled? And so on through the process, until we get to a stage
where they have had the operation, and then: Did it lead to a
transfer? Did it lead to a readmission? Did the patient then complain?
If the patient complained, was that complaint dealt with within
the appropriate time framework? Most KPIs are based on that sort
of idea and then there are a few additional ones tagged on to
the end.
Q294 Mr Burstow: Would it be possible,
if you had the disaggregated data, to reconstruct the KPIs in
a way that would allow you to draw more meaningful comparisons
with the equivalent data collected from direct NHS providers?
Mr Eayres: Yes. That is one of
the reasons why we recommend that the Department of Health do
it that way. If the ISTCs are submitting the same minimum data
sets that they are required to submit as the NHS do, we basically
have the same data for ISTCs and NHS hospitals, and we can then
write the same queries and create the same indicators for both.
Q295 Mr Burstow: You said they have
been agreed in principle. When do you think they might agree in
practice?
Mr Eayres: I cannot say.
Q296 Mr Burstow: Maybe we will ask
the Minister that question.
Mr Eayres: It is within the philosophy
of the national programme for IT within the NHS, in that we should
not be creating new return systems. Wherever possible, clinical
data should be collected, and then administrative/performance
management data should be extracted from that clinical information.
There is even a secondary user service being set up as part of
the information programme to do that. Our recommendation is that,
for the ISTC programme, that information flow is channelled in
through that programme.
Mr Burstow: Thank you.
Q297 Dr Taylor: Going back to aims
and objectives, I think it was Professor Appleby who said that
there has been no research or collection of data on the achievement
of aims and objectives. Have any of you any impressions of the
effect on waiting lists and how much of that has been due to the
independent sector treatment centres?
Dr Miller: Perhaps the one where
the data is clearest is in cataract surgery. The independent sector
treatment centre cataract programme so far had done 20,000 cataracts
by the end of January 2006, but that needs to be put into perspective.
The NHS itself is doing just over 300,000 a year, and the productivity
of the NHS increased very greatly in recent years as a result
of a joint project between the Department of Health and the Royal
College of Ophthalmologists. They sat down a few years ago together
and agreed a plan/arrangements to increase cataract operations
in the NHS. That was done successfully, so that we now have a
figure of 300,000 done on the NHS and the target per year is 9,000
in mobile cataract schemes. That gives you an idea of the relative
contributions.
Q298 Dr Taylor: We have had that
several times from several people. Professor Appleby wants to
come in.
Professor Appleby: I would like
to make the point, which I think partly Paul was making, that
the NHS has been tremendously successful in reducing waiting times
over the last three to four years. Actually, whether the ISTCs
have had any added effect to that is very difficult to say. The
one thing we do not know is how long patients have been waiting
who have been treated by ISTCs. This is part of the information
set we would like to have to which Daniel was referring earlier.
ISTCs are treating NHS patients. The information about their treatment,
their diagnosis, how long they have been on the list before they
get treated and so on should be treated in just the same way as
if they were treated in an NHS trust hospital; that is, it should
become part of what is known as the hospital episodes statistics
system, which we could then analyse in lots of different ways—and
then we can start to make comparisons as well. The other thing
I would like to mention is waiting times and waiting lists. We
know they have been coming down over the last few year—in
terms of waiting times, tremendously, and waiting lists have also
started reducing recently quite significantly. It is not enormously
clear why or how this has been happening. If you look at the numbers
of patients taken off the waiting lists to be treated in NHS hospitals,
it has actually been falling over the last five or six years.
One would perhaps expect that if waiting lists were going down
the NHS would be treating more patients. That does not seem to
be the entire story, in that it also seems that not so many patients
are going on to waiting lists in the first place. So the actual
reasons why waiting lists and waiting times are not coming down
is not solely a function of capacity. There is an issue around
that which I still think needs exploring.
Mr Johnson: I think one of the
biggest factors in bringing down waiting lists is the recognition
that if you separate acute care from elective care, you can guarantee
to do the elective care. You do not turn up, as I do not infrequently,
to do an operating list and find that all the beds are full of
acute medical admissions and my surgical patients have been sent
home. If you do not allow that to happen—because, effectively,
you deal with your elective patients in a separate institution
that does not have emergency medical admissions and you know when
you come in to do your operating list that you will do it—it
runs more efficiently. If you separate these things, then you
use the facilities far more efficiently. Probably that has had
more to do with bringing waiting lists down than the independent
sector ones, which, as I say, in the first phase some are not
even on line yet. The impact they have had, purely because they
have not been there very long, has not been very great. The sorts
of figures Paul Miller gave to you about cataracts indicate that,
although they have done a lot of cases, in terms of the total
numbers it is quite a small proportion.
Q299 Dr Taylor: Do we have any similar
figures for orthopaedics?
Mr Johnson: Not that I know of.
Professor Appleby: I think they
have been made available recently in a PQ. I cannot remember the
numbers offhand, though.
| 
