Memorandum from the Picker Institute
1. Overview
1.1 Picker Institute Europe welcomes the
Public Administration Select Committee's continued active interest
in the role of user voices in the design and delivery of public
services. The Picker Institute wishes to inform the Committee
that, based on its evidence of patients' experience in their NHS
treatment and care, it is simultaneously submitting a memorandum
to the Health Select Committee's Inquiry into public and patient
involvement in health care, which contains some specific recommendations.
1.2 This memorandum to PASC draws upon much
the same evidence and argument, but it will be for the Committee
itself to decide the relevance of these examples from the health
service to its overview of public services generally. This memorandum
focuses on only one of the questions raised in the Issues and
Questions Paper, namely the role of measures of customer satisfaction
in assessing the standards of public services.
1.3 Among the key points made below are
as follows:
— Crucial to enabling user voices to
be effective in influencing the design and delivery of services
is the collection of a high quality evidence base for what users
want, and what they get, in relation to those services.
— However, measures of user "satisfaction"
have been found to be problematic and of limited usefulness, and
the Picker Institute urges the Committee to examine the relevant
value of measuring user experience rather than "satisfaction".
— In relation to the health service
in England, a high quality evidence base in the form of a national
survey programme of patients' experiences of health care does
exist and is, in the view of the Picker Institute which has been
instrumental in developing and implementing it, both of high value
and cost effective.
— That programme's future is not necessarily
secure, given that the regulator which commissions it will be
merged with two other regulators in the near future; and the Picker
Institute requests PASC to express its support for the continuation
of the programme.
— The memorandum makes a number of
points about what is required to capitalise fully on this evidence
base in order to create positive improvements in the quality of
service to patients. These include the need for further adjustments
to regulation, performance measurement and inspection in the NHS
to require its service providers to make active use of the survey
feedback to involve patients in action planning for change.
— There is a need for action to ensure
that the new mechanisms for public and patient involvement in
health—the Local Involvement Networks' or "LINks"—make
use of the mass of evidence from patient surveys, and are not
left struggling with their information needs, duplicating existing
evidence or reinventing the wheel of survey techniques. The Picker
Institute is suggesting amendments to the Local Government and
Public Involvement in Health Bill to achieve this.
— The use of patient feedback to assess
standards relates both to Department of Health standards, where
there is not yet any meaningful enforcement of the need to engage
patients in health care, and to the professional standards of
the people who deliver care and treatment, and whose approach
to "patient/professional partnership" is crucial.
— There needs to be a wholesale culture
change in the approaches of health professionals if the experiences
of patients are to be improved, and their needs and desires to
play an active role in their own care are to be fulfilled. Patient
feedback should be used to revise professional standards, to reorient
professional education and training, and as part of the assessment
of professionals' performance within their revalidation and continuing
professional development.
2. About the Picker Institute
2.1 The Picker Institute is an independent
health charity which works with patients, professionals and policy
makers to promote understanding of the patient's perspective at
all levels of healthcare policy and practice.
2.2 It undertakes a unique combination of
research, development and policy activities which together work
to make patients' views count. These include:
— Researching and evaluating patients'
experiences.
— Leading initiatives that make improvements
happen.
— Building evidence to inform health
policy.
2.3 The Picker Institute led the development
of patient experience surveys in the UK. These go beyond simple
measures of "satisfaction", by enabling patients to
report on their actual experience of various aspects of their
care and treatment. The questionnaires are developed through holding
focus groups with relevant patients to discover what aspects of
care they focus on the most, followed by large scale surveys of
patients, producing results which enable health professionals
and managers to identify areas for improvement.
2.4 There is now a Department of Health
requirement for every NHS Trust to carry out an annual survey
of patients. The Picker Institute is an approved provider of surveys
for this programme, and acts as the national co-ordination centre
for all the surveys carried out by acute Trusts. The Picker Institute
also provides bespoke survey services to NHS bodies and other
organisations, including staff surveys as well as patient surveys.
On the basis of this evidence and its other, independent research,
the Picker Institute works to improve the quality of patient care
with a variety of organisations involved in the NHS, in professional
regulation and in representing patients' views. It is consulted
frequently by government and the Department of Health, as well
as relevant All Party Groups in parliament.
2.5 This memorandum draws upon the Picker
Institute's experience of having surveyed over one and a quarter
million patients over the last seven years; of researching and
evaluating many different ways to measure patient satisfaction
and experience; and of analysing patients' reported experience
in the UK and in other advanced, industrialised countries.
3. That role do measures of public satisfaction
have in assessing the standards of public services? How should
user views be monitored? How can the cost effectiveness of user
surveys and feedback mechanisms be assessed?
Getting the evidence right: "Satisfaction"
versus "experience"
3.1 In its Issues and Questions paper for
this inquiry, the Public Administration Select Committee refers
several times to the use of "customer satisfaction"
measures to influence the design and development of public services.
Likewise it quotes the government-commissioned review of the Charter-Mark
as recommending that "customer satisfaction surveys should
be commissioned and owned by the organisations delivering the
services to the public". The Picker Institute urges the Committee
to review this terminology.
3.2 Based on its experience of designing,
developing and running large scale patient surveys relating to
health care, and of using these to create changes in services
to the benefit of patients, the Picker Institute believes measures
of customer "satisfaction" are riddled with problems
and have limited usefulness.
3.3 "Satisfaction" is an ill-defined
concept which has been measured in many different ways. There
is no consensus on which aspects of satisfaction should be included,
or which are most important. Sometimes patient satisfaction is
treated as an outcome measure—satisfaction with one's health
status after treatment—and sometimes as a process measure—satisfaction
with the manner in which care was delivered. Satisfaction ratings
reflect a number of variables which may include the personal preferences
of the patient, the patient's initial expectations, and the realities
of the care received. Disentangling the effects of these variables
upon the results is a major problem. For example, patients' expectations
may be influenced by cultural norms or by health status.
3.4 Further problems result from the design
of satisfaction surveys. These often start from assumptions about
what is important to patients—assumptions which may be influenced
by the service provider's own research needs or by the institutional
organisation of service delivery, or simply by the derivation
of questions from previous literature and surveys. These assumptions
may limit the scope of patients' reporting.
3.5 Some of the results may be too generalised.
Typically, measures of patient satisfaction use opinion poll-style
market research to ask patients to express levels of satisfaction
("I am very/fairly/dissatisfied/very dissatisfied")
or to rate a service ("excellent,/good/adequate/poor/very
poor") on a scale (Likert scale). The results provide very
generalised indicators of the quality of a service that do little
to capture the complexities of modern health care or the diversity
of patients' experiences. They do not necessarily allow service
providers to benchmark that service ("how does this compare
to last year/to similar organisations elsewhere, etc"). And,
most importantly, they may not readily enable service providers
to interrogate the nature of any weakness in the service with
a view to improvement. Knowing that, say 25% of users were "dissatisfied"
with a service may not tell service providers very much about
what, if anything, needs to change. In fact, within the NHS, which
tends to generate high levels of trust and support from patients
and the public, general patient satisfaction questions typically
generate overwhelmingly positive ratings, which may not reflect
reported experience.
3.6 Other results can be more specific and
informative, usually relating to likes and dislikes, and highlighting
items like hospital food, car parking charges, or amenities. While
these issues have some importance, the fact that they show up
clearer results tends to draw a focus onto them which is disproportionate
to their importance compared to, for example, patients' concerns
about their illness and clinical care.
3.7 Patient "experience" surveys
go beyond these measures. In designing a new questionnaire for
a survey, the Picker Institute will initially involve patients
(current or recent users of health care) in qualitative research
in order to ensure the questions cover those areas about which
patients themselves are most keen to talk.
3.8 The questionnaires that are then developed
break down the patient experience into detailed reporting on what
happened with a particular provider at a specific point in time,
by asking patients to respond to questions about whether or not
certain processes or events occurred during a specific process
of care. The results are intended to be factual, rather than evaluative.
This enables a patient to reflect their good experience of the
service as a whole, or of constituent parts of it, while also
reporting those elements that provided a poorer experience. In
this way, even a "very satisfied" patient can contribute
to identifying areas for service improvement.
3.9 For service providers, the results provide
a clearer view of what they need to do to improve the quality
of the service. If 90% of patients report that they were given
good information about the treatment they were to receive in a
hospital, but only 35% say that they were given adequate information
to look after themselves upon discharge, then the priority for
action is clear.
3.10 Moreover, by being part of a national
survey programme, the results can be instantly benchmarked. An
individual health service trust can look at its scores and compare
them to the national picture.
3.11 To demonstrate to the Committee the
value of this "experience" approach, it may be worth
quoting figures from the Picker Institute's analysis of national
trends. In 2005 the Picker Institute reviewed evidence from the
19 national patient surveys that had been completed since 1998,
involving more than one million patients1.
3.12 Clear improvements over time had been
experienced with regard to several of the areas targeted for action
by the government and the NHS—including waiting times for
GP and outpatient appointments, and faster access to cancer specialists,
for example. Patients continued to express high levels of trust
and confidence in health professionals.
3.13 However, it was also possible to identify
a number of key areas in which patients' experience had not improved
over time. For example, in the key area of involvement in decisions
and respect for patients' preferences:
— In 2005, 69% of primary care patients
said they were definitely involved as much as they wanted to be
in decisions about their care—fewer than in 2003 when the
figure was 73%. Only 59% were involved as much as they wanted
to be in medication decisions.
— In 2004, 21% of outpatients and 26%
of A&E patients said staff didn't always listen carefully
to what they were saying—no improvement on previous years.
— In 2004, only 53% of inpatients said
they definitely had a say in decisions about their treatment.
For outpatients the figure was 70%; for A&E patients 64%;
for coronary heart disease patients 64%.
— The proportion of cancer patients
expressing satisfaction with their involvement in decisions fell
from 89%-84% between 2000 and 2004.
3.14 As another example, in the key area of
"clear, comprehensible information and support for self-care",
there had been significant improvements in the provision of information
to patients with cancer and coronary heart disease (both priority
areas within the NHS plan), but there were still severe shortcomings
more generally:
— In 2005, only 61% of primary care
patients said they received enough information about the possible
side-effects of their medicines, the same proportion as in 2003.
— In 2005, 43% of stroke patients said
they were not given information about dietary changes that might
prevent another stroke, and 33% said they were not given information
about physical exercise.
— In 2004, 40% of inpatients, 37% of
outpatients and 61% of A&E patients said they were not told
about danger signals to watch out for—the same proportions
as in previous years' surveys. Repeat surveys in 2005 again showed
little change.
With these results, at a national level, it
is possible for managers, planners, regulators and others to look
for the specific elements of care that should be the focus for
service improvement. Each NHS trust will have similar results
for the same domains of care within its own service, and the ability
to benchmark these against the national data.
Based on its considerable experience of developing,
implementing and analysing the results of these large scale questionnaires,
the Picker Institute belies it is now possible to know—objectively
and authoritatively—both what patients want, and what they
get, within the National Health Service. What patients want are:
— Fast access to reliable health advice
— Effective treatment by trusted professionals
— Shared decisions and respect for
patients' preferences
— Clear information and support for
self-care
— Attention to physical and environmental
needs
— Emotional support, empathy, respect
— Involvement of and support for family
& carers
— Continuity of care and smooth transitions
The value of the evidence base
3.17 The future of patient involvement depends
critically on the continued compilation of an evidence base that
can be used to analyse patients' experiences en masse, to identify
areas for improvement, and to measure progress. The key to this
is the national patient survey programme. This provides several
main benefits.
3.18 The first benefit is that the results
contribute to a national picture of the quality and effectiveness
of health care. They go beyond simple measures of "satisfaction"
by breaking down the care experience into its various elements
in order to identify the strengths and weaknesses of NHS care
nationally—and therefore the potential targets for improvement—through
patients' own reported experiences.
3.19 The second main benefit is that they
inform each trust of its own strengths and weaknesses in patients'
eyes. Moreover, the results each trust receives can be "benchmarked"
against the national picture, showing how well the trust performs
in each area vis a vis its peers. The straightforward statistic
that, for example, 30% of this trust's inpatients were given inadequate
information to manage their own care upon discharge, will be given
context by knowing that this placed it among the worst 20% of
trusts.
3.20 A third benefit is that this system
makes available a set of resources and a "market" of
providers upon which NHS bodies can draw to conduct additional
surveys of their users or client populations outside the national
programme itself. For example, commissioners may ask a survey
provider to help them identify the priorities of a particular
patient group in their locality. The Picker Institute and a number
of other approved survey providers can be commissioned to provide
these "bespoke" surveys. This can save health service
managers and researchers from having continually to reinvent the
wheel of patient feedback techniques.
3.21 A fourth potential benefit of the evidence
base is to feed into structures of public involvement. The publication
of this data by the national regulator and by individual service
providers can be part of transparency and accountability, and
is a necessary precondition to deepening public and patient involvement
in health, which has emerged as a government priority.
3.22 For these benefits, the programme is
relatively cost-effective. It merely requires each trust to spend
a few thousand pounds annually on "customer care" surveys
which, as a ratio to their expenditure budgets as a whole, is
probably minimal compared to customer care expenditure in other
parts of the public sector, and certainly to private sector market
research.
3.23 The national patient survey programme
is, however, not secure. In its short lifetime of less than a
decade it has already had three masters and is due to have a fourth,
when the Healthcare Commission is amalgamated with the National
Social Care Inspectorate and the Mental Health Commission.
3.24 The Picker Institute therefore urges
the Select Committee:
— to express its support for the continuation
of the national patient experience survey programme under the
future regulator.
Unlocking the value of the evidence base
3.25 In its Issues and Questions Paper the
Committee asks what role measures of customer satisfaction should
have in assessing standards. Possessing an evidence base is not,
in itself, any guarantee that there will be an improvement in
service standards. For this, active measures must be taken at
various levels to ensure that the data is actively and effectively
used to identify action plans for change.
3.26 In relation to health services, the
patient survey is open to the criticism that in itself it does
not create change. One has to look at the levers available to
ensure that the evidence is put to active use.
3.27 There is a requirement for trusts to
carry out the surveys, but no requirement to put the results to
work for quality improvement. The Picker Institute, in its role
as a survey provider, offers a package to trusts which includes
a bespoke service to help the trust use the results to involve
patients in developing an Action Plan for quality improvement.
This remains voluntary and is an additional expense for the Trust.
3.28 There are some mechanisms in place
that are supposed to take advantage of the evidence base. For
instance, the existing "Public and Patient Involvement Forums"
based around trusts are required to monitor the action plans that
are supposed to result from trusts' patient surveys. The extent
to which this has been done—and the extent to which trusts
respect and enable this role—are uncertain.
3.29 The PPI Forums are about to be replaced.
Legislation before the House of Commons—the Local Government
and Public Involvement in Health Bill—provides for their
abolition and replacement with "Local Involvement Networks"
or "LINks". These will be organised in geographical
areas (not attached to specific health provider institutions)
and are expected to be voluntary-sector led. The government is
attempting to give maximum flexibility to each local authority
area to establish LINks in their own way.
3.30 The Picker Institute is concerned that
LINks may struggle to clarify and establish their roles. This
is a particular danger with regard to the gathering, analysis
and use of information and intelligence on what patients and the
public want. Making effective use of such resources is challenging
for small organisations, and especially for lay people. But the
problems will be greatly exacerbated if government guidance and
regulation does not clearly point the way.
3.31 The government's most recent explanation
of what LINks could do places considerable emphasis on the gathering
of intelligence and feedback from patients1. For example:
"LINks will therefore be engaged in monitoring
by actively seeking views directly through contributions from
individuals and groups, and indirectly from representatives or
advocates, from complaints and PALs, through surveys, through
comment cards, through websites, and through other methods. Their
strength will be that they are able to engage with a large number
of people rather than relying on the experiences of a few centrally
appointed members."1
3.32 Surprisingly, the national patient
survey programme is not mentioned once in this document. There
is no reference to the existing evidence base. There is no suggestion
that there should be a responsibility for trusts to publish and
share this information with relevant LINks in their user area;
let alone a requirement for commissioners and providers to involve
LINKs in the planning and design of future surveys.
3.33 As a result there is a real and present
danger that LINks will waste time, energy and resources duplicating
the evidence, reinventing the wheel of survey techniques, or worst
of all, struggling with poorer quality evidence when a high quality,
properly validated evidence base already exists and continues
to develop.
3.34 The Picker Institute is recommending
that this could be remedied by amending Section 155 of the Bill,
to enable the Secretary of State, through regulation, to impose
duties on health service providers to: to publish and share promptly
with relevant LINks information from patient surveys; to assist
LINks to analyse and understand the data; and to involve LINks
in the design and commissioning of future patient surveys.
3.35 In relation to the NHS system, there
are departmental standards, against which service providers' performance
is measured and regulated. These Standards for Better Health (Department
for Health 2006) include "core" (compulsory) and "developmental"
standards, which are in effect good practice guidelines but do
not place duties on the committee or the trust.
3.36 The "core" standard that
relates to patient involvement states that:
C17 The views of patients, their carers and
others are sought and taken into account in designing, planning,
delivering and improving health care services.
The Healthcare Commission, as regulator, will
measure performance against this standard during its annual "health
check" of the trust. But it is a rather general standard
that may not be too testing for trusts to uphold.
3.37 Among the "developmental"
standards are some which are more exacting and specific in relation
to patient involvement:
D8: Health care organisations continuously
improve the patient experience, based on the feedback of patients,
carers and relatives.
D9: Patients, service users and, where
appropriate, carers receive timely and suitable information, when
they need and want it, on treatment, care, services, prevention
and health promotion and are:
(a) encouraged to express
their preferences
(b) supported to
make choices and shared decisions about their own health care.
D10: Patients and service users, particularly
those with long-term conditions, are helped to contribute to planning
of their care and are provided with opportunities and resources
to develop competence in self-care.
3.38 These "developmental standards"
are those to which trusts should be "progressing". The
Healthcare Commission states that it will begin assessing such
progress within its "annual health check" programme;
but in 2006-07 this will not include standards 8-101.
3.39 In short, the formal "levers"
within the NHS system, that could incentivise as well as enforce
the use of patient feedback for quality improvement are currently
weak. In its simultaneous memorandum to the House of Commons Health
Select Committee's Inquiry into public and patient involvement
in health, the Picker Institute urges that committee:
— to examine the potential roles of
regulation, inspection and other interventions to require trusts
to demonstrate that they are involving patients in using their
survey results to set priorities for quality improvement.
Professional Standards
3.40 The eight key areas of "what patients
want" outlined at 3.16, above, are significant in demonstrating
that patients' experience of health care centres, not only on
service provider organisations, but much more importantly, on
their interaction with the health professionals who deliver the
care and treatment. Patients want professionals who are good communicators;
who are interested and sympathetic; who guide them to and through
clear and comprehensible information about their health and about
treatment choices; who involve them in decisions; and who support
them in their self-care.
3.41 It is therefore also crucial to bring
patients' feedback to bear on these professionals, as well as
on the institution for which they work. There is strong evidence
from patient experience surveys that outdated hierarchical and
paternalistic approaches continue to dominate health professionals'
transactions with patients. If the government goal of a "patient-centred"
NHS is to be achieved, there must be a shift to a culture of "patient/professional
partnership". Professionals will need to acquire and demonstrate
new skill sets revolving around health literacy, communication
skills, enabling shared decision-making, and supporting patients
to manage their own care.
3.42 This means that the feedback from patients'
reporting of their experience and what matters to them should
now be used to revise and set professional practice standards
and to revise professional education and training.
3.43 As a good example, the General Medical
Council recently revised its guidance to doctors on the professional
standards they should follow. "Good Medical Practice"
now states that it is a duty of any doctor registered with the
GMC to "Work in partnership with patients", which means:
"listen to patients and respond to their
concerns and preferences; give patients the information they want
or need in a way they can understand; respect patients' right
to reach decisions with you about their treatment and care; and
support patients in caring for themselves to improve and maintain
their health".
3.44 But the GMC covers only one part of
the health workforce. The Picker Institute wants to see similar
standards adoped and pursued by the range of other health professions.
3.45 Furthermore, guidance like "Good
Medical Practice" will only have an effect if there is momentum
at key points throughout the system to get the professionals to
uphold those standards: momentum from the government, the NHS
executive, the bodies delivering professional education and training,
the professional councils, the NHS trusts and the commissioners
of NHS care to set the right kinds of incentives for the professionals
to gain these skills and meet these standards.
3.46 Patient feedback could be useful, too,
in assessing the performance of professionals as individuals and
within their clinical teams. There is no reason why patient's
feedback on their experience should not be used as part of the
assessment of the technical competence and the interpersonal skills
of professionals. This could in turn contribute to the revalidation
and continuing professional development of the health professionals.
3.47 However, at this more individual level,
there is a priority need to conduct further research to develop
up to date and effective models for patient feedback questionnaires
relating to professional performance.
January 2007
| 
