Memorandum by The National Audit Office
(COM 119)
TELEPHONE SURVEY OF PRIMARY CARE TRUST COMMISSIONERS
CONTENTS
Summary
Part 1: Background, methodology and previous NAO
findings on commissioning
Background
Methodology
Previous NAO findings on commissioning
Part 2: Results of the telephone survey
The current state of commissioning
The main challenges faced by PCT commissioners
Key actions to overcome the challenges identified
Practice-based commissioning
World Class Commissioning
SUMMARY
1. Commissioning in the NHS is the process
of deciding what health and care services are needed, acquiring
them and ensuring that they meet requirements. It is a complex
process with responsibilities ranging from assessing population
needs, prioritising health outcomes, procuring products and services,
and managing service providers.
2. The Department of Health's vision for
healthcare commissioning is framed by "World Class Commissioning"
which aims to deliver a more strategic and long-term approach
to commissioning services, with a focus on delivering improved
health outcomes. This vision is underpinned by a set of 11 organisational
competencies which requires primary care trusts (PCTs) to demonstrate
their ability to: locally lead the NHS; work with local partners;
engage with public and patients; collaborate with clinicians;
manage knowledge and assess needs; prioritise investment; stimulate
the market; promote improvement and innovation; secure procurement
skills; manage the local health system and make sound financial
investments. The World Class Commissioning programme incorporates
a yearly assessment of all PCTs performance against these 11 key
commissioning competencies.
3. Practice-based commissioning is seen
by the Department of Health as a key driver to achieving world-class
commissioning. It involves increasing the participation of GPs
and local clinicians in commissioning health and wellbeing services,
as well as specialist services.
4. In November 2009, the House of Commons
Health Select Committee requested the support of the National
Audit Office (NAO) to help it obtain the views of PCT commissioners
for its inquiry into healthcare commissioning. This memorandum
has been prepared in response.
5. This memorandum is based on the results
of a telephone survey with PCT commissioners undertaken for the
National Audit Office by Ipsos MORI between 5 and 15 January
2010. A total of 114 telephone interviews were conducted,
which represented a response rate of 75%. A subsequent focus group
was held with PCT commissioners and senior NHS staff with a strategic
view of commissioning to explore the results of the telephone
survey in more detail. This has also fed into the memorandum.
MAIN FINDINGS
6. The survey found that PCT commissioners
were generally very positive about the state of commissioning
in its entirety across the PCT, with 95% stating that commissioning
was going very well or fairly well. When commissioners were asked
specifically in which key service areas commissioning was going
well, the percentage of respondents varied from 50% for specialised
commissioning to 77% for primary care services.
7. Our focus group participants thought
that these results were a fair reflection of the current state
of commissioning, but commented that the results needed to be
set in context. Commissioning has improved from where it was a
year or two ago, but there is clear evidence of the need for further
improvement. The World Class Commissioning assurance process results
for 2009 showed that PCTs received an average score of 1.65 across
all competencies out of a possible score of four. Competencies
that received lower than average scores were market development,
procurement skills and prioritising investment.
8. The survey results can also be set in
context by the results of recent National Audit Office value for
money reports on a range of health services. These reports have
highlighted weaknesses, at PCT level, in all three stages of the
commissioning cycle: strategic planning; procuring services; and
monitoring and evaluation.
9. Strategic planning issues identified
in our reports were a lack of knowledge of local needs, insufficient
use being made of available data in planning activity, poor understanding
of costs related to addressing issues and a lack of clinical evidence
to commission services effectively. Procurement issues we identified
were fragmented procurement activities not realising scales of
economy, poor understanding of costs of activity and limited benchmarks
to guide PCT spending. Monitoring and evaluation issues we identified
were poor understanding of whether services provided achieve value
for money, the impact of activity not being assessed adequately,
a lack of influence over providers and a lack of reliable methods
in place to measure predicted benefits.
Key challenges facing PCT commissioners
10. The survey highlighted a wide range
of key challenges facing PCT commissioners. The most frequently
mentioned challenges (percentage of commissioners mentioning the
challenge as one of their top three challenges) were:
— financial pressures and constraints (58%)—40%
of commissioners mentioned this as their first major challenge;
— commissioning acute services (21%);
— clinical engagement to support change
(15%);
— improving commissioning skills within
the PCT (14%);
— greater commissioning capacity within
the PCT (14%); and
— ability to decide how to prioritise services
(10%).
11. Focus group participants supported these
as the key challenges facing commissioners and commented that
many of the challenges were inter-linked. For example, the focus
group participants noted that tackling the top challenge, financial
pressures and constraints, will require action in the area where
the majority of the commissioning budget is spent, which is acute
services (the second most frequently cited challenge), which will
require engagement with clinicians (the third most frequently
cited challenge). Another inter-related theme identified by the
focus group was knowledge management and prioritisation—the
availability and quality of information and how it is used to
drive service improvements. The focus group also noted that these
issues also relate to skills and capacity.
Key actions identified to address the challenges
12. PCT commissioners identified a number
of key actions which would enable them to address the challenges
they face. The most frequently mentioned actions by commissioners
were:
— stronger commissioning skills and capacity
(28%);
— clearer central messages about prioritisation
(22%);
— encouraging PCTs to work together where
appropriate (13%);
— clearer political leadership about challenges
facing PCT (12%);
— more and better performance information
about patient experiences and outcomes (11%); and
— greater leverage for PCTs over providers
(11%).
13. For those commissioners that identified
financial pressures and constraints as a key challenge, the most
frequently cited actions to address this challenge were using
the financial crisis to promote change (18%), clearer messages
about prioritisation (16%), and encouraging PCTs to merge where
appropriate to local conditions (15%).
14. The focus group supported the actions
identified by the survey respondents and in particular drew out
two issues: the opportunity to use the current and future financial
situation to promote change and the fact that there needs to be
an understanding that contractual levers are not enough to tackle
the issues facing commissioners. Focus group participants noted
that it will require strong leadership from PCTs to drive through
changes along with effective engagement with both clinicians and
the public to realise the changes necessary.
Practice-based commissioning and World Class Commissioning
15. PCTs commissioner's views on practice-based
commissioning were also positive with 77% of commissioners tending
to agree or strongly agree that their PCT is well placed for practice-based
commissioning to take on a more prominent commissioning role.
However, there was wide variation in the reported proportion of
a PCT's commissioning budget that is directly determined by practice-based
commissioning, from none up to the largest category, 76-100%.
Our focus group suggested that this wide variation reflected the
fact that PCTs are at different stages of development in their
practice-based commissioning function—some are very developed,
whilst others are only just beginning to develop this function.
16. PCT commissioners' general perception
of World Class Commissioning was that it had had a positive impact
on local commissioning, with 84% tending to agree or strongly
agreeing that World Class Commissioning has made their PCT more
effective at commissioning and 61% tending to agree or strongly
agreeing that World Class Commissioning has delivered measurable
benefits to patients.
17. When PCT commissioners were asked how
they had changed as a result of World Class Commissioning, a number
of positive themes emerged:
— a greater focus on outcomes;
— increased collaboration;
— more emphasis on commissioning skills;
— meeting patients needs; and
— improved systems and processes and re-organisation
where necessary.
18. Although the majority of comments on
what had changed as a result of World Class Commissioning were
positive, there were a number of more mixed responses highlighting
a number of concerns with World Class Commissioning. Examples
included views that World Class Commissioning is just a tick box
exercise and that the World Class Commissioning assurance process
takes time to complete when there are other things staff could
be getting on with. Some concerns were also expressed as to whether
the World Class Commissioning programme had the right balance
between being an assurance programme and a development programme.
PART : BACKGROUND,
METHODOLOGY AND
PREVIOUS NAO FINDINGS
ON COMMISSIONING
Background
1.1 Commissioning in the NHS is the process
of deciding what health and care services are needed, acquiring
them and ensuring that they meet requirements. It is a complex
process with responsibilities ranging from assessing population
needs, prioritising health outcomes, procuring products and services,
and managing service providers.
1.2 The Department of Health's vision for
healthcare commissioning is framed by "World Class Commissioning"
which aims to deliver a more strategic and long-term approach
to commissioning services, with a focus on delivering improved
health outcomes. This vision is underpinned by a set of 11 organisational
competencies which requires primary care trusts (PCTs) to demonstrate
their ability to: locally lead the NHS; work with local partners;
engage with public and patients; collaborate with clinicians;
manage knowledge and assess needs; prioritise investment; stimulate
the market; promote improvement and innovation; secure procurement
skills; manage the local health system and make sound financial
investments. The World Class Commissioning programme incorporates
a yearly assessment of all PCTs performance against these 11 key
commissioning competencies.
1.3 Practice-based commissioning is seen
by the Department of Health as a key driver to achieving world-class
commissioning. It involves increasing the participation of GPs
and local clinicians in commissioning health and wellbeing services,
as well as specialist services.
1.4 In November 2009, the House of Commons
Health Select Committee requested the support of the National
Audit Office (NAO) to help it obtain the views of PCT commissioners
for its inquiry into healthcare commissioning. This memorandum
has been prepared in response.
Methodology
1.5 To seek the views of PCT commissioners,
in the timescale that was needed, we undertook a telephone survey
of PCT commissioners. To inform the development of the survey
questionnaire, we held a focus group with four senior PCT staff
with a strategic view of commissioning. We also received written
feedback on potential survey questions from two PCT commissioners.
The survey was undertaken on our behalf by Ipsos MORI.
1.6 The telephone interview for the survey
sought PCT commissioners' views on the following:
— how well commissioning was going in its
entirety and particular commissioning areas where it was going
well;
— the three main challenges faced by PCT
commissioners when commissioning services and the particular areas
of commissioning in which these challenges are faced;
— the actions that would be most helpful
in overcoming these challenges;
— whether practice-based commissioning
was well placed in their PCT to take on a more prominent role
in commissioning local health services and what proportion of
their total commissioning budget was spent on practice-based commissioning;
— whether World Class Commissioning has
made PCTs more effective at commissioning local health services
and has delivered measurable benefits to patients;
— what has changed as a result of World
Class Commissioning; and
— whether there were any commissioning
challenges that are specific to local circumstances not covered
in the previous questions.
1.7 For each PCT, we identified two senior
commissioners, including the Director of Commissioning or an equivalent
role. Only one commissioner from each PCT was interviewed by Ipsos
MORI. Over the period, 5 to 15 January 2010, 114 telephone
interviews were conducted, which represented a response rate of
75%.
1.8 Once the telephone survey was completed,
we held another focus group with four PCT commissioners and other
senior NHS staff with a strategic view of commissioning to explore
the results of the survey in more detail.
Previous NAO findings on commissioning
1.9 The NAO has not undertaken a value for
money study focused solely on healthcare commissioning. Nevertheless,
healthcare commissioning has featured in many of our value for
money reports on health services. These reports have highlighted
weaknesses, at PCT level, in all three stages of the commissioning
cycle: strategic planning (Figure 1); procuring services (Figure
2); and monitoring and evaluation (Figure 3).
Figure 1
NAO FINDINGS ON COMMISSIONING—STRATEGIC
PLANNING
|
| Issue identified | Report area
|
|
| Insufficient use was made of available data in planning activity
| Mental Health[19] Autism[20]
|
| PCTs lacked adequate knowledge of local needs
| Services to reduce alcohol harm[21]End of life care[22]AutismDementia[23] Rheumatoid arthritis[24]
|
| Poor understanding of costs related to addressing issue
| End of life careNational Chlamydia Screening Programme[25]
|
| Insufficient use of cost benefit analysis |
Services to reduce alcohol harmEnd of life care
|
| PCTs lack clinical evidence to commission services effectively
| Rheumatoid arthritis |
|
| Source: National Audit Office reports
| |
Figure 2
NAO FINDINGS ON COMMISSIONING—PROCUREMENT
|
| Issue identified | Report area
|
|
| Fragmented procurement activities not realising scales of economy
| National Chlamydia Screening Programme |
| Poor understanding of costs of activity |
Services to reduce alcohol harmEnd of life careNational Chlamydia Screening Programme
|
| Limited benchmarks to guide PCT spending |
National Chlamydia Screening Programme |
|
| Source: National Audit Office reports
| |
Figure 3
NAO FINDINGS ON COMMISSIONING—MONITORING AND EVALUATION
|
| Issue identified | Report area
|
|
| Poor understanding of whether services provided achieve value for money
| Services to reduce alcohol dependenceEnd of life care
|
| Impact of activity not assessed adequately
| National Chlamydia Screening Programme |
| Lack of influence over providers | National Chlamydia Screening Programme
|
| No reliable method in place to measure predicted benefits
| National Chlamydia Screening Programme |
|
| Source: National Audit Office reports
| |
PART : RESULTS
OF THE
TELEPHONE SURVEY
The current state of commissioning
2.1 Primary Care Trust (PCT) commissioners were asked
to comment on how well commissioning, in its entirety across the
PCT, is going at the moment (Figure 4). Almost all of the commissioners
(95%) stated that commissioning is going very well or fairly well.
No respondents perceived commissioning across the PCT to be going
not at all well.
Figure 4
HOW WELL IS COMMISSIONING GOING IN ITS ENTIRETY ACROSS
THE PCT?
Source: National Audit Office analysis of telephone survey
with PCT commissioners
2.2 PCT commissioners were asked to comment on which
areas of commissioning are going well (Figure 5). More than six
in ten commissioners thought it was going well in each area of
commissioning apart from specialised commissioning, where 50%
thought it was going well. Only three per cent of commissioners
thought that it was going well in all areas.
Figure 5
AREAS OF COMMISSIONING THAT ARE GOING WELL
Source: National Audit Office analysis of telephone survey
with PCT commissioners
The main challenges faced by PCT commissioners
2.3 PCT commissioners were asked to identify the three
main challenges they faced when commissioning services in their
PCT. Figure 6 displays the challenges most frequently mentioned
by commissioners. When listing their major challenges, 40% of
commissioners cited financial pressures and constraints as their
first major challenge. Commissioners mentioned a wide range of
challenges—individual challenges cited by less than 5% of
respondents made up almost one-third (33%) of the total responses.
Some of the other challenges cited are highlighted in Figure 7.
Figure 6
THE MAIN CHALLENGES FACING PCT COMMISSIONERS
Source: National Audit Office analysis of telephone survey
with PCT commissioners
Figure 7
OTHER KEY CHALLENGES FACED BY COMMISSIONERS
Source: National Audit Office analysis of telephone survey
with PCT commissioners
2.4 Our focus group, established to explore the results
of the survey, supported these as the key challenges facing commissioners
and commented that many of the challenges were inter-linked. For
example, the focus group participants noted that tackling the
top challenge, financial pressures and constraints, will require
action in the area where the majority of the commissioning budget
is spent, which is acute services (the second most frequently
cited challenge), which will require engagement with clinicians
(the third most frequently cited challenge). Another inter-related
theme identified by the focus group was knowledge management and
prioritisation—the availability and quality of information
and how it is used to drive service improvements. The focus group
also noted that these issues also relate to skills and capacity.
2.5 For each of the key challenges identified, commissioners
were asked to identify which commissioning areas the challenge
particularly applied to (Figure 8). Over three-quarters (77%)
of the challenges mentioned apply to acute services and 69% apply
to commissioning in its entirety.
Figure 8
THE COMMISSIONING AREAS WHERE THE KEY CHALLENGES ARE FACED
Source: National Audit Office analysis of telephone survey
with PCT commissioners
2.6 PCT commissioners were asked to comment on whether
there were any commissioning challenges that are specific to their
local circumstances that were not covered in the previous questions.
Although individual commissioners identified a number of challenges
that were specific to their local circumstances, most of these
had already been cited by other commissioners under the three
key challenges. Local challenges not mentioned as one of the three
key challenges were local population changes, such as an ageing
population or an increasing ethnic minority population, and the
PCT not being coterminous with their local authority.
Key actions to overcome the challenges identified
2.7 PCT commissioners were asked to identify what actions
would be most helpful to them to overcome each of the three challenges
identified earlier. Figure 9 displays the most frequently
cited actions. Other actions mentioned, not presented in Figure
9, accounted for 18% of all the actions cited. Figure 10 displays
some of these other actions.
Figure 9
KEY ACTIONS TO ADDRESS THE CHALLENGES FACED BY PCT COMMISSIONERS
Source: National Audit Office analysis of telephone survey
with PCT commissioners
Figure 10
OTHER ACTIONS IDENTIFIED BY COMMISSIONERS TO ADDRESS THE
CHALLENGES THEY FACE
Source: National Audit Office analysis of telephone survey
with PCT commissioners
2.8 Figure 11 displays the most frequently mentioned
actions to address the most common challenge—financial pressures
and constraints.
Figure 11
KEY ACTIONS TO ADDRESS THE MOST COMMON KEY CHALLENGE—FINANCIAL
PRESSURES AND CONSTRAINTS
Source: National Audit Office analysis of telephone survey
with PCT commissioners
Practice-based commissioning
2.9 PCT commissioners were asked to identify approximately
what proportion of their PCT's total commissioning budget is directly
determined by practice-based commissioning (Figure 12). There
is a wide variation in the proportion that commissioner reported.
Our subsequent focus group to discuss the survey finding suggested
that this wide variation reflected the fact that PCTs are at different
stages of development with their practice-based commissioning
function.
Figure 12
PROPORTION OF PCT'S COMMISSIONING BUDGET SPENT ON PRACTICE-BASED
COMMISSIONING
Source: National Audit Office analysis of telephone survey
with PCT commissioners
2.10 PCT commissioners were asked to comment on the extent
to which they agreed or disagreed with the statement "My
PCT is well placed for practice-based commissioning to take on
a more prominent role in commissioning local health services"
(Figure 13). Over three-quarters of commissioners (77%) tended
to agree or strongly agreed with the previous statement, with
15% tending to disagree or strongly disagreeing.
Figure 13
"MY PCT IS WELL PLACED FOR PRACTICE-BASED COMMISSIONING
TO TAKE ON A MORE PROMINENT ROLE IN COMMISSIONING LOCAL HEALTH
SERVICES"
Source: National Audit Office analysis of telephone survey
with PCT commissioners
World Class Commissioning
2.11 PCT commissioners were asked to comment on the extent
to which they agreed or disagreed that World Class Commissioning
has made their PCT more effective at commissioning local health
services (Figure 14) and whether World Class Commissioning had
delivered measurable benefits to patients in their PCT (Figure
15). More than four in five (84%) commissioners strongly agreed
or tended to agree that it has made their PCT more effective at
commissioning health services, of which more than one in three
(37%) strongly agreed. Less than one in ten disagreed (9%), whilst
7% either don't know or neither agreed nor disagreed. The majority
of commissioners also agreed that World Class Commissioning has
delivered measurable benefits to patients (61%). However, 18%
disagreed and 22% either don't know or say they neither agreed
nor disagreed. This indicates that two in five commissioners are
yet to see measurable benefits from World Class Commissioning
for patients.
Figure 14
"WORLD CLASS COMMISSIONING HAS MADE YOUR PCT MORE
EFFECTIVE AT COMMISSIONING LOCAL HEALTH SERVICES"
Source: National Audit Office analysis of telephone survey
with PCT commissioners
Figure 15
"WORLD CLASS COMMISSIONING HAS DELIVERED MEASURABLE
BENEFITS TO PATIENTS IN YOUR PCT"
Source: National Audit Office analysis of telephone survey
with PCT commissioners
2.12 PCT commissioners were also asked to comment on
whether they had changed what they do as a result of World Class
Commissioning. Commissioners generally responded that the policy
has had a positive impact. Key positive themes emerging from their
comments (see Figure 16) were:
— a greater focus on outcomes;
— increased collaboration;
— more emphasis on commissioning skills;
— meeting patients needs; and
— improved systems and processes, and re-organisation
where necessary.
Figure 16
POSITIVE COMMENTS FROM COMMISSIONERS ON HOW THEY HAVE
CHANGED WHAT THEY DO AS A RESULT OF WORLD CLASS COMMISSIONING
A greater focus on outcomes
— "WCC helped us at a strategic level to become
more focused on what we have to achieve and to decide what we
as an organisation will focus overall our efforts and funds on."
— "We've become more much more focussed. We
are commissioning against the competencies so it gives us much
more clarity and leverage and given us more status and authority
with our providers."
— "It focuses us more on outcomes rather than
just on number crunching."
— "There's been more of a focus on outcomes
and of stakeholder engagement and an increasing ability to manage
providers."
Increased collaboration
— "I think we have become more aware of the
wider implications of our actions and the need to involve and
collaborate on a wider stage."
— "We have extended our partnership work with
the whole health and social care community and we have focused
more closely on developing PBCs and supporting them so that they
can form their own consortium."
More emphasis on commissioning skills
— "There is more emphasis on needs assessment
and more emphasis on the skills required in order to deliver WCC."
— "We've got more focus on ensuring the appropriate
commissioning skills within the staff and also increased focus
on outcomes."
— "We've got better procurement processes in
place and better contracting in place."
Meeting patients needs
— "There's a lot more awareness of following
a process that focuses on and identifies patient need and enhancing
and changing services as a result."
— "We are looking objectively at where we can
make changes and work more closely with patients and meeting their
needs."
— "WCC changed the way we engage with our community
and particularly the public. We are much more active in public
engagement and WCC has really embedded clinical engagement as
part of the normal business of the PCT."
Improved systems and processes and re-organisation where
necessary
— "We've become much more structured and use
competencies to understand good practice."
— "We have made some changes to some of our
systems and to some of our processes and made a lot of progress
in engaging with the public in a more systematic way."
— "We have improved the systems and processes
that underpin commissioning."
— "I think there is a more rigorous and systematic
deployment of commissioning techniques across the whole commissioning
agenda."
— "We much more focused on market management
and the procurement of clinical services as an area."
— "Far more strategic and structured in the
way we approach our services around assessing needs and priorities."
Source: National Audit Office analysis of telephone
survey with PCT commissioners
2.13 Although the majority of comments were positive,
there were a number of more mixed responses highlighting a number
of concerns with World Class Commissioning (Figure 17). Examples
include views that World Class Commissioning is just a tick box
exercise and that the World Class Commissioning assurance process
takes time to complete when there are other things staff could
be getting on with.
Figure 17
NEGATIVE COMMENTS FROM COMMISSIONERS ON HOW THEY HAVE
CHANGED WHAT THEY DO AS A RESULT OF WORLD CLASS COMMISSIONING
— "The negatives have been very much around
this assurance process. We've just finished the assurance process
submission and it's taken a number of people over three or four
months, when there are other things they could be getting on with."
— "There is a danger with all the pressure put
on us with WCC that it can slow the process down and time is of
the essence at the moment given current financial situation."
— "My take is WCC is regarded as an assurance
process first and foremost and may have changed ways individuals
operate but in itself it hasn't had the radical effect it might
have done in commissioning process."
— "Working on a day to day level as we do I
can say that it has had little effect on my day to day work."
— "Highlighting the gaps in the system its often
short-termist and that causes problems as we don't know how real
it is. ie we don't know if we are doing this [WCC] just because
we have to or because its better."
— "We've looked at the competencies and the
actions we've taken to address these competencies and where there
are gaps whether that translates to benefits to patients the jury
is out."
— "I think WCC isn't inherent in everything
we do. It's not within us it is a process which is added afterwards,
so at the end of the year we are trying to find proof of how we
have done it. In fact it should be the other way round so we should
be including it in everything we do… we have only been doing
it for a couple of years so it is too early to have it as part
of our culture just yet—a lot of our processes have changed
recently but that is more through audit that WCC."
— "[WCC] has felt like a tick box exercise to
show you have that competency. I would question the value of the
competencies and sub competencies as some of the sub competencies
are not good value for money and increase costs."
Source: National Audit Office analysis of telephone
survey with PCT commissioners
February 2010
19
Helping people through mental health crisis: the role of crisis
resolution and home treatment centres, Report by the Comptroller
and Auditor General, HC 5 Session 2007-08. Back
20
Supporting people with autism through adulthood, Report
by the Comptroller and Auditor General, HC 556 Session 2008-09. Back
21
Reducing alcohol harm: health services in England for alcohol
misuse, Report by the Comptroller and Auditor General, HC
1049 Session 2007-08. Back
22
End of life care, Report by the Comptroller and Auditor
General, HC1043 Session 2007-08. Back
23
Improving services and support for people with dementia,
Report by the Comptroller and Auditor General, HC 604 Session
2006-07. Back
24
Services for people with rheumatoid arthritis, Report by
the Comptroller and Auditor General, HC 823 Session 2008-09. Back
25
Young people's sexual health: the National Chlamydia Screening
Programme, Report by the Comptroller and Auditor General,
HC 963 Session 2008-09. Back
| 
