Memorandum from Tetlow Associates
This memorandum represents the response of Tetlow
Associates to the issues raised in the Issues and Questions Paper:
"Public Services: putting people first?"
It follows a research project we have undertaken
recently on behalf of the Local Government Association and National
Consumer Council on Developing Customer Satisfaction Measures
for Local Government Services. A summary of this report is available
at: http://www.lga.gov.uk/Documents/Briefing/projects/developing%20measures.pdf.
We provide responses below to your questions
1, 2, 3, 8 and 10.
Question 1: Can public services learn from
the way that either non public sector organisations or overseas
governments make use of user experience in service delivery and
design?
A: learning from the private sector
This is not only possible but essential. The
context in which people experience public services is framed by
the offerings of all the organisations with which we interact
including the private sector, and this influences our expectations.
The fact that supermarkets are now open 24 hours a day, or that
many services are available on-line inevitably frames expectations
about the way the world can be experienced and puts pressure on
public services to learn from such examples if they are to avoid
a public perception that they are anachronistic and outdated.
The techniques used by the private sector have
much to offer to public services. Specific techniques designed
to enhance customer insight (eg segmentation, techniques for understanding
the nature of the customer experience and the relative importance
to the customer of aspects of products and services) have informed
many of the developments of commercial products and services of
recent years and have given leading companies significant competitive
edge.
Some of these techniques are being applied to
public services but they are hampered by the fact that there is
no generally accepted standard and practice is diverse. Thus wisdom
derived from research about the responses of some groups in society
to one service is rarely used in the design of others. The use
of basic sociodemographic categories in customer research on public
services is widespread but sophisticated segmentation methodologies
as an essential element in service design is not. Very few public
service surveys seek to understand the reasons why people feel
pleased or displeased with the service they receive. Most rely
on answers to simple satisfaction questions which give a general
impression but do not help pinpoint the precise nature of improvement
customers might welcome.
In order to deepen understanding, many public
services use qualitative techniques including deliberative forums,
focus groups, citizens' juries and other methodologies which give
people greater freedom to express their views in depth and discuss
their experience than is possible using quantitative surveys.
Many of these techniques are based on private sector examples
and are extremely useful. However, their use is patchy and only
a small population can be involved, so the use of surveys with
greater rigour is also required.
A related key area where public services can
learn from the private sector is in the use to which the data,
once gathered, is put. Public service organisations are often
not structured in a way which allows the information which is
known about customer responses to change the way they work. The
results of surveys often do not transparently inform crucial decision
making processes and are not part of the essential management
information which boards draw on in making strategic decisions.
Examples of the ways private companies have changed processes
to respond directly to customer preferences are common. This is
less evident in public services.
B. Learning from other countries
International best practice has useful lessons.
The best known example is that of Canada, where there are national
standards for measuring the drivers of satisfaction with services
and regular national surveys ("Citizens First"). The
key lessons for the UK from the Canadian example are:
— Rigorous focus on understanding the
reasons for people's satisfaction and dissatisfaction has produced
an overall significant increase in satisfaction with services.
— The use of rigorous, standardised
methodologies for understanding customers across services has
proved successful for a wide range of services at all jurisdictional
levels.
— Achieving good results has not been
costly and many Canadian officials believe that money has been
saved.
Further information about the Canadian example
is available from the Institute of Citizen Centred Service. (http://www.iccs-isac.org).
Other examples worth looking at include the
US American Customer Satisfaction Index, itself the basis for
the European Customer Satisfaction Index which has been used on
a small scale in Scandinavia. These examples provide an interesting
example of a standardised way of managing data about customers
but the results in terms of driving change are less clear than
is the case in Canada.
Question 2: Is it possible to set minimum
standards for public services? If so, how is this best done?
There are risks in setting minimum standards,
especially those related to customer satisfaction or which depend
on perceptions. Satisfaction is influenced by many issues which
may not be directly related to the quality of the service received.
(There is evidence, for example, that satisfaction with services
varies between sociodemographic groups.) Thus achieving, say,
a standard for customer satisfaction of 65% satisfied or very
satisfied may be relatively easy to achieve in one area but very
challenging in another, and the difference may be explained largely
by differences in the populations of the two areas.
More effective are minimum standards for improving
people's experience based on previous survey evidence. This approach
was taken in Canada—each service was set a target of a 5%
improvement in satisfaction over a five year period, and this
was achieved by 2005.
Setting minimum standards for performance other
than customer satisfaction or measures of customer experience
is widespread. The effect of these on people's relationship to
services depends on the extent to which the standard relates to
what people want and need from the service. Proxy targets are
often used in this area—for example waiting time targets
setting minimum standards in the NHS. Proxy targets carry risks—whilst
it may be reasonable, provided there is robust supporting evidence,
to assume that improvement in the target will improve the customer
experience, organisations tend to focus on the content of the
target or minimum standard rather than the spirit, especially
if there are financial or reputational issues for the organisation
in meeting the minimum standard. This produces perverse incentives
and runs risks of damaging the customer experience when, for example,
hospitals focus on ensuring that no-one waits more than four hours
in A and E as their first priority and pays less attention to
the actual experience of the patient, who may simply find themselves
waiting in another part of the hospital instead, with a possible
resulting adverse effect on their overall experience of their
treatment.
Setting standards needs to be based on current,
specific information about the issues which matter to people.
Whilst there may be widespread evidence that people value being
treated with dignity and respect, simple questions of social care
customers such as "were you treated with dignity and respect"
may produce a "yes/no" answer which could produce a
result that a certain percentage of customers felt they were treated
with dignity and respect. However, we know that elderly people
are more likely to answer "yes" to this question than
young ones, and thus the answer may not tell us a great deal about
the service itself but is more likely to be a reflection of social
attitudes to authority. More effective are specific question derived
from the detail of the experience, such as "Did staff talk
about you as if you were not there". Using measures built
around such questions can be used to derive a meaningful overall
score for the customer experience, and measuring this over time
could form a useful minimum standard for services.
Question 3A: What role do measures of customer
satisfaction have in assessing the standard of public services?
Overall measures of customer satisfaction do
have a role in assessing the standard of public services. However,
taken on its own there are difficulties with this concept:
(a) Policy goals may not be consistent with
high levels of customer satisfaction. For example, we do not necessarily
wish to prioritise simple satisfaction with higher education—students
may be satisfied with undemanding coursework, if asked, but their
longer term interests and that of society more widely are better
served by sacrificing simple immediate satisfaction for greater
challenge.
(b) Satisfaction levels vary by sociodemographic
groups, with, for example, older women very much more satisfied
than younger men. Simple satisfaction scores need to be understood
carefully, and the underlying detail explored.
Measures of the extent to which organisations
focus on their customers as part of their day to day processes
may be more helpful. For example, the extent to which they gather
data about the user experience and use it to drive processes,
alongside evidence that satisfaction scores and rating of specific
service experiences are improving, may be more meaningful.
Question 3B: How should user views be monitored?
A variety of techniques are appropriate for
understanding user views and experience. "Views"—which
may be a matter largely of opinion can be a different concept
from "experience". Thus a user may hold a view that
a particular service is not the way he or she might ideally like
it to be—for example he or she may dislike the building
or the style of a publication, but nonetheless the service experience
is satisfactory because he/she was dealt with expertly by staff.
It is probably useful to measure both, but important to be clear
which is being investigated.
The most important aspect of the user/customer
relationship with the service to measure is the direct experience.
Qualitative work is necessary to develop a detailed understanding
of the nature of the experience and the issues and details which
matter to people. This can then be used as a basis for general
surveying using qualitative and/or quantitative techniques. If
similar instruments are used on a widespread basis, and repeated
over time, important trends can be tracked which can be used to
inform policy development at local and national levels. This is
not possible at present and requires improvement.
Question 3C: How can the cost effectiveness
of user surveys and feedback be assessed?
The issue of the cost effectiveness of understanding
customers is often raised. During our research for the Local Government
Association and National Consumer Council we found evidence that
local authorities gather a great deal of data at currently unquantified
expense (estimating the actual cost may be the subject of further
research) much of which is little used to drive their processes.
Additionally, some processes undertaken by public
service organisations in the belief that they are a necessary
part of delivering a good customer service may prove, on the basis
of sound evidence, to be unnecessary.
We believe strongly that simply looking at this
issue from the perspective that additional survey work represents
an extra "cost" to organisations is mistaken. Experts
in Canada involved in managing the Citizens' First initiative
and the use of their "Common Measurements Tool" believe
that this initiative has at worst been cost neutral and have anecdotal
evidence that individual institutions have saved money as a result.
Private companies regard the cost of understanding
customers as an essential element of their business and crucial
to their survival. We believe that effective public service organisations
need to give this issue similar priority if their democratic mandate
is to be honoured. It is not to be regarded as an optional "extra"
in addition to other aspects of strategic management—strategic
management is impossible to undertake effectively without it.
Assessment of cost effectiveness may include
measurement of the achievement of desired policy outcomes. These
are often behavioural—for example reducing crime, encouraging
environmental responsibility or healthy living. These depend on
effective communication with customers which can only be based
on detailed understanding of them and their experience. Thus even
in areas where direct service experience is less relevant measurement
of the extent of behavioural change may be an indication of the
cost effectiveness of customer insight. Similarly, satisfied public
service customers, especially those for personal services such
as social services, are likely in some cases to be helped to manage
their lives effectively and perhaps reduce their dependence on
acute health services.
Question 8: Do official consultations typically
manage to capture the views of the right people? What kinds of
consultation are most effective in engaging with the appropriate
people?
The simple answer to this question is no. Engagement
with many forms of consultation requires social capital which
many people simply do not have—the language of documents
and the processes to be engaged with are inaccessible to many
and poorly advertised. Only those people familiar with such processes
are inclined to engage, producing the "usual suspects"
syndrome and the sense that consultation has become a ritual which
passes for real public engagement but is in fact a poor substitute.
This requires further work. There are many techniques
available for understanding peoples' perspectives on policy issues,
many of which are in widespread use in the market research industry.
Government has much to learn from these techniques and their use
might make for more effective policy making and easier acceptance
of policy announcements and legislation.
This problem exists at national and local level,
and the capacity of some individuals to be heard can lead to an
understandable perception by politicians that they are in touch
with public opinion, when in fact they are simply hearing the
loudest voices. It is essential that rigorous research, geared
especially to ensure true representation of the population, forms
part of the policy engagement process.
Question 10: How does user influence relate
to wider issues of democratic accountability?
This issue is of critical importance where services
are intensively used but only by a minority of people—for
example looked after children. Many people will have opinions
about such services, especially when they are the subject of press
coverage, but few will have close first hand knowledge and it
is difficult for politicians to determine the extent to which
widespread public opinion should influence decision making rather
than knowledge of the user experience, which may be rather different.
This situation is made worse when good data about the actual performance
of the service in the eyes of its users is not readily available.
It is essential that sound democratic processes
of engagement are supported by good data about customer experiences,
so that public opinion can be balanced alongside real evidence
about the way services are performing in the eyes of their customers.
January 2007
| 
