Examination of witnesses (Questions 101
- 119)
TUESDAY 7 DECEMBER 1999
MR J REES
and MR B PAGE
Chairman
101. May I formally welcome you to the Committee
this morning? Jonathan Rees, you are described here as Director,
Modernising Public Services. Is this not the Service First programme?
It is now called Modernising Public Services?
(Mr Rees) Yes, it is, indeed.
102. And Ben Page, who is Director of MORI.
A director or the director?
(Mr Page) No, I am not the director, I am Director
of Government Research at MORI.
103. Perhaps not quite as impressive as it sounds
here but nevertheless awfully impressive. Thank you very much
for coming to talk to us and help us with our inquiry into innovations
in participation. I understand that you would like to make a presentation
to start with and we can ask questions about it afterwards. Off
you go.
(Mr Page) If you are happy with that.
104. That would be excellent.
(Mr Rees) (figure 1) As you say, I am Director
of the Modernising Public Services group in the Cabinet Office,
which is the group which is responsible for improving the quality
and responsiveness of public services. As such we report to Ian
McCartney and Dr Mowlam. Another way of putting it is that we
were responsible for chapters 3 and 4 of the Modernising Government
White Paper. That is some of the context. What we thought we would
do is we would give you a very brief overview of the People's
Panel, why we set it up and so on and then Mr Page will talk about
some of the first results. (figure 3) We set up the People's
Panel in the summer of 1998. It was a decision of the new Government
to set it up. It is run for us by MORI; we have a contract with
them. The Panel itself comprises 5,000 people who, as it says
there, are representative of the UK population in as many different
ways as we can make them representative. We can use it for either
quantitative research, big opinion surveys, or qualitative, smaller
focus groups. All the results are published. So that in a nutshell
is what the Panel is. (figure 2) I thought I would set
out a little of the context in which we set up the Panel, why
we went for a Panel, how we set it up, what we have used it for,
what we do with the results and our future plans. (figure 4)
The overall objective was set out in the Modernising Government
White Paper with which you will be familiar, which is that we
want public services to be more responsive to both groups and
individuals. It was in that context that the decision that the
Panel should be set up was taken. (figure 5) It is not
the only new initiative that the Cabinet Office is taking forward.
For instance, Ministers have spent the last six months running
a series of listening to older people events up and down the country.
We have a programme called Better Government for Older People,
which has 28 pilots which are experimenting in different ways
of trying to engage people about public services generally. There
was an exercise which was run by another bit of the Cabinet Office,
the Women's Unit, Listening to Women, which resulted in a publication
which the Prime Minister launched at the beginning of October.
We have Charter Mark and that has been around for about eight
years now, but that is another of the mechanisms for encouraging
different bits of the public sector to be more responsive to their
users. So successful organisations have to show that they have
consulted users and got complaints handling processes in place.
We issue best practice from the Cabinet Office on how you go about
consulting users and we have done a lot of work with the National
Consumer Council on what the right mechanisms are. The White Paper
said that we were going to introduce a new consumer test which
Ministers will be making an announcement about quite soon, which
is designed to try to ensure that for central government and its
agencies the consumers' interests are put first. Then we have
the People's Panel. Broadly speaking there is a range of different
ways in which we are trying to find new ways of getting to people
and making sure that their views inform what we are doing. (figure
6) The People's Panel itself was set up because Ministers
took the view that they needed a new mechanism to try to find
out what people thought, particularly about public services and
what needed to be done to improve them. Why a panel rather than
any other form of research? I think the Panel has three key advantages.
One, it enables us to track views over time; you are going back
more or less to the same people, so you can see how their views
have changed. Secondly, by going for a Panel of 5,000 we have
a group of people whom we have asked about their usage of public
services. If you want to talk to people who have used the legal
system in the last year or you want to find out what people think
about bus services in the North East, it is relatively easy for
MORI to pick those people out and for us to ask them specific
questions. That is a sort of second advantage. The third is that
it enables us to do what we call here cross-sectoral research.
We are talking to people about their experiences of government
as opposed to what they think of the Benefits Agency or what they
think of the Employment Service. That is a third advantage. Finally,
it raises the profile of consultation. There is a lot of consultation,
a lot of research which goes on across government. The People's
Panel is one of the mechanisms which has perhaps attracted a bit
of public attention. That is why we set up a panel. (figure
7) When Ministers decided they wanted to do this, we consulted
fairly widely about how best it was to be done. We also had a
pilot to see whether it worked before we began the main recruitment.
That all took place in the spring of 1998. We then recruited the
full 5,000, who are representative in terms of age, gender, regional
distribution and so on. Unlike some of the local authority panels,
where you can if you want apply to be a member, in this case,
as Mr Page will explain later, recruitment is totally random so
that is why we know it is a representative section. It was recruited
between June and September last year. (figure 8) It is
available for use across the public sector, so it is not just
something the Cabinet Office uses. To date eight other government
departments have put questions in either one of our quantitative
surveys or done qualitative work with us. Secondly, we are committed
to being totally open with the results. All the results are published
on our website. We produce a four-page or a six-page summary of
the results which we try to circulate widely and we have sent
the Committee copies of those. We have what we nicely call a support
group. This is a group of people to help advise us, both from
other government departments and from outside. We have some academics
on it and we have a representative of the National Consumer Council,
designed to try to ensure that we are tapping into other research
that is going on in government but also that we are asking the
right questions. Finally, as the Panel is a new approach, we are
committed to evaluating it. Ministers said that it would be set
up for three years. We are in the process of conducting the evaluation
at the end of the first year to try to see how it is working.
That is part of the overall process. (figure 9) What do
we we use the Panel for? We have done three fairly major quantitative
surveys. The first wave was as a result of consulting all 5,000
people. Since then we have not needed to consult 5,000. The second
and the third waves have both been around 1,000 people. The first
wave asked generally about people's usage of services and was
part of the recruitment exercise, but it also asked about things
like attitudes to electronic government, one-stop shops and so
on. The second wave included questions from DETR about local democracy,
complaints handling, transport. The third wave, which we published
in July, included some general questions about public service
standards, how long you would expect to wait for a reply to letters
and so on, as well as research which helped inform the DETR's
Urban White Paper, Housing Green Paper. All of those three waves
have tended to be an amalgam of a range of different questions,
in part to keep people interested when they are being asked. We
have also done a fair degree of qualitative work with focus groups,
with smaller groups of people on issues like biosciences and Modernising
Government. Before we produced the White Paper we asked a selected
number of people about their experiences at certain life episodes
such as bereavement or when they needed care, just to find out
what the public perception was about how joined up government
was, how responsive it was. We have used it for things like that.
The Women's Unit have done a number of focus groups with it to
try to define what women's real concerns are. I thought you would
probably ask me about the cost so I would say that the cost in
total of the Panel to date is about £632,000, of which the
Cabinet Office has spent about £500,000; the remainder coming
from the other departments which have used it. (figure 10)
What do we do with the results? Clearly we publish the results.
We are also wanting to use the results to inform the policy-making
process. I mentioned the example where we had asked people about
their experiences of government as they saw it. As a result of
that, it has helped prompt the setting up of some action teams
which are looking at life episodes from the users' point of view.
DSS used it and they used the results to help determine how they
should take forward their own modernisation programme. MAFF, for
instance, carried out some qualitative research into an information
booklet on GM foods. They got a group together, showed them the
booklet, asked them what they thought about it. As a result the
booklet is to be rewritten. There is some evidence that it is
informing policy making but clearly that is one of the aspects
we are going to be looking at in the evaluation. There is a time
lag between getting the results and actually seeing the results
in policy making. Clearly it is only one of the aspects. There
is consultation, there is research, other research, there is a
whole range of other issues which Ministers take into account.
(figure 11) Looking at the future, we shall be publishing
shortly, probably early in the new year, some research we have
done on older people's attitudes to public services. This is an
example where we have taken evidence from the first three waves
and actually looked at it through the eyes of a particular group,
in this case older people, to see whether their attitudes are
different from the general public's. That will come out. We are
engaged at the moment in what we call the fourth wave, the fourth
main quantitative wave. This is looking at the key issue of demand
for 24 x 7 services, that is services seven days a week, 24 hours
a day, where do people see the greatest need for extended hours.
This is again fulfilling one of the White Paper's main commitments.
We are also using it to look at public awareness of Charter Mark.
The fifth wave will repeat some of the questions we asked in the
first wave, so we shall be able to see what the change is, but
it will also be a base line in some senses for where we stand
on modernising government across the board, against which we shall
be able to compare in three years' time. Finally, we are engaged
in what we call an ethnic minority booster. Although the Panel
is representative of the population as a whole, clearly some groups,
though representative, will be fairly small. So the number of
Bangladeshi women will actually be three or four as a representative
sample. If we want to look at the particular interests of those
groups, we need to boost them. That is why Ministers decided to
do an ethnic minority booster which will enable us to tap into
the views of that group more closely. (figure 12) In conclusion,
we like to call it a world first at national level. We are not
sure any other country has done it, though there are about 100
local authority panels in this country. It is flexible. It can
be used in a number of different ways. We have used it in some
of them; there are other things we still have not done with it.
It is a key part of the overall Modernising Government drive.
If you are happy, Mr Page will now run through some of the technical
aspects.
105. Yes, thank you very much.
(Mr Page) (figure 14) I do not know how much
detail you would like on the methodology, but Mr Rees has given
you some of the key figures. I can give you a few slides on the
absolute detail. People have been recruited face to face in their
own homes, so we are trying to get people's considered views in
a properly representative spread of locations right across the
UK, which is very, very important. The interviews are actually
captured on computer so that we can process the information accurately
and quickly.
106. Do people get paid?
(Mr Page) Do the people get paid for taking part?
No. I know it looks like a long time. The British public, as long
as they understand why they are being asked to do it, are amazingly
sympathetic towards taking part in very long interviews. Some
government departments actually run interviews which take an hour
and a half.
107. MORI of course gets paid.
(Mr Page) MORI gets paid because the interviewers
collecting the information need to be fed and watered, yes.
108. So MORI is getting paid but the punters
are not getting paid.
(Mr Page) The punters do not get paid, no.
109. It is a good system, is it not?
(Mr Page) You are welcome to try it. It does not pay
as well as others.
(Mr Rees) The punters do get paid for certain sorts
of things. If you are going to do a focus group, the convention
is that you will pay people to attend the focus group say £10.
(Mr Page) More than that; it has gone up.
(Mr Rees) We do not pay them for being a member of
the Panel. We do give them a free People's Panel pen when they
become members.
110. Gosh. I can see why you are not short of
volunteers then.
(Mr Page) If people give up a whole day to come to
an event, we would actually pay them £50 and get them there.
It is commensurate. In terms of the sample design, it is a random
selection of enumeration districts; these are the small buildings
blocks of the census. These have been selected at random and we
made sure we had the correct proportion in each region, the correct
proportion of different types of housing, different types of neighbourhood.
Within each one there is a random selection of addresses so that
we are not just allowing the interviewers to go to the ground
floor of the tower block because he or she does not like to climb
up the stairs because the lift is not working. If the flat selected
is at the top of the tower block he or she will be sent to the
top of the tower block and not allowed to interview anywhere else.
It is also very important in stopping us just speaking to people
who wish to take part or who have plenty of time to take part.
The samples are clustered in enumeration districts which is important.
This exercise, as with all of these things, is always a tradeoff
between absolute accuracy and cost. Also, if one wishes to go
back to people, if one has a pure random sample right across the
country of one home selected on the Isle of Muck, the cost of
sending an interviewer back to visit that person on the Isle of
Muck by ferry is extremely important, but the clustering means
we do have a proper spread of locations and in aggregate it is
representative. In terms of recruiting the Panel, some of the
piloting work was very interesting in seeing how people react
to an approach by government to engage with them. Many people
find that quite a threatening experience and because of that we
wrote to people who lived in middle class areas, because they
were quite keen to join and to tell the Government what they thought
of them, but we found people living in more working class areas
were rather suspicious. So the initial contact there was by the
interviewer with a letter and explanatory brochure as well. We
left all the people whom we recruited a brief leaflet detailing
the approach, why it was being done, how it was going to be used
and of course the famous pen as a small incentive to remind them
they now belonged to the People's Panel. (figure 16) Once
we actually get to each home, a person is chosen at random. It
is not the person who has lots of opinions on everything and who
will tell us what they think, it is one person at random within
that household. If it is a couple, it does not mean that it is
always the husband or always the wife who is chosen, they are
selected at random. The interviewer is making up to five calls
back to find the selected person. If the selected person was not
in, the interviewers would actually keep calling back to get the
correct person, because if one does not do that, one ends up with
a sample which disproportionately represents the elderly and the
unemployed and people who tend to be at home more. If people said
no, we sent back a different interviewer. So we are trying to
minimise refusals all the time, not by antagonising people, because
that is not good for either MORI or the Cabinet Office, but to
try to make sure that all the time we are getting a representative
cross section of people to join. We have been very careful in
terms of the representativeness of the sample. We perhaps do not
have time to go into it today, but I am more than happy to provide
you with figures on the exact profile. (figure 18) The
data which is collected covers all of the things on that chart:
gender, age, ethnicity, in detail, disability, the actual income,
the qualifications of people in the household, which newspapers
they read, whether or not they have access to the internet, their
e-mail addresses. We tried to get a very broad picture about these
people so we can dissect the groups in any way we need to for
future research, very cost effectively to target people who work
part time. If you do not ask that question at the start you will
not know which ones to go back to.
Mr Oaten
111. Do you ask about their politics?
(Mr Page) To be honest, I wanted to but I was forbidden.
The reason I wanted to of course was to check and see whether
I did have the correct proportion as the polls were showing at
the time for the different parties. However, there are absolutely
no party politics in this whatsoever, so no. On service delivery,
we have looked at a range of services and tried to get detail
on a great many of them, both as a base line and also to be able
to go back and actually see how these people's views have changed.
One of the key benefits of a panel as opposed to snapshot or ad
hoc surveys you read about all the time, is that you can actually
see how individuals' views are changing as opposed to the aggregate.
You can see that the British public might be becoming more or
less satisfied with train services, for example, but it could
be that different people are becoming more satisfied. You do not
know that unless you have a panel. Other things we have looked
at: expectations. I am going to give you some details of some
of these results in a minute. Also looking at what people see
as the most important services to them. There is a whole range
of things in a 52-minute interview. (figure 21) This just
gives you some examples of some of the results. They really are
just snippets. Overall levels of satisfaction; here are some of
the very highest ones; these are net figures, those who are satisfied,
minus dissatisfied, amongst those who say they use each one; very
high levels of satisfaction, again consistent with other research,
certainly that MORI has conducted, even on behalf of those utilities
or indeed locally for libraries, for local government. Road and
pavement maintenance was one of the worst rated services; again
utterly consistent with all MORI's work locally and local government
gets rather worse ratings than some of the services it delivers.
There is a selection of scores. (figure 23) This chart
is interesting in the sense that it shows some of the correlations
that one is able to look at with a large data set of this type,
where you can see on the X axis, the horizontal axis, how well
informed people feel each of those services keep them if they
use it and in the vertical axis how satisfied they are with each
one. You can see that there is a reasonable correlation in the
sense that there are no services where there is very high feeling
that they are kept very well informed but where satisfaction is
low. One of the messages which is coming out of this is that all
other things being equal, the public service which keeps people
well informed about what it is doing will tend to be better regarded
for example than the one which does not tell you anything at all.
Chairman
112. Is not the conclusion quite the opposite?
(Mr Page) That if you are satisfied you feel well
informed.
113. Fire and emergency services. Is that not
the test case, that people do not care about being informed about
it so long as it turns up when something goes wrong?
(Mr Page) Absolutely correct, but that is one of the
exceptions.
114. But surely it is an absolutely compelling
exception.
(Mr Page) It depends very much upon your relationship
with the service. You probably do not want to have intimate contact
with the fire brigade every day of the week. If on the other hand
you are a council tenant and your flat roof is leaking, and also
there is an investment programme for your estate, and you are
also claiming housing benefit from your local authority, you will
be very interested in when they are going to get round to fixing
the roof. The fire service is very much about "Come and put
it right". Certainly I can show you separately from this—
115. I do not want to make a meal of it, but
I should have thought that the conclusion quite simply is that
tenants want their repairs to be done. If their repairs are being
done, they are more interested in that than being informed when
their repairs are not going to be done.
(Mr Page) They are; absolutely. I do not disagree
with you at all. However, if you take two local authorities and
you compare the Audit Commission performance indicators for each
of those authorities and the actual level of council tax, any
count of widgets one could look at in terms of what those two
authorities are delivering, the one which is also keeping people
informed about its plans, about when they are actually going to
get the money to do the SRB programme, etcetera, will tend to
be better regarded, which is what I am measuring here, than the
one which does not. There are exceptions. The Health Service is
one. We do not need it to keep us informed; it makes us well.
We have different relationships with the Fire Service, some of
these emergency services. (figure 25) I shall push on.
On the whole do you think public services are better or worse?
One quarter said they were better, one third said they were worse.
Just some examples: 38 per cent no change over the last five years.
It would be interesting to see how those views change as we go
on.
Mr Oaten
116. What date are those?
(Mr Page) Summer 1998; June to September.
117. Quite old data then.
(Mr Page) Yes. The other thing is that with some of
these questions you will find that there will always be people
who say things are getting worse and there will always be people
who say things are getting better. Do they meet your expectations?
One per cent of the public say public services greatly exceed
their expectations; ten per cent said they fall a long way short.
Of course it is very interesting to think about what people's
expectations actually are and whether they are rising. Fifty-one
per cent say they are about what they expect. These are just examples.
I am not developing a thesis here or anything else.
Chairman
118. You report this in your first report on
the People's Panel which says half the Panel think that public
services are what they would expect. What on earth does that mean?
(Mr Page) MORI was asked to ask the question.
Mr Oaten
119. Do you give advice on the questions?
(Mr Page) It is an interesting question. The other
point about some of these questions is that it is not the absolute
answer that is interesting, it is the trend.
|
