Examination of witnesses (Questions 160
- 179)
TUESDAY 7 DECEMBER 1999
MR J REES
and MR B PAGE
160. What if it has not when you do it with
5,000 people, if there is a divergence between the views of these
5,000 people as measured and the base?
(Mr Page) They have all become extremely cheerful
about everything.
161. Absolutely. If they have all become very
interested in local government and learned how difficult it actually
is to run a local council, very sympathetic to the local councils
and very positively supportive of them in large numbers. What
are you going to do?
(Mr Page) I think it extremely unlikely that would
happen and there is no evidence that that is happening on the
Panel. Then you are left with one of the interesting things which
is why they have become like that? Was that by being on the Panel?
Was it because of the relatively spartan newsletter they got sent
three times? Was it the pen? Was it the process of joining the
Panel? It is very interesting to see that because if it does have
that effect, which I must admit I do not think it will have, we
would want to see why and we may need then to ask how reliable
the Panel is. You would obviously call it into question if it
were wildly out.
(Mr Rees) Conditioning is potentially a real problem,
but to date it has not been one. The reasons for that are that
we do not go back to people that often. We have talked about four,
five, six waves, but after the first one, we have only gone back
to 1,000 rather than to the full 5,000. If somebody is pulled
out to go to a focus group, then we do not go back to them for
a considerable period of time, if at all. In both those ways I
suspect of the 5,000, I do not know and it is one of the things
for which we are going to use the evaluation, I suspect some of
them do not realise they are on the People's Panel at all, which
is very, very different from a local authority panel where you
go along and you talk about what you are going to do with dustbins
and so on. They are very different animals.
Mr White
162. I am interested in how you have done the
evaluation and the relationship between the Cabinet Office and
the individual departments. If departments are misusing the People's
Panel, what sanctions do the Cabinet Office have or will it be,
"Oh, they're misusing it so they're not going to get the
benefits of it"?
(Mr Rees) We have set the Panel up, we are therefore
the guardians of its use. We have more requests from departments
to use it than we particularly want to use it in the sense that
if it is overused it is actually we who are going to have to pay
for it rather than the department. What we have is a system whereby
we want to get a balanced range of questions. If you are asking
a sample of questions, you might find, as we did, that the Health
& Safety Executive wanted to ask questions about gas safety.
We put that in. I doubt if anybody would want to have a full half
hour questionnaire about gas safety, but four or five questions
in a longer run yield good answers. We do not want to use the
Panel for inappropriate questions and that is our responsibility
to make sure it does not get misused.
163. If department X is saying they have certain
questions and you say that is a question which department Y ought
to be asking as well, do you go to department Y and say, "Should
we not be doing this right across government rather than just
department X"?
(Mr Rees) Yes; indeed. Part of what we want to achieve
is that department X and department Y both know that they are
asking very similar questions.
164. Department Y is not asking any questions
at all.
(Mr Rees) If it is not asking any questions then indeed
the other half of that is to go to them and say, "Department
X is doing these questions and it seems they are pretty relevant
to what you are doing. Do you want to add on, piggyback, have
a view as to whether these are the right sorts of questions?"
Yes.
Chairman
165. How many departments have not used the
People's Panel at all?
(Mr Rees) Eight have used it. I can give you a list
but we have not actually had any questions from the territorials.
I suspect that the overwhelming majority of government departments
have used it in one way or another.
166. That is Cabinet-Office-speak for the territorial
departments, is it not?
(Mr Rees) Sorry?
167. By "territorials" do you mean
Scotland, Wales?
(Mr Rees) Perhaps I should not have called them territorials.
Scotland, Wales, Northern Ireland.
(Mr Page) Scotland have commissioned an analysis.
(Mr Rees) Scotland have commissioned an analysis.
Coming back to the point about breaking down, the number of people
from 5,000 in Scotland is sufficient to have a sensible breakdown.
We have also done it for London as well.
168. Treasury?
(Mr Rees) The Treasury are commissioning questions
in the next wave. We are working very closely with the Treasury
on the consumer test for which the next wave will be used as a
base line.
Mr Browne
169. One of the principal purposes of this,
as I understand it from your presentation, is to help formulate
policy. Are there any examples of it helping to formulate policy
up until now?
(Mr Rees) I said it is to help inform policy, which
is probably Cabinet-Office-speak as well, for which I apologise.
There are examples. I gave the example for instance of MAFF, but
another example is that we used a focus group by DfEE to look
at value added league tables, something which as a general policy
everybody is in favour of. DfEE ran through and of course found
that people found them awfully difficult to understand and that
then went back to the way they presented that information. It
is at that sort of micro level where I can give you some examples.
At the macro level, on a major policy area like electronic government,
it will have an impact over time, but I cannot say yet that there
is a causal effect. I do not think there ever will be a direct
causal effect.
Mr White
170. But not welfare reform or some of the more
contentious ones.
(Mr Rees) No; no, it is fair to say not.
Mr Browne
171. Both of the examples you have given us
at the micro level informing policy have been presentational issues.
Both of them have been about presentation. The MAFF one was a
document which was put out and changed.
(Mr Rees) It is about communication and how well you
communicate with people, which is actually one of the core issues
that government has to deal with. It is one of the core issues
behind modernising government. If we are not communicating with
people effectively, then we are wasting our time.
Mr Browne: I accept that.
Mr Oaten
172. I am very interested in the bit about the
Treasury deciding what questions they may like to be in the future
survey. I understand that the findings are public. At what point
would somebody like me, a backbencher, discover and be able to
get access to the question the Treasury wanted to ask?
(Mr Rees) At the moment it would only be when the
results were published. We have not been asked. Mr Page may have
a view, but once we have started the research I suspect there
is a potential risk, before we have actually talked to all of
the people, if the questions are then publicised, because they
can then begin to think what their responses might be. I do not
know how much that is a potential risk, how much a practical risk.
Once we have actually done all the questions and the research,
there is clearly a time delay whilst the data is crunched and
so on. I cannot see any reason why—
173. There would be nothing to stop me tabling
a Parliamentary Question to ask what questions the Treasury have
asked be included in the next People's Panel survey.
(Mr Rees) What I am saying is that I would rather
you did not do it until we have done the actual research because
that could impact on the research. Once the research is done I
can see no reason why the information could not be available.
174. Have any Ministers attended any of the
focus groups behind a screen just to hear what is literally said
rather than read the reports?
(Mr Page) They attended some before the Panel was
set up, but I do not think they have actually attended anything
to do with the Panel that I am aware of, although in the Listening
to Women exercise obviously the Ministers will have been out,
not necessarily directly with the Panel, but they will have been
out doing various road shows and all sorts of other things. Peter
Kilfoyle attended a workshop we ran on Better Government for Older
People but that was actually separate from the Panel. Anybody
who was terribly well known and the public was likely to recognise—
175. Behind a screen.
(Mr Page) We do it with local government. MORI has
not done it much with Westminster politicians, although there
are one or two I can think of.
Mr White
176. One of the areas in which I am quite interested
is how you translate a pilot into general practice. I was interested
that you did a pilot first. How did you actually translate that
pilot into general practice and make a success of it?
(Mr Page) The reason we did the pilot was really to
see how people would feel about joining a panel run by Whitehall.
Will certain sorts of people be more willing to join than others,
logistically what will be involved, etcetera, etcetera? What we
found was that it was about four times as difficult in time and
effort to get somebody under 24 to agree as somebody over 24.
It is partly because they are busy and also people on lower incomes
are much more suspicious of government public services per
se than, say, a retired judge. As a result of that, we then
changed the methodology to boost up, we changed the design of
the survey. Two major things came out of the pilot. One was not
putting the words "citizens' jury" in the recruitment
literature because people found it frightening and they thought
it was about judgement by your peers rather than a research exercise
because they do not know what it is, they do not know what a citizens'
jury is. We sent out letters from the then Chancellor of the Duchy
of Lancaster to the pilot people and asked if they would like
to join the panel, because the one hypothesis was that it would
be very interesting and exciting to be invited by a Government
Minister to take part in this exercise. What we found was that
people who lived in wealthy parts of the South East rang up and
were dead keen, whatever their politics, to tell the Government
what they thought, but perhaps people of different extraction
in East London were much less keen to ring up, or in fact did
not ring up at all, in fact would not even answer the door. As
a result of that, we changed the way in which we approached people
and given that there was more enthusiasm amongst middle class
types, to coin a phrase, we wrote to them, because they seemed
to respond positively to a letter. However, because we obviously
wanted to get the correct proportion of working class people,
people on low incomes, we decided not to send them a letter. So
you get a friendly local interviewer on your doorstep who would
explain it to you and they realise it is not somebody investigating
them or their takeup of benefits or anything else. The other thing
was a booster sample of young people to make sure we did actually
get the young people. Because we knew it was going to be very,
very difficult, we had a core random sample of addresses and then
neighbouring addresses at each one, which we screened for 16 to
24-year-olds whom we could get to join the Panel. We actually
had to make more approaches to get them onto it. In the end we
got the right number.
177. Was the budget for the pilot and the budget
for the overall People's Panel the same budget?
(Mr Rees) Yes.
178. It was quite clear you were going immediately
from a pilot to the full-scale thing, that the pilot was ironing
out the problems rather than checking the concept?
(Mr Rees) Yes. You are right, what we had was a decision
by Ministers that they wished to set it up. So the pilot was about
the best way of setting it up, the best way of approaching people,
the most cost effective way. It was not a pilot to determine whether
or not this was a good idea and that is why we have set it up
for three years, to do the evaluation. Given some of the time
lags in this, I am not sure we could have piloted it before going
to the full 5,000.
179. How did you get to the socially excluded?
(Mr Page) Basically it is random. The methodology
actually used for the recruitment of the Panel is very similar
to that used for things like the English House Condition Survey,
which DETR has run since the 1970s, if not further back. Basically
it is a random selection of addresses, so every home in the sample
and every enumeration district has an equal chance of selection,
and once you have sampled that, every address within the enumeration
district has an equal chance.
|
