Select Committee on Public Administration Minutes of Evidence

Examination of witnesses (Questions 160 - 179)



  160. What if it has not when you do it with 5,000 people, if there is a divergence between the views of these 5,000 people as measured and the base?
  (Mr Page) They have all become extremely cheerful about everything.

  161. Absolutely. If they have all become very interested in local government and learned how difficult it actually is to run a local council, very sympathetic to the local councils and very positively supportive of them in large numbers. What are you going to do?
  (Mr Page) I think it extremely unlikely that would happen and there is no evidence that that is happening on the Panel. Then you are left with one of the interesting things which is why they have become like that? Was that by being on the Panel? Was it because of the relatively spartan newsletter they got sent three times? Was it the pen? Was it the process of joining the Panel? It is very interesting to see that because if it does have that effect, which I must admit I do not think it will have, we would want to see why and we may need then to ask how reliable the Panel is. You would obviously call it into question if it were wildly out.
  (Mr Rees) Conditioning is potentially a real problem, but to date it has not been one. The reasons for that are that we do not go back to people that often. We have talked about four, five, six waves, but after the first one, we have only gone back to 1,000 rather than to the full 5,000. If somebody is pulled out to go to a focus group, then we do not go back to them for a considerable period of time, if at all. In both those ways I suspect of the 5,000, I do not know and it is one of the things for which we are going to use the evaluation, I suspect some of them do not realise they are on the People's Panel at all, which is very, very different from a local authority panel where you go along and you talk about what you are going to do with dustbins and so on. They are very different animals.

Mr White

  162. I am interested in how you have done the evaluation and the relationship between the Cabinet Office and the individual departments. If departments are misusing the People's Panel, what sanctions do the Cabinet Office have or will it be, "Oh, they're misusing it so they're not going to get the benefits of it"?
  (Mr Rees) We have set the Panel up, we are therefore the guardians of its use. We have more requests from departments to use it than we particularly want to use it in the sense that if it is overused it is actually we who are going to have to pay for it rather than the department. What we have is a system whereby we want to get a balanced range of questions. If you are asking a sample of questions, you might find, as we did, that the Health & Safety Executive wanted to ask questions about gas safety. We put that in. I doubt if anybody would want to have a full half hour questionnaire about gas safety, but four or five questions in a longer run yield good answers. We do not want to use the Panel for inappropriate questions and that is our responsibility to make sure it does not get misused.

  163. If department X is saying they have certain questions and you say that is a question which department Y ought to be asking as well, do you go to department Y and say, "Should we not be doing this right across government rather than just department X"?
  (Mr Rees) Yes; indeed. Part of what we want to achieve is that department X and department Y both know that they are asking very similar questions.

  164. Department Y is not asking any questions at all.
  (Mr Rees) If it is not asking any questions then indeed the other half of that is to go to them and say, "Department X is doing these questions and it seems they are pretty relevant to what you are doing. Do you want to add on, piggyback, have a view as to whether these are the right sorts of questions?" Yes.


  165. How many departments have not used the People's Panel at all?
  (Mr Rees) Eight have used it. I can give you a list but we have not actually had any questions from the territorials. I suspect that the overwhelming majority of government departments have used it in one way or another.

  166. That is Cabinet-Office-speak for the territorial departments, is it not?
  (Mr Rees) Sorry?

  167. By "territorials" do you mean Scotland, Wales?
  (Mr Rees) Perhaps I should not have called them territorials. Scotland, Wales, Northern Ireland.
  (Mr Page) Scotland have commissioned an analysis.
  (Mr Rees) Scotland have commissioned an analysis. Coming back to the point about breaking down, the number of people from 5,000 in Scotland is sufficient to have a sensible breakdown. We have also done it for London as well.

  168. Treasury?
  (Mr Rees) The Treasury are commissioning questions in the next wave. We are working very closely with the Treasury on the consumer test for which the next wave will be used as a base line.

Mr Browne

  169. One of the principal purposes of this, as I understand it from your presentation, is to help formulate policy. Are there any examples of it helping to formulate policy up until now?
  (Mr Rees) I said it is to help inform policy, which is probably Cabinet-Office-speak as well, for which I apologise. There are examples. I gave the example for instance of MAFF, but another example is that we used a focus group by DfEE to look at value added league tables, something which as a general policy everybody is in favour of. DfEE ran through and of course found that people found them awfully difficult to understand and that then went back to the way they presented that information. It is at that sort of micro level where I can give you some examples. At the macro level, on a major policy area like electronic government, it will have an impact over time, but I cannot say yet that there is a causal effect. I do not think there ever will be a direct causal effect.

Mr White

  170. But not welfare reform or some of the more contentious ones.
  (Mr Rees) No; no, it is fair to say not.

Mr Browne

  171. Both of the examples you have given us at the micro level informing policy have been presentational issues. Both of them have been about presentation. The MAFF one was a document which was put out and changed.
  (Mr Rees) It is about communication and how well you communicate with people, which is actually one of the core issues that government has to deal with. It is one of the core issues behind modernising government. If we are not communicating with people effectively, then we are wasting our time.

  Mr Browne: I accept that.

Mr Oaten

  172. I am very interested in the bit about the Treasury deciding what questions they may like to be in the future survey. I understand that the findings are public. At what point would somebody like me, a backbencher, discover and be able to get access to the question the Treasury wanted to ask?
  (Mr Rees) At the moment it would only be when the results were published. We have not been asked. Mr Page may have a view, but once we have started the research I suspect there is a potential risk, before we have actually talked to all of the people, if the questions are then publicised, because they can then begin to think what their responses might be. I do not know how much that is a potential risk, how much a practical risk. Once we have actually done all the questions and the research, there is clearly a time delay whilst the data is crunched and so on. I cannot see any reason why—

  173. There would be nothing to stop me tabling a Parliamentary Question to ask what questions the Treasury have asked be included in the next People's Panel survey.
  (Mr Rees) What I am saying is that I would rather you did not do it until we have done the actual research because that could impact on the research. Once the research is done I can see no reason why the information could not be available.

  174. Have any Ministers attended any of the focus groups behind a screen just to hear what is literally said rather than read the reports?
  (Mr Page) They attended some before the Panel was set up, but I do not think they have actually attended anything to do with the Panel that I am aware of, although in the Listening to Women exercise obviously the Ministers will have been out, not necessarily directly with the Panel, but they will have been out doing various road shows and all sorts of other things. Peter Kilfoyle attended a workshop we ran on Better Government for Older People but that was actually separate from the Panel. Anybody who was terribly well known and the public was likely to recognise—

  175. Behind a screen.
  (Mr Page) We do it with local government. MORI has not done it much with Westminster politicians, although there are one or two I can think of.

Mr White

  176. One of the areas in which I am quite interested is how you translate a pilot into general practice. I was interested that you did a pilot first. How did you actually translate that pilot into general practice and make a success of it?
  (Mr Page) The reason we did the pilot was really to see how people would feel about joining a panel run by Whitehall. Will certain sorts of people be more willing to join than others, logistically what will be involved, etcetera, etcetera? What we found was that it was about four times as difficult in time and effort to get somebody under 24 to agree as somebody over 24. It is partly because they are busy and also people on lower incomes are much more suspicious of government public services per se than, say, a retired judge. As a result of that, we then changed the methodology to boost up, we changed the design of the survey. Two major things came out of the pilot. One was not putting the words "citizens' jury" in the recruitment literature because people found it frightening and they thought it was about judgement by your peers rather than a research exercise because they do not know what it is, they do not know what a citizens' jury is. We sent out letters from the then Chancellor of the Duchy of Lancaster to the pilot people and asked if they would like to join the panel, because the one hypothesis was that it would be very interesting and exciting to be invited by a Government Minister to take part in this exercise. What we found was that people who lived in wealthy parts of the South East rang up and were dead keen, whatever their politics, to tell the Government what they thought, but perhaps people of different extraction in East London were much less keen to ring up, or in fact did not ring up at all, in fact would not even answer the door. As a result of that, we changed the way in which we approached people and given that there was more enthusiasm amongst middle class types, to coin a phrase, we wrote to them, because they seemed to respond positively to a letter. However, because we obviously wanted to get the correct proportion of working class people, people on low incomes, we decided not to send them a letter. So you get a friendly local interviewer on your doorstep who would explain it to you and they realise it is not somebody investigating them or their takeup of benefits or anything else. The other thing was a booster sample of young people to make sure we did actually get the young people. Because we knew it was going to be very, very difficult, we had a core random sample of addresses and then neighbouring addresses at each one, which we screened for 16 to 24-year-olds whom we could get to join the Panel. We actually had to make more approaches to get them onto it. In the end we got the right number.

  177. Was the budget for the pilot and the budget for the overall People's Panel the same budget?
  (Mr Rees) Yes.

  178. It was quite clear you were going immediately from a pilot to the full-scale thing, that the pilot was ironing out the problems rather than checking the concept?
  (Mr Rees) Yes. You are right, what we had was a decision by Ministers that they wished to set it up. So the pilot was about the best way of setting it up, the best way of approaching people, the most cost effective way. It was not a pilot to determine whether or not this was a good idea and that is why we have set it up for three years, to do the evaluation. Given some of the time lags in this, I am not sure we could have piloted it before going to the full 5,000.

  179. How did you get to the socially excluded?
  (Mr Page) Basically it is random. The methodology actually used for the recruitment of the Panel is very similar to that used for things like the English House Condition Survey, which DETR has run since the 1970s, if not further back. Basically it is a random selection of addresses, so every home in the sample and every enumeration district has an equal chance of selection, and once you have sampled that, every address within the enumeration district has an equal chance.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2000
Prepared 15 May 2000