Select Committee on Work and Pensions Minutes of Evidence

Examination of Witnesses (Questions 250-259)




  250. Welcome to our next set of witnesses, who are from the Evaluation, Development and Review Unit of the Tavistock Institute: John Kelleher, Elliot Stern and Penny Youll. You are all very welcome. Maybe you can just say a little about what your own assessment is of the evaluation you have done, just to start, and then we have some questions on the areas we have a particular interest in.
  (Mr Stern) I would start by putting in context what we have been doing. We have been responsible for a number of the evaluations of the Welfare to Work programmes and we have experienced the New Deal for Young People and the New Deal for Disabled People, especially innovative schemes. John has been responsible for the ONE evaluation we are talking about this evening, Penny has also worked on the New Deal for Disabled People, and I did the design work on all of those evaluations. It is worth saying that, because we have cumulatively been learning a lot about the delivery processes and how delivery affects effectiveness. If we are going to talk about the specifics and details of our impressions of this evaluation, John should lead on that.
  (Mr Kelleher) The setting up and delivery of ONE was a very considerable accomplishment by the staff and by the management with limited resources and limited time. That is the first thing to say. Secondly, across ONE, there does seem to be considerable enthusiasm for the ONE concept, the ONE idea, from staff, from management, from the clients that we met and indeed from some of the researchers involved in it. There were considerable problems to be overcome in the ONE pilots, mainly to do with resources; to do with quality of information technology; to do with quality of training; to do with the stretch of management—the very limited number of managers in the teams; and to do with the autonomy of area managers to innovate and adjust matters on the ground. But we felt the major constraints on the pilots were not so much these practical issues, important though they were, but more issues about the overall communication of the purposes of ONE and the focus of what the personal advisers were doing when they met with the clients, and the priorities which were given to different parts of the client meeting to deal with benefits, to deal with work and how work was dealt with. It seems to us that many of the aspects of ONE have been demonstrated to be valuable in the pilots, but it is difficult to point to any single area or office where all the elements have been put together at the same time, and therefore it is not so surprising that we are not seeing great impact at this stage. What we have seen in this delivery evaluation is to some extent a proof of concept: what we have not actually seen is a fine-tuned machine working in practice.

  251. Penny, anything to add at this stage?
  (Ms Youll) The only thing I would add as far as the conduct of the evaluation was concerned is that we were very mindful, and kept mindful, of the fact that this was a pilot in nature, supposedly, and that did have a particular impact on the way managers in the pilot areas were able to conduct their particular work. We felt it put some constraints on them which we tried to recognise, identify and track. But as time went on, we had to adjust to our commentaries that clearly something else was happening and its pilot nature had been translated into something a little more like a pathfinder.
  (Mr Stern) I would underline that. I think there is a familiar tension in this evaluation, as in many other so-called pilots, between the kinds of methods and responses you get when you initiate something which is called a pilot and very soon it becomes clear that there is a likelihood of national roll-out on that basis, and people's orientation changes because they begin to see themselves as preparing for something rather than testing a hypothesis. This has come about in most major programmes we have looked at over the years because there is always that ambivalence in the UK about piloting, pathfinding and national roll-out.

  252. This substantial piece of work was described as interim, does that mean it is going on?
  (Mr Kelleher) Our research has been concluded and we are dotting the Is and crossing the Ts for our final report.

  253. Which will be available when?
  (Mr Kelleher) It will be in final form to go to the Department for publication within the next week or two. I understand it takes four to six weeks for something to be published.

  254. At least that. You make mention in your research of the industrial dispute that was going on. Was that a significant factor in a pilot which was probably struggling to get going in any case? Was that an additional factor and is it something significant? Do we need to bother about the impact?
  (Mr Kelleher) Our research was completed before the industrial dispute in Jobcentre Plus began. Secondly, on the particular issue about screened environments, we were asked on several occasions what we could say about this. In the famous Sherlock Holmes story, there is the case of the dog which did not bark in the night. Our experience of screened environments was that it simply did not seem to be an issue with the staff we interviewed; it simply did not come up as an issue.

  255. Are you professionally surprised there is not a proper evaluation before we go to Jobcentre Plus roll-out?
  (Ms Youll) Professionally surprised, no.

  Chairman: That is a good answer.

Mr Dismore

  256. I would like to pick up from where John Kelleher left off about the vision, the whole object of trying to change the culture from benefit dependency, to independence and to work. To what extent do you think the ONE project was able to achieve that, both in terms of claimants and organisational structures, that you could evaluate?
  (Mr Kelleher) I think a beginning was made. I think the vision is not yet clear. There was a tension within the vision between the notion of a ONE integrated service and a work-focussed service. There was often confusion in the mind of personal advisers as to whether the focus should be round the client or round getting a job. I think we began to identify how a greater concentration could be made on the moments within the process where personal advisers could really make an employment difference. There was a difficulty that there are many clients who are already job-ready, and that is not going to make a great deal of difference to them. There are also clients with fairly profound problems—motivation problems, intrinsic problems, disabilities, etc.—which ONE has to work on in quite a complex fashion and it is not something that a single meeting or a series of meetings with the personal adviser can deal with. However, there is an intermediate group of clients that have to confront certain practical difficulties in reaching employment and the process does need to be fine tuned to concentrate more on that client group, to better inform personal advisers and to better prepare personal advisers to effectively deal with that client group. We have learned a lot about that but there is much to be put into practice.
  (Mr Stern) Can I just add, this is a familiar problem because the tailoring and making specific and personalising of the service and being able to recognise what category or group you are dealing with has occurred in virtually all of the welfare-to-work programmes we looked at. You sometimes have a very finely honed machine which is finely honed to deal with what it is faced with at that moment. On other occasions, you may not recognise that you have people in front of you for whom you do have a responsibility and it takes a while to go through that diagnostic process.

  257. Are you recommending a triage system, does that lead to quick wins?
  (Mr Kelleher) I think we are recommending clarity of our priorities. There is a large group of clients that need to be swiftly dealt with and swiftly referred and there is a smaller group of clients that need intervention; that may well be a triage system. I think the point about a triage system is, it is normally based on a lot of medical expertise. If we use the same analogy, there actually needs to be the same expertise about what kind of client you have in front of you. We have seen people identified by PAs as effectively work shy, when to our observer they were clearly depressed, to give one example, and it is very important to be able to dig beneath that. It is usually only at the second PA meeting that this work begins to be done. The idea of case loading—where you really come to grips with the client, so you can help—if it is to be effective, PAs have to get these other extraneous clients off their workload so they can concentrate on the client where they can make a difference.

  258. Another of the visions, this is more on the organisation side, is the partnership concept of the different agencies working together, local authorities, and so on. To what extent do you think this concept has worked in practice? What are the lessons to be learned?
  (Ms Youll) My own metaphor for this was two motorways merging, two highly complex operations having to merge into a single point of delivery, and that is actually to miss the point, because the core partnership was intended to be a threesome. I think we became very clear that the local authority element was not fully and successfully integrated for a variety of reasons. As far as our observations of the actual operation of the Employment Service and the Benefits Agency is concerned, these worked really quite surprisingly well. As a layperson, in one sense, moving in and observing people at work, there seemed to be really quite a good degree, a high degree of cooperation and teamwork, almost irrespective of where people had been seconded from, they were learning from each other on the job. I think that was partly because they had an identity as ONE people, they had their own logos and desk styles and particular environments. I would like to add that it was also very clear to us that the site within which the ONE team was located also had a significant influence on the way the staff in that particular ONE site operated. ONE would be more suffused with Employment Service expectations and targets, because that is where they were located, and Benefits Agency staff would be rather surrounded by Benefits Agency objectives. I think, to this extent, it was my point about a pilot, that ONE never had really its own completely freestanding identity, it was essentially dependent on and connected with the core operations of both the Employment Agency and the Benefits Agency.

  259. Is the conclusion from that that, if you are operating from a local authority purpose, then that would have the same effect?
  (Ms Youll) Yes, but where they were located in the local authority office, the particular arrangement, the impact was not quite so great and the contribution of the local authorities was, to some extent, confined to the housing benefit issue, which was not the most central part of benefit claims that most of the clients were coming with, although it was absolutely essential.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 26 February 2002