Examination of Witnesses (Questions 250-259)
MR JOHN
KELLEHER, MR
ELLIOT STERN
AND MS
PENNY YOULL
WEDNESDAY 16 JANUARY 2002
Chairman
250. Welcome to our next set of witnesses, who
are from the Evaluation, Development and Review Unit of the Tavistock
Institute: John Kelleher, Elliot Stern and Penny Youll. You are
all very welcome. Maybe you can just say a little about what your
own assessment is of the evaluation you have done, just to start,
and then we have some questions on the areas we have a particular
interest in.
(Mr Stern) I would start by putting in context what
we have been doing. We have been responsible for a number of the
evaluations of the Welfare to Work programmes and we have experienced
the New Deal for Young People and the New Deal for Disabled People,
especially innovative schemes. John has been responsible for the
ONE evaluation we are talking about this evening, Penny has also
worked on the New Deal for Disabled People, and I did the design
work on all of those evaluations. It is worth saying that, because
we have cumulatively been learning a lot about the delivery processes
and how delivery affects effectiveness. If we are going to talk
about the specifics and details of our impressions of this evaluation,
John should lead on that.
(Mr Kelleher) The setting up and delivery of ONE was
a very considerable accomplishment by the staff and by the management
with limited resources and limited time. That is the first thing
to say. Secondly, across ONE, there does seem to be considerable
enthusiasm for the ONE concept, the ONE idea, from staff, from
management, from the clients that we met and indeed from some
of the researchers involved in it. There were considerable problems
to be overcome in the ONE pilots, mainly to do with resources;
to do with quality of information technology; to do with quality
of training; to do with the stretch of management—the very
limited number of managers in the teams; and to do with the autonomy
of area managers to innovate and adjust matters on the ground.
But we felt the major constraints on the pilots were not so much
these practical issues, important though they were, but more issues
about the overall communication of the purposes of ONE and the
focus of what the personal advisers were doing when they met with
the clients, and the priorities which were given to different
parts of the client meeting to deal with benefits, to deal with
work and how work was dealt with. It seems to us that many of
the aspects of ONE have been demonstrated to be valuable in the
pilots, but it is difficult to point to any single area or office
where all the elements have been put together at the same time,
and therefore it is not so surprising that we are not seeing great
impact at this stage. What we have seen in this delivery evaluation
is to some extent a proof of concept: what we have not actually
seen is a fine-tuned machine working in practice.
251. Penny, anything to add at this stage?
(Ms Youll) The only thing I would add as far as the
conduct of the evaluation was concerned is that we were very mindful,
and kept mindful, of the fact that this was a pilot in nature,
supposedly, and that did have a particular impact on the way managers
in the pilot areas were able to conduct their particular work.
We felt it put some constraints on them which we tried to recognise,
identify and track. But as time went on, we had to adjust to our
commentaries that clearly something else was happening and its
pilot nature had been translated into something a little more
like a pathfinder.
(Mr Stern) I would underline that. I think there is
a familiar tension in this evaluation, as in many other so-called
pilots, between the kinds of methods and responses you get when
you initiate something which is called a pilot and very soon it
becomes clear that there is a likelihood of national roll-out
on that basis, and people's orientation changes because they begin
to see themselves as preparing for something rather than testing
a hypothesis. This has come about in most major programmes we
have looked at over the years because there is always that ambivalence
in the UK about piloting, pathfinding and national roll-out.
252. This substantial piece of work was described
as interim, does that mean it is going on?
(Mr Kelleher) Our research has been concluded and
we are dotting the Is and crossing the Ts for our final report.
253. Which will be available when?
(Mr Kelleher) It will be in final form to go to the
Department for publication within the next week or two. I understand
it takes four to six weeks for something to be published.
254. At least that. You make mention in your
research of the industrial dispute that was going on. Was that
a significant factor in a pilot which was probably struggling
to get going in any case? Was that an additional factor and is
it something significant? Do we need to bother about the impact?
(Mr Kelleher) Our research was completed before the
industrial dispute in Jobcentre Plus began. Secondly, on the particular
issue about screened environments, we were asked on several occasions
what we could say about this. In the famous Sherlock Holmes story,
there is the case of the dog which did not bark in the night.
Our experience of screened environments was that it simply did
not seem to be an issue with the staff we interviewed; it simply
did not come up as an issue.
255. Are you professionally surprised there
is not a proper evaluation before we go to Jobcentre Plus roll-out?
(Ms Youll) Professionally surprised, no.
Chairman: That is a good answer.
Mr Dismore
256. I would like to pick up from where John
Kelleher left off about the vision, the whole object of trying
to change the culture from benefit dependency, to independence
and to work. To what extent do you think the ONE project was able
to achieve that, both in terms of claimants and organisational
structures, that you could evaluate?
(Mr Kelleher) I think a beginning was made. I think
the vision is not yet clear. There was a tension within the vision
between the notion of a ONE integrated service and a work-focussed
service. There was often confusion in the mind of personal advisers
as to whether the focus should be round the client or round getting
a job. I think we began to identify how a greater concentration
could be made on the moments within the process where personal
advisers could really make an employment difference. There was
a difficulty that there are many clients who are already job-ready,
and that is not going to make a great deal of difference to them.
There are also clients with fairly profound problems—motivation
problems, intrinsic problems, disabilities, etc.—which ONE
has to work on in quite a complex fashion and it is not something
that a single meeting or a series of meetings with the personal
adviser can deal with. However, there is an intermediate group
of clients that have to confront certain practical difficulties
in reaching employment and the process does need to be fine tuned
to concentrate more on that client group, to better inform personal
advisers and to better prepare personal advisers to effectively
deal with that client group. We have learned a lot about that
but there is much to be put into practice.
(Mr Stern) Can I just add, this is a familiar problem
because the tailoring and making specific and personalising of
the service and being able to recognise what category or group
you are dealing with has occurred in virtually all of the welfare-to-work
programmes we looked at. You sometimes have a very finely honed
machine which is finely honed to deal with what it is faced with
at that moment. On other occasions, you may not recognise that
you have people in front of you for whom you do have a responsibility
and it takes a while to go through that diagnostic process.
257. Are you recommending a triage system, does
that lead to quick wins?
(Mr Kelleher) I think we are recommending clarity
of our priorities. There is a large group of clients that need
to be swiftly dealt with and swiftly referred and there is a smaller
group of clients that need intervention; that may well be a triage
system. I think the point about a triage system is, it is normally
based on a lot of medical expertise. If we use the same analogy,
there actually needs to be the same expertise about what kind
of client you have in front of you. We have seen people identified
by PAs as effectively work shy, when to our observer they were
clearly depressed, to give one example, and it is very important
to be able to dig beneath that. It is usually only at the second
PA meeting that this work begins to be done. The idea of case
loading—where you really come to grips with the client,
so you can help—if it is to be effective, PAs have to get
these other extraneous clients off their workload so they can
concentrate on the client where they can make a difference.
258. Another of the visions, this is more on
the organisation side, is the partnership concept of the different
agencies working together, local authorities, and so on. To what
extent do you think this concept has worked in practice? What
are the lessons to be learned?
(Ms Youll) My own metaphor for this was two motorways
merging, two highly complex operations having to merge into a
single point of delivery, and that is actually to miss the point,
because the core partnership was intended to be a threesome. I
think we became very clear that the local authority element was
not fully and successfully integrated for a variety of reasons.
As far as our observations of the actual operation of the Employment
Service and the Benefits Agency is concerned, these worked really
quite surprisingly well. As a layperson, in one sense, moving
in and observing people at work, there seemed to be really quite
a good degree, a high degree of cooperation and teamwork, almost
irrespective of where people had been seconded from, they were
learning from each other on the job. I think that was partly because
they had an identity as ONE people, they had their own logos and
desk styles and particular environments. I would like to add that
it was also very clear to us that the site within which the ONE
team was located also had a significant influence on the way the
staff in that particular ONE site operated. ONE would be more
suffused with Employment Service expectations and targets, because
that is where they were located, and Benefits Agency staff would
be rather surrounded by Benefits Agency objectives. I think, to
this extent, it was my point about a pilot, that ONE never had
really its own completely freestanding identity, it was essentially
dependent on and connected with the core operations of both the
Employment Agency and the Benefits Agency.
259. Is the conclusion from that that, if you
are operating from a local authority purpose, then that would
have the same effect?
(Ms Youll) Yes, but where they were located in the
local authority office, the particular arrangement, the impact
was not quite so great and the contribution of the local authorities
was, to some extent, confined to the housing benefit issue, which
was not the most central part of benefit claims that most of the
clients were coming with, although it was absolutely essential.
|
