Examination of Witnesses (Questions 20
- 39)
WEDNESDAY 13 MARCH 2002
MS RACHEL
LOMAX, MR
LEIGH LEWIS,
MR MATTHEW
NICHOLAS AND
MR BILL
WELLS
20. I am glad to hear you say that, but your
main justification seemed to be in answer to the Chairman not
that there was a straightforward payback for the economy, which
interpretation of these figures in terms of how worthwhile that
has been to the economy might be slightly confusing, but let us
leave that on one side for the moment. Your answer to the Chairman
a few moments ago appeared to be that it was not just a question
of how many jobs you created, it was also a question of the other
objectives you were looking at, including long-term employability.
Is that correct?
(Ms Lomax) Let us take this head-on. All employment
programmes are open to the criticism you and the Chairman have
just mentioned; all of them. It is very difficult to say that
every extra person who goes into a job would not have gone into
a job otherwise. What the New Deal for Young People has done is
got people into work, even on the evaluation evidence we have
here, more successfully than most previous programmes, or most
programmes in other countries, even on that narrow definition
of success.
21. If I may say so, you are moving a bit away
from the point I was trying to get to, which was this question
of improving employability. As I understand it, that was what
you were saying to the Chairman just now was one of the major
reasons why you felt, whether it is a waste of money or not, the
issue was not purely about how many jobs you had created or how
many people you had removed from unemployment, the issue was also
about improving long-term employability.
(Ms Lomax) Yes; it is indeed. I wanted to challenge
your summary of my reply to the Chairman which implied that I
was accepting that the scheme was a waste of money and I most
definitely do not.
22. You have challenged it very well but I do
not want to stick on that. I want to get onto employability.
(Ms Lomax) Let there be no misunderstanding.
23. The point here is that if it is true that
the question of improving employability is an important aspect
of the whole scheme, then I want to ask you why in that case you
have apparently no measure for measuring whether that has worked
or not.
(Ms Lomax) It is quite difficult just to capture employability
in a target.
24. Is it a sensible way then of justifying
the scheme if you cannot know really whether it has worked or
not?
(Ms Lomax) You can capture the impact of a scheme
by evaluating, as we have done, with specific evaluation studies.
For example, I think it was the Policy Studies Institute (PSI)
which did an evaluation looking precisely at the employability
issue and concluded that on a number of different dimensions you
could safely conclude that the New Deal for Young People was improving
employability in terms of whether people were progressing in jobs,
getting more pay, getting more responsibility, retaining jobs
longer—
25. Were they able to quantify that?
(Ms Lomax)—and they quantified that.
26. So why was no target set? If you can quantify
it, you might as well set a target and see whether you have achieved
the target.
(Ms Lomax) Because we cannot trace everybody in their
post-New Deal work record. What we have tried to capture is retention
in terms of the people who come back onto JSA and that has been
reflected in Employment Service targets and Ministers are going
to be announcing the Jobcentre Plus targets next week and we shall
be trying to capture a bit more of some of these ideas in the
targets we set for Jobcentre Plus. It is a moving target. We are
trying to develop our ability to capture it. We are learning how
to do this as we go along and we also need to improve our ability
to track people once they move into work. That is why in the Employment
Bill at the moment, which is before the House of Lords, we are
taking powers to enable us to join up the Employment Service records
with Inland Revenue records so that we can track what happens
to people once they enter work. That is exactly so we can begin
to get some meaningful measures of employability.
27. You seemed to say just now that one of the
things people have been looking at is the question of whether,
once they go into jobs, people's salaries then increase and this
is a measure of employability. People have been measuring that
and you appear to think that some of the work you are doing therefore
has been worthwhile. I am not trying to deny that for a moment.
What I am suggesting is that if this was always possible and people
have been doing it, why, when you set up the scheme, did you not
set a target that was measurable—okay you cannot track them
all—in terms of a percentage of those you can track whose
salaries increased over the year when they went into a job, or
whatever?
(Mr Lewis) May I say something pretty fundamental
on this about what target setting is about within any major organisation
or business? One sets targets for things which the organisation
itself can directly and clearly change and make happen, because
that is the motivation, that you want to ensure things happen
within the organisation. There are many things you measure and
you want to happen, but which you do not believe you can directly
impact day by day. To give an example of that, we know from very
extensive evaluation, just to put a few numbers on what Ms Lomax
was saying, that 75 per cent of those leaving the New Deal for
jobs have received a pay rise within six months, we know that
40 per cent have gained more responsibility within their jobs
within six months. We know, and it is the acid test for me, that
90 per cent of option participants themselves identified employability
benefits. It is hard to say to an individual member of staff in
the Employment Service that their target is to ensure that somebody
receives a pay rise within X months, because they cannot directly
impact on that. That is the difference between on the one hand
setting targets where you can have a direct and clear impact,
helping to place someone into a job, and measuring outcomes which
also matter to you.
(Ms Lomax) What we can do is incentivise the agency
to put people into sustained jobs, into quality jobs and that
is what the retention targets were about.
28. I find this one a bit hard to take. It does
seem to me that if you are going to set an objective for your
organisation with no measurable target at all and then say you
can measure all sorts of things but you have not actually said
how much you want these things to increase in order to improve,
you have met that objective, I would have thought that was a fairly
feeble way of monitoring what you are doing.
(Ms Lomax) No. May I have another go at this because
it is quite a fundamental point.
29. It is a very fundamental point and I am
rather worried about the reply.
(Ms Lomax) Let us go back. You can set outcomes for
objectives and outcomes for programmes which are ambitious and
long-term and go beyond those things you can immediately control.
Employability is a very long-term concept. It is relevant, it
is very important for employment programmes but it would be quite
wrong to pretend that it was easily measured or was something
under our immediate control. As Mr Lewis said, what you do by
way of target setting for an operational business is try to focus
their attention on things they can impact. It is supposed to motivate
them and focus their efforts. I really think that there is a distinction
between the outcomes you are seeking to influence and operational
targets which you are setting to try to focus business efforts.
The two are not the same.
30. I have to say that setting objectives and
having no way of measuring whether you are meeting them or not
is an unfortunate way of managing a business.
(Ms Lomax) I would contest that we have no means of
measuring whether we are having any impact on employability.
31. But if you do have a method of measuring
it, why do you not set targets to measure against?
(Ms Lomax) Because setting targets and evaluating
the impact of the programme are two separate things. That is the
argument I am trying to make at the moment. We have an evaluation
strategy which tends to look at quantitative evidence, long-term
evidence, things which are difficult to quantify and which might
be out there in the world. That is why we have a complex evaluation
strategy. It is to get at the many dimensions of the things we
are attempting to influence over a period of time. Targets are
about trying to focus the efforts of the operational business
in a way which will be supportive of getting performance, which
will help contribute towards this. Any operational business which
has woolly targets, complex targets, excessively bureaucratic
targets, is not going to perform well.
32. You do not have any targets; that is the
point.
(Ms Lomax) That is not true.
33. It is not a woolly target, you do not have
any targets.
(Ms Lomax) The Employment Service has a large number
of targets—
34. But not for this programme.
(Ms Lomax)—and some of them are related to things
which we think they can control which are relevant to employability.
That is why I mentioned retention. It is an important point.
(Mr Lewis) If you were, as the NAO did, to go out
and speak to Employment Service staff, personal advisers and other
staff who worked on this programme and have been proud to do so
for the last five years, they would certainly say that they have
known and been very clear from the very first day that they had
very clear targets to achieve. Those targets, in terms of employability,
were fundamentally about helping a very significant number of
young people coming through that programme into jobs and that
is the fundamental starting point. You cannot develop your employability,
if you do not first go into a job. We have measured many, many
other things and made very clear that we attach great importance
to them.
35. Actually the measure should not perhaps
have been just how many people went into jobs, but how many people
went into jobs who would not otherwise have and Ms Lomax said
that was very, very difficult to measure. So the one measurable
target you set yourselves everyone knows you cannot really use
terribly well to monitor the effectiveness of the programme and
the others do not have targets at all. That is what worries me
about the whole thing.
(Ms Lomax) No, that is not true; that is just not
true.
36. Could I turn you to page 9 and Figure 4
where it looks as though the overall reduction in number of young
claimants is very well worthwhile and I do not deny that for a
moment, but it looks rather as though the reduction is almost
entirely in terms of the over-six-months unemployed. I can understand
very good reasons for wanting the long-term unemployed number
to come down as quickly as possible. I just wonder whether you
could comment at all on the fact that since the reduction is almost
entirely in terms of the long-term unemployed, whether that is
to some extent at least at the expense of the short-term unemployed,
those who are more likely to get jobs anyway.
(Mr Wells) It is true that some people, when they
leave the Options, or leave a job, will come back into short-term
unemployment. However, the rate at which they are leaving short-term
unemployment has also improved over this period.
37. But the number of short-term unemployed
has not gone down. The dark blue lines on the graph are almost
exactly the same.
(Mr Wells) It is partly an optical illusion; they
have actually gone down. The rate at which people are coming back
on who are leaving the count has improved for the short-term unemployed
as well as the long-term unemployed. There was an increase of
short-term unemployed as people came round again, but it was nowhere
near as big as the reduction in long-term unemployment and the
net effect of both of those figures was a substantial reduction
in overall youth unemployment and the evaluation evidence suggested
that a substantial part of that was due to the New Deal.
Mr Gibb
38. Chairman, I should like to beg your patience
for 30 seconds while I say thank you to Ms Lomax for the figures
I received this morning on income support and Jobseeker's Allowance
fraud. You very helpfully split the figures between fraud and
customer error: 290,000 cases of fraud and 316,000 cases of customer
error in those two benefits. I was baffled by the answer I received
from the Minister yesterday morning when he said that these figures
could not be split because to do so would involve excessive cost
because he would have to go through every case individually to
see whether it was fraud or customer error; he said the split
could not be made. Is the Minister's answer correct? In which
case are the figures you have given me correct?
(Ms Lomax) Yes, they are. Would you like me to write
about it or does the Committee want to hear this answer.
39. Yes, write, as it is not relevant to this
hearing.
(Ms Lomax) I have looked at this. I will write to
you to explain how they marry up.
| 
