Examination of Witnesses (Questions 20
- 39)
WEDNESDAY 16 NOVEMBER 2005
MR NICHOLAS
MACPHERSON, MR
JONATHAN STEPHENS,
MS MARY
KEEGAN, MS
SUE OWEN
AND MR
JOHN OUGHTON
Q20 Peter Viggers: The National Audit
Office reported in March 2005 that in 20% of the PSA data systems
departments are not collecting sufficient data for the targets
to be accurately assessed. What action did you take on that?
Mr Stephens: We worked with departments
to produce a new set of targets as part of the 2004 Spending Review.
We endeavoured to learn a number of the lessons that should have
been learnt from the earlier set of targets, and in particular
we ensured that early in the process, before the targets were
agreed or set, there was an early engagement on the technical
specification of the target. This enabled us, with the 2004 set
of PSAs, to publish within a couple of weeks of the Spending Review
pretty well a full set of technical notes, setting out the detailed
technical specification of precisely in technical terms what the
target meant, how it could be measured, the period it would be
measured over, the data sources and how they would be validated,
et cetera. I think the NAO recognised in its report that this
represented a significant step forward on the previous approach,
where a number of these technical issues had only been identified
well after the targets were set, and in some cases had proved
impossible to resolve. I think we have made significant progress
in specification and technical understanding.
Q21 Peter Viggers: In some areas
meeting of targets is a matter of ticking boxes and does not promote
efficiency at all—for example, in the National Health Service
where patients have been left in ambulances rather than being
taken out of the ambulances, because time starts running when
they come out of the ambulance. This is a matter of fact. How
do you monitor whether the targets are valuable in promoting efficiency?
Mr Stephens: I think there has
been an awful lot of valuable focus on the role that targets play
in promoting efficiency and effectiveness. There have been a large
number of studies. The Treasury contributed its own side of this
area in the Budget 2004 document on devolved decision-making.
Undoubtedly, there are important messages that we have learnt
since the introduction of PSAs in 1998. For example, most targets
are now outcome-focused. The input intermediate targets have been
abolished, and that has enabled us to reduce markedly the numbers
of targets, recognising that it is important to focus down the
attention and effort. We have also learnt important lessons about
the specification targets to avoid just this sort of perverse
impact that you might be referring to. For example, on NHS waits,
that has led us in the course of the Spending Review 2004 to develop
a target that embraced the whole period from a patient coming
in to the system to being treated and not splitting it up. We
have improved the involvement and consultation with frontline
deliverers. There have been significant lessons learned from our
own experience from the independent external reports that have
been done; and internationally, in terms of measurement of public
services, performance management of public services, the UK is
generally reckoned to be world-class in its system. That said,
there are inevitably some dilemmas remaining. Do you go, for example,
through a simple target that provides a high degree of focus but
risk some perverse effects; or do you go for a more complex target
that then becomes more difficult to monitor and follow through?
Q22 Peter Viggers: Does this focus
on targets and micro management sometimes miss the big picture?
How do you measure, for instance, the fact that pensions are in
crisis, or that means testing is now rather seriously eroding
people's trust in savings?
Mr Stephens: Targets are certainly
not the be-all and end-all, nor do I think that in PSAs they amount
to micro management. We have 110 PSAs across the whole of the
public services—£250 billion of money, and that works
out at about six or seven targets per department, which is not
an unreasonable setting of national priorities for that amount
of public money being spent. Indeed, the focus on target outcomes
and a devolved responsibility around resources to deliver those,
enable departments and public services to have a high degree of
freedom on how to deliver that. I recognise too that as the system
gets devolved out, you can have a multiplication of targets. That
is one of the things we addressed in devolving decision-making
in the Budget 04, and we took a number of steps to try and reduce
that and to retain the focus on a few key national priorities.
Q23 Peter Viggers: What are the sanctions
if departments do not meet their targets?
Mr Stephens: The key sanction
is that the departments are publicly accountable. They are committed
to this. Regular performance information is published, and as
a matter of fact it is a matter of public accountability, including
in this place for Ministers, as to whether or not they have met
those targets.
Mr Macpherson: I would also add
that permanent secretarial pay is determined, amongst other things,
by performance against targets. We have a strong incentive in
ensuring that our departments deliver.
Q24 Chairman: Has any permanent secretary's
pay actually been affected?
Mr Macpherson: Undoubtedly.
Q25 Chairman: By the failure to deliver?
Mr Macpherson: I am too new to
this post to have seen a complete pay round, but my understanding
is, yes.
Mr Stephens: Can I say that "off
course" does not automatically mean that a sanction is required;
the first question is: "That is interesting; what has happened;
what shall we do?" It may mean that some external event has
changed, which then causes delivery plans to be changed.
Q26 Chairman: And gets the Permanent
Secretary off the hook! Is that right?
Mr Stephens: Well . . .
Q27 Mr Love: This is a rich seam
but I think I will avoid it just at the moment! I want to carry
on where Peter Viggers left off in relation to PSA targets, and
the Treasury's own targets. Last February the then Chief Secretary
to the Treasury, who has now gone off to South Africa, said that
the Treasury was responsible for its own PSA targets. What arrangements
are in place for peer review of the Treasury's targets?
Mr Stephens: As I said, these
are publicly set targets. In Treasury's case a high amount of
information is published very readily—the main economic
statistics are published monthly or even weekly. I would guess
the performance of the economy is the single most scrutinised
area of government. In addition, the Treasury Board, with its
independent non-executive directors, regularly reviews performance
against the Treasury targets.
Q28 Mr Love: I am somewhat bemused
by that answer. The Treasury prides itself on its tough approach
to the targets it sets for other departments. Are we to assume
that you are objective enough to set the same tough standards
for yourself?
Mr Macpherson: I think you can
assume that. I should only add that this Committee should be holding
us to account, and indeed does.
Q29 Chairman: Does that mean you
will dock your own pay?
Mr Macpherson: I am accountable,
amongst other things, to the Cabinet Secretary and Head of the
Home Civil Service, Sir Gus O'Donnell, who will no doubt take
a very dim view if, following his very excellent performance,
Treasury performance against targets goes into freefall—but
it will not.
Mr Ruffley: Is that a guarantee?
Q30 Mr Love: You are therefore telling
us that the targets you set are sufficiently challenging. I will
assume that that is the case. Let me turn to some of the targets
we are talking about, specifically nos 5 and 6. I see words like
"will reduce the persistent gaps"; "will define
measures to improve performance"; "it will demonstrate
progress". It seems rather nebulous, if I may say so, and
I would ask you to comment on that; but going a little further,
in your report on the Spending Review 2002 you indicated that
both targets were "on course". Why wasn't a higher,
more challenging, specific level of attainment included in your
PSA targets for 2004?
Mr Macpherson: First, these targets
are I think very challenging. These are very fundamental issues
which the Treasury is seeking to address, like the underlying
rate of growth of the economy, productivity growth, child poverty,
employment and so on. These are massively challenging objectives.
In many ways the 2004 targets are moving the agenda on, so that
whereas in 2002 the objective was, say, to reduce child poverty
by a quarter by 2004-05, now we are seeking to halve it by the
end of the decade. In other areas too we have adapted the target
to reflect new circumstances. There are some areas like changing
the composition of regional growth, in relation to which we are
seeking to change trends that have gone on for 80 odd years. I
just do not accept the premise that these targets are not challenging.
Q31 Mr Love: But it is very difficult
for an external judgment to be made. You are saying, taking your
example of regional growth rates, that the target is to reduce
the persisting gap. I fully accept the difficulty, and fully accept
that the long-term trend is a theme in this area. However, do
you not think that a more quantitative target would assist those
who are looking at the performance of the Treasury to make a real
judgment on whether the improvements are being made?
Mr Macpherson: There are two issues
here. One is whether the target is quantitative; to which the
answer is that it very much is quantitative. If you look at the
technical note that sets out the data around the PSA, it is very
explicit and clear, and very demanding. There is that issue, but
then there is the issue about how you monitor performance. You
make a good point here: you get data on regional output once a
year—the last data we have is from December last year—so
what we need to do, and what we have been doing increasingly,
is to develop proxy indicators such as the unemployment rate in
the North-West, the performance at GCSE and so on, which would
give us some intermediate measures that will allow us to monitor
progress.
Q32 Mr Love: Coming back to the final
point, which I do not think you have addressed, you indicated
in 2002 that you were making progress, and yet you did not set
a more challenging target. If there is a quantitative arrangement,
why have you not set more challenging targets for yourself at
2004? Presumably, we are asking the question looking into the
future in the next Spending Review. Will you set yourself an even
more challenging target for that? After all, if you take the child
poverty issue that you started off with, plainly there are more
challenging targets as we go forward. Whether the Government will
deliver them is an issue that no doubt we will debate in the future.
Mr Macpherson: There is a trade-off
here, with all targets, between continuity and change. Having
a target and then keeping it in the same form for more than five
minutes helps you in managing the organisation because it is only
over a period of years that these targets really take root in
terms of how people approach the job. As you say, where you are
making demonstrable progress it is important to make the target
more ambitious. We do not want a situation where everybody achieves
all their targets; that would suggest that—
Q33 Mr Love: It would be good for
permanent secretaries!
Mr Macpherson: It would be, but
somebody has to stand up against that, and the Treasury does.
I think a system where everybody is achieving their targets probably
has something wrong with it.
Q34 Chairman: The Treasury target
just says you have to demonstrate progress, and is not really
a target.
Mr Macpherson: I am willing to
send you the relevant part of the technical note because there
is a difference. We have to keep these targets to two sentences
or so in terms of the high-level presentation; but underpinning
that is some really hard data explaining what the process means.
If you have time, Jonathan will read it out, but I am happy to
send it to you.[2]
Q35 Mr Love: The only thing I would worry
about is that you mentioned earlier—and I have taken the
point—that these targets are in areas that the media and
other bodies are interested in as far as Government targets are
concerned; yet many in the media would not look to the detail
of the technical notes. Is it important that we make full publicity
use of the value of the overall PSA targets to make sure we are
getting our message across?
Mr Macpherson: I am quite sure
that everybody can work harder on the presentation. One of the
issues about PSAs, which we were very keen to ensure when they
were created, is that they should take root and become the currency
of debate, particularly political debate but debate more generally.
In a sense they are trying to encapsulate the issues that the
citizens and taxpayers of this country care about. Anything we
can do to get the presentation better to encourage that debate
can only be a good thing.
Q36 Mr Todd: While one can appreciate
that the setting of targets is a political process, and that the
focus on particular areas is one that politicians should make,
the technical process of deciding whether that target produces
meaningful information ought to be one that can be assessed objectively
and, I would have to say, externally. What attempts have been
made to ensure that there is an audit of the technical side of
whether the targets that have been chosen are appropriate and
meaningful? Proposals have been made by committees in the past
that there should be such a step, either by the NAO or some other
body. What approach have you taken on that?
Mr Macpherson: It is fair to say
that we have been quite receptive to people who have commented
on target-setting. I remember the Public Affairs Select Committee
produced a very thoughtful report on PSAs two or three years ago.
There were two responses. One was to enter into a greater dialogue
with so-called stakeholders about what you are trying to achieve,
so that there is buy-in and understanding say from the police
or from doctors; but the other thing at a more technical level
is ensuring that through the NAO's audit of data systems we have
credible, sensible data systems underpinning the targets. That
was raised earlier. Clearly, the NAO has suggested that in certain
areas the targets are falling short. One of the reasons why we
wanted the NAO to make those kinds of suggestions was to seek
to improve them.
Mr Stephens: There is a process
of technical scrutiny of the technical issues as targets are developed,
before they are finally agreed. A number of outside experts are
involved in that, including the NAO. It is not a formal audit
at that stage, but their advice is available and effective.
Q37 Mr Todd: For this to work, you
have to make sure that firstly the data systems are in place that
allow you to measure what you are doing. The NAO has criticised
the absence of data systems in some areas, to measure what you
are attempting to achieve. The second thing is to ask if that
is the right thing to measure if the outcome that you are seeking
is this one. Peter has made a reasonable point about precisely
where you measure delivery of somebody into hospital or something
of that kind. That is perfectly reasonable. There is a technical
issue of what you should be measuring in this process. Should
there not be a thorough external audit of those?
Mr Stephens: I agree. I think
those are very valid issues to look at. They are important lessons
to have learned from the technical specification targets and they
are formally addressed in validation of systems carried out by
the NAO. It has been recognised by the NAO that this can point
to a significant improvement in that area of technical specification,
understanding the data systems underpinning targets. I would also
say that sometimes you come across a dilemma in this area. You
can sometimes have a very reliable source of information, which
meets the criteria of being reliable and easy to measure, but
may not be measuring quite the things you are wanting to impact.
Then you face a bit of a choice—do we go with that, with
an understood, well-validated measure; or do we try to create
a new measure that more accurately reflects exactly what we are
trying to reflect—but of course that new development can
potentially carry new burdens with it? That is a genuine dilemma
in target-setting.
Q38 Mr Todd: The criticism I make
is that that choice is internalised; you are making it, and although
you suggested that there is an iterative process with a number
of outside bodies making suggestions, it is undoubtedly the case
that there is not a full external process of asking, "have
they got this right; does it make sense to do it this way; have
they got the data in place to deliver the information required?"
Can I turn to the question about productivity, which is one of
your targets—"productivity gap narrowing". Your
chart, particularly figure 3.2 on page 23, shows the narrowing
of the gap. To some extent I suppose it depends what time lines
you choose to use on your graph as to how meaningful that is.
One would suspect that if you extended that graph backwards by
20 years, say, you would see perhaps more meaningful information
of change. How did you select a particular point to present that
by?
Mr Macpherson: We selected 1995
as a starting point because that was a year where, on the basis
of macroeconomic analysis, the key countries seemed to be at broadly
the same point in the economic cycle. Part of the problem with
measuring productivity is that it is influenced by—
Q39 Mr Todd: As we know, cycles are
subject to some review as to quite where we sit in them, are they
not?
Mr Macpherson: It is always important
to know where you are in the cycle, but it is fair to say that
in terms of the UK, USA, France and Germany, 1995 is a sensible
starting point.
2 The technical notes are available on HM Treasury's
website, www.hm-treasury.gov.uk Back
| 
