Examination of Witnesses (Questions 120-139)
DEPARTMENT FOR
INTERNATIONAL DEVELOPMENT
3 MARCH 2008
Q120 Keith Hill: It seems to me that
DFID and the budget support approach come out of the Report pretty
well but being able to measure the results is obviously key to
the judgments, so I will concentrate in my few questions on part
four which deals with monitoring progress. However, let me just
draw your attention to part three, paragraph 12, which says, "To
date, DFID and others have not collected sufficient evidence to
show that transaction costs are reducing". Given the fact
that the reduction in transaction costs was absolutely essential
to the move to the budget support approach, why have you not assessed
why budget support has not reduced transaction costs both for
donors and for recipients?
Dr Shafik: We do have evidence
that transaction costs have come down for us. In fact table 15
in the Report shows that aid relative to our own administration
costs has come down over time as a result of budget support, so
we can show that for us. It has been harder for us to show it
for the recipients and there are a couple of reasons. A bit of
it is proving a negative and I will just give you a couple of
examples. Vietnam is a good case in point; Vietnam received 791
visits from donors in one year. The average number of donors a
government has to deal with in a poor country has gone up from
about 12 in the 1960s to over 30 now on average. It has been very
hard against those trends of growing numbers of donors for them
to deal with and it is hard to be able to show declining transaction
costs. The other problem is that the up-front costs of budget
support are high. To get a dozen donors round the table to all
agree what the priorities are, to all agree on the indicators
we are going to use to measure performance against, to all agree
on the audit and accountability measures, so we have seen higher
transaction costs in the beginning than we expected.
Q121 Keith Hill: Is this an area
you intend to continue to monitor?
Dr Shafik: Yes.
Q122 Keith Hill: Both the donors
and the recipients?
Dr Shafik: Yes.
Q123 Keith Hill: I think that is
quite important because otherwise what is the justification for
that approach. Paragraph 4.17, why did four of the six budget
support programme reviews looked at by the NAO consider progress
against only half or fewer of the programme objectives?
Dr Shafik: This is an area where
a lot of the key outcomes that were measured were being monitored
at a different level, not at the level of general budget support
or sector budget support. The equivalent I can think of is the
way we monitor public service agreements in this country where
you have 31 cross cutting governmental PSAs but a lot of the key
things we care about are monitored at the departmental level or
at the sectoral level. Several of the cases here where the indicators
were not monitored in the programme of formal documents were actually
monitored somewhere else in the system.
Q124 Keith Hill: Why did you not
tell that to the NAO?
Dr Shafik: We did, but they have
a point, to be honest. If we agree on a set of indicators up front
we need to be able to say that we are going to monitor them through
some other mechanism. That is not really good enough and that
is an example where we just need to clean up our systems and make
sure we are doing it properly. It is not an issue of substance
but of format.
Q125 Keith Hill: Maybe the next time
round—because I am sure they will be looking at this again—that
is something you could inject into the Report. Moving onto this
issue of measuring results, let me draw your attention to paragraphs
4.9 to 4.16 of the Report which show that gaps in country data
on the services delivered and outcomes obviously limit your monitoring
and also your management of the partner governments. Should you
not be doing something more to remedy these weaknesses?
Dr Shafik: Yes, and as I said
we probably do more than any other donor in terms of trying to
build statistical capacity and better data. We now have work with
the World Bank where we have 20 countries which now have clear
national strategies to improve their statistical systems and ten
more countries developing them. We are working in ten of the 15
budget support countries to improve the data and monitoring that
they are able to deliver.
Q126 Keith Hill: We all agree that
is terribly important. Paragraph 4.6 reveals that again of the
programmes looked at by the NAO 15% of the indicators have no
specific time-bound targets and baselines were missing in 22%
of relevant cases. How on earth can you monitor progress effectively
when such key indicators simply are not available?
Dr Shafik: In many countries we
are again working from a very low base. Rwanda is a very good
example; after the genocide we had no baselines on anything to
work with. In many cases what we do is that we might approve a
project but the first thing they have to do in terms of the budget
support programme is establish a baseline. Sometimes it comes
later if we do not have it on day one. Given the circumstances
we think it is still worth starting the process rather than waiting
for the baseline, but that is an example of where again we need
to clean up our systems, make sure that we have much more quantitative
rigour both at the beginning and throughout the process.
Q127 Keith Hill: Paragraph 4.14 on
the issue of monitoring progress, without monitoring unit costs
and other measures of efficiency how to you assess whether you
are getting value for money?
Dr Shafik: We do monitor unit
costs but we do not have clear benchmarks that we use globally
in DFID and that is something I would like to develop in this
new Investment Committee we established last week. We will actually
be doing precisely that, introducing much more data on unit costs,
average costs, typical costs, rates of return to different types
of development investments and forcing our teams to measure their
interventions against those sorts of benchmarks.
Q128 Keith Hill: Finally, paragraph
4.16 draws attention to weaknesses identified by the NAO in the
design of monitoring frameworks and poor matching of objectives
to indicators. How quickly do you think you are going to be in
a position to remedy those weaknesses?
Dr Shafik: I think we can move
very quickly on new projects. I think with some of the backlog
it will take a bit more time, partly because those documents have
to be negotiated with other donors and with governments, so a
kind of retro-fitting of some our performance frameworks will
take some time. Certainly I think on everything going forward
from now on we can expect much more rigour around quantification
and monitoring frameworks.
Chairman: There are one or two supplementary
questions before we finish, one from Mr Mitchell and one from
Mr Bacon.
Q129 Mr Mitchell: Can you put in
perspective for us, we are devoting an increasing proportion of
our aid budget to budget support, what is the proportion of the
aid budget of other countries that is devoted to budget support
and what is the proportion of EU aid that is devoted to budget
support?
Dr Shafik: The largest budget
support donor in the world is the World Bank followed by the EU.
It is 10% of DFID total spend and 18% of our bilateral programme
in DFID. I would prefer to give you a note on that to give you
the precise figures.
Q130 Chairman: If you do not know
the answer, give us a note, it does not matter.
Dr Shafik: In terms of ranking
we come after the World Bank and the EC. We are probably the largest
bilateral donor that uses budget support.
Q131 Mr Bacon: Dr Shafik, there is
enough in this Report to find good news if you want it and to
find plenty of evidence that things are looking pretty shaky in
terms of measurement, assessment and analysis if that is what
you want to find. The criteria that you employ for the 15 countries
seem pretty broad; it is a wide range of different countries within
those 15. However, it is only 15 out of 120, in other words nearly
90% of the countries to whom you give aid you do not give budget
support is appropriate for presumably because the potential benefits
are not worth the risks. For the vast majority of the countries
to whom you give this type of support the benefits are not worth
the risks, why not?
Dr Shafik: There are very different
reasons depending on the country. For example, for middle income
countries the amount of money we could give in budget support
would be so small relative to the government's own resources that
our judgment is that we will not have systemic impact. To be frank,
a lot of middle income countries will not listen to us and on
their entire budget we are giving them the equivalent of 0.05%
of their budget. So that takes out all middle income countries.
There is another whole group of countries where we are primarily
providing humanitarian assistance and where it is not a situation
in which supporting a government budget would be appropriate so
that also knocks out a whole list of countries where we do not
think it is the right instrument. Then there is a whole group
of countries where we judge the fiduciary risk as just too high
so countries like DRC or Kenya or Nigeria are ones where we do
not feel comfortable taking the risk.
Q132 Mr Bacon: I thought you did
support the Kenyan Ministry of Education. When I was in Kenya
I was told that the Ministry of Education was supported by the
DFID guys on the ground.
Dr Shafik: That is correct. It
is technically not budget support though because the money is
earmarked to specific expenditures in the education sector and
we get independent audits to check that that money is being used
for the correct purpose.
Q133 Mr Bacon: I have been sitting
listening to DFID coming to this Committee for six or seven years;
it is all good stuff and there is never quite as much measurement
as you want, but you are asking us to believe something that is
deeply counter-intuitive because we see things from all the domestic
policy departments where the government struggles to get the kind
of delivery and extract the value out of the system that it wants
when it is here in this country and you would have thought it
would have much more direct control over it. You are asking us
to believe that this process is as likely or more likely to work
when it is tens of thousands of miles away where you have very
few people on the ground.
Dr Shafik: I think we are able
to show delivery. The numbers I cited in terms of school enrolment,
immunisation, roads built and that kind of thing is convincing
on delivery. I think we have quite a good story to tell.
Q134 Mr Bacon: Why is that attributable
to budget support? If you had just gone in and said, "We
will build this road" or "We will enrol this number
of pupils" you could have achieved the same effect directly
through a project.
Dr Shafik: We could but it would
have been a much narrower effect.
Q135 Mr Bacon: An organisation such
as the Gates Foundation, which is a big donor, do they give budget
support?
Dr Shafik: No.
Q136 Mr Bacon: Not at all.
Dr Shafik: No, they do not.
Q137 Mr Bacon: Why not, do you know?
Dr Shafik: Yes, because Gates
have defined their strategy quite differently. Gates see themselves
as the venture capitalists of the development business. They are
not funding core service delivery; that is not what they are about.
They realise that even though they are big they are actually quite
small compared to us and they cannot afford to fund service delivery.
What they fund are the really innovative upstream inventions like
new technologies for treating malaria or new technologies for
new vaccines for HIV. They will invest in the R&D and the
up-front invention and then people like DFID will then see the
new malaria vaccine or new HIV vaccine and they will roll that
out and deliver it in the countries and finance the massive service
delivery. Their strategy is quite complementary to ours.
Chairman: Dr Shafik, thank you very much.
That concludes our hearing. It has been a most interesting hearing
on a very important subject on which we will obviously need to
reflect very carefully on everything you have told us. Clearly
there are advantages in giving money to governments to improve
their civic governance and I have always been an enthusiast for
this, but we are the Public Accounts Committee and we have to
be clear that this money has been spent in accordance with what
Parliament wants and it has achieved what it set out to do. We
will reflect on what you have had to tell us this afternoon. Thank
you very much.
| 
