Select Committee on Public Accounts Minutes of Evidence

Examination of Witnesses (Questions 140-159)



  140. I will give you another example. One of my colleagues was in one country when he said to an elder leader, "Don't you think this person should be defeated at the next election?" and they said, "No, no, no we cannot defeat them." He said, "Why not?" "Because he has now got to a position where he has stolen millions off us, he has got a lot of money in his overseas' bank account and now he has stopped stealing off us. If we elect a new one, the new one will start stealing off us to build up their overseas bank account so we must keep him there." It causes a little bit of despondency until you read the report and I think it is page 39 where they imported some instruments in for future leaders. How much of your programme is committed to influencing future leaders in getting a better level of governance and less corruption in these countries?
  (Mr Chakrabarti) I will ask Mr Lowcock to give some examples from his own experience in East Africa but from what I have seen on the ground quite a lot actually. Increasingly say in Kenya, I know that our people we work with on governance issues spend a lot of time with other members of civil society, not the Government necessarily, opposition groups, NGOs and so on, churches, discussing the way forward to get a more pluralistic system in Kenya. I think that has paid off. I have not been to Kenya for ten years. Clearly a lot of the policies of Government are not good enough but undoubtedly it is a freer society with a freer press than there was ten years ago. One does have some impact, it may take a long, long time when you do it collectively but I think you can attribute some of that success to the efforts that Mr Lowcock and others of his colleagues have had.
  (Mr Lowcock) Just on Kenya. We do spend time discussing these kinds of issues with the people we work with in the countries where we work. We try also through dialogue with Government to persuade them, for example, to put more money in to the primary health care budget or primary education budget and less into something else. This question of influence is one that we are very alive to.

  141. In the report, if I have read it correctly, it is not undertaken as a routine part of your country planning to ensure the level of governance has achieved a standard which you are comfortable with, is that right?
  (Mr Chakrabarti) No, we take governance very, very seriously indeed. We are always trying to improve it. I do not want to leave you with the impression that we would only operate when governance was at Westminster standards.

  142. No, no.
  (Mr Chakrabarti) Indeed not. The question is are we making progress in the right direction or are we travelling backwards? Is that Government committed to good standards, getting there, putting in anti-corruption bureaux, what have you, getting a more free press, allowing the opposition to speak freely. Where that is happening we are much more comfortable operating with that Government.

  143. You do an assessment of the quality of governance, albeit it might not be at the level you would like but it is moving forward, you have put this assessment of it in as a routine part of your country planning agreements?
  (Mr Chakrabarti) Yes. Each country strategy paper that we have that our teams draw up will address governance issues in the way you describe.

  144. Excellent. It says in the report it is not a routine part of it.
  (Mr Chakrabarti) We have just put some new guidance out and it is absolutely crystal clear.

  145. I accept this is a difficult situation because if you are going to put a resource in you have an expected outcome so you sit down and develop this plan and say "If we put this resource in, the outcome given in, for instance, poverty reduction will be X over one or two years." Is that part of your country planning?
  (Mr Chakrabarti) It has not consistently been set out in that way. I think the NAO team looked at a number of country strategy papers and saw some had them in and some did not. With the new guidance what we are doing is insisting the country teams specify exactly what impact they will have with their efforts.
  (Mr Lowcock) One point the report makes is that we have been much better at doing exactly what you say when we are looking at investment in particular activities. So in the example on page 37 it lists in Bangladesh all the things we thought we were going to get from this £55 million investment. The criticism in the report, which we accept, is that we have not been very good at aggregating, to look at the country as a whole, to forecast what from all our activities the country as a whole will be able to do. We have been better at doing it when we are looking at individual projects and, as I say, we accept that criticism.

  146. You have answered my next question because my next question was about the risk assessment. I know how difficult it can be but do you automatically do a risk assessment on a possible outcome and the reasons why you might not achieve either level A, level B or level C?
  (Mr Chakrabarti) I think you are absolutely right at project level we always have done. We have models which our project teams think through of where the risks are to not achieving their objective. In fact those models have been passed on to domestic policy makers in the aftermath of BSE and foot and mouth to try and think through risk assessment in the UK. Where we have not been so good is at country level where we have not thought through the shocks that could divert us from hitting our targets. Again the new guidance makes it very clear that country managers will have to do this. Also we have director's statements now where we are going to insist that directors responsible for a PSA objective will have to set out the risk of not achieving them.

  147. The last question. I realise how difficult it is to work with other countries which have different strategies. One example I was looking at, this country went in and built a lovely hospital and it had the national flag on top, not the flag of the country the hospital was in but the country which was providing the hospital, the only difficulty was there were no doctors, there was no doctors' training programme, there was no nurse training programme and this building was an empty shell. The equipment had been supplied by the host country. This was an example of a lot of money which could have been much, much better spent. How do we make progress and bring countries on board and in the European context is it the time to start maybe naming and shaming some of these countries with their policies which are in part being counter-productive?
  (Mr Chakrabarti) I think peer pressure is the key thing, whether it is naming or shaming but being much more open and explicit about the sort of projects you spoke of, the appalling waste of money which is clearly not helping the poor in any way. There is a process at international level which is run by the OECD which reviews each country's aid programmes to see whether they are any good or not. That process, that peer review process does bring to light some of these appalling failures. That requires them I think, parliamentarians in those countries and elsewhere but also in public to speak up and put pressure on the governments of those countries to do better, frankly.

Mr Bacon

  148. Mr Chakrabarti, I wonder if I could start by asking a very general question. As a student I shared a flat with a development economist and it was very obvious to me watching him for many years, from being an under-graduate going through to doing a post-doc, how his own attitude to development economics changed. You have at one extreme the Peter Bauer view of the world which says basically it is a waste of money, I am not advocating that at all. Could you characterise, briefly, how the debate among international development economists has changed over the last 20 years or so from how it was to where it is now and where you stand in that debate? Just two minutes.
  (Mr Chakrabarti) Okay. Twenty years ago I would have characterised the development economics profession as being on the cusp, I suppose, of change in terms of its thinking. At that point a lot of people still believed in protectionist economies, very strong state dominated approaches to development. Then there was a shift in the 1980s as we saw the debt crisis take hold, many of these countries unable to pay back their debt, realising that actually they had to move to perhaps more market orientated economies. That was very much a theme of the 1980s I think. I would say in the 1990s and since there has been a recognition that some of that was perhaps a little bit of an over-reaction. You do need a viable state involved to run effective services for the poor in particular so there has been a slight reaction against that. The key biggest change I have seen in the development profession in the last ten years, it has been an evolving thing, has been the recognition that it is really country's strategies that we must buy in to rather than donor strategies. I think for many of us, we were designing programmes 20 years ago, 15 years ago, which were very much focused on what we could deliver as opposed to what that country maybe required. We are moving much more to trying to support their policies, their strategies rather than ours.

  149. That is very helpful. It helps to lead into my next question which is really about the whole environment of measurement. You have these Public Service Agreements and International Development Targets, you have these Millennium Development Goals, this laudable list of things which are achievable or should be achievable. Mr Steinberg earlier mentioned the notion on Page 26 in Paragraph 2.13: "...with the majority of measures it is not possible to determine the extent to which any achievement is the result of DFID's efforts..." Later on in the Report at Paragraph 3.39 the NAO says that its analysis of country planning documents shows that there were no quantified links between development activities proposed, resources committed, and progress against PSAs ir IDTs. Then on Page 43, Paragraph 3.43 it says: "Country Strategy Papers explicitly identify planned development expenditure over the strategy period. But there is little discussion or analysis of the relation between resources committed and impacts or outcomes resulting." Page 32, to go back to the point you were making about the central importance that has been recognised in the last few years of the overall country plans, flags this up as an even more important factor. Paragraph 3.6: "None of the Country Strategy Papers chart the projected trend in poverty status indicators over the plan period." It does not sound to me like there is much of a culture of measurement or performance evaluation in your Department at all. I do not think anyone doubts that you are doing good work but it does not sound to me—this is all or nothing language—that any or many of your staff seem to take the business of performance evaluation or measurement at all seriously.
  (Mr Chakrabarti) That is a slightly unfair characterisation. They take it very seriously at project level and we are world renowned for it at project level. The problem has been to try and get an aggregation at country level which the NAO Report quite rightly picks up on.

  150. This is the point you made earlier?
  (Mr Chakrabarti) Absolutely. There is no doubt about that. Having read the papers I can see that. The new guidance which we have just put out tries to change all that. We need to work it through with country teams. It gets them to try and identify the outputs and the impacts from what we are doing.

  151. Why do you think they do not do it at the moment?
  (Mr Chakrabarti) Partly it is a culture change. It has taken some time to shift from the project level up to the country level.

  152. On Page 48, and this is referring to the country level, there is a quote: "... it's a very interesting debate which I think is going on internally is how we use our Evaluation Department, it's very unclear what its role is at the moment..." "... I've always been very sceptical about the worth of evaluation in DFID,"—a startling statement—"not because it's a bad thing to do, it's not but because as an organisation we rarely take much notice of it."
  (Mr Chakrabarti) I think that is absolutely accurate in terms of the recent output of our Evaluation Department, which had been much more focused, I am afraid, on lesson learning too late almost, so the programme managers who are quoted there are saying, "It is all very useful but I am not being taught lessons when I do my job."

  153. It says in Paragraph 4.12: "But there are indications that formal evaluation reports are not read by country teams. This raises the risk that evaluation findings are not receiving the attention they need to influence development practice ..." They are saying, are they, Page 53, that sometimes you do not get the information until it is of academic interest rather than when you are implementing policy? They are saying it is all very well, it is glossy, but it is a complete waste of time because it is too late?
  (Mr Chakrabarti) That is right. That has certainly been the case with much of it. There have been more generic lessons which we have learned and which have affected us in real time, lessons about aid effectiveness more broadly and how projects should be done over a longer timeframe and so on, but the lessons around some of the sector work are too late for them to be really useful on the day to a programme manager. What programme managers have done, quite rightly, is to use personal networks and retreats to learn lessons in real time.

  154. Are you saying the formal evaluation you undertake is of so little value that their attitude would be—and I do not want to unfairly characterise it but crudely they might be saying—"We know we are doing good work, why don't you just let us get on with it and stop bothering us with all this evaluation and measurement?"
  (Mr Chakrabarti) That is not the reaction on the ground. The reaction is it would be more helpful to have a proper evaluation done which really did impact in real time as opposed to two or three years late.

  155. How do you go about getting what you call real time evaluation?
  (Mr Chakrabarti) You have to have the evaluation teams working much more closely with the country teams than you have had in the past. We have had an unhappy separation of functions and I would like them to be much closer together to do more work in terms of monitoring the implementation of programmes than we have done in the past.

  156. How much money is spent on evaluation?
  (Mr Lowcock) The budget of the Evaluation Department—

  157. That was not my question. Maybe they are the same thing.
  (Mr Lowcock) They are not the same thing unfortunately. That budget is £2 million. There is quite a lot of evaluation work that is done by people who are running country programmes. For example, every year for all our major activities we carry out a review of whether they are achieving their objectives. We do 500 of those a year and they are done by the people running the projects. When all our major projects finish the people who have been responsible for them commission a project completion report. Those things contribute to the overall evaluation effort and, of course, that costs us money, having commissioned that activity. We do not have a number on that unfortunately because it is done by people who do lots of other things as well. The bit of the organisation which just does ex post impact assessment independently costs us £2 million per annum.

  158. You mention the case of Bangladesh on Page 37 and talk about improved life expectancy from 57 to 62 and a list of things for a specific amount of money. They all sound extremely measurable, including the number of children who get full-time primary education. It sounds like there could be a very high degree of specicivity. What is so difficult about taking that in these various pockets where you have got these specific projects and adding it all together?
  (Mr Chakrabarti) I hope very much that the next generation strategy papers will do that and that they will have targets of primary enrolment at the level of the country. In Bangladesh that should be possible. The Government of Bangladesh should be able to come up with those targets in its poverty paper and then we should just adopt these targets if we are happy with this new paper they are writing.

  159. On Page 29 is says that the EU receives 55 per cent of the money that goes from DFID to multi-lateral institutions, £709 million. I think you said earlier—and correct me if I am wrong—that the result of giving some money through the EU is that it makes it less easy to achieve your targets. Is that right?
  (Mr Chakrabarti) It is a less effective programme than many others.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 1 August 2002