Select Committee on International Development Minutes of Evidence


Examination of Witnesses (Questions 27-39)

MR ANTONIO TUJAN JR AND MR HOWARD WHITE

29 APRIL 2008

  Q27 Chairman: Thank you very much for coming in. You have both been listening to the previous evidence session so you will understand the context. It is very good of you to come along. Perhaps you could briefly introduce yourselves for the record.

  Mr Tujan: I am Antonio Tujan, the international director of IBON. It is a research and education NGO based in Manila in the Philippines. I am also the current chairperson of the Reality of Aid Network, the global, north-south network of NGO platforms which monitor aid.

  Mr White: I am Howard White. I have spent most of my career as an academic working, amongst other things, on issues that affect aid management. For the last five years I have been leading the Impact Evaluation Programme of the Evaluation Department of the World Bank and I am now the head of the new International Initiative for Impact Evaluation.

  Q28  Chairman: Which likes to call itself 3ie. We have obviously been looking at the Paris Declaration as part of the framework for coordination. When we were in Ghana, we got some slightly negative feedback from the Ghanaians about the commitment of members of the Paris club, if you like, and the expectations for the forum that is going to take place in September. From your point of view—you can be as frank as you wish—how well do you think both the donors and of course the recipients, because they are part of the process, have lived up to the commitments that they have made under the Paris Declaration? If you are able to quantify it or be specific, that would be helpful.

  Mr Tujan: I just arrived from Ghana this morning where we had the results of prior research that began in preparation for Accra. In this process they raised a number of key points that civil society in Ghana is particularly interested in on the questions of gender equity, environment and production. They felt that there is a large gap between what they thought was necessary and could bring Ghana along the path forward and what has been delivered. The monitoring survey that was conducted by the OECD in 2006 has shown that there has been a very low level of performance to the Paris Declaration commitments. To a certain degree this is understandable because, on the one hand, it was a new survey and our indicators were unclear and so on. The new survey is presently being processed. There are initial results and the peek into that that was announced by the EDCDE in Bangkok last week was that there is a slight improvement. In short, it is still slight.

  Mr White: Forgive me if I take a slightly longer-term view, but I think it is very useful to do that. When the Paris Declaration was first signed, there were no particular grounds for being optimistic. This is not the first declaration the international community has made and it is certainly not the first that will not be met. It is not the case that aid coordination is a new idea. The DAC[12] has existed for well over 40 years, going on for 50 years. The DAC was founded as an aid coordination body, so what has it been doing for the last 50 years? There was mention in the last session that the multilateral agencies are there to play a coordination role. This is patently not the case. No one thinks the EU is coordinating the aid of the various EU donors on the ground in the recipient countries. The multilateral agencies, by and large, are not playing that role either. We come into a context where general declarations and the history of aid coordination have not done terribly well. We have to ask ourselves what is different now. What has changed that will mean this might actually work? There are two sides you have to look at because the Paris Declaration talks about aid coordination and about recipient control. I would rather say "recipient control" than "ownership" because without control there is no ownership. I think the modest progress that has been made is on the aid coordination side amongst the donors themselves. The progress towards recipient control has been very slight, if there at all. Without that, there is very little incentive for the recipients to get seriously engaged with the Paris agenda and very little likelihood that the Paris agenda is realised. Let us look even on the aid coordination side at whether there is some modest movement. I think we have to recognise that change there is necessarily slow. Institutional change is slow, unless you are talking about stroke of a pen reforms which you can do just by signing a document. Untying aid is one example where the UK indeed did that. Untying aid over the last decade is one success story in aid coordination. You ask what the DAC has been doing for 50 years: it was trying for a long time to get progress made in untying, as has now happened. On day-to-day management of aid, it is what is happening at the ground level in country programmes that matters and there things take a very long time to happen. You have the aid managers signing the Paris Declaration, but they are not the people who are down there in the country programme actually doing the work on a day-to-day basis. These are people still in DFID for example who come from a background of being project staff, project managers and are now having to change their roles to be sector specialists and change the sorts of things they look at. It is a very slow process. One thing I worked on some years ago now was the predecessor of budget support, which was import support, programme aid and various forms of macro support like that. The Strategic Programme for Africa, the SPA, was trying to coordinate the aid management of adjustment lending to Africa. It took pretty much 10 years to get some alignment on import support and disbursement of procurement systems, even though all the major donors agreed on it. In that case a particular sticking point was the World Bank. It was not until the World Bank introduced its simplified disbursement procedures that it was possible to coordinate just one particular aid programme to one particular group of countries. When I think of a recent experience I had in Bangladesh, you can arguably say you have had a health sector programme in Bangladesh since independence. They have always had very coordinated aid. When they started the health sector programme proper in the 1990s, only a small amount of money was going into the pooled finance. The rest was still projectised and even the amount going to the pooled finance was not in the first project part of the government procurement system. It was done as a separate pool, managed by the Ministry, and it was not until six years later with the second programme that they brought the procurement system more in-line with the government procurement system. It took a number of years to achieve that. These things take time. I want to use Bangladesh to illustrate a previous point I made about control because it really is my belief that donors are very reluctant to relinquish control. Bangladesh is a very striking case. I was there during the pre-negotiations for the health, nutrition and population sector programme and we were discussing with government officials the new strategy document. We had all had a copy for the last couple of weeks. We were going in to discuss it and the senior government officials would say, "We would like to discuss that document with you but we have never seen it." It was written by donor-financed consultants. This is very peculiar in Bangladesh because it wrote its own five-year plan and had the donors coming in on that, financing its plan very successfully. Bangladesh in the 1980s produced the first ever drugs policy for use with generic drugs and, against US and World Bank opposition, introduced that policy. Suddenly, 20 years later, the country cannot write its own strategy. When I asked a senior health official about this, his answer was, "The donors know what they want so why not let them write it?"

  Chairman: We did get a complaint from Afghanistan that donors were not telling them what was forthcoming. I happened to notice a report from the Afghan Government about forward expectations and the figure from DFID was zero. We all know that DFID has a commitment and they can read our reports and see it but they have not told the Afghan Government what it is apparently.

  Q29  Mr Singh: It seems that your report on the Paris Declaration is, "Could have done a lot better." What are your expectations for the High-Level Forum in Ghana? What do you expect to come out of that? Is there a way, if concrete commitments come out of that forum, that they can be sustained and implemented?

  Mr Tujan: In the first place I would like to emphasise that the Paris Declaration has gone way beyond the issue of coordination which was the agreement in Rome in 2003. In 2005 in Paris, it became aid effectiveness precisely because it was no longer about the donors coordinating. You bring in the recipient countries and they were there in Paris. They were spectators in Rome. They still felt like they were spectators in Paris, so they said. The important thing was that in Paris you already had the notion of ownership. The problem with the Paris Declaration is that the notion of ownership in Paris is about control. When you talk about control, it is about governance. We do know that development is not just about governance; it is about people. Therefore, you will find that in the Paris Declaration CSOs[13] are not there. Parliaments are not there. The commitments to accountability are very thin because obviously you cannot have proper accountability without democratic systems. That includes other players. That is why in Accra there is a movement beyond Paris. So in a sense we are moving in three-year stages in terms of commitments. It appears that in Accra there is going to be discussion now on democratic ownership, recognising that ownership means the whole process in a recipient country. That includes CSOs, parliaments, media and so on. There is a discussion of putting in CSOs to enrich the whole Paris Declaration which means you talk about accountability with the CSOs in the picture. You talk about donor accountability but it is no longer just governments. It includes citizens and CSOs in the recipient countries. There is a lot more that is coming in. Also, in the Accra Agenda there is language on conflict affected states but that language so far as formulated in the first consultative draft is very problematic. The important thing is it is there. There is also language that is being pushed on the so-called cross cutting issues on gender equality, human rights and environment. Accra, by optimistic expectations, would be quite a movement forward from the Paris Declaration and the discussion now is how to move it forward. There are some sectors which are much more conservative about it. Of course the question of implementation is another story. It always happens that way, that implementation lags behind commitment.

  Mr White: By and large, I would agree with that. There is great potential in Accra still, despite my earlier pessimism. I always come out being an optimist in the end. There is potential to try and realise the ideals of the Paris Declaration, to use it as a means for empowering the recipients to take more control of the use of aid funds. I fully accept that recipient does not just mean government; it means all forces of society. There is a great danger that Accra will result in noble statements along those lines and very little that is concrete will come out of it. It would be useful to think ahead as to how one might practically engage in the process to look for concrete things that come out of that that would result in improved aid management practices where recipients take an important part in that process. For example, something I have mentioned many times on previous occasions, the DAC peer review mechanism should be replaced by a recipient peer review mechanism whereby it is a group of recipient governments that is doing the review of donor performance, not DAC members.

  Q30  Mr Crabb: Moving on to impact evaluation, there seems to be some consensus that impact evaluation is one of the keys to achieving better coordinated, more effective aid. How credible do you think are the self-assessments that are currently being carried out by DFID and other donors? How could they be improved?

  Mr Tujan: We have been referred to the drafts of a study on that. Indeed, there is a plethora of assessments that are being made. The question is how credible are they to change behaviour. The OECD has embarked on a 12-country evaluation of the implementation of the Paris Declaration. I think that is a process that can go forward, except that for civil society we think there should be an independent monitoring and evaluation process, which means it is not by the recipient countries and it is not by the donor community, but an independent process. To have that in place, we have to look at the aid architecture because the question is who will now manage that process.

  Mr White: In terms of the assessments made by the development agencies, you have both the self-assessments carried out by the operational side of the agency, and then you have the evaluations carried out by the evaluation department of the agency. Then you have the possibility of evaluations carried out by some other, nominally independent body. To date we have essentially the self-assessments which are the vast bulk of all assessment work being done in any agency. Then, the evaluation departments are carrying out a smaller number of normally thematic or sectoral studies rather than project specific studies and you do not as yet particularly have any systematic, independent evaluation function which is the purpose of 3ie, the organisation I am heading. For both the evaluation departments' work and the self-assessment operations, the quality varies from agency to agency. I have worked for many agencies and there are some like the EU where you know that if you are critical you never work for them again. There are others like the Swedes who love to be told they are bad and do things wrong. The more critical you are, the more they like your report. DFID falls somewhere between those on the evaluation department side but on the self-assessment side DFID ranks pretty badly in terms of the operational studies that are done, particularly the routine completion reports. There is no quality-control function on those, so managers can put in pretty much what they like. They are not anyway particularly detailed studies. On the side of those I would give a pretty low mark. On the side of the evaluation departments I would give a reasonable mark of seven out of ten or so.

  Q31  Mr Crabb: In terms of the weaknesses you have identified in the operational self-evaluation, how is that manifested? What is coming out which is incorrect?

  Mr White: Evaluations serve two functions, accountability and lesson learning. I do not think they can serve either function satisfactorily. It does not send any clear signal about whether taxpayers' money is being used effectively or not because there are biases in the reporting of the outcomes of the interventions and it serves no lesson learning purpose because it is not particularly well structured and there is probably no lesson learning anyway. It is critical to draw out lessons.

  Q32  Mr Crabb: Could I ask Mr White a couple of questions about the 3ie initiative? How do you intend to ensure that it does not cut across any other similar initiatives? How would you describe the specific unique added value that you bring to this area?

  Mr White: There are three sets of similar initiatives of which 3ie is one. The first set of initiatives are those undertaken by agencies themselves to have more impact evaluations. The World Bank has been particularly active in that area in what is called the Development Impact Evaluation Initiative, DIME, which was initiated by the World Bank's research department. Last time I checked the portfolio which was in January of this year, DIME had 230 ongoing impact evaluations, which is a fantastically large increase in the number of impact evaluations going on in any one agency. There are other initiatives in the World Bank, particularly the Impact Evaluation Fund which is also funding impact evaluations of World Bank interventions. There are the agency specific initiatives. They have a very clear role because they are doing impact evaluations of initiatives funded by that agency as part of the project and the system as self-evaluations. The second initiative for which the World Bank's evaluation department has a secretariat is NONIE, the Network of Networks on Impact Evaluation. NONIE was created to create quality impact evaluation amongst, in the first instance, the official development agencies. It was created at a meeting of the DAC, the ECG, the Evaluation Cooperation Group of Multilaterals, and the UNEG, the United Nations Evaluation Group. It has since expanded to incorporate the evaluation associations of developing countries, particularly AFREA, the African Research and Evaluation Association. NONIE has no money per se to do impact studies. NONIE's role is to create awareness of the importance of quality impact evaluation and to equip people with the skills to manage, interpret or read these studies by producing guidelines on quality impact evaluation. 3ie's role is really rather different. 3ie's role is going to be as a funder of quality impact evaluations and to carry out advocacy work on evidence based policy making on the findings of impact studies financed by 3ie and from other sources. I would see quite a lot of complementarity between the roles of the different initiatives. Also, the World Bank carries out direct capacity-building work to create the capacity to do impact evaluation work in developing countries. You have NONIE which is meant to be there primarily to create demand and 3ie which is there to service that demand, if you like, a demand that will emanate from developing country governments which will then be filled by the money coming from 3ie. We have one example even though 3ie has not even started yet. One of the NONIE sub-committee members is from the Pakistan Rehabilitation Authority and 3ie is financing a study for him on how you do impact evaluation of disaster relief. Given that demand is created, the initiatives coming from the World Bank and other agencies can also help satisfy that demand by having better quality impact evaluation carried out within the World Bank financed projects and so on. I believe particularly strongly that it is important that the evaluation function rests with the recipient governments and that they see the aid money as being their money, which in the end it is. The evaluation is one way of achieving this because they will see ways for those funds to be used much better.

  Q33  Mr Crabb: What do you think is the likelihood of achieving some common quality standards across all these different evaluation mechanisms?

  Mr White: Surprisingly high. I am now in 3ie but I was involved in setting up NONIE and at the first meeting we had a couple of years ago I went in quite pessimistically, thinking that like minded donors would all sign up. Then we would have the UN, the French saying, "Oh no, it is not what we want to do." Then we would have a middle group of donors who were not one way or the other. In fact, there was unanimous agreement in that first meeting, even from UNEG members, that this was something we had to do. Whilst NONIE is having quite a hard time keeping on board with the particular view of impact evaluation which is being promoted, it is succeeding. I do not think it is something that is going to happen overnight but I believe the core supporters of both NONIE and 3ie will adopt common standards very quickly. They will make sure the impact work they do will meet those standards. There have even been movements by previous non-supporters like the Danes, who have said now very clearly, "We recognise the impact work we did previously was not to standard. We are not going to finance work for that any more. We are only going to finance studies that satisfy certain quality standards." The UN system is, if you like, the Soviet Bloc of moves to improve impact evaluation. I thought it would be the last to crumble but at their meeting in Geneva last month they did a pre-AGM session on impact evaluation and discussed both NONIE, 3ie and UNEG's role in NONIE. They are formally represented in NONIE, so there seems to be also some crumbling of the wall there as well.

  Q34  Sir Robert Smith: One of the ways donor countries try to assist is through technical assistance, providing expertise, advice and experts in-country. The Paris Declaration requires far greater coordination and alignment with developing country strategies. How can that technical assistance be enhanced by coordination? What are the obstacles to better coordinated technical assistance?

  Mr Tujan: I will predicate that with ownership. The reason why I am saying that is first of all because under the Paris Declaration coordination is very important for ownership. It does not mean that there is no coordination acting without ownership. In fact there is. That can create problems and is creating problems. We have the example of Indonesia where you have donors coordinating around the World Bank but without the ownership of the government, so in effect you have the donors running different sector-wide projects and the Indonesian Government is left out. On the question of technical assistance, coordination is premised on the leadership that is exercised by the government which would in fact enhance coordination such as clarifying first of all which donor would be in the best position to provide the technical assistance that is requested by the government. The assumption of technical assistance with ownership is that technical assistance is not simply provided by the donor; it is demanded. It is demand driven and when it is demand driven and there is coordination, then in effect the demanded TA will be paired up with which donor has the comparative advantage and that way it works better. The question of comparative advantage, the question of improving the quality of technical assistance, I think is predicated not so much on coordination but coordination under ownership.

  Mr White: I would agree with that. There are many historical examples of successful technical assistance with Meiji restoration in Japan and reclaiming land in Essex, where I come from, using Dutch technical assistance. Botswana in the 1980s and 1990s was using government purchased technical assistance. There is demand for international technical expertise that can be purchased by government from the international market for technical assistance. That market has been very seriously distorted by the donors who have required governments to take technical experts that the donor wants them to have, largely because of the desire of the donors to retain control and the lack of trust of the donor in the recipients' ability. I would agree entirely with what Antonio was saying. The key here is to put the government in charge of demand for technical assistance and only to supply technical assistance that is demanded by the government and then the coordination function falls into place.

  Q35  Chairman: Is that an opportunity for DFID to use its constraints on staffing more creatively? We have discussed that in connection with other issues. For example, instead of putting in technical experts on the DFID payroll, those technical experts would be on the recipient government's payroll. What DFID is doing is making a financial contribution to enable them to strengthen their own technical assistance. Could that be a better way of doing it?

  Mr White: Absolutely, provided those technical experts are demanded by the government.

  Mr Tujan: Even more so because the government will also be able to take certain technical experts and they would come out cheaper.

  Q36  Sir Robert Smith: The crucial bit of coordination is ownership by the recipient country rather than all the countries getting together and deciding what is needed?

  Mr White: Yes.

  Q37  Hugh Bayley: How fundamental is ownership to aid effectiveness? Can you think of examples of effective aid achieved without ownership?

  Mr Tujan: The Paris Declaration defines ownership as the most fundamental of the five principles. It is the over-arching principle. Alignment, harmonisation, mutual accountability are supposedly premised on ownership. Ownership has four dimensions to it. First, there is the political dimension which relates to state-to-state relations. After all, aid is a function of development cooperation and international relations. Also you have the problem of new administrations so when you have elections new governments come in and it changes ownership because it affects the political will of the recipient government when there is a government change. Then there is the dimension of the aid relationship and how that works when in the first place we must understand that it is almost essentially two different social systems at work. You have a very developed, industrialised country that is providing assistance to a country that is coming out of feudal or very undeveloped democratic systems and state mechanisms, and where institutions are weak. Ownership in that relationship is something that is negotiated. It is not automatic. It becomes a function of political will and capacity. A country can have the political will but not the capacity in its institutions to be able to own development. That is why we have this problem of ownership from the country side as well. The donor comes in and has to assist in capacity building. Before long, the relationship becomes paternalistic. It is a very difficult relationship when you assume that the country owns its development but that country may or may not have the political will or the capacity. The third dimension is the question of democratic ownership. These countries generally do not have good functioning democracies. Therefore, ownership for them also has to be understood in the context of how parliaments work, how civil society works and so on. There is a fourth dimension of ownership and this is the ownership by the poor themselves. They are the ones who are in the end the objects of this development issue. Do they own their development? If you build a dam or a bridge, how does it affect them? Do they own that? Most often, they are the ones who are removed from their own communities when somebody decides to dam their river. Ownership is a difficult, complicated issue but in the end the decisive dimensions are ownership in aid management and democratic ownership—the middle two. I agree it is not easy to be a donor in a situation where it is so easy to become paternalistic. Let us face it: ownership is also about capacity.

  Q38  Hugh Bayley: What you say obviously is drawn in part from your own experience in the Philippines. There is a difference between decision making and consultation, but are you really saying in a state where you do not have an open democracy, you do not have freedom of the press, you do not have freedom of a civil society to criticise the government, you do not have a multi-party system, that country ownership is a bit of a sham? The donor will own the aid programme but it should as a matter of good practice consult as broadly as it can with the poor, with parliamentarians, with civil society and the state, but because you have in liberal terms a malfunctioning state the donor will ultimately retain decision making.

  Mr Tujan: There is a great danger in that relationship that the tendency is for the donor to own it. While we keep saying ownership, ownership, ownership, it is not an easy thing. My premise is not just based on the Philippines. Our NGO works also in Indonesia, Africa, India and south Asia, as we are an international NGO. What I would like to emphasise is that you can have efficient aid. I would not say it is effective aid. That is aid that is so very focused towards the delivery of certain services, is completely transparent and so on. That can happen but in the end how effective would that aid be in terms of the poor? How would you measure the effectiveness of that aid? In the end it should be measured by how it actually impacts on the lives of the poor. That is why impact evaluation in the end matters because it is not simply about delivering some support efficiently. In the end, is it actually impacting on development? When we talk about impacting on development, it becomes a complicated process because development itself is a political and complicated political process.

  Q39  Hugh Bayley: Mr White, if ownership is good practice as a means to an end and you are assessing the technical efficiency and effectiveness of aid, should not the Paris mantra be effectiveness, effectiveness, effectiveness, not ownership, ownership, ownership? Measure the outcome and then determine what is good and what is bad.

  Mr White: The answer lies in the relationship between the two so I will go back to the earlier question because I think it does address that point also. Let us not get carried away. Of course it would be possible to design an intervention, target it and achieve positive development impacts with no local consultation whatsoever. We have a lot of experience of that. In conflict zones, we often have to do something like that but vertical programmes also do that to a large extent. They are designed top down, inserted top down. The expanded programme for immunisation for example is a largely top down programme and highly successful. It becomes horizontal over a long period of time. We have an example now of the Millennium Villages Project which is exactly that, a very top down intervention from donor countries with no consultation or government involvement. They claim it will show a very positive development outcome. That remains to be seen. One problem is the political one. In working that way you are setting aside democratic principles of consultation which I will come back to. The second one is I genuinely believe that you will undermine the efficiency and sustainability of the activity by working in that way. You undermine efficiency because there exist structures to deliver these services on the ground throughout developing countries. There is a lot of nonsense spoken about the lack of capacity in developing-country governments, particularly at district level. It is not uncommon to find now district-level workers who have bachelor's degrees and may even have overseas master's degrees. The reason why these people are not doing their jobs is because they are inadequately resourced, not because of lack of motivation or anything else. I have one example of a district building officer in central Zambia. The World Bank and EU financed social fund was doing a lot of school construction in the area and he was busy, going out visiting these schools that were being constructed, inspecting the ones that were under construction. I asked him, "How much of your time do you spend looking at social fund constructive facilities?" He said, "It is about 50% of my time or more." I asked, "Does this not take away from doing your proper job if you are spending all your time on this project?" He said, "No. This is my job. Before there was nothing to do. I am a buildings officer. These are buildings. There were no buildings before. I just used to sit and read the paper." People like this are there, able to implement these projects so why would you put in parallel structures and say that these people do not have capacity? These people do not have the money for their petrol to go to the village. Like you, we went to one resettlement village in northern Zambia. We took the resettlement officer. He had not been to the village in three years because he could not afford to go there. There was no way in which he could carry out his function simply because of the lack of a couple of pounds a year, basically. You create parallel structures where you say the capacity is not there. It is not the lack of institutional capacity; it is the lack of a very small amount of resources. In the end, you do not want to run projects that are going to be externally financed forever. You want to have these administered through the government aid which is at district level where that money can be allocated through district block grants. You have to work with these people so that you have their buy-in. This comes down to the point about the definition of ownership. I agree we need broad consultation but let us not think that in developing countries you have a much greater desire to participate and be consulted than we have in our own country. You do not. The people you really need to involve are those who are responsible for implementing the intervention. If you do not do that you are not going to get very far. You need to consult the politicians, both national and local, who are in the area where the intervention is going to be carried out. Otherwise you will not get very far. You will not get much support. You need to have some sort of consultation with the beneficiaries to make sure it is really what they want and that they understand the nature of the investment. The call for general, broad consultation should not be overstated. Back to your claim: effectiveness, effectiveness, effectiveness. I absolutely would agree but there are aspects of ownership that necessarily underlie that.


12   Development Assistance Committee, Organisation for Economic Co-operation and Development Back

13   civil society organisations Back


 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2008
Prepared 17 July 2008