Evaluation
59. Evaluation is of paramount importance to any
organisation. Evaluation is the basis for accountability, transparency,
learning, and informed decision-making. High-quality evaluation
is fundamental to any assessment of the Commission's progress
in implementing its reforms successfully. In particular, evaluation
in terms of a standard framework is a prerequisite if comparisons
are to be made between the poverty focus and effectiveness of
European development assistance and that of member states' bilateral
programmes.
60. The first Annual Report on the Implementation
of the European Commission's external assistance was published
in January 2002.[124]
As a baseline report, it refers to the situation at 1 January
2001, the date of establishment of EuropeAid. We join with others,
including Ministers attending the European Development Council
on 8 November 2001, in welcoming the recognition that it is important
to make such a report, but in being extremely disappointed with
the structure and content of the report itself. The Annual Report
makes no attempt to measure impact, fails to say how the Commission
intends to measure impact in future, and neglects to discuss priorities
or compare the different regional or sectoral programmes. The
Annual Report is a step in the right direction, but a disappointingly
small step which must be improved upon next year.
61. Evaluations of specific programmes are managed
by the Evaluation Unit. Located within EuropeAid, this unit is
in charge of the evaluation of the Commission's development and
cooperation programmes. It covers all geographical regions, as
well as sectoral programmes. Its stated objectives are, firstly
to learn lessons to improve policy and practice, and secondly
to improve transparency and accountability. The evaluation unit
reports directly to the five RELEX Commissioners on the board
of EuropeAid, and as such might be described not as being independent
of the Commission, but as having an independence within the Commission.[125]
This placing of the evaluation unit within the implementing agency
- whilst apparently peculiar - is not dissimilar to the approach
employed by DFID. Poul Nielson responded to our questions about
the location of the evaluation unit within EuropeAid, saying that:
firstly, the EC's evaluation unit makes use of external evaluators
as much as other donors do; secondly, that internal evaluators
tend to communicate their findings more clearly and with less
fear of losing subsequent contracts; and thirdly, that were the
evaluation unit to be located in DG Development or DG External
Relations, it would then be too close to the programming phase
of the project cycle.[126]
62. We agree that for the purposes of learning, an
internal evaluation unit can be best, but feel that this might
not be the case as far as accountability and transparency are
concerned. In addition, we were not convinced by Poul Nielson's
suggestion that an external unit is likely to be less willing
than an internal one to want to rock the boat. We are also aware
of serious concerns about the independence of the evaluation unit
and the transparency of its processes, concerns which have been
voiced by professional evaluators which much experience of working
with and for the EC.[127]
We recommend that further consideration is given to the multiple
objectives of evaluation and the best way of achieving these in
a transparent manner.
63. No matter whether conducted by an internal or
an external unit, any assessment of the (cost)-effectiveness of
development assistance depends upon information about both spending,
and about the impacts of that spending. Attempts to assess the
effectiveness of the European Commission's development assistance
programmes are lacking on both counts. Echoing a point made earlierparagraph
24the Development and Cooperation Committee of the European
Parliament stated in its opinion on the Budget for 2002 that,
"The system for budget classification is beneath contempt,
making it more or less impossible to establish the detailed distribution
of EU aid money sector by sector".[128]
Lacking proper systems for information management, the EC has,
up until now, failed to provide a sectoral breakdown of its development
spending, something which would enable comparisons to be made
between the EC and other DAC donors. The EC accepts that this
needs addressing. Poul Nielson described the process of retrieving
data to report on what the EC does in terms of development assistance
as "archaeology",[129]
and said that being able to report in the same way as other DAC
donors do would be "a measure of going normal, going mainstream."[130]
We were encouraged to hear that the Common Relex Information System
(CRIS) will be operational by mid-2002, and that the clear intention
is to be able to produce a sectoral breakdown for spending in
2002, and to perhaps produce a retrospective breakdown for 2000
and 2001.[131] We congratulate
EuropeAid on its progress in this area, and encourage further
endeavoursparticularly in terms of developing project-profiles
which go beyond the DAC sectoral breakdownsto improve transparency
and accountability, and to inform decision-making.
64. We are pleased that evaluation is being taken
increasingly seriously in the Commission, and are encouraged that
there has been some progress with producing an Annual Report,
improving the information systems, and with beginning to report
on the sectoral breakdown of spending. Work remains to be done
with clarifying which budget categories are focussed on development
objectives, and care must be taken to guarantee the independence
of the evaluation unit and the transparency of its processes.
We do not wish to demand more from the EC than we expect from
other donors, but we trust that the EC will continue to work with
other donors to develop a set of useful performance indicators.
64