Background

This is the eleventh edition of the Annual Report on Results and Impact of IFAD Operations (ARRI), prepared by the Independent Office of Evaluation of IFAD (IOE). It consolidates and summarizes the results and impact of IFAD-funded operations on the basis of evaluations conducted during 2012 and in previous years.

The Fund is one of the very few multilateral and bilateral development organizations that produces an annual report similar to the ARRI. Its production is a reflection of IFAD's overall commitment to promoting accountability, learning and transparency in reporting on performance. While the primary audience for the ARRI is IFAD Management, staff and consultants, and the Fund's Evaluation Committee and Executive Board, the report is also of interest to recipient countries and the wider development community.

Objectives

The objectives of the ARRI are to: (i) present a synthesis of the performance of IFAD-supported operations based on a common methodology for evaluation; (ii) highlight key learning issues and development challenges that IFAD and recipient countries need to address to enhance the development effectiveness of these operations.

Methodology

IOE is able to produce the ARRI because all its evaluations follow a consistent methodology, captured in the Evaluation Manual, which builds on international good practice. In preparing the 2013 ARRI, IOE took serious note of feedback on last year's edition by IFAD Management and the Evaluation Committee and Executive Board, and, accordingly, adjusted the way independent evaluation data have been analysed and reported on in this edition.

Two changes have been made. First, the 2013 ARRI has analysed and reported evaluation results based on the year in which projects were completed, rather than the year in which evaluations were undertaken (the past practice). This is consistent with the practice in other international financial institutions and has ensured that the results for each time period are representative of a similar cohort of projects.

Second, a new data series has been introduced based only on evaluation data from project completion report validations (PCRVs) and project performance assessments (PPAs). This avoids a possible problem associated with combining results across a diverse mix of evaluation types. The one limitation of the new data series is that IOE only began conducting PCRVs and PPAs in 2010. Thus it is difficult to conduct trend analysis based on these data at this stage, a limitation that will be overcome once data from other years become available. Consequently, the series based on data from all types of evaluations will continue to be presented for the time being in order to report on trends.

Main messages

The relevance of IFAD-supported projects remains generally high, illustrating the importance of the Fund as a global organization promoting sustainable small-scale agricultural development. The rural poverty impact of IFAD operations has shown improvement since 2005-2007. An upward trend is also visible in project performance and overall project achievement in projects that closed since 2009-2011.

There are three further areas of good performance that deserve to be highlighted. First, IFAD's own performance as a partner over 2011-2013 is the best since the ARRI was first issued in 2003. This is noteworthy, and the coherent Change and Reform Agenda introduced in 2009 is likely to have also been a key ingredient in improved performance in this area. Direct supervision and the consolidation of existing, and establishment of new, IFAD country offices are two fundamental changes to IFAD's operating model that are contributing to better results.

Second, IFAD operations score high in promoting innovative approaches and in scaling up, which are fundamental to achieving a wider impact on rural poverty. In fact, for a relatively small organization such as IFAD (compared with the World Bank or other regional development banks), the capacity to develop and test innovative solutions in agriculture and rural development is critical, because, if successful, these can eventually be scaled up by governments, donors and other partners. Third, IFAD operations are very good at promoting gender equality and women's empowerment, an area in which the Fund is developing a comparative advantage, a track record and specialization. This, too, is fundamental, given that a significant number of poor rural women in developing countries work in agriculture and derive their livelihoods also from related activities.

The performance of IFAD operations is generally on a par with the agriculture-sector operations of the World Bank globally, similar to that of the African Development Bank in Africa and better than the Asian Development Bank in Asia and the Pacific. This is reassuring, given that the contexts in which IFAD works are often more challenging (i.e. remote and marginalized areas with a focus on poor rural people) and the nature of its operations is generally more demanding (i.e. with a focus on participatory approaches, targeting and empowerment).

There are areas, however, that deserve added attention moving forward. Notwithstanding the positive picture in general, a large number of projects continue to manifest moderately satisfactory performance and hardly any are highly satisfactory in the evaluation criteria assessed. Thus, there are opportunities for further improvement overall.

Moreover, two areas flagged in the past remain problematic: efficiency of operations and sustainability of benefits. These are the two weakest performing evaluation criteria. Government performance as a partner has also not shown much improvement over time. This is particularly important, as projects financed by IFAD are ultimately executed by governments, and their performance is thus critical to successful outcomes.

With regard to the performance of country strategies, the ARRI underlines that country programme evaluation ratings for relevance, effectiveness and overall performance, as well as for non-lending activities (partnership-building, policy dialogue and knowledge management), have improved overall since 2006-2008. Lack of resources – human and financial – remains the main limiting factor for non-lending activities.

The ARRI also notes that the grants programme is a very important instrument at IFAD's disposal, yet there are opportunities for a more strategic use of grants, including tighter synergies at the country level with investment projects financed by the Fund. There is also room for improvement in grant monitoring, supervision and reporting on outcomes.

This edition covers one learning theme: understanding exceptional projects. The review revealed a strong association between factors in all types of countries. Poor designs, poorly managed in difficult contexts, lead to exceptionally unsuccessful project outcomes. Good designs, well managed in supportive contexts, lead to exceptionally successful project outcomes. The few exceptionally successful projects in difficult contexts generally had good designs, high-quality project management, and good support from IFAD and government. IOE will deepen this initial analysis in 2014, when it undertakes the corporate-level evaluation of IFAD's engagement in fragile states and prepares an evaluation synthesis report of the organization's opportunities and challenges in working in middle-income countries.

Conclusions

IFAD has a unique role to play in the international aid architecture in reducing rural poverty, and its projects and programmes are making important contributions to government efforts to promote sustainable small-scale agriculture development. It is an organization on the move, concerned with making adjustments to its operating model, with the aim of achieving better results. There are, however, opportunities for improvement to achieve excellence, which will require continued focus on improving internal business processes, supporting governments in improving their capacity, including in the area of monitoring and evaluation (M&E), and greater customization to country contexts.

Three specific conclusions are worth highlighting:

• There is good evidence of improvement in some aspects of IFAD's performance. However, there are a number of persistent challenges, such as in efficiency, sustainability and M&E. These require a step-change in approach. Business as usual, and incremental improvement efforts, are as unlikely to lead to significant change in the next few years as they have in the past few years.
• Considerable progress has been made in improving project design and project supervision. A similar focus and effort is now required for project management. This is emerging as a very important, but underemphasized, determinant of project performance in all country contexts. A more consistent, more effective approach to ensuring and supporting high-quality project management is required, particularly in the early stages of project implementation.
• Fragile and/or conflict-affected states are an important focus for IFAD in both low- and middle-income countries. Fragility, conflict and poverty increasingly coincide. Thus, IFAD's relatively poorer performance in these situations represents an important challenge. The forthcoming corporate evaluation of performance in fragile situations will provide a critical opportunity for significant rethinking and change.

Recommendations

The 2013 ARRI makes the following four recommendations:

• the corporate-level evaluation on fragile states planned for 2014 should be expanded to cover fragile and conflict-affected situations in both low- and middle-income countries;
• the 2014 learning theme should examine the role of government, with a particular emphasis on strengthening the performance of project management teams;
• IOE should work with IFAD Management to design and implement a corporate-level evaluation on project management, including project-level M&E; and
• IFAD Management should pay special attention in next year's Report on IFAD's Development Effectiveness and the Annual Project Portfolio Review to the persistent challenges identified in this and previous ARRIs.