Country Programme Evaluation

The core learning partnership and the users of the evaluation

From February until October 2005, the Office of Evaluation of the International Fund for Agricultural Development (IFAD) carried out a Country Programme Evaluation (CPE) for Rwanda, to take stock of the results and impact of the programme's operations in the last ten years (1994 – 2004) and to provide an input into the ongoing revision process of the Country Strategic Opportunities Paper (COSOP).

A Core Learning Partnership (CLP) of this evaluation was constituted. The CLP members will be involved in the implementation of the evaluation's recommendations by participating in the ICFG (IFAD COSOP Focal Group). It is composed of representatives of the Rwandan Authorities (Ministry of Finance and Economic Planning – MINECOFIN; Ministry of Agriculture and Animal Resources – MINAGRI; Ministry of Local Government, Community Development and Social Affairs – MINALOC; Ministry of Commerce, Industry, Investment Promotion, Tourism and Cooperatives – MINICOM; and the Ministry of Gender and Family Promotion – MIGEPROFE), farmer organizations (Network of Farmer Organizations of Rwanda – ROPARWA) and Rwandan financial institutions (Union of Popular Banks and National Bank of Rwanda). The CLP also includes the Coordinator of the United Nations Agencies in Rwanda, the United Nations Office for Project Support (UNOPS) Country Portfolio Manager and the respective directors of IFAD's Programme Management Department, IFAD's Eastern and Southern Africa Division and IFAD's Office of Evaluation.

The CLP discussed the Approach Paper for the evaluation in May 2005. The evaluation mission was fielded in June 2005 after a desk review of strategic and project documents had taken place. The evaluation team met a large number of stakeholders concerned with rural and agriculture development, both in the capital (Kigali) and in rural areas where projects funded by IFAD were or are implemented. Immediately after the evaluation mission, the aide memoire of the evaluation team was presented to the CLP members at a wrap-up workshop. During August and September 2005 the draft report was reviewed by members of the CLP, whose comments were taken into account when finalizing the report. The final report of the CPE was communicated to the Government and stakeholders in November 2005.

This Agreement at Completion Point (ACP) presents a summary of the main findings and recommendations of the CPE, and the modalities agreed upon by the ACP signatories on how the evaluation's recommendations will be acted upon. The preparation of the ACP involved discussions with a large number of stakeholders in Rwanda during a workshop in Kigali on 28 November 2005, followed by written exchanges on the draft ACP between IFAD and the Government of Rwanda. The process leading to the ACP was facilitated by the Office of Evaluation of IFAD. The ACP was agreed upon and signed by the Government of Rwanda, represented by MINECOFIN and MINAGRI, and IFAD's Programme Management Department on 13 January 2006, at IFAD headquarters in Rome, Italy.

Main findings of the evaluation

Main findings on IFAD's programme and strategy in Rwanda

Programme approach. During the period under review (1994-2004), IFAD's country strategy in Rwanda was broadly relevant to the national priorities reflected since 2000 in the Poverty Reduction Strategy Paper (PRSP) and the Vision 2020 of Rwanda, and to IFAD's mandate. Comparing the COSOPs of 1999 and 2002, analytical underpinnings improved in the second COSOP. However, the analyses in both of these country strategies were inadequate for promoting a programme approach – as the concept is understood today – that would be coherent with and complementary to the national priorities in the agriculture sector. While the COSOPs anticipated synergies within the IFAD portfolio and between projects sponsored by IFAD and other development partners, both COSOPs remained essentially administrative documents instead of tools for a strategic and dynamic management of IFAD's programme in Rwanda.

IFAD's role and focus in Rwanda. The Fund targets the poor geographically and focuses on vulnerable categories of the population. It promotes the development of farmer groups and aims at offering economic alternatives in rural areas. In its efforts to tackle the fundamental causes of poverty, IFAD distinguishes itself from other development partners through its innovative and flexible approaches. However, there are still weaknesses in prioritizing target groups during implementation, and new opportunities emerge from the Land Reform Policy, the decentralization and ongoing administrative reforms and, the Strategic Plan for Agricultural Transformation (PSTA).

Policy dialogue. Many stakeholders in Rwanda considered IFAD's recent efforts in assisting the formulation of the PSTA as an important contribution to policy formulation and regard the upcoming project for support to the implementation of the PSTA mainly through pilot schemes as highly relevant. Yet, policy dialogue was mostly considered as an evident by-product of project level interventions and little attention was given to feeding project experiences into policy dialogue at the national level. In addition, IFAD's lack of a permanent field presence, numerous changes in programme and project management staff, and the shortage of adequate financial and human resources have limited IFAD's capacity to engage actively and effectively in such a policy dialogue.

Cross-cutting themes: gender equality, environment and post-conflict management. For all issues the analytical underpinnings of IFAD's country strategy and project design documents were not well rooted in a comprehensive and accurate understanding of social and economic realities. As a result, neither design nor implementation of projects involved appropriate features to improve gender equality, to avoid negative impact on natural resources or to contribute to long-term post-conflict reconstruction and reconciliation.

Capacity building. Capacity building at all levels, for the stakeholders as well as project staff, remains a major challenge for IFAD's programme in Rwanda although it is an essential ingredient of sustainability. Project disbursements regarding capacity building are usually slower and lower than the amounts originally allocated, an indication of the lack of priority given to it by the implementation bodies of the projects.

Sustainability strategies in project design. Project design documents are usually well written, however, giving too little attention to the analysis of critical conditions and risks, like the withdrawal of a co-financier. Proper exit strategies are also often absent in these documents.

Microfinance. The mechanisms under which projects provided guarantee funds or credit to microfinance institutions and the weak recovery rate make the sustainability of the proposed credit lines very unlikely. Since 2000, notable progress has been made in the way micro-credit was tackled by the IFAD programme, which is reflected in project design since 2004. Tangible results are soon expected from the new approach adopted in different projects to support the financial sustainability of local institutions.

Main findings on IFAD funded project implementation in Rwanda

Participation and ownership. At the national level, apart from the Lead Ministry, other ministries concerned by IFAD-funded interventions are too little involved in project design and implementation. This curbs opportunities for working with cross-cutting viewpoints and for following an integrated approach. Project Steering Committees in general do not take up their roles regarding strategic discussion about project orientations or possible adjustments that need to be made in order to ensure the achievement of the project's development goals. Ownership of the project documents by the lead ministries and the Project Management Units (PMU) is usually limited. This sometimes leads to differences of opinion about project priorities, insufficient respect of these priorities and inadequate compliance to the implementation approaches that were planned. Participatory methods are not yet effectively used in IFAD-funded projects. The populations and institutions supported by IFAD are often considered as beneficiaries rather than stakeholders or partners invited and capacitated to take part in the decision process of IFAD-funded projects. It should also be noted that there is need for clarity of the role of the Community Development Committees (CDC) at district level who tend to take part too much in the operational management of projects instead of solely in strategic planning and monitoring as was intended in the national framework.

Partner performance. Shortcomings in project management have implications on the performance of IFAD's partners, for instance the high turn-over of PMU staff and delays in disbursements of counterpart funds. The performance of service providers to the projects is sometimes curbed by a lack of resources and a lack of flexibility in terms of time and procedures, which would be necessary to enable them to make the projects benefit entirely from their expertise.

Fiduciary issues. Projects often suffer disbursement difficulties because of the complex approval process in four steps (project, Lead Ministry, cooperating institution and IFAD) and, above all, because the lack of familiarity of the project staff with the administrative, financial and procurement regulations. In some cases, delays in disbursements are due to delays in processing disbursement requests.

Monitoring and evaluation (M&E). The M&E system, particularly regarding the achievement of impacts, is still inadequate. While some useful tools for M&E have been developed by the projects, there is a lack of consistency between the projects and the Government's poverty monitoring system. This makes, for instance, an assessment of IFAD's contribution to poverty alleviation and, at a national level, to the achievement of the Millennium Development Goals very difficult.

Recommendations of the evaluation agreed upon

Recommendations on IFAD's programme and strategy in Rwanda

Programme approach. IFAD, building upon a stronger field presence, should put into place a programme (rather than a project) strategy and management approach through the development of new coordination mechanisms. This approach should include:

• projects designed and managed as pillars of a wider programme, owned and managed by the Government, in support to the development and implementation of the Government's policies for the supported sectors;
• well planned synergies and complementarities between the projects financed by IFAD within the programme;
• a clear understanding of the roles of and relationships between all stakeholders in the supported sectors, at all levels (national, provincial and local) with reference to the institutional framework of the country and the decentralization policy;
• an analysis of the support that could be given to these stakeholders, including to the Government, civil society organizations, farmer organizations, the private sector and academic institutions;
• in accordance with the Paris Declaration on Development Aid Harmonization, broader and stronger relationships with other relevant development partners, and Government assessment of partner performance;
• a monitoring and evaluation system at the programme level in harmony with the project M&E system and the Government's monitoring system for the PRSP, which also permits monitoring of the continuously changing external environment in order to steer IFAD's programme and projects.

To ensure an effective coordination of the IFAD programme, a Programme Steering Committee comprised of IFAD, the Government and the other main stakeholders should be formed. This Committee will, among others, organize an annual joint review of the programme. A Country Team should also be formed in order to facilitate and support programme implementation.

Considering the numerous requirements of an integrated programme approach, it is recommended to allow more time and resources for the development of the next COSOP. Considering that country programme allocations are now based upon a Performance Based Allocation System, the assessment process should ensure that the Government is fully involved and that it is informed about the criteria and ratings.

IFAD's role and focus in Rwanda. IFAD and its partners, in their reflection upon IFAD's role in Rwanda and the strategic orientations of the next COSOP, should concentrate IFAD's intervention on following target groups (in particular in the field of agriculture, animal husbandry and off-farm activities):

• the poor rural populations of the country, making use of the categories of the poor as defined in the PRSP;
• productive and economically active women in rural areas;
• farmer organizations, given their role as important political and economic players representing the farmers in the context of decentralization;
• young people in rural areas, and specifically on developing their business capacity to address the prevalent unemployment rates.

Furthermore, recent changes in policy and the socio-economic environment should receive special attention in IFAD's programme:

• the Land Reform and its implications for rural and agricultural development;
• the PSTA and its potential positive effects for the rural poor, including strengthening of research and extension in agriculture, transformation and marketing of agricultural produce;
• the Decentralization Policy and all the requirements it brings about related to developing the capacity of district and sector level administrations and, in particular, the Regional Development Centres;
• the upcoming policy on micro-finance and the support that existing micro-finance institutions in rural areas will need to reach the poor while meeting the new regulations put in place by the National Bank of Rwanda;
• HIV/AIDS and its impact on rural communities;
• the recent developments in telecommunications and information technology soon accessible in rural areas.

Policy dialogue. The Government and IFAD should define the broad priority areas for policy dialogue to which the Fund can contribute in Rwanda, and make its objectives clear in the next COSOP. Policy dialogue should be carried out in collaboration with other relevant strategic partners, including civil society and farmer organizations. IFAD-funded project experiences should provide the main input for IFAD's contribution at policy dialogue. Active involvement in this matter will require a stronger local presence and specific material resources. This could realistically be achieved by formalizing the Country Team facilitated by the locally appointed IFAD Liaison Officer.

Policy dialogue should be carried out in partnership, through channels put into place by the Government of Rwanda. Accordingly, IFAD should take part more actively in the Development Partners Consultative Group (DPCG), comprised of the main development partners and the partner ministries. In that respect, the IFAD Liaison Officer and the Coordinators of IFAD-funded projects should continue their participation in the DPCG's Rural Development Cluster, Private Sector Cluster and Decentralization Cluster.

Cross-cutting themes: gender, environment and post-conflict management.

• In order to improve its impact on women, IFAD should develop a systematic approach to mainstream gender in its entire programme in Rwanda. Differentiated indicators according to gender should be used and reflected in the M&E system at project level by collecting and analysing disaggregated data. At the same time, a particular effort should be made to increase gender awareness and sensitivity of project teams and technical assistance teams, and this at all levels of responsibility.
• IFAD-funded projects should promote sustainable use of natural resources (soil, water and forests) and mitigate environmental impact of project activities.
• In similar situations, IFAD should adopt more elements and tools for analysis, at the programme level as well as at the level of the projects, to ensure its contribution to post-conflict reconstruction and reconciliation. These elements should take particularly into account reconciliation, contribution to the climate of trust and the preservation of equity between citizens.

Capacity building. IFAD, the Government and the project teams should put greater emphasis on capacity building for grassroots organizations and public institutions to promote sustainable development. IFAD's programme and projects should:

• integrate capacity development as a long term process in the design and implementation of its programme;
• devote greater attention to the development of planning, management, organization and dialogue skills for the stakeholders and, in particular, for farmer associations;
• base capacity building activities on needs assessments and monitor progress in order to focus efforts;
• use a broader approach to capacity development than solely trainings, including learning by doing, exchange of experiences between peers, field visits, mass communication (magazines, community radio) etc.;
• provide training for trainers and ensure the use of participatory training methods, using people's knowledge and experiences as the basis for their own capacity development;
• consider capacity building within IFAD's programme and project structures in the larger context of human resources management, comprising clear job descriptions, recruitment of the right people, regular performance monitoring, the proper use of incentives and the creation of a stimulating working environment in order to retain capable staff.

Sustainability strategies in project design. Project design should give more attention to the assessment of risks and should promote a more flexible process approach rather than a blueprint, to ensure sustainability of the structures put into place by the projects. Progressive exit strategies should be taken into account from the start of the interventions by building upon partnerships, in particular with local authorities and civil society organizations at the grass-roots level, which should operate as co-managers of project activities. Exit strategies should be re-discussed and re-agreed upon by IFAD and its partners at the mid-term review of each project.

Microfinance. IFAD's programme and projects should continue to use and support existing financial institutions in rural areas, in order to promote the access of the rural poor to sustainable financial services, considering those institutions as genuine organizations and not solely as service providers to the projects. An in-depth study of the actual status of micro-finance institutions operating in rural areas is necessary to focus IFAD's support to these institutions, particularly in order to enable them to adopt the new regulatory framework and to ensure access for the poor to their services.

Recommendations on IFAD funded project implementation in Rwanda

Participation and ownership. Active participation of all stakeholders – of which not the least the supported people and institutions – in the decision processes in the planning and implementation of IFAD's programme and projects, as well as in monitoring and evaluation, appears crucial to guarantee ownership of activities and results. Because participation and ownership are essential ingredients for sustainability, no efforts should be spared to promote them at all levels. Therefore:

• The Government and IFAD should develop an approach and proper tools in order to capacitate the different stakeholders in its projects on participatory approaches. The Government and IFAD should give particular attention as to the implementation of these approaches by the PMUs and by the decentralized administrative bodies. To this end, projects should promote methods for rural facilitation (animation rurale), enabling better communication between parties, especially between stakeholders and project teams, service providers and rural communities.
• IFAD and the Government should develop and promote a consultative process during the design and the implementation of IFAD-funded projects through the enlargement of the Steering Committees to other stakeholders and through the strengthening of the mandate of these committees for strategic supervision. This consultation would enable a better strategic integration of IFAD-supported interventions and, above all, a better integrated management of each project. Well thought participation mechanisms and frequent dialogue between key partners in project implementation would ensure a common and continuous understanding of project objectives and approaches.

Performance of partners. The Government should aim at improving stability of PMU staff through sound selection procedures and ensuring a stimulating working environment. A particular effort by the Government is required, when changing personnel, to ensure the transfer of knowledge internally. The Government should facilitate project implementation through avoiding delays in disbursement of the counterpart funds and monitoring smooth implementation of contracts with service providers. A greater flexibility in the employment of skills and knowledge may be allowed where and when necessary to enable projects to benefit optimally from external expertise.

Fiduciary issues. It would be beneficial to IFAD, the Government and the Cooperating Institution to review, simplify where possible and harmonize administrative and financial project management procedures. These procedures should clearly state deadlines for different administrative and financial steps to be undertaken by the institutions concerned. From their side, PMUs should make an effort to prepare more realistic, less ambitious annual work plans and budgets. They should also make sure that their staff have sufficient knowledge of procurement procedures and other IFAD and UNOPS procedures.

Monitoring and evaluation. The project and programme level M&E system and indicators should be harmonized and articulated with the tools developed by the Government of Rwanda for the monitoring of the national poverty reduction programme (PRSP). The M&E tools developed by IFAD to measure impact of its interventions on rural poverty (Results and Impact Management System) should be used as a complement to the national system both at the programme and the project level. To improve M&E in IFAD-funded projects, the M&E units need to be provided with adequate human and material resources and, if needed, adequate training. Regular meetings between M&E units should be held to discuss monitoring techniques and results. Moreover, IFAD and the Government should, in their partnership approach, put emphasis upon building the key stakeholders' capacity for carrying out self-monitoring and self-evaluations.

Signatures

This Agreement at Completion Point reflects the main findings and recommendations of the Country Programme Evaluation, as agreed between the Government of Rwanda and the IFAD Programme Management Department. These recommendations will be taken into account in the design of IFAD future strategy in Rwanda, which will be presented in the 2006-2011 Country Strategic Opportunities Paper.