Completion evaluation

Background

IFAD started its operations in Yemen in 1979 and since then has funded 20 agriculture and rural development projects in the country. The total costs of these projects has been around US$646 million, which includes US$202 million in loans from IFAD on highly concessional terms and US$13 million on grant basis. Currently, five IFAD-funded projects are ongoing in Yemen. 

The Raymah Area Development Project (RADP) was approved by the IFAD Executive Board in December 1997. The loan was declared effective in July 1998 and the project closed in December 2007, after a two-year extension was granted to the original closing date. The total costs of the project were US$16 million, including an IFAD loan of US$12.1 million. The project had two main objectives to: (i) improve living conditions in Raymah through the provision of sustainable rural infrastructure and services, and the setting up of strong community organizations able to express community demands and aspirations; and (ii) increase rural incomes on a sustainable basis by improving the productivity of smallholdings. It had four main components: (i) rural infrastructure; (ii) community development; (iii) agriculture development; and (iv) project management.

As requested by the Executive Board, the IFAD Office of Evaluation (IOE) undertook a completion evaluation of the RADP in 2009/10. The main objectives of the evaluation were to: (i) assess the results and impact of the project; and (ii) develop a series of findings and recommendations that could serve as inputs for future similar operations in the country.

This Agreement at Completion Point (ACP) contains a summary of the main findings and recommendations from the project evaluation. The ACP has been reached between IFAD Management (represented by the Programme Management Department) and the Government of Yemen (represented by the Ministry of Planning and International Co-operation, as well as the Ministry of Agriculture and Irrigation). It reflects their understanding of the main findings from the evaluation (see section B below) as well as their commitment to adopt and implement the recommendations contained in section C of the ACP, within the specified timeframes. The evaluation was also discussed in the 64th session of the IFAD Evaluation Committee with the participation of the Deputy Minister of Agriculture and Irrigation of Yemen. Among other issues, he underlined at the session, the importance of the RADP in promoting development in Raymah governorate and that lessons from its implementation are being applied by the Government in new agriculture and rural development projects in the country.

Main evaluation findings

The RADP was implemented in a geographic region marked with challenging characteristics, especially in terms of its topography as well as agro-ecological, socio-political and cultural context. The region had largely been neglected by the Government and other donors in the past, and the evaluation therefore commends the Fund for its initiative to finance the RADP in the late 1990s. The evaluation found that the project played a very important role in opening up the Raymah region to more comprehensive development investment activities by the Government of Yemen, and for bringing critical services to the area. Moreover, the RADP can be credited, at least to some extent, for the transformation of Raymah into a full-fledged governorate in 2004, thereby increasing its visibility among policy makers, donors and development practitioners.

In addition to the above, the project made some useful contributions to rural poverty reduction in the region, even though more could have been achieved. The small scale infrastructure developed by the project (e.g., small dams, water harvesting reservoirs, community drinking water supply schemes, and feeder roads) provides important benefits to small farmers. The project design was instrumental for further enshrining the concept in Yemen to participatory planning and encouraging community involvement in resource allocation and implementation of development activities. However, project design over-estimated the capacity of the Government at the time in community mobilization and building participation especially among beneficiaries who were not exposed to such approaches previously.

The big dams constructed have not on the whole proved to be as successful for a variety of reasons including technical and operational and maintenance limitations. The Bait Al-Faqih to Al-Hadia road, which is extremely important for the region, has not yet been fully constructed as envisaged at appraisal. Its construction and quality was marked by a difficult relationship between the project management unit and the selected contractor. Environmental issues were also not given due consideration in the design and construction of the road. The only electric scheme constructed in Kusmah is no longer functioning, among other reasons, due to high costs for its operations and maintenance and weakness in system for collecting fees.

The RADP supported the development of the agriculture extension system, even though only 43 out of the 150 extension intermediaries were selected and trained. Salaries and allowances for extension staff were often provided late, affecting their performance and morale. A number of veterinary officers trained under the project eventually set themselves up as private service providers, operating livestock pharmacies, which is a useful achievement. Research was conducted through the Agricultural Research and Extension Authority on a number of new crop varieties, even though some crops could not be adopted due to lack of knowledge by farmers and limited access to markets. Four of the five nurseries set up under the project to supply farmers with horticulture plants and forestry crops were no longer under operation at the time of the evaluation, due to lack of water and unwillingness of private entrepreneurs to take over. Even though close to 700 individual loans were provided for a variety of on and off farm activities, the repayment rates were very low, jeopardising the sustainability of the rural financial activities. The project however did pay adequate attention to gender issues, for which an international adviser was hired for two years in 2002.

Project management was troublesome, as recognised by the Government. For example, the first project director (who remained in the position for 4 years) was not selected through a competitive process and was recognised not to have the required skills and experience to manage such an operation. Likewise, Government acknowledged that locating the project management unit – at least in the initial phases of the project – in Sana'a was an inappropriate decision.

IFAD on its side did not follow-up promptly to address key constraints during execution, in spite of classifying RADP as a ‘project at risk' for 8 out of the 9.5 years of its implementation. This was a significant shortcoming, as no specific measures were implemented to improve project effectiveness. There is also evidence that the IFAD country programme managers for Yemen at the time only visited the project on three occasions throughout its implementation. However, the recent decisions to undertake direct supervision and implementation support in Yemen, strengthen portfolio monitoring and management, consolidate IFAD country presence in Sana'a, and the assignment of a new and creative IFAD country programme manager for Yemen in 2008 are steps in the right direction, which appear to be already contributing to a better overall partnership between IFAD and the Government.   

Recommendations

The below are the six main recommendations from the project evaluation:

Allocate required resources for ensuring adequate understanding of country context. In the design of future projects, it is important that objectives and activities are commensurate with the institutional, social and economic capabilities of the country. In this regard, sufficient administrative resources should be allocated by the management to ensure that adequate analytic work can be undertaken during project design in Yemen that would enable a thorough understanding of the country's policy and institutional environment.

Pay attention to natural resources management and environmental issues. Given the concerns for depletion of underground water resources in Yemen, the environmental impact implications of IFAD-financed operations and the proposed coping strategies should be clearly articulated in design. Construction of large infrastructure projects should systematically include comprehensive and rigorous environmental and social impact assessments, before activities are commissioned. This would be consistent with the Fund's environmental and social assessment procedures, which were in fact followed in the design of the most recent operation in Yemen ¹.

Agriculture research and extension. In light of past experiences and capacities of the government's agriculture research and extension, it is recommended that opportunities be further developed to involve more actively producers associations and other value chain actors in the provision of such services to the rural poor. This would be in line with the directions adopted in the more recent IFAD-funded portfolio in Yemen of a wider engagement of private sector entities.

Focus on development results. There are a number of measures that can be emphasized to ensure a continued focus on development results. These include ensuring project objectives are achievable and measurable, that project supervision pays attention to results, in addition to achievement of physical and financial targets, the implementation of supervision recommendations are monitored and followed-up in a timely manner, project-level monitoring and evaluation systems are well-resourced, and that baseline surveys are undertaken during design or at the outset of execution. During the annual portfolio review process, IFAD Management should ensure that ‘projects at risk' such as RADP are thoroughly discussed and an action plan devised to ensure better performance in the future.

Improve sustainability.  In the context of small-scale infrastructure development activities, attention should be given to building management capabilities among beneficiaries and their groups for operations and maintenance. Exit strategies should be developed early on the process, well ahead of project closure.  

Complete the Beit al Faqih to al Hadiah road. The Government of Yemen should consider including in its own development plans the completion of the road from Beit al Faqih to Al Hadia, which is critical to further improve connectivity, access to markets, and communication.

Implementation responsibilities and timeframes. IFAD and the Government would be jointly responsible for implementing the first six recommendations in the context of the development of the new Yemen results-based country strategic opportunities programmes for 2014-2019 as well as in future projects funded by IFAD in the country. The Government would be responsible for implementing the sixth recommendation.

---

1/ Economic Opportunities Programme approved by the Board in April 2010.