Raymah Area Development Project (2010) - IOE
Raymah Area Development Project (2010)
Introduction
Objectives. As agreed with the Executive Board, the IFAD Office of Evaluation (IOE) undertook an evaluation of the IFAD-financed Raymah Area Development Project (RADP) in the Republic of Yemen in 2009. The main objectives of the evaluation were to: (i) assess the results of the RADP; and (ii) generate a series of findings and recommendations that could inform future agriculture and rural development projects in the country. The evaluation was undertaken in line with the provisions contained in IFAD's Evaluation Manual.
Project background. The RADP was approved by the Executive Board in December 1997. It became effective in July 1998 and closed in December 1997. The total project cost was US$17 million, out of which IFAD provided a loan of USD 1.2.1 million, the Government and beneficiaries provided USD 3.9 million, and USD 1 million was to be provided in grant by a cofinancier to be identified. The cooperating institution was the United Nations Office for Project Services (UNOPS) responsible for project supervision and loan administration.
The project objectives were: (1) to improve living conditions in Raymah through the provision of sustainable rural infrastructure and services and consolidating strong community organizations to articulate the community aspirations; and (2) to increase rural incomes on a sustainable basis by enhancing the productivity of small holdings. The project had four components: rural infrastructure, community development, agriculture development and project management.
Project performance
Relevance
RADP objectives were aligned with the Government's strategy as captured in its five-year development plan 1996-2000, which included an emphasis on promoting development in Raymah. Objectives were also congruent with the 1997 COSOP, which identified the needs of poor rural people in terms of, among others, rural infrastructure (water supply, and rural roads for improved access), and the development of technologies to enhance productivity and food security. At the same time, however, the project took a multisector approach by including in its design the financing of many activities in a diverse range of subsectors, adding to the project's implementation and coordination complexities. The implications of the prevailing sociocultural context, insufficient community participation experience, and the Government's institutional capacities and outreach were not adequately taken into account at entry, which led to challenges during implementation. Finally, the choice of the Cooperative and Credit Agriculture Bank (CACB) is questionable given the bank's limited experience in group lending at the time. All in all, the relevance of the RADP is assessed as moderately satisfactory.
The effectiveness of the RADP is assessed in relation to its two main objectives.
Improving living conditions in Raymah through the provision of sustainable rural infrastructure and services and the setting up of strong community organizations able to express community demands and aspirations. The provision of small-scale infrastructure (such as small dams, feeder roads, drinking water supply) contributed to improving living conditions in the project area. Poor rural communities have better access to safe drinking water and some access to market opportunities. Women's workload has been reduced as they no longer need to fetch drinking water. Feeder roads contributed to lowering transport costs (e.g. the costs of transporting a 50 kg sack of wheat in Al-Malqah was reduced from 200 to 75 Yemeni rials (YER). 1
However, the larger and much more cost-intensive rural infrastructure (e.g. the construction of large dams and the main trunk road) was not properly carried out, and therefore its contribution to effectiveness has been below expectation at design and generally unsatisfactory. Similarly, an important project activity was to make available sustainable rural finance, through which the target group could enhance their on- and off-farm income-generating activities and derive better incomes. However, on the whole, the rural finance activities under the project did not work.
Although the project made efforts to mobilize and organize communities, there were only a few groups and associations still functioning at the time of this evaluation and the involvement of women in them was low overall. There were a variety of reasons for this, including (i) insufficient capacity-building provided to community groups/associations; (ii) community groups/associations formed primarily to access project resources, rather than as lasting instruments for rural poverty reduction; and (iii) the Government's decision not to recruit an international NGO with expertise in participatory approaches and community mobilization, as originally envisaged at appraisal.
Increasing rural incomes on a sustainable basis by improving the productivity of smallholdings. There were improvements in productivity, for example as a result of the small dams for irrigation of vegetable and fruit plots, improved animal husbandry practices, and investments in small-scale irrigation infrastructure. Livestock farmers benefited from an average increase in milk productivity from 2.6 to 3.9 litres. Farm revenue increased by YER 50 per hectare for sorghum and YER 200,000 for corn in irrigated areas. However, these have not translated in sustainable improvements in incomes especially among poor rural people. A main reason for this was inadequate access to agroprocessing facilities and marketing opportunities. In conclusion, the evaluation's assessment of RADP effectiveness is moderately unsatisfactory. It is important to bear in mind that this overall judgement on effectiveness incorporates significant variation among project activities.
Efficiency
On the positive side, the time lag between RADP loan approval and loan effectiveness was seven months, which is better than the average time taken for IFAD-funded projects in Yemen. However, the RADP took 9.5 years for implementation, as compared with an average of 6.8 years taken for all IFAD-funded closed projects in Yemen. The longer implementation duration of the RADP caused additional unforeseen administrative and transactions costs for IFAD and the Government alike, thereby reducing efficiency. The complex and lengthy procedures for release of funds within the Government was a factor affecting the efficiency of implementation. CACB faced high transaction costs in loan approval and management processes. The Beit Al-Faqih to Al-Hadia trunk road was to be constructed within 18 months of loan effectiveness, but protracted delays and poor management meant that the works had not completed even at project completion. There are many other examples that illustrate poor efficiency, including the resources spent on big dams, which are largely not operational. Overall, RADP efficiency is rated as moderately unsatisfactory.
Rural poverty impact
The construction of the main trunk road, in spite of the shortcoming reported earlier, and the feeder roads have provided some access to markets and employment opportunities. The feeder roads are benefiting around 13,000 people in remote areas of Raymah. Women's incomes in some cases have also improved as a result of training in income-generating activities (sewing and embroidery, food making), but the most important aspect has been better access to safe drinking and irrigation water. In terms of human and social capital and empowerment, the RADP had a positive impact on health and human security, especially through the water reservoirs and supply systems. As a result, life risks have been reduced and women's workloads have been reduced. It is estimated that households save an average of 180 hours per year in fetching water.
The impact on creating strong community organizations and individual empowerment has been rather limited, which has constrained communities' ability to advocate with local or national authorities for better services and development activities. Some impact was achieved on agriculture productivity and food security as documented in various self-evaluation reports, which, however, was not entirely evident at the time of evaluation. The least favourable impact is in the area of natural resources management and environment. Raymah is characterized by great environmental fragility and erosion potential. The project did not take sufficiently into account the area's topography and agroecological conditions, especially in the building of large dams. For example, the large dams were constructed without a slope stabilization programme through plantations or terrace rehabilitation in the upper reaches of the dams. In fact, the RADP lacked a systematic and rigorous assessment of environmental impact. The mountainous segments of the trunk and feeder roads constructed by the RADP are exposed to the risk of landslides, as construction works were carried out with minimal protection of mountain sides and downhill agricultural land. In summary, the rural poverty impact of the RADP is considered moderately unsatisfactory.
Sustainability and innovation
Sustainability. The evaluation found the prospects for sustainability of some of the small infrastructure to be promising (e.g. open water reservoirs, drinking water schemes, feeder roads). Better sustainability is expected in those cases where investments were identified and developed with the participation of beneficiaries.
However, the sustainability of dams (especially the larger ones), the trunk road, the electricity scheme and the mechanized drinking water supply system is weak. Major financial resources were invested for these activities from the RADP. For large infrastructure, the lack of sustainability stems from inadequate technical design, low maintenance capacity and limited allocation of counterpart funds for recurrent costs. As already mentioned, the evaluation also concluded that the sustainability of community organizations and associations was weak, and noted that the provision of research and extension services had already ceased during project implementation. The rural financial services system is not considered sustainable given the extremely low repayment rates, and the women's centres constructed are mostly no longer in use. No exit strategy was developed during implementation or close to project completion to define an approach for sustainability. In conclusion, the sustainability of benefits is therefore considered unsatisfactory.
Innovation and scaling up. The promotion of community participation and empowerment and the development of social capital were the most innovative aspects of the RADP. The extension system based on a network of extension intermediaries was also innovative, as was the development of dedicated centres and the hiring of women as rural extension officers as a means to encourage their involvement in development activities. Efforts were made to innovate through research in new agriculture technologies, for example, by introducing biogas plants in households with large livestock herds.
Although the design included innovative elements, a large number of these initiatives could not be successfully implemented. For instance, RADP design severely underestimated the time and effort that would be required to change mindsets in government and among the beneficiaries to promote participatory approaches, and undertake activities that would contribute to developing gender equality and women's empowerment. In this regard, as mentioned earlier, the non-hiring of an international NGO to support social capital-building and empowerment did not help either. The adoption rates of new technology were low, as adaptive research activities did not always build on community priorities, and there was limited knowledge and understanding among beneficiaries of the farming systems promoted. Finally, the project did not have an approach for scaling up, even though it left an impression among those who were involved in its implementation of having attempted to introduce approaches that were not common in the country at the time. Innovation and scaling up are rated moderately unsatisfactory.
Performance of partners
IFAD. One of IFAD's merits was to assist the Government in undertaking a project in a remote and barely accessible area that had not previously been the focus of any major development intervention. The RADP contributed to increasing government policymakers' and donors' attention to Raymah. During implementation, Raymah was granted a full governorate status, which has far-reaching implications for its development in the future. On the other hand, IFAD shares responsibility for the design of a project that proved unsuitable to local conditions and capacities. The troubled project implementation was repeatedly emphasized in the annual supervision mission reports and the mid-term review. The fact that IFAD Management itself, in the context of its annual internal portfolio review process, classified the RADP as "at risk" for 8 years in a 9.5 year period of implementation is further confirmation of the severe problems that the project faced during implementation.
IFAD on its part did not take a firmer position with the Government to ensure the implementation of supervision mission and mid-term review recommendations, which could have contributed to better effectiveness in the end. There is also evidence that the IFAD Country Programme Managers for Yemen at the time only visited the project on three occasions throughout its implementation, even though supervision and implementation support was outsourced to UNOPS, in accordance with the IFAD operating model of the time. That said, the evaluation is aware that IFAD Management in general, in the recent past, is increasingly paying attention to portfolio management issues and there are examples of projects similar to the RADP that have been cancelled prematurely before their intended completion date. The situation in terms of dialogue and communication with the Government is remarkably different now. The current IFAD Country Programme Manager is responsible for direct supervision and implementation support of all ongoing projects in the portfolio, and maintains a close, constructive and appreciated dialogue with partners in-country. All in all, however, the performance of IFAD in the RADP is considered unsatisfactory by the evaluation.
Government of Yemen. The Government should be commended for taking the important initiative for transforming Raymah into a governorate in 2004, during the implementation of the RADP. The Government has shown maturity in accepting the findings of this evaluation constructively, and in being self-critical of past actions, which they are committed to avoiding in the future. However, in general, the poor design of the project is a responsibility the Government shares with IFAD. The fact that the RADP was financed as a loan should have served to instil in the Government a greater sense of ownership and responsibility, and motivated it to ensure better management of activities and use of resources. The Government also had a key role in nominating the first project director, which was done on a non-competitive basis, leading to the designation of an individual whose performance was inadequate. Delayed provision of counterpart funds and payment of salaries affected staff performance and motivation and hence RADP effectiveness. The performance of the Agricultural Research and Extension Authority (AREA) was inadequate in a number of aspects, especially its undertaking of research on activities that were not of priority to poor rural people. Despite the difficulties encountered during implementation, as with IFAD, the Government also did not take stern measures to monitor implementation (e.g. the project steering committee did not meet as regularly as required) and bring the project back on track. CACB, which was fully a government undertaking during implementation of the RADP, on its part did not make special efforts to ensure that loans went to the IFAD target group, and most of its portfolio was non-performing.1 In conclusion, the Government's performance overall is rated as unsatisfactory.
UNOPS. The evaluation found that UNOPS organized missions regularly, summarizing implementation progress adequately. On a less positive note, the evaluation found that supervision missions were limited in terms of their technical depth and analysis, which could be to some extent attributed to the restricted level of resources made available by IFAD for the function. UNOPS did not rigorously follow up on the implementation of the recommendations of the mid-term review. In sum, the performance of UNOPS2 was also unsatisfactory, even though it is of relative unimportance at this stage, given that IFAD is now undertaking direct supervision and implementation support of the Yemen portfolio.
Conclusions and recommendations
Conclusions
The RADP was implemented in a geographic region marked by challenging characteristics, especially in terms of its topography and its agroecological, socio-political and cultural context. The region had largely been neglected by the Government and other donors, and the evaluation therefore commends the Fund for its initiative to finance the RADP in 1997. The evaluation found that the project played an important role in opening up the Raymah region to more comprehensive development investment activities by the Government, and in bringing critical services to the area. Moreover, the RADP can be credited, at least to some extent, for the transformation of Raymah into a full-fledged governorate in 2004, thereby increasing its visibility among policymakers, donors and development practitioners.
In addition to the above, the project made some useful contributions to rural poverty reduction in the region, even though more could have been achieved. The small- scale infrastructure developed by the project (e.g. small dams, water harvesting reservoirs, community drinking water supply schemes, and feeder roads) provide important benefits to small farmers.
The large dams constructed have not proved to be successful for a variety of reasons including technical, operational and maintenance limitations. The Beit Al-Faqih to Al-Hadia trunk road, which is extremely important for the region, has not yet been fully constructed as envisaged at appraisal. Its construction and quality was affected by the difficult relationship between the project management unit and the selected contractor. The capacity and performance of the contractor was also inadequate. Moreover, as for large dams and the water supply system in general, the construction of the trunk road increased attention to several natural resources management and environmental issues that were not adequately addressed by the RADP and remain a priority. For example, in constructing roads through mountainous terrain, the project did not consider protection of downhill agricultural land and landslides of material removed to make the roads.
On another issue, the only electric scheme constructed, in Kusmah, is no longer functioning mainly because of the high costs for its operations and maintenance, and a weak system for collecting fees. This is unfortunate, as the provision of electricity had contributed to improving livelihoods among poor rural people, for example, in terms of the mechanization of the water supply system and for literacy promotion activities.
The project was instrumental in furthering understanding of the concept of participatory planning in Yemen, and encouraging community involvement in resource allocation and the implementation of development activities. However, project design overestimated government capacity at the time for community mobilization and participation-building, especially among beneficiaries who had not previously been exposed to such approaches. As such, the sustainability of community organizations and associations is a cause for concern. The expected support of an international NGO for developing social capital and mobilizing beneficiaries did not materialize, which could have proved useful in achieving this specific RADP objective. Finally, the project did pay some attention to gender issues, for which an international adviser was hired for two years in 2002, even though in the end little progress was made in promoting gender equality and women's empowerment.
The RADP supported the development of the agriculture extension system, even though only 43 out of the 150 extension intermediaries foreseen were selected and trained. Salaries and allowances for extension staff were either not provided or often provided late, affecting staff performance and morale. This led to the collapse of the extension system in 2005. A number of veterinary officers trained under the project eventually set themselves up as private service providers operating livestock pharmacies, which is a useful achievement. Research was conducted through AREA on a number of new crop varieties, even though some crops (e.g. high-yielding varieties of sorghum and peanuts) could not be adopted for various reasons: farmers lacked the requisite knowledge to do so; not all of the new technologies responded to community needs and priorities; and access to markets was limited. Four of the five nurseries set up under the project to supply farmers with horticulture plants and forestry crops were no longer operating at the time of the evaluation, due to lack of water and the unwillingness of private entrepreneurs to take over.
With regard to rural financial services, even though close to 700 individual loans were provided for a variety of on- and off-farm activities, the repayment rates were very low, jeopardizing the sustainability of the rural financial activities. Several of the loans benefited relatively more influential individuals and only a small portion of funds were provided to women.
Project management was troublesome, as recognized by the Government. For example, the first project director (who remained in the position for four years) was not selected through a competitive process and was recognized not to have the required skills and experience to manage such an operation. Likewise, the Government acknowledged that initially locating the project management unit in Sana'a was an inappropriate decision. Government monitoring efforts were limited and well below the levels required given the challenges faced in implementing the RADP.
IFAD, on its side, did not follow up promptly to redress key constraints during implementation, despite having itself classified the RADP as a "project at risk" for 8 out of the 9.5 years of the project's implementation. The evaluation considers that allowing a project with serious problems such as the RADP to continue underperforming year after year was a significant shortcoming on the Fund's part at the time. However, the recent decisions to undertake direct supervision and implementation support, strengthen portfolio monitoring and management in general, consolidate country presence, and assign a new IFAD Country Programme Manager for Yemen in 2008 are steps in the right direction, which are collectively expected to improve the overall partnership between IFAD and the Government in the future.
Recommendations
Allocate required resources for ensuring adequate understanding of country context. In the design of future projects, it is important that objectives and activities are commensurate with the institutional, social and economic capabilities of the country. In this regard, sufficient administrative resources should be allocated by Management to ensure that adequate analytic work can be undertaken during project design in Yemen, which would enable a thorough understanding of the country's policy and institutional environment. This is especially important in light of the weak institutional and policy framework in the country.
Pay attention to natural resources management and environmental issues. Given the concerns for depletion of underground water resources in Yemen, the environmental impact implications of IFAD-financed operations and the proposed coping strategies should be clearly articulated in design. Construction of large infrastructure projects should systematically include comprehensive and rigorous environmental and social impact assessments, before activities are commissioned. This would be consistent with the Fund's environmental and social assessment procedures, which were in fact followed in the design of the most recent operation in Yemen. 3
Increase involvement of the private sector in agriculture research and extension. In the light of the past experiences with and the capacities of the Government's institute for agricultural research and extension, it is recommended that opportunities be further developed to involve producers' associations and other value chain actors more actively in the provision of such services to poor rural communities. This would be in line with the strategy adopted in the more recent IFAD-funded portfolio in Yemen of a wider engagement of private-sector entities.
Build on community priorities. It is essential that future adaptive agriculture research and extension activities be adequately tailored to ensure that farmers receive technologies and advice that allow them to address their priorities for agriculture and food security.
Focus on development results. Measures to ensure a continued focus on development results are many, but the main ones could be ensuring that: project objectives are achievable and measurable; project supervision pays attention to results in addition to achievement of physical and financial targets; the implementation of supervision recommendations are monitored and promptly followed up; project-level monitoring and evaluation systems are well resourced; and baseline surveys are undertaken during design or at the outset of implementation.
Improve sustainability. In the context of small-scale infrastructure development activities, attention should be given to building management capabilities among beneficiaries and their groups for operations and maintenance. Exit strategies should be developed early on in the process, well ahead of project closure.
Complete the Beit Al-Faqih to Al-Hadia road. The Government should consider including in its own development plans the completion of the road from Beit Al-Faqih to Al-Hadia, which is critical to further improvements in connectivity, access to markets and communication.
2/ This is also confirmed in a report on microfinance in Yemen issued in June 2005 by the Consultative Group to Assist the Poor.
3/ The Economic Opportunities Programme, approved by the Board in April 2010.