Completion Evaluation  

Evaluation objectives. A completion evaluation of the Oudomxay Community Initiatives Support Project (OCISP) was conducted in Lao People's Democratic Republic (PDR) in 2010. The main objectives of the evaluation were: (i) to assess the performance and impact of the project; and (ii) to generate findings and recommendations useful for ongoing and future agriculture and rural development projects and programmes in Lao PDR.

Project context. Lao PDR's performance with respect to poverty reduction has been impressive during the last decade. National and rural poverty rates have fallen; the Human Development Index has risen; primary school enrolment rates have risen and access to safe drinking water and basic sanitation has increased. Infant and maternal mortality rates have fallen. However, there is still substantial rural poverty, especially in the northern region where the OCISP was implemented. Shifting cultivation and opium production have been widely prevalent in the north, although it has been officially opium free since 2005 and shifting agriculture has been substantially reduced. Household food insecurity is still a problem, especially in the rainy season.

Project description. The project aimed to improve the livelihoods of some 11,700 poor households living in 149 remote villages in all seven districts of the northern province of Oudomxay. It had five components: community development; agricultural and natural resources management (ANRM); rural financial services (RFS); rural infrastructure; and institutional strengthening. Total project cost at approval was US$21.4 million, of which the IFAD loan was US$13.41 million; the Government contribution was US$3.67 million; Luxembourg co-financed US$1.77 million as a grant (for the RFS and institutional strengthening components) and World Food Programme (WFP) contributed Food for Work assistance equivalent to US$1.76 million (for construction and training activities).

The project was implemented by provincial and district planning offices, five line agencies and two mass organizations (the Lao Women's Union and the Lao Youth Union). A Project Coordination Unit, housed in the provincial Department of Planning and Investment, was responsible for overall project management, coordination and monitoring. Technical and policy guidance was provided by Provincial and District Steering Committees.

Project design and timeline. OCISP was formulated in March 2001 and appraised in October/November 2001 jointly with the Lao Government, the German Technical Cooperation, the United Nations Office on Drugs and Crime, Luxembourg Development and WFP. It was approved by the IFAD Executive Board in April 2002, signed in July and declared effective in September 2002. The project was completed in March 2010 and closed in September 2010.

Project Performance

A major change in the project context was an increase in trade with China and Vietnam and an inflow of foreign direct investment from these countries. In particular, there was rapid increase of commercial maize growing in the Province, driven by private investment in response to increased demand from China. This development offered an important new alternative to target farmers and increased cash availability in the villages, although some villages could not benefit because of unsuitable soils or climate. Other contextual changes included the rolling out of land use planning, village consolidation, and an increased focus on village clusters (kumbans) as a sub district level of planning. In 2008, the province was affected by climatic disasters and pests (rats), which produced rice shortages in some of the target villages.

The only major project design change was a redesign of the RFS component in 2004 due to the restructuring of the banking sector.

Overall, the project was managed well but implementation was uneven because of problems with the ANRM component and delays with the RFS component. It was implemented within the planned period and had high levels of disbursement. By July 2010, the project had reached 92 per cent disbursement overall.

Community development. This component consisted mainly of support to participatory village planning and beneficiary project monitoring, gender mainstreaming and a variety of general development activities. The community development teams successfully established village development plans, on the basis of which a workplan and budget was elaborated with service providers and updated annually. The Lao Women's Union carried out gender awareness training for villagers and project staff and the Lao Youth Union carried out awareness raising activities on drugs, HIV/AIDS and the environment. Other activities carried out by various line agencies included the installation of village speaker systems, certification of cultural villages, health and hygiene training and adult literacy classes.

Agriculture and natural resources management. This component got off to a slow start due to disorganisation, limited technical direction and delays in the arrival of the international technical advisors. Implementation picked up around the mid-term but it remained weak. The most significant activities were support to maize cultivation and the expansion of paddy rice cultivation through irrigation schemes in the lowlands. However, overall beneficiary coverage was limited; female participation was low; extension activities were too diverse, and focused on the lowlands rather than on the uplands where shifting cultivation had been practiced; and the adoption rates were low.

Rural finance services. Village Savings and Credit Schemes (VSCSs) were set up in 63 villages. The total start-up fund was 315,000 Euro. By 2010, these village funds had yielded 84,000 Euro in interest (returns over borrowing) and villagers also added 79,000 euro of their own savings. The total fund now amounts to 478,000 Euro. The 63 VSCSs had 4,700 members of which 97 per cent were women. The borrowing rate averaged 78 per cent of the total capital and the overall repayment rate was 96 per cent. Most loans were used for income generation: to buy maize seed, small animals or for small trading businesses. Progress with the development of secondary and tertiary credit associations was slower and they were still not fully functional at project end.

Rural infrastructure. This component performed extremely well, meeting or surpassing most of its output targets. About 422 km of rural roads were constructed or rehabilitated. This provided improved road access to 132 villages and 9,478 households. Village road maintenance committees were set up and trained by the project to conduct routine maintenance activities, using voluntary village labour and hand tools, but they were not able to carry out major repairs such as landslides. Over 60 per cent of households now have access to improved water via the construction/rehabilitation of 91 gravity-fed systems and a few boreholes and dug wells. Water user committees were set up and were fully functional by the end of the project. A total of 28 dormitories were constructed, providing accommodation for more than 3,000 pupils (of which 49 per cent were girls).

Institutional strengthening. The new project and district coordination units were established rapidly with a full complement of staff for all positions. A comprehensive M&E system was developed in the course of the third year of project implementation. The Provincial and District Steering Committees were set up and met regularly to monitor implementation progress. With support from Lux-Development an ample training programme was carried out covering general management, financial management, M&E and computer skills, supplemented by ongoing advice from national and international technical assistance staff.

Relevance. OCISP was highly relevant to the Lao Government's policies for poverty reduction and decentralisation. Its objectives were consistent with IFAD's regional and country strategies. It was also relevant to the needs of the rural poor as it targeted the poorest villages in priority poor districts of the country's second poorest province. The project approach of developing participatory processes in the villages and strengthening the capacity of project implementers to plan, manage and monitor service delivery to the villages was in line with government policy. However, there was an ambiguity in the design about whether the ultimate project goal and purpose was to be narrowly focused on ANRM objectives or more broadly focused on rural development. There was also some inconsistency in the treatment of cross-cutting objectives such as gender mainstreaming and capacity building. These ambiguities made for some loss of focus in project monitoring.

Effectiveness. OCISP was effective in meeting its objectives in four of the five components. It was particularly successful in meeting the institutional objectives, i.e. pro poor decentralised service delivery. Major successes were also achieved in terms of improving villagers' access to markets and services through the development of financial and physical infrastructure. However, there was very limited success in meeting the ANRM objectives.

Efficiency. Initially the project had problems in establishing efficient project and financial management. There were also problems of overstaffing and staff rotation; the mechanisms for outsourcing technical consultants did not prove to be satisfactory and an early attempt to set up a customised database for integrated project management and monitoring proved to be time consuming and costly. However, with close monitoring by supervision missions, a comprehensive analysis by the Mid-Term Review, and the ongoing provision of technical assistance advice and training, efficiency improved considerably. An analysis by the Project Completion Review showed that the economic internal rate of return increased to 12.2 per cent compared with 8.7 per cent at appraisal.

Rural Poverty Impact

The overall picture is that household incomes rose and poverty diminished in the target villages during the project period. Agricultural output increased, but only in some areas, on some types of land and specific types of crops. It increased through an extension of irrigated land and substitution of maize for rice, rather than through improved productivity. These positive effects were probably limited to households with access to valley bottom land, roads and markets. There were fewer improvements in the uplands, where rice output fell and food security may have deteriorated. On the other hand, access to urban centres, schooling, and safe water were increased through the investments in rural infrastructure, with positive impacts on health and education. There were limited negative effects on the environment. Villagers' empowerment increased through the support to their village institutions and participation in development planning. Women were empowered to participate in public meetings, their workload was reduced because of easier access to water and markets, and their personal well-being improved because of better health, hygiene and literacy. The capacity of government staff to provide pro-poor services significantly increased.

Sustainability and Innovation

Participatory village planning is probably sustainable because of continued government commitment and resources, although there is uncertainty over the effect of the proposed kumban plans as an intermediate layer of planning between the villages and the districts. The ANRM and RFS outcomes are not sustainable. The rural infrastructures are only partly sustainable as the current maintenance arrangements can only deal with minor repairs. Some institutional management capacities are sustainable because project staff will use them in their new jobs. There is a high level of government commitment to sustaining project activities and outcomes, but there are budget constraints.

Most of OCISP's innovations consisted of institutional processes that were adopted in Oudomxay province for the first time, such as working through government structures, the use of participatory processes, gender mainstreaming, and capacity building. These processes are now widely used and can no longer be considered innovative. Few innovative agricultural technologies were developed, despite the urgent need for them in the upland areas. There has been limited scaling up of project processes within wider government structures and there has been little investment in capturing the good practices developed in OCISP for wider adoption.

Performance of Partners

IFAD's performance as a project partner was generally satisfactory; there was good follow-up and flexibility in accepting supervision recommendations and there was particularly good support to the monitoring and evaluation system in the carrying out of the Results and Impact Management System (RIMS) surveys. However, IFAD's timeliness in responding to project queries or requests was sometimes slow. The Lao government considered IFAD to be a collaborative and flexible partner, although it did not participate much in donor fora at national level. UNOPS provided good supervision and implementation support to the project between 2002-2006 and there was a smooth handover to direct supervision by IFAD in 2007.

The Lao Government has strong ownership of the project, which has undoubtedly been helped by the fact that it was fully implemented within government. Initial problems with project procurement, financial management and staffing were resolved and full compliance with the requisite procedures was achieved by project end.

The collaboration with cofinanciers Lux Development and the World Food Programme was extremely good. This was illustrated by the fact that there was a single annual workplan and budget for all donor and government-financed activities. WFP stated that the cooperation between the government and all the donors was exemplary and a good model of cooperation for other projects. The Food for Work activities financed by WFP were particularly important in encouraging villagers' participation in the project, and the rural finance and institutional strengthening components financed by Lux-Development had significant impact in terms of increasing village access to savings and credit and in improving project management generally.

Conclusions

OCISP ratings for project performance and impact are moderately satisfactory, pulled down in each case by ANRM. In terms of rural poverty impact, there are good ratings for village empowerment and pro-poor service delivery; but lower ratings on the impact on agricultural productivity and food security. It has a low rating on innovation and is only moderately satisfactory in terms of sustainability. So the overall project achievement rating is 4 (moderately satisfactory).

Summary of Project Performance and Impact - Ratings*

Ratings are assigned on a scale of 1 to 6 (6 = very satisfactory; 5 = satisfactory; 4 = moderately satisfactory; 3 = moderately unsatisfactory; 2 = unsatisfactory;1 = very unsatisfactory).

OCISP's main purpose was to improve the livelihoods of villagers living in remote, mountainous areas of Lao PDR where shifting cultivation and opium production had been reduced. The project was primarily designed with narrow focus on ANRM. But the design did not fully appreciate the difficulties of delivering this objective in a context with a poor, dispersed and largely illiterate population, an underdeveloped private sector and weak government capacity.

In the event, the project was helped by external forces, which expanded the cash economy through trade with neighbouring countries and provided new opportunities for many farmers to participate in the maize boom. Although the main driver for this boom was external, the project contributed to it through the expansion of the rural roads network, provision of seed and agricultural extension advice on maize growing and the mobilisation of funds through the VSCSs. But OCISP was unable to develop other economic alternatives to compensate for the decline in upland rice growing. The lesson here is that a narrowly defined ANRM project requires certain minimal market conditions and institutional capacity to be successful.

However, OCISP had broader objectives. It also aimed to change village institutions and culture, making them more participatory, more outward oriented, and more urbanised. This was a modernisation project as well as a poverty reduction one. The community development and rural infrastructure components expanded to encompass these broader objectives. The question is whether such objectives should have been reflected in the project goal and purpose at the outset, rather than the narrower economic one. Given the greater success and impact of some of these activities, compared with the ANRM ones, a broader goal was probably more appropriate at the time when the project was formulated.

OCISP was successful in building the capacity of the government agencies to fulfil their mandates and roles within the framework of the Lao Government's decentralisation policy. Training, technical assistance and guidance from supervision missions played an important part in this process, but the capacity was also built through learning by doing. OCISP acquired a reputation for successful implementation amongst government and donors. A public sector capacity building project on its own would not have achieved these results; it required the resources and experience of implementation for the built capacity to become embedded.

The challenge now is sustainability. New economic alternatives are still needed for the uplands, especially since maize cultivation is not sustainable in the medium term; the savings and credit schemes still require external support, and the rural infrastructure maintenance arrangements are still weak. The capacity of village and district level institutions to plan and manage service delivery will crucially depend on scarce government resources. There is a need to consolidate the work done to date and focus on the development of sustainable economic alternatives.

Recommendations

The main question is whether to extend the project to other non-target villages in the Province, focus on correcting the deficiencies of the current project in the existing target villages, or both. The argument in favour of consolidating within existing villages is that the enabling environment is now in place, but still weak. There is a case for extending to more remote areas because they are inhabited by extremely poor ethnic groups that have been underserved to date. The argument against replicating the project in its current form to the other villages in the province is that the government should be able to do this without further technical support and could draw on resources for it from other donors, NGOs or private sector operators.

Consolidate successful interventions in existing project villages. Consolidating project interventions in order to improve sustainability should include: (i) improving the capacity of villagers to manage their own VSCSs while continuing to strengthen and supervise the recently established district and provincial microfinance institutions; (ii) strengthening the Agricultural Technical Service Centres; and (iii) monitoring the maintenance of rural infrastructure, linking it to district services for major repairs and finding ways to increase the commitment of resources from government departments at all levels.

Focus on improving the ANRM component. The main consideration for any future project in Oudomxay must be to address the ANRM problem. Ultimately, any improvement in the livelihoods of villagers will depend on the development of sustainable economic alternatives. There are physical limits to the development of lowland rice production, land use planning policies have been limited to the amount of land available for upland agricultural production, and maize cultivation is not sustainable in the long run without measures to offset declining soil fertility. Any future ANRM strategy should focus more explicitly on the uplands and include: (i) agricultural intensification; (ii) agricultural diversification; (iii) increasing livestock productivity through forage planting; (iv) improved harvesting of non-timber forest products; (v) a value chain approach that will strengthen the links between farmers, transporters and traders; and (vi) participatory land and forest management and awareness raising on villagers' rights to use and manage natural resources.

Any new project that focused primarily on ANRM would have to address the deficiencies of the agricultural extension system, not only increasing resources and capacity building but also improving institutional management and commitment. The extension system also needs to be much more focused on innovation. Extension officers and researchers need to work together to identify problems and find solutions for upland agriculture and natural resource management. The new ANRM component should include a broader range of partnerships, including private sector operators, research institutions, the National Agricultural and Forestry Extension Service and training establishments. Government departments that have an interest in the sector could also be involved, such as the National Land Management Authority and the Ministry of Industry and Commerce. The primary responsibility for project management, coordination and decision-making should continue to be located in the provincial and district planning offices, with oversight from the local Steering Committees, but a mechanism for accessing advice from relevant national line ministries should also be established.

Incorporate more remote ethnic villages. Project activities should be extended to the more remote ethnic villages; however, the range of activities should be considered carefully. OCISP already found that it was difficult to work in these villages; transport was time consuming and there were few staff with knowledge of ethnic languages. Any future project should combine quick wins through the provision of rural infrastructure with a longer term goal to develop agriculture and natural resource management. The community development approach should be more narrowly focused on rural infrastructure and developing agricultural and natural resources, building local participatory capacities to interface with project implementers. The broad-based community development approach with a proliferation of implementers and activities may not be cost effective in the more remote areas.

Build knowledge management for wider scaling up. OCISP provided a good source of lesson learning that could be useful for other projects, government policy-makers and donors. However, little time or resources have been available to take advantage of this. A future project should systematically build in a fully-resourced knowledge management component, which analyses the lessons from OCISP and future project experiences, develops knowledge products, and organises dissemination activities with links to other projects, researchers, policy makers and beneficiaries.