Completion Evaluation

Introduction

In line with the Evaluation Policy of the International Fund for Agricultural Development (IFAD), the Fund's Office of Evaluation (IOE) has undertaken in 2009 a completion evaluation of the IFAD-financed West Guangxi Poverty-Alleviation Project (WGPAP) in the People's Republic of China. The main purpose of this evaluation was to assess the performance and impact of the project and generate lessons learnt that will guide the Government of China and IFAD in ongoing and future agriculture and rural development programmes in the country.

The basis for the completion evaluation was the self-assessment prepared by the Asia and Pacific Division of IFAD (APR), building upon own insight during project implementation and on the Project Completion Report (PCR). The Desk Review Note, prepared by IOE prior to the main mission, was another other important source of evidence. Mission activities concentrated, among others, on discussions with focus groups and individual households, and drawing contests organized in five primary schools, one in each county visited. These contests aimed at revealing impact domains in terms of poverty impact as perceived by schoolchildren. The mission also prepared 48 scorecards focusing on village implementation groups (VIGs), 20 detailed household interview records, related to the rural finance component, and conducted a ranking exercise of project strengths and weaknesses at the wrap-up meeting in Nanning, which will be referred to in this evaluation report. Thus, the mission attempted to triangulate perceptions of project performance and impact, by taking into account viewpoints of the project's stakeholders, beyond the documentation prepared by the provincial and county Project Management Offices (PMOs) for the evaluation.

With a population of 1.31 billion inhabitants in 2007 and a land area of about 9.6 million square kilometres, China is the most populous country in the world and the third largest in terms of land size.

During the past 25 years, China achieved remarkable economic growth of more than 9 per cent per year and managed to keep inflation in check over the last ten years. In 2005, the country ranked fourth in the world in terms of gross domestic product (GDP) as a result of the economic reforms introduced three decades ago. Because of this sustained rapid economic growth, China is ahead of schedule in achieving the Millennium Development Goals for national poverty reduction. More than 400 million people have moved out of poverty by increasing agricultural productivity and creating productive employment opportunities in the service and manufacturing sectors.

The economy has become more urbanized than before, increasing the urban poor population up to 41.8 per cent in 2004. The distribution of economic benefits among economic and social groups has not been even, as inequalities still persist in three dimensions: i) rural-urban; ii) east-west (geographic); and iii) inland-coastal regions.

The national poverty reduction strategy had a marked focus on mountainous and other marginal areas, and on multi-sector poverty reduction programmes. Poverty alleviation efforts were focused on poorer administrative villages and on reaching out to all natural villages within one administrative village. Participatory approaches and capacity building at community level were increasingly seen as a major leverage factor in the development of marginal areas.

The WGPAP was approved by the IFAD Executive Board in December 2000 and declared effective in March 2002, with an IFAD loan portion of US$30 million and total project costs of US$106 million. The long term goal of the project was to achieve sustainable and equitable poverty reduction in the karst area of the Guangxi Zhuang Autonomous Region (GZAR). The overall objective was to achieve a sustainable increase in productive capacity, both on- and off-farm, and to offer increased access to economic and social resources, including financial services, education, health and social networks in ten of the poorest counties of GZAR. WGPAP was one among five integrated rural development projects that have been launched during the period 1999-2005 in the framework of the 1999 country strategic opportunities paper (COSOP), but the only IFAD-funded project designed in Southwest China during that period. The project included five components, namely: i) Agricultural Development; ii) Financial Services; iii) Social Development; iv) Rural Infrastructure; and v) Project Management. In line with the 1999 COSOP, explicit partnerships were built into the design of WGPAP with the World Food Programme (WFP) and the Rural Credit Cooperative Union (RCCU) network in the project area. While a provincial PMO oversaw project implementation, the bulk of project activities was carried out by county and township PMOs.

Project performance

WGPAP was relevant for the needs of the rural poor and addressed all important dimensions of poverty. Its design was consistent with both national policies and IFAD's strategies, and integrated lessons from other IFAD projects in China. The whole design process was predominantly based on national expertise and can be considered as good practice. Effectiveness was highly satisfactory specifically measured by indicators in the dimensions of food security, income generation, health, education and enterprise start-ups. One of the key ingredients of effectiveness was the high coverage achieved across the ten project counties. Overall efficiency was satisfactory, but the project failed to conclusively prove that the economic rate or return (ERR) at completion was probably higher than the ERR assumed at appraisal. The study commissioned to this effect by the project, while commendable as such, lacked the necessary methodological rigour.

Poverty reduction impacts of WGPAP were in the satisfactory range for all five impact domains. Despite the existence of mainstream poverty-alleviation programs funded by the Government, impact in the project counties and townships could be attributed to the project with a fair degree of certainty, whereby the mission consciously triangulated impact perception from observation points that were additional to the PCR, which was admittedly the main source of evidence. Household income and assets progressed at a significant pace, inferring that the poverty reduction targets have been attained.

In the domains of social capital and empowerment, food security and natural resources and environment, the evaluation mission also perceived significant impacts, derived from capacity building thanks to broad based literacy courses and agricultural training of which women were key beneficiaries. Food security was improved due to substantial productivity increases in cereals and sweeping crop and animal production systems diversification. The evaluation mission noticed, in relation to the impact domain of institutions and policies, remarkable progress in service delivery capacity of county and township authorities as well as of the Rural Credit Cooperative (RCC) network, but also had to recognize that generic institutional capacity at administrative village level remained weak.

Overall sustainability is rated as satisfactory. Two key factors of sustainability are the existence of mainstream government programmes in poverty alleviation - to which WGPAP was complementary - and the seamless integration of the PMO at county and township levels into local government structure. These factors outweigh areas of constrained sustainability, such as the still perceivable subsidy elements in the formation of interest and refinancing rates of the RCC network and the high level of public service delivery in remote areas. In terms of innovation and replication, the project went at length in capturing and documenting innovations, but it received insufficient innovation inputs from IFAD, despite the pronounced innovation drive in the 2005 COSOP.

A salient trait of the WGPAP evaluation is that all involved partners exhibit performance ratings in the satisfactory range. Government and beneficiary performance ranked best, not only for their ability to implement the project effectively and efficiently, but also for the determination of the PMOs at county and provincial level to establish and run a comprehensive monitoring and evaluation (M&E) system. WGPAP would have had the opportunity of transforming this M&E system into a genuine and real time management information system (MIS) if appropriate guidance and support from IFAD had been provided. Despite this shortfall, both IFAD and the United Nations Office for Project Services (UNOPS) performance ratings are in the satisfactory range, marked by continuity and consistency in personnel and concepts that have marked a clear line of business. In fact, WGPAP was seen as a largely problem-free operation by all involved actors.

Findings and Recommendations

Findings

Summarizing the main evaluation findings, a storyline of relative success emerges. The table below underlines this with ratings that are all in the category of satisfactory achievements. Compared to the ratings contained in the Desk Review Note and to those made in the self-assessment of APR, only minor variations emerge. On the other hand, WGPAP ratings fare significantly better than the ones of the Annual Report on Results and Impact of IFAD Operations (ARRI) 2008.

Evaluation rating summary

Evaluation Criteria	Ratings Completion Evaluation
A. Project performance
Relevance	5
Effectiveness	6
Efficiency	5
Overall project performance	5
B. Rural poverty impact
Household income and assets	6
Social capital and empowerment	5
Food security and agricultural productivity	5
Natural resources and the environment	5
Institutions and policies	5
Overall rural poverty impact	5
C. Overarching factors
Sustainability	5
Innovation, replication and scaling up	5
D. Performance of partners
IFAD	5
WFP	5
UNOPS	4
Government	5
E. Overall project achievement	5

 

 Although the rating summary is a comprehensive and well tested analytical framework for the assessment of project success or failure, it falls short of convincingly explaining its underlying contributing factors. In a simplified manner, project success may be seen as a function of quality at entry, implementation capacity and conducive environment. Quality at entry of WGPAP was marked by: (i) an early and unwavering determination of and agreement on key fundamentals of the project, by and between the main partners, (ii) solid partnerships established along proven patterns, (iii) strong participation and ownership of the national partners during design, (iv) participatory mechanisms involving the target population on a broad basis, and (v) a component mix and cost sharing mode that reflected perceived needs as well as comparative advantages. WGPAP's implementation capacity was driven by: (i) the experience and credibility of the provincial PMO to provide decisive guidance, (ii) the presence of committed and qualified human resources, sufficient in numbers and able to transform guidance into action, and (iii) the continuity of these human resources. The fact that poverty impact, sustainability and innovation were satisfactory stemmed from such implementation capacity, on one hand, and from a conducive environment, on the other hand. The project environment was conducive because: (i) project implementation was supported by a set of government policies that had a clear rural poverty reduction focus and that emerged when the WGPAP was designed, (ii) local government structure was capable, possibly by design and certainly in reality, to practically integrate the project as a mainstream undertaking, and (iii) a stable political environment and a booming economy during project.

Recommendations

The evaluation mission makes four recommendations as detailed below. The first refers to the design of future IFAD projects in China, the second to key aspects of institutional partnerships, in particular regarding rural financial services, while the third recommendation points to innovation and IFAD's role of its promotion. The fourth recommendation deals with M&E and MIS. It is understood that the government of GZAR intends to propose a second project to IFAD that would be based on lessons learnt form the WGPAP.

Recommendation 1: Design of future IFAD-funded projects in China. The design of future IFAD-funded rural development projects in China should continue to be responsive to the multidimensional character of poverty and the growing expertise of national stakeholders in this kind of operations, in particular: 

• Targeting of project townships should continue to be based on factual indicators. In projects focused on service provision, all villages within one township should be included in the project, and natural villages more clearly identified. Likewise, a transparent assessment of the special needs of the various ethnic minority groups is critical.
• The design process should replicate WGPAP's good practice of strong involvement of local expertise at all levels, including from the anticipated target population.
• In settings where food-for-training and food-for-work is deemed essential for broad participation and coverage, a second-generation solution regarding food aid from domestic sources should be devised with the Chinese authorities, in order to replace WFP food aid.

Recommendation 2: Institutional partnerships. Project partnerships should extend over the boundaries of provincial and local government in the strict sense, e.g.:

• VIGs or analogous bodies at village level should not only play a role for planning and monitoring purposes, but should be strengthened in their core management capacity, with special regard to service delivery functions and equitable gender representation in positions of responsibility.
• Partnership with provincial RCC networks should be put on a completely new footing, by taking into account the ongoing reform and by agreeing on a set of information to be shared that is conducive to a real-time assessment of loan portfolios and banking performance indicators in general.
• Key areas of policy dialogue should be determined systematically and given due and documented attention over the project implementation period.

Recommendation 3: Role of innovation in IFAD-funded projects. Building on the orientations of the 2005 COSOP, new IFAD-funded projects should put significantly more emphasis on innovation and its promotion, in particular:

• Opportunities for innovations should be screened systematically from formulation stage onwards and be periodically reassessed.
• Particular attention should be devoted to the identification of local sources of innovation and their mobilization during project implementation.
• Innovations that exist or emerge in the global context should be made available to IFAD-funded projects more pro-actively, possibly also via IFAD grants.
• Priority areas should encompass innovations that respond to global challenges, such as food security, soil fertility, alternative energies and climate change.

Recommendation 4: From M&E to MIS. New IFAD-funded projects should take stock from the accumulated experience of the WGPAP and actively foster a quantum leap in the state-of-the-art of M&E and MIS, with the aim to make advanced M&E and MIS methods a mainstream feature in China. This would include the following:

• The generic requirements of an overarching, yet concise, M&E system for multidimensional poverty-alleviation projects in marginal rural areas should be defined, best in a joint effort between national expertise and IFAD resources.
• The basic architecture of a real-time MIS, based upon such M&E requirements, should be designed in a way that it can be used for a wide array of IFAD-funded projects, in China and possibly also elsewhere. Again, a partnership between national expertise and IFAD would be advisable.
• The above recommendations would give a particularly concrete meaning to the development of local evaluation capacity, an initiative that IFAD has decided to support in China.