Interim Evaluation

Introduction

Evaluation objectives. An interim evaluation of the Qinling Mountain Area Poverty-Alleviation Project (QMAPAP) in China was conducted in 2008. The main objectives were to: (i) assess the performance and impact of the project; and (ii) generate a series of findings and recommendations to guide the Government of China (GOC) and the International Fund for Agricultural Development (IFAD) in financing a follow-on project. The evaluation covered all components of the project and not only those items financed by IFAD.

Poverty situation. Poverty in China fell substantially from 360.6 million people in 1990 to 101.2 million in 2006¹.  Yet, in that same year, 128 million Chinese were still estimated to live on below a dollar per day. The poorest communities in China are mainly found in remote and resource-poor areas in the western and interior regions, often without sufficient access to clean water, arable land or adequate health and education services. The QMAPAP focussed in one of these remote areas: the Qinling Mountain range which covers areas in the Shaanxi and Hubei provinces. The QMAPAP aimed to target 310,000 households in the nine poorest counties of the two provinces and reach about 90 per cent of the village population in the 128 targeted towns.

Project description. The project aimed to achieve a sustainable increase in productive capacity, and offer increased access to economic and social resources for the targeted towns. The project design included five components: agriculture development, rural infrastructure, rural financial services, social development, and project management. The management structure for the project built on existing local Government structures at Provincial, Prefecture, County and Town level. Project Leading Groups (PLGs) were established at each level to facilitate coordination between project partners.

The total cost of the project was estimated to be US$106.3 million² , of which IFAD's loan contribution was US$29 million³. Other contributors were: US$10.4 million in grant co-financing from the World Food Programme (WFP), US$4.1 million from project participants and US$62.6 million from the Chinese Government. The IFAD loan largely supported a credit fund to stimulate rural financial services plus a contribution towards agriculture development and project management.

Project design and timeline. The project was formulated during April 1999 and approved by the IFAD Executive Board in December 1999. The Loan Agreement was signed in February 2000, but loan effectiveness was not declared until August 2001. The delay was largely due to difficulties in the establishment of partnership arrangements with WFP and finalising the sub-loan agreement for the rural finance component which were conditions for loan effectiveness. The loan closed as scheduled in March 2008.

Summary of implementation results

Overall, the project has performed well. Of the total financial allocation, 93 per cent has been expended, with expenditure per component being close to the targeted amounts. The implementation results of the project have been in accordance with targets, with most key objectives being met or exceeded. The initial delays in project effectiveness created implementation difficulties which reduced the potential synergy of rural credit with other components early in the project. However, there were substantial improvements in overall performance after the Mid-Term Review (MTR) in 2005.

The changing context of China had a major influence on the performance. The effects of policy and market changes during the project period positively influenced farming and labour practices. The project aligned well with the policy changes overall and contributed towards increased access to markets and credit funds. The project investments combined with Government investments to build local capacity and on-ground improvements in infrastructure, agricultural practices and community capacity. The project inputs not only created development opportunities within the towns and villages but also increased access by villagers in remote area to economic opportunities in other areas.

Agriculture development. Terracing, land levelling and land improvement have been carried out in the majority of project villages. Irrigation facilities were improved with a total investment of US$19.8 million. Productivity of food crops increased by an average of approximately 20-50 per cent per household, both through introduction of higher yielding varieties, diversification and improved soil and farm management. Forest coverage rate has greatly increased with both commercial tree crops and environmental trees to protect watershed areas.

Infrastructure development. Construction of infrastructure has been successful, with most types of infrastructure attaining the targets set. The new and improved roads and bridges reduced transport costs and increased access to markets.  Irrigation and flood protection resulted in increased crop yields. Potable drinking water systems have improved quality of life through substantially reduced time in carrying water, as well as associated health benefits. The expenditure for rural infrastructure was slightly higher than estimated, partly due to high demand from the communities and rising prices per unit of infrastructure during the project period. The biogas units built by the project served as an effective pilot for government projects and the number of biogas units in the project area has substantially increased through other government and private investment.

Rural financial services. The project credit fund was channelled through the existing network of Rural Credit Cooperatives (RCCs) which were the best option for rural lending at the time of design. Despite initial delays, around 85 per cent of the loan funds were disbursed. Local RCCs operate independently so the performance of RCCs varied considerably. An estimated 459,300 loans were disbursed by the time of the evaluation compared to the target of 572,000 (80 per cent accomplishment). This result was partly due to late commencement of credit activities so that the expected number of loans was not attained. Insufficient training of RCC and project staff regarding credit promotion and management was also a contributory factor. Furthermore, credit demand changed during the project. For some villagers the project loan amounts were insufficient for their investment needs. Others were able to borrow from friends and relatives so had less demand than expected for formal credit. The project did respond to the changing demand after MTR by increasing maximum loan amounts from CNY 5,000 to 7,000, but demand in the villages by the end of the project was still for higher loan amounts.

An overall loan repayment rate of 92 per cent has been estimated but again, different RCCs had varying definitions of what constituted delinquency. There was also merging of banking and local customary practices which led to different loan conditions and lending practices being applied in each area. Nevertheless, the level of credit availability has increased in all areas and clients who joined RCCs during the project have largely remained as active clients. The project-supported loans did target the poor and have contributed to stimulation of the village economy.

Social development. Participation in the project was achieved through the Village Implementation Groups (VIGs). The VIGs comprised Village Committee members plus women's and farmer's representatives. The aim of establishing the VIGs was to increase the participation of village members in local governance in general and in identifying priority activities for investment in particular. The VIGs were effective during the project period and were instrumental in coordinating installation of infrastructure, training programmes and other activities. Few remain active beyond the project period in the same format as during the project but the existing Village Committees have generally retained the remaining roles of the VIGs. The social development activities included training for women which was highly appreciated, particularly the technical training.

The participation of women in project activities grew as the proportion of migration labour for men increased. Women's status in the household increased as they were more frequently left to be the key workers and decision makers in the village in the absence of the men. 

Project management. Project Leading Groups and Project Management Offices (PMOs) were established and operated effectively. The organizational structure had two provincial PMOs, one for each province.  The structure operated reasonably well and was appropriate to maximise gains through capacity-building of local government. In the early years of the project, the IFAD processes were cumbersome and not clearly understood by the PMOs. After MTR these issues were addressed, formats were streamlined and processing time reduced. There has been a persistent issue with the design and operation of the monitoring and evaluation system which has created difficulties in data analysis and been a barrier to effective project management.

Project performance

Relevance. The QMAPAP design had high relevance to the project area and target groups. The project objectives were consistent with IFAD and national agriculture and rural development strategies and policies.  The project inputs were relevant and based on local needs, and the balance of cost allocation for each component generally matched the highest priority demands for project support in the project area. The project operated with a demand driven targeting approach which was managed at the local level. As a result, the evaluation found that results and approaches varied across the project area.

The amount of investment per village varied to respond to the highest needs identified by the villages. Nevertheless, some villages were less able to access enough resources and thus key priorities in their isolated sub-village groups were not fully addressed.

Effectiveness. The project was timely given the wider economic opportunities in the country, enabling poor families to take advantage of wider contextual opportunities. The project has effectively reached its objectives largely due to the combination of positive policy advances in the rural areas and project investments. The villages appeared to have benefited substantially from project inputs. There were elements of project operation that did not reach their full potential due to natural calamities, staffing issues and insufficient reach to the most remote households. Nevertheless, overall the quality of life in the villages has substantially improved in all key indicators. The credit component processes could have been implemented in a timelier manner so that greater synergy could have been achieved with other project initiatives.

Efficiency. The project exceeded the Appraisal Economic Internal Rate of Return (EIRR) estimates of 17.8 per cent; with estimated EIRRs of 27 per cent for Hubei and 24.5 per cent for Shaanxi.  QMAPAP also compares favourably to other similar World Bank (China) and other IFAD projects, having achieved relatively low costs per beneficiary and operating costs. The project has not exclusively achieved these benefits but has been an important contributor to improving livelihood security as well as contributing to an improvement in overall prosperity in the project areas. In general the project was assessed as efficient.

Rural poverty impact

Household income and assets have shown substantial increase. Migration work has brought the biggest increase in income for many households. While not directly attributable to the project, QMAPAP initiatives facilitated access to employment. In terms of human and social capital and empowerment, participation by both men and women in skills training allowed access to new technology and broadened knowledge of changing agricultural techniques. Involvement of women in village decision-making increased markedly. Some broader self-mobilisation and self-organization of people occurred in some villages but in general villagers' organizations are still underdeveloped. 

As agricultural productivity increased, surplus food production has increased and food security is no longer a pressing issue in the project area. Agricultural production has been greatly commercialized with large areas of cash crops planted. The rural finance component played an important part in financing new or expanded activities.

Training and livestock services had positive impact on livestock productivity, particularly for lower income households. There are some concerns regarding adoption of cash crops which are price sensitive by lower income households who are less resilient to market fluctuations. Natural resources have been more effectively utilized for food, cash and tree crops. Forest coverage has been greatly increased.

There are some negative impacts on the environment caused by intensive farming, especially for cash crops and fruit tree production because chemical fertilizers and pesticides are increasingly used.

The most significant improvements in institutions and policies came from the good cooperation between different partners. The capabilities of local public institutions in servicing the poor improved.  Yet, the project had little impact on the RCCs. The project approach conflicted with their strategy to avoid risks and to reduce operational costs. There was little policy dialogue on issues related to the project either in design or implementation. Overall, the project demonstrated strong achievement in terms of rural poverty impact.

Sustainability and innovation

Project initiatives have been a strong vehicle for sustainable change in all levels of the project. The benefits generated through infrastructure and agricultural development are likely to continue, and the credit will remain available to clients linked to the RCCs during project implementation. However, there was no formal exit strategy, and roles and responsibilities of the local government in follow on activities are not clear in all areas, which would be the key factor for sustainability as most project operations have been absorbed in existing Government processes. There is concern regarding the sustainability of some infrastructure facilities due to lack of maintenance mechanisms. The rural financial services initiated through the project are liable to continue at current levels, however there are concerns regarding the continuation of the RCCs reaching out to new poor clients once the IFAD funds are no longer directly supervised. There were few major innovations in the project, but some minor innovations, such as linkage of skills training and extension services with RCC loans, piloting of biogas units, VIGs supporting RCCs in identifying potential borrowers and registration of irrigated facilities as common ownership helped to improve project performance.

Performance of partners

The performance of IFAD was considered as low in the first half of the project. Delay in implementation of the IFAD-supported activities caused implementation issues. The Government pre-financed some activities to be financed by IFAD to avert delays in key implementation activities. The rural finance activities commenced late affecting the potential synergy of the project. The delays were compounded by processes which were considered by the project management to be unclear and overly time-consuming. At MTR, IFAD took action on the concerns, particularly reducing unnecessary documentary requirements and increasing loan ceilings. As a result there were clear improvements in the latter half of the project. The opening of the IFAD Coordination Office in Beijing was considered as very valuable in improving coordination between IFAD and GOC. As cooperating institution for project supervision, United Nations Office for Project Services (UNOPS) performed moderately satisfactorily, conducting regular supervision missions but which were considered to be overly focussed on credit and gender rather than the overall context of the project. The GOC provided the required financial and human resources for the project in an effective manner. The various ministries and line agencies (at national, provincial and local levels) involved in project implementation performed well with strong coordination and efficient use of resources. The role of the Women's Federations was well-integrated in project operation and their capacities of training and women supporting were enhanced.

However, the performance of the RCCs was rated as moderately unsatisfactory due to the delays and difficulties in delivering credit services to target groups.

Conclusions

In line with the assessment, the project's overall achievement has been rated as satisfactory. The ratings in relation to each criterion are presented in Table 1, and the ratings for partners' performance are presented below

Summary of project performance and impact

Evaluation criteria	Project evaluation ratings
A. Core performance criteria
Relevance	5
Effectiveness	5
Efficiency	5
Project Performance	5
B. Rural poverty impact	5
Household income and assets	5
Human and social capital and empowerment	5
Food security and agricultural productivity	5
Natural resources and the environment	5
Institutions and policies	4
C. Other performance criteria
Sustainability	4
Innovation, replication and scaling up	4
D. Overall project achievement	5

 

Ratings of partners performance

Partners	Project evaluation ratings
IFAD	4
Government	5
Cooperating institution (UNOPS)	4
WFP	5
RCCs	3
WF	5

 

Changing context and a shifting poverty definition.  Much policy change occurred in the project areas during the project period.

Policy changes had substantial influence on the pattern of farming, stimulating on-farm investment and a turn towards more commercial farming.  The project was only a partial contributor to the advances in the villages but it was a timely intervention that accelerated development and provided an opportunity for very poor households to benefit from the economic opportunities available. The poverty targeting in the project was initially appropriate because it assisted villages to classify households into the better-off, poor, poorer and very poor categories, with project clients being selected from the three latter groups. At the same time, there was a correlation between poor households and lower ability to access employment, particularly migration work. Within the changing economic context, the employment circumstances for households changed during the project period. Families that might be considered affluent one year, may become a poor family the following year.  This pattern meant that the project approach of defining the poor in three categories became confusing for implementation. Counties that adhered to the project definition faced greater challenges than those which worked more strategically to support economic development for the whole village.

Migration employment. Income from migrant labour makes a substantial contribution to the increase in household income in the project area. During the project period, the income was used to increase living standards, invest in assets, create local employment and provide informal loans, all of which complemented project activities. Yet there are negative impacts of migration. The labour structure of agriculture has greatly changed. Women and elders do most of the labour work. Families are separated and household decision-making is affected. The long term impact of migration on the community is not yet fully recognized.  There is likely to be a high sensitivity to employment changes in relation to external economic forces.

Investment scale versus equity. Project investments were not evenly spread across the targeted villages. Benefits were higher in villages where the investment was sufficient for multiple and synergistic inputs. Yet, there was capture of a greater proportion of benefits by VIGs with greater capability in planning and lobbying for resources. Villages that only accessed funds to assist the main village centre and or where there was a greater focus on one project intervention did not achieve the same level of development.  In some areas, remote village households were missed out by the project. 

Local capacity-building. Management and maintenance of infrastructure is still problematic because in most villages there was no formal maintenance mechanism. Accordingly, the efficiency of the project was reduced by the destruction of infrastructure caused by calamities such as floods and land slides. Apart from the insufficient capacity in local infrastructure management, in broad the project did not implement a strategic capacity-building approach.  Training tended to be adhoc at all levels.  Training and orientation in aspects important for good project implementation such as project management, rural finance and gender approaches was insufficient.

The project design called for support for developing community organizations such as farmers associations and infrastructure maintenance groups.  Such activities did not eventuate as a project strategy. 

Management information system. Insufficient assistance was provided to the PMOs to adjust the management information system at an early stage to generate the required reporting data in relation to the project logical framework. The lack of an effectively functioning management information system contributed to lost opportunities in adapting the project design to the requirements of the project area. With improved information available from the PMOs, the Ministry of Finance and IFAD would have facilitated better knowledge management and management decision-making.

Recommendations

Proceed with a follow-on project. The positive performance of the project demonstrates the value of investment in remote areas of the country. The continuing pockets of need for poverty reduction suggest that there is a valid reason for proceeding with a follow-on project. The design process should retain key elements such as: the focus on targeting of the poorest households, increased participation of women and farmers in village planning processes, an integrated package of interventions, and a project management process that strengthens local governance. The design of a follow-on project should also consider the following recommendations.

Increased flexibility in project design. The pace of change in rural China is very rapid. There are many initiatives that have overlapping impacts in the project area.  It is recommended that there is a more explicit recognition of the changing context in project design and implementation. It may be more appropriate to consider eligibility for progress redesign at each mission rather than delaying until the project mid-term. Similarly, definitions of poverty groups should not be rigid so that the project can respond to changing needs of the households in the project areas. In particular, any credit support should allow for a mix of credit sources, depending on the capacity of the potential borrower at different stages of the project. This could include community development funds and small group lending for the lowest income groups, collective credit mechanisms for cooperative activities, and RCCs for larger enterprise loans. IFAD and the GOC should work together on identifying which aspects of project design must be compliant for the whole project period and which aspects can be subject to change in line with project conditions.

Support to employment generation as a design strategy. It is likely that migration employment will continue to be a major factor in rural areas for the near future, but long term changes may bring other changes and challenges. If a further project is approved, further analysis is needed to understand the characteristics of migrant labour and the attitude of households towards external employment opportunities. This may result in consideration of special credit facilities to support migration, or organizational support for workers rights. It is also worth to consider the impact on agriculture production and community resilience for those residents remaining in the village.

Scale of operations per village. A clearer investment mechanism is required per village such that the greatest economies of scale and optimum short and long term benefits can be achieved. In project design, guidelines for an investment quota could be defined.  The participatory decision making approach of the VDP could be replicated but with greater analysis of feasibility and more stringent approval processes for any proposals that exceed the quota; and additional capacity-building for VIGs which are having difficulty in reaching the quota should be considered.

Greater local mobilisation and capacity-building. More focus should be placed on investing in local organizations, leadership and community activities that will assist in achieving greater levels of results across all impact indicators. Establishing infrastructure operations and maintenance groups would provide greater protection of project investments in physical facilities. Initiating and supporting farmers' producers and marketing organizations would provide an additional stimulus to the local economy. It would also align strongly with the current GOC policy towards strengthening farmers organizations. Strengthening local organizations would require a specific sub-component, with relevant performance indicators and suitably qualified staff.

Management information and analysis. IFAD must work with the GOC to install effective Management Information Systems (MIS) at the commencement of a project. It is critical that data is collected in a way that allows data to be easily checked for accuracy and is linked to project outcomes. Supervision missions require specific guidance on checking compliance with the MIS design and ensuring that the guidelines are followed. The generation of quality data would be a valuable resource for the GOC and IFAD in future to provide deeper insight to strategic considerations such as the attribution of project investments to the economy and social well-being in the project areas, and more broadly, learning regarding the key success factors in poverty reduction in China context.

---

1/ World Bank Development Indicators, 2006.

2/ Report and recommendation of the President to the Executive Board on a proposed loan to the People's Republic of China for the Qinling Mountain Area Poverty-Alleviation Project.

3/ Loan terms were highly concessional - 40 years, with a grace period of ten years and service charge of 0.75 per cent per annum.