Project completion evaluation

Introduction

The Southwest Anhui Integrated Agricultural Development Project (SWAIADP) co-funded by the Government of China, the International Fund for Agricultural Development (IFAD) and World Food Programme (WFP), was implemented between December 1997 and June 2004 and aimed at reducing the chronic food shortages among half a million people living in the poorest mountainous areas of Anhui province. IFAD's Office of Evaluation (OE) undertook an evaluation of SWAIADP between May and September 2005, with fieldwork conducted between 23 June and 15 July 2005.

For this completion evaluation a Core Learning Partnership (CLP) was set up, consisting of the people and institutions that would have a stake in the recommendations of the evaluation and/or responsibilities and decisional power to implement them. This CLP has provided overall guidance to the evaluation team at different stages. In particular, it was invited to review and comment on the proposed evaluation process and methodology, participate in key discussions, as well as review and comment on various outputs from the evaluation such as the approach paper and draft evaluation report. The CLP consisted of representatives of the Ministry of Finance, the Anhui province Commission of Agriculture, the Project Management Offices (PMO) of Jinzhai, Huoshan, Yuexi, Qianshan, Taihu counties, WFP, UNOPS, and IFAD's Division for Asia and the Pacific.

This Agreement at Completion Point is the final step in the evaluation process, and defines the main findings and recommendations drawn from the evaluation, agreed upon by the Ministry of Finance, the Commission of Agriculture of Anhui province and IFAD's Division for Asia and the Pacific. OE has facilitated the process leading up to the conclusion of this agreement.

Main evaluation findings

The SWAIADP was a successful project: well-designed, well-managed and effectively implemented. It has been of great benefit to the project area as a whole, and to the vast majority of the households living within it. The keys to its success were the commitment and efficiency of government departments at all levels, and the very good lines of communication, official and unofficial, combined with true partnerships and a strong sense of ownership by all, including the beneficiaries. Moreover, the design of SWAIADP was highly relevant, partly because the idea for the project and its chief elements were home-grown and also because the formulation was carried out largely by local consultants with a close understanding of how things function at the various levels of the administration.

Strategic issues

Reaching the poor. The project targeted the rural poor geographically, as it was carried out in the 34 poorest townships of the five poorest counties of Anhui province. Project assistance, particularly credit, was not directed towards the poorest households in the first place. Instead, an approach was used where active, progressive, and often better-off farmers were directly involved in project activities to serve as models for poorer households. Because this approach was used in a relatively homogeneous area as far as poverty is concerned, with relatively few inequalities between households, also because the poorer households were consulted for the preparation of the Village Development Plans through Participatory Rural Appraisal techniques, and, finally, because measures were put into place to ensure that poorer households had access to project services (food-for-training, collateral-free loans…) the SWAIADP was gradually capable of reaching the poorer households.

Learning and replication. IFAD's initiative in attempting to mainstream loan funds through the existing network of Rural Credit Cooperatives (RCC), instead of utilizing government bureaus, has been a significant contribution to the development and sustainability of the rural finance sector in China, clearly recognized by the Ministry of Finance as well as the provincial finance bureaus. While the PMOs bore the credit risk and kept a final say in the approval of loans, the RCCs in the project area have enhanced their loan management capacity and enlarged the scope of their activities to include the delivery of a microfinance service to poor households. Later IFAD-funded projects such as the ongoing Rural Finance Programme have learned from the SWAIADP experience, putting even more emphasis on strengthening RCC capacities and enabling RCCs to independently provide microfinance services under a poverty reduction programme.

Operational issues

Market risks. In some townships there are indications that market saturation of raw silk and tea may pose a risk to the economic sustainability of project interventions. The Government is aware of this risk, and has prepared certain measures to safeguard the cultivators. Some townships are planning to re-direct the focus of loans away from these two commodities.

Agricultural extension. With the increasing cultivation of various new cash crops in the project area and the need for continued diversification, greater resources and capacities are required for extension services. The approach to extension has also been changing rapidly to reflect the needs and priorities of farmers operating within a market economy.

Training for women. Training programmes for women were popular and well-attended, but more can be done to design courses to suit the specific needs of women. A lot of emphasis was put on traditional literacy training, substituting, in part, the existing government-supported programme, while functional literacy and skills training received less attention. The manuals used for skills and technical training consist of too much text and not enough illustration. Credit training was found useful, but most women had no previous experience with banks and bank loans and some were puzzled by interest rates and repayment planning. Healthcare training was also appreciated, but doctors do not always make good teachers. In general, class sizes were large, and included people of very different levels of education and teaching requirements.

Loan recovery. At present, the loan recovery rate in actual credit operations of the RCCs is relatively lower than expected at appraisal. PMOs claim the recovery rate will gradually improve over the coming years, as defaulters are expected to repay their loans in due course. However, it appears that the only available sanction for defaulters today is the refusal of further loans.

Forest management. Farmers with permits to fell trees from their wood lots often select those trees with round and straight trunks to harvest for sale first. The mission findings suggest that harvesting at a later stage could reap higher prices and benefits to the farmers, subject to market conditions.

Management issues

Baseline survey. The baseline survey seemed over-complicated given the ample statistics available from the State Bureau of Statistics, including surveys of all the poorest households collected at the village level. The questionnaire for the baseline survey ran to many pages and included superfluous data. At the same time, and perhaps as a corollary of this, the sample of households included in the survey had to be kept too small to be representative.

Approval of the Annual Work Plans and Budget (AWPB). To be approved, the AWPB passed through three levels of aggregation and approval before entering the donor approval process involving IFAD, WFP and UNOPS. This made for a lengthy and cumbersome procedure, with IFAD's approval sometimes taking up to one month. On one occasion the provincial PMO found it necessary to start implementation before approval was received. 

Monitoring and evaluation. While the prescribed reporting procedures were carefully followed for monitoring project activities, the analysis of impact during project monitoring and evaluation appeared inadequate. Data on the incidence of disease was collected during check-ups by the township health workers, but was not effectively utilised.

Translation. Donor-funded projects require an ever increasing amount of documentation and reporting in English. The series of missions, observers and workshops also require interpretation and translation services. SWAIADP placed this burden – without perhaps being aware of its full extent – on the provincial level M&E officer, who had her own considerable portfolio to manage.

Recommendations

A. Strategic issues

Reaching the poorest. If the "model farmer" approach is applied, measures need to be put in place to ensure that the process of dissemination is encouraged and continued. Therefore, the following recommendations are given: (i) priority should be given by the RRCs and Village Implementation Groups (VIGs) to smaller "pro-poor" loans, especially those intended for the replication of successful low-cost, labour-intensive production models; (ii) visits to such successful models should be organized for the poorer households by the township PMOs, and the farmers having successfully adopted such models should be stimulated to share their experience at meetings and trainings organized for the poorer households by the township PMOs through the VIGs; (iii) VIGs should try to understand what prevents certain poor households from borrowing and search for solutions; and (iv) further capacity building in bottom-up participatory approaches for planning at the village level and targeting of the poor should be envisaged. Exchanges on how to better reach the poor between VIGs and the RCC of a township, as well as between RCCs of different townships, could be beneficial.

Learning and replication. In view of the great efforts made by the provincial authorities and the valuable experience they have gained in planning and implementing IFAD-funded activities, continuing contact between the Fund and the province is highly desirable and elements of this project should be replicated elsewhere. Support to pilot schemes for the latest generation of microfinance approaches and/or the proposed Village Development Funds, in which planning and management of schemes are devolved to village level, would be suitable ways of continuing IFAD's interest in Anhui province. It is also desirable that the Anhui provincial authorities, with the assistance of the IFAD Liaison Office, actively encourage workshops and study tours in the project area in order to disseminate the project's achievements. On the other hand, Anhui province could also benefit from learning, in particular from the later generation of IFAD projects that support the development of sustainable microfinance services in the rural areas of China. In those projects, RCCs are supported to develop their capacity for microfinance and to operate in an autonomous way in collaboration with a poverty reduction programme.

Operational issues

Market risks. The diversification of income does not consist of a single, irreversible measure, but must be seen as an ongoing process sensitive to market needs and changes and requiring further support. This support, provided by county and township agricultural departments, should include appropriate training programmes for agricultural extension officers and ready access to market information for producers. Continuing efforts should also be made by the provincial agricultural commission to introduce better quality crop and animal varieties, especially varieties indigenous to other regions of China.

Agricultural extension. The substance of agricultural extension, in terms of training and services, should be expanded to include broader concepts, such as regular communication between farmers, the need for ‘informal' agricultural education, and the challenges posed by a market economy. This process should be promoted and guided by the Anhui province's Agricultural Commission. An increase in resources (in terms of both human resources and funding) will be required to enable the extension services to meet the new challenges.

Training for women. The All-China Women's Federation should be involved to help assess women's needs and organize small-scale training programmes. The traditional literacy training provided by the project should be replaced by functional training, and its content focused on women's needs. The training programme should be complemented by technical and household skills training as well as courses on loan management. As for all training activities, appropriate content and training methodology should be developed to suit different types of trainees. More illustration, a less densely- presented text and a larger font would assist the illiterate and semi-literate. Training courses need also to be more frequent and shorter and classes need to be smaller. To improve the quality of the training courses, the instructors should be trained in adult pedagogy and participatory training methodology.

Loan recovery. The aim of reaching close to 100% loan recovery by the RCCs should be re-emphasised. How it can be achieved requires analysis and discussion at the county and township levels. The provincial PMO could engage in consultations with RCCs. Workshops, staff training and a review of current operational procedures could be considered as proven tools to address this issue with success.

Forest management. The provincial Forestry Department is encouraged to continue their good work while exploring the potential to support the development of high quality woodlots of farmers, for example through appropriate training and systematic dissemination of information in wood marketing, where appropriate.

Management issues

Baseline survey. For comparable IFAD projects in the future, the baseline questionnaire should be reduced in length and simplified to the essential, while the size of the sample should be increased in order to make the baseline survey more representative. Key ‘qualitative' impact indicators should be included in the survey.

Approval of AWPB. Simultaneous submission of the AWPB to all administrative levels in China and the donors would make a more participatory discussion possible and accelerate the process of approval.

Monitoring and evaluation. In project monitoring, more attention should be given to analysing collected data, so as to enable a better understanding of ongoing development processes and a more qualitative evaluation of project performance in terms of impact on poverty. For example, after each medical check-up, the township healthcare workers should summarize the results and submit them to the township PMOs. With the technical support of township hospitals, the PMOs could then analyse the incidence of diseases in each village, investigate the causes of high rates of disease in particular villages and tailor the healthcare training in the villages accordingly.

Translation. Future IFAD-funded projects should stipulate the recruitment of a dedicated translator/interpreter.