Interim Evaluation¹

Insight 1: project fesign and supervision

Project design was appropriate and innovative with an objective that was relevant for the rural poor in the Tihama and is consistent with the Country Strategic Opportunities Paper, the Government's objective of improving the standards of living of the rural population and the Interim Poverty Reduction Strategy Paper. Further the focus on women and the environment is in line with IFAD's mandate.

Nevertheless, with the benefit of hindside, for some design areas the necessary information to facilitate implementation was lacking: (i) despite recognising that beneficiary participation was crucial for the sustainability of project interventions, the participatory processes to be followed was not sufficiently spelled out, nor was there clear indication that this area should be further investigated during implementation. This seriously influenced the adequacy of the participatory processes during implementation; (ii) the incompatibility of the national and IFAD's procurement procedures in respect of the ceiling above which it is necessary to advertise and request tenders as not sufficiently addressed, which delayed procurement; (iii) there was no requirement for the project to record project expenditure by component as well as by category of expenditure, which has prevented a comparison of project expenditure by component with the appraisal estimates, while the auditing requirements were too brief and general; (iv) drafts of the proposed Subsidiary Agreements, which help ensuring that these agreements are consistent with project design, were not included in the appraisal report; (v) the design of the technical assistance input did not initially include a technical assistance team leader, which is essential when a large technical assistance team is involved to coordinate the programme and ensure consistent reporting and modern management techniques; and (vi) cost recovery/sharing arrangements for project-financed services were not sufficiently stressed by the design and were introduced towards the end of the project in an attempt to ensure sustainability.

Unperceived gaps in any project design can be minimized if project supervision is able to provide the administrative and managerial support needed by project management. However, the limited resources for supervision has contributed to the problems faced by UNOPS in this project, in particular, the reduction in the number of supervision missions from two to one per year and the insufficient technical inputs during supervision. A related lesson concerns the need during supervision to insist that the resources proposed at appraisal and needed for implementation are provided. Because this course of action was not followed with respect of the support to rural women component, its outreach and potential impact were reduced. Similarly it had adverse consequences on the capability of the M&E unit.

Recommendations

• Future project design documents should include; (i) the details of the participatory process to be followed, including the arrangements at village level, who will undertake this work; and a sufficiently detailed analysis to demonstrate that the organization(s) to be involved have the capacity and experience to carryout the work involved; (ii) a review of the Government procurement procedures to ensure compatibility with the IFAD procurement guidelines; and (iii) draft subsidiary agreements so that they are part of the design and discussed prior to loan negotiations.
• The project design documents should specify: (i) a requirement for the project to record project costs by component as well as by category of expenditure so management can monitor project costs by component during implementation; and (ii) greater details concerning the audit requirements in the design documents and in the Loan Agreement. The latter should include the following: (i) that the audit will be in accordance with International Standards on Auditing and the audited accounts and financial statements, will be in a format agreed with IFAD and include separate opinions on the statements of expenditure and the operations of the Special and Project Accounts; and (iii), the audit report will be in a Long Form audit and will contain a separate management letter to which the Project Management Unit will reply within one month.
• When a large technical assistance team is proposed a TA Team Leader should be included with terms of reference that include the introduction of modern management techniques (e.g. planning, budgeting, accounting, monitoring etc.).
• Cost recovery/sharing arrangements for project-financed services should be included as part of the design, and discussed in details at project's start up workshop.
• In a challenging socio-economic context like that of Yemen there is a need to provide increased resources to the Cooperating Institution to give the support required during implementation. Two full supervision missions a year are necessary with the presence of technical specialists when required.
• In addition, IFAD needs to take prompt action in respect of non-compliance with the loan agreement if the problems identified are not to continue hampering project implementation.

Insight 2: physical achievement, beneficiary participation and sustainability

The physical achievements of the project in comparison to appraisal targets are impressive and much experience has been gained during implementation. This can be seen in the length of sand dune stabilization belts established, the associated infrastructure provided and the area protected by on-farm shelterbelts and others. In general, the project must be commended for achieving physical appraisal targets and sometimes surpassing them.

These achievements, however, are under threat because of doubt over the sustainability of the investments in sand dune stabilization and hence their replicability. This is caused by the insufficient commitment by the surrounding communities to manage the physical assets built by the project. A main reason for this is that local communities have not fully participated in site selection and were not adequately mobilized from the beginning of the process. Sustainability is also unlikely due to the continuing need to irrigate the trees after the end of the project and concerns about the future availability in many places of suitable groundwater for this purpose.

The project has recently recognized the need for community involvement, and supported for this purpose the Environmental Protection and Development Associations. The ability of these associations to fulfil the role envisaged for them could not be established.

The Tihama Development Authority should be commended for its efforts to develop its team of female extension workers, adult literacy activities and for the recruitment and training for two years of 22 women to work as midwives. Some activities have made an appreciable effect on women's workload, productivity and self-confidence e.g. the provision of domestic water supplies, health services, interest free livestock grants and the literacy classes. But so far the sustainability and replicability of these activities are in doubt. The work of the component has also been hampered because the overall provision of staff training and TA for this component was far less than envisaged at appraisal. In addition, the work of the female staff is restricted due to the lack of supplies, equipment, and transport, a lack of leadership at the central level and lower financial incentives than their male colleagues.

Recommendations

• Efforts should be devoted to developing, supporting and testing the management capacity of the existing Environmental Protection and Development Associations and establishing them where they do not yet exist.
• The project should establish the actual costs of operating each of the wells and maintaining the sand dune stabilization belts so that a realistic estimate can be made of their financial viability.
• Dialogue with the Governor of Hodeidah, the Agricultural Co-operative Union, the newly elected local councils and local NGOs should be stepped up to assess the extent to which these organizations can help to undertake community mobilization at the village level for the maintenance and financing of the sand dune stabilization belts.
• Support to any future sand dune stabilization activities should wait until the project has developed a model of community participation that shows that the sand dune stabilization belts will be sustainable when outside assistance ceases or concluded that sand dune stabilization should be treated as a public good and financed accordingly. In the meantime The Tihama Development Authority should enhance its efforts to: (i) produce the planned sand dune stabilization manual; (ii) repair localised gaps in the sand dune stabilization belts; (iii) using existing studies look closely at the regional sand movements to determine where additional sand dune stabilization belts are needed; (vi) develop an approach suitable for the coastal areas, which are the source of the sand problem; and (vii) investigate a number of fast growing and productive exotics that have been successful in shelterbelts in Wadi Tuban and elsewhere in the region.
• Activities related to rural women, highly appreciated by the local communities, should receive further support by TDA through: (i) provide the female extension staff with additional vehicles on a full time basis; (ii) introduce gender equity in respect of the staff incentive allowances paid; (iii) provide the refresher training for female extension staff; and (iv) recruit qualified women staff to provide the leadership required.
• The commendable dialogue initiated with the Ministries of Education and Health should be followed-up to ensure the effective use of the extension staff trained in adult literacy, nutrition, family health, and develop a long-term arrangement for the effective deployment of the project-trained midwives. These arrangements should also consider the provision of basic equipment and supplies required by the midwives at the village level and introduce an element of cost-recovery that will ensure sustainable service provision by them at the village level. The project should identify teachers at the village level and help them develop a sustainable model for the delivery of adult literacy and primary education for girls in the villages through the introduction of school fees, etc.

Insight 3: monitoring and evaluation and impact assessment

The reporting on physical progress by the project has been regular and adequate, based on collection of reports from experts technical division and the extension services. However, the collection, analysis and storage of data is greatly hampered by a lack of staff (with only three people designated to carry out all M&E work for the Tihama Development Authority) computers and transport. The M&E Unit is supposed to compare the data collected with the annual work programme and budget and the appraisal estimates, identify gaps and problems, and give appropriate feedback to project management. However, no structure is in place to do this regularly and feedback seems to have been provided only rarely. M&E activities are regarded more as a donor requirement rather than a management tool. While some efforts have been made towards investigating beneficiaries reaction to some project activities, by the time of the evaluation there was no comprehensive assessment of emerging project impact. In addition, data on project expenditure is only available by category of expenditure thus preventing the comparison of project costs by component with the annual work programme and budget and appraisal estimates, and the undertaking of costs benefit analysis. The Cooperative and Agricultural Credit Bank has collected data concerning the socio-economic status of the borrowers but little analysis of this data was done.

Recommendations

• Monitoring and evaluation should be an integral part of the department that is responsible for project implementation and not be the responsibility of a separate donor financed M&E unit and should be provided with adequate resources.
• The project should enhance its efforts to identify key indicators for issues like environment, employment, food security, water management, and family well being so the Project Completion Report can include an assessment of the impact of the project. Data should be collected using a standardised format to establish the socio-economic profile of the beneficiaries and analysed by the M&E unit.
• The Cooperative and Agricultural Credit Bank should computerise and analyse the data on the landholding and livestock ownership pattern of each borrower to provide a clear socio-economic profile of the borrowers to assess project impact.
• In future, projects must record project expenditure by component as well as by category of expenditure so that an analysis of the cost effectiveness of project interventions by component can be assessed and a cost benefit analysis undertaken where this is appropriate.

Insight 4: provision of credit

Evaluation findings indicate that the borrowers are generally from the IFAD target group, although they are not the poorest households as most loans require collateral. The ceiling for collateral free loans is too low for women and sharecroppers, i.e. those without collateral, to make a worthwhile productive investment. These groups also require non-credit assistance if they are to be able to make the best use of credit funds. With 99% of the IFAD loan disbursed there are no additional loan funds available for the Cooperative and Agricultural Credit Bank to make new loans in the project area and the Bank will have to meet the expected demand from other sources including the repayments from the project-financed loans already made. However, the loan repayment rate is low (62% for project loans at the end of December 2001 but higher than that for non-project loans) and there is a need for a substantial increase to ensure the sustainability of the Bank's project-financed credit operations and the Bank itself. The Bank has yet to establish a single revolving fund for the repayments as required by the Subsidiary Loan Agreement from which to make new loans in the project area. The main insights are that: (i) there is a need to reform the system of collateral based lending if credit is to benefit the poorest rural groups and women; and (ii) greater attention needs to be given to the collection of loan repayments if lending is to be financially viable and credit operations sustainable in the long-term.

Recommendations

• To ensure an uninterrupted credit supply until the end of the project period the Tihama Development Authority and the Cooperative and Agriculture Credit Bank should jointly assess what action should be taken to provide additional funds if future project lending cannot be met from the repayments of existing project loans.
• To improve disbursements of loans for water conservation and for rural women, the Bank should review its loan eligibility and increase the ceiling for collateral free loans, use informal intermediaries such as savings and credit associations and self-help groups², and employ female credit officers.
• To improve the low repayment rates for project loans Bank branches should introduce concerted efforts through follow-up visits and loan recovery campaigns.
• The Bank should withdraw the instructions to the project area branches (including Bajil, Beit-el-Faki and Hays) to maintain a branch level separate Revolving Funds and instead take steps to set up one consolidated Revolving Fund and to maintain an up-to-date record of recoveries of project loans and their deployment for re-lending. This will also ensure that recoveries of past loans to farmers outside the project area are re-lent to project area villages.
• Early implementation of the entire restructuring initiative, which IFAD is supporting, would help the Bank become a sustainable financial institution.

The following additional lessons from project experience would benefit future rural credit operations in the Yemen: (i) when the scope and allocation for credit is increased significantly during implementation, supervision missions should examine the loan eligibility and collateral criteria and widen the availability of credit to the target group; (ii) the design of credit programmes for the rural poor, especially women, should also include specific interventions to increase their access to credit e.g. capacity building, community participation, training etc.; and (iii) project design should consider means of coordination between the agencies responsible for project implementation, at both top management and field levels to avoid delays and serious implementation problems.

Options for future IFAD assistance in the Tihama

Support for sand dune stabilization should be conditional upon clear evidence that project activities are sustainable and economically viable. Other options for IFAD support, based on the analysis of current experience could include: (i) support for rural women in one or more wadis with similar components to those included under the present project's Support to Rural Women with a focus of livestock, including forage production, water supplies, rural financial services, literacy and primary health care; and (ii) support for agricultural services to enhance extension, adaptive research, rural finance and animal health and plant protection services targeted at the smaller land and pump owners and their sharecroppers.

---

1. The members of the Core Learning Partnership were: Messrs Farid Mujawar (Deputy Minister of Agriculture and Irrigation), Anwar Al Harazi (Deputy Minister for Technical Cooperation or representative), Mohamad Yahia Al Gasham (Chairman Tihama Development Authority), Abdulmalik Al Thawr (General Director of Planning and Monitoring of the Ministry of Agriculture and Irrigation), Zain Haig (Project Manager of the TEPP), Mohamad Al Wadan (Chairman of the Cooperative and Agricultural Credit Bank), Mohammed Hassani (IFAD Country Portfolio Manager for Yemen), Mohamed Chaalala (Senior Project Implementation Officer, UNOPS) and Ms Mona Bishay (IFAD Senior Evaluator).

2. A start is being made under the IFAD-assisted Al-Mahara Rural Development Project and similar arrangements have been agreed for the Dhamar Highland Areas Participatory Development Programme.