Interim evaluation

Introduction ¹

The aim of the Special Country Programme (SCP) Phase II, building on that of Phase I which was conducted in Oromia and Southern Nations, Nationalities and People's Region (SNNPR) between 1987 and 1996, is to "improve food security and incomes amongst poor rural households by enhancing their resilience to drought, through intensification, diversification and commercialisation of smallholder agriculture."  SCP II has operated in Tigray, Oromia, Amhara, and SNNPR since 1999.  The objectives of the project are to improve and expand traditional small-scale irrigation schemes, enhance agricultural support services, and strengthen the government and community institutions responsible for project implementation. The project is complex, in terms of its aims, components and institutional arrangements.  The natural environment and socio-economic and political contexts are very challenging, and this must be taken into account in the assessment of project performance and impacts, as well as the formulation of a future phase of the project.

This interim evaluation (IE), taking place in the final full year of the project, was composed of a pre-mission socio-economic survey carried out in depth in three irrigation schemes and their adjacent communities, together with a four-week mission by four independent consultants, in which 22 irrigation schemes, 14 woredas and all four project regions were visited.  The methodology of the evaluation combined surveys of individual farmers (mainly gathering quantitative data) with semi-structured interviews with farmers and farmer groups, woreda officials, and regional and federal personnel, and observations on site. The IFAD Methodological Framework for Evaluation (MFE) is adopted.  The latter includes rating for project performance, impact and performance of project partners (provided in the main text).  The evaluation mission was preceded by a survey, comprising both qualitative (focus group) and quantitative (randomised administration of questionnaires) techniques.  The evaluation timing was not ideal, falling as it did in the rainy season and around the Meskel public holiday.

Main design features

Project components.  The project set out to improve irrigation infrastructure on 800ha of traditional irrigation schemes, and expand command areas by a further 4 500ha.  Dry season irrigation of a range of vegetable crops, the majority of which were to be sold into local markets, was to provide the justification for the investment of 70% of the total project funds.  The agriculture component would consist of enhanced extension services, soil conservation measures in the catchments where the irrigation schemes are located, development of seed multiplication activities, and promotion of vegetable plots for women, in or near to irrigation command areas.  A range of institutional strengthening measures would be implemented through technical assistance, training and provision of basic resources such as vehicles and equipment.

The target groups of the project would include 23 400 households farming approximately 0.25ha each in small-scale irrigation schemes; 10 000 farmers on rainfed land who would benefit from soil conservation measures; 2 400 women vegetable gardeners each cultivating a 200m2 plot; the community level, woreda, Regional and Federal institutions which would be supported directly or indirectly; and, supposedly, 2 million people in the wider community who would benefit from the availability of vegetables in local markets.  The last figure lacks credibility since the population involved is composed of very poor households in the most food-insecure woredas of Ethiopia whose purchasing power is extremely limited.

Implementing partners and funding.  The project is coordinated by a Project Coordination Unit (PCU) within the Federal Ministry of Water Resources, which liaises with the relevant organs of Regional Government in the four regions of Tigray, Amhara, Oromia, and SNNPR.  Funding amounting to USD32.4m (including physical and price contingencies) has been made available by a loan from IFAD (USD 22.5m), GoE (USD 6.2m), farming communities (USD 3.1m, in kind), and an Irish Government grant (USD 1.34m).  Supervision is provided by UNOPS from its base in Nairobi.

Changes at Mid Term Review (MTR).  The main formal changes to the project design appeared at MTR in 2002.  The MTR envisaged a final total of 40 Small-scale Irrigation (SSI) schemes, covering 5 190ha and benefiting approximately 21 000 households, being completed by end of project.  The MTR also introduced limited provisions for the rehabilitation of schemes developed under SCP I.  A new component of Water Management was added, which absorbed the activity of Water Users Associations (WUAs) establishment and added several new activities.  The agricultural component was significantly restructured, with a more detailed breakdown of activities.  The MTR proposed several changes to the project period, targets and costs.  The MTR noted that the project had not commenced until February 1999, and so the six-year implementation period envisaged at Appraisal would run until February 2005.  The MTR recommended completion at the end of the 2004-05 fiscal year, i.e. 6th July 2005, and a project closing date of 31st December 2005.

Summary implementation results

Monitoring of project achievements has been very poor.  This evaluation concludes that, due to the weak M&E system, neither IFAD, nor UNOPS, nor the PCU, nor GoE more widely has more than a very general impression (from disbursements and expenditures) of the achievements to date of this project.  Probably the most reliable information concerns the number of irrigation schemes completed (46 out of 58 by September 2004).  There appear to have been very limited achievements in the agriculture component of the project, the most alarming single area being that of soil conservation.  Achievements in relation to institutional strengthening have been mixed.

Expenditure.  Overall, to July 2004, 55% of the IFAD funds had been spent, the level of expenditure across the regions varying from 30 to 60%.  Only 7% of the Irish Government grant had been spent, but the main reason for this appeared to be due to misunderstandings in the Ministry of Water Resources. 

Performance of the project

Relevance.  The project is very relevant to its target communities, to national policies and to IFAD principles and strategic thrusts.  However, little analysis appears to exist of the traditional irrigation schemes which SCP II sets out to improve.  It may be that lower (capital) input options could effectively and efficiently spread project benefits to a wider target group than under the present project design.  Also, while a key element of the project is the commercialisation of irrigated farming, the challenges of linking farmers in remote areas to markets which can absorb their production are very great.  Difficulties of physical access, the domination of the market by traders who dictate price to farmers who have no bargaining power, and the small overall size of the market, are major hindrances to the achievement of the goal of commercialisation.

Effectiveness.  In a project of this type, it is very early to be attempting to measure impact and effectiveness.  In relation to irrigation development, the peak year of construction will turn out to be 2003/04, with the expectation of full realisation of benefits by farmers only after a minimum of six years (according to the assumptions at Appraisal, which tend to be optimistic) from then.  The limited expenditure and achievements in the agriculture component have already been mentioned.

Targeting.  The project has been well targeted at woreda level, with 70% of irrigation schemes being located in woredas defined by an international consensus to be highly or very highly vulnerable.  The evaluation found that significant efforts had been made to reach vulnerable areas.  Some landless individuals and households may benefit from the project through providing labour, usually through share-cropping arrangements in the developed irrigation schemes.

Efficiency.  The unit (per ha) costs of small-scale irrigation development in SCP II lie between USD1 100 and 6 500 (EB9 800 and 56 700), depending on whether only direct construction costs, or full project costs, or something in-between, are included.  These are commensurate with norms for this type of infrastructure in sub-Saharan Africa.  It is too early to observe the full benefits, but a critique of the benefits assumed at appraisal shows that many of these assumptions (yields, percentage of crop sold, prices, need for maintenance) were optimistic.  Economic rate of return is almost certainly substantially less than proposed at appraisal (15%), but it is difficult to identify interventions with demonstrably higher rates of return, as well as attractiveness to farmers and Government stakeholders.

Rural poverty impact

Physical and financial assets of target irrigation farmers have started to improve.  There are instances in which ‘modernisation' of irrigation has made matters worse for farmers, but overall these are few in number.  Impact on financial assets has been limited by low producer prices when middlemen are involved, poor roads, gluts, and consequently farmers' preference to retain for consumption crops which otherwise might be sold.

Impact on human assets, in the form of skills and knowledge, has been limited by the generally poor quality of extension work, unimaginative use of trials and demonstrations, and limited institutional support provided so far by the project.  Concerning the impact on social capital, the establishment, strengthening and empowerment of local organisations for water management has generated confusion.  Traditional water user groups have not been exploited effectively in the move toward ‘modern' organisational forms (WUAs and cooperatives).  The stakeholder mandated to strengthen WUAs is focused on the promotion of cooperatives, which are unattractive to some (perhaps many) farmers because of associations with the former Government (the Derg).  The situation remains confused, and a resolution to this issue is urgently needed.

Food security, in the sense of increased and more reliable production and increased income, is improving, for irrigation farmers.²

The range of dietary intake is also widening due to crop diversification.  The cash generated from selling vegetables and other produce is commonly used to buy food to cover the household food demand during the food deficit months.  Some farmers spoke of a reduction in hungry months from about six to two (July and August).

A positive impact on the environment, through soil conservation, is crucial to the sustainability of the physical irrigation assets, as well as to the wider spread of benefits beyond the command areas.  So far very little has been achieved in this respect, although funds are available for this purpose.  There is a danger of soil degradation within the command areas, if soil nutrients are not managed carefully through inclusion of legumes within crop rotation and use of fertiliser and manure.  Waterlogging and salinity risks need to be regularly assessed.

The institutional context – especially government re-organisations and decentralisation – has limited the impacts of the project.  Coordination among the various federal and regional stakeholders is weak and participation in planning at the woreda level is not encouraged.  Conversely the project has had little demonstrable impact on institutions (in the sense of ‘rules of the game'), policies and regulatory framework.  The potential for policy dialogue between donor, project stakeholders and higher levels of government is significant, and initiatives need to be taken to engage in such dialogue.

The project's impact on women through the promotion of women's vegetable gardens has been small in numerical terms (only 3.2% of available funds spent), but significant in terms of depth.  This is an area of real and demonstrable importance to family life, nutrition, and women's empowerment, and should be given the importance it merits.

Sustainability of SCP II irrigation schemes depends particularly on careful site selection (especially in relation to markets and other irrigation abstractions); proper attention to social structures; respect and recognition of indigenous knowledge in the study and design process; and formal recognition of the need for post-construction support to irrigation communities.  There are weaknesses in all these areas, which threaten sustainability.  Two important physical threats to the irrigation schemes concern the damage caused by catchment soil erosion, and limited dry season water resources.  These both highlight the importance of an integrated approach to catchment planning and management which takes account of all relevant land and water uses and users.

The project is innovative in its particular combination of components and target farmers, but its replication in the same form will remain dependent on donor funds for the foreseeable future.  Some non-target farmers have been observed to imitate the technology which they see, but mostly using local materials in what amount to contemporary ‘traditional' schemes.

In many cases visited in this evaluation we have observed competition and conflict between upstream and downstream water users.  At least one region takes a whole-catchment approach to the study and design of potential new schemes, cataloguing existing abstractions and computing a water balance for the site under study.  This should be standard practice, and indeed Ethiopian law provides for the "Supervising body" to issue permits for water abstractions (not for traditional irrigation schemes, but presumably for ‘modern' schemes of the type included in SCP II).

The sense of ownership of irrigation schemes by the regions is an important plus-point in regard to sustained impact.  The fact that the regions take responsibility for post-construction maintenance and repair, when this lies beyond the capacity of the farmers, is a recognition of reality, and an important contributor to sustainability.

Performance of partners

Partner performance.  IFAD has been strong in terms of direct support to GoE, but weak in the management of supervision and technical assistance. 

Opportunities to initiate policy dialogue and donor coordination in some key areas relevant to SCP have not been exploited, although it is not too late to initiate this. ³

The Cooperating Institution (UNOPS) has performed conscientiously within severe time constraints.  The supervision process however has major shortcomings.  GoE has demonstrated strong commitment at the level of policy; less delivery in terms of ensuring adequate human resources at all times; poor monitoring of activities; and weak coordination among government stakeholders.

Overall assessment

 As would be expected in a project as complex as SCP II, the overall assessment is mixed, with some areas of significant strength and others of weaknesses which are great enough to undermine sustained impact.  Apart from those aspects already mentioned, the evaluation would particularly highlight as strengths of the project the high degree of commitment and the positive attitudes of many individuals at the level of the PCU, the Regional Programme Coordination Unit (RPCU), within the various regional government agencies involved, at woreda level, and among the many farming communities involved.  Significant weaknesses exist in the processes of disbursement of funds; in the degree of joint (consultant-GoE) participation possible within the processes of appraisal, MTR, technical assistance, supervision and evaluation; and in the priority given to non-engineering aspects of the project.  There are also several aspects of the original project design (especially its optimism about sustainability, crop yields, and the possibilities of commercialisation) which the present evaluation challenges.

Insights and recommendations

Part 1 - Extension of SCP II.  Expenditure of available funds, especially from the Irish grant and in the agriculture component, has been limited.  Monitoring of project achievements has been weak.  The project is due to be completed in July 2005, with a closing date of 31st December 2005.  It is unlikely that full expenditure and complete and accurate reporting can realistically take place by these dates.  An extension of these dates by at least 12 months is recommended.  If this is accepted, then a number of issues can be addressed even before taking any decision on future interventions.

Participative processes for the formulation of next phase.  We recommend that planning begin as a matter of urgency, to define a third phase of the SCP, in full consultation with those responsible for implementing Phase II.  The general and more detailed recommendations which follow Table 14 relate to the content of the formulation activity.  We urge that every attempt should be made in future project formulation to make project design a fully participative process.  The knowledge and experience which exists at federal and regional level of all aspects of project implementation should be utilised to the fullest extent possible.  We recommend that a joint team of SCP II personnel, Ethiopian consultants, international consultants and IFAD personnel be assembled for the purpose.  It is likely that formulation in this fashion will take longer than under present procedures, but the benefits in terms of realism of design and ownership of the outcome will be significant.

Formulation – producing limited but necessary paperwork.  Large quantities of detailed prescriptive documentation of variable quality and usefulness, simply gather dust and fails to fulfil a useful function.  We recommend the production of the minimum amount of paperwork, in formats which are agreed by all stakeholders to be necessary and useful for project management at various levels.  Concise formats such as logframes which fulfil multiple necessary functions are to be encouraged.

Formulation – produce a flexible project design and recognise the need for long-term programming. The project design should be sufficiently flexible to allow variation in approach from region to region, and evolution of approach over time, as better procedures are learnt by those implementing the project.  Any future project addressing food security in Ethiopia through a package of small-scale irrigation and agricultural support components should recognise the long-term nature of such an intervention.  The full adoption of the project by government, and the full realisation of the benefits by target groups of farmers may take as 10-12 years or more.  Continuity of effort is needed to achieve expected outcomes.

Formulation – producing procedures of sector-wide applicability.  Regions and woredas in food-insecure parts of Ethiopia have their own, and other donor-supported, programmes of assistance to small-scale irrigation.  Any future project focusing on SSI should endeavour to the greatest extent possible to integrate with regional, woreda and donor programmes, in order to simplify and strengthen programmes in this sector and move toward a sector wide approach.  Approaches vary, but the common goal of household and national food security is shared.  We recommend a joint donor-stakeholder forum to share experiences across donor programmes, broaden the menu of options with a view to the possible development of a common approach within the sector.

Part 2 - Insights – summary of conclusions from the Main Report.  Table ES1 sets out in summary form the main points made at the relevant places in the IE (2004) Main Report.

Table ES1 Interim Evaluation (2004) Insights

Project achievements are not adequately monitored.  Identifying and quantifying the project achievements has proved extremely difficult.  None of the stakeholders has accurate overall knowledge of project achievements to date.  This information is needed for effective project management by regions, PCU and IFAD.

Limited achievements in the agriculture component.  The agriculture component of the project is severely under-spent.  The limited activities in soil conservation, women's vegetable gardens, seed production, and extension services threaten to undermine impact on key target groups.

Little is known about traditional irrigation systems.  It appears that many assumptions have been made about the weaknesses of traditional irrigation systems, without the foundation of detailed investigation and diagnosis.  It may be that less capital-intensive interventions to improve traditional systems could have significant benefits, potentially spreading benefits more widely.

Market for vegetables is limited.  The assumption that there is a large accessible market for vegetables is questionable.  Physical access to markets is challenging.  Prices given by traders are very low.  Purchasing capacity of rural populations in food-insecure woredas are extremely limited.  High transaction costs limit the possibilities for exporting produce.  Producer prices can be very low, unless farmers sell directly into the market (without middle-men).

There is a great deal of institutional learning to share.  The experience gained by GoE and IFAD in SCP I and II puts both stakeholders in a very strong position to engage in dialogue over the outworking of policies in the areas of: food security, land and water management, coordination of Government agencies, cost recovery, and on-going support to communities.  Little has been done in this area to date.

The logframe is not used.  An agreed, detailed and up-to-date logical framework (logframe) is an extremely useful management tool.  SCP II's logframe is weak, incomplete, and not used.

Long term commitment is needed.  The impact of SCP II will only be fully realised if there is continuity of project activities over a minimum of 10-12 years.  A six year project is too short to achieve significant impacts.  It has taken until PY5 to reach a peak in irrigation scheme construction, and longer in the agriculture component.  Benefits to farmers will take another 6-10 years to realise.

Targeting at woreda level is good.  The project is well targeted at woreda level, with 70% of SCP II irrigation schemes being located in food-insecure woredas.

The appraisal assumptions were optimistic.  Many optimistic assumptions were made in the economic analysis of the project at appraisal.  In particular we highlight the high yields, high producer prices, low post-harvest losses, no water scarcity, and no maintenance costs assumed at appraisal.

Modern irrigation development is sometimes flawed.  Not all ‘modern' irrigation development has benefitted all of the target farmers.  Mistakes have been made in particular when engineers have ignored the knowledge or wishes of farmers, when hydrological assessments have been flawed, or where upstream developments have deprived schemes of water.

SCP II may have reduced grazing areas.  SSI development, combined with area enclosures and re-afforestation, may have reduced grazing areas and livestock numbers in some cases.  The impact of this on the environment, on financial assets, and on diet needs further investigation.

Insecurity of land tenure remains a matter of concern.  Insecurity of land tenure, both within SSI schemes and outside, continues to be a widespread problem, of perception, and in reality.

Financial assets are increasing.  Financial assets of irrigation farmers are rising, but slowly, because of the market problems raised earlier.

Access to credit is mixed.  Access to credit by SCP II farmers is mixed.  But low levels of usage of bought inputs (seed, fertiliser, pesticides, herbicides) limit the perception of this as a significant issue by farmers.

Extension services are of poor quality.  The quality of extension work is low, and specific SCP II interventions such as demonstration plots and trial sites have limited impact.

Social organisation for water management needs to be resolved.  Traditional water management organisations tend to be ignored in the establishment of ‘modern' WUAs and cooperatives.  This threatens the viability of the modern structures, and is disempowering.  The stakeholder charged with responsibility for strengthening WUAs is only interested in promoting cooperatives.  WUAs do not have legal status to enable them to operate a bank account and access credit.  Neither WUAs nor cooperatives fully represent the water users farming within irrigation command areas.

Limited impacts on women farmers are nevertheless encouraging.  Where the project has facilitated home agents at woreda level, and women within irrgation schemes, the initial results have been very encouraging.  Much more remains to be done in this key area of impact.

Attitudes to commercial farming are changing.  Irrigation farmer attitudes to commercialisation of crop production appear to be changing, and some of this change is attributable to the efforts of SCP II.  Whether these changes will persist in the face of marketing difficulties faced by farmers, remains to be seen.

Crop diversification is taking place; yields are mixed.  Crop diversification within SCP II schemes is occurring, but yields are variable from scheme to scheme.  Some vegetable yields are still well below those assumed at appraisal, even for year one of production.

Irrigation households are eating more vegetables.  Significant dietary intakes of vegetables appear to be taking place.

Soil erosion is a major environmental threat, and SCP II could have a major impact.  Soil erosion threatens the viability of both rainfed and irrigated farming.  SCP II includes a significant component of soil conservation work, but very little has been achieved so far.

Intensive multiple cropping in irrigation command areas will lead to soil degradation.  Without specific measures to manage soil fertility, such as rotation including legumes, and use of fertiliser and manure, soil nutrients will be rapidly depleted.

There is limited evidence of the beneficial effects of area enclosures on natural vegetation.

Government re-organisation and decentralisation have limited the impact of the project.  At woreda level, under-staffing, under-resourcing, and rapid turnover of staff are major issues.

Impact of the project on women is more likely to be achieved through targeted activities such as women's vegetable gardens, than through women's membership of WUA executives.

Vegetable cultivation increases labour requirements significantly.

The achievement of sustainability depends on site selection in relation to markets; establishing or strengthening sound social structures; study and design which takes account of local knowledge; and formal recognition of the need for post-construction support.

The project is innovative in its combination of irrigation, soil conservation, female-focused and institutional support activities.  At the present level of capital-intensity, it is not directly replicable without continuing donor support.  However, non-target farmers are already copying what they see, and developing new ‘traditional' irrigation systems.

Downstream developments compete for water with those upstream, and already there is significant competition and sometimes conflict over limited water resources.  We stress the importance of an approach based on integrated catchment planning, in order to limit and manage such conflicts.

The performance of IFAD and UNOPS has been limited by the shortcomings of brief foreign ‘expert' inputs which place more emphasis on outputs than on process.  The system prevents effective development of partnership, inter-dependence and joint ownership.

The performance of Government has been mixed.  Commitment at policy level has been high, while maintenance of staffing levels, monitoring and coordination have been weak

Performance of community social organisations has been limited by the confusion over traditional water management structures, WUAs and cooperatives.

Part 3 – Recommendations applicable to project implementation and content

Policy dialogue.  Areas in which the project's experience on the ground could make a valuable contribution to national policies and institutional frameworks include at least the following: water resource management at catchment level (including use of permits and the application of legislation such as Proclamations 92/1994 and 197/2000); adaptation of national water resource policies and legislation to regional level; marketing and price regulations (protecting farmers from unscrupulous merchants); policies on WUAs and so-called irrigation cooperatives; policies and practice relating to land title; and understandings and practices in relation to post-construction maintenance and rehabilitation.  The project could take a highly constructive lead in future in facilitating debate and movement on these issues, all of which strongly affect the impact and sustainability of small-scale irrigation schemes.

Institutional arrangements.  This project has experienced complex, and too frequently changing institutional arrangements.  In light of the many organisational changes which have taken place, it is remarkable what the project has been able to achieve – despite, rather than because of, the location of organisational authority and the linkages and coordination between stakeholders.  We now have major concerns about organisational changes in process at regional and federal level, and recommend that a careful and thorough internal review of the implications of these changes be set in train.  Ways need to be found to avoid the loss of institutional learning and experience built up now over many years, and incorporate it into future project implementation.

Social organisation.  The wide variety of approaches taken in this project toward traditional water management structures, WUAs and cooperatives speaks as much of the variety of perceptions of these organisations as of the site specific needs.  Within the project as a whole there is a great deal of confusion, created by lack of respect for farmers' traditional structures, the ‘modern' belief in a standardised WUA, and the dogmatic promotion of cooperatives.  We do not promote a single solution to this complex situation, but our recommendation is for regional and national debate and experience-sharing on the subject, and a high degree of flexibility in the solutions developed in different places and at different times.

Catchment planning and development.  It is essential that any individual irrigation scheme is appreciated in the context of the entire catchment in which it lies.  This is important from the point of view of water resource evaluation, of the assessment of soil and water conservation requirements and of the prevention and resolution of conflicts between user groups.  At least one of the SCP II regions sets its scheme study and design (feasibility study) process in the context of a database of water developments in the entire catchment.  This good practice should be extended to those which at present treat each scheme as an isolated entity.

Consolidation and component balance.  In the early days of water sector infrastructure projects, it is common for more emphasis to be placed on physical construction than on supportive actions to extend impact and ensure sustainability.  SCP II is no exception to this general rule.  It has been more convenient for funds to be focused on construction expenditure by the regional irrigation authorities than to disburse money to other stakeholders such as the Bureaux of Agriculture and Cooperatives, and the woredas.  We recommend that this imbalance be re-dressed in any third phase, with greater expenditure on agricultural support activities, soil conservation, women's gardens, and woreda level institutional support.  We also recommend consideration of a higher level of expenditure on market access roads.

Financial aspects.  The project can only move as fast as its cash flow.  Some regions find the revolving fund ceiling very low, especially during peak construction periods.  The system of settling accounts after disbursement is also found to be cumbersome, time consuming and inefficient.  Individual receipts have to be collected from remote woredas and carried to the regions, from which in turn they are taken by an accountant in person to Addis Ababa after consolidation.  Accountants who take receipts to regional offices for settlement go back to their centers many kilometers away to do their accounting work all over again in cases where errors are observed in filling forms.  Informants in the regions feel that the IFAD system has to be improved, perhaps in line with the simpler procedures of some other donors.  We recommend a detailed analysis of present procedures, with the aim of simplification.  The pool system of accounting at regional and woreda level creates unnecessary difficulties.  The new AfD system ⁴ has much to recommend it, and IFAD should explore this further.

Personnel and learning.  There is significant turnover of staff at regional and woreda levels, resulting in a need for frequent staff orientations.  Recognising this reality, the project should conduct orientation workshops for new staff, perhaps as often as every six months.  More generally, there is great value in shared learning, such as that which took place at the August 2004 workshop in Adama (Nazaret).  Such workshops should become regular annual events.  Further learning at regional and national levels could be brought about by the establishment of policy fora in which key issues of policy and strategy could be discussed. 

Scheme audit.  Few consolidated data exist on the SCP II schemes constructed to date, and even less on the phase I schemes.  All regions made an undertaking at the Adama workshop to compile profiles for their SCP II schemes.  One region (Amhara) already undertakes a biennial review of a sample of schemes (non-IFAD) under its care.  We recommend that all IFAD Phase I and II schemes be properly catalogued, and that these profiles be regularly updated.  If such an exercise could be extended to non-IFAD schemes, to provide regional databases, this would greatly extend the baseline and monitoring data on which future decision-making rests.

Participative planning process.  The criteria by which scheme locations are selected are not fully clear, and not necessarily the best to ensure impact and sustainability.  We recommend the establishment of a participative process, involving all relevant Regional and woreda level stakeholders, to develop clear scheme selection criteria based on need, institutional capacity, and likely viability.  Woredas appear to have little or no involvement in scheme study and design.  We therefore further recommend the full involvement of woreda personnel in the study and design process, with more flexibility than at present in the balance of project components at any particular site.  Farmer involvement in scheme planning and design is still limited, and so we press for the adoption of more fully participative processes of planning and design, in which all professional disciplines are trained and to which they are committed.  One thrust of such approaches could focus on minor (low-cost) improvements to traditional irrigation which are able to significantly improve performance for water users.

Construction, maintenance and rehabilitation.  We are aware of several cases in which initial construction budgets have been insufficient to complete schemes.  In most regions, scheme repairs and maintenance (which are beyond the capacity of farmers, but are relatively minor for the irrigation authorities) are carried out routinely by the regional authorities.  In other cases, true rehabilitation (major reconstruction/repair, often with social re-organisation) is carried out.  In all these instances, funds may be obtained from SCP II surplus construction funds, from non-IFAD regional sources, from NGOs, or from funds specifically designated for "rehabilitation".  While this situation demonstrates the commitment and the flexibility of the regional authorities, it gives rise to two areas of confusion: first, in establishing what is the true investment cost in a particular scheme; and second, in classifying as "rehabilitation" activities which really constitute minor maintenance.  In particular we urge the realistic recognition by both IFAD and regional authorities that regular (annual) minor maintenance is needed, and that this should have clear and transparent planning procedures and an adequate (and increasing) budget line.  Long-term support is a necessity, not an option.

Reporting, M&E, information flow and documentation.  Quarterly reports from the regions are not presented in a consistent manner, and lack rigorous analysis and reflection.  In order to ensure transparent information flows, the reporting structure and content require review.  A number of implementation documents exist (including Appraisal, Project Implementation Manual, Operating Manual, Financial Manual, Supervision Reports, Mid-Term Review).  Given the sheer volume of material, it is unsurprising if these documents are not fully internalised or extensively used for project management.  This problem is exacerbated by staff turnover.  We recommend that the Project Coordinator and Regional Coordinators meet to design a simple progress reporting structure, especially to fulfil the requirements for imminent end-of-project reporting.

---

1/ The evaluation was led by Professor Richard Carter (Water Sector Specialist, Cranfield University, UK), with Ato Ayele Gebre-Mariam (Socio-economist), Dr Kerstin Danert (Independent Water Sector Researcher and Consultant, Uganda), Dr Tilahun Amede (Consultant Agronomist, Ethiopia) and Ato Merkorewos Hiwet (Consultant, Marketing and Economics, Ethiopia). Mr Fabrizio Felloni, Lead Evaluator, designed the evaluation methodology, conducted a preparatory mission in May 2004 and accompanied the evaluation team during its first and last week.

2/ In the main text we argue that this is a narrow definition in view of more recent findings on nutrition security.

3/ On the matter of policy dialogue, IFAD has been engaged in the preparation of two recent programmes in Ethiopia: the Agricultural Marketing Improvement Programme, and the Rural Finance Intermediation Programme.  These areas do not fully coincide with those referred to in this paragraph but are relevant to rural poverty alleviation.

4/ Which uses payment certificates rather than receipts.