Interim evaluation¹

The Special Programme for Soil and Water Conservation and Agroforestry in the Central Plateau of Burkina Faso commenced in October 1988 as one of the most representative and most ambitious investment projects in IFAD's Special Programme for Africa. Phase 1 covered four provinces (Passoré, Yatenga, Bam and Sanmatenga). A second phase was approved in December 1994 and began in May 1996, expanding the programme area to Boulkiemdé, Sanguié and Namentanga provinces.

The total cost of the programme over its 15-year implementation period is USD 38.1 million, of which USD 26.7 million represents IFAD lending (SRS 011-BF in Phase 1, 80% disbursed; and SRS 044-BF and 0369-BF in Phase 2, 99% disbursed). The second phase was cofinanced by the West African Development Bank (BOAD) in the amount of USD 1.5 million. BOAD also provided programme supervision. Projected completion for Phase 2 is the end of June 2003, following a one-year extension.

In view of the foregoing, and given the possibility of further action in the area, IFAD's Office of Evaluation was asked to conduct an interim evaluation of the programme². The mission's objective was to assess, in a spirit of collaboration and partnership, the relevance of the actions and approaches taken, measure their impact and any changes made, and draw lessons for the future as to what ought to be pursued, ended or improved.

The interim evaluation focused on three major thrusts of programme intervention: (i) conserving productive resources and securing agricultural production; (ii) rural financing (credits and subsidies for income-generating activities and equipment); and (iii) building local capacity.

The mission worked in the programme area from 24 January to 20 February 2003. The mission members were three consultants and three officials from the regulatory authority. They began with a facilitated self-evaluation by the programme team, visits to the field, meetings with stakeholders and others, and a workshop in Yako to allow for reflection and an exchange of views on the mission's preliminary findings.

Programme context and design

The central plateau covers one quarter of Burkina Faso's surface area and holds 43% of its population. With GNP per capita of less than USD 240, Burkina Faso belongs to the group of least developed countries. Agriculture is the source of livelihood for 92% of the population. The central plateau has been and continues to be one of the most disadvantaged regions of the country owing to its poor soil resources (exacerbated by serious erosion) and high population density (from 70 inhabitants/km² to more than 100 in some provinces).

The programme's overall objectives were, on the one hand, conserving natural resources and, on the other, effecting lasting improvements in production, income and living standards for the population concerned. The target group was composed mainly of rural households farming on less than three hectares of land, or approximately 40 000 smallholdings (20% of them headed by women), representing a population estimated at between 320 000 and 400 000 people living in the 27 departments selected within the seven provinces of the programme area. The programme was to benefit more than 4 000 young people in addition to 1 000 women through its women's advancement and rural credit components.

The main quantitative objectives of Phase 1 were soil and water conservation (SWC) treatment of 28 000 ha of collectively farmed enclosed fields and 10 250 ha of individually farmed outfield plots. The creation of 160 nurseries under the agroforestry (AGF) component was to reach at least 10% of the area covered by SWC measures, i.e. 2 800 hectares. Twenty per cent of these same areas were to undergo crop intensification using various technology packages; 2 643 tonnes of Burkina phosphate and 470 tonnes of NPK fertilizer were to be distributed under this exercise. The research and development component was to address the need for innovation in the spheres of crop intensification and agroforestry, with pre-determined subject matter. A village development fund was to promote the mainstreaming of women in the development process through local savings and credit associations, as well as finance other agriculture-related activities to generate income for the rest of the target population.

Phase 2 called for SWC treatment of 27 500 ha of collectively-farmed land and 30 000 ha of individually-farmed land, as well as the construction of 780 irrigation dikes. Sixty new nurseries were to supply plants for AGF activities on 40 000 ha. Crop intensification activities were intended to restore soil fertility, promote better soil use and integrate crop and livestock activities. Twenty-four groupings of livestock breeders (from eight villages) were to conduct a pilot operation in semi-stabling and rangeland management. The promotion of women's activities, working through 300 women's groups, called for the provision of 900 wheelbarrows, 50 grain mills, the introduction of improved cookstoves, literacy activities and support for income-generating activities by means of microcredit. Training was to be provided to 27 000 people in an effort to build the capacity of farmers' organizations. Agricultural credit was to be used to promote the use of farm equipment, providing partial guarantees to enable the financial institution, the National Agricultural Credit Bank (CNCA), to finance 1 500 carts. An improvement in the institutional capacity of partner service agencies was included but not precisely quantified. Finally, Phase 2 called for the construction or rehabilitation of 180 wells, accompanied by orientation on hygiene and infrastructure management and maintenance.

Implementation and subsequent developments

The organization and management strategy for Phase 1 was based on massive voluntary participation in carrying out the work by the beneficiaries on a contractual basis, in order to: (i) minimize recurring costs; (ii) maximize the use of local resources and competencies; and (iii) lead to self-management by the programme beneficiaries in the medium term. The programme was to be carried out in collaboration with government institutions (CNCA, provincial agricultural research and producer support services). Phase 2 continued on the same basis. The project management unit provided overall direction and financial back-up through technical, administrative and financial management for the programme.

Burkina Faso has undergone far-reaching institutional changes since 1987 as a result of: (i) two major reforms in the Ministry of Agriculture; (ii) implementation in 1992 of an agricultural sector adjustment policy that placed a hiring freeze on all public technical services; (iii) passage of Law 14/99 setting up a procedure to establish farmers' organizations by subsector; (iv) creation by interministerial decree in 2000 of the village land management committees, introducing an important new actor in the process of decentralization, local development and local land management; (v) creation of the province of Zondoma within Yatenga Province (Law 9/96/ADP of 24 April 1996); and (vi) in the economic sphere, the devaluation of the CFA franc in 1994.

The programme design has changed very little overall over the course of the two phases, with both the major objectives and the modus operandi remaining the same since 1987.

Principal results

The seven provinces within the programme area contain 80 departments and 1 710 villages. By the end of the second phase, the programme had been active in 27 departments and a total of 459 villages (75% of the villages in the departments selected and 27% of all the villages in the seven provinces).

Conserving productive resources and securing agricultural production

In the area of soil and water conservation (SWC), programme documentation indicates that, since 1987, activities involving stone bunding have been carried out on close to 89 600 ha (60% collectively-farmed land and 40% individually-farmed land), an area approximately equal to the objective set. In addition, the programme has built 748 irrigation dikes, improved the traditional zaï system on 32 500 ha, and introduced demi-lune conservation structures on 324 ha. The mission cannot however judge the reliability of the database. The plans for the treated land are not geo-referenced to the site level. The condition and use of land prior to treatment are not specified, and the translation to "treated area" of linear structures such as stone bunding installed on unsurveyed land is not easily verifiable. Still, the technical quality of what has been achieved appears to be satisfactory overall. The implementation process was based on public technical services such as the regional departments of agriculture, water and wind resources (DRAHRH), the programme's outreach teams and village associations. Implementation remained linear through both phases, with no real increase in responsibility on the part of villagers (farmers carrying out the work). In latter years this approach felt the effects of staff cuts at the regional departments (50% of field staff on average as of this writing). The average cost of one hectare receiving SWC treatment ranges from USD 82 to USD 302³ depending on the method of calculation.

In the area of agroforestry (AGF) over the same period, the results posted by the programme are as follows: production of 2.6 million plants (90% of the target), installation of 182 nurseries (107%), assisted natural regeneration (ANR) on 17 058 ha (79%), organization of 45 groups of livestock breeders in 15 villages, implementation of a pilot livestock operation in semi-stabling as well as plantings of ligneous and herbaceous vegetation along stone bunding. Technically speaking, some of these results appear promising and of interest, especially the ANR procedure and the pilot livestock project. On the other hand, the approach taken is not sustainable for the planting and nursery component (a collective, non-economic approach) and not well adapted (the final ownership of the plantations and profits generated by them is not clear to farmers). Stakeholder involvement in plantations is also observed to be low. The survival rate after three years is not encouraging (around 30%) compared to the amount of effort and cost involved.

In the area of crop intensification, since 1987 the programme has installed 29 000 compost pits (112%), distributed 2 936 tons of Burkina phosphate and trained 21 producers of improved seed for total yield of 13.1 tonnes (maize, millet and sorghum).The composting technique was not appropriate to the constraints and capacities of the rural environment in terms of the availability of water and raw materials and the difficulty of work, and compost production volumes are far from proportional to the number of pits dug. In addition, the approach used to distribute the improved seed does not allow for determining the dissemination rate in the rural environment or ensuring the permanence of the system.

In the area of research and development, the national research institute INERA has produced 16 technical fiches on SWC, AGF and crop intensification that describe the various techniques and quantify the results obtained, but the economic and social data remain to be developed. Certain assumptions need to be better adapted to the farmers' logic, strategy and resources (as in the case of compost production) if a more satisfactory acceptance rate is to be achieved.

In the area of village water management, the programme has installed or rehabilitated 201 functional wells using an efficient procedure; 60 new wells are to be built under a contract that is to be executed by programme completion. The benefits of this type of intervention are well known and need not be demonstrated.

Financing of rural activities

Medium-term equipment credits. Overall, 1 036 credits have been extended at a rate of 9% per annum, exclusively for the purchase of carts, totalling 194 million CFA francs (two thirds of the target amount): 351 village associations have benefited from this operation in eight provinces, and approximately 16% of the credits were made to women. The repayment rate is reported to be 87%. Difficulties have arisen in the partnership with the Agricultural and Commercial Bank (BACB, formerly CNCA), which has in some cases taken a very long time to review applications and issue financing (up to 13 months); nor has it fulfilled its commitments in terms of credit recoveries since 2001.

Short-term credits for income-generating activities. A total of 1.09 billion CFA francs in credits has been granted, 64% to women. These are mainly small business credits (87%) and start-up credits (9%). A total of 292 local savings and loan organizations have been created and/or assisted (with a total of 8 338 members, of whom 76% are women), including 249 village banks belonging to the national federation of credit unions (RCPB). Half of the villages in the programme area have benefited from the establishment of these 292 local savings and loan organizations, but not all of them were established by the programme (some pre-existed it). The penetration rate of these organizations remains quite low (on the order of 4%).

Subsidies for women's advancement and soil and water conservation activities (subsidies for equipment). A total of 688 carts and 1 197 wheelbarrows have been distributed through partial or full subsidies. Although the wheelbarrows were originally intended exclusively for women, the carts were to be distributed to both men and women. In fact, they were distributed exclusively to women as well. Virtually no equipment has been used for SWC activities as planned. In almost all cases, it has been used for hire.

Building local capacity

The pilot approach to land management that was implemented in 8 of the 15 villages targeted, led - following a participatory, informational and educational procedure - to the elaboration of village development plans. Despite the request made at the time of the mid-term review in 1999, no funds were made available to implement the village development plans. The programme did train leaders in fund-raising, with the expected difficulties in making use of such training (remote locations, illiteracy, lack of negotiating experience). Therefore it is difficult, even after seven years, to reach any conclusion on this component, except for the preparatory phase which, although long (2.5 years), seems to have been assimilated by the villagers.

Support for farmers' organizations. The main results have been two identification surveys conducted (1 264 village associations selected), a typology of the latter (93% are non-functional, i.e. not engaged in any real activity), training of 21 684 producers, including 7 316 women, in matters such as cooperative management, management of collective equipment and economic units, planning and programming of activities, mobilizing local resources, and so on. In addition, 362 officers from partner services (DPA) have received similar training, and assistance was provided for the restructuring of farmers' organizations and their legal recognition in accordance with Law 14/99.

Training. The programme, in Phase 2, provided 264 000 person-days of training on 37 subjects to producers, support services and PMU staff, including 30 000 person-days on crop intensification and techniques and more than 21 000 person-days on the operation of farmers' organizations. Specialized training was provided to 324 professional herders, 256 nursery workers and 19 well-repair workers; 8 265 women were trained in improved cookstove techniques and 1 129 women received literacy training.

Institution-building. The principal results are as follows: (i) the establishment of 17 private and public-sector agreements and seven outreach teams in Phases 1 and 2; and (ii) elimination of annual contracts with villages; (iii) active participation in the implementation of eight technical provincial consultation meetings and the holding of seven annual review and programming workshops with all programme stakeholders.

Impact, relevance and efficiency of the programme

The evaluation confirmed that the programme had many positive effects. Among the main ones are an improvement in productive capital (land) and crop production: SWC interventions on some 90 000 ha⁴ (including 5 000 ha of reclaimed formerly non-productive land) and crop intensification measures led to an average increase in cereal yields on the order of 25% (from 400 to 500 kg/ha on average) on 20 to 30% of cultivated land in 459 villages. An estimated 50% of the land cultivated by the 34 500 households and association members covered by the programme received treatment, for an average of 2 ha per household. In addition, about 5 000 households applied treatment to an additional 20 000 ha of ‘non-programme' land, following the example of neighbouring fields. Overall, an estimated one half of all households in the villages within the programme area have benefited either directly or indirectly. In the programme area as a whole, an estimated 7 000 to 9 000 tonnes of millet and sorghum have been added to cereal production thanks to the programme. Many farms have improved their equipment as well (in particular hand tools and carts).

Access to local credit for the 8 300 members of local savings and credit associations has led to a significant rise in income generated by small business and livestock activities. Overall, an estimated 815 million CFA francs in profits have been generated thanks to microcredits granted.

Food security in households having improved their land resources increased, benefiting at least 350 000 people. Average annual cereal production per farm reportedly increased from 1 600 to 1 800 kg of millet/sorghum, i.e. from 80% to 90% coverage of annual cereal needs for an average household (estimated at 2 tonnes per year for a household of 10). These average figures naturally break down quite differently in actual households. In particular, the technical solutions put forward by the programme (stone bunding, traditional zaï techniques, organic compost) are all very labour-intensive. Those households with a high ratio of consumers to assets and unable to obtain outside labour undoubtedly remain at a disadvantage.

The effects of SWC interventions on agricultural production are much more noticeable in years of low rainfall, since they reduce vulnerability to climatic fluctuations. Before attributing the general improvement in food production and food security in the area concerned to the programme, we should take into account the fact that average rainfall has been higher in recent years than during the initial programme.

Some 233 000 people in 204 villages have benefited from improved access to drinking water and therefore improved health conditions (especially for children) thanks to the installation of 261 wells. Women have clearly benefited from this village water component as well as other measures taken to relieve their workload (transportation equipment in particular) and enable them to engage in revenue-generating activities. Their position within village society has improved thanks to their economic weight (they generally represent a healthy majority in local savings and credit organizations) and the new knowledge they have acquired.

However, the expected impact from building social capital and local and collective capacities did not materialize in proportion to the investment made. With respect to SWC activities, the programme implementation strategy essentially depended on state technical services and outreach teams, with villagers taking on little responsibility and contributing mainly manual labour. The changes in public services in the past 10 years (SAP of 1992) at the regional and provincial levels clearly make this an outdated approach. The programme was unable to prepare villages and village land development committees to take on responsibility for managing their own land resources, nor did it stimulate the emergence of private operators (with the exception of some technical training for nursery operators and well-repair workers). This was clearly a major error in design and implementation of the second phase of the programme, and had an adverse impact on prospects for a sustainable development process.

The programme's contribution to certain positive changes – such as the overall improvement in plant cover, a slowing in the rural exodus and the sharp decrease over the last 20 years in the proportion of households living below the poverty line – is difficult to assess with any certainty. Many exogenous factors, such as changes in rainfall, the rise in gold washing activity, and opportunities for emigration and employment in Côte d'Ivoire, have all played an important part. Also, although 35 000 to 40 000 farms have benefited from the programme interventions, the proportion of the target group reached, i.e. the poorest and most disadvantaged households, cannot be determined using the approaches applied.

The programme has made it possible to validate or confirm certain techniques in SWC, agroforestry (ANR), livestock raising, village water management and microcredit, which can today be replicated on a larger scale. On the other hand, little progress was made in the areas of fertilization, crop intensification and reforestation, all of which are essential in a context of crop settling. The research and development component did not contribute any major innovations to address farmers' expectations but did however refine already existing techniques and verify their quantitative results. Little progress has been made on evaluating the economic and financial returns on these techniques or analysing their relevance to different types of farming based on the availability of labour, conditions of access to land, and so on.

The programme has once more illustrated the importance and potential of local credit and the excellent response of populations in using such credit to engage in income-generating activities (averaging 75% net profits after repayment). What is needed at this point is an expansion and consolidation of rural financial services networks, and an increase in the still-low penetration rate. Also, institution-building through the components of support for farmers' organizations, village land management committees and public technical services did not meet expectations and will require an in-depth revision of approach. The local land management (gestion de terroirs) approach was not effective either in putting in place overall management plans contractually at the village level or in reflecting on and implementing rules for managing natural resources within communities.

The assessment of the programme results in the main domains of impact is summarized in the table below:

Domains of impact	Rating of Impact
1. Physical and financial Assets (principally land and agricultural equipment)	3
2. Human Assets (principally improvement in access to potable water and training)	2
3. Social Capital and People's Empowerment (strengthening of groups and management unit)	2
4. Food and economic security (reduction of food-deficit and food risk)	3
5. Environment (reduction of erosion and retention of waters)	3
6. Institutions, policies and regulatory framework	1

(4= high; 3= substantial ; 2= modest ; 1= négligible)

Lessons learned

From the point of view of methodology, the approach planned at the outset was pertinent (a contractual transfer of responsibility in response to organized village demand through permanent local providers) but was not implemented in the spirit of the appraisal report. A systematic approach of working with groups was not the most appropriate in certain areas (crop intensification, reforestation and agroforestry) and was not able to provide assurances of reaching the target population (only 30% of the population was represented by these village associations). The SWC and AGF approaches suffered from a lack of reflection and support for implementation strategies in order to achieve goals, as well as conditions for sustainability of the dynamics used. The programme concentrated its attention and energy on physical achievement of quantitative objectives and paid only limited attention to social and institutional conditions and to ensuring the permanence of the development process. What is needed is a better definition of the development objectives of programmes, based on a more accurate and more qualitatively-oriented logical framework and –once again – putting in place monitoring and evaluation mechanisms that can (assuming adequate resources) not only monitor compliance but also evaluate the timeliness of actions and relevance of methods with respect to the strategic objectives and the philosophy of the contractual documents. Also needed is better capacity on the part of decision-makers (project management, oversight ministry and providers of funding) to fine-tune a programme in the course of execution. Certain pertinent recommendations were made by external evaluations in 1993, and then again in 1997, but were never taken into account. In this essential area of ‘strategic direction' and evaluation in the course of implementation, supervision by the cooperating institution has once again shown its limitations.

The institutional arrangements and partnerships have been shown to be difficult to apply at times in a constantly changing environment. Having virtually all of the programme contingent on public technical services very soon showed itself to be an ephemeral approach. It would be advisable for future plans to leave open the possibility of distributing tasks differently (advisory assistance, training, implementation, monitoring) according to the type of provider (public, private) and not to hesitate to favour the emergence of private operators if the opportunity arises. An approach that provides for flexible assistance that is gradually phased out should be planned from the outset. A prior audit of institutions operating in the field should be carried out before any project is identified, to reach a better understanding of the competencies, resources and capacities of local operators as well as their objectives, strategies and costs as providers.

The experience of this programme also shows the importance of having a regional and local strategy based on a technical targeting process. It is not possible today to determine the significance of the 90 000 ha of treated land vis-à-vis the size of the problem in the area, or where emergency regions remain to be dealt with, or where there is land with good potential for supplying stone. A spatial targeting strategy based on constraints and potential does not rule out a sound partnership that can be formalized in contracts with the villages concerned. The overall watershed approach warrants a review to better build in protection for surface water plans (e.g. silting of the Bam, Yako and Ouahigouya reservoirs). Also, it is difficult to evaluate actions already concluded and programme investments for the future on a large scale unless it is possible to determine and measure, on a map and on each plot of land, the baseline situation, programme achievements, and what remains to be done. Simple ways of achieving this by using survey photographs and GPS could provide ad hoc support for this type of action.

The programme evaluation reveals the need to move systematically toward an economic, financial and commercial approach to the proposed actions. It is important that each participant be able to place a cost on investments and activities and measure their potential returns, whether with respect to soil and water conservation, compost production, improved seed or reforestation activities. It is quite regrettable that after 15 years of interventions the economic and financial returns on the proposed investments are not better defined, since the ‘transferability' of the functions and estimates of necessary and sufficient public subsidy levels depend on such a definition. Programme management, as well as partners providing support and investigators, must adopt a new attitude in this sense. The ‘beneficiaries' already possess this mentality and express it more or less intuitively through the intensity of their response to the actions proposed.

The evaluation has once again demonstrated the need for reliable baseline studies and a better knowledge of the constraints but also the strategies and practices of the target population. This information, coupled with a pragmatism that is currently lacking in the research, would result in proposals of actions more in tune with the environment and facilitate their adoption by the target group.

It is important to maintain and strengthen consultative agreements between projects and between public actors and civil society (farmers' organizations, NGOs, etc.) to ensure harmonization of interventions in the same villages in related subject areas (and/or distribute them better geographically).

This programme also shows the importance of formalizing the programme commitments with the target population, and the importance of having local stakeholders gradually take on maximum responsibility (for logistical, administrative, and then financial aspects) and phasing out external assistance over time. In parallel, staffing ratios should be reviewed and a strategy adopted to gradually increase the number of villages involved.

Addressing the issues relating to soil reclamation and conservation in the central plateau is a race against time. In 15 years, the population has grown from 2 to 3 million inhabitants. It has not been possible over the same period of time to treat more than 20% to 30% of the surface area, and the process will tail off significantly during the post-project period. The lessons learned from this experience should enable programmes to be developed to accelerate the dynamics of development activities built much more upon sustainable local initiatives (that generate income if possible) that take on a management role and depend less on project institutions. Much is at stake. The central plateau now holds more than half the country's population.

Recommendations

Based on the lessons learned over the course of this programme and the size of the environmental, agricultural and human challenge at hand, the interim evaluation mission recommends the following:

• Preparation of a new sector intervention in the area of agricultural land management issues (SWC, AGF, surface water and small-scale irrigation) and crop settling (fertility management), following prior conduct of in-depth studies which are indispensable for any new project, namely: (i) a zoning study to measure the development of and current trends in water and wind erosion in the central plateau; (ii) a study of the situation and capacity of current local operators and providers; and (iii) implementation of pilot mini-operations to determine the feasibility of new rural activities (groups of young people engaged in land conservation, for example).

• Consideration of the lessons learned during this programme in order to commit to a fundamentally different approach, establishing from the outset an evolving strategy based on local actors (farmers' organizations, SMEs and local pieceworkers, village land development committees) and a planned phase-out of the project structure. Under this approach, the project could call for a support fund for agricultural land development to be set up as a starting point, to be managed directly by village land management committees and/or agencies and accompanied by capacity-building and research-action measures.

Establishment of a programme to develop financial services (microcredit and others) within the strategy of local financial institutions but with more ambitious, better distributed penetration rates (just 4% currently after 15 years of intervention).

---

1. The interim evaluation mission was made up of Mr Hubert Boirard, mission leader and rural institutions expert; Mr Bertrand Guibert, agricultural engineer, Ms Corinne Riquet, economist specialized in microfinance, and three officials from the line ministries.

2. The programme had been the subject of a mid-term evaluation during the first phase (IFAD, 1993), a case study as part of the evaluation of the Special Programme for Africa (IFAD, 1997) and a study of the impact of the SWC, agroforestry and crop intensification components during the second phase (IFAD, 2001).

3. Based on a rate of 65O CFA francs per USD.

4. The evaluation mission was not able to verify the total land surface improved under the programme. The positive impact on beneficiary villages and households is clear, but only a large-scale survey could confirm the actual extent of the impact at the overall programme level.