Project interim evaluation¹

Introduction

The agreement concluding the PDAOK (Ouadis of Kanem Agricultural Development Project) interim evaluation comprises two parts. The first part addresses short-tem actions that will be taken after the evaluation validation workshop (25/26th February 2002). The actions will ensure completion of activities from the social and health component; provide support for the self-administered local credit unions (CLA) and repayments of current loans; and ensure transition to the new project within the specified design process schedule. This agreement was concluded on 27th February in Mao, the main town in the Kanem region, between the members of the expanded core learning partnership (PEC).

The second part of the agreement presents the major lessons learned from the PDAOK experience as well as recommendations for the new project now being prepared. It was drawn up by the IFAD evaluation officer after completion of the evaluation report and was distributed to the members of the CLP for approval in November 2002.

The two parts are complementary and are based on unanimous agreement by the independent evaluation mission, the CLP and all of the participants in the Mao evaluation workshop. The PDAOK results and impact on the date of the evaluation are clearly lower than expected and the conditions for sustainability of the achievements have not been met in almost all of the intervention areas. However, the investments made and a number of positive effects are relevant in view of a starting point characterized by the extreme vulnerability of the population. A new rural development project in Kanem, supported by IFAD, is desirable and should be presented to the IFAD Executive Board in April 2003. The main reasons for this recommendation are:

• The PDAOK allowed the emergence of many grassroots groups providing important functions for the local inhabitants and for women in particular. A social transformation process favorable to economic development is under way. However, these groups are still very fragile and the services to which they had access have no prospect of sustainability without a new development and institutional consolidation project.

• The local credit unions, still at a very early phase, offer a viable outlook for the development of microfinance institutions in the region. Demand is great and is already partially organized and familiar with credit management. A competent national NGO, SECADEV, is prepared to provide support to the credit unions. This new outlook for the region can only be realized with new IFAD funding for a sufficient period of time.

• From the perspective of water lifting technologies for ouadi cultivation, there are now several options (PVC boreholes, single tilting can, motor pumps) but there is still a great deal to be done upstream and downstream of production to ensure profitability. The private suppliers of maintenance and equipment need to be reinforced. Partial investment subsidies will certainly still be needed for the poorest to have access to these innovations.

• The Government decided to maintain ONDR presence in the region and to refocus its functions on agricultural and rural advisory services, withdrawing from credit and procurement. This service is indispensable and should be accompanied by the capacity for research and innovation. In the near future, the Government will probably not be able to ensure quality agricultural services without external funding. Ultimately, it can be expected that the financial resources from oil exploitation could open new doors to sustainability.

Given that IFAD, and then the Belgian Survival Fund, decided to support the Government of Chad in this region in 1994, it seems that the potential for success is even greater today. The human resource base and social capital have both evolved for the better in the villages and lessons have been learned. The improvement in project performance shows that implementation problems are not insurmountable as long as peace and the rule of law exists regionally and nationally and responsibilities are clearly delineated. However, the new project should not be a mere continuation or expansion of the PDAOK. Changes are needed in approach and institutional arrangements. Therefore, the PDAOK should be closed in June 2002 and a transition phase organized with a view to future prospects.

Part One: Short-term recommendations
(Record of the meeting held in Mao on 27th February 2002, following the evaluation validation workshop)

Following the PDAOK interim evaluation validation workshop, a meeting was held on 27th February 2002² on the PDAOK premises in Mao to decide on the mission recommendations about the short term, particularly steps to be taken over the coming months. The participants decided by mutual agreement on the following:

Schedule for preparing the new project

• Formulation mission in April 2002;
• Pre-evaluation mission in August 2002;
• Presentation to IFAD Executive Board in April 2003

If the schedule can be met, and given the amount of time required to process loans, it can be expected that the new project will begin in the first half of 2004.

Programme of the social and health component from March to June 2002

The programme of work and budget proposed by the validation workshop is approved, with the understanding that the activities in the "health" sub-component have priority (completion of centres under construction, training of traditional birth attendants and health promoters).

The project management unit (PMU) and IFAD agree to verify quickly with UNOPS the availability of funds needed to finance the activities proposed by the "literacy" and "nutritional education" sub-components.

Programme of support for self-administered local credit unions (CLAs) and issues related to credit

Support for the CLAs and use of collections from credits outstanding and in arrears

• Their share of the fund will be returned to the 4 existing CLAs, "GDF Interest", once the agencies have received the necessary training and tools for proper management of their funds. The procedures for returning these funds will be defined before the loan is closed.

• The CLA offices will collect credit that is outstanding - and in arrears from the member economic interest groups of the 4 existing CLAs with support from SECADEV. These funds will remain the property of the CLAs.

• The economic interest groups that are not members of the 4 existing CLAs will be supported for collection of their credits outstanding and in arrears by the Kanem ONDR district. The amounts recovered will be credited to the "GDF Interest" accounts on behalf of the economic interest groups in question. They will be returned as equity capital once these economic interest groups form CLAs. The Directorate General of Agriculture announces its intention to put in place quickly the ONDR staff needed to cover the entire district.

• The Kanem ONDR district and SECADEV will work together so that the transition process towards formation of new CLAs proceeds under the best conditions.

Continuity of support for the CLAs between 30 June 2002 and the start of the new project

• The ongoing presence of the operator to support the local credit unions must be ensured between 30 June 2002 and the start of the new project in 2004;

• Therefore, IFAD will examine the option of mobilizing the necessary funding for year 2003 in its scheduled grants to NGOs as this type of funding cannot be arranged for the second half of 2002;

• For the second half of 2002, the Government and IFAD will examine the possibility of funding from a new agreement with SECADEV for its intervention in Kanem from "PSANG 2" loan funds;

• SECADEV and the PMU agree to prepare the outlines of a draft agreement that could commit SECADEV and PSANG 2 for the second half of 2002. This draft should be available before 23 March 2002, date of the PSANG 2 startup seminar and of preparation of its first AWP/B;

• If these steps are not sufficient to ensure the ongoing presence of the operator in Kanem until the new project starts, the Government may use available funds from the "GDF Capital" account theoretically intended for future refinancing of the CLAs.

PDAOK / SECADEV Agreement: After careful examination of the various articles in the agreement, a new version was approved by the stakeholders and will be signed on Friday, 1 March 2002. The direct payment request will be sent to UNOPS by DHL once the bank guarantee to be provided by SECADEV has been secured, and before 8 March 2002, if possible. While awaiting receipt of this direct payment, the PMU agrees to advance the necessary funds to start the operator's activities under the schedule.

Part Two: Main Lessons Learned from the PDAOK Experience and Recommendations for a New Project

Conclusions and lessons learned from the evaluation

The overall impact of the PDAOK was lower than expected, especially in increasing irrigated agricultural production, where expectations were over optimistic given available technologies. The impact on health and nutrition is still emerging from project work still in progress. Yet, changes due to the project are significant compared to the original situation. The creation of small functional economic interest groups based on affinities and common interests, especially women's groups, is a fundamental, though limited, achievement. The experience of credit for small businesses, (non-nomadic) small livestock and seed banks showed potential. The conditions for improving the health status of villagers have now been met (through wells, health centres, and nutritional education). The most successful actions were those taken with, or specifically targeting, rural women.

Despite this initial positive impact, the conditions for sustainability were not met in nearly all intervention domains. The extreme vulnerability of the rural populations, the weakness and often the absence of essential public and private services called for and still call for a much longer-term effort in developing the institutions and services concerned, especially community and private services.

In addition to the issue of the length of the project, which was clearly inadequate for PDAOK's ambitions, there were mistakes in design and drifts in implementation. The most striking example of this is credit. The original design addressed the concern for sustainability of financial services but the proposed scenario included technical and social flaws and, above all, the development of a microfinance institution was not presented as a development objective. No specialized operator was planned for that purpose. The GDFs were mobilized and the loans were granted under the "agricultural extension" component to serve agricultural development and environmental protection. Credit became merely a way to solve the immediate problems without worrying about institutional sustainability. The ONDR agents devoted most of their time to administering this system to the detriment of agricultural advisory services and training. Within a context of huge poverty there is strong pressure for immediate results. The idea of creating a viable financial institution, with all the implications of prior training and progress in granting loans, took a back seat.

The drifts in implementation, to the detriment of long-term investment and sustainability, also appeared in Research & Development, which was neglected, and in training.

Regarding support for agricultural innovation, PDAOK demonstrated the risks and limitations of an approach that transfers exogenous innovations. In very specific agrarian systems faced with a multitude of constraints, innovation is essential but must originate from the rural location, knowledge, practices and capacities. To facilitate such a process – and perhaps to integrate external inputs – requires specialized competencies in Action Research, mobilized over a sufficiently long period of time. The mobilization of these competencies in a region such as Kanem is objectively difficult and requires appropriate incentives.

The cereal banks had positive effects in communities with high exposure to food insecurity but the conditions for profitability and sustainability of the services were neither identified or achieved. The design of the cereal banks should be reviewed. Management of these banks is difficult within the context of unpredictable cereal market conditions and almost impossible if the distribution of imported and free food supplies is as frequent as it was and occurs without coordination or a search for synergies with the local cereal banks.

The impact of the PDAOK on the millet crops reveals that the margins of progress possible in rainfed agriculture, and the importance of millet crops in the equilibrium of household economies, should not be underestimated. Progress is currently less in improving crop techniques than it is in resolving constraints upstream and downstream of production (such as establishing a seed stock and the possibility of obtaining supplies on credit).

The principle of a "Service Provider" – a lead operator - on behalf of a PMU with little field presence tends to create turf battles, and weakens project leadership and its responsibility for the people. The function of strategic guidance – not only of administrative and financial management – associated with good capacity for monitoring and evaluation (M&E) and true decision-making power should be clearly allocated to the director and supported by the responsible ministry and IFAD.

19. PDAOK suffered greatly from repeated changes in management. The complexity of the project and the number of its components and operators mobilized the energies of successive managements to tackle immediate administrative challenges rather than strategic issues. Given the challenging context, simplification of the project and of procedures now seems inevitable. The principle of annual renewal of operator agreements should be reviewed. Although this has the advantage of requiring operators to provide regular reporting, it also leads to disproportionate administrative costs and delays.

Major recommendations for the new project

The overall design and the institutional arrangement of the new project should above all be simple and adapted to the difficult conditions in the area. The specific objectives should be limited and clear, the responsibilities focused and very clearly allocated. Operational procedures – especially those for planning, approval and execution – should be as short and as simple as possible.

Within the intervention zone, selection of the sectors to concentrate on should consider the real potential for improvement, local dynamics, and effectiveness. Geographic dispersion under the mere pretext of equity and reaching the poorest of the poor should be avoided. The new project should have a broader vision of the concept of participation by involving the target populations from the time of the original design. Economic interest groups also need to be involved with project design and implementation. Project design and implementation should be formalized in a procedures manual and receive specific support to be of use.

The organizational form of the small economic interest groups – especially women's groups – should be retained but oriented towards more professionalism and market integration. Alliances among economic interest groups by proximity zone should be facilitated for supply and marketing functions. Village approaches are to be avoided, as they produced no results in the PDAOK.

Generally speaking, the Chadian Sahel very rarely benefited from customized agricultural innovations and research. Everything, or nearly all, remains to be done in Kanem, for tree crops and annual crops of the ouadi as well as for rainfed crops and non-nomadic small livestock. The future programme should put in place substantial participatory research in collaboration with research institutions, the operators supporting development and the producers.

The impact of the "petty trade" credit transactions and the high demand in this domain show that development of Kanem need not be solely through support for agricultural production. It is necessary to continue along the same path as PDAOK, acknowledging the importance of diversifying income sources, especially for women. The local artisans and merchants should playa role in developing the local economy for the following reasons:

i) establishing a network of local financial services that is autonomous and financially viable, starting with the existing CLAs, should be a specific objective of the new project, and

ii) inputs, equipment, transport, and the maintenance of infrastructure and equipment, should be provided by local organizations and the private sector and not by the project or public operators.

Establishing a self-administered, institutionally viable rural finance arrangement in south Kanem requires the intervention of a competent, motivated operator over a period of at least seven or eight years. This should be a separate sub-project, autonomous within the new project and not just a component. The operator should be a Chadian NGO supported by international technical assistance as needed. SECADEV is already involved in this work, which matches the NGOs own strategic priorities. It is recommended that SECADEV position itself – and be acknowledged by the Government and IFAD – as a full partner in the design and implementation of the new project and not just as a service provider. International technical assistance will undoubtedly be needed to reinforce the NGO's capacities and competencies.

The bylaws and internal regulations of the existing CLAs need to be re-examined alongside their management to correct any inconsistencies, complete them, and bring them into compliance with current regulatory provisions to obtain official recognition. In particular, a sustainable system cannot be planned by limiting access to the economic interest groups. Membership in the CLAs should be open to individuals. If credit is made available at an individual level the specific guarantee modalities will need examining.

Considering the income levels of the people in the project area, it is inconceivable that a microfinance system could operate in Kanem solely on the basis of collection of deposits. Resources in the form of loan funds or equity capital need to be made available to the CLAs under the control of the supporting NGO.

To guarantee the financial viability of the CLAs, financing investments in ouadi exploitation, especially those for water lifting which entail a high level of risk, should initially be avoided. During the first years of the project, the CLAs should not be involved in medium-term financing. There should be consideration about how to identify other medium-term financing options.

Creation of new CLAs should be studied as soon as possible for the purposes of economies of scale and consolidation of PDAOK achievements in most villages affected. Location selection should be guided by local economic potential, population density and the dynamism of the local inhabitants. The first milestones in a future umbrella structure, to be built as the network expands, will be established by the organization of meetings for exchanges among the managers of existing CLAs. This umbrella structure should be designed so that after support is withdrawn, it can continue operating under the CLAs who will have created it.

Credit is a powerful tool to meet the needs of the local population and it should be made sustainable in a healthy way. However, financing in the form of subsidies (matching grants) also seems to be essential for investments with deferred profitability and positive externalities and for innovations involving a substantial risk to the producers (water lifting systems in the ouadis with deep water tables, for example). A partial subsidy fund (local development fund) should be created under the new project, completely separate and independent from the relevant CLA savings and loan system.

Frequent changes in personnel and political interference in the project's management greatly impeded operations. Consequently, it is recommended that implementation of a future project be assigned to a more autonomous PMU with personnel recruited by competitive bidding in the private sector. The PDAOK regional and national supervisory bodies, which were badly run and caused significant delays in approval of the AWP/Bs, should be replaced by a single steering committee chaired by the appropriate ministry, to approve the AWP/Bs by no later than November 2002.

The problems the PMU faced in carrying out its work, show that special attention needs to be paid to financial management in the future project and support provided by an accounting firm. The specialized operators to which certain activities are subcontracted – including public services – should have greater autonomy (agreements covering two or three years, operational autonomy over vehicles and personnel) to avoid any flaws that could affect the PMUs.

The project should establish and use a simple, effective M&E system, the results of which could be a condition for approval of the AWP/Bs. Subsequent support will be provided in setting up and monitoring this system over the first three years of the project.

---

1. The Core Learning Partnership comprised: Mr. Hassanty Oumar Chaib, Director General of the Ministry of Agriculture; Mr. Sebey Boutna, Director of the PDAOK; Mr. Souleyman Abakar Ahamat, Anchor of IFAD projects in Chad; Mr. Demembaye Tongongar, ONDR; M. Ngaba Sou Ngadoy, Director General of SECADEV; Mr. Ngoniri Gos, Interim Evaluation Mission consultant; Mr. Luyaku Nsimpasi, Portfolio Manager, IFAD; Mr. Marc Empain, Portfolio Manager, UNOPS; Mr. Muaba Matamba, BSF; Mr. Jean-Philippe Audinet, Evaluation Officer, IFAD.

2. The following were present:
- For the government party: HASSANTY Oumar Chaib (DG of the Ministry of Agriculture); Abdoulaye SENOUSSI (DEAFPR/MA); Djibert LOOL (DA/GRHA); Demembaye TONGONGAR (representative of DN/ONDR); Ngaye TORDINA (DEPP/MA); Sebey BOUTNA (Director of PDAOK); Abacar OUMAROU (District Manager, ONDR – Kanem/Bahr-El-Gazal
- For SECADEV: Mahamat BODINGAR and Théophile MOYANGAR
- For BSF: Mujika ‘Muaba MATAMBA
- For IFAD (Operations): Luyaku L. NSIMPASI
- For IFAD (Office of Evaluation): Jean-Philippe AUDINET; Didier EYCHENNE (Consultant); Ngoniri GOS (Consultant)