Completion evaluation

Agreement at completion point ¹

Introduction and the Core Learning Partnership

In 2004, IFAD's Office of Evaluation (OE) conducted a completion evaluation of the District Development Support Programme (DDSP) in Uganda. The evaluation mission took place in November/December 2004 and the evaluation report was finalised at the end of February 2005. An evaluation workshop was held in Kampala on 11 March 2005. The objectives of the workshop were to: (a) have a general discussion on the evaluation's results; and (b) discuss the draft evaluation Agreement at Completion Point (ACP). This ACP illustrates the stakeholders' understanding of the key evaluation findings and recommendations, their proposal to implement them and their commitment to act upon them. In particular, the ACP builds on the DDSP evaluation results and the discussions that took place during the workshop on 11 March. It includes, in section B, a brief summary of the key findings from the evaluation (which are fully substantiated in the main report). Section C includes the evaluation's recommendations, as agreed by the main evaluation partners and users (namely, the Government of Uganda (GOU) represented by the Ministry of Local Government (MOLG), IFAD's Eastern and Southern Africa Division (PF), the Belgian Survival Fund (BSF) and Ireland).

The evaluation core learning partnership (CLP) members included representatives of the MOLG, the chairperson of the donor working group on decentralisation, PF, BSF and Ireland. OE facilitated the work of the CLP and the ACP process. Discussions were held with CLP members at critical stages in the evaluation, and their inputs in the preparation and discussion of the draft ACP were much appreciated.

Some of the main evaluation findings

Main achievements of DDSP

The major achievements of the programme according to the completion evaluation have been:
(i) the successful implementation of infrastructural improvements such as safe water supplies, community access roads, health centres and parish headquarters; (ii) the provision of nearly 12 000 short-term and medium-term loans to the rural poor via the network of Subcounty Integrated Development Agencies (SIDAs); (iii) the establishment of community-based initiatives such as adult literacy programmes; (iv) the progress made by the district authorities towards the sustainable delivery of adequate services and a culture of participatory governance; (v) an extremely vigorous and effective supervision process.

Main constraints

The mission identified the principal constraints as follows: (i) an imprecise targeting policy which has meant the exclusion of the poorest from the benefits available under the credit and agricultural development components; (ii) the difficulty of providing motivation and incentives for the effective operation of the various users' committees; (iii) the limitations of bottom-up planning processes, which were never fully clarified; (iv) the failure to set up a fully operational M&E system, with central data repository.

Issues and recommendations

Strategic issues

The main recommendation is that IFAD should continue its support to the five districts involved in the DDSP on the grounds that the considerable achievements made by the programme will not prove sustainable if external funding is withdrawn, particularly in the three newest districts² . The evaluation pointed out that the sector-based ‘basket funding' approach currently favoured by the Government of Uganda does not necessarily match IFAD's specific brief and comparative advantages as emphasised in the report of the external evaluation of IFAD.

Recommendations

The evaluation recommends funding for a follow-up programme in the five districts, possibly with the addition of Masindi district (as also recommended in the DDSP Mid-term Review); the nature and direction of the interventions under the follow-up programme should be determined in the light of present needs and former investments, on a componentwise and districtwise basis. In its ongoing negotiations with the Government, IFAD should stress its unique function as a ‘laboratory for innovative ideas'.  

Suggested timing: Inception through appraisal by end 2006 subject to an official request from Government to IFAD by latest end-October 2005.
Partners involved: IFAD and GOU (and potential co-financiers and co-operating institution)

Policy issues

Despite a supportive policy framework and the radical decentralisation measures which resulted in the transferral of the responsibility for administering government services to the district-level – thus making possible the whole strategy of DDSP –  the evaluation feels that further steps are now necessary in terms of the fiscal empowerment of the counties and parishes.  The most urgent of these is the provision of alternatives to Graduated Tax, soon to be suspended.

Recommendations

The Government should consider further measures to decentralise budgetary authority in order to enable councils at subcounty and parish levels to fund and manage local schemes. The Government should promptly come up with alternative sources of subcounty and parish revenue to minimise the effect of the revenue hiatus at these levels. In this regard, PF should continue its policy dialogue with the GOU to ensure that the required measures are introduced as soon as possible.

Suggested timing: as soon as possible
Partners involved: GOU and IFAD (in collaboration with other development partners, in particular the donor working group on decentralisation)

Targeting issues

The evaluation was concerned that the poorest segments of the population had not been successfully targeted by the rural finance and agricultural development components. The implementing agencies had come to take the view that the ‘active poor' were the appropriate primary target group, the very poor being either inaccessible, or unable to take advantage of the opportunities offered by the programme. Although the targeting of the ‘active poor' is a legitimate policy, it was not what was intended at project design. Wealth ranking exercises carried out by the evaluation indicated that there were specific groups of very poor people willing and able to benefit from the programme had the targeting been more precise. There was extensive discussion of this issue at the Kampala workshop, with various stakeholders claiming either that the poorest had been reached or pointing out the severe problems in doing so. However, the consensus of the meeting was that any future project should give close attention to the matter of poverty focus.

Recommendations

Any follow-up programme should develop and pilot a medium-term agricultural loan product suitable for the poorest households, consulting best practices within Uganda and elsewhere. The objectives of the Agricultural Development component should be carefully examined to ensure that they match the needs of very poor farmers; in particular, support for the raising of poultry and small livestock might be included in future interventions. Staff of implementing agencies should be trained in PRA techniques such as wealth mapping and wealth ranking in order to achieve a greater precision in targeting.

Suggested timing: Inception through Appraisal by end 2006
Partners involved: IFAD and GOU (and potential co-financiers and co-operating institution)

Volunteer fatigue

The evaluation found evidence that some community-based committees lacked enthusiasm and commitment, leading in certain cases to ineffective maintenance of schemes and poor attendance at meetings. The programme relies heavily on the work of volunteers, not only for O&M but also in terms of the various participatory processes involved in bottom-up planning. For all volunteers, it seems likely that incentives – whether financial or otherwise – will be necessary to ensure a continuing supply of people willing to undertake these tasks. There also appear to be unrealistic expectations raised by the participatory planning process. Most of the proposals submitted at village and parish level will not be acted on and a sense of frustration will gradually set in. Some evidence for this is already present.

Recommendations

Incentives for volunteers must be found, either through allowances or, if the funds are not available, through study tours and minor privileges.  The limitations to the bottom-up planning process should be clarified, first of all at design, and after that at every stage of implementation. Care must be taken in the future to avoid an excessive reliance on community volunteers and any unnecessary multiplication of committees at community level.

Suggested timing: Inception through Appraisal by end 2006
Partners involved: IFAD and GOU (and potential co-financiers and co-operating institution)

Monitoring and evaluation

Despite the expenditure of much time and effort, the M&E system was not fully operational at the time of the visit of the evaluation mission. Among the main constraints in M&E especially at the district level is the low know-how in the use of software and the lack of a fully dedicated M&E capacity. 

Recommendations

The evaluation recommends to contract out the routine M&E activities (data collection, handling and reporting) to an outside entity that would have a representative co-opted to the management team to facilitate the use of the M&E outputs in performance management. This has the advantage that it brings in flexibility in how multidisciplinary resources are brought together to deal with the types of spikes that occur in M&E workload. For example, within the provision of one or two full-time person-equivalent(s) an outside entity can deploy up to four people for different aspects of the same assignment. This is a more efficient way of utilising resources. The main requirement is to embed the outside entity into the performance management processes. 

Suggested timing: By end July 2006
Partners involved: GOU (DDSP project authorities)

The Bunyoro Toro Development Company (BUTO)

Although supporting the initiative of the districts to retain the control of the revolving credit fund at district level, the evaluation queried (a) whether BUTO was able to match the need for a businesslike approach with the specific targeting of the poor in terms of savings mobilisation and loan products and (b) whether the company's declared aim that SIDAs should progressively purchase shares in the company – leading to eventual ownership of the company by the SIDAs – was realistic. The evaluation also recommended clarification of the exact position of BUTO in terms of accountability and reporting procedures. These issues provoked a lively debate during the Kampala workshop, ranging over many aspects of microfinance. The main feeling of the meeting was that it is unrealistic to suppose that the very poor can be easily or quickly reached through credit schemes and that it must be seen as a gradual process. However it was also agreed that the design of a future project should include particular efforts to focus on the very poor in this respect.

Recommendations

BUTO should be required by the districts to allocate a specific proportion of credit funds as pro-poor agricultural loans. The company should produce a detailed time-oriented plan concerning the purchase of shares by SIDAs. BUTO should be required to follow the same reporting procedures as other implementing agencies. The process of establishing it as a legal entity requires completion.

Suggested timing: by end July 2006
Partners involved: BUTO and GOU (DDSP project authorities)

Infrastructural investments in water and health

Boreholes constitute the preferred technology for rural water supply in Uganda and were chosen as the main technology in the programme districts. However, they are not appropriate or economical at all sites and some new boreholes have been neglected by communities in favour of other existing sources. Regarding the health centres constructed under the programme, the evaluation feels that certain essential amenities were missing from the package, notably running water and electric power in the maternity units. In remote areas, the health centres, together with the school, are the chief symbol of the development process. It is essential that they give an impression of efficiency, cleanliness and orderliness.

Recommendations

Boreholes are expensive to construct and maintain and should be adopted only where there is no viable alternative. The initial unit cost of health centres should be increased in order to provide for whatever is considered a basic necessity, including running water, DC electric power supply, a minimum number of beds and mattresses, and a perimeter fence.

Suggested timing: Inception through Appraisal by end 2006
Partners involved: IFAD and GOU (and potential co-financiers)

Management and finance

Three separate issues are addressed under this heading: budget flexibility, collaborative initiatives between sectors and between districts, and the need for a reliable flow of counterpart funding.

Recommendations

The evaluation recommends the introduction in any future programme of budgetary flexibility for district chiefs consisting of a tolerance level of up to 10%, allowing districts to deviate from the annual workplan and budget without specific application and subject only to proper audit controls. The evaluation also recommends brief monthly status update meetings between the component heads. For districts, a biennial workshop would provide an opportunity for the exchange of ideas. The Government should consider setting up monthly or quarterly automated disbursement schedules for counterpart funding. 

Suggested timing: Inception through Appraisal by end 2006
Partners involved: IFAD and GOU (DDSP project authorities in collaboration with district councils and the Ministry of Local Government in consultation with the Ministry of Finance)

---

1/ This agreement reflects an understanding among partners (see paragraph 1) to adopt and implement the recommendations stemming from the evaluation.

2/ Kamwenge and Kyenjojo are newly created. Kabarole, much reduced in size and with few senior staff remaining, is also effectively a new district.