Country programme evaluation¹

The Core Learning Partnership (CLP) and the users of the evaluation

The members of the CLP constituted representatives from the Government of Indonesia (represented by the Ministry of Agriculture and the National Development Planning Agency), the NGO community (represented by NGOs PPSW and Bina Swadaya), four IFAD-supported projects², AsDB and IFAD (represented by the Asia and Pacific Division and the Office of Evaluation). Some of the most important stages in the interaction among some CLP members included exchanges on the approach paper as well as discussions at the outset of the evaluation mission, the wrap-up meeting at the end of the Country Programme Evaluation (CPE) mission on 1 September 2003, the stakeholders' workshop on 14 January 2004 in Indonesia to discuss the draft CPE report, and the CPE national roundtable workshop on 11-12 March 2004. The CLP is a vital part of the evaluation process and needs to be strengthened.

Some of the main evaluation findings

The CPE concluded that IFAD should continue and strengthen its role in Indonesia as a promoter of innovation in policy, institutional and operational terms. One of IFAD's comparative advantages lies in introducing and testing innovative ideas and approaches that can be expanded nationwide. IFAD's small size and flexibility should be used as an advantage in Indonesia to take up new models of rural development and to make them work and to abandon them if they don't. It has already enhanced flexibility in its lending instruments well suited to this task. By building on existing evidence from the ground, IFAD could further increase and deepen its contribution to policy change and assume a position of knowledge and influence in councils such as the Donors' Group for Indonesia giving vibrancy and vitality to its arguments. IFAD's unique mandate provides a powerful imperative for it to take a leading role in showing how rural development reduces poverty.

One of the most important findings of the CPE relates to IFAD's strategic choice for rural poverty alleviation in Indonesia. In this regard, the CPE has highlighted that the 1998 country strategy and the projects that followed put significant emphasis on the formation of social capital as a prelude to economic empowerment and a means for rural poverty reduction. The evaluation acknowledges the importance of social mobilisation and building social capital as a key dimension of IFAD's work in Indonesia. The evaluation also argues that the formation of social capital is a necessary but not sufficient condition for successful rural poverty reduction, and that the next country strategic opportunities paper (COSOP) and consequent operations should have a balance between the promotion of social capital on one side and the economic empowerment of the rural poor through, but not limited to, agricultural development as well as the promotion of sustainable off-farm opportunities.

In terms of impact on rural poverty, five of the ten projects³ had substantial impact on poverty, in particular in terms of income effects. Evidence of impact on the poor, beyond increased incomes, is limited partly because this was not explicitly sought in early projects. Impact on women and institutional development was limited in earlier projects but is improving in recent operations. The portfolio does not score well on sustainability (only 5 of the 11 past and current projects rated are likely to be sustainable). The lack of sustainability in the portfolio is regarded as a major concern by the CPE.

Another finding relates to policy dialogue. Although IFAD had committed to promoting policy dialogue in several areas with a variety of partners in Indonesia, the CPE notes that due to various factors, the Fund has not been in a position to contribute sufficiently to key policy dialogue processes established by both the government and donor community, especially at the national and sectoral levels. The CPE further illustrates the importance of articulating up front the need for clear, measurable objectives for policy dialogue, with allocation of resources and the definition of work plans that includes prioritisation of activities and indicators for outcome assessment.

The CPE highlights the need for strong partnerships at various levels with different partners, as diverse partnerships with different institutions serve specific purposes. Overall, IFAD's partnerships within GOI especially those in the MOA have been productive. However, some would like to see IFAD widening its partners at the national level in the Government. Partnerships with NGOs are important. They have grown and are improving, whereas those with co-operating institutions and with co-financiers need enhancement. Among other issues, the CPE identified the need for continuous engagement and timely communication as necessary ingredients for successful partnerships.

Corruption is widespread in Indonesia. The CPE assessed the tendering procedures in selected IFAD-supported and noted that some of their financial procedures have been tightened. Nevertheless, the CPE argues that the Fund needs to be forthright in recognising that corruption is a disruptive phenomenon, especially in Indonesia, and take sterner and more consistent anti-corruption steps.

Back to Top

Recommendations agreed upon by all partners

8. Overarching recommendations. There are two broad strategic recommendations that IFAD and its partners agree to include in the next Indonesia COSOP and subsequent operations:

1. Ensure that the IFAD strategy to empower the poor includes efforts to raise farm and non-farm productivity in a sustainable manner. This may require, inter-alia, stronger linkages with formal and non-formal agriculture research systems and promoting the development of rural micro enterprises, markets and other aspects of market-linkages to capture benefits from production increases, such as rural infrastructure, market information and agro-processing; and

2. Increase inputs devoted to knowledge generation, advocacy and policy dialogue. In this regard, attention should be paid to documenting what works, which would help carry the policy and advocacy dialogue forward. Moreover, it should use networking (both real and virtual) and experimentation on the ground as key instruments in knowledge generation.

To give effect to these recommendations, three lines of actions should be considered:

1. IFAD and its partners should continue to establish and nourish strategic partnerships. First, would come partnerships to find new and workable solutions to raising incomes and empowering the rural poor. Second, would be partnerships to identify and introduce new ways of building capacity of the enablers for effective rural poverty reduction. And third, would be to enhance partnerships to provide an audience for new policies and ideas and tested poverty reduction projects in the rural economy. Such partnerships can be established with NGOs and CBOs, government agencies as well as other aid agencies;

2. IFAD should provide greater support during implementation, and secure better supervision of and better monitoring and evaluation in the operations it supports. Implementing agencies require clear accountabilities and project staff need more support during project execution possibly through a highly competent, well resourced and well mandated in-country group of mainly local staff who will also be required to ensure pro-active and preventive anti-corruption measures. To capture the knowledge generated by learning whilst doing requires appropriate and participatory project monitoring and evaluation and more intensive implementation support (see paragraph 9iii & 9v). Similarly, logframes need to be revised and updated to help improve strategic management and more attention must be given to outcome monitoring and impact evaluation; and

3. IFAD and its partners should define objectives in its next COSOP for Indonesia in accordance with the resources that can be allocated. The COSOP should include a coherent hierarchy of objectives, for both lending and non-lending operations, which should be time-bound. It should contain performance indicators to monitor the implementation of the strategy, which will serve eventually to measure performance and outcomes of the COSOP. The preparation of the COSOP should be based on a thorough analysis of the inputs, processes, and activities required to achieve its objectives, as well as include a prioritisation and a time plan for the delivery of its expected outputs.

10. Other recommendations. In addition to the above, this section includes recommendations that were developed during the CPE national roundtable workshop in Yogjakarta on 11-12 March 2004, attended by various partners including representatives of the Government of Indonesia, (11) members of the IFAD Evaluation Committee and the Executive Board, the NGO community, research institutes and universities, project staff, international development organisations and IFAD management and staff (from the operations and evaluation divisions, as well as the Vice President's Office). The recommendations have been grouped according to the three CPE workshop themes (i.e., Strategic Mix of IFAD Operations, Policy Dialogue and Portfolio Management):

Strategic mix of IFAD operations. The discussions under this theme explored how IFAD, working with GOI, NGOs and other partners, can identify and pursue the most effective and efficient route(s) to reducing rural poverty in Indonesia. The following recommendations should be taken into account in the preparation of the forthcoming COSOP:

1. The COSOP should articulate IFAD's comparative advantage in Indonesia and its complementarities with other International Financial Institutions and UN agencies working in agriculture and rural development;

2. IFAD activities should also cover the rural poor in coastal areas engaged in agriculture and fisheries;

3. The Fund should allocate greater resources to agriculture to ensure increased productivity that would lead to increased income. Agriculture and off farm economic activities should be given at least equal emphasis as social capital formation;
   (iv) There is need to devote greater attention in IFAD operations to promoting pro-poor, low-cost technical change in on and off farm activities using local knowledge and as identified by the poor;

4. Forestry and promoting access to forest lands should be included as part of IFAD's natural resources management agenda in the next COSOP; and
   (vi) IFAD's lending and non-lending (e.g., policy dialogue, research funded through grants, and knowledge management) activities should be linked in a mutually reinforcing and strategic manner.

Policy dialogue. This theme explored the objectives and nature of IFAD's policy dialogue and advocacy work, the modus operandi of policy dialogue in light of the absence of IFAD staff permanently present in the field, the platforms and processes in which the Fund must engage actively, as well as the human and financial resources implications to effectively achieve the established objectives. The following recommendations should be taken into account in the development of the COSOP:

1. IFAD should seek national stakeholders' contribution to the identification of policy dialogue objectives and commitment to their achievement during the next COSOP formulation. The next COSOP should consider policy dialogue as an integral dimension of IFAD activities in Indonesia. Policy dialogue activities should have measurable objectives, components and activities, outputs, resource allocation and performance indicators;

2. Resources need to be earmarked to identify, document and communicate local policy changes promoted in the context of IFAD operations that could have potential for upscaling and replication at a higher level beyond the operation under consideration;

3. IFAD should improve its efforts in promoting dialogue among national stakeholders on policy changes identified in IFAD operations. In this regard, IFAD should strengthen further its partnership with the National Development Planning Agency (Bappenas), to promote the discussion on policy issues of mutual concern with IFAD;

4. The Fund should review and intensify its engagement and participation in selected national and thematic policy dialogue fora on rural and agriculture development; and

5. The preparation of the next Indonesia COSOP should be used as an opportunity for engaging the government, the donor community and other national stakeholders in policy dialogue on issues of pressing concern.

Portfolio management. Various aspects of portfolio management are fundamental for impact achievement. For example, the issue of partnerships and institutional choices are important including the role of NGOs, as are issues related to project design, co-financing, implementation-support, direct supervision and supervision through co-operating institutions, as well as monitoring and evaluation. Moreover, problems associated with corruption cannot be ignored. If IFAD-assisted projects are to optimise their effectiveness and maximise their impact, improvements in the institutional processes and management actions bearing on these and other dimensions of portfolio management are of central importance. The following key recommendations on this topic should be given attention in the preparation of the COSOP:

Partnership with NGOs, IFIs and UN agencies.

• There is need to promote a stronger partnership between government and NGOs at the project level. This can be achieved, inter-alia, by a clearer definition of the objectives, roles and responsibilities of NGOs in project design and implementation, and clarity about the resources allocated and funding mechanisms to NGOs for implementation purposes;
• Priority should be given to working with committed NGO partners with a good track record and adequate institutional capacity. Moreover, NGOs should be entrusted responsibilities commensurate with their capacities and outreach; and
• IFAD should proactively engage and intensify its co-operation and co-ordination with AsDB, FAO, UNDP, World Bank and other international development organisations in developing and implementing projects and programmes, policy dialogue, knowledge sharing and other activities.

Project quality at entry.

• Within a programme-oriented approach, there is need to focus on smaller geographic areas in future operations and ensure appropriate integration between different project sub-activities; and
• The need to strengthen multi-stakeholder participation in project design and institute discussion fora (virtual and non-virtual) during design as a means to stimulate debate is important.

Supervision and implementation-support.

• The benefits of direct supervision work by IFAD was highlighted;
• Better supervision in general is essential for improving implementation performance. In this regard, supervision (particularly processes related to performance assessment and improvement as well as learning) needs to be adapted to the evolving nature of IFAD operations. For this purpose, additional resources are required to enhance supervision quality;
• More use of local experts should be made in direct supervision and supervision through co-operating institutions;
• Co-ordination, synergies and feedback between the IFAD-led implementation support activities and the supervision exercises should be clearly defined; and
• There is need to streamline and improve co-ordination, communication and follow-up between supervising institutions and agencies responsible for project audits.

Corruption.

• In close co-operation with concerned stakeholders including the GOI and NGOs, the new COSOP should appropriately analyse and pay due attention to the issue of corruption, inter-alia, in the selection of partner institutions and programme provinces/districts;
• There is need to establish a dialogue with national counterparts on how to incorporate proactive and preventive anti-corruption measures in all future projects and programmes;
• It is recommended to strengthen NGO capacity as partners who could play a greater role in monitoring and reporting on financial matters; and
• Provide more responsibility for implementation and management of corresponding resources to community based organisations as a means to combating corruption.

Monitoring & Evaluation.

• Ensure baseline surveys are undertaken systematically at the beginning of each project;
• Systematize the introduction of the IFAD M&E Practical Guide in all operations; and
• Document and share the experiences in PIDRA in using participatory monitoring and evaluation.

---

1. This agreement reflects an understanding among partners (see paragraph 1) to adopt and implement the recommendations stemming from the evaluation.

2. Eastern Islands Smallholder Cashew Development Project, Eastern Islands Smallholder Farming System and Livestock Development Project, Rural Income Generation Project and the Post Crisis Programme for Integrated Development in Rainfed Areas.

3. IFAD has financed a total of 12 projects in Indonesia. At the time of the CPE one project was not yet effective and due to lack of data another project was not rated.