Completion evaluation

Background and Core Learning Partnership

The Cordillera Highland Agricultural Resource Management (CHARM) Project targeted Indigenous communities in three provinces of the Cordillera Administrative Region (CAR) of the Philippines.  The CHARM project was executed by the Government of the Philippines (GOP) through the Department of Agriculture (DA) and jointly funded by Asian Development Bank (AsDB) and IFAD between 1997 and 2004. The evaluation of the CHARM project was conducted in 2006 by IFAD's Office of Evaluation (OE).

As per usual practice for OE evaluations, a Core Learning Partnership¹ (CLP) was established providing critical inputs at key stages in the evaluation, including towards the preparation of the Agreement at Completion Point (ACP).

This ACP reflects an understanding between the Government of the Philippines represented by the DA and the National Economic and Development Agency and International Fund for Agricultural Development (IFAD) represented by the Asia and Pacific Division on the key findings from the evaluation (see section II below), and to adopt and implement the evaluation's recommendations listed in section III, according to the set timeframes.

Main evaluation findings

Design features.  The combination of sustainable agriculture development and Natural Resource Management (NRM) reflects the specific conditions and needs amongst poor communities in CAR.  The components and sub-components were generally appropriate, apart from the Rural Finance sub-component which had various design weaknesses.  A number of key design features that had appeared in the project feasibility study were later dropped in the Project Appraisal document and final logical framework which led to design gaps.

Implementation and outputs.  There are extensive sources of data for the project which provide a composite picture of a successfully implemented project.  Supervision reports show consistent and satisfactory performance throughout the project period.  Physical targets were largely achieved, with some targets being exceeded. However, there are mixed results across the different component activities.

Attaining project objectives.  The rural infrastructure sub-projects resulted in increased yields and reduced input and marketing costs in most instances but on-going maintenance is an issue.  Reforestation activities provided opportunities for short-term local employment. Agriculture development activities were not sufficiently field-oriented so the potential in scope and outputs was lower than expected. Notably, rural finance achievements did not attain the expected results and the sub-component is rated unsatisfactory. Peoples Organizations (POs) and Local Government Unit (LGU) training did not achieve the expected results due to topics not being relevant and multiple training being accessed by leaders rather than spread across the community. Planning activities drew the partners together in identifying and addressing local priorities in a coordinated way. Implementation activities provided opportunities for government agencies to harmonize policies, procedures and practices, particularly in relation to Indigenous self-determination.

Relevance, effectiveness and efficiency.  The CHARM project design was relevant to the needs of the targeted communities. The substantial support for Indigenous processes and practices was not only appropriate to the community but contributed to national policies and practices related to Indigenous land and cultural integrity. The project was largely effective; however, delayed contracting of Non Government Organizations (NGOs) during the early stages of the project reduced the effectiveness of planning and participation. Participation processes of partners with local leadership were very effective but wider community participation has consistently been raised in reports as insufficient.  The CHARM project can be considered a fairly efficient operation. For example, the Economic Internal Rate of Return (EIRR) estimates (20.06 per cent) have exceeded project appraisal estimates (18.4 per cent).

Performance of IFAD and its partners.  The regular coordination activities of the project, particularly at higher levels, were important forums for integrated action by partners that extended beyond the scope of CHARM activities alone and contributed to other governance activities.  IFAD's involvement in implementation was minimal through much of the project, but increased in latter years.  The Government of the Philippines (GOP) and AsDB performed satisfactorily.

Rural poverty reduction impacts. A positive impact on project participants has been achieved. Yet, the target of reducing the level of poverty from 70 per cent to 25 per cent across all targeted municipalities was overly ambitious and did not adequately take into account the unique situation in CAR. The impact on poverty is considered only modest.  However, the extent of project impact should not be underestimated. There were impacts for the Indigenous People in the Cordilleras that have far reaching effects for improving their lives in the future. Policy dialogue, partnership building, and assisting in innovation in land tenure processes were not explicit objectives of CHARM. Nevertheless, the project investments have resulted in opportunities for partners in the CHARM project to strongly engage in institutional development opportunities that are considered highly important in the region.

Sustainability and ownership. CHARM was implemented during a critical period for Indigenous People in the Philippines. The aspirations of the local Indigenous Communities in terms of poverty reduction, the changing policy context, and the unique challenges faced in the Cordilleras were considered and supported proactively by CHARM implementers. Assisting national recognition of Indigenous land ownership has built significant foundations for future appropriate development.

Unfortunately, foundation processes to build capacity for sustaining project gains were not pursued until the end of the project. At this stage POs were still weak. At both the municipal and barangay level, a continuing attitude of institutional dependency amongst all POs was observed ². Improved participation, ownership and wider capacity building could have contributed to a greater likelihood of sustainability. 

Innovation, replicability, and scaling-up.  CHARM supported the formulation of some of the first Ancestral Domain Sustainable Development and Protection Plans (ADSDPP) in the country. The ADSDPP is intricately linked with land tenure processes for Indigenous Communities.  CAR is now being promoted as a national model in Indigenous land tenure processes.  As most POs are weak and there has been little orientation towards replication systems, no replication or scaling up could be discerned.

Overall assessment. In sum, CHARM has been an important project for CAR and the Indigenous communities that it reached.  Project performance has been satisfactory in achievement of physical targets and in attainment of goals.  Outcomes and impact have been lower than expected due to the fact that targets were over-ambitious, but there is strong justification for continuing IFAD and AsDB support for the processes in CAR.  There are important lessons to be learned from CHARM that will benefit targeted communities in a follow-on project and also provide potential for further policy dialogue and improved processes. 

Strengths.  The main strengths of the project have been in the improved coordination between the implementing partners in CAR.  The project activities have provided a means for interagency and Government/NGO collaboration.  The gains in Land Tenure Improvement (LTI) were significant and contributed to attaining not only the project objectives in terms of improved resource management but also to IFAD broader objectives of strengthening local asset ownership. Barangay natural resource planning assisted in identifying areas for reforestation, as well as contributing to broader land use planning initiatives of the local government units.  The rural infrastructure installation has been a major contributory factor to improved market access and improved facilities in most project areas.

Weaknesses. The main area of weakness was in the technical services delivered through the Agriculture Services Support Component.  The level of adoption from technology training, information kits, agri-business activities or technology transfer reached only 5 per cent at best, apart from Integrated Pest Management, where adoption rates ranged between 15-25 per cent. The low uptake rate seems to be related to perceived lack of relevance of topics, and method of training. Tangible results from the research activities were also not evident. Comprehensive studies on the "Key Commodity System" concept and agro-forestry based technology synthesis did not include mechanisms for applying the recommendations of the research. Consequently utilization of proposed new technologies at the farmers' level did not eventuate.

Other weaknesses. Existing Indigenous Knowledge Systems and Practices (IKSP) in the traditional communities covered by the project have not been clearly embedded in the design of the research activities. The actual cost of access infrastructure rehabilitation, though within the acceptable range of unit cost parameters during the implementation year, exceeded the appraisal estimates. This was due to the underestimated cost at appraisal given the topography of the project sites.

Lessons learned.  An overall lesson learned is that National Standards cannot apply in CAR.  The standards for rural infrastructure, agriculture and reforestation did not match the local conditions.  Consequently there is a need for a more flexible approach at local level in line with community needs. Local knowledge and locally appropriate designs could have had greater support. The tenuous link between enhanced agricultural support services and results at the farmer level particularly highlights lower than expected effectiveness in training and other extension services.

Key recommendations agreed by partners

The following recommendations from the evaluation have been agreed upon by the concerned partners. They have also benefited from discussions during a final CHARM project evaluation stakeholders' workshop held in Manila on 26 January 2007.

Recommendation 1

Proceed with CHARM2. There is opportunity to build from the successful processes in CHARM and consider a second phase project.  This is important for both the sustainability of the CHARM interventions and expanding the project to other deserving communities.

Actions

Incorporate learning from the evaluation.  Recommendations for project design include: (i) a revised definition of poverty reduction incorporating community values of quality of life and sustainability considerations rather than only income increase; (ii) sustainable agriculture development should be balanced with IKSP, natural resource management and enrichment planning; (iii) broader participation and equity focused on comprehensive community development and a local learning approach including a participatory M&E system; (iv) an outcome rather than target orientation should be taken with built-in flexibility through the annual work plan and budget to allow adjustment to changing context; and (v) a clear exit strategy with a mainstreaming of project components into local institutions and processes.

Build on existing information

The preparation for CHARM2 should strongly consider the issues raised during the CHARM feasibility study, based on lessons learned from Highland Agriculture Development Project (HADP). Many issues remain relevant, as do the solutions proposed that were not adequately incorporated into the CHARM Appraisal design.

Strengthen and extend existing approaches

The opportunity that CHARM2 provides for building substantial knowledge in CAR is unparalleled.  The Project Support Office (PSO) already has an extensive library and staff with vast amounts of intellectual knowledge related to project implementation in CAR, and wider development issues such as Indigenous Peoples Development and Land Tenure Improvements.  DA has an established Project Coordination Office (PCO) with a core of experienced staff that holds the intellectual and institutional knowledge of CHARM. The current processes include inter alia: a well-staffed PSO within the CAR DA; strong agency coordination; integrated components; a strong focus on policy dialogue and advocacy for Indigenous Knowledge Systems and Practices (IKSPs), and provision of critical infrastructure.   CHARM took a proactive approach to innovating in administrative procedures and polices related to IP concerns. The limiting factor in the level of policy impact is that these are still fledgling processes which are still subject to conflict, unclear guidelines and delays in implementation.  Thus CHARM2 should both solidify gains made in existing project areas and look to expand to new areas of CAR not served by HADP or CHARM.

Improve partnerships

The partnership between the GOP, AsDB and IFAD should be continued, although communication and co-operation between IFAD and AsDB should be improved. IFAD needs to have a greater role in providing implementation support. If AsDB is a partner in the next phase, IFAD/AsDB need to better coordinate to ensure IFAD's participation in supervision and implementation support missions.  If AsDB is not a partner, IFAD should consider direct supervision and implementation support given the number IFAD priorities being addressed in the project such as IP concerns, participation, empowerment, and policy dialogue. Stronger analysis and building links between sub-components e.g. agriculture and agroforestry at the local level is required to encourage local learning and innovations that would progress learning related to poverty reduction. Building on the social capital available within the project itself and a more analytical and knowledge management approach could build CHARM into an international model for Indigenous and watershed development.

Use CAR specific approaches

Support for the emerging IP policies and best practices should continue to be supported. Continued lobbying is required to consider CAR as a "special case" for national standards in recognition of the unique environment is still required to assist in effective development of the target areas and to consolidate the gains achieved through CHARM.

Time frame.Immediately, starting from the Appraisal Report

Partners involved. Relevant GOP agencies, Regional governments, IFAD, NGOs, POs and AsDB (if it participates in CHARM2)

Recommendation 2

Balance project objectives towards greater sustainability. Clarity in objectives is required to balance the potentially conflicting objectives in social, economic and environmental activities.  A follow on project should aim to achieve greater alignment of support at component and sub-component level to achieve coordinated and multiplier effects in each project site. Development of systems for valuation and payment for environmental services is an innovative area that needs to be continued. This would not only give greater recognition of the value of the Cordillera watershed to the Northern Luzon super-region, but also pilot systems for replication by other communities in watershed areas.   While most rural infrastructure packages under CHARM are categorized as "small scale" and are not considered as Environmentally Critical Projects, they may cause negative environmental impacts because they are located in CAR an environmentally critical area. The Indigenous communities in CAR have already shown that there are many local innovations that are appropriate to the development agenda of the local communities that could be incorporated into a more relevant and innovative approach.

Actions

Formalize Environmental processes.  The level of environmental best practice should be further developed by continuing work on valuing environmental services, improving environmental assessment for infrastructure construction, and strengthening the link between sustainable agriculture and forest management. A CHARM2 should include an Initial Environmental Examination (IEE) during the Project preparation stage and an Environmental Management and Monitoring Plan (EMMP) for every proposed sub-project to be financed.

Sustainability measures and processes should be instituted at commencement of project to build processes during the project operations that will be more able to be sustained by the participants themselves. Operational activities such as improved orientation, adoption of results-based management approaches, developing long term partnerships, more focus on transparency, use of Information Education Communication (IEC), community-based monitoring, using Indigenous systems and greater reliance on local knowledge management systems would all contribute to a project that has greater local ownership and a higher understanding of roles and responsibilities in sustaining project investments.

A wider menu of small productive infrastructure, Information Communication Technology (ICT) infrastructure and support, partnerships with private sector would provide an opportunity for new partnerships and sharing of ideas, as well as combining local innovations with introduced technology. Higher cost parameters should be allowed on critical access infrastructure given the topography of agricultural areas in CAR and to allow flexibility on design and specifications to fit local conditions. One particular area of innovation that needs attention is that of enviro-hazard mapping and risk management to assist in building risk scenarios and mitigation plans.

Time frame. Immediately, starting from the Appraisal Report

Partners involved.Relevant GOP agencies, Regional governments, IFAD, NGOs, and POs

Recommendation 3

Improve participation and capacity building processes. Improved coordination was a key focus of the CHARM design. The coordination activities of the PSO did result in significantly improved liaison between regional and provincial partners and formation of active working agency partnerships in the project sites.  Local implementation now needs to be focused at the municipal and barangay level, with greater emphasis on building engagement and self reliance of the local government units and community groups.  The LGUs were largely bypassed in the rural infrastructure and agriculture services components. There were positive initiatives through the ADSDPP formulation processes, municipal staff training and in other specific activities. Many barangay plans were used to contribute data towards the Ancestral Domain planning processes, as well as the municipal development plans, municipal comprehensive land use plans and provincial development plans.

Actions

Increased participation at the local level.These initiatives need to be given greater prominence in a follow-on project, with the Barangay Development Council as a focal point for broader community participation. Broader community participation must be encouraged by poverty profiling, local capacity building, and strategies to have a more equitable spread of benefits through out each barangay locality or sitio.

Greater capacity building for existing agency and LGU staff so that they can conduct the required activities would be a more sustainable approach rather than the extensive use of consultants that occurred in CHARM.  This can include exposure trips to other areas in the country to assess how successful processes might be applied in the CAR context.  

Introduce a Capacity Development component.In CHARM2, a specific Capacity Development component/unit is required that has the specific role of synchronizing training activities of the different agencies, as well as the different project components so that they clearly contribute towards the overall project outcomes.  The tasks for the unit would include: (i) improving training needs assessment so that training provided is tailored to the specific needs of the participants; (ii) improved training delivery methods, particularly increasing the number of courses delivered within the communities, (iii) improve relevance of training design and including re-entry plans for participants to increase the likelihood that learning will be applied; and (iv) conduct post-training assessments. Clearer systems to support application of training and replication within the communities could considerably increase the level of impact. More hands-on trainings are required for all components but especially agriculture technology and infrastructure operation and maintenance.

Time frame. Immediately, starting from the Appraisal Report

Partners involved. Relevant GOP agencies, Regional governments, LGUs, IFAD, NGOs, and POs

---