OE NEW EVALUATION PROCESS

 

1/ These are some aspects to which OE will pay particular attention in 2000, which will be a year of testing our new evaluation process.

Annex 2

The Core Learning Partnership-CLP

Learning together will not only increase the quality and relevance of evaluations, but also promote ownership and commitment in the evaluation exercise and to its results, leading to a higher rate of adoption of evaluation output. The main role of the CLP is to produce a set of agreed upon recommendations and lessons learned, and an understanding of the concrete follow-up actions that are required from the evaluation. An underlying assumption is that CLP members will be exposed to learning experiences during the evaluation, which is a necessary condition for them to provide an effective contribution to the process. This may be achieved through the CLP member's participation in the critical steps of the evaluation process.

The main purpose of the CLP is to steer the evaluation process throughout and shape critical learning stages in the evaluation process. Typically, the CLP can have various functions: to develop an evaluation action plan and design the stakeholder participation approach; to discuss the progress of the evaluation efforts, etc. However, the CLP's main purpose is threefold:

• Collaborate in the design and approval of the Approach Paper;
• Discuss the draft evaluation report and the preliminary findings and extract as well as develop the recommendations; lessons learned and related follow up; and
• Plan the process leading to negotiation and approval of the agreement/understanding among the partners on the results of the evaluation.

The composition of each CLP is likely to differ, depending on the type of evaluation activity, the country involved, the theme being examined, etc. However, since the overall objective is to bring together a core group of partners who could effectively participate in the evaluation, the CLP members will be representatives of PMD, government representatives, OE, and the project level staff. The involvement in the evaluation process of other partners, including CBOs, NGOs, beneficiaries, co-operating institutions is important and will be pursued wherever this is feasible and realistic. It is necessary to carefully consider the composition of the CLPs, as the objective is to involve in the core group those partners who are able to effectively contribute in various stages of the evaluation process. Finally, for the CLP concept to be successfully implemented, an agreement is also required up-front amongst all participating members to dedicate sufficient time for their contribution in the process.

The concerned EO will take the lead in steering the CLPs, in close consultation and co-ordination with its members. The CLP may be established soon after an evaluation activity has entered OE's pipeline. Or it may be convened later in the process. Each evaluation will present different opportunities and for this reason the CLP model needs to be flexible. All members would take an active part in the various stages of the evaluation process, beginning with planning and implementation, and on to the final stages of the evaluation exercise. The CLP would jointly review the outputs generated at various stages, providing their comments and suggestions.

Finally, by introducing CLPs, OE would provide a strong signal to its partners about its seriousness and commitment in making evaluation and learning a more transparent, collegial and joint effort towards meeting common goals, in which the role and input of each partner is important. The CLPs would also dispel fears among some partners that evaluation is about finding fault and a proxy for measuring individual or institutional performance.

The CLP is a key element in our participatory methodology, specifically for Country Programme and Thematic Evaluations. The main challenge would be to monitor the working time of members involved in CLPs and other costs to an acceptable level while maximising the learning effort and experience. During 2000 OE will start to adopt the CLP to the extent possible on an experimental basis and the experience will be assessed together with the partners at the end of the year.

Annex 3

The Approach Paper

An Approach Paper will be included in each future Evaluation activity. The Approach Paper provides the guiding framework for undertaking the evaluation/study. The purpose of the Approach Paper is:

to clarify the overall objectives of the evaluation, define the CLP and the partners' respective roles, expected achievements, processes involved, budget and time frame.

The Evaluation Officer is responsible for drafting the Approach Paper. It should be short, focused and clear. It should provide the background and rationale of the evaluation activity, including the genesis of the demand for the evaluation, the needs it is meant to address, the core and broad-based learning partnerships involved, geographic coverage, specific focus, sources of initial knowledge and basic information needed, expected outcome and targeted users.

The main processes involved in the evaluation, from inception to completion, should be described in the paper, including the various phases of the activity, output envisaged at the end of each phase, the consultation processes (within and outside OE), workshops planned, the role of core and broad partners, the role of the EO, as well as the role of national and international expertise and local resource persons needed to undertake the evaluation.

The paper also describes the methodology to be used in the evaluation, including approach to fieldwork and the planned involvement of field partners, CBOs and project beneficiaries. It will highlight the use of participatory evaluation tools.

To help determine the focus of evaluation and highlight issues of emphasis, the preparation of the Approach Paper may be preceded by a short country/field consultation with partners. The modality of this consultation will be determined on a case-by-case basis.

An important step in the evaluation process is the stocktaking of existing knowledge, i.e., the collection and verification of existing knowledge within and outside the house on the subject matter of evaluation. Such exercise is of prime importance in helping focus the evaluation, building on previous work done by partners and push forward the knowledge frontier.

The CLP will discuss and finalize the Approach Paper. (See Annex 2 – CLP.)

The Approach Paper

An Indicative Outline

I Background and Rationale

• Background and justification of the Evaluation
• Expected Focus of the Evaluation

II Objectives

• Key questions to be addressed
• Expected Outcome

III Partnership involved

• Core Learning Partnership
• Broad Partnership
• Defines the roles and responsibilities of the EO, consultants, and core learning partners.

IV The Process

• Phases of the NEP Process

Defines how each of the NEP steps will be organized.

1. Stocktaking of existing Knowledge (how to do it, who is involved, role of partners, sources, expected outcome, timing)
2. Sharpening Focus of Evaluation, determining Expertise required and Modality of Field Work
3. Finalizing TORs and Recruitment of Consultants (local/international resource persons)
4. Methodology of Evaluation including local surveys, PRAs, focused groups etc. (partners involved with special emphasis on the role of CBOs)
5. Conducting Fieldwork including Workshops (roles of partners, expected outcome, timing)
6. Agreement on Conclusions, Recommendations and Follow-Up Actions (partners involved, consultation process, expected outcome, timing)
7. Articulation of Lessons Learned (expected outcome, timing

IV Work Plan & Budget Proposal

• Budget, with consultants/resource persons involved and timetable for the missions by phases of the process
• Draft Terms of Reference for consultants/resource persons