Issue #7: September 2008

One of the main findings of the workshop organized by the Office of Evaluation (OE) and the Programme Management Department (PMD) prior to preparation of the 2008 Annual Report on Results and Impact of IFAD Operations (ARRI) was that a full understanding of the mix of conditions for supply and demand is key to improving the performance of monitoring and evaluation (M&E) systems in IFAD-financed projects. An institution-wide initiative is to be launched jointly by OE and PMD at the end of 2008 with a view to identifying, in close partnership with the Programme Management Department, the most appropriate strategies and instruments for strengthening the M&E of IFAD-financed projects.

The 2002 External Review of the Results and Impact of IFAD Operations concluded that efficient project M&E systems, followed up by rigorous independent evaluations, constituted the "basic building blocks for assuring quality control and maximizing results and impact". While considerable progress has since been made in terms of strengthening IFAD's independent evaluation function, concern with regard to inefficient M&E has been a recurrent theme of project evaluations. The 2005 Independent External Evaluation of IFAD expressed concern as to the Fund's poor record in terms of data collection and self-evaluation, and with regard to arrangements for project-level M&E. The ARRI for 2006 identified the poor performance of M&E systems as one of the major problems affecting IFAD-financed projects.

An analysis may be made of factors affecting the performance of M&E systems by distinguishing between the conditions for supply and demand. Supply conditions refer to the capacity to generate M&E information. In this regard, appropriate methods and tools are needed, as well as staff with the necessary skills and experience, technical support, and adequate financial and human resources. A balance with regard to such capacity must also exist within both IFAD and its partners, particularly when projects are implemented by weak institutions and under difficult conditions.

Supporting M&E by acting solely in terms of supply may have undesirable effects as far as the ownership of such systems is concerned. To consider the generation of information (e.g. through expensive and (often) unused surveys) as an inherently "good thing", increases the perception of M&E as a donor concern that borrowers are required to comply with. While generating information is indeed necessary, incentives are also called for to ensure that M&E is performed efficiently and that the information produced is both valuable and useful to all levels of stakeholders.

The only valid test of the quality of any M&E system is that it should produce knowledge relevant to key stakeholders, that it is used to manage, learn from and report on project activities, and that it improves the performance of current and future projects.

In view of the foregoing, it is fundamentally important to act on conditions for demand: conditions that are determined by the interest of stakeholders in knowledge generated by M&E systems. The Fund's Executive Board, Senior Management and country programme managers, as well as all levels of government authorities, implementing agencies and beneficiaries, are likely to have diverging views, priorities and interests with regard to M&E, based on cultural factors, time frame, accountability, relationships with other processes (e.g. performance budgeting), and so on. Stakeholders attach their own degrees of importance to the measurement of results or to information produced (quantitative vs. qualitative; outputs vs. impact; etc.) because diverse incentives and interests apply to the various actors and types of data involved.The challenge for IFAD is to disaggregate and understand the mix of supply and demand conditions that explain the varying performance of M&E in the different contexts involved. Acting on the supply side alone may not be sufficient to ensure that M&E knowledge is used for decision-making: the importance of incentives must also be acknowledged. Analysing and acting upon incentives that affect the demand for M&E from different stakeholders within IFAD and its partners is fundamental to understanding why M&E systems perform as they do and for mobilizing appropriate support instruments. Until recently, IFAD's understanding of the system of demand and incentives among country stakeholders and institutions has been limited. This may, however, change thanks to the Fund's shift to direct supervision and implementation support, as well as increased country presence.

Account must also be taken of the close relationship that exists between conditions for supply and demand.

While the monitoring of project activities and outputs is both easier and of keen interest to project stakeholders, assessing whether the expected impact is being obtained and making use of that knowledge has a strong political dimension and, from a methodological point of view, is more difficult to achieve. Greater commitment and leadership is needed to support impact studies, especially when beneficiary feedback is involved.

A limited capacity to undertake and analyse such studies may further reduce interest among decision-makers.

Technical solutions alone are not sufficient

Standardized or donor-led technical solutions that do not generate a response among stakeholders are counterproductive and may even reduce any long-term interest in the principles of M&E. As such, the contextual features of M&E should be acknowledged and, as and when needed, technical solutions (e.g. surveys, data processing software, etc.) designed to fit available skills and respond to the specific demands of potential users.

Promoting tailor-made and incremental solutions

In cases of poor accountability and limited management capacity, IFAD expectations with regard to M&E should be modest rather than ambitious. In these cases, ensuring a robust and systematic monitoring of outputs may be seen as a real achievement. Simple systems for gathering beneficiary feedback with regard to the quality of project services are more likely to generate interest among partner authorities and stakeholders than sophisticated (time- and resource-consuming) techniques. Overall, efforts for building up countries' M&E capacity should be modest, incremental, long-term, go beyond technical skills, and tailored to the particular institutional context

Enforcing the harmonisation and alignment agenda

As a signatory to the Paris Declaration for Aid Effectiveness, IFAD is committed "to rely, as far as possible, on partner countries' results-oriented reporting and monitoring frameworks" and to "work together [with partner countries and other donors] in a participatory approach to strengthen country capacity and demand for results-based management". These principles and commitments should be taken into account in the context of IFAD-promoted initiatives for strengthening M&E capacity. Systematic and country-wide obstacles to mainstreaming a results-based culture cannot, and should not, be addressed by IFAD in isolation. For this reason, the Fund should play an active role in forging and promoting partnerships for strengthening M&E capacity at the national and sector-wide levels.

Enhancing partnerships at national level

In partner countries, IFAD should seek to forge partnerships with local research centres, think-tanks, universities and other civil-society organizations that may play an active role in M&E during the implementation of IFAD-financed projects. The mainstreaming of M&E and results-based principles within project management does not necessarily entail the centralization and implementation of all M&E functions into project and government structures. Separating some evaluation functions from project management may even be beneficial.

Working on the demand side at all levels

IFAD's efforts to strengthen M&E should take account of the demands and expectations of all stakeholders involved in the M&E process and, in this regard, every effort should be made to facilitate the alignment of all layers of stakeholders involved therein. This will include promoting a new vision of project-level M&E within IFAD's governing bodies, based on the principles of usefulness, alignment and partnership. Capacity-building initiatives should focus not only on the supply side of the system but also involve potential users of M&E knowledge, particularly programme directors. M&E must be seen as an integral part of effective, results-based management rather than as a separate function or process.