Malaga, 2 February 2024 – There was a time when evaluators were looked upon as being the secret police, an enemy from which it was wise to conceal honest thoughts and genuine experiences. In many institutions, that may still be the case. For the Independent Office of Evaluation of IFAD (IOE), however, that time is a thing of the past. IOE’s drive to perfect its communication craft, including by horning-in neuroscience-based principles, has resulted in a mindset shift among stakeholders and evaluands alike. Co-creation and enhanced engagement are now a daily reality in the work of IOE, which maintains its full independence whilst thriving on co-creating milestone publications such as the IFAD Evaluation Manual, which was jointly produced with Management in an unprecedented collaborative effort. The roots of these results run deep and date back to the beginning of 2021, when IOE embarked on its ‘revolutionary’ journey under the leadership of its newly appointed Director, Dr Indran A. Naidoo.

“We need to move from evaluation phobia to evaluation embracing. To do so we need to create a psychologically safe environment where people feel free expressing themselves. The way to do so is by enhancing opportunities for dialogue and engagement, and by improving communication. Engagement must happen at all stages of the evaluation process, from workplan and design to data collection and analysis”, explained Dr Naidoo during an Evaluation Practice Exchange (EPE) of the United Nations Evaluation Group (UNEG) EvalWeek 2024, held in Malaga on 30 January 2024.

The UNEG EvalWeek 2024 took place from 29 January to 2 February 2024. Hosted by the World Intellectual Property Organization (WIPO) and the United Nations Office on Drugs and Crime (UNODC), in collaboration with CIFAL Malaga, the event comprised of three elements: The Professional Development Seminars (PDS), on 29 January; twenty EPE sessions, on 30-31 January; and the Annual General Meeting (AGM), on 1 – 2 February. Seasoned evaluators and Directors of UN evaluation offices from all across the globe joined emerging practitioners and external observers for an intense, enriching and insightful week of discussions and sharing of experiences.

“Evaluation is critical to ensure that our work is making an impact on the ground and a difference in the lives of people. When I became Secretary-General, I challenged our organization to strengthen our commitment to evaluation […]. Since then, we have reached some key milestones. Your annual meeting is an opportunity to build on these achievements […]. We need credible, high-quality evaluation to cover more programme areas — especially peace and security — and ensure that evidence, results, transparency, and continuous learning and improvement guide our work, every step of the way”, stated the UN Secretary-General, António Guterres, in his message to the UNEG event.

IOE presented a strong offering, with a five-person team in attendance. On 30 January, Dr Naidoo and Dr Alexander Voccia, Coordinator of the Evaluation  Communication Unit, delivered a joint presentation in the EPE seminar titled ‘What does it take to build a credible, independent evaluation function?’, which was chaired by Judit Jankovic, Senior Evaluation Specialist at the International Criminal Court (ICC). Later the same day, Fabrizio Felloni, IOE Deputy Director, Dr Kouessi Maximin Kodjo, Lead Evaluation Officer, and Massiel Jiménez, Evaluation Research Analyst, hosted a session titled ‘The crisis of linear thinking -- Combining theory of change and systems thinking in complex strategic and policy evaluations’, in which they also delivered a joint presentation. In addition, Mr Felloni acted as a presenter during the EPE session titled ‘Evaluating Policy Influence’.

Drawing from his over thirty-year long career, Dr Naidoo explained that there tends to be an assumption that evaluators can affect change and enhance programme quality. Unfortunately, that would appear to not be always the case. Cognizant of this, IOE’s new approach builds on a dynamic and engaging process of triangulation that is based on independence, credibility and utility. This has resulted in enhanced opportunities for dialogue and mutual learning throughout the evaluation process.

“Evidence, on its own, does not drive change. As a matter of fact, 70 per cent of initiatives that attempt to trigger change fail. Instead, organizations that develop initiatives geared towards changing mindsets are two times more likely to succeed in achieving the progress desired. We must be empathetic listeners. People must look forward to an evaluator coming, because we all share the common goal of making programmes better. Disagreement is also fine, it’s part of a healthy process of interaction”, clarified the IOE Director.  

To spearhead this effort, IOE adopted a ground-breaking neuroscience-based approach to its communication efforts which resulted in the production of a pioneering on-line training, and a communication module within the IFAD Evaluation Manual, in addition to a video series knows as the ‘evaluation pills’, among other outputs. This undertaking is bearing notable fruits, as IOE has refined its interactions with Management, whilst continuing to engage in global dialogues and networks. A large selection of innovative communication products provides enhanced visibility to these efforts and achievements. These include a state-of-the art fully independent website, and a first-of-a-kind on-line magazine, which have already reached tens of thousands of users across all countries in the world, since their launch two years ago.    

“IOE’s communication offerings present users with an immersive experience. Opportunities for real, tangible change are fast emerging, as the products facilitate greater uptake of IOE’s work”, underscored Dr Voccia.

In addition to contributing experiences vis-à-vis its state-of-the-art approach to communication, during EvalWeek, IOE also stimulated thought-provoking conversations on some of the core methodological issues pertaining to evaluations. In this regard, in their joint EPE session, Mr Felloni, Dr Kodjo and Ms Jiménez explained that a theory of change (ToC) does not necessarily have to be linear, despite the linear thinking that it normally entails.   

“We need to go beyond rigid theories of change, and try to work with the reality of multi-faceted contexts. This means having complexity represented in the design of evaluations”, noted Mr Felloni.

A linear ToC risks over-simplifying reality. As a result, it is inadequate for humanitarian programmes, due to the dynamic and unpredictable natures of crises, as well as for complex development programmes. 

“In contexts of fragility, it is important to develop an explicit resilience framework that takes into account key fragility drivers, in order to guide the operations”, highlighted Dr Kodjo.

These considerations notwithstanding, a ToC has the merit of helping to understand intended results, causal links and main assumptions. Therefore, ToCs should not be abandoned. Instead, they should be enriched by incorporating additional frameworks, such as a system thinking approach that can help better capture and unpack complexity.

“The adoption of a system-thinking theory of change has allowed IOE to explore contextual aspects, territorial dynamics and expected contributions of IFAD”, said Ms Jiménez.

Mr Felloni provided further points of reflection in his presentation during the EPE session titled ‘Evaluating Policy Influence’. In particular, the IOE Deputy Director used the example of IFAD’s support to the Central America Strategy for Territorial Development, approved in 2010 by the Summit of the Central America Integration System. In his presentation, Mr Felloni showed several steps of policy development that can be evaluated with limited resources through desk reviews, virtual interviews and brief in-country work. Instead, an assessment of socio-economic effects of a policy typically requires primary data collection, including surveys.

Following the closing of the EPE sessions, Dr Naidoo and Mr Felloni attended the UNEG annual general meeting. During the meeting, the UNEG working groups provided an updated on past and on-going activities. 

UNEG is an interagency professional network bringing together the evaluation units of the UN system, including UN departments, specialized agencies, funds and programmes, and affiliated organizations. UNEG's current operational strategies are outlined in the UNEG Principles of Working Together 2022 and UNEG Strategy 2020-2024. The UNEG EvalWeek 2025 will be held in Tokyo and hosted by the UN University, while the 2026 edition will be held in Rome.

For further information, please contact Alexander Voccia [here]

• To access the joint presentation by Dr Naidoo and Dr Voccia, please click here.
• To access the joint presentation by Dr Kodjo and Ms Jiménez, please click here.
• To access the IFAD Evaluation Manual and related on-line trainings, please click here.
• To access more information about UNEG EvalWeek 2024, please click here.

• To access the brochure ‘More than a journey | 20 years of independence, please click here.
• To access Fabrizio Felloni’s interview on the evolution of independence of IOE, please click here.
• To learn why independent evaluation makes IFAD a more credible institution, please click here.

ioe.ifad.org

Independent Magazine

Twitter

Youtube

Linkedin