Steyning, 23 May 2022 – Many evaluators appear to have been caught flat-footed by restrictions resulting from the pandemic environment, especially limits on travel and onsite work. Moving from this premise, the forthcoming book titled ‘Evaluation in the Era of COVID-19’ takes several institutional, national and disciplinary perspectives to explore both the shortcomings of evaluation but also the innovations and successes. The editors of the publication, Indran A. Naidoo, Director of the Independent Office of Evaluation of IFAD (IOE), Ray Rist, former president of the International Development Evaluation Association (IDEAS), and Pearl Eliadis, Associate Professor at McGill University, discussed these issues during the annual meeting of the International Research Group for Policy and Program Evaluation (INTEVAL), which took place on 23-25 May 2022, at Wilton Park.

“This book will make a significant contribution to the field of evaluation, given that COVID-19 has changed in many ways the manner in which evaluation has been working. Evaluation, with its focus on accountability and good governance, should have been well-positioned to respond to the COVID-19 crisis. But, it is far from clear that evaluation systems actually rose to the challenge”, affirmed Dr Naidoo.

Hosted on the prestigious grounds of Wiston House, in the UK, the event was organized by Ida Lindkvist, Jos Vaessen, Ray Rist, Rob D. van den Berg, and by the UK Foreign, Commonwealth and Development Office. Attended by over forty INTEVAL members, the meeting provided a space for distinguished authors to present recent and upcoming publications.

“This is the 37^th INTEVAL annual gathering, which has taken place uninterruptedly since 1986. When we gather each year, a significant constellation of evaluation leaders and authors emerge to help carry the group forward into the future.  To date, the group has published 31 books on various themes of evaluation”, explained Dr Rist. 

Presentations on ‘Evaluation in the Era of COVID-19’ offered multidisciplinary perspectives and critical assessments on how evaluation and evaluators responded to the pandemic. Presentations included Peter Wilkins (Australia), Ray Pawson (United Kingdom), and crosscutting insights into the impacts on the SDGs (Dorothy Lucks and Robery Lahey). Jan-Eric Furubo presented his work on evaluation in times of turbulence, providing key insights into how knowledge is selected by policymakers during crises. Dr Naidoo analyzed the UN’s socio-economic response plans, demonstrating how traditional oversight activities were suspended and unable to respond with sufficient agility.

“During the meeting, distinguished participants will be taking stock of the latest books that have been prepared, including the upcoming publication titled ‘Evaluation in the Era of COVID-19’”, Dr Naidoo noted. 

Discussing the forthcoming volume, Dr Naidoo, Dr Rist and Prof. Eliadis noted that the evaluation sector had evolved in environments that possessed a degree of stability; governments had predictable and structured planning processes with clearly established sets of users for reporting results. The COVID-19 crisis disrupted many of these systems and processes, as well as the connections among them. The demand for discrete studies based on simple attribution models declined while a move towards broader knowledge streams, capable of responding to current crises, accelerated.

“An original meta-analysis of evaluation literature reiterated points made by several authors: in the first half of the pandemic, evaluators were overwhelmingly focused on practical and operational matters, while other knowledge actors rapidly occupied the field. In some cases, evaluators were often absent in critical conversations about what the pandemic meant for evaluation”, stated Prof. Eliadis.

Few evaluations were able to address the implications of the new democratic deficits nor the human rights violations that were revealed by exacerbated or emerging forms of inequality: these are not areas of comfort for traditional evaluation practice. Evaluators were being asked to move beyond methodological problems and technical solutions to engage with substantive and high-level inquiries that were content-driven, consistent with human rights norms, and informed by what has been learned during the pandemic.

“The book argues that evaluators paid too little attention to the compelling need for substantive need for transformation in the profession as compellingly argued by Michael Quinn Patton. That is the context for extracting lessons to address the major systems transformations needed, with a focus on human rights as argued by Pearl Eliadis, and by Michael Quinn Patton who notes the need for that transformation in health, climate, food, and social justice”, said Prof. Eliadis.

The session closed with remarks from discussant Dr Leslie Fierro, the Sydney Duder Professor of Evaluation at the Max Bell School of Public Policy (McGill University) who noted the implications of this research for teaching, capacity building and the new generation of evaluators.

The INTEVAL gathering also included a seminar on ‘the future of evaluations’ hosted by the Independent Evaluation Group of the World Bank (IEG), which co-founded the event.

“Looking ahead, during the meeting, we agreed to start work on a new book examining the interrelations of evaluation with artificial Intelligence.  Three other books are presently in progress; including evaluation and theory of change; evaluation and the post-truth environment; evaluation and sustainability”, highlighted Dr Rist.

In 2023, the INTEVAL group plans to meet at IFAD, in Rome, Italy.

For further information, please contact Alexander Voccia [here]