Amplifying Voices and Knowledge through Remote Participatory Video Evaluation (PVEs)

 Martina Pluviano

In today’s rapidly changing development landscape, organizations are looking for innovative approaches to enhance their impact and extend their reach. Within evaluation, a recurring challenge has been the inclusion of the voices of local and indigenous communities. This blog presents an innovative approach for evaluations to effectively engage with and incorporate local and traditional knowledge.

In the context of the Corporate Level Evaluation on Knowledge Management (KM), the IOE has been testing innovative tools. In 2022, the CLE team conducted two Remote Participatory Video Evaluation Pilots (PVEs) in Brazil and Peru, which led to the creation of multiple videos, including one that captures the shared experiences from both countries. This blog therefore seeks to explore the transformative potential of PVE, particularly in light of recent technological changes brought about by COVID-19, and reflect on the key takeaways from piloting this methodology in Brazil and Peru, in order to contemplate its future scalability and suitability in evaluation.

Addressing Underrepresentation and Remote Access

PVE is a methodology that aims to address the underrepresentation of local and indigenous voices in evaluations and the logistical challenges of reaching remote communities. It combines participatory video and evaluation principles to engage with stakeholders in distant settings, to gather insights, perspectives, and experiences from communities beyond traditional evaluation schemes. In PVE, participants are actively involved, capturing their stories through video recordings and offering an authentic representation of their realities. This people-centred approach is designed to empower people to share their knowledge from their perspectives, to ensure their voices are included in evaluations, making the process more impactful. Also, this allows evaluators in the development landscape to directly hear from those they serve, gaining a deeper understanding of their needs and successes. For the pilots in Brazil and Peru, a hybrid approach was adopted, including semi-structured video interviews via Zoom and a collection of participants' footage and photos to enrich the final products.

COVID-19: A Potential Game-Changer for Evaluation and Inclusive Remote Engagements

It is undeniable that COVID-19 brought out significant changes to many sectors, and the field of knowledge management and evaluation is no exception. With the pandemic prompting governmental efforts to prevent isolation by increasing Wi-Fi accessibility in certain rural communities (e.g., in Brazil), the relevance of remote participatory video evaluation greatly increased. This technological advancement provides a great chance to connect with communities that now have internet access, making it easier to share and spread knowledge. Also, this newfound access to online resources can bridge the knowledge gap and promote more informed and inclusive evaluations. As we move forward, it might be interesting to reflect on these transformative changes in evaluation approaches and adjust to this evolving internet setup.

A step-by-step recipe to inclusive evaluation?

In Brazil and Peru, we conducted the pilots with a step-by-step approach. First, we carefully crafted an interview guide with customized questions for the audience, guided by Isabela Souza (Ar &Terra Consultancy). With help from the IFAD country teams, we chose two projects from IFAD's portfolio for in-depth evaluation. The selection of interviewees was done in close collaboration with the Project Management teams, by prioritizing participants who were “not the usual subjects” to ensure a diverse and inclusive representation. Before the interviews, we reached out to participants via WhatsApp for a friendly chat with the evaluators. This served multiple purposes: clarifying the interview's purpose and objectives, setting the interview date, and most importantly, making them feel comfortable and at ease. This friendly interaction helped build rapport, break the ice, and create a positive atmosphere for our discussions. During the interviews, we followed the detailed guides, but we also went beyond specific questions related to project activities, delving into the interviewees' lives and their connection to IFAD as a whole, embracing a Whole Person Learning (WPL) approach.

In the 8 interviews with community members involved in IFAD projects, we gained valuable insights into various aspects of knowledge management, particularly on how local and indigenous communities acquire knowledge. The interviews revealed three main learning methods: hands-on learning from previous generations, technical assistance, and necessity; social learning through exchange programs and communication with people from other countries; and formal learning in schools, universities, and IFAD project training. Also, though not the primary focus, the interviews offered crucial feedback on the value added by IFAD projects, the challenges they encounter, and potential ideas for improvement.

Lessons from the Process: Surprises, Concerns, and Inspiration

Amongst the many surprises brought by the video pilots, the most inspiring one was how willingly the participants shared their personal experiences. The direct engagement of the evaluators with community members during the interviews led to open and informal conversations where they happily expressed their thoughts and felt heard. This approach provided a genuine opportunity to gather authentic feedback, making the participants feel at ease, resulting in candid and unfiltered responses.

However, the process also faced several challenges and setbacks. One major concern was the limited Wi-Fi availability in rural areas and the unstable internet access in remote locations, making it difficult to conduct interviews via Zoom. In Peru, language barriers and local dialects added complexity to communication. Another striking discovery was that the entire PVE process was quite time consuming, and took more time than expected, involving the co-design of the interview guide, collaboration with local teams, interview duration, and video editing. Also, it's essential to acknowledge that the involvement of country teams in selecting participants and projects might introduce potential selection bias, which needs careful consideration during the evaluation process.

Questions for the future: Is PVE scalable and fit for IOE Evaluations?

Looking back at the successes and challenges from the pilots in Brazil and Peru, we can't help but wonder:  is this approach suitable and scalable for IOE evaluations? The potential use of PVE holds exciting possibilities, like using video interviews for communication materials and adding value to traditional evaluations. But we must also tackle the challenges head-on, striving for inclusivity and accuracy. Offering remuneration for participants to acknowledge their time and cover expenses related to transportation and internet access, could be a step in the right direction. As we embrace the participatory nature of PVE, we need to find ways to make it scalable while ensuring meaningful and impactful involvement of local communities. Moreover, the role of PVE in enhancing KM practices within IFAD is also worth exploring, as it could potentially foster greater engagement with indigenous and local knowledge and bring evaluation closer to the heart of the communities it serves. By digging deeper into this remote methodology's potential, we can pave the way to more inclusive and insightful evaluations, amplifying the voices of local communities and gaining a deeper understanding of development efforts.

In conclusion, the pilots in Brazil and Peru have shown that engaging with local and indigenous communities effectively through a remote setting is achievable. Despite some challenges, which should be addressed, our experiences with PVE revealed the power of open and informal conversations, allowing us to capture the essence of individuals and truly hear their voices. Local communities were enthusiastic about sharing their knowledge, and PVE could be valuable tool for inclusive and informed evaluations. Let these insights inspire us to further embrace participatory methods, empowering communities and fostering meaningful connections as evaluators.

If you would like to know more about the video pilots or to provide us with comments and feedback, please reach out to Johanna Pennarz ([email protected]) and Martina Pluviano ([email protected]) and Isabela Souza ([email protected]).