Evaluation of IFAD's technical assistance grants programme for agricultural research - IOE
Evaluation of IFAD's technical assistance grants programme for agricultural research
1. The Strategic Framework for IFAD 2002-2006 (SF) identifies "Improving Access to Productive Natural Resource and Technology" as one of the three strategic objectives to enable the rural poor to overcome their poverty. In the context of high pressure on land and water resources and the choice faced by poor farmers to restore land fertility or migrate to cities, it states that:
Appropriate technologies and research to improve farm productivity by boosting returns to land and labour are essential if the former choice is to be a viable option. As solutions are often context-specific, technologies need to be developed through appropriate research and validated working together with the rural poor –something that is still quite rare. Full appreciation needs to be given to the existing risk-management strategies of small farmers. These often differ for men and women farmers, requiring gender-differentiated approaches (SF, page 10).
2. The importance of agricultural research for rural poverty reduction has been recognized by IFAD since its inception. Financing agricultural research on a grant basis as part of IFAD's technical assistance programme was explicitly mentioned in IFAD's Lending Policies and Criteria. At that time the agenda was defined very broadly. IFAD's programme of technical assistance grants for agricultural research (AR/TAGs) was established in 1979 to provide grant support to the international agricultural research centres (IARCs) and, through them, to the national agricultural research systems (NARS). In all, a total of USD 171.5 million has been allocated for 199 grants from 1979 to December 2001. Traditionally, such grant recipient organizations are made up of two groups: centres of the Consultative Group on International Agricultural Research (CGIAR) and non-CGIAR centres1.
3. Goal and objectives of the programme. IFAD's approach in supporting agricultural research is embodied in a number of Executive Board policy documents between 1979 and 1991 and an internal 1997 document. In all of these documents the objectives and coverage of the programme remained broad, without clear prioritization. This has led to a wide-ranging interpretation of the role of the programme in-house and by IFAD partners. The linking of AR/TAGs with IFAD projects to enhance their poverty impact has always been central to the programme. There has been consistent emphasis on the contribution of grant-financed research to the technological base of the IFAD investment programme. The overall goal of the AR/TAG programme is seen as contributing to the reduction of rural poverty through the following means. These can be considered programme objectives, although they have never been stated clearly as such in a policy paper:
- develop and adapt appropriate and sustainable technologies within a reasonable span of time in support of resource-poor farmers and the rural poor;
- promote IFAD's partnership with IARCs so as to influence their agenda towards pro-poor research;
- strengthen the capacity of these institutions and NARS for pro-poor research and training;
- support technology-related socio-economic research to ensure relevance and sustainability;
- generate knowledge and information on appropriate agricultural technologies and practices.
II. The Evaluation
4. There had been no comprehensive evaluation of the AR/TAG programme in the more than two decades of its operation. This evaluation was requested in the context of the formulation of an IFAD policy for the use of grant funds. The evaluation's main objectives are to: (i) assess the achievements of the programme in relation to its objectives; (ii) analyse main trends in AR funding and the current relevance of the programme to IFAD's strategy and priorities; (iii) identify and analyse factors that have affected the programme's operations and likely impact; (iv) provide recommendations for future orientation of the programme and building blocks for articulating a strategy of grant resource allocation.
5. The evaluation methodology. The evaluation was designed to be participatory, involving recipient organizations and IFAD staff. The evaluation process was both desk and field based. It adopted a four-pronged approach: (i) a desk review of all available documents for a sample of 42 grant programmes (mostly closed) involving 67 TAGs out of the 199 approved between 1979 and 2001 (i.e. 34% of the total); (ii) discussion with a range of IFAD staff, particularly those involved in AR/TAG processing and management; (iii) a formal survey of recipient institutions (31 institutions were surveyed and 25 responded, i.e. an 80% response rate); and (iv) field visits to nine selected grant recipients in Africa (ICIPE, ICRAF, IITA, ILRI), Asia (IRRI), the Near East (ACSAD, ICARDA) and Latin America (CIP, IICA) and to some cooperating NARS.
III. Overview of the Programme
A. Programme History and Development
6. During the very early years, the programme's research emphasized commodities and was heavily food-crop oriented. In these years, IFAD supported CGIAR centres2. with the aim of adapting the technology thus created to the needs of resource-poor smallholders, and of influencing the research agenda of the CGIAR system towards resource-poor, smallholder production systems. The early emphasis on commodities soon gave way to a progressive shift towards farming systems and related socio-economic research and sustainability issues. In time, the programme became more end-user oriented and aware of the location-specific nature of pro-poor research and it widened its institutional coverage.
7. Over the years, the programme itself became more systematized. A set of formal guidelines for AR/TAGs were first prepared in 1997. The programme was seen as an instrument that "focuses on the development, through applied and adaptive research, of innovative and effective means to eradicate rural poverty"3 These guidelines and the associated internal screening processes represented an attempt to direct the programme in accordance with IFAD's strategic focus and priorities and to make it more useful to the Fund's loan portfolio. In mid-2000, grant screening and selection procedures were strengthened and applied within a competitive grants system, based on scoring against specific criteria, to identify the most appropriate research and training grant ideas for entry into the pipeline4.
B. Allocation of Programme Funds
Box 1: Twenty-two Years of AR/TAG Operations
IV. Evaluation Findings
A. Research Priorities of the Programme
8. The evaluation found large differences of opinion about the research priorities of the programme. These are evident not only among grantee institutions, but also within IFAD. At present, technical, methodological, institution building and, to a much smaller extent, policy priorities appear to coexist. A clear policy and strategic framework, pinpointing the priorities, is still to be provided. Selection of TAGs for screening seems to occur on a case-by-case basis, with chance and personal factors still playing a role. Some grants are clearly supply driven and others are initiated by IFAD. In many cases an interaction takes place, and often a reconciliation between the research agenda pursued by the applicant institute and that of IFAD. Within IFAD, there are clear differences among regional divisions in agricultural technology and research priorities, some matching the existing AR/TAG selection criteria better than others6. At the grantee level, IARCs claim that the TAG priority-setting process has usually involved consultation with farmers, as well as diagnostic surveys with a multidisciplinary team, but such evidence is often lacking in reports.
B. Trends in Approach
9. Positive trends are evident in the evolving research approach of the programme from 1979 to the present. The grants approved show increasing concern for poverty, environmental sustainability and major production systems in arid and semi-arid regions of the world. This trend has been gradual and the 1997 guidelines for AR/TAGs served to legitimize these trends rather than breaking new ground.
Box 2. Trends in the Research Approach 1979-2001
C. Research Design
10. The large majority of reviewed AR/TAG Executive Board proposals are well designed in terms of rationale, objectives, description of main components and core research. Recently, a few grant design documents have attempted a logical framework presentation. The most common weaknesses are: (i) overambitious design in terms of numbers of countries and objectives; (ii) lack of reference to relevant earlier TAGs; (iii) absence of information on implementation capacity, particularly regarding national partners; (iv) inadequate or missing description of monitoring and evaluation (M&E) arrangements; and (v) lack of clear guidelines for impact assessment. Proposals of CGIAR applicants were usually of a better quality than those of non-CGIAR. About two thirds of the TAGs reviewed were found to have made fairly important changes in the TAG design during implementation, only some of which have a clear justification. These appear to have occurred because of an incomplete match between the IFAD research agenda and the institutional agenda of the grantee.
D. Relevance of the Research
11. The objectives of TAGs, as stated in Executive Board proposals, have been relevant to the needs of the rural poor, to the AR/TAG programme, and to IFAD regional priorities and strategies. Some 86% of these proposals had stated goals and objectives with clear poverty relevance. But the poverty relevance of outputs is much lower, in part because of above-noted changes in focus during implementation. The documents reviewed suggest that only about 60% of completed technology outputs can be described as clearly appropriate to the rural poor. The gap between the relevance of objectives and the relevance of outputs is mostly caused by an insufficient attention to the livelihoods and constraints of the poor and to insufficient farmer participation in determining research priorities. However, the evaluation noted a marked improvement in this aspect over the years through the increasing integration of socio-economic research and farmer participation. Implementation capacity constraints at the national level have also been a cause.
E. Research Partnerships
12. NARS7, particularly government agricultural research institutions (GARIs), remain the main research partners of IARCs. The institutional survey results show that this is more the case with CGIAR than with non-CGIAR centres (100% for the first group versus 56% for the second). With few exceptions, in most countries this partnership is often strained by financial and capacity constraints of NARS. They have little cofinancing available for implementing project activities and are often short of human resources. IARCs also noted GARI's capacity weaknesses, inadequate reporting and accounting performance, poor facilities, lack of long-term plans, and limited social science expertise. These weaknesses have hampered research implementation.
13. CGIAR centres were found to have established more working relations with non-governmental organizations (NGOs) and extension systems than the non-CGIAR centres. Overall, NGO partnerships are increasing, particularly for facilitating farmer involvement in on-farm research and for technology dissemination, but CBOs appear to be less involved. NGOs were praised as partners. But there is far less evidence of active NGO partnership in the TAG reports than is claimed in the survey responses. Most research tries to work with extension systems, which occasionally benefited from grants and training activities. These systems also collaborated in research, particularly farmer-participatory field testing. However, extension partners were found to be underfinanced, with increasing problems of staffing, incentives and mobility. Overall, the focus on multidisciplinary research has led to a search for new partners that can provide socio-economic research expertise (e.g. universities, social science research institutes and NGOs). The emergence of the private sector as a strong player in the field of agricultural research calls for innovative forms of partnership that safeguard the interests of the poor. The evaluation also demonstrates that there are cross-fertilization benefits in promoting cooperation between NARS themselves in multicountry TAGs.
14. IFAD was instrumental in promoting steering committees (SCs) as a mechanism for TAG governance. Both the survey and report reviews indicate that they are now used by a large majority of TAGs, often in combination with workshops8. CGIAR centres appear to have an almost unanimous belief in the value of SCs for purposes of review, planning coordination, monitoring, ensuring transparency and developing a sense of ownership of the activity. IFAD has almost always been a member of the SCs of its supported research, together with other implementation partners. Overall, the SC mechanism has worked well for field coordination and management of AR/TAGs.
F. Grant Linkages with IFAD Loan Projects
15. Linking the AR grant portfolio to the loan portfolio, to enhance IFAD's investment projects, has always been central to the AR/TAG programme. IFAD loan projects were expected to use technology developed by the AR/TAGs to increase their impact on rural poverty reduction. The evaluation found that such direct linkage is more likely to happen with downstream, farmer-participatory research, which produces poverty-appropriate technology of visible benefit to and attraction for farmers. Longer-term research will usually have a time-lagged and indirect input. Some IARC respondents to the institutional survey understood the concept of linkages in a much broader sense, to include activities that would set the stage for such linkages to eventually take place (e.g. on-farm technology testing in an IFAD project area or IARC staff joining an IFAD project mission).
16. Fully 78% of Executive Board proposals for AR/TAGs named specific IFAD loan projects that "would benefit" (or similar wording) from the TAG. Forward, parallel and backward types of linkage expectations were found. However, very rarely were linkages with IFAD loans mentioned in the original IARC proposals. In a few cases, memoranda of agreement were attached to the Board proposal providing details of such linkages.
17. Evidence of achievement of linkages was much lower. Among the reviewed TAGs that had anticipated linkages, only 46% had evidence of any form of linkage, even when considerable latitude of definition was allowed (i.e. only 36% of the TAGs reviewed could be considered successful in achieving linkages). Where multiple phases were involved, linkages were seldom achieved during the first phase. Research reports, including those of supervision, rarely dealt with the question.
Box 3. Reasons for Weak Linkages of AR/TAGs with IFAD Projects
G. Achievements of Research Objectives
1. Overview of Achievements and Constraints
18. Achievements. In line with programme objectives, the achievements of the agricultural research TAGs are not restricted to technology outputs. Nevertheless, the majority of TAGs reviewed stated their objectives in terms of technology development. The evaluation found that TAGs performed less well in terms of achievement of stated objectives than of design or relevance. About 60% were found to have achieved most or a good part of their objectives, about one third achieved some of their objectives and 10% achieved little. CGIAR-implemented TAGs were much more likely to have satisfactory or higher performance than the non-CGIAR ones9. There were no clear regional trends. Although some IARCs appeared less effective than others, definite judgement on institutional performance could not be made on the basis of the sample, which does not include all TAGs implemented by all IARCs. In some instances, there were major differences in performance between different TAGs implemented by the same grantee.
19. Research time horizon and other constraints. From its inception, the AR/TAG programme has stressed achievement of outputs over the short to medium term. During implementation this has not always been the case. In the early stages of the programme, multiphased grants were more common, with IFAD supporting several longer-term research programmes. Even in later stages, delays in producing research output in the short term (three years) have also been observed, requiring follow-up TAGs. A number of explanations can be given. Agricultural research is, inherently, a long-term process. In addition, production systems of smallholders in unfavourable environments, for example in rainfed areas of Asia, Africa and the NENA region, are by nature complex and difficult to address. Livestock, agroforestry, and some other types of research supported by the programme need a longer time horizon because of life-cycle factors. Where NARS are weak, more time is needed initially for capacity-building. Farmer-participatory research also takes extra time but is essential for generating appropriate outputs. These factors often make the search for technically, environmentally, socially and economically sustainable production technologies a medium-to-long-term task.
20. The above arguments notwithstanding, producing short-term output proved feasible when research projects supported a slice of an established research programme, building on accumulated knowledge from past research. Such TAGs must therefore carefully choose the entry point for IFAD support, making sure that it is towards the end of the research programme, preferably at the technology validation stage. However, research restricted to this shorter-term focus can be restricted in terms of innovativeness and relevance as well. There could be, therefore, a trade-off in the choice between a short-term approach to produce immediately usable output and longer-term research. Better correspondence was observed between short-term output expectations and research approaches in recent years, due to the increased downstream nature of IFAD support. Other constraints faced by TAGs in the achievement of objectives include: (i) over-ambitious technology objectives at the design stage; (ii) inadequate capacity (especially among NARS); (iii) funding shortfalls (e.g. where cofinancing did not materialize, costs were underestimated, or inadequately allocated); (iv) climatic factors that delayed technology testing and therefore completion of outputs; (v) difficulties in partnering with NGOs and CBOs or establishing linkages with extension systems.
2. Technology Outputs of Different Types of Research
21. Short-to-medium-term crop research on established crops. A good part of the programme-financed crop research has been short or medium term, building on earlier research and products in established food crops, especially in grain crops such as rice, wheat, maize and grain legumes. Frequently such research uses new and improved varieties to develop and refine location-specific technological packages and associated management practices. Sometimes this research has built on indigenous knowledge and technology rather than, or in addition to, building on scientific research. This has usually resulted in small improvements in existing crop and livestock management practices (such as planting time, spacing arrangements, integrated pest management (IPM)), but which can be easily adoptable and result in significant improvements in subsistence crops for smallholders.
22. Longer-term crop research on established crops. The programme also financed long-term research (multiphase, multiyear) for established crops, as in the case of support for rainfed rice by IRRI and WARDA and for the faba bean by ICARDA. In the latter case (TAG 1-ICARDA), the first two phases of research generated new varieties and some component technologies (i.e. weed control, fertilizer application, pest management, with research conducted on-station). Later, some integrated production packages were developed in farmers' fields. The third phase developed linkages with development projects to accelerate dissemination. The ICARDA faba bean research (the first research grant in IFAD) was a pioneering grant for multidisciplinary and participatory research, with many technology outputs and positive nutritional impact on the poor. Its main weakness from IFAD's perspective was that technology outputs generated were too input-intensive to be used by the poorest farmers.
23. Research on ‘neglected' crops. IFAD has taken a lead in mobilizing interest in and donor support for research on some important ‘neglected' food crops of the poor, with notable success, especially in Africa. Research on plantain, bamboo and rattan, and cassava are examples. For instance, cassava research has been supported over the entire lifespan of the AR/TAG programme, with a range of technology products generated along the way. These have included: improved cassava varieties, highly cost-effective biological control technology of two major cassava pests, transfer of improved cassava varieties from Africa to Latin America, and development of a global cassava policy. The impact of long-term involvement in the cassava programme on the rural poor in Africa has been remarkable (see Box 6, page 11).
24. Integrated crop pest-management research. Since the early 1990s, the AR/TAG programme has financed several TAGs for development of IPM technologies in Asia, Africa and Latin America. Some of these have generated a range of outputs, including adapted new varieties, IPM technology, crop management practices, and methodological adaptations of the farmer-field-school methodology. The sample TAGs reviewed on IPM research suggest on average a four-to-five-year research period for generating outputs. Resolving production and legal issues as well as collective action at the community level can take a longer time and need more initial attention under IPM TAGs.
25. Livestock research. Four important areas of TAG livestock research have been supported: (i) improvement of quality and quantity of livestock feed; (ii) breed improvement and reproduction; (iii) pest control and disease management (development of control measures and surveillance systems); and (iv) crop/livestock integration. Most of the livestock feed-related research has focused on developing low-cost alternative feeding strategies, improving quality of crop residues and promoting forage, especially legume forage, in rotation10. Much of this research has revived old technologies that were ‘gathering dust on the shelf,' adapting them to the needs of smallholders. TAG-supported livestock research produced some worthwhile output despite the acknowledged challenges of livestock as compared to crop research in semi-arid areas. Research on livestock faces diverse and complex constraints related to the longer life cycle of animals, the intricate role of livestock in the livelihood strategies of the poor, its linkage with natural resource degradation and the specific role played by poor rural women in the livestock subsector. The evaluation noted the increasingly better integration of socio-economic studies into recent livestock research and the firmer rooting of this research at the rural community level. This was associated with progressive improvement in the quality of research output in livestock TAGs.
26. Research on commercial insects. Since 1995 IFAD has provided support to ICIPE's Commercial Insect Unit (TAGs 308 and 491). Some of this research has built on traditional knowledge and practice to generate improved technologies for sericulture and apiculture for African farmers. It is one of the few TAG research areas that have been effective in generating post-harvest technologies as well as production technologies.
27. Fisheries research. These TAGs have been limited to inland fish farming, primarily in Bangladesh. This cluster of grants to ICLARM shows a clear learning curve in understanding the relevance of technology to poverty, as well as in linking grants to IFAD loan projects.
28. Research on NRM. This highly relevant body of research has focused on generating technology outputs for water, soil and agroforestry, mainly in NENA and West Africa. The specific outputs have included technologies, strategies and policies for conserving water and for the management and rehabilitation of pasture and rangeland. Agroforestry research has been comparatively less successful to date in completing appropriate technology outputs – one reason being that it requires a long lead-time, at least five years or more, and skills for intensive multidisciplinary work at the community level, which not all IARCs possess. Recently designed agroforestry TAGs have integrated past lessons learned, and serious efforts are being undertaken by IARCs to widen research partnership and integrate socio-economic issues at community levels.
Box 4. Lessons on the Generation of Technology Outputs
Overall the programme has been reasonably effective in achieving the technology generation objectives. Crop and cropping-system research began early in the programme and benefited from accumulated knowledge and lessons. Livestock and NRM research face particularly difficult challenges in the context of poor rural communities, but notable improvement has been observed in tackling these issues in the more recent TAGs. Lessons to be noted are:
H. Poverty Impact of Agricultural Research
29. Agricultural research can have a major impact on poverty; the difficulty lies in proving such impact on a case-by-case basis. In the case of the AR/TAG programme, this is made even more difficult by a lack of relevant data. In the review sample, evaluations were available for less than a quarter of the sample TAGs11.The survey results argue that more studies have actually been done, suggesting missing reports or delayed impact evaluations (after TAG completion). Time and budget limitations and lack of clarity among grantees on what kind of evaluation IFAD expects have been some constraints. Two main approaches were used by TAGs for impact assessment: technology adoption studies and economic impact assessment, which measures economic rates of return of research. Very few TAGs were able to go beyond these definitions to estimate the research impact on rural poverty.
30. The available data confirm that attribution of poverty impact to agricultural research is complex and based on a number of assumptions. It occurs indirectly, through the impact of research on agricultural productivity and through the effect of productivity changes, on a variety of other economic and social aspects at micro, sectoral and macro levels. The issues are as follows:
Box 5. Difficulties in Assessing the Poverty Impact of Agricultural Research
31. To compensate for the lack of data, the evaluation adopted a methodology for assessing the potential for poverty impact based on "appropriate products/available dissemination mechanisms". It defines TAGs as having poverty impact potential if they can meet the following four impact conditions or proxy indicators of impact:
- Impact condition # 1: usable technology outputs have been completed.
- Impact condition # 2: outputs are appropriate to resource poor farmers.
- Impact condition # 3: there are no major constraints to dissemination.
- Impact condition # 4: linkages have been established with a system for dissemination.
32. The evaluation checked the above-mentioned conditions as a proxy for poverty-reduction potential in the reviewed sample. Less than one third of the reviewed grants fulfilled these conditions. However, the evaluation would like to note that this figure has to be interpreted with care as only a rough proxy indicator. TAGs were more likely to meet the second condition – that of appropriate technology – if farmers had participated in the research. Notwithstanding the above findings, there are several cases of research TAGs in which the poverty impact has been established unambiguously (e.g. research on cassava, faba beans, rice, potatoes, plantain and some others).
33. Recognizing this scarcity of impact data and the methodological lacunae in the field, in recent years IFAD has emphasized impact achievement and assessment in its support to agricultural research. It has contributed to the development of impact evaluation methodologies that have become an important input for the CGIAR Special Programme on Impact through the Standing Panel for Impact Evaluation. IFAD is supporting this initiative to develop methods that identify the necessary conditions for favourable impact of agricultural research on the poor and determine the best methods for assessing this impact12.
Box 6. Poverty Impact of Agricultural Research: Biological Control of the Cassava Mealybug (TAGs 36, 136-IITA)The TAG research for biological control of the cassava mealybug (CMB) is a good illustration of the potential impact of agricultural research. Cassava is the staple crop of 200 million Africans, primarily the poor. In the 1970s a new pest, the CMB, began to devastate cassava fields throughout Africa and threaten the food security of millions. TAG 36-IITA supported strategic and applied research that identified a natural enemy of the CMB – a tiny wasp from Paraguay – as the control solution. After careful study, it was disseminated in Africa in the 1980s. Thus the solution did not involve the use of expensive pesticide inputs by the poor, but was essentially ‘free'. Under two phases of TAG 136, coverage was expanded throughout cassava-producing countries in Africa with excellent results. By 1994, some USD 27 million had been spent on CMB control. The benefits to poor farmers whose fields had been saved was estimated at USD 4.5 billion, or more than 160 times the cost of the control measures13. Others have estimated the benefit-cost ratio at 149:1.
Some of the factors that lead to the successful impact of this research:
I. Institutional and Policy Impact of Agricultural Research
34. The evaluation recognizes the important policy and advocacy role performed by IFAD through the AR/TAG programme. Together with other committed donors, IFAD has strongly advocated the poverty focus of the CGIAR system, became a founding member of the Global Forum on Agricultural Research (GFAR), and is playing an important role in the CGIAR Special Programme for Impact Assessment (see paragraph 33).
35. The role of the programme in promoting pro-poor research became particularly clear through the institutional survey and visits. IFAD efforts in this respect succeeded in sharpening the focus on poverty-related research in the work programme of the CGIAR system. In recognition of this role, in 2002 IFAD was invited to become an official co-sponsor of the system. It has also taken the lead in funding certain areas of poverty research (e.g. neglected crops) and has catalysed other donor support. At the regional or national levels, about one third of TAGs in the sample had important policy or strategy objectives. Many of these actually achieved some degree of policy impact. Some newer TAGs are explicitly building in activities for policy dialogue and influence.
36. IFAD's advocacy role in the creation of GFAR in 1996 is worth noting. The Fund was a lead agency in the global effort that led to its establishment, and chairs its support group to mobilize the international donor community around the GFAR agenda. GFAR aims at facilitating cost-effective partnerships and strategic alliances in research to reduce poverty and food insecurity and to conserve and manage biodiversity and natural resources. Main aspects of the GFAR agenda are: demand-driven research implemented through equal partnerships among stakeholders; strong farmer perspectives in setting the research agenda, taking into account the regional heterogeneity of farming systems; and effective involvement of the intended users (poor farmers) in research design and technology diffusion.
37. The AR/TAG programme has also had a positive impact on institutional capacity, at least in the short term and particularly at the NARS level. Almost all IFAD-financed TAGs have been engaged in NARS capacity-building, particularly for GARIs14. The evaluation findings suggest that at least 50% have achieved a significant impact at this level, with most of the remaining TAGs also having a positive impact. Capacity-building at the national level has consisted of short-term training on research through workshops or short, focused courses on technical subjects and methodological topics (e.g. impact assessment, farmer-participatory research and gender issues). Sometimes extension staff have been trained as well as researchers15.
38. However, several respondents to the institutional survey highlighted the difficulty of achieving field-level impact from training because of the serious financial and human resource constraints of the large majority of NARS. There are a number of factors involved, such as the high level of turnover in some countries of national staff (including those trained by TAGs), and the lack of equipment and of travel budgets for field research. Other types of capacity-building activities have included financing of higher degrees for researchers; provision of laboratory equipment or other essential capital investments needed for research.
J. Knowledge Impact of Agricultural Research
39. The review found that almost all TAGs had generated lessons worth disseminating. Actual dissemination of those lessons, and therefore impact on knowledge diffusion, still needs strengthening. Often the knowledge generated by TAGs has only been disseminated to a small group of ‘network' researchers and has not reached the larger development community, farmers or a wider circle of IFAD staff. Technical advisory notes (TANs) are potentially very useful tools for the dissemination of TAG research findings to a wider development audience, both inside and outside IFAD. Since 1998 some 55 TANs have been produced. Of these, 15-20 are ready for posting on the IFAD TAG subsite. TANs are short and user-friendly, and PT is taking measures to increase both the number and quality. Other methods used by TAGs to disseminate information include scientific journals, conferences and sourcebooks, dissemination through networks16 and through IARC information centres.
K. Programme Management
40. Management of the AR/TAG programme has undergone major improvements in recent years. Historically, the programme was entirely managed by IFAD/PT. In earlier years grant selection decisions were centralized, with little involvement of regional divisions. More recently there has been a conscious effort to address this situation, and to involve regional staff in TAG initiation and management, in order to ensure that financed research is in line with regional strategies and research priorities, and to enhance linkages with IFAD projects17. Steps have included: (i) the PD guidelines for AR/TAG, implemented since 1998, which introduced, inter alia, the possibility of TAG initiation and management by regional divisions; (ii) new TAG screening criteria and procedures, developed by an in-house task force in 2000, with immediate application; (iii) the institution of TANs to generate and disseminate learning from the programme, which have been very well received by IARCs and other partners; and (iv) annual reviews of ongoing grants, starting in 1999, which perform important functions, e.g. feedback to grant recipients and reporting to IFAD's Executive Board.
41. Despite the significant progress noted above, there is still room to enhance efficiency. Overall, IARCs commended IFAD's management of TAGs and the professionalism of the staff involved. While fully acknowledging improvements introduced by IFAD over the years, they and IFAD staff highlighted the following areas where further improvement can be made: (i) streamlining and systematizing the approval processes to increase efficiency; (ii) providing more comprehensive guidelines for progress and completion reporting, including a unified reporting format and some guidance on expected evaluation; (iii) speeding up disbursement processes; and (iv) providing practical means to improve linkages of TAGs with IFAD investment projects. Supervision of TAGs continues to be variable both in frequency and quality. Funding constraints are a major reason. One solution used by several TAGs has been to include funding under individual TAGs for supervision purposes, but difference of opinion exists on this practice.
V. Main Conclusions and Recommendations
A. Overall Performance of the Programme
42. The AR/TAG programme has now been operating for more than two decades. During that time, IFAD has played a strong advocacy role in redirecting the focus of the CGIAR system towards more poverty-focused research, taken the lead in opening up new research areas, and continued to play a pro-poor advocacy role in a number of international forums related to agricultural research. The programme has achieved several well-known successes in agricultural research. There have also been some less-publicized failures. The majority of TAGs fall somewhere between these two extremes. TAGs are usually well designed, and overall they have been reasonably effective in achieving stated objectives. Due to unavailability of impact assessment studies, the impact of most TAGs on poverty cannot be rigorously established. TAG impact on establishing effective partnership for research and strengthening national research institutions appears to be highly positive, though sustainability cannot be rigorously verified.
43. Overall, the programme remains relevant to the IFAD poverty mandate and its current SF. It has progressively focused on enabling the rural poor to access appropriate technology for improving their livelihood and on the technical and socio-economic needs of those living in ecologically fragile environments. Better use of farmer-participatory research and multidisciplinary approaches has helped, but more can be done to involve farmers and CBOs in setting research priorities. As the programme has increasingly included new research areas, it has become too diffuse in its focus. Supply factors and individual interests still play too large a part, and IFAD has yet to provide coherent institutional direction for the programme and better means to link it with its loan projects.
44. The overall programme impact is clearest in terms of institutional capacity-building, both at the IARC and NARS levels. Its poverty impact is least easily pinpointed, in part because linkages with IFAD loan projects have rarely succeeded and because adequate impact evaluations are not systematically undertaken. Impact on knowledge is generally agreed to be much weaker than it could and should be, and yet is probably the easiest to improve. TANs are a good step in that direction. Room for improvement exists with respect to the programme's efficiency, in particular proposal review procedures and implementation. Traditionally, the programme focus has been shared between CGIAR and non-CGIAR centres, while distribution of grant resources between institutions within these two categories has become highly uneven. Programme efficiency and systematization of procedures can be further improved.
B. Conclusions on Programme Policy, Strategy and Procedures
45. The AR/TAG programme needs a clear strategy, priorities and better focus. While grant approval processes have received considerable attention, the technical or research subject matter priorities have not. The programme is attempting to cover too many areas within a framework of zero growth in resources. In addition, basic issues, such as the main goal and objectives of the programme, research priorities, longer versus short-to-medium term research, and upstream versus downstream focus are still not quite clear. Some regional divisions have elaborated regional research strategies that guide their own TAG selection, but these are not positioned in a well-defined institutional policy/strategy of support for agricultural research, and will benefit from further focus and prioritization. A ground-level policy and strategy discussion is needed to determine what IFAD research priorities are, given the new strategic framework, how these can be linked to regional priorities, and how the programme can address them effectively.
46. The AR/TAG programme needs to establish a niche in innovative research for poverty reduction. Despite emphasis in all programme documents, including the 1997 guidelines, on "the need to develop through applied and adaptive research innovative and effective means to eradicate rural poverty", innovation has not been a main criterion in assessing grant proposals. The SF and the recent document, Evaluation of IFAD's Capacity as a Promoter of Replicable Innovations, highlighted IFAD's potential catalytic role as an innovator. There is a need for the programme to carve out a niche in the generation of an innovative research agenda. Possible areas include: no-tillage farming; water harvesting; higher-nutrition food crops for the poor; new research partnerships that include emerging actors in the field of agricultural research (e.g. the private sector and NGOs); more-effective integration of the poor in the setting of research priorities and in implementation; and similar topics of relevance to IFAD investment projects.
47. The consistency between resource allocation in the AR/TAG programme and that of IFAD loans should be increased. Ideally, the allocation of resources in the programme should be consistent in thematic terms with the planned allocation of resources in the Fund's loan portfolio. This is important if grants are to address research issues identified by operations and to link with future projects. Such planning is not currently done. Synchronization between TAG and loan programmes (the former preceding the latter chronologically) should be done on the basis of regional and location-specific identification of research needs and the tailoring of AR/TAG programme priority areas accordingly.
48. Programme procedures need strengthening. TAG procedures need to be more efficient. A series of positive moves in recent years culminated in the implementation, in May 2000, of the AR/TAG screening criteria. These have made programme procedures more transparent and participatory. However, the programme needs to further enhance selection rigour and efficiency. Proposal selection criteria and processing procedures need revisiting and further systematization. IARCs have asked for more guidance at all stages.
49. Reporting should be more appropriate to IFAD concerns. Implementation completion reports do not appear to be prepared consistently. Nor are progress, completion or supervision reports comparable in terms of topics covered and adequacy. Impact assessment is not systematically performed. Problems faced during implementation and solutions proposed are not sufficiently discussed, and yet these are among the most informative parts of the report for IFAD, and also for future TANs. Overall, there is a tendency to produce either publicity documents or technical dissertations. Linkages with IFAD loan projects are usually not covered.
C. Conclusions on the Research Funded
50. AR/TAG grants show wide variation in quality. There are some very well-conceived and executed TAGs and some poor ones, with CGIAR institutes performing better overall in quality of proposals, performance and impact. Research proposals need to give more attention to capacity for implementation both at IARC and NARS levels. This also raises the issue of whether IFAD should give priority to the better-performing IARCs and NARS, that is those with a good track-record, and phase out support to those that are consistently low performers.
51. Linkages between TAGs and IFAD loan projects have been difficult to achieve. In the evaluation assessment, this is the weakest aspect of the programme. Limited forward planning, lack of joint grant/loan coordination, unclear research priorities, difficulties in identifying technology needs and delays are frequent causes. Past experience has provided some useful lessons on how to better achieve linkages. There are some good models among the TAGs. More could be done at the TAG proposal and start-up stages to lay a better basis for linkages, particularly by IFAD staff. Loan projects can play a more effective role in enhancing such linkages. They should not be viewed only as a platform for dissemination of TAG-financed research output, but can also provide the field context in which downstream research should be designed and adapted. Encouraging cooperation and exchange of knowledge among country portfolio managers (CPMs), IFAD/PT technical advisers, grant managers and project field staff is essential to the fulfilment of this role. Supervision and reporting have not paid adequate attention to the linkage question.
52. There has been a general trend in the programme towards more multidisciplinary, multipartner and participatory research, which has been accelerated in recent years. This is in line with IFAD's SF and priorities, and ensures better impact of research on poverty. Although social aspects are becoming increasingly well integrated, economic and policy issues need more attention. Cost-benefit analysis, initial policy-constraint analysis and policy dialogue need greater focus in the future. Scope for improvement exists in increasing the participation of farmers and CBOs in determining research priorities and in providing insights into traditional practices and innovations. CGIAR and some non-CGIAR centres are gradually developing the necessary capacities.
53. Some TAG projects are moving beyond technology validation to technology dissemination activities. While in some ways, this is desirable, it also raises strategic issues. At the IFAD level, there is danger of overlap between AR/TAG focus and activities and those of the IFAD/NGO Extended Cooperation Programme and, indeed, those of the IFAD loan programme. Replacing national research and extension systems in some of their activities is another danger. At the level of IARCs, there is the question of value added and the match between such research and the skills of grant recipients.
54. There is a danger of research achievements and impact being undermined by the narrow time frame. The very large majority of research activities financed have a duration of three years, even though IFAD Executive Board documents allow three to five. Extensions, funding of subsequent phases and sometimes small grants are occasionally used to ‘patch up' grant projects in order to allow them to achieve their objectives. This approach is inefficient and is advantageous neither to the grantees nor to IFAD. Apart from capacity constraints, it suggests that research activities, and particularly the newer impact-oriented ones, need a longer implementation period than the usual three years. A longer time frame of four to five years would result in better research, particularly where life cycles are long or considerable initial capacity-building or background social research is needed. It would also permit a realistic assessment of impact.
55. The programme has made a good contribution to capacity-building, particularly in participatory methodologies and poverty-oriented research at the national level. But in spite of efforts made to help NARS, there are a number of constraints on longer-term impact: the generally weak financial situation of NARS and staff attrition and rotation. IFAD needs to better recognize the implications of these constraints. If capacity-building is to be a major objective of the programme, then adequate time and funding should be allowed for this purpose under TAGs. If production of output is the overriding goal, then the programme should be more selective of national partners, favouring those with existing capacity.
56. Too little is known about the poverty impact of individual TAGs or clusters of TAGs. This argues for making impact evaluation a more important part of TAGs, with expectations laid out clearly at the design stage, adequate time, and funding earmarked for the purpose.
(a) Developing a Policy and Strategy for IFAD's Support to Agricultural Research
The programme has not had an agreed-upon strategy for guiding IFAD's contribution in the area of agricultural research. Preparation of a research strategy for the programme will first need to feed into, and then build upon, the output of the new task force that will recommend a general policy and strategy for IFAD's grant programme. Preparation of the research strategy should therefore build on the following axes: (i) IFAD's SF for 2002-2006; (ii) the new IFAD strategy for grants in general, and synergies between research grants and other grant lines; (iii) programme experience and lessons as captured by this evaluation; and (iv) regional technology gaps and research needs, as articulated by regional strategies. The strategy would need to cover:
- programme goal and general objectives;
- link with the IFAD SF;
- research focus of the programme, types of research IFAD should and should not finance, including extent to which it should be strategic or downstream, and respective time horizon;
- specific thematic priorities or technology gaps that have greatest importance during 2002-2006 (to be reviewed periodically);
- expected emphasis on innovative research;
- expected linkages between AR grant programmes and IFAD's loan portfolio;
- broad spectrum of partnership in setting research priorities;
- range and types of organizations the programme should support;
- relative importance to be given to agricultural research for technology development, as compared to socio-economic and policy research;
- relationship between grant-funded and loan-funded research;
- complementarity and linkages of AR/TAGs with other IFAD grant lines (particularly the Extended Cooperation Programme and small grants);
- expected role of farmers, CBOs and local participatory processes in setting research priorities and in implementation of research programmes; and
- guiding principles for generation and dissemination of knowledge from TAGs.
(b) Strengthening the Linkages between Grant-Financed Research and the IFAD Investment Programme
- Establish a system for joint loan-grant planning that would also strengthen communication between IFAD CPMs, grant coordinators, grant managers and the loan project staff concerned;
- synchronize grant/loan implementation;
- define the role of IFAD projects at the field level in forging linkages with grant-financed research;
- prepare/finalize divisional agricultural research strategies for both loan- and grant-funded research, with clearly identified priorities;
- establish a corporate-access database for the programme that is not limited to closed and ongoing grants, but includes pipeline applications in order to ensure greater transparency and to inform the joint planning process; and
- share information on technology outputs of the programme more widely through TANs on the IFAD website, as well as through other information networks and dissemination mechanisms.
(c) Enhancing the Poverty and Institutional Impact of the Programme
- Increase TAG duration to up to five years, as indicated in policy documents, to allow initial time for situational assessment and post-research time for impact evaluation;
- systematically include farmers, CBOs and NGOs as effective partners in setting research priorities and implementing research programmes;
- direct greater attention to both assessment of national capacities and further building of capacity for participatory research;
- systematically evaluate the impact of all TAGs, with earmarked funding for the purpose and agreement on indicators, including measures of utilization of grant outputs by IFAD investment projects; and
- identify consistently low IARC performers and determine steps to be taken.
(d) Improving Internal Processes and Procedures
- Further systematize grant review and selection procedures to enhance transparency, ensure fair competition among applicants and assign appropriate weight to innovative research;
- conduct better reviews of final proposals, particularly of institutional arrangements and capacity, M&E arrangements and research budget;
- review the impact of the 2000 screening procedures and processes during their ‘trial' period to determine any need for improvement; and
- provide more comprehensive guidelines to grant applicants and recipients, and for supervision, evaluation and impact assessment of AR/TAGs.
(e) Resources Required Should be Reassessed and Adequate Allocations Made
- The recommended refocusing of the programme should be associated with a reassessment of the financial resources needed within the existing overall resource constraints.
- Human resource needs should also be reassessed, with a view to enhancing the programme's management and coordination, strengthening linkages with IFAD projects, and continuing technical backstopping and quality control. Such assessment requires a detailed analysis of the workload and time budget for IFAD staff concerned and is outside the scope of the present evaluation.
- Adequate resources should be allocated for supervision, and new, more effective modalities examined.
- The decentralization process introduced since May 2000, though highly desirable, needs to be reassessed in terms of its effect on linkages with IFAD loan projects. At the time of the evaluation, none of the new TAGs (post 2000) had been completed and hence could not be included in the assessment.
(f) Knowledge Generation and Dissemination Requires Immediate Attention
- TANs are a positive step in the right direction, but delays in their production need to be addressed. The notes could be fine-tuned to make them more useful to institutions and projects that might wish to consider the technology.
- A system needs to be set up to capture and share the many non-technical but useful lessons being generated on topics such as: institutional partnerships, participatory processes in research, methodologies such as impact monitoring and evaluation, and on transferability, sustainability and technology adoption processes.
1. The ‘non-CGIAR' group is used for classification convenience by IFAD and is not an internationally identifiable group as such.
2. The CGIAR started in 1971 as an early endeavour of the international community to develop a global agricultural research system based on donor funding. This system is currently sponsored by IFAD, the United Nations Development Programme and the World Bank. Starting with a few international centres, it has grown into an association of 58 public and private members that supports a system of 16 specialized international agricultural research centres. The most recently formulated mission of the system (2001) is "to achieve sustainable food security and reduce poverty in developing countries through scientific research and research-related activities in the fields of agriculture, forestry, fisheries, policy and environment". In an effort to maximize the effectiveness of the global research efforts, at the end of 2001 the CGIAR system introduced the global challenge programmes to support high-impact research that tackles issues of overwhelming global and/or regional significance and requires partnerships among a wide range of institutions.
3. Guidelines for Agricultural Research and Training TAGs, Programme Management Department (PD), December 1997 (internal document), p. 1.
4. The main criteria are the following: (a) the grant proposal should address problems and opportunities of high priority to the rural poor; (b) the proposal should address issues and concerns of relevance to the regional strategies and the current and future IFAD loan portfolio; (c) the institution(s) identified should have competence and comparative advantage in the activities proposed; and (d) the technical approach should be feasible and should have potential to deliver medium-term benefits to the rural poor.
5. Research typologies are defined as follows: strategic research – quest for the solution of specific research problems; applied research – application of scientific knowledge to the solution of a practical problem; adaptive research – development of technological packages using solutions to practical problems from applied research; technology validation – on-farm trials to test applicability of technological packages to specific locations/situations.
6. At the time of the writing of this report, two regional divisions' research strategies were available for review: Western and Central Africa, and Near East and North Africa.
7. NARS refer to all governmental and non-governmental organizations involved in agricultural and related research at the national level.
8. But SCs are used far less frequently by non-CGIAR centres (56% compared to 93% for CGIAR).
9. Differences in reporting quality may also be a factor.
10. IFAD, Assessing the Impact of the IFAD TAG Programme on Agricultural Research and Technology Transfer in the NENA Region 1980-1998 – working document, November 2001.
11. Some IARCs emphasize impact assessment more than others. ICARDA alone has produced more than six impact-assessment studies on its received grants.
12. Fourteen country case studies will be completed by April 2003, and a paper synthesizing the preliminary results of the case studies is in preparation. A major feature of these studies is that they go beyond conventional treatment of poverty as solely a matter of income, expenditure, food intake or nutritional status. Drawing on participatory poverty assessments, the studies look at the vulnerability of poor people to various trends and shocks and use the sustainable livelihood framework, thus paying attention to a wide range of capital assets.
13. Main source: L. D. Swindale, Globalization of Agricultural Research: A Case Study of the Control of the Cassava Mealybug in Africa, available at www.worldbank.org.
14. The need for such capacity-building has varied among regions and countries.
15. Some TAGs have trained more than 100 scientists (in one case, 500).
16. About a quarter of the agricultural research TAGs have established networks of one kind or another, usually of researchers and, much less frequently, mixed networks that include researchers, extension agents, IFAD project staff and occasionally farmer association representatives.
17. PT maintains the coordination function. This includes: management of the grants pipeline (processing and reviewing of all grants); Executive Board document preparation and Board presentation; implementation progress reports to the Assistant President/PD and external reporting through the IFAD Annual Report and to various international research forums; liaison with other IFAD divisions on TAG-related matters; and implementation follow-up with TAG task managers in PT and the regional divisions.