Interim evaluation

The Independent Office of Evaluation of IFAD (IOE) conducted a joint interim evaluation of two IFAD-supported programmes: (i) the Rural Financial Services Programme (RFSP); and (ii) the Agricultural Marketing Systems Development Programme (AMSDP) in Tanzania in 2010. In line with the IFAD Evaluation Policy, this interim evaluation was undertaken as a standard procedure in preparation for a possible follow-up phase of the project. The main objectives of the evaluation were: (i) to assess the performance and impact of the programmes; (ii) to generate a series of findings and recommendations that would guide the one joint follow-on programme, "the Marketing Infrastructure, Value Addition, and Rural Finance Support Programme" (MIVARF), which was in the process of being developed by the Government of Tanzania, the African Development Bank (AfDB), the Alliance for a Green Revolution in Africa (AGRA), and IFAD.  

To assist in the evaluation process and to maximize the lessons from the evaluation, a Core Learning Partnership (CLP), consisting of IFAD, Government, and other stakeholders, was established and feedback gained from the CLP during the preparatory mission in March 2010 was incorporated into the evaluation Approach Paper.  The main evaluation mission was conducted in May 2010. Given the advance stage of the design of MIVARF a final evaluation workshop was not held.  Instead discussions were held, based on the evaluation findings and recommendations, between the evaluation team and IFAD and the government about key lessons from the evaluation. These discussions informed and set out the understanding between IFAD and the Government of Tanzania for the Agreement at Completion Point (ACP).

The ACP is the agreement of IFAD Management and the Government to the main evaluation findings and their commitment to adopt and implement the evaluation recommendations within specified time frames. This agreement will be signed by IFAD Management, the Director of the East and Southern Africa Division, and the Government, the Permanent Secretary of the Prime Minister's Office. The recommendations agreed upon will be tracked through the President's Report on the Implementation Status of Evaluation Recommendations and Management Actions.  The role of IOE is to facilitate the ACP process leading to its conclusion.

Main Evaluation Findings

The areas selected for support, rural finance and markets, were and are highly relevant.  Improving farmers' access to financial services and markets has over the last decades been a top government priority and the Government continues to consider these two areas as being crucial in its strategy to boost the relatively modest agricultural sector growth. 

The design of both programmes appropriately used holistic thematic approach addressing issues at macro, meso and micro level. However, the designs underestimated the time and resources required for establishing viable private/cooperative enterprises and did not fully appreciate that there are limits to what governments can do. The programmes targeted vast geographical areas in order to gain experiences from working in different agro-ecological and socio-cultural contexts. This scattered approach over a large geographical area resulted in high delivery costs. The programmes were covering many of the same geographical areas, had obvious linkages and were both implemented by the Prime Minister's, the designs did not adequately explore options for synergies and cost-savings e.g. through joint management arrangements. This was done later on during the implementation phase.

The evaluation finds that both programmes have made satisfactory contributions to the immediate objectives that were defined in the design such as RFSP's support in building management and technical capacity of gMFIs to strengthen their operational and financial performance and AMSDP's success in forming and/or strengthening existing and new producer groups.  The contributions were more significant at the micro/field level than at national level where they had a modest influence on the national policy and institutional frameworks. And at meso level, RFSP's support did not result in strong apex organizations for micro and rural finance.

The combination of operating largely in project mode and the ambitious area coverage reduced the project efficiency and resulted in a situation at project completion where the expenditure on Coordination & Management and Monitoring & Evaluation accounted for 41 per cent of total expenditure in the case of RFSP (against 25 per cent in design) and 17 per cent for AMSDP (against 10 per cent in design).

Both programmes achieved their quantitative targets for establishment of SACCOS and agricultural producer/marketing groups whose services have impacted positively on many of the members in terms of improving household income, crop prices, and agricultural productivity and food security.

Furthermore, AMSDP's investments in road and other market infrastructure have provided thousands of households with better access to markets and higher prices.

While the sustainability prospects of supported infrastructures are found to be reasonable, there are sustainability challenges for several of the SACCOS and the majority of the agricultural marketing groups. While the institutional development of SACCOS and producer groups has been substantial, at programme completion few have reached the stage where they are self-sustaining.

Both programmes introduced innovations and positive experiences that provide an agenda for consolidation and scaling up under MIVARF. For example, thanks to cooperation between the two programmes, a warehouse receipt system was successfully introduced, allowing farmers to postpone their crop sales till prices become more attractive.  Moreover, RFSP introduced the concept of complementing individual membership in SACCOS with group membership whereby a group of poor women could join while converting to individual membership once they had the financial capacity to do so. RFSP's collection and distribution of performance data of the SACCOS was an innovative way of introducing benchmarks and a sense of competition among the SACCOS.  Another innovative aspect in AMSDP was supporting the capacity of producer/marketing groups largely through contracting of private/civil society service providers and developing the capacity of the district administrations to manage such outsourcing and contracting.

Agreement at Completion Point

The Government of Tanzania in collaboration with IFAD, AfDB and AGRA has designed the Marketing Infrastructure, Value Addition and Rural Finance Support Programme (MIVARF). The design of MIVARF took into consideration lessons learnt from the implementation of AMSDP and RFSP based on the recommendations from the draft interim evaluation report. The design team also had the opportunity of meeting with the interim evaluation team to discuss on the findings. The Programme is expected to begin implementation in July 2011.

The government agrees with all the recommendations given by the interim evaluation team and these have been incorporated in the design of MIVARF and will be followed up during its implementation. The four recommendations and the responses by the government are:

Recommendation 1: Scale up the positive experiences of RFSP and AMSDP and avoid support to organizations and activities with little potential for success

Opportunities for replication and scaling up. As highlighted in the report, there are many positive experiences from RFSP and AMSDP. Many of these positive experiences may need to be refined and further developed but they already represent lessons that MIVARF can consolidate, replicate, build upon, and scale up. However, the scaling up phase may also involve a transition from the project mode (to a large extent a feature of RFSP and AMSDP) to a more decentralized approach relying on mainstreaming support into government systems (see the discussion in paragraph 251 of the evaluation report). The risks associated with this and other issues related to scaling up should be considered in MIVARF.

Avoid support to unsuccessful activities and organizations. In some of the programme interventions, RFSP and AMSDP did not achieve the expected results and it is unlikely that continued or different forms of support under MIVARF will produce acceptable results.  IFAD, the Government, and/or local government authorities (LGAs) should resist different kinds of pressure for continuing the support in these cases "in order to save the organization".  Similarly, certain levels of mortality are to be expected for the supported gMFIs, producer groups and agribusinesses. The design of MIVARF should attempt to define acceptable mortality rates (or targets/success indicators for survival) for the different target groups/enterprises.

In order to improve transparency and mutual accountability, it would be useful to define the support for beneficiary groups in a contract or Memorandum of Understanding between the district/programme and the beneficiary group. Such a contract or Memorandum of Understanding would specify mutual responsibilities and obligations, what type of support will be provided and for how long, etc. This would eradicate any wrong perceptions among groups that they are entitled to eternal support, and it would also give the district/programme the tool to terminate support if the group fails to meet its obligations.

Response: The Government of Tanzania has already implemented this recommendation in the design of MIVARF. Implementation of MIVARF is scheduled to commence in July 2011. The Prime Minister's Office is the lead agency for this Programme. In order to ensure that support is provided where there is potential for success, the design underlines "competition for resources" as a principle for engaging the various levels of beneficiaries in implementation.

Deadline date for implementation: Done during the design phase of MIVARF project

Entities responsible for implementation: MIVARF Project Design Team under the leadership of Tanzania CPM

This recommendation has been implemented during the design of MIVARF

Recommendation 2: In design, maintain the focus on improving access to financial and marketing services of the rural economically active poor but emphasize financial and commercial viability and sustainability in the support to the beneficiaries.

Targeting. Though medium and large scale farmers and agribusinesses are important to growth, and indirectly also for the poor, IFAD should - given its mandate – continue concentrating on support that more directly benefits the rural active poor.¹   IFAD support for small and medium sized agribusinesses, such as through guarantees from financial institutions, should be a secondary priority or carried out in partnership (e.g. with AGRA). In addition, IFAD should support linkages between producer groups of small farmers and larger producers and agribusinesses.

Approach (pursuit of economic viability).While focus should continue to be on the rural active poor, it needs to be emphasized in selection and support strategies that grassroots financial institutions as well as producer/trader/processing groups are market-based and that only a good social rationale is insufficient to ensure their survival. Financial and commercial viability must be at the forefront when deciding whom to support and how to do it. This will also require that more attention is given to scale – e.g. instead of supporting the emergence of many small and weak SACCOS the programme should consider to support the expansion of larger SACCOS, having good survival potential, so that these SACCOS may open branches in non-served areas. In this context, there may also be a need for facilitating mergers. This will improve sustainability prospects and reduce the cost of delivering support and financial inclusion.  Also, it may not be necessary to work with SACCOS in some areas that now benefit from the availability of other financial services (e.g. MFIs and banks).A similar approach should be applied in support for producer/marketing groups. Working with fewer and larger groups with good prospects for expansion and increasing their value added is preferable to giving temporary and insufficient support to many small groups who are left in nowhere with limited prospects of survival as the support ends.

MIVARF duration. In sequencing and phasing the support for beneficiary groups there is now sufficient evidence to indicate the duration of support that groups at different developmental levels require in order to become self-sustaining. Thus, it should be possible to avoid inclusion (towards the end of the implementation period) of groups for which the support duration will be too short. In this context, it is of concern that the partners are considering an implementation period for MIVARF of only five years. ²  IFAD should consider extending the MIVARF implementation period to ensure sufficient time to establish well-functioning, sustainable groups.

Response: The design of MIVARF has taken into consideration this aspect and a lot of resources have been allocated to improving access to financial and marketing services. The eligibility criteria and principles of engagement to be employed by the Programme are geared towards ensuring financial and commercial viability and sustainability of supported beneficiaries.

Deadline date for implementation: During the seven years implementation period of MIVARF project

Entities responsible for implementation: MIVARF Project Management Unit under the directive of its steering committee Chaired by Permanent Secretary, Prime Ministers Office

This recommendation will be implemented during implementation of MIVARF

Recommendation 3: Reduce delivery costs and investment per beneficiary, and improve cost effectiveness

The high delivery costs and investment per beneficiary would not be acceptable in the scaling up phase. If the ambition is to scale up to national level coverage, this will multiply the challenges faced in RFSP and AMSDP manifold but may, in principle, present opportunities to improve the cost effectiveness and efficiency of the implementation and management of MIVARF. 

Clarifying implementation approaches and introducing lower cost options. First, MIVARF, by combining the two programmes which have many synergies may reduce some of the administrative and management costs.  Also, where possible in moving to a national programme, MIVARF should try to concentrate its investments to deepen impacts and avoid scattering resources within or across too many districts. This would enable MIVARF to succeed in the Value Addition (VA) part of its title.

The district administrations remain the appropriate place for anchoring the field implementation, however, they will require substantial external hand-holding and support (from a PCU) in order to handle the business and value chain development activities. Such hand-holding is costly and will result in very high delivery costs if the investment per district is modest (because the MIVARF budget may be spread over many districts). To be most cost effective, MIVARF should clearly articulate how it will work with both the LGA system and private service providers to optimize service delivery to the beneficiaries and reduce costs. For example, while the investments in marketing infrastructure (e.g. feeder road rehabilitation) clearly are within the mandate, daily activities and competence of district administrations, this is less so when it comes to facilitating establishment and development of private enterprises (i.e. gMFIs, producer/trader groups, agribusinesses) and value chains, which are the focus of IFAD support under MIVARF. Such facilitation requires special skills and substantial resources.  It may not be a first priority for the district administrations which already struggle to meet the demands for education and health services and economic and social infrastructures.

There are other options for reducing delivery costs and improving cost-effectiveness. Within rural finance, the landscape is changing rapidly. MFI/NGOs, such as BRAC, are becoming major players and mobile-phone banking is becoming accessible to rural communities while expanding its range of services, which may offer low-cost opportunities for financial inclusion of the rural poor. Similarly, within agricultural marketing, there are options for using well-functioning producer groups as service providers to assist newly formed groups in their development. There may also be options in contract farming schemes where large nucleus farmers or contract farming enterprises may be facilitated to support the capacity development of the small farmer groups who supply the enterprise. Such "service providers" may only require a small financial contribution as compared to the cost of contracting professional business development services providers and consultants.

Build on existing knowledge and experiences from AMSDP and RFSP. Finally, the lessons and experiences of RFSP and AMSDP make it possible to avoid costly searches and experiments on how to do things. Either as part of the closing of AMSDP and RFSP or as part of the inception phase of MIVARF, IFAD should facilitate the production of practical what-to-do manuals, developed based on the experiences and lessons learnt. Such manuals should inform district staff and other implementers about what to do and how to do it for different activity areas of the programme. Development of such manuals could be done in facilitated "write-shops" with participation of a few selected beneficiaries and stakeholders. The manuals would contribute to speed and cost-efficiency in the scaling up phase. In developing such manuals, it should be an overriding priority to reduce delivery and unit costs.    

Response: The two programmes preceding MIVARF have produced a number of key lessons which will enable a reduction in the delivery cost and investment per beneficiary to improve cost effectiveness. As from July when the implementation begins MIVARF will strive to use the lessons learnt from the two preceding programmes to improve cost-effectiveness. This will entail building up on efforts by other development partners during implementation as well as ensuring a close link to support provided by the Agricultural Sector Development Programme (ASDP) through District Agricultural Development Plans. MIVARF will also make use of the existing investment that has been developed by RFSP and AMSDP in terms of human, physical and technical investments.

Deadline date for implementation: During the seven years implementation period of MIVARF project

Entities responsible for implementation: MIVARF Project Management Unit under the directive of its steering committee Chaired by Permanent Secretary, Prime Ministers Office

This recommendation will be implemented during implementation of MIVARF

Recommendation 4: Emphasize development of partnerships and linkages at all levels

Strategic partnerships. There are many organizations and programmes that support the development of rural finance, agricultural marketing and value chains. Some address levels (e.g. investment finance for large agribusinesses) that are beyond the scope of IFAD's support but still important for development of the value chains. Some may fill other gaps in the MIVARF support while some have a time horizon that goes beyond the completion date for MIVARF. For these and other reasons, it is important that the MIVARF develops operational partnerships with such programmes and organizations and also facilitates the MIVARF target districts to access such cooperation.

Sustainable field-based partnerships.  It is also important to eliminate the perception of "MIVARF groups" and instead promote an approach whereby producer groups and gMFIs that are supported by MIVARF are facilitated support and services from other organizations and programmes. From the start of MIVARF's support for a group, attention should be given to avoiding the development of "MIVARF-dependency" and to promoting self-reliance.
Response: One key aspect of MIVARF is building partnerships and linkages for more effectiveness and greater impact. The MIVARF design has positively considered this aspect by bringing together IFAD, AfDB and AGRA in financing the Programme. During implementation, MIVARF will use a set of criteria developed to encourage partnership and linkages at different levels which will ensure success and lead to sustainability of its interventions. Moreover, MIVARF has also been nominated as a member to form a committee (under the ASDP) that will strive to leverage the various efforts within the agricultural sector to achieve a bigger impact on a wider area.
Deadline date for implementation: During the seven years implementation period of MIVARF project

Entities responsible for implementation: MIVARF Project Management Unit under the directive of its steering committee Chaired by Permanent Secretary, Prime Minister's Office

This recommendation will be implemented during implementation of MIVARF

Signed by:

Mr Ramadhani Khijah
Permanent Secretary
Ministry of Finance and Economic Affairs
Date:  04/07/2011

Mr Ides de Willebois
Director, East and Southern Africa Division
Programme Management Department, IFAD
Date: 15/06/2001

---