The Smallholder Cash and Export Crops Development Project (PDCRE ), which became effective September 2003, will close in September 2011.  Because the Government of Rwanda has indicated that it may ask IFAD to consider financing a follow-on project for PDCRE, IFAD procedures require that an interim evaluation be undertaken at this stage. The evaluation was carried out by the Independent Office of Evaluation (IOE). Besides providing an overview of the performance of PDCRE until now, this evaluation is expected to help guide the design of the potential follow-on project. The interim evaluation is also timely in the light of the planned Rwanda country programme evaluation (CPE) by IOE in 2011, as it will help identify strengths and weaknesses and how the project could be scaled up.

From 23 June to 13 July 2010, IOE fielded an evaluation mission who worked with PDCRE staff, beneficiaries, Government agencies, local governments, civil society, private-sector stakeholders, and development partners.

The evaluators used the IOE methodology specified in the Evaluation Manual of April 2009. The project authorities prepared a self-evaluation and results framework that were helpful for identifying the main achievements attained during the project period. The interim evaluation benefited from the project progress reports, the extensive supervision mission and mid-term reviews and their technical annexes, the reformulation document, and the policy and progress reports by PDCRE, stakeholder organizations and service providers. In addition, the Impact Assessment of 2010 prepared by the consulting group CIBLE Sarl, completed only a few months before the evaluation mission, provided comprehensive and timely primary data. Furthermore, the mission attempted to verify the information presented in the various documents as much as possible during its fieldwork, which included interviews of the project staff, public agencies, stakeholders and beneficiaries in Kigali and project areas. The mission visited all project areas and all but one of the cooperatives supported by the project. The mission then formed its independent views on the bases of its observations and the new information. In December 2010 a final learning workshop was held with the Government and IFAD staff to consider key findings and recommendations from the evaluation, which informed the evaluation's Agreement at Completion Point.

The country and its economy. Rwanda is a small, landlocked country of 26,338 square kilometres located near the centre of Africa, a few degrees south of the Equator. In 2010, Rwanda's population was 10.4 million people, and its annual population growth rate was about 2 per cent. With an average of 340 persons per square kilometer, Rwanda has the highest population density in Africa. The economy is largely subsistence agriculture; more than 85 per cent of the population lives in rural areas. Rwanda is one of the poorest countries in Africa. In 2009, its gross domestic product (GDP) per capita was US$866, making it No. 100 out of the 135 countries measured for the Human Poverty Index by the United Nations Development Programme (UNDP).

More than 80 per cent of Rwandans work in agriculture, and agriculture contributed an average of about 40 per cent of the total GDP in 2009. (Services contributed 46 per cent and industry 14.2 per cent). The average size of smallholdings is little more than half a hectare and relatively low yields are major concerns for Rwandan agriculture. The genocide-caused turmoil of the early-to-mid-1990s greatly reduced agricultural production, but it has subsequently recovered, reaching pre-war levels in 1998. Agricultural exports have long been dominated by coffee (US$47.1 million in 2009) and tea (US$39.8 million); together they account for more than 50 per cent of export earnings. Despite volatile international prices for coffee over the past few years, the country's best Arabica coffee is in strong demand and attracts a premium price on the international market. A major challenge for the Government is to ensure that food production keeps up with population growth. The key strategies at the time of appraisal were set forth in the Government's Poverty Reduction Strategy Paper in 2001 and IFAD's 2002 country strategic opportunities paper. To meet the current challenges the Government has articulated its long-term development goals in the Rwanda Vision 2020 document. In September 2007 the Government completed and adopted its Economic Development and Poverty Reduction Strategy (EDPRS).

The Project and its Performance

The project and its financing. PDCRE was approved by the IFAD Executive Board in December 2002. The Loan Agreement between the Government of Rwanda and IFAD was signed on 7 February 2003 and became effective on 19 September 2003. The project will be completed on 30 September 2011 and the loan will close in 31 March 2012. The total cost was estimated to be US$25.09 million, of which IFAD's loan amounted to US$16.26 million. The goal of the project is to maximise and diversify the income of poor smallholder producers of coffee, tea, and new cash and export crops. The project comprises five components: (i) coffee diversification; (ii) tea development with two sub-components: (a) integrated smallholder tea development in the Nshili district and (b) smallholder tea plantations in the Mushubi district; (iii) guaranteed credit to smallholders tea and coffee producers; (iv) development of new cash and export crops; and (v) project coordination. Project achievements are presented in chapter IV.

The project's objectives and component have remained identical to those set at appraisal, but many political and economic changes—some policy-related, others dictated by circumstance—have affected the project. One financier, BADEA (Banque Arabe pour le Développement Économique de l'Afrique), withdrew because the Government decided to involve the private sector in tea development and BADEA's planned contribution especially electrification of the tea complexes was no longer needed.  The Government cancelled its contract with the only international service provider, TWIN Trading Ltd., and introduced a public-private partnership policy. The Government also introduced a "zero-vehicle" policy for all development projects, established savings and credit cooperatives in all local administrative areas (Umurenges), and introduced a new cooperative law, which introduced a new approach to cooperative development by emphasising the role of the members and instituting strict penalties for mismanagement of cooperative funds, among other things. IFAD transferred supervision of the project from the United Nations Office for Project Services to its own consultants. These actions and the reformulation of the project at mid-term made the project look quite different from the one appraised in 2002 and made the evaluation mission's assessments of the project's efficiency more complex.

Assessment of performance. The project was relevant (and continues to be so) because its design was consistent with national policies and with IFAD's strategies, and because it integrated lessons from other IFAD projects in Rwanda. The project has addressed the needs of the rural poor and women. Although the project does not deal directly with food security, it has addressed important dimensions of poverty such as income generation, human and social capital, and the empowerment and development of poor people's institutions. Some of the potential problems related to natural resources and environment were not addressed. Although IFAD managed the project design process, it was based on a common understanding of the Government's policies and regular communication with the Rwandan authorities. The Government and IFAD understood the obvious needs of the rural poor and have kept their strategies consistent, the design simple and the components appropriate, and they have used the lessons learned from previous projects. These positive aspects outweigh the weaknesses in the logical framework and the undue optimism in some aspects of component design.

Although the development goal ("maximising the farmers' incomes and produce prices") was vaguely worded, the specific objectives and an analysis of their attainment provided an adequate basis for evaluating the effectiveness of the project. The coffee component did not enrol as many families as had been originally targeted, and the target should have been modified at mid-term when the number of coffee-washing stations was reduced from 16 to 10 (in 2010 it was raised to 12 because funds were still available under the project). Most of the objectives have been attained to varying degrees, the most successful ones being the establishment of cooperatives and coffee-washing stations and substantial progress (though delayed) in reaching the objectives of tea development and "new export crops." Some cooperatives have yet to achieve a sound financial footing, and only four coffee-washing stations (CWSs) operate at full capacity as productivity and capacity vary greatly through the project areas. It is too early to fully appreciate the effectiveness of the project because it is incomplete and productivity of coffee and tea plants is low because plants have not reached their fertility so that full benefits will come only when the coffee trees and tea bushes are mature and cocoon production reaches its planned capacity. Experts from OCIR-Café, OCIR-Thé and the project coordination unit agree that the yields are substantially below potential as a result of management practices generally being suboptimal. However, the evaluation mission was confident that farmers' incomes and assets will improve by the time the project closes and allow the project effectiveness to be rated at least as moderately satisfactory.

In the case of the efficiency of PDCRE, the mission found that it would not be possible to prepare an economic analysis that would be comparable to the economic analyses prepared for the coffee and tea components at appraisal. Too many changes had taken place in the coffee and tea components during implementation of the project to allow comparison. In addition, the yields of both coffee trees and tea bushes are still very low because they have been recently planted and the need for further improvement in management practices, and forecasting their productivity increases is too uncertain to allow meaningful computations of an economic rate of return (ERR). However, trends have already begun to improve, and so the mission recommends that an ERR be calculated for the models at the time of project closing (that is, for coffee cooperatives with coffee-washing stations and tea complexes that use a public-private partnership—PPP approach).

The evaluation mission used various other methods to estimate the efficiency of the project. These included calculation of financial rates of return for a well-functioning CWS and a cooperative society with the results of 29 per cent and 60 per cent financial rate of return, respectively (taking into account the 50 per cent subsidy on investments). In reviewing the administrative costs, the mission concluded that, at 15 per cent, they were at the same level as similar projects financed by IFAD or other financial institutions, if one takes into account the fact that all consultant costs were charged to the project coordination component rather than to the operational components for which the consultant studies had been made. The mission also computed per family costs of the coffee and the tea components of the project (for cooperative members only). These costs appear to be high, but the evaluation mission did not have benchmarks from other coffee or tea projects and cannot evaluate whether these figures are reasonable. Moreover, a large number of families who were not members of cooperatives received partial benefits from the project (seedlings and a place to sell their produce). As the project became effective and will have been completed on time, related to similar projects' duration (and despite the inability to compute an ERR at this time), the evaluation mission projected that the efficiency will be at least moderately satisfactory at project closing.

Poverty reduction impacts. The poverty reduction impacts of PDCRE were in the satisfactory range for three domains, that is, household income and assets, human and social capital and empowerment, and institutions and policies. It is important to note, however, that the coffee and tea incomes for the smaller farmers (those with less than 0.5 ha of tea or 250 coffee trees) cannot raise them above the poverty line but can only contribute to their reaching that level. Also, the main vehicles for improving the incomes of the coffee and tea farmers under PDCRE are cooperatives, and cooperatives are economic institutions owned by their members, who, in the case of processing and marketing cooperatives, must own enough assets to have surplus production to sell in addition to subsistence crops. Thus, these cooperatives are not for the poorest of rural people. However, the poorest social groups, that is, those who do not have agricultural land, have to a limited extent benefitted from other PDCRE-supported activities through project-created employment. Moreover, because also the poorest people wish and can save (and consequently may qualify for loans), they could benefit further from the projects' support to savings and credit cooperatives (SACCOs), because SACCOs admit  as members anybody who can pay the memberships fee, which is a lower barrier to entry for the poorest as compared to  other financial institutions.

While not directly addressing food security, the project has contributed to its improvement through increased income for smallholders and employment for the poorest sections of rural population. Human capital has improved on several levels as a result of extensive training and advisory work, but it has improved perhaps most remarkably among women, who now occupy from 30 to 60 per cent of committee seats in the PDCRE-supported cooperatives. As for environmental impacts, fertilizers and pesticides are little used in coffee or tea production zones and thus are not considered detrimental to the environment. In contrast, the evaluation mission considered all 10 PDCRE-supported coffee-washing stations and two tea factories to cause more pollution of the environment than was estimated at appraisal.

The overall sustainability of project activities is still uncertain. The coffee cooperatives find it hard to get the funds required to buy all the coffee berries offered. The cooperatives are heavily indebted and rely on the Rwanda Development Bank for long-term and short-term loans. However, because 40 per cent of them are already working at full capacity and are profitable, the evaluation mission considers their future sustainability achievable, especially for coffee where the international prices are now at an 11-year high. The outlook for tea and sericulture cooperatives seems less certain, for both financial and managerial reasons. Also, because the tea cooperatives are quite new, they need time to improve their administrative and financial management techniques, and their productivity is still low (although it is expected to improve once the tea bushes mature). The financial sustainability of sericulture looks problematic, and it is too early to forecast its future results.

Evaluation Criteria	Interim Evaluation Ratings
Project performance
Relevance	5
Effectiveness	4
Efficiency	4
Overall project performance	4.33
Rural poverty impact
Household income and assets	4
Social capital and empowerment	5
Food security and agricultural productivity	4
Natural resources and the environment	3
Institutions and policies	4
Overall rural poverty impact	4
Sustainability and Innovations
Sustainability	4
Innovations, replication and scaling up	4
Performance of partners
IFAD	5
UNOPS	4
Government	4
Overall project achievement	4

* Ratings are assigned on a scale of 1 to 6 (6 = highly satisfactory; 5 = satisfactory; 4 = moderately satisfactory; 3 = moderately unsatisfactory; 2 = unsatisfactory; 1 = highly unsatisfactory).

 

Recommendations

The interim evaluation has focused on the recommendations summarized below:

Potential Follow-on Project

The evaluation mission agrees with the Government that a new project similar to PDCRE is justifiable. The main focus of the follow-on project should be to consolidate the progress made and scale up the lessons learned with particular emphasis on coffee and tea. The cooperatives currently assisted by PDCRE must be able to consolidate their activities and become financially viable. New cooperatives may be added—and would be up and running faster as the project coordination and oversight agencies learn the lessons from PDCRE 1. The evaluation team has the following recommendations for the team that prepares the new project:

Meeting the processing challenges.The next phase should emphasize expanding the capacity to process coffee. When the 15.5 million coffee tree plants that were distributed in the PDCRE areas to cooperative and non-cooperative farmers (paragraphs 44, 90) become mature, they may produce up to 48,000 tons of cherry beans. Because the capacity of PDCRE-supported cooperatives is only 4,600 tons, some 80 to 90 new CWSs of the same size as under PDCRE will be needed to meet the processing requirement. This situation justifies planning more CWSs for the existing cooperatives or expanding the present ones, even if the new project extends outside the current area. However, the "old cooperatives" should rely more and more on bank loans for their investments, possibly with some kind of loan guarantees from external financiers. Also, in the next project a modular approach should be considered for CWS to avoid excessive loan burdens as happened during the present PDCRE with a standard-size of 500 tons CWS design (paragraphs 44, 100).

Capacity and institution building.The next phase should expand the training capacities of central cooperative agencies. Cooperative staffs and committees—as well as other members—will need training, and this should happen through expansion of the training capacities of the central agencies [Rwanda Coffee Cooperative Federation (RCCF), Federation of Tea Growers Cooperatives (FERWACOTHE), Rwanda Cooperative Agency (RCA)] (paragraphs 92, 120-123). Given the relatively limited capacity of these agencies, in the near term, a mixed approach using these agencies as well as NGOs and companies to provide capacity building services may be the most appropriate approach. It would be important to develop common procedures and practices for cooperatives (to have uniform training materials, manuals, accounting systems, and management information systems). Moreover, cooperative audit and control capacities need to be enhanced because private audit firms are not interested in small cooperatives. Because Rwanda has more than 4,200 cooperatives, establishment of a cooperative college should be considered (such a college has operated for instance in Kenya since 1969).

Financial services.The next phase should support savings and credit cooperatives (SACCOs) as a means to facilitate financial services for small farmers.Under PDCRE, support to SACCOs was a positive addition to the project as they can provide a useful service for small farmers and should be further supported.  However, in the context of promoting SACCOs (paragraphs 65 and 68), some individuals in Rwanda have proposed that SACCOs could serve as intermediaries for outside loans to local farmers. In terms of providing credit to cooperatives, BRD should be considered a viable partner as it has demonstrated its willingness to work with PDCRE and has performed satisfactorily. Although, if interested, commercial banks should be given the opportunity to participate.

Development goal. The development goal of the next phase should include an institutional element as institutional sustainability is required. In the current project, the development goal and even some specific objectives were expressed in obscure language, that is, the wording did not permit direct verification of whether the goal and objectives had really been attained (paragraphs 40, 87). For the follow-on project the mission suggests that the goal and objectives be made more specific and measurable, and that the main goal would in part be institutional and include an exit strategy rather than only focusing on farmer incomes or produce price

Cooperatives and their Management

In the next phase strengthening cooperatives and their management should be a priority. To do so a detailed analysis of the challenges of sustainability should be undertaken. To address the challenges that have been highlighted in the report and summarized in paragraphs 168-169, cooperative members and committee members need training and education, and it is equally important to improve the control and monitoring functions. To find out where improvements are most urgent, a detailed analysis of management's shortcomings and of gaps in the skills and competencies among the members, committee members and staff is needed.

Studies or Reviews

In addition to the need for analysis of the cooperative management problems just mentioned, the evaluation mission recommends a number of other studies. During the project period, PDCRE has organized carrying out a large number of studies and reviews on specific topics. They are listed in appendix 3(b). In addition, specialists hired by UNOPS and IFAD have prepared numerous reports on project-related topics during the mid-term review and supervision missions. However, some of these are out of date or the evaluation mission was concerned about their validity in the current situation. The evaluation mission recommends the following:

Environment: A study on the environmental effects of the coffee-washing stations and tea factories (paragraphs 118);

Cash crops: An updated study on "other cash crops" because the earlier one is 6 to 7 years old (paragraph 72) and the economy of sericulture appears uncertain; and

Coffee: A cost-benefit analysis of organic coffee production in Rwanda coffee cooperatives (paragraphs 43, 48). A technical and economic analysis of why parchment coffee processing accounts for more than one-half of the costs at CWSs (paragraph 89). A feasibility study of establishing a hulling factory for cooperatives (as in the original project design) (paragraphs 43, 94, 99, 136).

Public-private Partnership

The next phase of the project should work to ensure that the PPP is pro-poor. The PPP introduced in the tea component offers potential benefit to small farmers. However greater attention and support is needed to ensure that benefits accrue to all parties. The tea cooperatives require advice on the economic and managerial aspects of cooperatives, and on fair practices toward cooperative members under the public-private partnership arrangement (paragraphs 146-147). IFAD should look to learn lessons from other projects with experience with PPPs such as the Vegetable Oil Development Project in Uganda where there are lessons from using a pricing formula involving representation of smallholders with the private company.

Food Security and Cash for Work Programme

In the next phase the project should introduce a "money-for-work program". Land in Rwanda is so scarce, particularly in the Mushubi area, that smallholders could not always afford to allocate any part of their small plots to coffee and tea crops because they needed to grow food (paragraph 114). They and some stakeholders have recently suggested that farmers with the smallest lots get cash loans to carry them over until the new crops produce enough income to allow them to buy food. However, the mission believes that such cash loans could create undesirable dependency, or simply be misused.

Instead, the evaluation recommends introducing a "money-for-work program" into the follow-on project. This will contribute to smallholder's income and improve local conditions and develop local resources, for instance, by financing labour-intensive works on low-quality and stony roads, improving the very poor, unsafe roads (there are no safety rails or even poles on the very steep mountainside feeder road), supporting afforestation and forest maintenance, developing village water schemes.