Interim evaluation

Introduction

In April 2008, IFAD's Office of Evaluation (OE) conducted an interim evaluation of the IFAD-funded Uplands Food Security Project (UFSP) in the Democratic People's Republic of Korea (DPRK). This was the first evaluation by OE in the country. Its main objectives were to: (i) assess the performance and impact of the project; and (ii) generate a series of findings and recommendations for future projects and programmes financed by IFAD in the country.

On 26 November 2008, a learning workshop was held in Pyongyang to discuss the main results of the evaluation, and to provide inputs for the preparation of this agreement at completion point (ACP). The latter presents an agreement between IFAD (represented by the Asia and the Pacific Division) and the Government of DPRK (represented by the Ministry of Agriculture) on the key evaluation findings and recommendations, and proposals on how to act upon them.

Main evaluation findings

Main results per component. Of the seven project components, the sustainable crop production systems component was the most significant, with 56 percent of the original base cost in favour of 37 cooperative farms (CFs). It promoted the introduction of three-year crop rotations, albeit without the broad-based insertion of fodder crops. The component also included the procurement of farm machinery and farm inputs, mostly fertilizer. The second component, potato seed supply development, was successful as it responded to a key limitation of potato production, i.e. disease-free quality seed. Under the third component, environment preservation, the project encouraged 46 CFs to plant and enrich fuel wood plots on 5,500 hectares of nearby sloping lands.¹

Credits for CFs and individual households were the object of the fourth component. Both types of borrowers took up small livestock breeding and fattening activities thanks to available credit, with a ceiling of less than US$70 per household. The target of reaching close to 18,000 households was achieved, covering 46 CFs. The fifth and sixth components, both very small in terms of financing, aimed at strengthening community facilities and services in 37 CFs. Both were implemented rather late, but still resulted in the construction of 162 community facilities such as kindergartens, nurseries, clinics and training centres and the procurement of more than 100 items of farm output processing equipment, which are now in operation and range from flour and rice mills to oil and potato starch extractors.

Under the seventh component, the project management unit (PMU) was strengthened, enabling it to assure project implementation. 

Project performance. The project was relevant in that it addressed significant rural poverty in relatively remote and resource poor areas of the country. Its design drew lessons from former IFAD operations in the country, especially with regard to the credit component. However, relevance was hampered by substantial information gaps at design and a deficient involvement of local expertise.

Thus, project design was unclear regarding the cropping systems to be promoted under the main component, and did not take into account the pervasive soil acidity in most of the participating CFs.

As it turned out, crop response to fertilizer applications was constrained because of this phenomenon, which the project did not address. Moreover, project design ignored the communication problems between the PMU and project partners, including IFAD, and failed to formalize co-financing and technical assistance agreements. As a result, neither food-for-work from the World Food Programme (WFP) nor technical assistance from the Food and Agriculture Organization (FAO) and the United Nations Development Programme (UNDP) were delivered as envisaged.

During several years the project had very limited success in introducing sound crop rotations because of inadequate approaches and technical advice. However, after a less directive approach was adopted favouring capacity-building in collaboration with the National Academy for Agricultural Sciences (NAAS), the project was effective in introducing sound crop rotations on more than 31,000 hectares against a target of 18,000 hectares. The project also succeeded in providing farm machinery, farm inputs, community facilities and services, and the farm output processing equipment. It also put in place the credit component in favour of 46 CFs and 18,000 individual households. In terms of efficiency, the goods were procured at comparatively competitive prices. However, end-user efficiency was constrained due to bottlenecks in maintaining farm machinery and the less than satisfactory crop response to fertiliser application because of widespread soil acidity. The credit component suffered an important efficiency loss.

Rural poverty impact. The impacts achieved in the domains of household assets and income, social capital and empowerment as well as agricultural productivity and food security were satisfactory, with the credit component having played a pivotal role. Over project life, household incomes increased by a factor of two in average, while livestock income – a direct consequence of the credit component – tended to grow more rapidly than farm dividends. The credit component empowered women in particular, as more than 90 percent of the loans were underwritten by women, and talent was unleashed to run small scale livestock operations at own risk and responsibility. This has contributed to enhanced food security. The impacts regarding natural resources and the environment were respectable in the immediate vicinity of the CFs, but unavoidably limited when considering the degree of degradation of sloping land on a broader scale.

Sustainability. The project has invested in areas with a distinct potential for sustainability, such as new crop rotations, the technically sound production of disease free potato seeds, and small livestock breeding and fattening systems, with CF work teams and households working in tandem. Increased household incomes and assets are likely to protect the households from future shock and stress. However, farm machinery procured by the project is losing its usefulness due to difficult access to spare parts and lack of adequate machinery maintenance capacity. The overall resource base of the CFs is still fragile, and the challenge of producing enough food for the country's population remains substantial. The encroaching of sloping lands is a matter of environmental concern, which the Government expects to resolve in cooperation with international partners.

Innovation. The project was successful in devising a low-cost and zero-risk credit component, which can be considered innovative. There is no evidence, however, of replication or scaling up of the household credit scheme. Although the new crop rotations took time to be adopted they may be the expression of local expertise and resolve rather than project-driven innovation. In matters of comprehensive soil fertility management, the project failed to take advantage of available local knowledge as no specific activities, such as systematic on-farm trials, had been included in the design.

Performance of partners. IFAD deserves the credit for having designed a relevant project in rural poverty reduction. On the other hand, important aspects, such as on-site fact-finding, the formation of solid partnerships with co-financiers and the securing of reliable communication links, were neglected both at design and during project implementation. The United Nations Office for Project Support (UNOPS) was in a similar position, confronted with the task of supervising a relatively complex project in an environment characterized by scarce availability of information and restricted access. While the performance of the PMU was respectable, considering all the odds, the other national partners remained reclusive in terms of information sharing and did not provide all relevant technical, economic and social data in time. The partners that exhibited outstanding performance were the beneficiaries, the CFs and involved households alike. The key potential for future success lies with the talent and determination of these men and women.

Recommendations

As this was an interim evaluation, the recommendations aim at setting a preliminary stage for the design of a subsequent IFAD operation in DPRK. They will be taken into account in the forthcoming results-based IFAD country strategic opportunities paper (COSOP) – a necessary step before initiating the design of a follow-up IFAD investment loan.

This table presents the evaluation recommendations ² and the agreement status by IFAD and the Pacific and the Government of DPRK. In accordance with the IFAD Evaluation Policy, in case of disagreement, views are justified by both parties.

---

1/ Sloping lands are defined in DPRK as lands with slopes of 15 per cent or more. These lands are managed by the Ministry of Land and Environmental Protection

2/ Additions made to the interim evaluation recommendations, based upon the deliberations held during the learning workshop in Pyongyang on 26 November 2008 with the main project partners, are shown in italics