Executive summary

Introduction

At its eighty-ninth session in December 2006, the Executive Board requested the Office of Evaluation (OE) to undertake a Country Programme Evaluation (CPE) in Pakistan in 2007/2008.

Evaluation objectives, methodology and process. The main objective of the evaluation was to: (i) assess the performance and impact of IFAD operations in the country; and (ii) generate findings and recommendations to serve as building blocks for preparation of the next results-based Country Strategic Opportunities Programme (COSOP) for Pakistan. To that end, the CPE assessed the quality and implementation of the COSOP, and the performance and impact of IFAD operations (including non-lending activities such as policy dialogue, the forging of partnerships and knowledge management). The evaluation covers 14 IFAD loan-funded projects approved for Pakistan since 1990.

A CPE preparatory mission took place in April 2007. The main CPE mission took place from 2 to 31 July which visited IFAD-funded project areas, held discussions with beneficiaries, project staff, federal and provincial government authorities, international organizations and others. In January 2007, the Asia and the Pacific Division of IFAD (PI) undertook a self-evaluation of its operations in Pakistan to serve as an input for the CPE.

Economy and poverty. Approximately 67.5 per cent of Pakistan's population and 80 per cent of its poor live in rural areas. The majority (57 per cent) of the rural poor are from non-farm households (excluding agricultural labourer households, which are a minority), with the poorest 40 per cent of rural households deriving only 30 per cent of their income from agriculture. According to the 2008 World Development Report, Pakistan is categorized as a 'transforming country', where agriculture is no longer a major contributor to economic growth and poverty remains a largely rural phenomenon. Remittances amounted to US$5.5 billion in fiscal year 2006-2007, representing 9 per cent of total rural income. Pakistan has one of most developed irrigation systems in the world. Only 20 per cent of the cropped areas remain rain-fed (barani).

Agriculture's contribution to gross domestic product has declined from a little over 25 per cent in 1990 to 23.1 per cent by 2005. Average annual official development assistance (ODA) commitments to agriculture and rural development during the period evaluated amounted to US$146 million. IFAD's average annual commitment in the period 2001-2005 was US$11.2 million, equivalent to 7.7 per cent of ODA dedicated to the same category. 

IFAD strategy in Pakistan

IFAD's work in Pakistan has been governed by a country strategy developed in 1991 and by the COSOP of 2003. The 1991 strategy targeted less favoured areas, following a rural development model adapted from the Aga Khan Rural Support Project (AKRSP). The 2003 COSOP continued the process of consolidating and improving that model, and proposed assisting the Government in five main directions: agricultural and rural development; women's empowerment; access to resources; decentralization; and household food security and diversification of production. These main directions were consistent both with key country priorities and strategies, and with IFAD's regional strategy. Moreover, decentralization, in the sense of government having a greater local experience with appropriate delegation of authorities, is particularly timely and relevant to the devolution process initiated in Pakistan in 2000.

The geographic niche selected by the COSOP contains a number of remote, disadvantaged and conflict-affected areas including the Federally Administered Tribal Areas (FATAs), parts of the North-West Frontier Province (NWFP) and Azad Jammu and Kashmir (AJK). These areas account for some of the lowest social indicators in the country and are characterized by poor infrastructure, scarce communications and weak administrative institutions.

Nevertheless, IFAD and the Government do not appear to have taken a differentiated approach to project design and implementation in these areas, and have not paid enough attention to special measures aimed at ensuring flexibility, deploying suitable expertise and seeking alternative partnerships in comparison with other parts of the country. This led to difficulties in implementation and jeopardized results.

There are some important gaps in the COSOP. The development of the non-farm sector, including the development of rural microenterprises as well as the role of remittances, received little attention. Moreover, access to markets -which is fundamental in order to ensure wider opportunities to the rural poor– and promoting linkages to the private sector were not addressed in the strategy. Environmental issues also received little attention in the COSOP.

Despite the limitations of the Pakistan COSOP as a strategic instrument when it was prepared the COSOP presents little synergy among the projects and limited interface with non-lending activities or with technical assistance grants.

The COSOP was also rather vague as to who IFAD's partners might be.

It seems that, in most projects, IFAD's business model of implementing projects/programmes in partnership with government has meant that areas with the most complex rural problems have been handled by agencies with the least capacity to deal with them.

However, the more recent projects, approved after the 2003 COSOP was prepared, represent a move in the right direction as they are being implemented with the support of the Pakistan Poverty Alleviation Fund (PPAF).

Performance and results of IFAD-supported operations

Portfolio performance. By and large, the results of IFAD-funded projects in Pakistan are in the satisfactory zone, despite delays and other challenges to implementation. Projects have increased both the asset base of beneficiaries and their preparedness to deal more actively with government, which has also changed the way it perceives the beneficiaries. The IFAD-supported programme has been particularly strong in improving agricultural productivity; constructing priority community assets to be owned by the community; increasing beneficiaries' financial assets; mobilizing community organizations; boosting the confidence of women to engage in economic activities; and enhancing food security.

Projects scored satisfactorily with regard to relevance, demonstrating that such interventions fit well into Pakistan's agricultural policies, although in quite a number of cases insufficient attention has been given to effectively targeting the poor. The evaluation noted, however, that efforts were made through mid-term reviews and follow-up missions for three projects ¹ to sharpen the focus on targeting. Overall efficiency has been moderately satisfactory despite time overruns (two years on average). By and large, gender has been an area of success in the country programme. However, performance fell short in some areas. IFAD's contribution to the devolution process and to strengthening the capacity of local government institutions has been limited. Results in the area of microfinance have been weak, mainly for reasons having to do with the operating environment.

Despite being recognized as important in project design, livestock –as well high-value crops– have not yet received the attention necessary to maximize their potential for rural poverty reduction.

Sustainability remains a challenge, particularly as it pertains to the maturity of community-based organizations (CBOs), public infrastructure and access to credit², despite progress made by a number of microfinance institutions supported by IFAD in Pakistan towards profitability. With regards to innovation, IFAD promoted the participation – for the first time in a government-financed and –managed project – of non-governmental organizations (NGOs) and has contributed to introduce new agricultural products and innovative financial instruments (particularly through the most recent microfinance projects). There are few examples, however, of a systematic approach to the replication and scaling up of successful innovations by the Government, the larger international financial institutions (IFIs) and the private sector.

The performance of partners. In the past, IFAD has been a rather passive player in terms of its non-lending activities in Pakistan. The change in the Country Portfolio Manager for Pakistan in 2004, together with the decision to establish a proxy country presence (PCP) (in the form of a retainer consultancy) in 2005, has improved IFAD's visibility, donor coordination and relations with government partners. However, even though the PCP seems to be well-established, it is not institutionalized, having limited authority and resources allocated to it. IFAD's decision to assume responsibility for the supervision and implementation support of four ongoing projects as of January 2008 deserves recognition inasmuch as it swiftly responded to corporate priorities.

Pakistan is one of eight United Nations Member States selected to pilot UN reform in 2007-2008. The country presence has also facilitated IFAD's engagement in the 'One UN' initiative, including participation in all UN Country Team meetings.

In the projects supervised by the United Nations Office for Project Services, the fiduciary aspects were supervised well although technical aspects and implementation support tended to be neglected.

The Government performed as well as could be expected within the limitations imposed by its capacity constraints, especially in the remote areas targeted by the Fund.

IFAD has established good relations with rural support programmes and the PPAF.

Non-lending activities were not a strong part of the programme. Policy dialogue has been reinvigorated by IFAD's country presence, which has allowed for more frequent discussions and interaction with the Government. All in all, however, IFAD has not managed to translate its project-based experience into providing effective pro-poor policy advice to the Government. Good relations have been established with federal and provincial levels of government, two major IFIs operating in Pakistan, the PPAF and with some research institutions such as the International Center for Agricultural Research in the Dry Areas.

However, the range of partnerships could be expanded further, including the private sector. Knowledge management has been pursued through various means such as project exchanges, regional electronic networking and a bi-annual newsletter. However, more could have been done to systematically review and analyse IFAD's experience in Pakistan with a view to extracting lessons and knowledge for sharing across projects.

Aggregate evaluation ratings^a of IFAD-funded projects in Pakistan

a) OE uses a six-point rating scale, whereby 6 is the highest score and 1 the lowest.

b) Ratings considered here are those of five projects for which substantial documented evaluative evidence is available: Mansehra Village Support Project, Pat Feeder Command Area Development Project, DIR Area Support Project, Barani Village Development Project and the North-West Frontier Province Barani Area Development Project.

Conclusions

The Fund has made an important contribution to agriculture and rural development in Pakistan, which is even more significant in the light of the current surge in food, commodity prices and related shortages. This has been achieved despite its relatively limited investments in the country and the lack, until recently, of a permanent country presence. It is also particularly noteworthy as several IFAD operations have been implemented in highly challenging environments.

The Fund was instrumental in further developing the successful AKRSP model to grass-roots development, by scaling it up and adapting it to a government implementation model. IFAD has also contributed to strengthening CBOs, to women's empowerment (including in difficult contexts such as FATAs) and to improving the agricultural productivity of small farmers, which led to better food security and incomes. These achievements are the result of IFAD's focus on pursuing largely agricultural-based interventions.

However, a number of areas such as environmental issues, rural financial services and market linkages, as well as livestock development and the promotion of high-value crops, did not receive the attention they deserved.

Notwithstanding the above, the CPE concludes that even greater results could have been achieved by IFAD through wider consideration of and investments in non-farm activities and employment, including attention to the development of rural microenterprises with adequate linkages to rural financial services. Moreover, greater attention to the consequences of migration, and to ways of tapping the vast amount of remittances flows would have been useful. This is particularly relevant in light of Pakistan's categorization as a ‘transforming country' and agriculture's modest 30 per cent contribution to the incomes of the poorest rural people.

Good results were seen with regard to social mobilization and the building of CBOs. However, the CPE concludes that the Fund could have taken a more broad-based approach to supporting Pakistan's devolution plan of 2000 and to overall decentralization, including greater attention being paid to strengthening local governments and representatives of elected bodies through capacity building of locally based employees of different levels of government and encouraging a service orientated culture, as well as proactively seeking partnerships with the private sector.

IFAD has worked in various remote, disadvantaged and conflict-affected areas of Pakistan, including the FATAs, parts of the NWFP and AJK.

Despite the difficulties associated to working in these geographic areas, the performance of IFAD-funded activities in such areas in Pakistan has been moderately satisfactory on the whole, and future IFAD assistance in these areas of the country deserves serious consideration. While the CPE noted the strong desire of the Government to ensure the Fund's continued engagement in such areas, the Fund cannot continue with an undifferentiated approach.

Two overarching factors call for special consideration by the Fund. First, sustainability – an institution-wide issue for IFAD and also a concern in the Pakistan portfolio; and second, innovation, which despite various examples in the portfolio, has not been conspicuous in the country.

Moreover, results remain weak in terms of the replication and scaling up of innovations promoted through IFAD operations, which is partly attributed to inadequate attention to non-lending activities, as well as to poor links between grants and loans. Even though limited in terms of resources and authority, the establishment of a proxy country presence in 2005 has contributed to better positioning of IFAD in Pakistan. Monitoring and evaluation systems were weak. 

Recommendations

The CPE proposes five overarching recommendations with regard to IFAD's country programme for Pakistan:

• The need for a better balance between agricultural and non-farm investments in the rural sector. The CPE recommends that more resources should be devoted to non-farm opportunities, and underlines the importance of promoting rural financial services and wider market linkages for both agricultural and non-farm outputs. In terms of agricultural activities, greater attention should be given to livestock and high-value crops that would provide higher returns on investments as well as to domestic production of edible oil, which provides an opportunity to reduce imports and enhance food security. Consideration should be also given strengthen measures for improving environmental and natural resource management.
• Provide capacity development support to decentralized entities and other bodies working at the local level. This will call for continued attention to social mobilization and the strengthening of CBOs, local NGOs and rural civil society in general. At the same time, the Fund should take a more inclusive approach to supporting decentralization by establishing the building blocks for a more service orientated relationship between governments and local organizations. This entails building up the capacity of local governments and representatives of elected bodies that play an important role in planning and resource allocation for rural poverty alleviations at the grass-roots level Greater participation of private-sector groups of farmers and enterprises is also warranted to ensure better results.
• The CPE recommends that the Fund should continue to support the Government in its engagement in disadvantaged, remote and conflict-ridden areas of the country — but taking a much more differentiated approach, i.e. one that is both flexible and adapted to the context of such areas. The mobilization of expertise, particularly in tribal affairs, conflict resolution and peace-building, is essential.
  
  The importance of ensuring the commitment of provincial and federal governments to continued IFAD support in these areas cannot be over-emphasized. In fact, IFAD could play a complementary developmental role - in support of the rural poor - to the Government's own initiatives and those of other donors working in such environments.
• Strengthening IFAD's capacity to promote innovations that can be scaled up and replicated by the Government, donor organizations and the private sector, merits more attention and resource allocations. This includes a more systematic approach to identifying, piloting, documenting and sharing innovative approaches to agriculture and rural development, additional resources and capacity to engage in policy dialogue, and careful selection of partner institutions. This will also necessitate closer synergies between, and wider use of, the mix of instruments available. A number of areas are in need of innovative approaches: remittances, migration, promotion of local governance and use of grants (as opposed loans) to support the efforts of larger development actors in conflict-affected areas such as the FATAs.
• The Fund's overall development effectiveness would be further enhanced by adjusting its operating model in accordance with the size and specificities of its programme in Pakistan.
  
  This includes establishing a more consolidated, permanent and better-funded country presence – one option to strengthen country presence in Pakistan is to outpost the CPM from Rome–, undertaking direct supervision and implementation support of IFAD-funded projects and programmes, and seeking to improve both knowledge management and project and country-level monitoring and evaluation systems.

---

1/ Barani Village Development Project, Dir Area Support Project, and Northern Areas Development Project

2/ The designs of the last two microfinance projects (Microfinance Innovation and Outreach Programme, and the Programme for Increasing Sustainable Microfinance) pay careful attention to sustainability and exit strategies.