Corporate-level evaluation

Introduction

Background. In December 2003, at its eightieth session, the International Fund for Agricultural Development (IFAD's) Executive Board approved the three-year Field Presence Pilot Programme (FPPP). The main aim of the pilot programme was to enhance the effectiveness of IFAD operations by focusing on four interrelated dimensions: implementation support, policy dialogue, partnership-building and knowledge management. While approving the pilot, the Board requested the Office of Evaluation (OE) to evaluate the FPPP during its third year of implementation and present its results to the Board.

Evaluation objectives. Accordingly, the main objective of this evaluation was to: (i) assess the performance and impact of the FPPP in achieving IFAD's overall objectives; and (ii) generate findings and recommendations to help IFAD's management and Board decide on the pilot's future and lay the basis for the formulation of an IFAD country presence policy.

Evaluation methodology. The evaluation: (i) established a comparator group of countries without any form of IFAD field presence to gain a better appreciation of the results in countries with and without field presence; (ii) gave emphasis to the assessment of results achieved before and after the establishment of field presence by mainly obtaining the views of stakeholders; (iii) secured systematic feedback about the benefits of an actual or potential field presence within IFAD and at the country level; and (iv) carried out a comprehensive benchmarking study to identify good practice in country presence and learn from the experience of other development organizations.

While the focus was on the FPPP (including satellite countries¹), the evaluation also examined the experience gained with: (i) two outposted Country Programme Managers (CPMs) in Panama and Peru; and (ii) proxy field presence² arrangements. This facilitated the assessment of alternative field presence arrangements pursued by IFAD.

The evaluation faced three major challenges: (i) limited availability of self-evaluation data, incomplete accounting of FPPP costs and lack of any baseline data on the performance and outcome indicators adopted by the Board; (ii) the short and incomplete implementation period for most of FPPP initiatives³; and (iii) unrealistic expectations regarding the assessment and attribution of the pilot programme's impact on the rural poor⁴.

Country visits took place in 25 of the 35 countries included in the evaluation sample, some with and others without any form of field presence. In line with the IFAD Evaluation Policy, OE set up a Core Learning Partnership (CLP) and benefited from the inputs of two Senior Advisers (Dr Nafis Sadik and Professor Robert Picciotto), who provided advice on the design of the evaluation and reviewed all major evaluation deliverables.

Assessment of IFAD'S Field Presence experience

Design and management of the FPPP. The evaluation found that the focus of the FPPP on the four interrelated dimensions (implementation support, policy dialogue, partnership building and knowledge management) was appropriate for furthering the objectives of IFAD country programmes. However, the FPPP was critically under-funded, and the human resources allocated to the pilot were inadequate. The Consultation on the Sixth Replenishment of IFAD's Resources and the Executive Board were quite involved in the design of the FPPP, including establishing the pilot's objectives and resource allocation. For example, a group of IFAD member states prepared and circulated a "non-paper" outlining the objectives and design of the FPPP in 2002, and a specific Ad-hoc Working Group of the Executive Board on Field Presence⁵ was established in December 2002 to oversee its development and implementation. This involvement delineated a clear framework that the management followed in preparing the final proposal on the FPPP.

The FPPP design devoted inadequate attention to the interface between CPMs, IFAD's cooperating institutions and the field presence officers in general, but in particular with regard to their contribution to implementation support activities. This led to confusion on the ground among key partners about their respective roles and responsibilities. The self-assessment report on the FPPP issued by Programme Management Department (PMD) echoes these sentiments.

The Fund did not take advantage of the FPPP by systematically experimenting with alternative models to field presence except for the inclusion of satellite countries – a subregional approach to field presence connected to the establishment of field presence in a particular country. Most pilot countries followed the same model. It involved appointing a local staff member and arranging for office space. Nor did the Fund outpost any CPMs from Headquarters under the FPPP, although such experimentation was specifically envisaged by the pilot.

The implementation of the FPPP was also characterised by the inability to capture reliable cost data and the absence of a platform for systematic knowledge sharing among FPPP officers and CPMs, as well as inadequate reporting on performance indicators. Furthermore, no human or financial resources were specifically dedicated by IFAD for the management of the FPPP, so the pilot had to be implemented within existing management and staff capacities. As such, it can be said that pilot has not provided IFAD's management and Board with appropriate guidance for the formulation of an authoritative country presence policy at this stage.

Organizational aspects. Field presence officers draw on the administrative services of host organizations. This inevitably leads to some loss of IFAD identity and visibility. The majority of field presence officers are recruited and hosted by United Nations Development Programme (UNDP). The others are recruited and hosted by the Food and Agriculture Organization of the United Nations (FAO) and the World Food Programme (WFP), except for four field presence officers directly recruited as consultants by IFAD. No pilot is housed in the offices of international financial institutions⁶ – a lost opportunity for enhanced partnership with organizations that are especially well placed to help ‘upscale' IFAD-funded activities.

The Peru outposted CPM works from a privately rented office, whereas the Panama CPM is hosted in the UNDP office. Most proxy field presence officers work from private offices (or their homes). They are all recruited directly as IFAD consultants.

The effectiveness of the FPPP and proxy field presence officers has been constrained by limited delegation of authority. Nor has systematic coaching been provided.  Field presence officers are not authorized to represent IFAD formally or to take decisions on operational or financial matters. Partners at the country level are aware of this and tend to contact the headquarters directly. This contrasts with the orderly country relations pattern experienced in the two countries where IFAD has outposted CPMs. They enjoy the same status as their colleagues in Rome and this is recognized by country partners.

IFAD has attracted highly qualified field presence officers, although and understandably they are not equally competent to implement all four dimensions of the FPPP. Proxies have focused on one or two main areas of work (such as policy dialogue and donor coordination). On the other hand, outposted CPMs with delegated authority have been able to mobilise national expertise to pursue all four dimensions. In general, no systematic induction and training was provided at the outset of the FPPP or for proxy field presence officers. On-the-job training has been ad hoc. No special training provisions were made for outposted CPMs either. While field presence officers have recently been given access to the IFAD Intranet and provided with IFAD email accounts, they still do not have access to other key information systems such as the Project and Portfolio Management System (PPMS) and the Loans and Grants System (LGS).

Financial issues.  It is difficult to draw an accurate picture of the pilot's costs - and for the other models of field presence. Managers and staff did not use available accounting systems in a way that would have enabled proper tracking of FPPP costs. It appears that several country pilots have spent more than the anticipated amounts, largely as a result of the escalation in staff costs. According to a recent internal audit, the actual costs of the FPPP will be around US$4 million (on the basis of a full three year implementation period for all FPPP countries), rather than the US$3 million approved by the Board for the 15 country pilots⁷. The evaluation notes that individual pilots with an average of US$67 000 per year (with a maximum of US$80 000 per country each year) are severely under-resourced to handle the variety of tasks implied by the four FPPP dimensions.

A cost analysis conducted by the evaluation on the outposting of CPMs found that this is likely to involve substantial costs. Outposting a P4-level staff member could involve either a savings of around US$12 000 or additional costs to the Fund of around US$34 000 per year, depending on the duty station and the related post adjustment entitlement. For a P5-level staff member, savings could be around US$17 000 or additional costs around US$35 000. These estimates make no provision for a hazard allowance (an entitlement in some cases), for costs related to rental subsidies or for one-time costs of more than US$50 000 per staff for duty travel, family travel and household goods removal related to the outposting of headquarters staff. The investment costs in infrastructure required to make outposted staff operational also need to be factored in. On the other hand, savings can be generated by recruiting local administrative and secretarial staff to support the outposted CPMs. All in all, it would seem that a budget neutral outcome (and in some cases savings) might be achieved only if much of the operational work arising from the planned expansion in the programme of work⁸ is transferred to field offices in countries where professional salary scales are lower than at headquarters.

Assessment of Country with and without Field Presence
Key dimensions Implementation Support Policy Dialogue Partnership Development Knowledge Management Overall	Field Presence   5.2 4.5 4.5 4.1 4.6	Comparator countries 4.6 3.4 4.4 3.6 4.0
(1= lowest score, 6 = highest score)

Performance and results. As a group, the results related to the FPPP, proxy field presence and two outposted CPMs are better across the four inter-related dimensions, in relation to the cohort of countries in the comparator group without any form of IFAD field presence (see Table 1). Performance is even better in countries where field presence was established two or more years ago. However, these results must be interpreted with caution since the FPPP was directed to countries where borrowers' attitudes and capacities were relatively favourable.

While some examples of innovations were found in comparator group countries, the results in field presence countries are better in terms of replication and scaling up of innovations. In this regard, the role of the outposted CPM in Peru stands out in terms of the promotion of innovations, as confirmed by other OE evaluations. While such innovations would not have taken place without the incumbent's special skills, the delegation of responsibility to the field was a necessary condition of success.

Ratings of the four inter-related dimensions across the different types of Field Presence

As to the before and after scenarios, apart from knowledge management, the overall effectiveness of field presence countries is rated   between moderately high and high in the other three dimensions to which field presence is expected to contribute. All three field presence models appear to yield overall moderately high to high results in terms of improving IFAD activities in the four inter-related dimensions.

Within the FPPP countries, the best results are reportedly achieved in implementation support activities. Overall, the results achieved in knowledge management were not as good but this may be due to the lack (until recently) of an overall IFAD knowledge management strategy. On policy dialogue and donor coordination, it is revealing that the results are above the FPPP averages in countries such as Mozambique, Nicaragua, Tanzania and Uganda with emphasis on sector wide approach programmes in agriculture or rural development. The same can be said for partnership strengthening.

Although the outposting of CPMs emerges as the most successful model of IFAD field presence, it must be stressed that the results are based on a sample of only two countries where IFAD presently has outposted CPMs. The largest difference in performance between the outposted CPMs and the FPPP and proxy field presence officers is in the area of knowledge management.

The group of satellite countries covered by FPPP showed broadly the same results in implementation support, but lower overall effectiveness in policy dialogue, partnership strengthening and knowledge management. This is largely because the FPPP and therefore the satellites gave more priority to implementation support than to the remaining three dimensions. It also attests to the difficulties faced when engaging in policy dialogue activities outside the duty station country.

Most proxies cover only one or at most two of the four FPPP dimensions. The area of focus is driven largely by the most pressing operational needs. Proxy field presence has been effective especially in supporting policy dialogue, donor coordination activities, and less so implementation support. One problem is that some proxy field presence officers are hired on contracts of limited duration (e.g., on a retainer basis). This may lead to conflicts of interest, when the proxy field presence officers explore employment opportunities with institutions involved in IFAD operations.

Key findings from the benchmarking study

The benchmarking study generated a number of revealing findings. All comparator organisations considered field presence to be essential to their development effectiveness, especially in the framework of the evolving development architecture with emphasis on harmonisation, donor co-ordination and aid effectiveness. It has allowed them to pursue more systematically partnerships with like-minded development organisations at the country level. They emphasized that appropriate delegation of authority to country offices was crucial and that special training, induction and oversight arrangements are needed to secure full benefits from the field presence. Costs were merely one of the criteria considered by the comparator organisations in deciding to embark on decentralisation. Each organization has pursued alternative options for country presence in order to respond to different contexts, including the setting up of regional and subregional offices to complement the work of country offices. Finally, the study noted that decentralisation has consequences for the work of headquarters, and that ongoing institutional reform processes must take full account of decentralization of operations.

Conclusions

Overall, the evaluation concludes that the field presence model tested by the FPPP has had positive results. The same can be said of proxies, and of the CPM outposting model, although the size of the sample is small. The benchmarking study confirms that a permanent field presence is widely viewed by other development organizations as central to their effectiveness. But significant resources need to be invested for field presence to be effective. In sum, the central question for IFAD is not about the rationale of a field presence, but rather about the form of country presence most appropriate for the Fund and the countries it serves.

The overall effectiveness of IFAD measured along the four dimensions of implementation support, policy dialogue, partnership development and knowledge management has been greater in countries with field presence than in countries without. The FPPP has made IFAD more visible and effective and has allowed better and more consistent follow-up. This has had wholesome effects on the quality of country programmes and projects. The results would have been better and more solidly documented had the shortcomings in the design and implementation of the pilot been recognised and acted upon on a timely basis – particularly with respect to funding, delegation of authority, legal, logistical and training arrangements.

The FPPP had an ambitious design and was under-funding. This can be seen as a reflection of the compromise that had to be reached in order to garner the acceptability of Board members, several of whom strongly favoured IFAD field presence, whereas others did not.

Based on a small sample, the outposting of CPMs with full delegation of authority to advance IFAD's objectives at the country level emerges as a highly effective option. The evaluation made an initial attempt to determine the cost of outposting CPMs, which reveals that establishing this type (but also any other less effective type) of country presence for IFAD is not likely to be cost neutral and involves significant rethinking of the role, organizational structure and functioning of the institution as a whole, comprising of both outposted and headquarters staff.

The experimentation with the satellite country approach has also proven positive, particularly as far as implementation support activities are concerned. It is an interesting option from a cost perspective. Finally, the proxy field presence approach has been effective, when focused on one or two areas such as policy dialogue and/or aid coordination. 

In sum, in spite of the limitations of pilot design and implementation and the challenges involved in assessing FPPP results (see paragraph 5), the evaluation is able to conclude that an enhanced field presence would make a significant contribution to IFAD's development effectiveness in all four dimensions. However, the most promising approach to decentralization based, admittedly, on a very small sample (CPM outposting) was not tested under the pilot. Nor were the other options tested systematically in diverse country contexts and in conjunction with appropriate delegation of authority and suitable training and induction support. Moreover, it is not possible to conclude without access to better cost data that a budget neutral outcome can be guaranteed. In fact, available evidence (amply confirmed by the benchmarking survey) suggests that the full benefits of decentralization may require substantial incremental budget outlays.

Due to all these considerations, the pilot must be considered a missed opportunity even though enough reliable evidence now exists to confirm the need for an expanded field presence program in order to allow IFAD to play its distinctive role in a relevant, effective and efficient manner within a development environment in rapid transformation.

Recommendations

The evaluation has two specific recommendations: (i) embark on an expanded Country Presence Programme; and (ii) work towards the formulation of a Country Presence Policy by 2010.

Embark on an expanded Country Presence Programme

Given that the FPPP did not succeed in providing a conclusive indication of the most effective form of field presence for IFAD, the evaluation concludes that it is premature to propose a mainstreaming of the initiative. Instead, the FPPP should be transformed in a new programme - the IFAD Country Presence Programme (CPP)⁹ - that would aim at consolidating the evidence around the positive results as well as determining the most cost effective form of country presence that IFAD should adopt in diverse country contexts. The CPP would consist of two distinct tracks:

• • Continue implementation of existing FPPP country initiatives; and
  • Expand the programme to allow systematic experimentation with alternative country presence models.

First, the evaluation recommends the continued implementation, under the CPP of all FPPP country initiatives, whether they were due to complete their three year implementation by the end of 2007 or not.

In parallel, the evaluation recommends an expansion of the CPP to allow for experiments that were not undertaken in the first phase of the FPPP, namely with outposting of CPMs and establishing subregional offices. Specifically, it is suggested that the FPPP be expanded to cover an adequate number of countries in all IFAD regions, including 2 to 3 subregional offices located in different regions. The expansion should entail the outposting of around 10 CPMs in FPPP and other countries, with preference given to large and active country programmes.

For all countries in the CPP, a reassessment of budget allocations should be made, to ensure that each country pilot has access to the funding needed to achieve the intended objectives. Proper use of IFAD's accounting system should be ensured so that accurate monitoring of costs related to the CPP is carried out. Finally, a platform for sharing experiences across the concerned CPMs and field presence officers should be set up with full access to the PPMS and LGS provided for all country offices

Furthermore, it is imperative that all monitoring, evaluation and reporting measures be put in place to ensure the evaluability of the extension phase in order to avoid the shortcoming of the first pilot phase. The need to collect baseline data in all countries under the CPP is especially critical.

The Management should be comprehensively engaged in country presence issues, for example in ensuring that adequate delegation of authority is provided to field presence officers and that appropriate systems are in place for training, induction, coaching and oversight of outposted personnel. The delegation of authority to field presence officers from headquarters should be articulated with clarity and realignment of responsibilities between field and headquarters staff should be specified to minimize duplication and enhance accountability. Where field presence officers have consultancy contracts that have performed competently, IFAD should devise specific legal instruments that allow their contracting as local staff.

A cross-departmental committee should be established to facilitate organizational learning and discussion of cross-cutting issues emerging from the CPP. Furthermore, IFAD should consider taking the lead in forming a committee of Rome-based UN agencies on country presence issues, as a forum for exchanging experience and good practices.

Development of IFAD's Country Presence policy after 2010

The evaluation concludes that it is premature for IFAD to formulate its country presence policy, given the limited experience both in terms of the implementation duration and diversity of country presence models experimented under the FPPP. Therefore, it is recommended that a self-assessment of the CPP (including the FPPP) be undertaken by IFAD Management in 2010. This would serve as the basis for the development of IFAD's comprehensive country presence policy to be submitted for approval to the Executive Board following the final assessment in 2010.

---