Extract of Agreement at Completion Point

Completion evaluation¹

Core learning partnership and evaluation users

The Office of Evaluation of the International Fund for Agricultural Development (IFAD) conducted a completion evaluation of the Development Project of the Mayan Communities in the Yucatan Peninsula. An evaluation mission visited the country in November and December 2004, concluding its field activities with a workshop that took place in Mexico City on 16 December 2004. On that occasion, an aide-mémoire was presented to the stakeholders outlining preliminary conclusions adjusted on the basis of comments received at a regional evaluation review workshop held on 13 December in Merida, Yucatan.

The IFAD project evaluation process is built upon a shared learning approach involving all stakeholders. To this end, it calls for creation of and participation by a Core Learning Partnership (CLP)² and an Broad Learning Partnership (BLP). The latter includes major evaluation users: the Government of Mexico, represented in the project by the State Secretariats of External Relations (SRE) and Finance and Public Credit (SHCP), by Nacional Financiera SNC (NAFIN) and the National Indigenous Peoples Development Commission (CDI), which replaced the National Indigenous Institute (INI) initially responsible for project implementation; IFAD as the financial institution; the United Nations Office for Project Services (UNOPS), which is responsible for project supervision; and the target population organizations participating in and benefiting from project implementation, including regional funds and grass-roots organizations. Civil-society organizations responsible for performing national project evaluations are also members of the Broad Partnership.

On 15 March 2005, a seminar was held in Mérida to present and discuss the evaluation results with several BLP members, chiefly the regional funds, in order to share observations and views, listen to comments and suggestions, and adjust the final evaluation documents as appropriate.

At the final meeting, which took place in Mexico City on 18 March 2005 with the participation of BLP member institutions, the agreement at completion point on major findings and recommendations³ was discussed and agreed to.

Main evaluation findings

Performance of the project, which is now nearing completion, can be considered satisfactory from the point of view of achieving its physical and financial targets, despite two extensions⁴ that have taken the completion date to 31 December 2004 and the loan closing date to 30 June 2005.

The project has brought about some improvements in the beneficiary population's incomes and living conditions. Although the changes in production and/or productivity cannot be quantified⁵, field observations pointed up improvements in basic technology and diversification of production and an increase in the number of animals⁶. In any case, the financial and technical supports from regional funds (RFs) has helped to generate options for self-employment and income generation, diversifying livelihood options for families.⁷

When the trust fund system called for in the appraisal document was replaced with participation by RFs in funding projects, the RFs became the cornerstone of all project operations.

The project contributed to increasing the number of RFs and strengthening them through the allocation of financing, both to grant credits to the target population and for technical assistance, training for technicians and executives, and the purchase of equipment in support of management and administration. However, local capacity-building for autonomous, participatory management is still at an incipient stage.

The approaches and mechanisms introduced by the project in terms of financial, administrative and accounting management⁸ and support for productive investment have proved to be a useful contribution to the implementation of development programmes and have been institutionalized by the CDI and included in other regions and programmes.

The project has influenced the inclusion in RFs of a large number of working groups and grass-roots organizations, motivated by access to financing. These associations are generally family-based groups⁹ that frequently consider the RF not an institution in itself but rather a governmental agency from which they can obtain resources. In practice, many of them lack the ability to manage financing, obtain technical support for production, or work and market their products collectively.¹¹

The fact that the project explicitly set a minimum percentage for women's participation in its activities, and suggested conditions to give women more of a leading role and direct participation in decision-making by production groups and RFs, is viewed as having achieved significant concrete positive results.

Efforts by the implementing organization (to set up a monitoring system, improve procedures, create a coordination body, etc.) were insufficient to overcome weaknesses in the coordination of actions, systematization of experiences, and compilation and organization of relevant data, as well as dissemination of related information.

With respect to the environment, the project activities and orientations have, in the great majority of cases, been respectful of the traditions and values of the predominant cultures and of natural resources and the environment generally, providing support for organic, balanced production processes.

IFAD's involvement was decisive in making the loan operation a reality. The cooperating organization acted in accordance with its project supervision role.

Evaluation by specific themes

Policy, institutional and management issues

Changes of a policy and economic nature occurred in public organizations during the first few years of the project, in particular the transformation of INI to CDI, an autonomous decentralized agency. The project's budget was cut significantly as a result, affecting in turn the flow of loan proceeds. In addition, resource allocations and loan disbursements were often delayed.

For CDI, which handles a large number of projects and RFs nationally, the importance of the IFAD project is chiefly qualitative. Many of the conditions and procedures have been included in the RFs outside the project. CDI is undergoing an effort to modernize, streamline and decentralize management.

No systemic approach was put in place to promote and transfer knowledge, support grass-roots organizations and disseminate information. Project actions to structurally strengthen participating non-governmental institutions were less effective as a result.

The fact that CDI itself was implementing the project may have been decisive in introducing changes of approach and priority in its programmes, as well as adopting procedures that were gradually assimilated and entrenched as regular practices within the institution.

Financing for productive investment

The percentage of women among credit users rose significantly, from 23.65% in 1998 to 55.15% in 2004, and the presence of women¹² among RF executives increased as well. Considerable unmet demand persists however within groups comprising mainly women, despite their good performance as borrowers.¹³

From a financial point of view, most RFs associated with the project are gradually losing capital.

Technical services in support of production

The project effectively contributed to improving financing applications and preparation of technically and economically viable productive projects. The committee set up in 2003 to review project proposals for quality and approve applications had a favourable impact on recoveries during 2004.

The technical assistance introduced by the project made a significant contribution to the operation of the RFs. In some cases the desired results did not materialize, in part because of a non-specialized approach. Other factors were frequent changes in field personnel, insufficient availability of replacement agents suitable for working in Mayan areas, long distances and a lack of regular transportation, and a lag between approval and delivery of funds. Generally speaking, little attention was given to the technical aspects of crafts projects or marketing. The Technology Assistance Council was not consolidated, despite a collaborative effort by other specialized institutions.

Promoting demand for services

There are limitations on the current capacity of producers' groups and associations to arrange for financial, commercial and technical services in support of production. Such arrangements are generally made by the technical assistants of RFs and CDI operating units. The organizations have not made use of the opportunity to tap resources available within the community, and controls on resource management in RFs have been neglected.

The family-based nature of the project's producer groups and organizations lends cohesion and a certain permanence, but the lack of sufficient capacity¹⁴ and adequate integration with RFs make continuity of action difficult.

The training programme objectives were not fully realized among the target population, particularly women's groups.¹⁵

The communications subcomponent was not initially conceived as a development instrument in support of information and training processes. However, some positive results were achieved in this regard during implementation. The use of the Mayan language over the radio to broadcast project information demonstrated effectiveness and high potential.

The use of indigenous school residences was effective and displayed high potential in transferring innovative and best practices.

The teaching materials, designed by the project in support of training, promotion and dissemination, were of good quality but little used and limited in distribution.¹⁶

Implementing Unit

The fact that CDI itself was implementing the project may have been decisive in introducing changes of approach and priority in its programmes, as well as adopting procedures that were gradually assimilated and entrenched as regular practices within the institution.

The creation of a peninsular coordination team within the structure of CDI did not in practice make any significant contribution to the coordination of project components and activities.¹⁷

The monitoring and evaluation system adopted by CDI represents important progress in terms of design, scope and details. Still, it did not manage to make information available easily and on a timely basis, nor to enable information to be used as feedback for operating units of RFs with reports addressing their specific needs.¹⁸

Recommendations approved by all stakeholders¹⁹

Policy, institutional and management issues

Decentralizing implementation. Promote the transfer of decision-making on project implementation and resource administration to organizations and communities within the target population, with less intervention from the central government and regional offices, while at the same time promoting participation by organizations, in particular RFs, in planning and decision-making on their own activities, and in local and regional development. Follow-up: Government; IFAD; cooperating institution.

Coordinating project actions. Promote the incorporation of mechanisms to strengthen the coordination of project actions. More important than permanent coordination bodies are a free flow of information, opportunities for dialogue and instruments for consensus-building among the various stakeholders involved in implementation. Follow-up: Government.

Regularity in resource allocation. Take steps as necessary to ensure that project budget funds flow as programmed, and to reduce delays between approval and disbursement of cooperation funding. Follow-up: Government; IFAD; cooperating institution.

Regional funds

Financing productive investment

Defining regional funds. Define the financial and social nature of the RFs and, based on the criteria adopted, come up with strategic actions: development and operability of the structure, efficient management capacity, funding required, diversification of services. Follow-up: Government.

Institution-building for RFs: Protect RFs from political contingencies, strengthen their plural nature to reflect low-income rural populations, promote their involvement in participating communities and build together with those communities a future vision for the institution. Focus on improving organizational, administrative, management and service capacities of RFs and strengthen participation by executives in operational controls and monitoring. The goal is efficient resource management²⁰, with assistance from participating organizations²¹. Follow-up: Government.

Training for RF executives. Inform and train RF boards of directors adequately with respect to legal and institutional instruments relating to the institution's activities and specific circumstances, as well as basic aspects of accounting management and interpretation of accounting statements, and useful instruments²². Follow-up: Government.

Recovering subloans. Place emphasis²³ on the responsibility of borrowers to repay the loans granted, gradually generating a shift in behaviour and culture, while addressing other important objectives in social and organizational areas. Before an organization of very few borrowers is excluded, an effort should be made to rehabilitate it. In designing new projects for populations lacking minimum assets and productive resources, a realistic assessment of the likelihood of recovering small loans should be made and consideration given to other kinds of viable assistance. Follow-up: Government; IFAD.

Diversifying sources of financing. Arrange for making use of the various sources of financing in existence in Mexican institutions to alleviate poverty and promote rural development. The RFs should play an active role in seeking and obtaining non-traditional sources of financing, generating services of a financial or social nature. Follow-up: Government; IFAD.

Replicability of RFs. Consider the possibility that the RF model, with adjustments and refinements as needed, could be replicated for the organizational, economic and productive strengthening of rural villages and indigenous communities in Mexico. Follow-up: Government, IFAD.

Technical services in support of production

Diversification of production. Develop methodologies and instruments to promote diversified production and sustainable technological innovation (consistent with the culture, environment and sound economics). Follow-up: Government.

Specialized assistance. Guarantee specialized advisory assistance to the technical assistance service for productive projects financed by RFs and create mechanisms to provide support through experienced consultants specializing in relevant issues. Follow-up: Government.

Quality of technical assistance model. Conduct an in-depth review of the technical assistance model adopted, in order to make the service more effective and more efficient. Develop an intervention strategy diversified by category of family beneficiary, by sex and age. Consider, for the medium and long term and once productive units are operating efficiently and producers have raised their incomes, including part of its cost in the loan. Follow-up: Government.

Gender perspective. Build an explicit gender focus into the project design; in dealing with social and organizational issues, particularly training, highlight the importance of clear functions and qualified trainers, assistants and promoters in these areas. Follow-up: Government; IFAD.

Improvements in applications and approvals. Maintain the requirement to improve applications and productive projects and for transparency and rigour in selection processes. Follow-up: Government.

Support for demonstration plots in residences. Step up financial and particularly technical support for demonstration and dissemination activities in indigenous residences. Follow-up: Government; IFAD.

Exchange and systematization of experiences. Strengthen the holding of events to exchange and systematize experiences, both among RFs and among technical assistants from different regions and member organizations. Follow-up: Government.

Training of trainers. Emphasize support for training of trainers (technicians and producers), in order to empower the population and lend sustainability to the improvements introduced by the RF programme. Follow-up: Government.

Producers' demonstration plots. Recover for technicians and member groups, demonstration plots on land owned by members, for applied research and to verify improved techniques, while ensuring that they fulfill the assigned role of demonstration and dissemination. Follow-up: Government.

Promoting demand for services

Strengthening organizations. Promote the idea that the objective of creating a group should extend beyond a simple short-term purpose, need or interest. Help strengthen the internal organization of groups and communities for decision-making purposes, and step up participation by current and potential beneficiaries in assemblies and decision-making bodies. Review conditions and methodologies for institutional support of organizations, particularly in technical respects (production, accounting, organization and others) and marketing. Follow-up: Government.

Vision of RFs within organizations. Encourage groups associated with RFs to be aware that the RF is an organization that they belong to and that belongs to them, and that they should take an active part in its activities, decisions, monitoring and control, so as to ensure greater efficiency and transparency and to achieve the organization's objectives. Follow-up: Government.

Information and dissemination. Support RFs in the intensive dissemination of their programmes, regulations and requirements. Promote efficient and effective strategies and mechanisms, as a complement to technical assistance, in order to guarantee social and generational equity²⁴, as well as internal cohesion. Follow-up: Government.

Implementing Unit

Institutional experience with implementation. Conduct an in-depth review of experience resulting from the implementation of projects by a pre-existing national or regional institution having been strengthened to that end.²⁵ Follow-up: Government; IFAD.

Monitoring of implementation. Underscore and provide support so that implementing institutions develop and maintain up to date project indicators and instruments²⁶ that will facilitate monitoring and timely corrective action during implementation. Require that a baseline study be performed during the first year of project implementation and ensure timely and proper implementation of interim evaluations. Follow-up: IFAD; Government; cooperating institution.

Fulfilling recommendations. Take steps to ensure that recommendations arising from supervision visits and evaluations are actually analysed and applied by the implementing institution, to the extent appropriate and relevant. Follow-up: IFAD; cooperating institution.

Recommendations not agreed to by all stakeholders

With respect to the training programme, the Government of Mexico does not share the mission's views on training.

According to the evaluation mission, training activities addressed to leaders and delegates of RFs, particularly those relating to administration and management, were not undertaken properly. There was no differentiated training (with respect to relevance and coverage of issues, menu of offerings, training instruments, etc.), nor was training applied effectively.

According to the Government, teaching and learning processes are a central thrust for CDI and the project, and a differentiated strategy was implemented at the peninsular and state levels, including methodological topics (such as gender and the exchange of experiences), as well as technical issues relating to production and establishing a menu of offerings for project beneficiaries through regional training programmes.

Agreed to and signed in Mexico City on 18 March 2005.

Raúl Delgado
Deputy Director, Agricultural Financing Projects and Subnational Governments
Secretariat of Finance and Public Credit

Jesús Eugenio Huerta González
Director, Multilateral Financing Institutions
Secretariat of External Relations

Eduardo García Dagio
Sector Executive
Financial Agent
Nacional Financiera S.N.C.

Marco Antonio Del Castillo Hernández
General Coordinator, Special Programmes and Projects
National Indigenous Peoples Development Commission

Enrique Murguía Oropeza
Programme Manager for Mexico
Latin America and the Caribbean Division
Programme Management Department
International Fund for Agricultural Development

---

1/ The agreement reflects the understanding reached among the institutions associated with project implementation and included in the CLP with respect to the findings and recommendations set forth and the willingness to adopt and implement such recommendations where relevant.  The ACP was signed by Mr Raúl Delgado, Deputy Director, Agricultural Financing Projects and Subnational Goverments, Secretariat of Finance and Public Credit; Mr Jesús Eugenio Huerta González, Director, Multilateral Financing Institutions, Secretariat of External Relations; Mr Eduardo García Dagio, Sector Executive, Financial Agent, Nacional Financiera SNC; Mr Marco Antonio Del Castillo Hernández, General Coordinator, Special Programmes and Projects, National Indigenous Peoples Development Commission; and Mr Enrique Murguía Oropeza, Programme Manager for Mexico, Latin America and the Caribbean Division, Programme Management Department, IFAD.
2/ Representatives of the Secretariats of External Relations (SER) and Finance and Public Credit (SHCP), Nacional Financiera SNC (NAFIN), the National Indigenous Peoples Development Council (CDI), and IFAD.
3/ Referred to as "suggestions and proposals" in the Mexican institutional context.
4/ The institutional changes, budget cuts and decentralization of operating units within the implementing institution (INI/CDI) that occurred at the outset of project implementation limited the ability to disburse the loan proceeds.
5/ The project monitoring and evaluation unit plans to process this data by the time of administrative closure.
6/ Any surpluses generated by the production projects were used to invest in animals, reducing on a small scale the vulnerability (risk) of participating families and enabling contingencies to be dealt with.
7/ Actions by RFs within the project context have helped to raise the value of family work, having an impact on paid family employment and, generally, opening up options for productive employment for those involved. For the project's target population, the RF provides the only accessible credit, besides subsidy-based assistance programmes.
8/ Including operating regulations, manuals, standards and procedures.
9/ Extended families or various branches of the same family, with some close neighbours.
10/ Many of the weaknesses noted may be attributable to the lack of a systemic support strategy for the community and groups within it, as well as the lack of technical staff with training in social fields (organization, training and communication, anthropology and gender issues).
11/ On the other hand, the very limited scale of the operations for which financing was provided, especially in terms of land, reduces any potential environmental impact.
12/ Most of those interviewed indicated that in connection with their requests for financing they: (i) had not been given a timely response; (ii) had been redirected to other branches of production; (iii) had not received needed technical support; and (iv) had not received training appropriate to their needs and opportunities.
13/ In addition to their good payment record, women in participating groups having received support for traditional tasks had accumulated family savings (generally in the form of livestock), obtained a minimum income that lent their daily work a value and, following a second loan, acquired some goods or equipment to carry out their productive activity.
14/ In technical, commercial, financial, administrative and organizational terms.
15/ Very few of those interviewed demonstrated that they had acquired additional knowledge and skills such that they could carry out their new activities independently.
16/ The mission frequently could not find these materials in use outside of CDI offices.
17/ The peninsular coordination concept presupposes performing functions that in practice are carried out through an exchange of experiences among those responsible for programmes at the state level. Their functions have focused mainly on supporting the monitoring of implementation, documentation of indicators, systematization of experiences and training activities, creating an overlap with other state agencies in some cases.
18/ Continuing efforts are required to complement and harmonize information among state agencies and RFs.
19/ Follow-up on these recommendations will apply to IFAD/the cooperating institution as and when new operations are developed with IFAD.
20/ Interest rates that protect monetary value, payment terms appropriate to the target population, technical management of production projects, portfolio diversification, and others.
21/ Facilitating loan repayment, providing support, preapproving credit, fulfilling and participating in rights and responsibilities assumed as members.
22/ This training should include assembly delegates and/or their children.
23/ However, loan collection activities should not be incumbent upon individual executives, nor should organizations and their members become accustomed to paying on the occasion, or as a result, of a visit by an executive.
24/ To take advantage of opportunities (experience and knowledge of elders, capability and energy of youth) and reduce the inequities that exist within participating families (limited access to goods and services, excessive workloads for women).
25/ One important advantage is the institutionalization of programmes, along with measures or instruments introduced, possible economies of scale and improved prospects for sustainability of the actions undertaken. Among the negative aspects are the risk of diluting efforts and attention, partial diversion of project resources towards other programmes, programme changes and greater impact of party politics.
26/ Providing information requires a constant effort to ensure complementary and harmonized action by implementing agencies, so that it is easily available in a timely fashion, and priority should be given to using data for feedback in the form of reports addressed to specific needs.