Interim Evaluation Report

Background and present status

Agriculture is the main sector of the Guatemalan economy, accounting for about 26% of GDP and providing employment for over 50% of the population. Census figures (1980) indicate that there were at that time over 450.000 small farmers (less than 3.5 ha), and estimates are that over 550.000 small farms exist today. These small farms make up over 80% of the total number but occupy only 10% of the area in farms.

Most small farms are concentrated in the highlands, the Altiplano, where the population is predominantly of mayan ancestry, and in the East or Oriente, whose population is mixed or mestizo. Increasing population pressure is forcing a major migration to the lowlands of the Peten Region. Strong cultural traditions exist among the people of the Altiplano, protected mainly by their use of various indian dialects, although now most male adults also speak Spanish as a second language. Both in the Altiplano and Oriente as well as Peten, rural poverty is widespread. Nationwide it was estimated (1980) 71% lived in poverty and 40% in extreme poverty.

The majority of small farmers produce food crops, most of which they keep for their own family's consumption, selling off some in order to buy other needed goods. In some areas of the Altiplano, where the average size of farm rarely exceeds half an hectare, families compensate by working off-farm and doing artisan work in textiles, pottery and other activities. The main food crop is corn, with beans, wheat, sorghum, potatoes and faba beans.

After a long period of violent disturbances, especially in the Altiplano, which were put down by successive military governments in the early 1980's, Guatemala entered a period of freely elected government in 1986, which substantially reduced restrictions in the countryside. Government development policies have emphasized export led growth, basically seeking to open the economy by creating incentives to export new products, mostly of agricultural origin. At the same time however, in order to improve internal food production conditions, public sector resources were focused on small farmers and poor rural dwellers, especially on those in the Altiplano. General economic policies however, did not provide incentives to food producers, as prices for most products tended to lag behind the general price level until 1990.

The Generation and Transfer of Agricultural Technology and Seed Production Project (PROGETTAPS), was designed in the late 1970's in response to policies directed at increasing food production and income of small farmers. Low farm productivity was identified as a major problem for making better use of resources by small farmers, so that the development and transfer of improved farm technology for this type of farmer was made the focus of this new project. PROGETTAPS was first of all to develop the institutional capacity to provide effective agricultural services, by creating the means to integrate research with extension. Secondly, PROGETTAPS would set up an efficient technology transfer system, thus ensuring that farmers received a continuing technological support for their production activities.

The original proposal for the project was presented to the Interamerican Development Bank (IDB) and accepted in 1983, when the Bank and the Government invited IFAD's participation in the project. Project appraisal was completed and loans approved by the Bank and by the Fund's Executive Board by late 1984. Actual project implementation started only in early 1986 and after approval of various loan extensions, IFAD funding is expected to end in 1993. Project implementation was assigned to ICTA (research component), DIGESA and DIGESEPE (transfer of technology components) ICTA/DIGESA (seed production component); administrative coordination and monitoring and evaluation activities were to be handled by USPADA.

The project rationale was "to support and strengthen both the existing research and transfer of technology efforts by consolidating the dispersed and scarcely coordinated actions, in order to maximize the impact of those integrated services on small (and some medium) farmer's incomes and food production". (IFAD. Report and Recommendation of the President to the Executive Board, August 1984, p.8).

Entering its last year of implementation, PROGETTAPS has substantially achieved its objectives. It has:

a. Strengthened the agricultural public sector's capacity to generate and especially to transfer new technology to small farmers;

b. Developed an effective methodology for integrating research and extension activities, which the Ministry of Agriculture is now seeking to extend to cover the whole country;

c. Managed to provide effective services to a large number of small farmers (some 30.000 directly, and many more indirectly).

PROGETTAPS has achieved the above, while using less than the US$ 24.2 million originally budgeted for the project. Even taking into account that the project implementation period increased from four to eight years, total cost is estimated to reach some US$ 21.9 million on completion in 1993. Devaluation of both the Quetzal with respect to the US dollar and of the US dollar with respect to the SDR, made it possible to stretch local funds. The IDB loan was cancelled partially in early 1992, as the remaining funds were considered undisbursable before the end of the loan. In regards to IFAD's loan, a substantial amount will also remain undisbursed by 31 January 1993, which is the present closing date for the loan.

Conclusions

Concerning project design

The project was kept reasonably simple, with just four components. The design feature helped to keep project coordination within acceptable limits and the project coordination system, based on one executive and one technical committee, served adequately throughout most of the project period, to successfully integrate the institutions involved.

The strategy incorporated in the project design was vindicated, specifically in regards to the premise that there existed a sufficient accumulated supply of technology, such that it could be used to promote an important change in the technology employed by small farmers. The project provided important support and resources to mobilize those public sector institutions involved in technology generation and transfer (TGT), basically by supplying funds to cover operational expenses, such as per-diem expenses of extensionists, fuel for vehicles and particularly agricultural inputs to allow for the setting up of technology transfer plots in the farm communities involved. The project thus allowed those institutions to make full use of their trained manpower (researchers, technology validators and extentionists), new integrated work methodology, and accumulated technology, to offer small and medium farmers substantially better production possibilities.

Another important project premise confirmed, was that the existence of improved varieties was a key factor for introducing new technology, and that availability of seed of these varieties was critical. ICTA's research before and during the project, made available new improved varieties. However, it was not PROGETTAPS's seed production component, designed to produce certified seed which eventually provided the required input, as this seed proved to be costly and not physically accessible to small farmers. Instead, the adoption of a grassroot proposal to have the project farmers produce their own seed, provided PROGETTAPS with a very important tool to involve the farmers themselves more directly in the TGT process to develop a new production alternative for some farmers (i.e. seed production), and to ensure the supply of sufficient good quality seed at a reasonable cost at the community level.

The modular approach on which PROGETTAPS based its work in the field, allowed it to expand its area coverage without much effort, once this expansion was accepted by the cooperating institution. This expansion was very substantial, as originally it was to cover only five sub-regions, and by 1992, it had reached 15 sub-regions.

The PROGETTAPS methodology

Through PROGETTAPS, the institutions involved with TGT for small and medium farmers, managed to develop a methodology for working together and for sharing responsibilities in regards to those phases where two or more institutions were involved. This methodology is now shared between ICTA and DIGESA (crops) and to a lesser extent by DIGESEPE (animal husbandry). Acceptance of the methodology by the professional and technical staff, especially in ICTA and DIGESA, has led them to develop a TGT institutional culture, in which they share certain common planning, operational and review procedures. In addition, it has lifted the status of the extension workers, by training them in the use of certain basic research tools, thus allowing them a greater participation in the TGT process than ever before, increasing their degree of self esteem.

The PROGETTAPS methodology for crop production: In the case of crop production, the methodology was evolved from that already in use by ICTA for carrying out farming systems research. This initial methodology provided for agro-socio-economic surveys (sondeos) to be used by multidisciplinary teams to identify with farmers the problems they faced, whereupon researchers would begin to investigate technically and economically feasible solutions, and the research results would be put to trail, especially in farmers fields to check results. ICTA's methodology provided for the next step of carrying out validation trails with farmers, but went no further towards transferring the new technology to other farmers. PROGETTAPS improved and innovated, by completing the design, so as to, on the one hand, involve extension staff in the last steps of the research process (on-farm research plots and validation trails), and providing them with an effective method for reaching farmers with a clear message whose results could be demonstrated (transfer plots); and on the other hand, systematically involving ICTA's technology validation teams in the follow-up of results, by working closely with the extension workers.

The initial joint ICTA-DIGESA methodology has proven its worth and although some modifications have been introduced, it operates essentially as designed. Targeted to reach upwards of 40.000 small farmers, it has nearly achieved this goal. Its success is based both on the simple technology being transmitted (improved varieties in almost every case) and in its ability to incorporate and take advantage of new tools that appeared as it was implemented. One, already mentioned was the artisanal production by small farmers of their own seed of improved varieties (PASM). The rapid expansion of the PASM programme after 1987 with the full support of PROGETTAPS, was instrumental in the widespread diffusion of new varieties that is now observed in the project areas. It also is generating a new production activity: seeds; a new source of income for small farmers who receive technical assistance to carry it out.

The other was the rapid and full incorporation into the methodology of the "representantes agricolas" or RAs, a mechanism developed by MAGA to better link the sector institutions with the farm communities where they worked. PROGETTAPS had a similar type of agent included in the original methodological proposal, who it was expected would serve in a relatively modest as an assistant to the extensionists. However, in practice the role of the RA has become of much greater importance to PROGETTAPS than originally envisaged. It was through the RAs, that the project managed to gain the confidence of the farmers, and by ensuring that, these in fact were considered good representatives of the communities they served, PROGETTAPS has been successful in reaching an approximately 30.000 or more farmers with the new varieties.

Finally, the methodology has also been flexible and adapted to others beyond the original group of farmers it was intended to serve. Thus, in 1988 PROGETTAPS's activities were extended to cover the whole of the extension agency's clientele, including both women groups and youths. Now both types of beneficiaries are receiving seeds of improved varieties and are planting them in individual or communal plots. In practice, however, not all aspects of the methodology were as well applied. The socio-economic data collected have not been consistently used to improve project performance as designed, and there is insufficient evidence to substantiate some project claims, such as adoption rates, productivity and income increases.

The PROGETTAPS methodology for animal production: For animal production, the method differs significantly. The techniques that were to be passed on to medium and small cattlemen comprised a complete and relatively expensive package, which meant that no simple transfer plot would suffice to show the benefits of the new technology. DIGESEPE in addition had no substantial experience in transferring technology to farmers and ICTA had no major stock of new techniques, the proposed technology depending on results obtained by a previous ICTA/CATIE project.

This project component targeted a relatively small number of producers (560), seeking the full incorporation of the proposed technical package. In fact, the method had to undergo substantial changes, since it rapidly became clear that the package was too expensive for the beneficiaries. During the third year of project implementation, a change was made to limit technology transfer to introducing animal feed for the dry season as the basic component. This has been diffused to small/medium cattlemen with substantially positive results. The work relations between ICTA and DIGESEPE, while adequate, lack the degree of coordination found in crop production. No attempt was made to incorporate the RAs as a tool for transfer of technology. Thus the method for animal production still requires review before it can be extended to a substantially greater number of producers.

Results obtained through 1992

Institutional Development

As stated in the previous section, the project managed to design and institutionalize a working methodology, and in doing so integrated research and extension work procedures at the field level, as well as achieved a substantial degree of institutional coordination at higher levels, through the participation of high level staff from each institution in the project's coordinating committees. Its relative success in this area is reflected in the decision to expand coverage countrywide. The project allowed a substantial re-equipping of ICTA, DIGESA and DIGESEPE, with vehicles and other equipment, as well as providing for graduate-level training for an important number of technical staff. PROGETTAPS also provided indispensable funds to cover participating institution's operating costs in the period since 1986. Where it was not successful, was in reverting the tendency to reduce government funding for TGT in real terms. The sustainability of PROGETTAPS in the future is thus linked to obtaining continued external funding.

Changes in crop production

Changes in technology generation capacity: The project has supported ICTA's crop production research both through direct funding of research operations, as well as through specialized training received by its staff. Much of the available technology at the start of the project has now been disseminated to farmers, and while not too many new varieties were generated in 1986-1992, the Institute carried out a continuing improvement of existing varietal lines. New materials are now undergoing trails and are expected to be released in one or two years time, thus assuring a continued supply of improved or more resistant varieties.

Crop production and technology adoption: The 1986-1992 implementation period has been marked by important geographical changes in Guatemala's food crop production. Generally, the areas covered by PROGETTAPS have increased crop production less rapidly than new areas on the Pacific coast (which substituted corn and other grains for cotton which practically vanished as a commercial crop) and in the Petan (newly opened as the agricultural frontier). Thus in terms of its contribution to increased production of food and grains, PROGETTAP's role has been minor in the aggregate.

Exceptions to this did occur. Thus the bean producing areas of the Oriente, where PROGETTAPS was very active, have increased their share of national production. More important, however than its contribution to national production levels, has been the adoption of improved varieties by small farmers. In corn, improved varieties have been adopted in a widespread manner, especially in those areas under 1.300 meters. At higher altitudes ICTA's varieties while adopted to some extent, are not markedly superior to local material, with adoption rates being probably not higher than 15%. However, in areas of the Altiplano where intensive horticulture for export is been introduced, ICTA's short-cycle materials are being preferred, as this allows for second cropping. In beans, adoption rates have been higher, perhaps as high as 30%, as ICTA varieties show more resistance to disease than local varieties do. In other crops such as potato and faba beans, PROGETTAPS has obtained success in having new varieties adopted, but its main impact has been in introducing these as new crops (potato in the Oriente), or in re-introducing important food crops abandoned in the recent past (faba beans in the Altiplano).

Yields: While partial data available would suggest that significant yield increases have been obtained by using PROGETTAPS's varieties, much work has still to be done to substantiate this.

Changes in animal production

In spite of its smaller size, the animal production component has reached only some 360 small and medium producers, less than its target figure. However, among those, the effects obtained from the adoption of improved cattle feeding in the dry season were very substantial, both in terms of milk production and of income. If DIGESEPE has shown greater interest in improving its technology transfer capacity, probably a much greater number of producers would have adopted the improved techniques.

Project Administration

Overall the coordination of project activities functioned as designed. The Executive Committee made up of the principal officers of all four participating institutions (including USPADA), however, did not meet as often as planned thus limiting its usefulness in solving important problems. Only after a General Coordinator was named in mid-1991, did many planning, budget and loan disbursement matters receive the required attention of the Executive Committee. The Technical Committee on the other hand, functioned well throughout much of the implementation period, formulating annual programmes, handling day-to-day technical matters, and carrying out a supervision based on periodic visits to the field, to observe project activities and discuss these with regional and local staff.

The channelling of project funds and handling of reimbursement requests to IFAD and IDB were generally unsatisfactory. Since no overall high-level coordinating body existed prior to mid-1991, the project's Administrative and Financial Coordinating Unit (UFCA), established since 1986 was ineffectual. IFAD funds assigned for a special account were never used as a revolving fund to facilitate implementation. Insufficient authority of UFCA over the financial and administrative units of ICTA, DIGESA and DIGESEPE, meant that these were often very late in providing reimbursement requests for IDB and IFAD loan funds, thus reducing funds available for field operations. Other problems arose with construction works funded by IDB, that were not completed, and with the late purchase of vehicles with IFAD funds. Some improvement is noticeable in project administration with the designation of a General Coordinator in 1991.

Supervision

IDB acted both as co-financier and as cooperating institution. Project supervision was carried out by IDB's Office in Guatemala. In general, supervision on technical matters was of good quality, especially from 1988. The assistance provided by IDB's agricultural loan officer was widely recognized as being helpful by the implementing agencies. Considerable flexibility in interpreting the project concept by the loan officer, resulted in expansion of the project area, in including women and youths as beneficiaries, in developing the artisanal seed programme, etc.

Problems were detected, however, in regards to loan disbursement procedures used. IDB requested too much detail in loan reimbursement requests from the project, thus unnecessarily increasing paperwork and being in addition very rigorous, by returning reimbursement requests which contained only minor errors that could have been handled as routine amendments to the documents. The non-use of IFAD's special account would indicate that the Bank office was not familiar with this long-standing Fund procedure.

Monitoring and evaluation

In spite of an early attempt in 1985 to establish an adequate M&E system, PROGETTAPS functioned throughout many years without an effective monitoring mechanism. Only since 1989 has the M&E Unit been fully operational. When designed, the M&E Unit was assigned duties beyond that of following up on PROGETTAPS, as it was also given functions as a sector M&E Unit. A number of useful studies have been made in monitoring project activities, and the Unit is considered as being very useful by the General Coordinator. A number of problems raised by the interim evaluation mission, indicate that it is necessary for the M&E Unit to improve its data gathering and in particular its data analysing and synthesizing functions, controlling also the reliability of the data.

Recommendations

For PROGETTAPS's completion

Scheduled completion of disbursements with IFAD's loan is 31 January 1993. The mission considers that by that date very substantial funds will still have remained uncommitted. On the other hand, additional government or other external funding to finance operational expenses of agencies participating in TGT efforts for small farmers will not be forthcoming in the short-term. Taking into account the substantial achievements of PROGETTAPS, it is recommended that the Fund and the Government negotiate an extension of the loan disbursement period, in order to allow Guatemala to continue to implement PROGETTAPS during part of 1993. Probably as much as the equivalent of US$ 1 million would be undisbursed by the present closing date. In case an extension is granted, it is strongly recommended that conditions be attached so as to ensure that priority be assigned for the full financing of operational expenses at the sub-regional level. Additionally, project resources should be used to carry out the following activities.

The project Technical Committee would carry out a review of the PROGETTAPS methodology, with the intention of analysing a number of variations that have been introduced informally as the project developed, but which have not been scrutinized in detail in order to determine how they relate to, or modify, the existing methodology on paper. The latest official documents defining this methodology date back to 1986, making it imperative that it be updated.

Since during implementation PROGETTAPS introduced a number of new activities (artisanal seed production, incorporation of RAs, inclusion of women and youth programmes), it is recommended that studies be carried out on how these functioned and what impact they have had on the overall project. In addition, due to insufficient attention paid during implementation, a new and more critical look has to be taken at the data concerning technology adoption rates, as well as goal achievements, that seem to include some duplication. A third area in which work is recommended, concerns the developing of a guide for M&E activities based on the PROGETTAPS experience, which would be useful for other projects, including on-going and future IFAD projects in the country.

Concerning IFAD on-going and future projects

PROGETTAPS and its relation to on-going or approved projects: The Fund has two other projects in Guatemala providing extension services to small farmers; the Zacapa-Chiquimula Small Farmer Agricultural Development Project (Loan 251-GM), and the Cuchumatanes Rural Development Project, under approval. Since in their design, both projects use modified versions of the PROGETTAPS methodology, it is recommended that a comparative analysis be made of all three projects, in order to identify the possible and actual effects of the proposed changes, and put these at the disposal of all three projects.

An expanded future PROGETTAPS: The Ministry of Agriculture, with technical cooperation from the Organisation of American States (OAS), is beginning the design of a possible new project based on the PROGETTAPS experience. The new project should focus on the technology generation and transfer activities that PROGETTAPS did best, avoiding the complications that would result from introducing additional components.