Interim Evaluation¹

The independent interim evaluation of the North Eastern Region Community Resource Management Project for Upland Areas (NERCRMP) was undertaken as a standard procedure in the process of preparation of a possible follow-up phase. The evaluation objectives were: to assess and document the results, impact and performance of the Project and of the partners; and to produce the basis for an agreement at completion point (ACP).  The findings, conclusions and recommendations in this report emanate from 17 days of field work, covering all six participating districts in the three states of Assam, Manipur and Maghalaya, 50 villages and intensive interaction with over 1 500 farmers, drawn from self-help groups (SHGs) and natural resource management groups (NaRMGs), with near equal representation of men and women.  The report takes into account the IFAD Strategic Framework, PI Regional Strategy and COSOPS of 1999 and 2001, and the development strategies and views of the Government of India, the North Eastern Council (NEC) and the Project Management

The Project was designed in the period 1994 to 1997. The main justification was the near total failure of previous top-down, culturally inept development initiatives; economic stagnation and resultant chronic poverty; inability of the traditional jhum system of shifting cultivation to cope with increasing population and decreasing fertility, with consequent threat to environment and biodiversity; and the general air of distrust and disillusion about Government intervention. Given this background, the primary thrust of the strategy was to introduce a development approach that was technically appropriate, culturally sensitive and institutionally effective. The aim at appraisal was to cover 460 villages in two phases of three then four years. Project area villages were to be selected to prioritise the target group criteria of: dependence on jhum; small farm acreage; rainfed cultivation; and prevalence of disadvantaged, vulnerable families. Specific emphasis was placed on empowerment of women; water catchment integrity; and pro-poor, differential benefit entitlement, to augment traditional tribal poverty alleviation customs and practices.

The goal of the Project was to improve the livelihood of vulnerable groups in a sustainable manner through improved management of their natural resource base.  Further specific objectives highlighted: local peoples, and women's participation; community organisation; environmental know-how; savings and thrift; and access to basic services and infrastructure. The original components were: Capacity Building of Communities and Participating Agencies; Economic Livelihood Activities (farm-based and non-farm); Community-based Biodiversity Conservation; Social Sector Activities; Village Roads and Rural Electrification; and Project Management. Following Mid-term Review in June 2002, the components were structured to comprise: Institutional Support (~7% of funding); Village Development Fund (VDF), including Livelihoods (~52%), Social Sector (~5%), and Village Infrastructure (~20%); Natural Resource Management (~4%); and Project Management (~11%).    

The implementing partners are Government of India, under the aegis of the North East Council (NEC), embracing the three State Governments and the Regional and District Societies, formed by the Project to address the inadequacy of existing development management capacity. These societies regroup representatives of NEC, relevant central and state line departments, parastatal agencies, District Administrations, participating communities, NGO partners and civil society; at the village level, the SHGs and NaRMGs, NGO facilitators and supervisors, district and block line departments and private sector service providers. Finally, the executive management rested with the Regional Programme Support Unit (RPSU) and six District Support Teams (DSTs), each with a District Project Manager and with a core cadre of six to eight officers.

The most notable policy and institutional change made by the Project was the successful establishment of village and community groups and their NGO service providers – all the more so that evidence suggests that emerging that future district development policy is likely to be aligned with these principles. The major design change took place at MTR: an increase of the target coverage from 460 to 1 000 villages; and introduction of the Village Development Fund.

As for the community development and institutional support, probably the most exemplary and significant achievement was the formation of SHGs, predominantly of women members: there are now 2 071 SHGs - as against an original target of 920, with 33 056 women members, and 996 NaRMGs. Targeting of the Project and of uptake of individual activities has been reasonably effective, although benefits of some activities are inevitably enjoyed by all sectors of society. The means and application of targeting have proved justified and practical; but there is a concern that outreach to the poorest of the poor is in many cases limited. The gender balance of Project activities has been sound; and Project management, after a chequered start, has progressed significantly, and become highly proficient.  

The main implementation results under the agro-biodiversity and on-farm livelihood were: the reduction by nearly half of the area under jhum; development or improvement of terraced and irrigated lands; the strong uptake of diversified cropping focusing on horticultural and cash crops, as well as longer term plantations; integration of livestock into cropping and homestead production systems - for over 20 000 households; and community and watershed protection forestry. On biodiversity awareness achievement is disappointing.

In non-farm livelihoods, more than 1 600 enterprises, ranging from small shops to weaving factories have been set up, and commercialisation of small farming and cottage businesses is under way, although further work is needed to consolidate marketing, financing and business management aspects. Already over USD 235 000 is being borrowed from formal sources, with so far sound repayment history. This follows the success of SHG credit operations, whereby over 30 000 women members have savings totalling USD 238 000 and a track record of own funds lending, with 100% repayment.

In the social sector, construction and rehabilitation of water supply schemes has exceeded target by 50%; nearly 600 villages now have low cost latrine facilities; and the community health and education interventions, while diverging from those planned, have had reasonable impact.

The village infrastructure activities have exceeded the road building target by a factor of 9 and constructed 60 bridges or culverts and 25 roadside shelters, making a signal difference to access and transport facilitation. Some 82 community halls have also been built. In electric power supply, 80 out of the targeted 115 villages have been connected to the grid; those outstanding are now waiting for an imminent Government rural electrification programme that will greatly widen the coverage. Only one of the proposed 20 micro-hydro-electricity investments has as yet materialised.

In terms of Project performance, given the circumstances of the region and its peoples, the precarious nature of livelihoods and the prevalent state of serious under-development and progressive resource degradation, the relevance of objectives of the goal of the Project was - and remains - very high, a score of 6 on the scale of 1 to 6. The relevance of the other component objectives were also judged to be either high - 5, or very high - 6, with the exception of the non-farm livelihoods, which is marked only moderately high - 4. 

The effectiveness and efficiency of the Project can best be judged from its physical and financial progress. For the majority of activities, the Project has exceeded physical targets, sometimes by exceptional margins; principal shortcomings are in modification of jhum cultivation practices, health worker training and electricity supply, for which there are reasonable explanations.

In financial terms, current expenditure is only around 35% of available funds, with deficiencies for Village Infrastructure, non-farm Livelihoods and Natural Resource Management.  Expenditures have been well controlled overall; and reasonably balanced between districts, with an average of USD 1.55m per district, to a total of USD 9.29m.

The rural poverty impact is a function of the numbers of households and people actively involved and the extent of improvement in their livelihood and well-being. In the absence of explicit appraisal targets for many of the activities and of systematic M&E records of impact, best estimates of overall coverage, direct benefit and changes in wealth are summarised below:  

The Project has covered 862 villages comprising 39 203 households (HHs)  and 223 450 people, compared to the respective numbers of 460 villages, 23 000 HHs and 131 000 people planned at appraisal; and 1 000, 5 000 and 285 000 targeted (or implied) at MTR;

The numbers of direct beneficiaries by component or sub-component, based on data from five districts are: for the Institutional Support component – 33 165 HHs and 172 68 people; for livelihoods – 29 388 HHs and 152 041 people; in the Social Sector -  21 081 HHs and 111 119 people; for Village Infrastructure – 16 460 HHs and 82 887; and for Natural Resource Management – 22 073 HHs and 140 478 people.  With reference to these numbers and the evidence of field work assessments, the Evaluation Mission concludes that at least 25 000 households have benefited to date – and received substantive advantage from one or more Project activities; and

The wealth ranking aggregate figures are rough indicators of movement in poverty incidence, show that of a total of 18 390 households assessed between 1999 and 2004, the numbers of "poorest" have fallen from 9 742 to 6 455 and those "better-off" have increased from 172 to 625; thus, 18% of the relevant Project households has moved out of the poorest category; and 2% of households have moved up into the better-off category.  Village responses confirmed this fact, reporting substantive improvements in family economic and welfare predicament.

The main and most direct sources of poverty impact have been the production and income-oriented activities of on-farm livelihoods; crop diversification, irrigation and terracing; and to a lesser degree, non-farm livelihoods and income generation.  The key evidence of impact is that of assured food security and higher cash income for the great majority of participating households.  The benefits are manifest mainly in ownership of physical assets, resilience of family financial predicament and improved nutrition and health status, in particular for children and women.  Poverty impact by domain is scored as shown in the listing below:

Poverty Impact Areas			Ranking
Physical and Financial Assets			5 – high
Human Assets			6 – very high
Social Capital and Empowerment			6 – very high
Food Security			5 – high
Environment and Communal Resource Bask			2 – low
Institutions, Policies and Regulatory Framework			2 – low
Gender			6 – very high
Sustainability			3 – moderately low
Innovation and Scope for Replication			5 – high
Overall Impact			5 – high

The Project has exceeded, met or come close to meeting nearly all of the targets set at appraisal or modified at mid-term.  It has had a commendable coverage of smaller and poorer farm households and a significant impact on a representative cross section of the district population, of whom most subsisted below the poverty line pre-Project, and now many fewer do; and among whom the majority have been women. The only weaknesses in impact have been those related to environment, policies and sustainability.

The performance of the partners in implementation has varied. The performance of IFAD is seen as having been generally good, but the Fund could have been more supportive in rectifying some of the earlier deficiencies and obstacles to progress, IFAD is ranked as moderately high - The performance of UNOPS was not satisfactory in the first four years of the Project due to staffing and personal relations problems, but has since improved, although better consultation with the Project as to the expertise needed for specific supervision missions is required; a score of moderately low is given -

The performance of Government and line agencies has been constrained by an overly cumbersome structure and procedures and a certain disinterest or lack of collaboration among the local departments; a low performance - 2. Project Management has performed well, after a chequered start and is ranked as high - 5; and NGOs and CBOs have made a distinguished contribution to Project progress and impact and are given a score of very high - 6.

The overall assessment of Project performance is that NERCRMP has been a notably successful development intervention, despite the very difficult environment in which it was implemented and the hugely ambitious nature of its geographical and subject matter coverage. In particular, in the empowerment of beneficiaries and in the promotion and progressing of commercialisation of the small farm subsistence sector through the encouragement of savings, thrift and credit, farm enterprise diversification and private sector linkage, the Project has achieved exemplary results. On the principal IFAD measure of performance, rural poverty impact, it is clear that there has been a marked positive change in the predicament of both direct and indirect beneficiaries.

The most important insight that emerged from the present situation is that a huge burden of further work is still to completed, specifically related to bringing the 540 newer villages enlisted since 2003 up to the standard of the more mature villages. This leads to the recommendations:

• that the Project restrict its activities to the current complement of 862 villages; and
• that operating strategies and plans be made to ensure that all villages and groups reach optimal self-reliance and proficiency, including through the assistance of additional staff and/or external service providers.

Despite the success of the Project, there is an urgent need to consolidate the empowerment of women and groups by assisting them to formalise their rights to access to land; and to address the predicament of the poorest echelon of the tribal societies, who have not yet become fully involved. Accordingly, the recommendations are made:

• • that a study on the topic of land tenure be undertaken urgently and an action plan be produced whereby the Project will assist with advice on tenure registration and possibly with financial help in the form of a loan or matching grant aid for the completion of the necessary legal formalities; and
  • that a separate strategy be devised as a matter of urgency, for application across the Project districts and particularly among new villages to ensure the inclusion of the poorest groups.

In response to the need expressed by some village and group respondents for more resources for water supply provision, the recommendation is:

• • that the guidelines of expenditure per household and expenditure per component and sub-component be interpreted in a more flexible and pragmatic manner so as not to disqualify crucial infrastructure investments for smaller and poorer communities.

Key constraints to the effectiveness of Project Management in taking forward its demanding workload are its limited jurisdiction over district operations and the somewhat erratic and unreliable flow of funds. In this regard, the main recommendations are: 

• • that the RPSU is given more authority through the aegis of the DC to influence and control district activities in the latter years of the Project; and

• • that a new understanding and binding arrangements be thrashed out and agreed by all parties to remedy the deficiencies and delays in funds flow.

Notwithstanding the importance of the recommendations presented above, there is a series of critical issues facing Project Management and stakeholders, namely:

• • consolidating the achievements and gains that have been made in all facets of Project activity;

• • optimising the sustainability of gains, achievements and supporting development operations; and

• • facilitating dissemination and replication of the Project approach, modalities and interventions in other villages, blocks and districts.

To address these issues, the recommendations are:

• • that the Project make greater efforts in most districts to involve the block and district line departments in the development and empowerment of villages and groups. This could be done if necessary by invoking the more active support of DCs to familiarise the cadres with the Project philosophy and operations and provide on-job training and experience;

• • that a specific programme of liaison and knowledge sharing with block and district line agency staff be devised and rigorously pursued with a view to devolution of  Project tasks and responsibilities as the Project winds down; 

• • that a similar programme be mounted of active engagement with - and technical and financial assistance to - the NGO sector, directed at district level and possibly also regional level considerations, to also prepare them for potential handover of group fostering and service provision; and

• • that District Societies and the Regional Society be given greater responsibility for the planning, the raising of financial resources from Government, donor and NGO sources, facilitation and supervision of development along Project lines, by the allocation of Project funding and the deployment of some of the Project senior cadre, for up to two years after Project closure.  

Further and finally, taking account of the immense task of bringing the large number of new villages to an acceptable level of self-reliance - two elements are of prime importance in seeing through a satisfactory outcome to NERCRMP, not least in the 540 new villages. On this subject, the crucial recommendations are:

• • that as far as practicable, the current Project Management senior cadre be committed and contracted to continue in service up to Project closure, but not necessarily in the same positions; and

• • that the Project duration be extended for two additional years, as a minimum, to complete the operations and additional tasks now envisaged, that is up to 31 March 2008 for completion and 30 September 2008 for loan closure.

 

---

 1/ The Interim Evaluation Mission comprised: Jim Semple, Mission Leader and Rural Development Specialist; Somesh Kumar, Community and Institutional Development Specialist; Wayne Borden, Agro-biodiversity Specialist; Sarath Mananwatte, Economist: Livelihoods, Marketing and Credit; and Mini Bhattacharyya, Sociologist. The Mission was supplemented and its work greatly facilitated by inputs from the national counterpart team, comprising selected members of the Regional Programme Support Unit and the various District cadres. Flemming Nichols, the IFAD Lead Evaluator, joined the Mission for the final week of field work and for the wrap-up meetings.