Practical and cost-effective indicators and procedures for
monitoring the impacts of rural development projects
on land quality and sustainable land management
Kurt Steiner
a,∗, Karl Herweg
b, Julian Dumanski
c,1aGTZ, Postfach 5180, 65726 Eschborn, Germany
bCentre for Development and Environment University of Bern, Bern, Switzerland c16 Burbank Street, Ottawa, Canada K2G OH4
Abstract
The impact(s) of development activities on land quality often become manifest only several years after termination of a project. Baseline studies and cost-effective monitoring needs to be initiated while activities are on-going, and this needs to be continued in post-project activities. This is possible, however, only when relatively simple and cost-efficient monitoring methods are available. These tools, however, are still rare and need to be further developed.
An international working group, initiated by GTZ and SDC (Swiss Development Co-operation) and strongly supported by the ‘Land Quality Indicator Initiative’ of the World Bank, developed guidelines for impact monitoring to address this constraint. These included a toolkit of relatively easy to handle and cost-effective methods. The impact monitoring procedure presented involves seven basic steps: identification of stakeholders, identification of core issues, formulation of impact hypotheses, identification and selection of indicators, selection and development of monitoring methods, data analysis and assessment of sustainable land management, as well as information management. © 2000 Elsevier Science B.V. All rights reserved.
Keywords:Sustainable land management; Impact monitoring; Practical indicators; Cost-effective monitoring methods
1. Introduction
Agricultural cropland, forested lands, range lands, and pasture lands are under increasing pressure be-cause of agricultural intensification on existing cul-tivated lands, and population migrations to marginal land areas. Increases in food supplies must come from agricultural intensification rather than expansion, but this must be done without degrading the land resource
∗Corresponding author. Tel.:+49-6196-79-1614; fax:+49-6196-79-6554.
E-mail address:kurt.steiner@gtz.de (K. Steiner).
1Former address: The World Bank, 1818 H Street, Washington,
DC 20433, USA.
on which production depends. The problem is, how-ever, in identifying which interventions give the best short-term benefits but are also sustainable over the longer term, and how to design monitoring systems to see if we are on the right track.
The objective of sustainable land management (SLM) is to harmonise the complementary goals of providing environmental, economic, and social op-portunities for the benefit of present and future gen-erations, while maintaining and enhancing the quality of the land resource (soil, water, and air) (Smyth and Dumanski, 1993). The promotion of sustainable land management is amongst the important set of objec-tives of most agricultural and resource management projects. However, without operational and
ate indicators and tools for monitoring and evaluation, sustainable land management is not more than an attractive but empty phrase.
Government officials, programme or project man-agers, and also land user groups are in need of informa-tion concerning the state of natural resources and land quality, and whether or not their past activities have led towards or away from sustainable land management. The problem that these decision-makers are facing is not the lack of indicators, but the difficulty to select an appropriate set of indicators reflecting changes in land quality without complicated, expensive and time consuming monitoring and evaluation procedures.
An international working group has addressed this problem and elaborated guidelines for participatory impact monitoring amended by a collection of appro-priate methods for indicator selection and monitoring. These guidelines have been field tested for appropri-ateness and ease of use in some development projects.
2. Principles and criteria for a sustainable land management monitoring program
Implementing procedures for impact monitoring in sustainable land management (SLM-IM) requires un-derstanding of some practical principles.
2.1. The impact of a single land management activity on land quality usually cannot be monitored
Monitoring changes in land management is a pro-cess of understanding the impacts of human inter-ventions on the landscape. Decision-makers, farmers, policy-makers, and project managers review their de-cisions and activities from time to time, and make adjustments for changing situations. Often, it is less important to isolate the impact of a single activity or project, than to estimate the overall direction and the extent of change and which factors are involved.
2.2. Project staff often have minimal time for additional tasks, such as impact monitoring
Project staff are confronted with an increasing num-ber of regulations, manuals and guidelines, and of-ten there is no time, nor incentives, to introduce and
manage new procedures. Therefore, unless SLM-IM is incorporated as part of the project objectives, it is unlikely that it will take place. Optimal entry points for SLM-IM are during project orientation and plan-ning, or during a mid-course/reorientation phase, as explained in Table 1.
2.3. Sustainability is a highly complex issue, and it cannot be assessed with certainty
There are universal principles and criteria for as-sessing sustainability, but activities and indicators to monitor progress towards this goal must be local. SLM-IM approaches and methods must always be adapted to local circumstances where they are ap-plied. The foremost requirement is a flexible SLM-IM procedure, as outlined in Table 1, adaptable to indi-vidual situations, but following scientifically sound protocols. Impact monitoring must be a participatory process involving all stakeholder groups during all steps of SLM-IM. The best options are incremental improvements in land management, starting with the most promising local practices and integrating exter-nal knowledge of project staff to gradually improve land management. The focus must always be on practical and cost-effective monitoring.
3. Developing cost-effective indicators and monitoring procedures
The ‘SLM — Guidelines for Impact Monitoring’ (Herweg et al., 1998), including a toolkit describ-ing selected cost-effective methods for monitordescrib-ing, was published as an aid to programme co-ordinators, project managers, task managers and others on imple-menting monitoring activities in different stages of a project cycle (Table 1). The guidelines describe the basis of SLM-IM as a seven-step procedure.
3.1. Step 1: Identification of stakeholders
Table 1
Fitting the steps of SLM-IM into a project cycle
Phases of project implementation and guiding questions Corresponding SLM-IM steps and tools for
Phase1:Project identification/orientation
What are the main land problems that can be addressed? Identification of stakeholders Identification of core issues
Phase2:Project planning
What activities are needed to address the main land problems? Identification of stakeholders Identification of core issues
Phase3:Environmental impact assessment
What impacts are the activities likely to have on the environment? Formulation of impact hypotheses Selection of indicator sets
Phase4:Performance monitoring
Is the project on target and on schedule? Selection of indicator sets
Selection/development of monitoring methods
Phase5:Mid-course evaluation
Were the established targets met and is there a need for corrective action? Selection/development of monitoring methods Data analysis and assessment of SLM Information management
Phase6:Continued planning/reorientation
What activities are needed to proceed towards SLM? Identification of stakeholders Identification of core issues
country desk officers, and policy-makers. Stakehold-ers must be actively involved and they must assume responsibility for SLM-IM from the beginning, to better ensure that land management monitoring will continue beyond the life-time of a project.
3.2. Step 2: Identification of core issues
Key questions: What is essential to make land man-agement more sustainable? What is most important to monitor?
Limited time and budgets in a project make it diffi-cult to address the complexity of SLM, and to monitor
Table 2
Examples of core issues of sustainable land management
Level Core issues
Natural resources Soil: nutrient depletion, soil erosion, salinisation, toxicity, etc.
Water: depletion of ground water table, declining water quality, flash floods, etc. Plants: reduced biodiversity, reduced biomass and nutritive value, etc.
Animals: malnutrition, diseases, overstocking, etc.
Land management Cultivation of marginal lands, overgrazing, deforestation, etc.
Society Impoverishment, migration, conflict over natural resources, imbalanced land reforms, instability of input prices, rapid population growth, etc.
all facets. Therefore, the most important land manage-ment issues, the so-called ‘core issues’ of SLM-IM, must be identified and monitored. Some core issues of SLM are identified in Table 2.
3.3. Step 3: Formulation of impact hypotheses
Key questions: Which impacts of project activities are desirable and expected? Can impacts other than the desired ones be expected?
Table 3
Examples of impact hypotheses for a hypothetical project involving terracing steep slopes to reduce soil erosiona
Impacts Different levels of impact hypotheses from the point of view of the
Ministry of agriculture
Local farmers
Project Local
merchants Desired impacts Soil loss reduced Incentives and
subsidies received
Technology adopted by farmers inside and outside the project area
Demand for tools and inputs increased
Soil fertility maintained
Crop yield increased Supply of agricultural
products increased Production improved
Undesired impacts – Labour demand for soil conservation increased
Incentives become more important than conservation
–
Unexpected impacts No adoption of the technology
Problems turning the ox-plough No adoption of the technology
–
Rodents settling in and waterlogging behind the SWC structures
aSDC (1997).
impacts. The variety of possible impacts should be es-timated beforehand by formulating impact hypotheses. Examples of impact hypothesis are shown in Table 3.
3.4. Step 4: Identification and selection of indicators
Key questions: How can suitable indicators be iden-tified? How can we move from measurement to as-sessment?
Indicators used for monitoring must be practical, measurable, manageable, and relevant (Pieri et al., 1996). The pressure–state–response framework
Fig. 1. The pressure–state–response framework (Pieri et al., 1996).
(Fig. 1) is recommended as a structural model for this purpose, since it assists in focusing on those is-sues that need to be addressed, and helps to identify a meaningful set of indicators, reflecting ecological, economic and social factors of sustainability, related to the issues. Examples are provided in Table 4.
Table 4
Indicator selection following the pressure–state–response frameworka,b
Pressure indicators State indicators Response indicators
Changes in cropping techniques Rate of deforestation Change of legislation Financial position of holdings Rate of soil erosion Investments Fuelwood/charcoal consumption Degree of salinisation Tree planting
Use of crop residues Soil crusting and compaction State conservation programmes Use of animal dung for fuel Crop productivity Farmer conservation groups Price of fuelwood/charcoal Livestock productivity Farmer adoption of tree planting
and soil and water conservation Nutrient balance (on-farm organic matter recycling)
aSahara and Sahel Observatory (1997).
bDriving forces causing pressure on natural resources are: population pressure, economic growth, urbanisation; policy failures/distortions
(stagnant technology, delayed intensification); imperfect markets (lack of markets, poor market access); transaction costs and imperfect information (limited access to information about market opportunities); social inequity, poverty; political and social instability.
3.5. Step 5: Selection and development of monitoring methods
Key questions: How can changes in land manage-ment be observed and measured? How can SLM-IM methods be developed?
The selection of appropriate methods depends on the objectives of SLM-IM. Useful criteria when se-lecting methods are requirements for data accuracy, investments (budgets, time of monitoring staff), level at which the data are required (plot, field, farm, dis-trict, national), need for large area coverage or sta-tistical sampling, frequency of observation, feedback and presentation of results. Often a complementary mix of simple and more sophisticated techniques is necessary.
3.6. Step 6: Data analysis and assessment of SLM
Key questions: Has the land management interven-tion (brought about by the project) increased the
sus-Table 5
Internationally identified indicators of sustainable land managementa
The five pillars of sustainability
Productivity Security Protection Viability Acceptability
Crop yield Soil cover Soil quality/quantity Net farm profitability Use of conservation practices Yield variability Water quality/quantity Input use efficiency (pesticides,
fertilisers, nutrients)
Farm decision-making criteria
Climate Biological diversity Off-farm income Return to labour
aDumanski (1994).
tainability of the land use system, compared to what was there before? Which processes need to be consid-ered in analysing the data?
The application of the SLM approach is based on using participatory approaches in field data collec-tion. This implies making farmers true partners in the research team, and involving them in all stages of sustainability evaluation, including identifying the criteria and indicators to be used in the evaluation and in the final rating on sustainability.
farmer in the final judgement. These procedures are recommended since it is usually the farmer who best knows his/her local situation, and is best able to judge whether or not certain management interventions are cost effective.
3.6.1. Examples of sustainability classifications
3.6.1.1. Aggregate indicator ratings. The perfor-mance of each indicator for each pillar is assessed (on a scale of 0–10) in view of its limitations to sustain-able land management. Numbers are assigned from the lowest to highest, then accumulated using additive or multiplicative procedures. Results are expressed in a sustainability classification.
3.6.1.2. Conditional sustainability. It is common that the sustainability analyses will show that some but not all requirements for the sustainability pillars are met. For example:
• In sloping areas in country ‘X’, a new land manage-ment technique may be better able to control soil erosion, while maintaining crop production levels, improving water quality, and so on, but profitability is marginal because of unexpected costs in the start up phase. In these cases, the systems are judged as conditionally sustainable, pending resolution of the residual problem of start up.
• In some situations, the analyses may show that the system is sustainable, but this is dependent on a par-ticular market niche. In this case, the system would also be conditionally sustainable, because mainte-nance of the special market is likely to be beyond the control of the farmer.
• In other cases, the analyses may indicate that sus-tainability may currently be adequate, but future sustainability is uncertain. In these instances, a tem-poral conditionality can be used.
3.6.1.3. Index of sustainability using sustainability polygons. This assessment is constructed by first ar-ranging the indicators around a central focus like the spokes of a wheel. The performance of each indicator is then rated (as described earlier), and this value is positioned on the respective spoke. All values are then connected to form a sustainability polygon (Fig. 2), and the degree of sustainability, as well as the
limita-Fig. 2. Index of sustainability using sustainability polygons.
tions thereof, are estimated from the asymmetry of the polygon.2
2The sustainability index can be used to compare two or more
Table 6
Examples of cost-effective monitoring proceduresa
Transectoral monitoring
Participatory methods Participatory rural appraisal — observation, transects and mapping Structured questionnaire
Local (indigenous) knowledge Local plant and soil classifications Biophysical stocks and flows Changes in resources, nutrients, etc.
Integrated transect method Detect changes in land use, management and degradation
Photo monitoring Pre- and post-project
Sectoral monitoring
Soil fertility Identifying deficiencies
Extended spade analyses
Estimating soil texture, consistency and aggregate stability
Erosion Field estimation
Sediment traps Erosion nails
Estimating ground cover
Yield assessment Pre- and post-harvest
Information technologies Expert systems integrating local and scientific knowledge
Economic estimates Cost-benefit analyses
aHerweg et al. (1998).
3.7. Step 7: Information management
Key question: How can information be presented and disseminated in a user-friendly manner, and stored for continual easy access?
Projects cannot afford to collect and accumulate data which are rarely analysed and used. Even data of excellent quality are irrelevant if there is neither the interest nor the capacity to use the information. Once collected, data should be stored by responsible insti-tutions to provide easy access for potential users.
4. Procedures for cost-effective monitoring
Effective monitoring requires not only indicators with which to monitor, but also cost-effective, field oriented procedures. A selection of such methods is described by Herweg et al. (1998), and summarised in Table 6. It is important to note that ‘cost-effective’ should not be equated with ‘cheap’; cheap monitoring may result in low-quality information, and it can be a waste of resources and a source of errors.
possibly learn. The same method used for several months or years will indicate whether or not corrective action fulfilled its purpose and led to more sustainable land management.
The choice of monitoring procedure ultimately is a trade-off among the objectives of the project, the budgets available, and the requirement for data accu-racy. For most land management projects, much can be achieved with qualitative approaches at much lower cost, particularly for exploratory investigations of a problem. Quantitative monitoring can then be focused on ‘hot spots’ originally identified using exploratory techniques. Herweg et al. (1998) estimate that quanti-tative procedures often exhibit error rates of 5–10%, compared to error rates of 30% or higher for qualita-tive techniques, but at much lower cost. World Bank projects often set targets for impact monitoring activ-ities at about 3–5% of project budgets.
5. Conclusions
Quantitative approaches are the most accurate methods for SLM-IM, but they are not always cost-effective. Qualitative approaches are sometimes more practical and cost-effective, and they can provide much useful information. However, there are limita-tions and these must be appreciated if these methods are to be used. If a project is in need of highly ac-curate results, then the cost of complex, quantitative monitoring must be included in the project budget.
References
Dumanski, J., 1994. Proceedings of the International Workshop on Sustainable Land Management for the 21st Century: Workshop Summary. The Organising Committee, Agricultural Institute of Canada, Ottawa.
Gameda, S., Dumanski, J., 1995. Framework for evaluation of sustainable land management: a case study of two rain-fed farming systems in the Black Chernozemic soil of southern Alberta, Canada. Can. J. Soil Sci. 75, 429–437.
Gomez, A.A., Swete Kelly, D.E., Syers, J.K., Coughlan, K.J., 1996. Measuring sustainability of agricultural systems at the farm level. In: Methods for Assessing Soil Quality. SSSA Special Publication no. 49, Soil Science Society of America, Madison, USA, pp. 401–409.
Herweg, K., Steiner, K., Slaats, J., 1998. Sustainable land management — guidelines for impact monitoring. Workbook and Toolkit, Working Documents for Public Discussion, Bern, pages 79 and 128.
Pieri, C., Dumanski, J., Hamblin, A., Young, A., 1996. Land Quality Indicators. World Bank Discussion Paper no. 315, World Bank, Washington, DC, USA.
Sahara and Sahel Observatory, 1997. Impact indicators and monitoring-evaluation for action programmes to combat desertification, 27 pp.
SDC (Swiss Development Co-operation), 1997. Monitoring — keeping in touch with reality. In: Series of Working Instruments for Planning, Evaluation, Monitoring, Transference into Action, Vol. 2. Strategic Controlling Unit, Beme, Switzerland, 77 pp.
