CHAPTER SIX: PRESENTATION OF THE CASE STUDY OF THE GADP
6.3 THE PERFORMANCE INDICATORS AS STRUCTURED BY THE GADP
Table 6-1: Performance indicators as structured by the GADP
Objectives Activities Achievement Sources of data Observation/ remarks
General objectives Farming
production increased
Increase farming production -Arable land farmed increased by 5%
-Livestock needs covered:
pigs (6.2%), rabbits (36%), and sheep (14.3%)
- Report of crops forecast
- Market price
- Survey report on farming Production
-Availability of financial resources of the project
-Release of funds by sponsors
-Government fund availed in time Specific objectives
1.Arable land increased
Increase arable land: marsh development
Rehabilitation of 230 ha up to 15 October 1998
-Report of Rural engineering activities
-Report of Evaluation and follow-up Department
Availability of technical documents before the end of July 1998
1.1 Progressive diminution of non-arable land in rural area
1. Guarantee the maintenance of vegetal material against erosion 2. Rehabilitate the main Drains
-¼ ha of vegetal material per district rehabilitated -one survey on farming production systems achieved
-Activities report -Follow-up report -Survey report
No information was provided for the project progress.
1.2 Marsh development
1.Canalize water during dry season in the marshes of Mwogo-Rwameru 2. Rehabilitate main drains in Mwogo-Rwamweru
-One temporary dam installed
by 30 October 1998 on Nkungu
-One channel of water (1.5km)
to the marshes Mwogo- Rwamweru dug by 30 October 1998
-Report of rural engineering -Report of final reception
No information was provided for the project progress
2. Agricultural inputs available
1.Provide farmers with inputs through the NGOs UNICOOPAGI and RITA 2. Get in time inputs as forecasted (3 months before agricultural season)
3. Provide in time inputs to the groupings who sold inputs (1 month before agricultural season)
Inputs available 16 131 500 Fr (local currency)
Orders established and submitted in time to suppliers
Schedule of transporting inputs to the stores respected
Distributions report Report on monthly activities Reception documents
-Expedition (delivery) documents
-Documents for entry of inputs in stores
No information was provided for project progress.
4. Increase seeds improved for distribution
Distribution of inputs to the contractors for increasing seeds of potatoes, beans, soya beans, sweet potatoes by 15 September 1998.
-Distribution report -Contracts for increasing seeds signed
-Report of follow-up
No information was provided for the project progress.
5. Establish credit system for inputs
One contract signed between the project and beneficiaries of credit (groupings, association of groupings).
Agreements signed No information was provided for the project progress.
6.Rehabilitate veterinary Dispensaries
10 dispensaries rehabilitated and equipped in 9 districts by the end of 1998
-Activities report -Follow-up report
No information was provided for the project progress
Objectives Activities Achievement of objectives Sources of data Observation/
remarks Specific objectives
3. Transport equipment and infrastructures
1. Transport of fertilizers and seeds to the storerooms of cooperatives
One vehicle (DAIHATSU) available to the project for transporting inputs
Financial report
No information was provided for the project progress.
2. Rehabilitate roads connecting project and hangars of stores
4 roads rehabilitated respectively of 12km, 3km, 1km, and 500m.
-Research and Development report
-Follow-up report
No information was provided.
4. Increased Projects generating jobs
1. Source contractors for improving the state of roads
2. Assure maintenance of forestry paths
3. Reforestation
4. Creation of forestry paths 5. Security guard
6. Put in place an autonomous saving and credit system (fund) for farmers
One contract signed for rehabilitating roads of 50km -55 000 forestry plants for the Province were distributed -26 000 fruit plants provided Forestry paths of 50km rehabilitated by December 1998 100 ha reconstituted
4 forestry paths created 60 security guards allocated in different districts
Workshop held between
beneficiaries and representatives of Financial Institutions, sponsors, MINAGRI, Ministry of Commerce
Rural engineering report
Follow-up report
Follow-up report Follow-up report -Personnel documents -Evaluation and follow-up Report
-Training report -Evaluation and follow-up report
No information was provided for the project progress.
5.Training 1. Train farmers about the right way of using land (crops rotation, fertilizer, combating diseases)
2. Training for farmers about grouping management
3. Train farmers about excavation work and its management
4. Field visits
- A syllabus on training in agricultural techniques:
70 copies produced and distributed
- A 2- day session about the usage of the syllabus held for 4 agricultural technicians at district level - A 3-day training session for 5 farmers per sector (local administration unit) Held
A 3-day session held and 3 members of each grouping from 3 groupings per district trained on organization and management of groupings
6 agronomists from districts of high altitude trained for 6 days in Gikongoro Province
14 farmers visited ARDI (NGO) for 30 days about the rearing of bees
-Syllabus of training -Training report -Follow-up report
-Training report - Follow-up report
-Visit report -Follow-up report
-Visit report -Follow-up report
No information was provided for the project progress.
Source: GADP (2001).
From Table 6-1, one can see that GADP performance indicators are not well structured and do not appear clearly in the table. They are vaguely presented. This is an understandable result of the fact that, as is obvious from GADP reports, some objectives, activities, components, and impacts were not clearly stated, nor was the relationship between them clearly defined.
Therefore the project’s achievements could not be objectively measured. Objective N° 4 in Table 6-1 illustrates the problem. In fact, an increase in the projects which would generate jobs has no direct link with the activities related to improving the state of roads, maintenance of forestry paths, reforestation, security guards, and credit system. It will be hard to measure the extent to which the objective has been achieved.
The column of the objective achievement in Table 6-1 indicates actual indicators (degree of objective achievement), but no planned indicators are mentioned elsewhere in the table. Hence, it is very difficult to come to a conclusion on project achievements. Performance is appreciated by comparing actual and planned achievement. The deviation between planned and actual achievement indicates whether the project is behind or ahead of schedule, so that corrective action may be taken while there is still time. That this aspect of planning was not properly taken care of could be the cause of delays, budget overruns, poor coordination of activities, poor relationships among the project stakeholders and conflicts, waste of resources, low productivity, and could finally have contributed to the overall failure of the GADP.
An example of the confused state of affairs is the objective of developing lowest level valley land of 450 ha and land of high altitude of 378 ha in collaboration with small cooperatives. The table of indicators indicates that 230 ha of marsh land had been rehabilitated up to the 15th October 1998. The objective, however, stipulates the development, not the rehabilitation, of the 450 ha land. A comparison between what has been planned and what was achieved becomes practically impossible because the terms of comparison are different. The initial objective is not what is measured. Another example is the objective of “construction or rehabilitation of 85.9 km of roads at district level using a high intensity workforce”. The table of indicators indicates that 4 roads respectively of 12km, 3km, 1km, and 500m were rehabilitated. In total, the rehabilitated roads covered a distance of 16.5 km. The way the objective was set is confusing. To make it possible to measure the project objectives and compare planned and achieved objectives, there should have been two separate objectives, the first: “construction of roads”, and the second:
“rehabilitation of roads”. To measure the achievement of the stated objective is not possible
because the distance of roads to be rehabilitated and the distance of roads to be constructed is unknown, the comparison between the roads rehabilitated and roads to be rehabilitated is impossible to make, and the performance of the road component is difficult to judge.
Moreover, the activity of training for farmers was not mentioned in the project objectives, but appears in the table of performance indicators. Whether the activity was achieved is impossible to say because it was not included in the planning. And even if it has been carried out, it could have unbalanced the financial forecasts as it was not included in the financial planning. There is evidence that the activity was not well planned and not achieved. According to the GADP (2001), trainees included farmers and employees from different levels of government bodies (central, provincial, district and levels). Training involved such subjects as computer skills, farm work, planning, management of stores, cooperatives management, intensive campaigns and popularization, survey, data analysis and research and development.
It would have been difficult to train people from different backgrounds, using the same training programs, and allocating the same time which, anyway, appears to have been too short to cover all the training needs. The records of the GADP (2001) indicate that the methods used for training involved sessions, meetings, study tours and practical training in foreign countries.
Training sessions and workshops could take 3 to 5 days. Considering time limitations, the training could not have been very productive to make positive impacts through the acquisition of sound, new knowledge and skills.
No doubt, the war and genocide were among the major causes for the non-achievement of training objectives, because some trainees and trainers were killed and others exiled to foreign countries. Besides, training funds had diminished because of the withdrawal of the main funders, as a result of what Rwanda was going through. But it can’t be denied, that the GADP planning was poor. Practical training in foreign countries in the form of courses or internship was positive, but is normally very expensive and became impossible when major sponsors were systematically withdrawing. It had been planned that training would be provided for 5571 people, but in the end only 3051 were trained, which means that the achievement rate was only 55% (Table 7-5). Furthermore, the content of the training programs was ambitious. The activities were too many to be achieved and the time was too short to cover the specific training needs required by beneficiaries and the organizations they worked for. As a result, the GADP failed to
contribute to an increase of farmers’ knowledge and skills, which were needed for the management of their businesses and cooperatives.
Except in the case of general objectives, and a few specific objectives where comments are made, column 5 of Table 6-1 does not provide any information about stakeholders involved in specific activities, resources needed, and reports or documents required. This shortcoming could have resulted in delays in the executing of tasks and the delivering of materials, and it would have complicated the assessment of the achievement of the related objectives and the coordination of project activities. In addition to this, it is not correct to list, for a given objective, the same activity and objective (Table 6.1). Poor planning is indicated once again, and in this case too, an inability to objectively estimate resources and schedule project activities, along with inefficient use of resources, could have resulted in delays and conflicts among GADP stakeholders, all of this due to the lack of clear roles and responsibilities or lack of resources.
The available GADP documentation, which contains only a few documents, does not provide any further figures in relation to planned and achieved objectives. This makes it difficult to compare the two. The problem of documentation would be attributable to the mismanagement of archives for which the GADP had no specific place reserved. The genocide worsened the situation, because some documents were burnt during that period or dispersed outside the GADP offices and, as it was in the rainy season, possibly destroyed by weather conditions.
The poor structuring of performance indicators affected the GADP implementation (management and evaluation). It was an indicator of poor strategic planning because the organizational structure was inadequate and objectives and activities were not clearly defined.
Table 6-1 shows that some activities were carried out, but without reference to planning, while others are missing. The problems surrounding the planning process were among the main obstacles to the project’s implementation, control and termination. Bguyonb and GADP (1993) confirm this, because they too discovered weaknesses in the Follow-up and Evaluation Department, which was incapable of structuring adequate performance indicators.
Recommendations were made to simplify the activities of the department and to reduce the indicators so that surveys could be concentrated on those actions that would affect necessary improvements in the project. The Public Health Agency of Canada (1996) highlights that the activities of planning, which include developing goals and objectives and setting budgetary requirements, demand a lot of time from project sponsors. Thoughtful planning activities include
the development of good success indicators and allow for continual commitment to the evaluation of impact issues as well as the adoption of realistic strategies and work plans in order to avoid project failure. In the case of the GADP, the success indicators should have been structured as indicated in table 6-2 below.
Table 6-2: Suggested success indicators
GADP Objective GADP Activity Success indicators
Performance indicators Impact indicators
Introduce agricultural intensification
Introduction of agricultural
intensification • Value ($) and quantity of cash crops (wheat, potato) produced and sold (tonnes, kg)
• Quantity of food crops (beans) produced (tonnes, kg)
• Increase of income ($)
• Increase of calories (Kcal)
• Decrease of diseases (rate)
Raise livestock Raising of livestock • Quantity of milk (litres) and meat (kg)
• Number of cows, sheep, or goats raised
• Increase of income ($)
• Increase of calories (Kcal) Develop and
distribute upland and valley bottom
Development of upland
and valley bottom • Land developed (ha)
• Land distributed (ha)
• Increase of income ($)
Develop soil conservation methods
Development of soil
conservation methods • Number of methods used
• Land developed by each method (ha)
• Applicability of each method (satisfaction of users, ha of land protected)
• Increase of income ($)
• Better landscape
Increase soil fertility Improvement of soil
fertility • Increase of crops (tonnes, kg)
• Improvement of productivity (increase of crops by kg or tonne for each crop)
• Increase of income ($)
• Increase of calories (Kcal)
• Decrease of diseases (rate) Facilitate access to
bank credit
Facilitation of access to bank credits
Bank loans granted to the beneficiaries:
• number of credit agreements,
• amount of value in $,
• number of applications for credit,
• number of applications accepted / rejected, etc
• Creation of new project generating income (number of projects created and number of new jobs created)
Carry out reforestation activities
Accomplishment of
reforestation activities • Forest and fruit trees planted (number and type of trees)
• Fruit trees planted (number and type of trees)
• Land developed (ha)
• Better landscape
• Increase of income ($)
• Increase of calories (Kcal)
• Decrease of diseases (rate Build stores, and
construct and rehabilitate roads
Construction of stores, and
rehabilitation of roads • Roads construction and rehabilitation (km, usability)
• Stores construction (number of buildings, usability)
• Agreements signed with subcontractors (number of agreements)
• Improvement of transport (number and quality of roads)
• Improvement of purchasing (number of new markets, reduction of delays) Train beneficiaries Training of beneficiaries • New knowledge in modern agriculture
• Improvement in using modern agricultural techniques
• Improvement of managerial performance
• Increase of agricultural productivity
• Increase of crop productivity (Increase of crop per kg, tonne, litre, etc)
• Soil fertility
• Soil conservation and protection
• Improvement of nutritional status (increase of Kcal)
• Improvement of management skills (increase of profits) Carry out research
and development activities
Accomplishment of research and development activities
• Reports (number)
• Publications (number)
• Domains of intervention (number and type)
• Partnership agreements (number)
• New knowledge
• Enhancement of management and farming techniques
• Community development (housing, communication, health care, incomes, education, etc)
Table 6-2 shows two kinds of success indicators: performance and impact indicators, which may be quantitative or/and qualitative:
• Performance indicators measure the extent to which the project has achieved the objectives.
• Impact indicators measure the changes introduced by the project.
Table 6-2 is presented to indicate ways of overcoming the weaknesses of the GADP in structuring its performance and impact indicators. Indicators, whether measuring the project performances or its impacts, can be qualitative (better landscape or improvement of health) or quantitative (number of roads constructed or increase of income). In the terms of reference of the GADP, some impacts were stated (reduction of food insecurity and improvement of nutritional quality of food)but their actual measurement was not indicated in any records or reports of the project. The Food and Agriculture Organisation of the United Nations (1999) found that the lack of impact assessment is an important weakness of project evaluation.
For the benefit of project planning and implementation and thereafter its evaluation, each goal must have its related objectives, which, in turn, are detailed into specific objectives. And each specific objective must have its related specific activities and indicators. Table 6-3 indicates a model that should be helpful in the processes of project planning and that clearly defines and structures the different elements, and their links, presented along the lines of Figure 6-2.
Table 6-3: Model of structuring performance and impact indicators in the process of the project planning
Goals (general objectives)
Specific objectives
Activities Success indicators
Performance indicators Impact indicators Planned
indicators
Actual indicators
Planned indicators
Actual indicators Goal № 1 Objective № 1 Activity 1.1.0
Activity 1.2.0
Indicator 1 Indicator 2
Indicator 1 Indicator 2
Indicator 1 Indicator 2
Indicator 1 Indicator 2 Objective № 2 Activity 2.1.0
Activity 2.2.0 Objective № 3 Activity 3.1.0 Activity 3.2.0 Goal № 2 Objective № 4 Activity 4.1.0 Activity 4.2.0 Objective № 5 Activity 5.1.0 Activity 5.2.0 Objective № 6 Activity 6.1.0 Activity 6.2.0 Goal № 3 Objective № 7 Activity 7.1.0 Objective № 8 Activity 8.1.0
Etc. Etc Etc Etc Etc Etc Etc
The information about the GADP organizational structure, objectives and performance indicators leads to the conclusion that the project was not adequately planned and that this negatively affected its implementation. Other elements, which were overlooked in the process of GADP planning and which complicated the project implementation, involved external environment factors. These are dealt with in the following section.