CHAPTER 2 21
2.3 Theory of Evaluation 25
2.3.3 Participatory Evaluation 34
Page 34
Mediation for Action: The approach in which Guba and Lincoln develop their argument in favour of Fourth Generation Evaluation is in itself an interesting example of the very polarisation process that they themselves are criticising. As mentioned earlier, FGE is posited as an alternative to a positivist approach. A different and useful interpretation is that the critical responsive and constructivist elements of the approach constitute one aspect of an appropriate evaluation process. Particularly, without incorporating an understanding of a mediated solution that is essentially deemed a
“consensual truth” by the stakeholders, consequential actions become almost impossible to determine. This implies that different stakeholders could interpret a particular event differently. However, for a consequential action to arise from the evaluation process requires ultimately that a “fact” be determined through this process that forms the basis for agreement and further action.
Zadek (2000) believes strict preconditions and polarisation of conventional approaches endanger the practical application of his proposed method, and its internal consistency.
Koch (2000:124) warns that the power aspects of the negotiation cannot be understated and evaluators would benefit from cautious appreciation and thorough prior analysis of situations. The rhetoric of FGE is that it may empower, liberate and emancipate (Guba and Lincoln, 1989). It seems the most it can give stakeholders is a sense of control over their lives and/or the workplace.
Another important approach of evaluation is participatory evaluation. Participatory evaluation differs from traditional evaluation by attempting to include all stakeholders in all aspects of the process, a characteristic which is of particular importance to this study.
Page 35
development and analysis (Bradley, Mayfield, Hehta, and Rukonge, (2002); Edun, (2000); and Estrella, 2000). Cornwall (1996) and Mtshali (2000) advocate that participatory evaluation has the potential to incorporate local knowledge into the process of evaluation as well as to build the analytical capacity of participants to evaluate their own needs and priorities, make decisions on these issues and take action to address problems (Estrella and Gaventa, 1998).
Weiss (1998) sees participatory evaluation ranged on a continuum with stakeholder, collaborative and empowerment evaluation along the continuum. Hall and Hall (2004) describe stakeholder evaluation as engaging with different stakeholder interests in order to understand their views, concerns and priorities, but maintain that the evaluator is in charge of the conduct of the evaluation. However, Cousins and Earl (1992) provide a differentiation between participatory evaluation and the conventional stakeholder-based model. Collaborative evaluation places the evaluator as “co- investigator” with programme staff, where the research skills of the one are combined with the empirical knowledge of the others to take joint responsibility for the evaluation. On the other end of the continuum, empowerment evaluation gives control of the evaluation to the practitioners, with advice and help from the evaluator.
Diez and Estaban (2000) maintain that participatory evaluation starts out from a recognition that evaluation develops within a pluralistic society and allows evaluation to be built upon the ideas, values and aspirations of those taking part at all levels and throughout the whole evaluation process. Similarly, the evaluation design gradually takes shape through the collaboration of all the stakeholders and their active participation in the analytical evaluation process. The implication is that the focus increases the probability that the results achieved through the evaluation will be used in an effective way to improve the policy, since it creates space for the actors in the programme to make the actual process and its results their own, thereby transforming the evaluation into a learning process.
Page 36
The evaluation of an academic department as a service provider should be understood as a participatory evaluation since this approach makes it possible to convert evaluation into an exercise contributing to achieving the goals of acceptable service quality. Diez and Estaban (2000) assert that it is precisely at the regional and/or local level where participatory evaluation can be easily implemented. Stame (1999:106) who is in support of the approach maintains that the evaluation of new regional policies (including those affecting service quality at a university) should be transformed into participatory evaluation and the viewpoints of the different actors in relation to both the methodology as well as the content of the evaluation should be taken into consideration. Equally, Kuhlmann (1998) stresses how the various interests and perceptions of the actors taking part must be explicitly taken into account.
Some of the advantages that participatory evaluation introduces into the evaluation practice will now be discussed:
Evaluation is understood as a learning process particularly from the perspectives of all the stakeholders involved. According to Kuhlmann (1998:131) evaluation ceases to be an exercise of assessment where the predominant perspective comes from only one source, when this only is the view of the policy designer, as the only criteria for evaluation, this becomes an exercise simulating the appearance of a learning process.
Kuhlmann (1998:132) believes this learning process allows the creation of a working framework where the evaluation process is used to build trust among stakeholders, managers, institutions and evaluators. Participatory evaluation provides the space to democratise the process of knowledge building when active participation is a practice applied throughout the entire evaluation process and be directed towards identifying and resolving problems and improving understanding.
Although evaluation is used to create useful knowledge for those involved in the process in order to achieve their objectives, Finne, Levin and Nilssen (1995:13) believe the process is aimed at creating a situation where new understanding is built on the best from all participants. In a pluralist society where there exist a multiplicity of
Page 37
viewpoints and perspectives, to expect to obtain an exact objective measure of impact, is neither possible nor desirable. Furthermore, when attempting to evaluate complex organisations the objective must be to create practical knowledge, instead of mechanistic judgements concerning the results, and attention must be fixed constantly on the learning process.
Participatory evaluation favours learning for action, since the evaluation process is used to propel action directed at improvement. Finne et al. (1995:14) suggest that it is a practical kind of knowledge that will stimulate the capacity of governments, community institutions and organisations like universities, to solve the pertinent problems.
Diez and Esteban (2000) claim that participatory evaluation makes it possible to strengthen the power of the participants to resolve their economic and social problems.
Against this background, some researchers highlight the capacity of evaluation to prepare the community for action (empowerment). Evaluation is understood as a process of collaborative change that combines knowledge creation and, through learning, facilitates mobilisation for action. Participatory evaluation may be conceptualised as a way of developing awareness, facilitating learning and empowering the different stakeholders to resolve the challenges confronting them.
Academic literature indicates that evaluation is seen as an instrument that makes it possible to observe the progress of the initiative, make short term corrections and centre on the proposed objectives. Through evaluation, the meaning of social reality can be explained from different perspectives, while there is an increased possibility that the stakeholders will feel that the results are relevant and proper to them and that there is a guarantee of them being put into practice.
Some of the drawbacks of participatory evaluation are that it may be much more time- consuming for both the evaluator and the organisation than a traditional goal-oriented evaluation where the question to be asked and the methodology to be followed are
Page 38
determined in advance. Participants will require time off from their regular duties in order to participate effectively in the evaluation process. In addition, clients and participants may require special assistance to become integrally involved in the evaluation. In order for the entire evaluative process to be participatory, the details of the evaluation cannot be fully identified in advance, because use of specific reporting criteria or other evaluation guidelines dictated by sponsors or funders, will limit the participation and input of both evaluators and non-evaluators. Upshur and Barreto- Cortez (1995:8) propose that a truly participatory process is entirely in the hands of the participants, not the evaluator or an outside source. This can empower participants, but it means that in order to use participatory evaluation, the organisation must be committed to the endeavour and the context must be appropriate.
Participatory evaluation allows evaluators, participants, managers and those directly and indirectly affected, to work together in an open exchange of information where all have an opportunity to participate in the debate. This process of collaboration creates new demands for evaluation. The aim of evaluation moves beyond its contribution towards an understanding of the effects of the phenomena under study and of the generation of learning processes, to a stage of how this knowledge and learning can be used in taking decisions. Evaluation plays a new role in which the interchange of information, and formal and informal learning, demystifies the process of evaluation itself, clarifies the function and expectations of what evaluation can offer, and generates mutual trust, helping actors to know and understand how others see the world.