CHAPTER 2 21
2.3 Theory of Evaluation 25
2.3.2 Fourth Generation Evaluation 30
Page 30
Page 31
Evaluation Review Technique (PERT) and Critical Path Method (CPM) will fall into the category of third generation evaluation methods.
Guba and Lincoln (1989) argued that there were some serious limitations to the perspectives on evaluation provided by all of these three generations. In particular, they failed to grasp:
The real power relations existing and pressuring the evaluator within the evaluation process. These power structures not only affected the relationships between those being evaluated, but also curtailed the practical ability of the evaluator to be a natural outsider;
The plurality of value bases existing simultaneously within the evaluation process, in addition to the multiple interests, agendas, and perceptions; and the consequential need to accept and cope with multiple perspectives within the evaluation process.
As a consequence to these limitations, a “fourth generation” by Guba and Lincoln (1989) refined the evaluation methodology. Guba and Lincoln (1989); Morse (1994) state that Fourth Generation Evaluation (FGE) is defined by the shift in epistemological positions, from a predominantly positivist to a constructivist paradigm. There are two main phases of a constructivist evaluation; discovery and assimilation.
The discovery phase is the phase where the evaluator provides a description of
“what is going on here”. The word here can be substituted with process, programme, person that is being evaluated.
The assimilation phase is where the evaluator incorporates the new discoveries into existing knowledge regarding the object to evaluate.
Page 32
The Fourth Generation Evaluation was defined as a hermeneutic dialectic negotiation (see Koch, 2000:117), where the evaluator is both a facilitator that elicits the views of different stakeholders, and a mediator in bringing the stakeholders to a level of consensus as to what happened in the past, and what should happen in the future.
Koch (2000) mentions that the appeal of a “negotiation” process is that the evaluation strives to give its stakeholders a voice or the opportunity “to have a say” about things that affect them. The fourth generation evaluator does not attempt to identify “facts”, but rather to highlight and mediate between varying views rooted in different interests and worldviews. Likewise, the fourth generation evaluator will not seek to determine a solution, but rather facilitates the discussion among the various stakeholders.
Trochim (2009:16) asserts that FGE methods can be seen to fall into the category of qualitative/anthropological models and participant-oriented models. In qualitative models the emphasis is on the significance of observation and the value of subjective human interpretation while participant-oriented models highlight the importance of the evaluation participants and the value of their knowledge, especially clients and users of the object to evaluate. Huebner and Betts (1999:342) believe the main advantages to such an evaluation approach, regarding the negotiation process of the stakeholders, are:
“(1) attempts to help them reach consensus about their reality or to recognise discrepancies; (2) educational because the process provides the stakeholder with the opportunity to incorporate others’ perspectives of the construct into their own; and (3) empowering because the entire process is built on negotiated stakeholders’
constructions of the evaluand”.
The process of FGE involves firstly, the identification of stakeholders, secondly, understanding and comprehending stakeholders’ claims, issues and concerns about the construct of the problem to evaluate; and thirdly, seeking consensus among stakeholders via discussion, negotiation and exchange (see Guba and Lincoln, 1989;
Huebner & Betts, 1999:343). It is during the last phase where the opportunities for co- learning and co-creation arise. Rebien (1996:155) suggests that social change is seen
Page 33
to take place when stakeholders themselves create their own solutions based on their understanding of the problem.
According to Zadek (2000), FGE has several shortcomings that if left unchallenged will undermine the value of the method from both theoretical and practical viewpoints.
The two particular shortcomings he underlines are:
Preconception of Preconditions: Guba and Lincoln (1989) are of the opinion that it is only through a hermeneutic process that effective mediation between different interests and insights become possible. They set out what they consider to be the preconditions for such a “productive hermeneutic dialectical negotiation”, which include: the need for all parties to work from a position of integrity; a willingness of all stakeholders to share power; a willingness of all stakeholders to change; and a willingness of all stakeholders to reconsider their value positions (see Guba and Lincoln, 1989:148-149).
Zadek (2000) believes there are some problems with these conditions and the most obvious being that these conditions are unlikely to pertain except in the rarest of cases.
Groups with relative power seldom want to share on any meaningful basis in practice, even those who say or actually think that they do. The danger of Guba and Lincoln’s severe conditions are that they marginalise Fourth Generation Evaluation from its practical process. Secondly, the hermeneutic dialectic process proposed by Guba and Lincoln would arguably not be necessary if these conditions did actually prevail. That is not to say that there is no space for a process to increase mutual understanding between the most collegial of people. Rather, it is amiss that the hermeneutic process is necessarily political, which in itself implies that Guba and Lincoln’s preconditions do not hold. Consequently, Fourth Generation Evaluation has potential value as an emancipatory process primarily because these preconditions do not prevail in our society and as such Guba and Lincoln undervalue their own insight by imposing these conditions.
Page 34
Mediation for Action: The approach in which Guba and Lincoln develop their argument in favour of Fourth Generation Evaluation is in itself an interesting example of the very polarisation process that they themselves are criticising. As mentioned earlier, FGE is posited as an alternative to a positivist approach. A different and useful interpretation is that the critical responsive and constructivist elements of the approach constitute one aspect of an appropriate evaluation process. Particularly, without incorporating an understanding of a mediated solution that is essentially deemed a
“consensual truth” by the stakeholders, consequential actions become almost impossible to determine. This implies that different stakeholders could interpret a particular event differently. However, for a consequential action to arise from the evaluation process requires ultimately that a “fact” be determined through this process that forms the basis for agreement and further action.
Zadek (2000) believes strict preconditions and polarisation of conventional approaches endanger the practical application of his proposed method, and its internal consistency.
Koch (2000:124) warns that the power aspects of the negotiation cannot be understated and evaluators would benefit from cautious appreciation and thorough prior analysis of situations. The rhetoric of FGE is that it may empower, liberate and emancipate (Guba and Lincoln, 1989). It seems the most it can give stakeholders is a sense of control over their lives and/or the workplace.
Another important approach of evaluation is participatory evaluation. Participatory evaluation differs from traditional evaluation by attempting to include all stakeholders in all aspects of the process, a characteristic which is of particular importance to this study.