Basics of Educational Evaluation
2.3 Important Distinctions in Evaluation Theory .1 Ideal-type stages in evaluation
2.3.3 Accountability and improvement perspectives reconsidered
The distinction between formative and summative roles of evaluation is closely related to two major perspectives from which evaluations are conducted, the first perspective is generally known as the “accountability perspective” and the second as the “improvement perspective”. Both orientations were introduced in Chapter 1 as 2 out of 3 basic functions of educational evaluation.
In general terms, accountability refers to holding public institutions and services responsible for the quality and output of their performance. Glass (1972) states that accountability involves several loosely connected strands: “disclosure concerning the product or service being provided; product or performance testing; and redress for poor performance (Glass, 1972). The third element implies that accountability is not just a matter of providing and judging information but at least also “foreshadows” actions by competent authorities in the sense of sanctions or rewards.
When evaluation is situated in a context of organizational learning and improvement this last element (“redress for poor performance”) is replaced by the assumption of a less
“controlling” type of use of evaluative information, where adjustment has the nature of mutual adaptation and evaluation is formative rather than summative.
Part of the complexities of the fitting of evaluation within the context of administrative or political decision-making have to do with real or alleged conflicts of interest between decision-makers, practitioners and evaluators. The sometimes even antagonistic nature of the relationships between these actors is much more associated with evaluation from an accountability perspective than is the case for improvement-oriented evaluation. In case of an accountability orientation, evaluation is more judgmental and controlling and closely tied to either vertical relationships within an administrative hierarchy or to demands from important external constituencies on which the existence of the organizations may depend. From the outside, particularly in the service sector, accountability is likely to be seen as a rightful requirement of tax-payers and other stakeholders to check the merits of their “investments”. From the inside, such control and assessment is oftentimes perceived as threatening or even unjust. The “cognitive”
complexity of decision-making and primary processes in the tertiary sector [e.g.
education], together with potentially threatening nature of accountability-oriented evaluation may give rise to a distrust of evaluation procedures. Cronbach and his colleagues (1980) go even as far as calling “accountability being dangerously close to totalitarisation”. Scheerens (1983) provides a case study of the evaluation of a pilot project in the sector of adult education in the Netherlands. Although in this case evaluators tried to combine “summative” and “formative” evaluation functions, teachers, however, reacted as if the project was only carried out for accountability purposes. They felt so threatened by the partly internal and partly external evaluation activities that a majority refused to cooperate in data collection activities which, of course, had a detrimental effect on the implementation of the evaluation program.
Even if this is considered too strong a qualification, evaluation practices carried out from an external accountability perspective may be hampered or frustrated by behavior of actors involved who feel threatened, when they consider the stakes to be high. It will be interesting to check and see to what extent external school evaluations, indicated from an accountability perspective suffer from such attitudes, and to what extent there are differences between countries which could be grounded in different educational cultures, and different traditions in acceptability of external control. The antagonistic nature of accountability-oriented evaluations in the education sector is supported by organizational theoretical constructs such as the “professional bureaucracy” (Mintzberg, 1979), of which resistance to rationalization and external review is one of the defining characteristics.
Educational professionals, i.e. teachers may contest the expertise and methodology of evaluation researchers, as when they consider themselves as the best medium and service of knowledge on what goes on in classrooms.
Evaluation from an improvement perspective has an altogether different orientation.
Here, learning, feedback, a formative role of evaluation, intrinsic interest in processes and a methodology that is controllable by teachers are the central characteristics. External school evaluation is more likely to be accountability-oriented and internal evaluation is more likely to be improvementoriented, although exceptions may occur, as when a school deploys an external consultant to review, for instance, its managerial structure.
While “control” is the key-term in accountability, “learning” is the key term in internal improvement-oriented evaluation. A question of internal “self evaluation”, however is
whether, it can be sufficiently objective and impartial to provide a firm basis for such learning processes.
Examples of accountability-oriented school evaluation are the public reporting of average pupil achievement tests in public media and the composition of so-called
“league” tables. The use of such approaches is much contested. The most severe criticism is that such procedures could stimulate selection processes which are detrimental to the principle of equality in education. Thus, schools might seek to enhance their average output, not by improving the school organization and teaching, but by attempting to obtain a “favorable” input of students. Particularly when school-scores have not been adjusted for intake or prior achievement, there is a danger that such practices might occur. But in addition, there might be other selection processes, having the same type of effect. Stronger schools with “better” intake could become more attractive to better teachers. And parents from more privileged socioeconomic backgrounds would be more likely to profit from the information that is made available by the publishing of school- scores.
Types of improvement-oriented school evaluation are school-based review procedures which are part of school improvement projects and school selfevaluations which monitor the “normal” functioning of a school.