Evaluation as a Tool for Planning and Management at School Level
5.2 The Rationality Paradigm Reconsidered
5.2.5 Retroactive planning
The idea of “retroactive planning” is best clarified by zooming in on the ways it differs from “synoptic rational planning”, while still clearly being part of the overall rationality paradigm.
The pure rationality model (Dror, 1968) formally enables the calculation of the optimal choice among alternatives after a complete preference ordering of the end states of a system has been made. This ideal is approached in mathematical decision theory, as in game theory where different preference orderings of different actors can also be taken into account. For most “real life” situations of organizational functioning the assumptions of pure rationality are too strong, however. Simon’s (1964) construct of “bounded rationality”, modifies these assumptions considerably by recognizing that the information capacity of decision-makers is usually limited to taking into consideration just a few possible end states and alternative means.
Cohen, March and Olsen (1972) and March and Olsen (1976) go even further in criticizing the descriptive reality of the pure rationality model. Cohen et al. (1972) describe organized anarchies as characterized by “problematic preferences”, “unclear technology” and “fluid participation”. With respect to problematic preferences, they state that the organization can “better be described as a loose collection of ideas than as a coherent structure; it discovers preferences through action more than it acts on the basis of preferences” (ibid, p. 1). Unclear technology means that the organization members do
not understand the organization’s production processes and that the organization operates on the basis of trial and error, “the residue of learning from the accidents of the past” and
“pragmatic inventions of necessity”. When there is fluid participation, participants vary in the amount of time and effort they devote to different domains of decision making (ibid, p. 1).
According to Cohen et al., decision making in organized anarchies is more like rationalizing after the fact than rational, goal-oriented planning. “From this point of view, an organization is a collection of choices looking for problems, issues and feelings looking for decision situations in which they might be aired, solutions looking for issues to which they might be the answer, and decision makers looking for work” (ibid, p. 2).
They see educational organizations as likely candidates for this type of decision making.
In terms of coordination, organized anarchies have a fuzzy structure of authority and little capacity for standardization mechanisms.
March and Olsen (1976) describe their reservations with respect to rational decision making in terms of limitations in the complete cycle of choice (see Figure 5.1, where this cycle is depicted).
The relationship between individual cognitions and preferences on the one hand and individual action on the other is limited, because of limitations in the capacity and willingness of individuals to attend to important preferences and because of discrepancies between intentions and actions: “…the capacity for beliefs, attitudes, and
Figure 5.1 The complete cycle of choice, cited from March & Olsen (1976).
concerns is larger than the capacity for action” (ibid, p. 14).
At the same time there may be a loose connection between individual action and organizational action, because internal individual action may be guided by other principles than producing substantive results (e.g. allocating of status, defining organizational truth and virtue). In the same vein they observe that actions and events in the environment sometimes have little to do with what the organization does and that it is sometimes hard to learn from environmental response.
Despite all these limitations on the descriptive reality of rational decision making and planning in organizations, even the most critical analyses leave some room for shaping
reality somewhat more to the core principles. The first type of activity which could bring this about is synoptic planning.
The earlier stated ideal of “synoptic” planning, namely to conceptualize a broad spectrum of long term goals and possible means to attain these goals, contains the basic logic of planned change. In models of planned change the various aspects of synoptic planning are usually structured as phase models (compare the previous chapter); the following description of the different phases is partly based on Ackoff, 1981, 74, 75).
In a first phase there is a reflection on values and normative aspects that should be attained through social programs or specific organizational behavior. This first phase can also be taken as the phase of defining the problem domain, in the sense of a system of threats and opportunities that face the organization.
In the second phase ends planning takes place in the sense that goals end objectives are specified.
In the third face means-planning takes place, where ideally there should be a rationale for selecting the means (examples in education are results of empirical educational effectiveness or practical experience on “what works” in education).
In a fourth phase resource planning is focused at determining “what resources will be required, when they will be required, and how to obtain those that will not otherwise be available” (ibid, 75).
In a fifth phase design of implementation and control determines “who is to do what, when, and where, and how the implementation and its consequences are to be controlled, that is, kept on track” (ibid, 75).
In a sixth phase (which, by the way, is not specifically mentioned by Ackoff), monitoring and evaluation, which can be seen as part of the control processes, are used for feedback and possible modification of means, goals or even values.
The feedback mentioned in this last phase turns the sequence in steps in fact into a circle that can go on and on. Many authors, including Ackoff, do not take the sequence of phases too seriously and say in fact that they make take place in any order. Others, however, see the way one “steps into” the planning, implementation and feedback circle as non-trivial. Borich and Jemelka (1981) see the planned change process as society’s attempts to “maintain equilibrium when the system threatens to become disadvantageously influenced by forces whose effects were previously neglected or would have been difficult to predict” (ibid, 216). They see a qualitative difference, however, in two ways of regaining equilibrium. The first being the traditional one where goals are formulated to determine behavior, and which one could see as a proactive orientation (J.S.) the second emphasizing that behavior provides impetus for goals, which they see as a more retrospective orientation. They illustrate the difference in these two orientations with a citation from Weick (1969):
“This sequence in which actions precede goals may well be a more accurate portrait of organizational functioning. The common assertion that goal consensus must occur prior to action obscures the fact that consensus is impossible unless there is something tangible around which it can occur. And this “something tangible” may well turn out to be actions already completed, Thus it is entirely possible that goal statements are
retrospective rather than prospective. (Weick, 1969, The Social Psychology of Organizing, Addison-Wesley, p. 8)
Borich and Jemelka (1982) use this view on retrospective analyses to explain two different views on program evaluation, which they indicate as forward evaluation and backward evaluation.
Forward looking evaluation is the traditional view, and means assessing discrepancies between a program’s objectives and outcomes.
“Rather than focus on discrepancies between program objectives and outcomes, backward evaluation has as its goal a statement of the values reflected by the program” (ibid, 220).
Once these values are made clear they can be compared to the official or externally established values.
The question should be raised what is gained by emphasizing the retrospective or rather the retroactive view on planned change. According to March and Olsen (1976), learning from experience meets the same fundamental limitations as rational planning.
When goals are ambiguous, which these authors assume, so are norms and standards for interpreting evaluative information. Another limitation is to determine the causality of observed events. They discern four major limitations to organizational learning:
• role-constrained experiential learning, if evaluative information is contrary to established routine and role definition it may be disregarded and not frustrated into individual action;
• superstitious experiential learning; in this case organizational action does not evoke an environmental response (i.e. is ineffective);
• audience experiential learning, when learning of individual organization members does not lead to organizational adaptation;
• experiential learning under ambiguity; in this situation it is not clear what happened or why it happened (ibid, p. 56–58).
Even though it is correct that in determining what to evaluate one is forced to address the same kind of selection about what is valuable as in the case of stating objectives, there are at least practical advantages in choosing the retroactive approach:
• on the basis of available instruments a broad scan of current functioning could be made, through which a more efficient selection process on what areas are relevant to address might be expected;
• the difficult and often lengthy deductive process of operationalizing goals can be avoided;
• the basis of subsequent discourse about objectives and means becomes firmly rooted in empirical evidence and is therefore likely to be more concrete and to the point as compared to choosing the “deduction from goals” route;
• particularly when evaluation is of the monitoring type, which means a regular
description of the system on key features, the retroactive approach could be viable in creating organizational learning and improvement.
In Table 5.1 the differences between proactive synoptic planning and retroactive planning are summarized.
In educational settings retroactive planning can take place at the level of the national educational system, at regional or district level, and at school level.
At each of these levels a particular type of evaluation or assessment should be put in place as the basis for retroactive planning.
At national level a national assessment program, possibly embedded in a larger system of indicators, would be the most likely instrument. At intermediate levels some kind of indicator or monitoring system could play this role, while at school level school self evaluation instruments are to be seen as the basis for retroactive planning.
The argument for retroactive planning, which one could also refer to as “evaluation centered planning” can be made more forcefully when considering the view on organizational functioning that is inherent in the concept of the learning organization.
Table 5.1 Schematic Comparison of Synoptic and Retroactive Planning.
characteristics synoptic planning Retroactive planning initial activity formulate encompassing goals Assess organization’s functioning choice of means and
methods deduce from scientific knowledge Induce as improvement of weaknesses in current functioning scope a broad scope encompassing all
major aspects of the organization a partial “piecemeal” approach time-frame long term short term
organizational structure
bureaucracy Learning organization
organizational participation
top-down Participative