D ECISION -M AKING 4
4.4 Naturalistic Decision-Making
114 COGNITIVE ENGINEERING AND SAFETY ORGANIZATION
have built on the concept of bounded rationality (Simon 1957), that is, rationality limited by the tractability of the decision problem, the cog-nitive limitations of the human mind, and the time available to make the decision. In this view, decision makers act as satisfiers, seeking a satisfactory solution rather than an optimal one.
Simon’s findings triggered new studies that looked into rational and other informal models of how experienced people make decisions.
Kahneman et al. (1982) presented research indicating that humans use a range of heuristics that involve less cognitive effort than rational decisions, although the final decision may not be optimal. Among the well-known heuristics have been the availability heuristic (i.e., a reli-ance on a good solution that is readily available and worked well in the past) and the confirmation bias (i.e., a selective attention to informa-tion that confirms a preferred opinforma-tion). Heuristics are powerful tools in making decisions under time pressure because they do not entail the same cognitive effort that rational decisions do. Another quasi-rational model of decision-making proposed by the same researchers was elimination by aspects. According to this model, people often do not have time to consider and weigh all attributes of different options.
In this situation, they would start by establishing a minimal crite-rion and eliminating the options that fail to satisfy it. Subsequently, another criterion can be selected to eliminate more options until a stage is reached where the last option satisfies all the criteria that remain. However, as options get eliminated in a serial fashion, people may miss an option that has a low score in the first few criteria but compensates with a highest aggregated score for all criteria.
DECISION-MAKING 115
emergence of the new paradigm boosted research in decision-making in complex organizations as it paid attention to several work con-straints in the real environment, such as:
• Multiple forms of uncertainty
• Environments where the situation is constantly changing
• Information rich but noisy environments which increase the monitoring workload
• Goals that may evolve in time and may compete with other goals elsewhere in the organization
• Severe time constraints that force practitioners to act quickly
• High consequences arising from decision errors
• Decisions dependent on multiple practitioners working in distant locations
The ATM system represents a typical domain for the application of the naturalistic decision-making (NDM) paradigm since most of these constraints are brought into play, especially in emergencies and abnormal situations.
A proponent of the NDM approach has been the Skill-Rule-Knowledge (SRK) model which describes the decision-making pro-cesses that people use, depending on their level of expertise and the context of the situation (Rasmussen 1986, 1993). The SRK model (or decision ladder) has been very popular because it emphasizes the role of other cognitive functions (e.g., situation assessment and planning) in decision-making and identifies several shortcuts for speeding up decisions. The SRK model (Figure 4.3) postulates that practitioners can make decisions at three levels, depending on the competence lev-els and the situational characteristics (i.e., familiarity, response time, and availability of rules).
Most situations in the ATM domain have been previously encoun-tered by experienced controllers and flight crews while supportive checklists and procedures exist for making suitable choices, especially under time pressure. Experts make decisions based on rules that are externally stored (e.g., procedures) or internally stored in their memory (e.g., acquired through experience and training). Rule-based decisions entail recognition of the problem symptom that helps experts clas-sify the problem and identify appropriate rules to make a choice; usu-ally a standard procedure may present practitioners with two or three
116 COGNITIVE ENGINEERING AND SAFETY ORGANIZATION
options and a method to make a quick choice. With extensive prac-tice, however, practitioners are able to associate an observed symptom with a preferred action pattern in an effortless way. Hence, behavior at the skill-level allows practitioners to time-share many tasks and speed up their performance. However, “perception–action” patterns may produce certain lapses and omissions that are difficult to capture since attention is shared among several tasks.
The most challenging decisions are made at the knowledge-based level where controllers are faced with unfamiliar situations and have to rely on their knowledge to identify candidate options and select the most appropriate one. Knowledge-based decisions may involve ana-lytical comparison of options or may require the creation of a new option. The SRK model is not dogmatic about the decision criteria that experts use. It does not require people to find an optimal decision through a formal comparison of options. Practitioners may use their own knowledge and judgment to make decisions using quasi-rational approaches (e.g., “elimination-by-aspects”). The important thing about
Knowledge-based decisions
Rule-based decisions
Action pattern Situation assessment
Recognition
Observation
Knowledge-based control
Rule-based control
Skill-based control
Figure 4.3 An adaptation of the Skill-Rule-Knowledge model.
DECISION-MAKING 117
knowledge-based decisions is that practitioners have to spend time in assessing the situation before making a decision. Situation assessment may involve thinking about the causes of the problem, projecting how the situation may evolve when an option is selected or even thinking how to share knowledge between team members to avoid side-effects.
Human cognitive control can be shared between two levels, such as when practitioners apply a set of rules but also try to interrogate them using deeper knowledge about the system. In this sense, knowledge can take a supervisory role in the application of rules and even provide a ground for learning from experience. Human heuristics in decision-making can be seen as shortcuts at the knowledge level where prac-titioners resort to rule-based decisions to speed up their cognitive processes; however, the risk remains that short-cuts prevent practi-tioners from introspecting their rules. Effective decision-making in safety critical domains depends on all three levels of cognitive control of the SRK model.
Another decision model that has been very popular in the NDM paradigm regards the recognition primed decision (RPD) model (Klein et al. 1993, 2004). The main tenant of the RPD model is that people make decisions by drawing analogies from their experience once they recognize that have encountered similar situations in the past; this pattern-matching process allows practitioners to identify suitable options and courses of action that may still apply to the cur-rent situation (see top loop in Figure 4.4).
In some cases, however, practitioners may wish to explore alterna-tive options either because they are not certain about the situation or because the cost of errors are very high. In the RPD model, options are evaluated by imaging how their consequences might unfold in the future according to a mental model of the system. All options are evaluated in a serial fashion until a suitable one is found that is fit for purpose. Hence, the middle-loop of evaluation (Figure 4.4) relies on a process of mental simulation that requires a good knowledge or model of the system.
The most complex case is where the situation looks unfamiliar, which requires a thorough assessment of the situation or fault diag-nosis. The situation assessment loop (at the bottom of Figure 4.4) can be supported by a good mental model of the system that guides the search for further information and helps practitioners identify possible
118 COGNITIVE ENGINEERING AND SAFETY ORGANIZATION
causal factors of the problem. Very often, the diagnosis of the situa-tion is likely to be followed by an explorasitua-tion of new opsitua-tions that are evaluated by mentally simulating their effects.
In contrast to the analytical decision theories, the RPD model includes consideration of situation assessment that interacts with option evaluation in several ways. For instance, situation assessment can make option evaluation easier by reducing the set of candidate options to choose from. Hence, the RPD model provides an integra-tion of situaintegra-tion assessment and decision-making that varies accord-ing to the characteristics of the context of work.
In novel situations, where no familiar patterns exist, proficient practitioners supplement situation assessment with a supervisory pro-cess that verifies the results of mental simulation and corrects any problems; this supervisory process has been referred to as metacog-nition. This higher order cognitive function has been addressed by the recognition/metacognition (R/M) model (Cohen et al. 1996). The R/M model describes a set of critical thinking strategies that supple-ment recognition processes in rapid decision-making. Metacognition
Situation
Data and cues
Patterns of the situation Action courses
and options
Mental simulation
Mental models which you
assess using
that affect the which
generates
which relies on
your
that let you recognize which
you assess using your
Pattern-matching
Situation diagnosis Course evaluation
Figure 4.4 The Recognition Primed Decision (RPD) model showing the functions of pattern-matching, situation diagnosis, and course evaluation. (From Klein, G.A., The Power of Intuition, Currency Books, New York, NY, 2004.)
DECISION-MAKING 119
involves a number of cognitive strategies regarding whether it is worthwhile to think more about a problem, how to critique a situ-ation model for incompleteness, conflict or unreliability, and how to improve it by collecting new information or revising assumptions.
The origins of recognition/metacognition model are traced into a United States Navy (USN) research program known as tactical decision-making under stress. TADMUS was launched after a tragic accident where the US Aegis cruiser Vincennes shot down a com-mercial Iranian airliner, killing 290 passengers (1989). The accident was attributed to many work factors with the most important being flaws in the decision-making process of practitioners. Among other approaches, the R/M model has been developed and validated in the context of the TADMUS program.
The recognition/metacognition model claims that practitioners build a mental model of the situation and of suitable plans that are subject to critique and correction. These metacognitive functions depend on the characteristics of the situation (i.e., time availability, high stakes, and uncertainty). Figure 4.5 shows how the functions of model building, critiquing, and correcting are adapted to the work environment as follows:
1. Carrying out a quick test, which rapidly assesses whether it is worth taking more time for critical thinking rather than act-ing immediately on the current recognition of the situation 2. Critiquing the results of recognition in order to handle three
kinds of uncertainty:
a. incompleteness in understanding the situation or in for-mulating response plans
b. conflicting evidence or goals
c. explicit or implicit assumptions made to simplify the problem
3. Correcting flaws by shifting attention to other evidence or making other assumptions.
Metacognition occurs when the benefits associated with criti-cal thinking outweigh its costs. This is likely to be the case when the situation is novel (i.e., the uncertainty is high), the cost of errors are considerable but there is sufficient time for critical thinking. The quick test considers these factors and, if conditions are appropriate,
12 0 COGNITIVE ENGINEERING AND SAFETY ORGANIZATION
interposes a process of critical thinking. The cornerstone of the R/M model is a critique of our current understanding of the situation and our earlier decisions. Critiquing models of the situations or decisions is a means of making decisions when uncertainty is high but there are expectations that additional information will be available for later improvements.