Principles of Intelligence Analysis and Synthesis
5.3 The Integrated Reasoning Process
• Negative evidence. Hypotheses that contain evidence of a nonoccurrence of an event (or nonexistence of an object) may confirm a hypothesis.
This is the kind of “dog that didn’t bark” evidence applied by Sherlock Holmes in the short story,Silver Blaze[16].
This process inherently demands a search for alternative hypotheses that extend beyond the hard evidence available. The U.S. Commission on Theater Ballistic Missile Threats has noted the importance of intelligence analysis exploring hypotheses that go beyond available evidence:
Yet, in a large number of cases examined, Commissioners found analysts unwilling to make estimates that extended beyond the hard evidence they had in hand, which effectively precluded developing and testing alternative hypothesis about the actual foreign programs taking place [17].
5.2.3.2 Hypothesis Selection
Abduction also poses the issue of defining which hypothesis provides the best explanation of the evidence. The criteria for comparing hypotheses, at the most fundamental level, can be based on two principle approaches established by phi- losophers for evaluating truth propositions about objective reality [18]. The cor- respondence theory of the truth of a proposition p is true is to maintain that “p corresponds to the facts.” For the intelligence analyst this would equate to
“hypothesis h corresponds to the evidence”—it explainsallof the pieces of evi- dence, with no expected evidence missing, all without having to leave out any contradictory evidence. The coherence theory of truth says that a proposition’s truth consists of its fitting into a coherent system of propositions that create the hypothesis. Both concepts contribute to practical criteria for evaluating compet- ing hypotheses (Table 5.5).
In the next chapter, we will introduce the practical implementation of abduction in the methodology of alternative competing hypotheses (ACH). We now turn to integrating these formal and informal methods of reasoning for practical analysis-synthesis in the intelligence problem-solving environment.
describe how the fundamental inference methods are notionally integrated into the intelligence analysis-synthesis process.
We can see the paths of reasoning in a simple flow process (Figure 5.4), which proceeds from a pool of evidence and aquestion(a query to explain the evidence) posed about the evidence. This process of proceeding from an eviden- tiarypoolto detections, explanations, or discovery has been calledevidence mar- shaling because the process seeks to marshal (assemble and organize) into a representation (a model) that:
• Detects the presence of evidence that match previously known premises (or patterns of data);
• Explains underlying processes that gave rise to the evidence;
• Discovers new patterns in the evidence—patterns of circumstances or behaviors not known before (learning).
The figure illustrates four basic paths that can proceed from the pool of evidence, our three fundamental inference modes and a fourth feedback path:
1. Deduction. The path of deduction tests the evidence in the pool against previously known patterns (or templates) that represent hypotheses of activities that we seek to detect. When the evidence fits the hypothesis template, we declare a match. When the evidence fits multiple
Table 5.5 Hypothesis Evaluation Criteria
Basis of Truth
Hypothesis Testing Criteria
Application to Intelligence Analysis-Synthesis Criteria
Correspondence The hypothesis corresponds to all of the data
1. Completeness—all expected data is present (e.g., there is no missing evidence)
2. Exclusivity—all available data matches the hypothesis; no data contradicts the hypothesis 3. Nonconflicting—there are no mutually exclusive hypotheses that also correspond to the data Coherence The hypothesis coheres
to (is consistent with) all propositions that make up the hypothesis
1. Consistency of logic—the hypothesis-creating system that leads from evidence, relationships (e.g., casual, organizational, or behavioral), and processes (e.g., laws of physics or rules of behavior) to predicted outcomes is logical and consistent 2. Consistency of hypotheses—all hypotheses follow the same consistent hypothesis-creating system
hypotheses simultaneously, the likelihood of each hypothesis (deter- mined by the strength of evidence for each) is assessed and reported.
(This likelihood may be computed probabilistically using Bayesian methods, where evidence uncertainty is quantified as a probability and prior probabilities of the hypotheses are known.)
2. Retroduction. This feedback path, recognized and named by C.S.
Peirce as yet another process of reasoning, occurs when the analyst conjectures (synthesizes) a new conceptual hypothesis (beyond the cur- rent frame of discernment) that causes a return to the evidence to seek evidence to match (or test) this new hypothesis. The insight Peirce provided is that in the testing of hypotheses, we are often inspired to realize new, different hypotheses that might also be tested. In the early implementation of reasoning systems, the forward path of deduction was often referred to asforward chaining by attempting to automati- cally fit data to previously stored hypothesis templates; the path of
Hypothesis testing –fit to known templates
Assess multiple hypothesis
likelihoods
Assessment criteria:
• Correspondence
• Coherence
• Pragmatic
Conjecture to form a generalized
hypothesis Conjecture best explanation
of evidence
Validate new hypothesis
likelihood Assemble
evidence to the generalized frame
of discernment Assemble evidence to best
frame of discernment Search for
evidence to affirm new hypothesis
Conjecture new hypothesis beyond the
current frame of discernment
Discovery Explanation Detection
Assess multiple hypothesis likelihoods Deduction
Retroduction
Induction Abduction Pool
of evidence
Figure 5.4 Integrating the basic reasoning flows.
retroduction was referred to as backward chaining, where the system searched for data to match hypotheses queried by an inspired human operator.
3. Abduction. The abduction process, like induction, creates explanatory hypotheses inspired by the pool evidence and then, like deduction, attempts to fit items of evidence to each hypothesis to seek the best explanation. In this process, the candidate hypotheses are refined and new hypotheses are conjectured. The process leads to comparison and ranking of the hypotheses, and ultimately the best is chosen as the explanation. As a part of the abductive process, the analyst returns to the pool of evidence to seek support for these candidate explanations;
this return path is calledretroduction.
4. Induction. The path of induction considers the entire pool of evidence to seek general statements (hypotheses) about the evidence. Not seek- ing point matches to the small sets of evidence, the inductive path conjectures new and generalized explanation of clusters of similar evi- dence; these generalizations may be tested across the evidence to determine the breadth of applicability before being declared as a new discovery.
Now we can examine how this process might flow in a typical intelligence process. Consider the case where a terrorist group (“ACQM”) has attacked a facility of country A, and the analyst is posed with the I&W question: “Is there evidence that that the group has capabilities, plans, or operations to conduct other imminent attacks?” The flow of analytic activities (numbered 1–8) is sequentially illustrated in Figure 5.5:
1. Deduction. The analyst immediately checks all intelligence sources and pools the evidence about ACQM to determine if the evidence fits any known patterns of attack of other facilities. This hypothesis-testing process seeks to deduce attack capabilities, plans, or operations initi- ated; if deduction fails, it may be due to lack of evidence, lack of breadth of hypothesis templates (not robust enough), or insufficient templates to cover new categories of attack.
2. The analyst hypothesis tests the evidence against known patterns of attack. No matches to existing templates deduce that attacks (of known types or for known targets) are not borne out by the evidence.
3. Retroduction. The analyst conjectures that ACQM may be planning attacks on other targets (people, transportation, media) using the same
modus operandi (MO). This new frame of discernment goes beyond the hypotheses that were considered within the deductive process.
4. This conjecture creates the basis for a search back through (retro) the evidence pool to explore other new patterns of attack that might target people, transportation, and media.
5. Abduction. Indeed, several hypotheses for new kinds of attacks on maritime transportation targets are quite feasible.
6. The evidence arrayed against these hypotheses is compared, additional collections of data are requested, and the results show that two target hypotheses (transportation around ports and rivers) are feasible and even likely. This provides a basis (the indication) for warning these categories of targets and an explanation for the warning.
Hypothesis testing–fit to known templates
Assess multiple hypothesis
likelihoods
Conjecture to form a generalized
hypothesis Conjecture best explanation
of evidence
Validate new hypothesis
likelihood Assemble
evidence to the generalized frame
of discernment Assemble evidence to best
frame of discernment Search for
evidence to affirm new hypothesis
Conjecture new hypothesis beyond the
current frame of discernment
Discovery Explanation Detection
Assess multiple hypothesis likelihood's Deduction
Retroduction
Induction Abduction Pool
of evidence
8 7
6 5
4 1
3 2
1
Conjecture a general pattern of attacks across all target categories Evidence approximately fits several other kinds of target attacks.
Most likely explanations of evidence offer reason for warnings
Validate an overall attack strategy, plan, and MO process No fits, but what about
same MO, different kind of target?
Do I find any evidence of patterns that fit previous, known attacks?
Do I find any evidence that fits new attack pattern for other kind of targets?
Figure 5.5 Representative intelligence problem search sequence.
7. Induction. Finally, as more evidence is accumulated over time, and the ACQM plans and conducts more attacks (some successes, some fail- ures), the evidence shows a more general pattern of behavior of the group—characterized by special forms of financing, a hatred for cer- tain cultural symbols, and special communications behaviors.
8. This generalized pattern is tested against all previous attacks and can be validated to provide a high-level template for future hypothesis testing in the deductive process.
This example illustrates onethreadof many possible flows through the rea- soning processes that analysts apply to iteratively analyze the growing pool of evidence and synthesize feasible hypotheses to be explored. The process also illustrates the validation of templates, created by induction, and their use in the deduction process. Once discovered by induction, these templates may be used for future attack detection by deduction.