Principles of Intelligence Analysis and Synthesis
5.4 Analysis and Synthesis As a Modeling Process
7. Induction. Finally, as more evidence is accumulated over time, and the ACQM plans and conducts more attacks (some successes, some fail- ures), the evidence shows a more general pattern of behavior of the group—characterized by special forms of financing, a hatred for cer- tain cultural symbols, and special communications behaviors.
8. This generalized pattern is tested against all previous attacks and can be validated to provide a high-level template for future hypothesis testing in the deductive process.
This example illustrates onethreadof many possible flows through the rea- soning processes that analysts apply to iteratively analyze the growing pool of evidence and synthesize feasible hypotheses to be explored. The process also illustrates the validation of templates, created by induction, and their use in the deduction process. Once discovered by induction, these templates may be used for future attack detection by deduction.
process is to sort through and organize data (analyze) and then to assemble (syn- thesize), or marshal related evidence to create a hypothesis—an instantiated model that represents one feasible representation of the intelligence subject (tar- get). The model is used to marshal evidence, evaluate logical argumentation, and provide a tool for explanation of how the available evidence best fits the analyst’s conclusion. The model also serves to help the analyst understand what evidence is missing, what strong evidence supports the model, and where nega- tive evidence might be expected. The terminology we use here can be clarified by the following distinctions:
• A real intelligencetargetis abstracted and represented bymodels.
• A model has descriptive and statedattributesorproperties.
• A particular instance of a model, populated with evidence-derived and conjectured properties, is ahypothesis.
A target may be described by multiple models, each with multiple instances (hypotheses). For example, if our target is the financial condition of a designated company, we might represent the financial condition with a single financial model in the form of a spreadsheet that enumerates many financial attributes.As data is collected, the model is populated with data elements, some reported publicly and others estimated. We might maintain three instances of the model (legitimate company, faltering legitimate company, and illicit front organization), each being a competing explanation (orhypothesis) of the incom- plete evidence. These hypotheses help guide the analyst to identify the data required to refine, affirm, or discard existing hypotheses or to create new hypotheses.
Inherent in this process is the explicit modeling of intelligence targets themselves, as well as multiple hypotheses regarding their description or state behavior. A collaborative intelligence analysis-synthesis process requires such explicit modeling. Tacit mental models in the minds of individual domain experts must be made explicit to be shared for collaborative analysis. Tacit men- tal models, exposed only as rationale for final intelligence judgments, are closed to independent scrutiny, while remaining vulnerable to errors of omission and cognitive biases of the owner. Explicit model representations provide a tool for collaborative construction, marshaling of evidence, decomposition, and critical examination. Mental and explicit modeling are complementary tools of the ana- lyst; judgment must be applied to balance the use of both.
Former U.S. National Intelligence Officer for Warning (1994–1996) Mary McCarthy has emphasized the importance of the explicit modeling to analysis:
Rigorous analysis helps overcome mindset, keeps analysts who are immersed in a mountain of new information from raising the bar on what they would consider an alarming threat situation, and allows their minds to expand other possibilities. Keeping chronologies, maintaining databases and arraying data are not fun or glamorous. These techniques are the heavy lifting of analysis, but this is what analysts are supposed to do [19].
Though not glamorous, modeling provides the rigor that enables deeper (structured) and broader (collaborative) analysis: The model is an abstract repre- sentation that serves two functions:
1. Model as hypothesis. Based on partial data or conjecture alone, a model may be instantiated as a feasible proposition to be assessed, a hypothe- sis. In a homicide investigation, each conjecture for “who did it” is a hypothesis, and the associated model instance is a feasible explanation for “how they did it.” The model provides a framework around which data is assembled, a mechanism for examining feasibility, and a basis for exploring data to confirm or refute the hypothesis. The model is often viewed as an abstract representation of an intelligence target: an organizational structure, a financial flow network, a military unit, a corporation, a trajectory of a submarine, or a computer-aided design (CAD) model of an adversary’s weapon or a competitor’s product.
2. Model as explanation. As evidence (relevant data that fits into the model) is assembled on the general model framework to form a hypothesis, different views of the model provide more robust explana- tions of that hypothesis. Narrative (story), timeline, organization rela- tionships, resources, and other views may be derived from a common model. In a criminal investigation, the explanation seeks to prove the case, without a doubt—a case that is both coherent (all elements of the hypothesis are consistent with the evidence and are noncontradic- tory) and correspondent (all hypothesis expectations are consistent with and not contradicted by evidence from the real world).
The process of implementing data decomposition (analysis) and model construction-examination (synthesis) can be depicted in three process phases or spaces of operation (Figure 5.6):
1. Data space. In this space, data (relevant and irrelevant, certain and ambiguous) are indexed and accumulated. Indexing by time (of collec- tion and arrival), source, content topic, and other factors is performed to allow subsequent search and access across many dimensions.
2. Argumentation space. The data is reviewed; selected elements of poten- tially relevant data (evidence) are correlated, grouped, and assembled into feasible categories of explanations, forming a set (structure) of high-level hypotheses to explain the observed data. This process applies exhaustive searches of the data space, accepting some as rele- vant and discarding others. In this phase, patterns in the data are dis- covered, although all the data in the patterns may not be present; these patterns lead to thecreationof hypotheses even though all the data may not exist. Examination of the data may lead to creation of hypotheses by conjecture, even though no data supports the hypothesis at this point. The hypotheses are examined to determine what data would be required to reinforce or reject each; hypotheses are ranked in terms of likelihood and needed data (to reinforce or refute). The models are tested and various excursions are examined. This space is thecourtin which the case is made for each hypothesis, and they are judged for completeness, sufficiency, and feasibility. This examination can lead to requests for additional data, refinements of the current hypotheses, and creation of new hypotheses.
3. Explanation space. Different “views” of the hypothesis model provide explanations that articulate the hypothesis and relate the supporting evidence. The intelligence report can include a single model and explanation that best fits the data (when data is adequate to assert the
Analysis
Explanation (Abduction)
Argumentation Data
Synthesis Data
Data
Searches
Decomposition Problem
Composition
Timeline Relationship
Map
Narrative Imagery Test, refine
hypotheses
Organize, structure evidence
Analysis
Figure 5.6 The model construction process.
single answer) or alternative competing models, as well as the sup- porting evidence for each and an assessment of the implications of each. Figure 5.6 illustrates several of the views often used: timelines of events, organization-relationship diagrams, annotated maps and imagery, and narrative story lines.
The form of the hypothesis-model is a function of the problem being addressed, and the model can have many views or perspectives of explanation. By a hypothesis or explanation, we can refer to a set of views or models that represent the single hypothesis. As in a criminal investigation, the model of a crime can be viewed from many perspectives as the evidence is fitted to a comprehensive expla- nation (e.g., the timeline of events, the path of the suspect on a map, or the spreadsheet of stolen assets matching evidence found in the suspect’s home).
Figure 5.7 illustrates several of the common forms of models, where each may provide a different perspective on a subject of investigation: an entity, an event, a process, or a target object. Robert Clark has enumerated and explained practical analytic methods to quantify and synthesize descriptive and normative models for a wide range of intelligence applications inIntelligence Analysis: Esti- mation and Prediction[20].Consider the range of analytic modeling activities that are required to answer the diverse questions posed by national and military intelligence consumers:
• What is the gross domestic product of a closed foreign regime? Economic questions regarding the gross domestic product of a closed foreign nation requires the development of a quantitative economic model, with inputs from measurement of crops, industrialized production, import-export activities, and other factors. The model provides a gross domestic product estimate and quantifies contributing factors and uncertainties.
• What is the status of a foreign nation’s weapon development? Questions regarding the status of science and technology programs require a pro- gram schedule (timeline) model to be hypothesized, and milestones on the schedule must be evaluated against observations (e.g., weapons test- ing or facilities construction) [21].
• What is the air order of battle of a foreign nation?Order of battle ques- tions require development of a model (the model is often a spreadsheet) that describes the force structure (organization) and enumerates the size of individual units (personnel and weapons).
• How can a military facility target be functionally destroyed?This targeting question requires the development of a functional model of the facility (e.g., the components that make up a radar installation, its electrical