CHAPTER FIVE
5.2 BOTTOM-UP APPROACHES TO OPERATIONAL RISK MEASUREMENT
It is a matter of judgment, however, which risk indicators are most relevant to the overall operational risk exposure of the firm.16
5.2 BOTTOM-UP APPROACHES
to identify possible behavioral lapses. Data are obtained using incid- ent reports, direct observation and empirical proxies. For example, figure 5.3 shows a process map for a transaction settlement. The transaction is broken into four steps. Then data regarding the number of days needed to complete the step is integrated into the process map to identify potential weak points in the operational cycle.
Scorecards require a great deal of knowledge about the nuts and bolts of each activity. However, the level of detail in the process map is a matter of judgment. If the process map contains too much detail, it may become unwieldy and provide extraneous data, detracting from the main focus of the analysis. Thus, the process map should identify the high risk steps of the operational process that are the focus of man- agerial concern. Then all events and factors that impact each high risk step are identified through interviews with employees and observation.
For example, the high risk steps in the transaction settlement process map shown in figure 5.3 relate to customer interaction and commun- ication. Thus, the process map focuses on the customer-directed steps, i.e., detailing the steps required to get customer confirmation, settle- ment instructions and payment notification. In contrast, the steps required to verify the price and position are not viewed by manage- ment as particularly high in operational risk and thus are summarized in the first box of the process map shown in figure 5.3.
Mapping the procedures is only the first step in the causal network model. Data on the relationship between high risk steps and com- ponent risk factors must be integrated into the process map. In the process map shown in figure 5.3, the major operational risk factor is
Distribution of days to complete
0 70%
1 30%
Distribution of days to complete
2 44%
3 56%
Distribution of days to complete
2 27%
3 73%
Distribution of days to complete
2 3 4
58%
26% 8%
Position
reconciliation Confirm Settlement
instructions
Payment notification
5 8%
Figure 5.3 Process map for a transaction settlement Source: Smithson (2000), p. 58.
assumed to be time to completion. Thus, data on completion times for each stage of the process are collected and input into the process map in figure 5.3. In terms of the number of days required to complete each task, figure 5.3 shows that most of the operational risk is con- tained in the last two steps of the process – settlement instructions and payment notification. However, there may be several different com- ponent risk factors for any particular process. If another operational risk factor were used, say the number of fails and errors at each stage of the process, then the major source of operational risk would be at another point of the process, say the position reconciliation stage.
Another technique used in causal networks is the event tree. The event tree evaluates each risk events’ direct and indirect impacts to determine a sequence of actions that may lead to an undesirable outcome. For example, figure 5.4 shows a generic event tree triggered by some external event. As an example, we can apply the generic event tree to Arthur Andersen’s operational risk in the wake of the external event of Enron’s bankruptcy declaration and the resulting SEC inves- tigation into Enron’s financial reporting. One can argue that Arthur Andersen employees, while detecting the event, failed to correctly interpret its significance for Arthur Andersen’s reputation as Enron’s auditor. In directing employees to shred documents, the staff mis- diagnosed the appropriate response, resulting in a failed outcome.
Event trees are particularly useful when there are long time lags between an event’s occurrence and the ultimate outcome. They help identify chronological dependencies within complex processes. However,
Event occurrence
Event detection
Response identification
Response implementation Outcome
External event occurs
Staff detects event
Staff correctly diagnoses response
Staff implements appropriate response
Success Time progression
Staff implements inappropriate response
Failure
Staff misdiagnoses response Failure
Staff fails to detect event. Failure
Figure 5.4 Generic event tree Source: Marshall (2001), p. 259.
both event trees and process maps are somewhat subjective. Manage- ment has to identify the critical risk factors, break down the process into the appropriate level of detail and apply the correct data proxies.
Moreover, by focusing on individual processes at the microlevel, the analysis omits macrolevel interdependencies that may result from a single failed activity that produces many failed processes. Moreover, there is no analysis of the likelihood of each external risk event.19
5.2.1.2 Connectivity models
Connectivity models are similar to causal networks, but they focus on cause rather than effect. That is, they identify the connections between the components in a process with an emphasis on finding where failure in a critical step may spread throughout the procedure.
Marshall (2001) shows that one technique used in connectivity models is fishbone analysis. Each potential problem in a process map is represented as an arrow. Each problem is then broken down into contributing problems. An example of fishbone analysis for errors in a settlement instruction is shown in figure 5.5. The root cause of
message content Broker error
Incorrect ISIN Invalid
format
Missing address
Invalid format
Error in
Incorrect format
Security error Free-text error
Trade-place error Safekeeping error
Invalid S/K account Incorrect
format S/K field missing
Figure 5.5 Example of a fishbone diagram for errors in a settlement instruction
Source: Marshall (2001), p. 252.
the error message is traced to either a safekeeping error, a broker error, a free-text error, or a security error. Within each of these possible problems, the specific cause of the error is identified.
Another technique used in connectivity models is fault tree ana- lysis. A fault tree integrates an event tree with fishbone analysis in that it links errors to individual steps in the production process.
Management specifies an operational risk event to trigger the ana- lysis. Then errors are identified at each stage of the process. In both fishbone and fault tree analysis, as well as for causal networks, care should be taken to avoid over-disaggregation which will make the analysis unnecessarily complex, thereby losing its focus. Connectivity models suffer from some of the same disadvantages as do causal networks. They are subjective and do not assess probabilities for each risk event. However, when combined with a scorecard to assess sub- jective probabilities, one obtains the fault tree shown in figure 5.6.
This is taken from Marshall’s (2001) example of the analysis of late
Missing trade
System failure
Human error
10% 40% 30%
2%
2%
OR
Booking error
Counterparty error
Product volume
Product complexity
60% 35%
Staff error Late confirmation
40% 5%
5%
Late settlement
OR
Telecom failure
Figure 5.6 Causal structure of late settlement losses Source: Marshall (2001), p. 95.
settlement losses for a financial institution. As shown in figure 5.6, late settlement occurs because of late confirmation (with a 40 percent probability), staff error (5 percent probability) or telecom failure (5 per- cent probability); the remainder of the cause of the late settlement operational risk event is the result of unknown factors (occurring with a 50 percent probability).20However, late confirmations themselves can be the result of several errors: missing trades, system failures, human errors, booking errors, or counterparty errors. Each of these operational risk events is assigned a probability in figure 5.6. Finally, the booking error cause can be the result of product complexity or product volume.
Thus, the fault tree measures the extent of interdependencies across steps that make up complex processes.
5.2.1.3 Reliability models
Reliability models use statistical quality control techniques to control for both the impact and the likelihood of operational risk events. They differ from causal networks and connectivity models in that they focus on the likelihood that a risk event will occur. Reliability models estimate the times between events rather than their frequency (the event failure rate).21This methodology is similar to intensity-based models of credit risk measurement (see chapter 4, section 4.2.2). If p(t) is the prob- ability that a particular operational risk event will occur at time t, then the time between events, denoted λ(t), can be calculated as follows:
λ(t) = . (5.2)
Thus, the reliability of a system is the probability that it will function without failure over a period of time t, which can be expressed as:
R(t) =1 − p(t)dt. (5.3)
External as well as internal data are needed to estimate the reliab- ility function R(t). Thus, the data requirements may be daunting.
Moreover, the model must be estimated separately for LFHS events in contrast to HFLS events. However, by focusing only on frequency
t
0p(t)
t
p(t)dt0
and not on impact, reliability models do not measure the severity of the risk event.
5.2.2 Actuarial approaches
The actuarial approach combines estimation of loss severity and fre- quency in order to construct operational loss distributions. Thus, the actuarial approach is closest to the VaR models discussed in the remainder of this book. There are three actuarial approaches: empir- ical loss distributions, explicit parametric loss distributions and extreme value theory.
5.2.2.1 Empirical loss distributions
Both internal and external data on operational losses are plotted in a histogram in order to draw the empirical loss distribution. External industry-wide data are important so as to include both LFHS and HFLS operational risk events. The relationship shown in figure 5.1 rep- resents an empirical loss distribution. This model assumes that the historical operational loss distribution is a good proxy for the future loss distribution. Gaps in the data can be filled in using Monte Carlo simulation techniques. Empirical loss distribution models do not require the specification of a particular distributional form, thereby avoiding potential errors that impact models that make parametric dis- tributional assumptions. However, they tend to understate tail events and overstate the importance of each firm’s idiosyncratic operational loss history. Moreover, there is still insufficient data available to back- test and validate empirical loss distributions.
5.2.2.2 Parametric loss distributions
Examining the empirical loss distribution in figure 5.1 shows that in certain ranges of the histogram, the model can be fit to a parametric loss distribution such as the exponential, Weibull or the beta distribu- tion. In contrast to the methodology used in market risk measurement,22 parametric operational loss distributions are often obtained using different assumptions of functional form for the frequency of losses and for the severity of operational losses. Typically, the frequency of operational risk events is assumed to follow a Poisson distribution.
The distribution of operational loss severity is assumed to be either