Qualitative and quantitative evaluation of human error in risk assessment
8.4 Criticality analysis
8.4.1 Developing the task inventory
The first stage of criticality analysis is the development of atask inventory. A task inventory is a high level description of the various human activities in the system of interest. Often, the task of inventory is classified in terms of the main engi- neering areas in the system. For example in a refinery, activities associated with process units such as furnaces, distillation columns and cat crackers, may be the natural way to organise the task inventory. In the railways, tasks might be organ- ised into groups such as train operations, signalling, train maintenance, and trackside activities.
Often there are discrete stages of operation that can be identified such as start-up, routine operation and shutdown. In the marine and aviation environments there are phases of operation such as starting and completing a voyage or flight sector that will have associated human activities. Because of the wide range of systems that HIRAM might be applied to, it is difficult to stipulate rules for organising the task inventory.
However, it must be comprehensive to ensure that no significant activity (in terms of risk potential) is omitted.
Initially, the activities will be considered at quite a high level of detail, e.g. ‘carry out maintenance on system X’. The criticality screening criteria, discussed below, are then applied at this level to eliminate areas that do not require detailed application of human reliability techniques. If the criteria cannot be applied at this coarse level of
detail, it will be necessary to break down the task further into subtasks and re-apply the screening criteria to each of the subtasks. Hierarchical Task Analysis (HTA) is a useful tool for this purpose (as illustrated in Section 8.5.3).
8.4.2 Screening the tasks
Once all of the human activities have been grouped into meaningful categories it is then necessary to apply the screening criteria to each of the human activities (tasks or subtasks). Some of the dimensions that are used to evaluate criticality are as follows:
• Likelihood of the task or subtask failing.
• Severity of consequences (safety, financial, quality, environmental impact, company reputation are common measures).
• Frequency of performance of the task (exposure).
• Likelihood of recovery from the error (recovery means some intervention that occurs to avert the consequences actually occurring).
8.4.2.1 Likelihood of task failure
At first sight it seems paradoxical that an evaluation of the failure likelihood of the tasks is required during the criticality analysis. It can be reasonably argued that the likelihood of the subtask failing is unlikely to be known at this stage. However, since the criticality analysis is primarily directed at rank ordering the tasks in terms of their priorities for analysis only relative likelihoods of failure, rather than absolute probabilities are required. Thus, a scale composed of qualitative statements of like- lihoods of failure, as shown in Table 8.1 would be adequate for this purpose. In the table some credible ranges for the failure frequencies in typical safety critical sys- tems are shown, but these are not strictly necessary for use in the criticality analysis, where a task classification of high, medium and low is usually sufficient. When the analyst is assigning tasks to these categories, a process of comparative judgements will be carried out, which are considerably more reliable than absolute judgements of probability. A similar approach of using high, medium and low assignments can also be used for the other dimensions described above.
Table 8.1 Mapping of qualitative scales on to probabilities
Verbal description Expected frequency of failure
High (H) 1 in 100
Medium (M) 1 in 1000
Low (L) 1 in 10,000
8.4.2.2 Severity of consequences
As discussed above, the severity of consequences measure could focus on safety, finance, quality, environmental impact, company reputation (or even some combi- nation of these factors), depending on the nature of the criticality analysis. In many risk analyses these factors are mapped onto a common dimension, which is usually monetary cost.
The process involves identifying and describing human interactions with the system that will have major impact on risk. A human interaction can in some cases comprise a single operation, e.g. operating a flap or detect- ing a temperature increase. Usually, however, a human interaction will con- sist of a task directed at achieving a particular system objective, for exam- ple, responding correctly in an emergency. Human interactions are obviously not confined to operational situations. They may also be involved in mainte- nance and system changes. Errors in these operations can give rise to latent failures, which are not manifested immediately, but which may produce fail- ures in the future when a particular combination of operational conditions occurs.
8.4.3 Developing a screening index
It is useful to develop a numerical measure of task criticality for screening purposes.
If such a measure is developed for each task in the set identified by the task inventory, tasks can then be rank ordered and placed into bands such as high, medium and low criticality. This provides a means of prioritising which tasks should be analysed first, and which should receive the most analytical resources.
• Approximate Error Likelihood (AEL). Normally, only assessments at the level of high (H), medium (M) or low (L) are possible at the screening stage. If nothing is known about the Performance Influencing Factors likely to affect failures in the situation being assessed, then this factor can be omitted from the screening analysis.
• The potential severity of the consequences if the hazard were released, referred to as the Outcome Severity Index (OSI).
• The likelihood that barriers (hardware or software) or recovery actions (actions taken by people after the accident sequence has commenced which prevent the consequences occurring) are effective (Recovery Likelihood, RL).
• The exposure to the task, e.g. the frequency (F) with which it is performed.
If these factors are evaluated on appropriate scales, an index of task criticality can be generated as follows:
Task Criticality Index (TCI)=AEL×OSI×RL×F
Each task can then be assessed on this basis to produce a ranking of criticality, which is essentially equivalent to a scale of risk potential. Only those tasks above a prede- termined level of the TCI (a function of time and resources) will be subjected to a detailed analysis.