3.3 The Framework…
3.3.3 Decision-making Process
This section presents the stepwise decision-making process shown in Figure 3.3.
The focus in this will be on step 2; alternatives. The starting point of the decision process is a crude description of the project or case being studied.
Framing
Description of Goals and Objectives. The definition of goals and objectives is a key element of the decision-making process. At this stage in the process we are concerned about high-level goals, relating the project activities and the decisions to the goals and objectives of the company or the organisation that the project is a part of.
Definition of the Decision Problem. In this activity focus is on the specific decision problem considered. What is the problem to be addressed and solved, and what are
the results to be obtained by making the right decision? Furthermore, what are the frame conditions, including relevant criteria and requirements for solving the problem? These criteria and requirements may relate to production, HES, functionality, quality etc.
Generation and Evaluation of Alternatives
Generation of Alternatives. This activity means generating a list of alternative solutions to the described decision problem. Sometimes this is quite a simple process, as there are some obvious candidates, for example if the problem concerns whether to implement a measure or not. In other cases there could be many possible solutions to the defined problem and each solution may have a subset of alternatives that need to be evaluated.
The process of generating alternatives is typically arranged as a creative work- shop with relevant personnel attending, leading to a list of alternatives for further evaluation.
Selection of Method for Risk-Informed Decision Assessment. The method used for evaluating alternatives will depend on the type of decision problem in question.
Some decisions are critical and need detailed analysis, whereas others are not so critical and more crude analyses may be sufficient. To adapt to this variation, we have designed a characterisation method to select among the alternative methods for decision-making.
The purpose of introducing this characterisation process is to be able to select the most efficient decision process for the relevant decision problem, using a struc- tured and standardised process that ensures the quality and the documentation of the decision process. The following three main categories of decision problems are introduced:
(1) Standard decision problem:
x This category applies for most decisions.
x The decision problem is characterised by limited expected loss/gain and limited uncertainties.
x The project management team facilitates the decision process without external assistance; there is no need for detailed quantitative analysis.
(2) Advanced decision problem:
x This decision problem is characterised by significant expected loss/gain and significant uncertainties.
x The project management team facilitates the decision process, but there is a need for detailed analysis on selected issues and external expertise will be required.
(3) Complex decision problem:
x This decision problem is characterised by large expected loss/gain and large uncertainties.
x
facilitation and documentation of the decision process.
x Detailed analysis will be required.
The project management team engages experienced people for
A set of features linked to expected consequences and uncertainties is established as a basis for performing the categorisation, see Table 3.1. The ranking categories obviously need to be tailor-made to the specific application. The system works as follows. For a specific decision alternative, we assess the investment costs, and assign an expected value giving a ranking from 1 to 3 depending on the result.
Similarly, we assess the potential profit by implementing the decision alternative.
Expected values provide the starting point for assessing the importance of the problem; however, there may be a number of factors that can cause or lead to out- comes (consequences) that are extreme compared to the expected values, and such factors are reflected by the second dimension, uncertainties.
Table 3.1. Characterisation of expected consequences and uncertainties. An illustrative example
Argument Comment
1 2 3
Expected effect of implementing decision alternative
Investment >10 mill 10< x < 100 > 100 mill Profit/production increase >10 mill 10< x < 100 > 100 mill Production loss >10 mill 10< x < 100 > 100 mill
Project schedule Little Medium Large
Safety (loss of lives) Little Medium Large
Working environment Little Medium Large
Environment Little Medium Large
Reputation Little Medium Large
Total
Uncertainties
Complexity of technology Low Medium High
Complexity of organisation Low Medium High
Availability of information High Medium Low
Time frame Short Medium Long
Vulnerabilities Low Medium High
Total
Ranking
The highest ranking on one of the questions gives the total ranking, ie. a ranking of 3 in one of the questions gives a total ranking of 3 wrt potential.
The highets ranking on one of the questions gives the total ranking, ie. a ranking of 3 in one of the questions gives a total ranking of 3 wrt uncertainty.
The following rules are proposed for the total ranking of the two dimensions:
x Expected consequences: The highest ranking for one of the issues gives the total ranking i.e., a ranking of 3 in one of the questions gives a total ranking of 3, etc..
x Uncertainties: The highest ranking for one of the questions gives the total ranking, i.e., a ranking of 3 in one of the questions gives a total ranking of 3.
In theory we should run through this process for each alternative, and the highest scores determine the overall score for the problem. However, in practice we
normally conduct only one run integrating the alternatives, and using a typical or the most extreme alternative as a basis for the categorisation.
The above characterisations provide a basis for determining an appropriate classification of the problem. High scores on each dimension mean that the problem is classified as complex, whereas low scores result in a standard decision problem. Of course, other factors than this risk assessment may also affect the final selection, such as time constraints and available resources.
Evaluation of Alternatives
In the following we describe the analysis approach for the different evaluation approaches:
x Standard decision problem x Advanced decision problem x Complex decision problem.
Standard Decision Problem. The decision alternatives are evaluated on the basis of checklists, codes and standards. A solution meeting the standards and codes is chosen. In some cases there is a need for ranking alternatives, and then a light version of the advanced decision problem analysis approach may be used.
Advanced Decision Problem. We refer to the complex decision problem. The difference between the two analysis levels is related to the degree of quantification.
All the aspects that are covered by the complex decision problem are also covered by the advanced decision problem. However, for the advanced decision problem, more crude analyses are performed providing results in categories, expressing different levels of risks, costs, etc. A comparison of options is performed summarising the pros and cons of the various alternatives, using some type of matrix showing the degree of performance for each alternative and each attribute.
Complex Decision Problem. For this type of analysis a complete quantitative analysis is required. That means that for all relevant attributes, observables are identified. As before we denote these X = (X1,X2, …). These may represent costs, incomes, NPV, production volumes, number of fatalities due to accidents, etc. By different types of risk and uncertainty analyses, these observables are predicted and uncertainties assessed. Expected values are assigned, but the analyses see beyond the expected values, covering:
1. Specific features of the possible consequences. It is not obvious what quantities Xi should be addressed. Some aspects of the possible conse- quences need special attention.
2. Specific features of the uncertainties. There may be large uncertainties in the underlying phenomena, and experts may have different views on criti- cal aspects and risks.
3. The level of manageability during project execution. To what extent is it possible to control and reduce the uncertainties, and obtain desired outcomes?
For Category 1 we suggest using the structure by Renn and Klinke (2002) and modified by Kristensen and Aven (2004). The scheme consists of eight conse- quence characteristics:
(a) Potential consequences (outcomes) – represented by representative perfor- mance measures (future observable quantities) such as costs, income, pro- duction volumes, deliveries, number of fatalities etc.
(b) Ubiquity – which describes the geographical dispersion of potential damage.
(c) Persistency – which describes the temporal extension of the potential damage.
(d) Delay effect – which describes the time of latency between the initial event and the actual impact of damage. The time of latency could be of physical, chemical or biological nature.
(e) Reversibility – which describes the possibility of restoring the situation to the state before damage occurred.
(f) Violation of equity – which describes the discrepancy between those who enjoy the benefits and those who bear the risk.
(g) Potential of mobilisation – which is to be understood as violation of individual, social and cultural interests and values generating social conflicts and psychological reactions by individuals and groups who feel afflicted by the risk consequences. Mobilisation potential could also differ as a result of perceived inequities in the distribution of risk and benefits.
(h) The difficulty of establishing appropriate (representative) performance measures (observable quantities on a high system level).
For Category 2 various types of uncertainty analyses can be used. The risk ana- lysis is to be seen as an uncertainty analysis of future observable quantities and events. The analysis structures the analysts’ knowledge of the risks and vulnera- bilities and of what the consequences of a hazard could be. Normally, a number of scenarios could develop from a specific hazard. There are uncertainties pre- sent, and these uncertainties need to be assessed and described. To assess the uncertainties about the possible consequences we may adopt a classification system as follows, in addition to using probabilities to express the uncertainties related to the outcomes of the various observables:
(a) Insight into phenomena and systems – which describes the current know- ledge and understanding about the underlying phenomena and the systems being studied.
(b) Complexity of technology – which describes the level of complexity of the technology being used, reflecting for example that new technology will be utilised.
(c) The ability to describe system performance based on its components.
(d) The level of predictability, from changes in input to changes in output.
(e) Experts’ competence – which describes the level of competence of the experts used in relation to for example the best available knowledge.
(f) Experience data – which describes the quality of the data being used in the analysis.
(g) Time frame – which describes the time frame of the project and how it influences the uncertainties.
(h) Vulnerability of system – which describes the vulnerability of the system to, for example, weather conditions and human error. A more robust technical system is more likely to withstand strains and this can reduce the likelihood of negative consequences.
(i) Flexibility – which describes the flexibility of the project and how this affects uncertainty, e.g., a high degree of flexibility allows adjustments to the project plan as more information becomes available and this can reduce the potential for negative outcomes.
(j) Level of detail – which describes the need for more detailed analysis to reduce uncertainty about the potential consequences.
The uncertainty aspects (a)–(j) can be assessed qualitatively, and discussed, or assessed by some type of classification and scoring system, describing the analysts’
and the experts’ knowledge and judgements.
For Category 3, the task is to assess the potential for obtaining desirable conse- quences, by proper uncertainty and safety management. By desirable consequences we mean desirable outcomes of the performance measures X, including desirable consequences of features of the consequences such as ubiquity, persistency, delay effect, reversibility etc. Such assessments can be qualitative and address factors that are important for obtaining such outcomes. The scientific disciplines would in many cases indicate areas that are important; unfortunately, though, in many cases establishing the appropriate measures is not straightforward. The reason for this is often a lack of knowledge about the effect of the measures, as well as a dispute among experts on the effects of these measures. An example is the importance of obtaining a good safety culture in a company – we all acknowledge that safety culture is important, but the effect is difficult to measure.
The costs of the uncertainty and safety management also need to be addressed and this leads to the use of some type of cost-benefit or cost-effectiveness analyses, measuring, for example, the expected cost per expected number of saved lives.
Such analyses provide, in many cases, essential decision support, but additional assessments are often needed, for example, addressing manageability characteris- tics such as:
(a) The ability to run processes reducing risks (uncertainties) to a level that is as low as reasonably practicable (ALARP),
(b) The ability to deal with human and organisational factors and obtain a good HES culture.
These aspects can be assessed qualitatively, and discussed, or assessed by some ty- pe of categorisations and scoring system, describing the analysts’ and the experts’
knowledge and judgements.
The above assessments provide decision support by structuring and commu- nicating the analysts’ knowledge to the decision-maker. As part of this support we
may use cost-benefit analyses and other types of analysis, that explicitly reflect the weight the decision-maker gives to the various attributes. However, such analyses still provide decision support – not decisions. There is a need for managerial review and judgement.
Managerial Review and Decision
The extent to which the analyses and evaluations provide clear recommendations on which decision alternative to choose would be case dependent. Purely mecha- nistic procedures for transforming the results of the analyses and evaluations into a decision cannot be justified. The analyses need to be evaluated in the light of the premises, assumptions and limitations of these analyses. The analyses are based on background information that must be reviewed together with the results of the analyses. The analyses and evaluations provide decision support – not a decision.
When evaluating the decision support the decision-maker needs to consider a number of issues, including
x Is the decision-making process managed and documented according to the decision principles and strategies?
x What is the ranking of the alternatives based on the analyses and evaluations? What assumptions are the analyses, evaluations, and ranking based on? What are the limitations of the analyses and evaluations?
x Are there concerns not taken into account in these analyses and evalua- tions?
x Are all relevant stakeholders taken into account? Would different weights of some stakeholders affect the conclusion?
x Robustness in the decision. What is required to change the decision?
It is often a requirement that the decision support should be available and the deci- sion traceable, which means the documentation of which elements have been given weight in the decision. Such a requirement does not apply in general, the point being that the process is in line with the decision principles and strategies defined.
A company may not consider it desirable to trace all weights given to the various attributes.
Implementation of Decisions and Evaluation
How a decision alternative is to be implemented must be considered when evalua- ting the different decision alternatives.
The effect of a decision alternative could change in time, the response of a measure could be different from that expected, concerns may change, etc. The decisions therefore have to be evaluated with hindsight, to see how they performed relative to the challenges and problems they were supposed to meet. From this, experience is gained and modifications in strategies, decision principles, etc. may be formulated.