Benchmark Framework for Risk Management

0

NOTE: THIS IS A PRELIMINARY VERSION PRINTED FOR LIMITED DISTRIBUTION FOR THE PURPOSE OF PROMOTING DISCUSSION AND

SOLICITING COMMENTS.

PORTIONS OF THIS PAPER HAVE BEEN PUBLISHED IN THE JOURNAL OF TOXICOLOGY AND ENVIRONMENTAL HEALTH Part B Vol 6, No. 6.

Benchmark Framework for Risk Management

Contributed by

J.H. Shortreed, L. Craig and S. McColl

Version of August 2000

Network for Environmental Risk Assessment and Management

0

Table of Contents

Acknowledgements

Preface... i

Figure A: Risk Management System for an Organization... vii

Figure B: Risk Assessment and Treatment Options ... viii

Figure C: Risks Estimation ... ix

Figure D: Steps in the Q850 Risk Management Decision-Making Process ...x

1. INTRODUCTION...1

1.1 The Nature of Risk Decisions ...1

1.2 What is Risk Management?...1

1.3 Basis for Development of a Benchmark Framework ...2

1.4 Organizational Dimensions of Risk Management Decision-Making...4

2. PROPOSED BENCHMARK FRAMEWORK ...6

2.1 Overview ...6

2.2 Elements of the Benchmark Framework: Functions ...10

2.3 Elements of the Benchmark Framework: Criteria...14

2.4 Elements of the Benchmark Framework: Capacity Requirements ...17

3. VALIDATION OF PROPOSED FRAMEWORK FUNCTIONS ...18

3.1 Comparison of Elements in the Frameworks ...18

4. REFERENCES...22 Appendix A: Excerpts from ISO Draft of Proposed Risk Management Definitions

Final Version Expected early 2002

Figure 1: Reflect relationship between terms, based on their

definitions regarding Risks... A-7 Figure 2: Reflect relationship between terms, based on their

definitions regarding Risk Management ... A-7 Figure 3: Reflect relationship between terms, based on their

definitions regarding Stakeholder ... A-7

1

Acknowledgements

We wish to thank the following for providing review comments on an earlier draft of this chapter: Chris Furgal of the Public Health Research Unit, Quebec Centre for Public Health; Kevin Knight, chair of the ISO working group on risk management terminology;

Felix Kloman, editor of Risk Management Reports; and Vince Gagner of the Canadian Federal Provincial Offshore Oil Safety Board.

i

PREFACE

Concept of the NERAM/IRR Benchmark Framework John Shortreed

September, 2001

A framework is defined as “an open structure that gives shape and support to something”; in this case the something is a systematic, efficient, and useful basis for risk management in an

organization. A framework is both a structure to guide the reduction of risks but also an expression of the best practice in a particular area.

The risk management Benchmark Framework was developed by NERAM (Network for

Environmental Risk Assessment and Management). It is a generic framework that was developed as a result of practical applications of risk assessment and management by NERAM and the Institute for Risk Research over the last 20 years, in a variety of fields, including; transport, blood systems, waste disposal, health, climate change, ISO and CSA risk “standards”, and marine navigation.

Over time, frameworks for risk management have evolved and improved as more and more applications of risk assessment and the risk based approach have been implemented. This

evolution involves the addition of essential functions to the framework. For example, the addition of risk communication with Stakeholders was an essential requirement that developed in the 1990s as a part of the CSA standard Q850¹.

In this historical context, it would be expected that the core element of any framework would also be the oldest element in the framework and that this core element would be found in all

frameworks. The core element of the Benchmark Framework for the health field for example, is the scientific estimation of the risks using the science of epidemiology, toxicology, cell studies, statistics, and so forth. This element was first developed about the 1920s for the issue of lead in gasoline and is present in virtually all of today’s health risk frameworks.

There has not been a consistent set of definitions for risk. However, the International Standards Organization, ISO, has recently developed and approved a standard set of risk terminology, which is a generic set of terminology for use by standard writers². This set of definitions is

comprehensive, mutually exclusive, expressed largely in dictionary definitions, and broadly consistent with existing terminology in risk management. For these reasons, this set of definitions was selected for the NERAM/IRR Benchmark Framework.

The Benchmark Framework is intended to be a middle-of-the-road, centralist formulation of the existing frameworks for risk management expressed in terms of the ISO terminology. The extent to which this is so can be judged by reviewing the mapping of existing frameworks onto the function and definitions of the Benchmark frameworks and observing the degree of agreement.

This was done in both an informal and subjective way, for over 80 existing frameworks, as the Benchmark framework was developed and modified.

1 Canadian Standards Association. 1997. Risk Management: Guideline for Decision-Makers (CAN/CSA- Q850-97). Canadian Standards Association. Rexdale, Ontario.

2 International Organization for Standardization (ISO) 2000. The Third Working Draft on Risk Management Terminology. ISO/TMB WG on Risk Management Terminology N33.

ii

The Benchmark framework is designed to easily fit into an organizational chart and it is innovative in that it explicitly differentiates between; functions, criteria, and capacity

requirements. This preface gives the overall concept of the Benchmark Framework while the main text provides the details of the framework.

The Benchmark framework is designed to assist decision-makers in making better decisions. It should be a help rather than a hindrance. In the sense that the ISO definitions are “generic”, it is expected that the Benchmark framework would be modified to better suit individual decision situations, but at the same time, in an ideal world, retaining the general conceptual understanding associated with a standard framework as well as the overarching definitions.

Figures A, B, and C present three hierarchical conceptual levels of the major elements of the Benchmark Framework, given elsewhere. These conceptual representations and the associated ISO terminology will be described, starting with the highest level of generalization which is given in Figure A.

A High Level Concept of the Benchmark Framework (Figure A)

Figure A illustrates the four main elements in a generic RISK MANAGEMENT SYSTEM, defined by ISO as “elements of an organization’s management systems concerned with managing risk”. The following four elements are designed to reduce risk and serve stakeholders in an effective, efficient and transparent way:

1. DECISION-MAKING

2. RISK ASSESSMENT and TREATMENT OPTIONS 3. OPERATIONS TO REDUCE RISK

4. STAKEHOLDERS

DECISION-MAKING is the process in the organization used to “accept” the risk, where risk acceptance is defined by ISO as “decision to accept a risk”. A risk is accepted, as controlled, by various TREATMENT OPTIONS or RISK CONTROL OPTIONS that are implemented as ACTIONS in the OPERATING SYSTEM. The ACTIONS taken in order for the risk to be accepted is the essence of risk management – expenditures to reduce risk to an acceptable level.

Risk CONTROL is defined by ISO as “actions implementing risk management”. Risk TREATMENT is defined by ISO as “process of selection and implementation of measures to modify risk”.

DECISION-MAKING functions in two ways: first the decision-makers can establish a set of CRITERIA contained in the CONTEXT for the management of the specific risk under consideration; secondly the decision-makers can review the EVALUATION of the risk, then factor in the STAKEHOLDER CONCERNS (these include possible factors not already included in the EVALUATION) and make a one-off decision. Clearly the CRITERIA can be applied by technical and administrative staff who are delegated decision-makers, within the limits of the CRITERIA.

Risk CRITERIA are defined by ISO as “terms of reference by which the significance of risk is assessed (Note Risk criteria can include associated costs and benefits, legal and statutory

requirements, socio-economic and environmental aspects, the concerns of stakeholders, priorities and other inputs to the assessment)”.

iii

CONTEXT is described in the AS/NZS risk management standard (1999)³ as “Establish the strategic, organizational and risk management context in which the rest of the process will take place. Criteria against which risk will be evaluated should be established and the structure of the analysis defined.”

STAKEHOLDER is defined by ISO as “any individual, group or organization that may affect, be affected by, or perceive itself to be affected by the risk (Note The decision-maker is also a stakeholder)”. The ENGAGING of STAKEHOLDERS includes RISK COMMUNICATION, defined by ISO as “exchange or sharing of information about risk between the decision-maker and other stakeholders (Note The information can relate to the existence, nature, form, probability, severity, acceptability, treatment or other aspects of risk)”. ENGAGING STAKEHOLDERS may also involve the forming of partnerships, establishing working relationships between the regulator and the regulated, and other relationships.

OPERATIONS TO REDUCE RISK shown in Figure A represent all activities by the organization to actually reduce the risk. These might include; line activities to screen for

parasites, regulatory activities, enforcement, training, establishment of standards, establishment of goals, audits of actual performance, and education. It is important to distinguish between an organization's OPERATIONS and their Policy Development and Analysis activities of RISK ASSESSMENT and TREATMENT OPTIONS. The latter are described in Figure B.

Figure A – the conceptual description of the Benchmark framework does not cover a number of important activities and functions. These are listed on Figure A. For example, the RISK

MANAGEMENT SYSTEM must itself be “managed”, there must be a human resources function, there must be training of staff to acceptable levels of competence, there must be indices to

measure the performance of the organizations activities, there must be a budget allocation and establishment of priorities between OPERATIONS and RISK ASSESSMENT, and so forth.

Finally, ISO defines risk management as “coordinated activities to direct and control an

organization with regard to risk (Note Risk management typically includes risk assessment, risk treatment, risk acceptance, and risk communication)”. Traditionally, in some fields, risk

management has been restricted to the task of identifying treatment options, decisions to accept and implement one of them; rather than the overall process which assumes that before actions are taken there must be scientific analysis, risk estimation, etc. Fortunately this difference can be avoided by not using risk management as a term in the Benchmark Framework and by using TREATMENT OPTIONS to represent the traditional risk management activity.

Middle Level Processes for Risk Assessment and Treatment Options (Figure B)

Figure B provides detail for the RISK ASSESSMENT and TREATMENT OPTIONS element of the overall concept of Figure A. This is the only one of the four basic elements that is expanded in this way as it is central to the main thrust of the Benchmark Risk – the planning, analysis, and evaluation of methods, policies, and activities to reduce risk.

Figure B continues the RISK COMMUNICATION activities of consulting and communicating with STAKEHOLDERS. This is delegated by decision-makers to this technical and

administrative activity. This element can also be expanded to deal with issues of transparency, timeliness, consistency, and so forth.

3 Standards Australia/Standards New Zealand. 1999. Risk Management. Australian/New Zealand Standard.

AS/NZS 4360:1999.

iv

RISK ASSESSMENT is defined by ISO as “overall process of risk analysis and risk evaluation”.

Risk analysis is defined by ISO as “systematic use of information to identify sources and to estimate the risk. Risk EVALUATION is defined by ISO as “process of comparing the estimated risk against given risk criteria to determine the significance of the risk”. RISK ESTIMATION is defined by ISO as “process used to assign values to the probability and consequences of a risk”.

In the Benchmark Framework, unlike the ISO definition, RISK ESTIMATION is limited to risk.

The other items estimated in the ISO risk estimation definition (costs, benefits and the concerns of stakeholders) are estimated as a part of the EVALUATION process in the Benchmark Framework; this change was made to be consistent with traditional usage in many fields.

In Figure B there are three main processes: RISK ESTIMATION to estimate the magnitude of the risk (probability and consequences); EVALUATION to compare the estimated risk against the CRITERIA including costs, benefits, stakeholder concerns and other non-risk criteria; and TREATMENT OPTIONS that are developed to reduce the risk to an acceptable level.

EVALUATION is applied to the estimated risk with existing control options or to the risk estimate with additional TREATMENT OPTIONS. In addition to the EVALUATION using the CRITERIA in the CONTEXT from the DECISION-MAKING it may be necessary to go directly to the DECISION-MAKING activity to seek guidance.

There is a PRELIMINARY ANALYSIS of the risk under investigation, prior to consideration of treatment, that involves a screening level application of: RISK ESTIMATION that identifies the risk (often called Hazard Identification, in ISO there is both a source identification and a risk identification defined, the first deals with the Hazard and the second with the risk) and in a preliminary way estimates the probability and consequence of a risk; followed by an initial EVALUATION of the risk; and consideration of the availability of options to treat the risk. With this preliminary information decisions can be made to identify PRIORITY RISKS that require a more in depth examination.

Implied in Figure B is that a back and forth iteration will likely be necessary between PRELIMINARY ANALYSIS and the establishment of the CONTEXT for the risk. Until

something is known about a risk it is difficult to establish the appropriate CRITERIA, to estimate how much RISK COMMUNICATION will be needed, and to estimate the resources required for RISK ASSESSMENT and TREATMENT OPTIONS.

Figure B indicates only two stages or tiers in the analysis of a risk; PRELIMINARY ANALYSIS, and the full-blown RISK ASSESSMENT and TREATMENT OPTIONS process. For some typical risk management activities, such as environmental site remediation, there may be three or more tiers of ever more detailed analysis; for example, a preliminary analysis, then if required an analysis using standard methods, and finally, if warranted, a detailed site specific analysis. There are an infinite number of variations on this theme – in general the decision-maker can refer back to the assessment/option process as many times as necessary to obtain sufficient information that they can make a decision.

v Details of Risk Estimation (Figure C)

Figure C provides a third level of detail for the Benchmark framework for RISK ESTIMATION, historically the oldest process in the framework, the most consistent across existing frameworks, and the best developed in terms of “standard” methods, techniques, parameters, etc. For these reasons only this element of Figure B is developed here, but it is noted that more detailed concepts are possible for EVALUATION, TREATMENT OPTIONS, PRELIMINARY ANALYSIS, and RISK COMMUNICATION.

Figure C is a reproduction of the US “RAGS” framework for the risk assessment of a

contaminated site⁴ (essentially the same as used in Ontario, Alberta and elsewhere in Canada).

For consistency with ISO the terms “probability” and “consequence” have been added as

alternative definitions of EXPOSURE and EFFECTS. ISO defines probability as “extent to which an event is likely to occur”; consequence as “an outcome of an event”; and event as “occurrence of a particular set of circumstances”. The correspondence between the two sets of terms is exact.

The RISK ESTIMATION process in Figure C first identifies the end points to be estimated, the estimation model to be used and the estimation (analysis) plan. Then the EXPOSURE

(probability) and EFFECTS (consequences) are estimated separately from the collected data and expressed as a profile, usually making uncertainty and differential impacts explicit. Finally the EXPOSURE and EFFECTS are combined to have a complete estimate of the risk and this estimate is “translated” into plain language for the decision-maker, a form that is also useful for RISK COMMUNICATION.

The final estimate of risk is passed on to EVALUATION and DECISION-MAKING; then there may be some TREATMENT OPTIONS generated and the “as treated” risk would come back for a subsequent estimation to determine the effectiveness of the proposed treatments. This in effect connects back to the process described in Figure B.

Before leaving the conceptual description and definition of the Benchmark framework it is of interest to consider the generic ISO definition of risk – “combination of the probability of an event and its consequence”. It has the usual three elements: an event; a consequence; and a probability.

Application of the Benchmark framework Concept

Having described and defined the concept of the Benchmark framework in Figures A, B, and C, it can now be used to examine the frameworks in use in a particular organization.

The objectives of this examination are often the following:

1. To identify the level of detail in the frameworks. Often there is confusion if comparing a high level framework with a middle level or detailed level.

2. To examine the flow of activities in a framework as compared to the Benchmark framework, and

3. To be able to identify what elements of the Benchmark framework are not present in an existing framework.

4 U.S. Environmental Protection Agency, 1999. Final Guidance: Ecological Risk Assessment and Risk Management Principles for Superfund Sites. OSWER Directive 9285.7-28P.

vi

The Canadian Q850 Standard

Figure D is the 1997 Canadian Risk Management Standard, Q850. The slight differences in definition post ISO Guide 73 (2002) are shown. It is consistent with the NERAM/IRR

Benchmark Risk Management Framework but is more limited in scope and detail. Adoption of Q850 would be enhanced by the application of the Benchmark Framework. It is expected that Q850 will be revised to conform to ISO Guide 73 and to recognize the rapid developments associated with the “Integrated Risk Management” and “Enterprise Wide Risk Management”

concepts.

vii

Figure A: Risk Management System for an Organization [Higher Level Framework] (One interpretation of the benchmark framework) (also known as Enterprise Wide Risk Management)

ENGAGE

CONCERNS

DECISION-

MAKING STAKEHOLDERS

CONTEXT

RISK CONTROL OPTIONS

ACTIONS

OPERATIONS TO REDUCE

RISK RISK

ASSESSMENT AND TREATMENT

OPTIONS (see Figure 2)

Not Shown are:

1. Functional Tasks, Processes, Roles, and Responsibilities.

2. Criteria for “Risk” Decisions 3. Capacity Requirements

4. “System” Performance Indicators, and other methods and techniques for managing risk management activities in an organization.

MONITOR

viii

Figure B: Risk Assessment and Treatment Options [Middle Level Process] (Detail of Figure 1)

CONTEXT

• Criteria

• Scope, budget

• etc.

PRELIMINARY ANALYSIS

RISK

ESTIMATION EVALUATION OPTIONS

? PRIORITY

RISKS

RISK ESTIMATION (see Figure C)

EVALUATION

TREATMENT OPTIONS

STAKEHOLDERS

RISK COMMUNICATION

NOTES:

1. Evaluation based on decision Context (Criteria) and other links to decision-maker for decisions.

2. Not shown are analysis issues and principles, including:

a. Explicit treatment of uncertainty b. Synthesis of “good” options.

c. Commonly accepted criteria (e.g. do more good than harm)

3. Evaluation may include models of cost-benefit, environmental impacts, risk finances, etc. RISK ESTIMATION may be exclusive to the RISK.

DECISIONS

ix

Figure C: Risks Estimation [Detailed Level Process/Techniques, one of many] (Example taken from US RAGS)

FORMULATION / IDENTIFICATION

Endpoints

Conceptual Model

Analysis Plan

analysis

profile profile

EXPOSURE

(PROBABILITY ANALYSIS)

EFFECTS

(CONSEQUENCE ANALYSIS)

RISK ESTIMATE & CHARACTERIZATION

Risk estimate

Risk Description for Decision-makers

analysis

data/models data/models

x

Figure D: Steps in the Q850 Risk Management Decision-Making Process – Detailed Model

• Develop an implementation plan.

• Implement chosen control , financing, and communication strategies.

• Evaluate effectiveness of risk management decision process.

• Establish a monitoring process, sunset, terminate as applicable.

Action/

Monitoring

• Identify feasible risk control options.

• Evaluate risk control options in terms of effectiveness, cost, and risks.

• Assess stakeholder acceptance of proposed action(s).

• Evaluate options for dealing with residual risk.

• Assess stakeholder acceptance of residual risk.

Risk Control

• Define scope of decision(s).

• Identify hazards using risk scenarios.

• Begin Stakeholder Analysis.

• Start the risk information library.

Preliminary Analysis

Risk Estimation

• Define methodology for estimating frequency and consequences.

• Estimate frequency of risk scenarios.

• Estimate consequences of risk scenarios.

• Refine Stakeholder Analysis through dialogue.

Initiation

• Define problem or opportunity and associated risk issue(s).

• Identify risk management team.

• Assign responsibility, authority, and resources.

• Identify potential stakeholders and begin to develop consultation process.

Risk Evaluation

• Estimate and integrate benefits and costs.

• Assess stakeholder acceptance of risk.

End Go Back

Take action Next step and/or take action Next step and/or take action Next step and/or take action

End Go Back End Go Back End Go Back

Source: CSA, July 1997. Risk Management: Guideline for Decision-Makers (CAN/CSA-Q850-97) Canadian Standards Association.

ISO “RISK TREATMENT”

ISO “RISK CRITERIA” (part of)

1

1 INTRODUCTION

1.1 The Nature of Risk Decisions

Risk is the chance of something happening that will have an impact upon objectives.

For public health organizations and other safety-related organizations the primary objective of the organization is to promote safety or reduce risk. In other organizations, there may be additional objectives of financial success, maintaining public trust,

supporting the arts, and so forth. In any organization there are both external and internal risks to manage. External risks refer to risk issues that the organization has a mandate to manage, for example to protect human health from environmental or occupational health risks. Internal risks threaten the ability of the organization to meet its mandate, for example due to insufficient capability within the organization or lack of public trust or credibility. While external risks are of primary concern to stakeholders, both types of risk are important to the organization.

Risk decisions are typically characterized by:

• high levels of uncertainty

• many unknowns

• major impacts such as death and catastrophic outcomes for society

• distributional impacts of risks, costs and benefits

• emotional involvement of stakeholders

• low probability of events

• high political content

• low willingness to spend now so nothing happens in the future

For these reasons, the management of risk requires the use of special “risk management”

techniques, tools, methods, and extensive communication and consultation with stakeholders and other interested parties.

1.2 What is Risk Management?

Risk management is a comprehensive, systematic process that assists decision-makers in identifying, analyzing, evaluating and treating all types of risks, both internal and

external to the organization. The objective of risk management is to ensure that

significant risks are identified and that appropriate action is taken to manage these risks to the extent that is reasonably achievable. Appropriate actions are determined based on a balance between: risk treatment (or control) strategies; their effectiveness and cost; and the needs, issues and concerns of stakeholders.

Risk management can also be thought of as managing the “unacceptable variation from the expected” of both positive and negative consequences of activities.

2

1.2.1 Guiding Principles for Risk Management

A number of guiding principles for effective risk management form the underlying basis for existing and emerging risk management frameworks in Canada, the U.S. and

Australia/New Zealand. These principles include the following:

• The decision process is documented and therefore open and transparent;

• The key role of governing board oversight and senior management commitment;

• Explicit consideration of stakeholders’ perceptions of the risk and risk management options through early and ongoing involvement in the decision process;

• Risks are considered in a comprehensive context, considering other public health or environmental health objectives. The organization should have a mandate to direct actions and resources where they will be most effective;

• A balancing of the costs of managing the risks, the benefits to be gained, and the level of risk management that is reasonable to apply;

• A standard set of terminology is used to describe the risk issues, thus contributing to more effective communication about risk issues;

• There is explicit treatment of uncertainty;

• The process is flexible and iterative to allow risk managers and stakeholders to revisit earlier stages of the process and to revise earlier deliberations and decisions in light of new information, ideas and perspectives.

1.3 Basis for Development of a Benchmark Framework

In Section 2.0 a benchmark framework for risk management is described. The framework is proposed as a comprehensive risk management process that can be applied and adapted to suit any organization responsible for managing environmental, human health and occupational health risks. The benchmark framework was developed to meet the following criteria:

1. The benchmark framework should reflect current concepts and underlying principles of environmental and health risk management. Examples include:

⇒ The US Presidential /Congressional Commission Risk Management Framework (1997)

⇒ Health Canada Decision-Making Framework for Identifying, Assessing, and Managing Health Risks (draft, October 1, 1999)

3

2. The framework should reflect current best practice for risk management. Examples include:

⇒ CSA Risk Management Guideline for Decision Makers (Q850, 1997)

⇒ Australian/New Zealand Risk Management Standard (AS/NZS 4360:1999)

⇒ Conference Board of Canada. A Conceptual Framework for Integrated Risk Management (1997)

⇒ ISO (third working draft of Risk Management Terminology, 2000).

3. The framework should be capable of being easily implemented by individual organizations within existing organizational structures. In addition to a concept and set of principles it must have identifiable functions, check lists of requirements, example case studies, etc.

4. The framework must be easily understood by risk makers, risk takers, stakeholders, and members of the public. Extensive documentation of scope, methods, and decisions is needed.

The US Presidential Congressional Framework (1997) reflects the international state of the art in environmental health risk management. The framework has been extensively reviewed and supported by a number of agencies and organizations, including Health Canada. Health Canada’s revised risk management framework reflects many of the concepts of the Presidential/Congressional Framework. The US framework serves as the initial starting point for development of the benchmark framework, however a number other principles of best practice are also included. For example, The Conference Board of Canada has introduced the concept of “integrated” risk management which is a proactive, comprehensive, organization wide response to internal and external sources of risk. Integrated risk management goes beyond managing hazards and loss to cover all aspects of an organization, from strategy to operations, and all types of risk – operational, legal, reputation, and financial.

There are two interrelated issues related to the value and practicality of a framework. The first is the degree to which an organization should conform to a standard approach and the second is the enhancement of communication of the risk decisions and risk reduction controls attributable to the use of a standard approach. The selected framework should allow for quicker and better communication between the decision-maker and all stakeholders but not at the cost of excessive complexity in the process. To achieve this there must be a careful distinction between those parts of risk management that should be

“standard” and those parts that should be “flexible.”

In general, standardization should be limited to those components of risk management that are similarly applied irrespective of the risk issue. Special and distinct attributes of a particular application should fit into the standard framework but will usually require additional explanatory exposition of the concepts and terms. This additional exposition should benefit from the standard framework concepts.

4

It should be noted that many of the framework functional elements that are introduced in this document are “under construction” as there is not yet a complete consensus on current best practice. However, many individual components in the proposed framework, such as the methods for health risk assessment and many of the scientific criteria for interpretation of cause and effect analysis (e.g. weight of evidence criteria) are well established and internationally accepted. The ideas concerning the ISO defined “risk management system” have been identified conceptually but the practical aspects of an operational framework for different decision environments are still being developed.

1.4 Organizational Dimensions of Risk Management Decision-Making Everyone is a risk manager. Every member of an organization “takes risks” by making decisions. Decisions are about day to day operations, about changes in training, about improvements in quality control, about new programs and products, about corporate mergers and partnerships, and so forth.

There are many levels of risk management decision-making within an organization. The objective of a risk management framework is to provide a realistic representation of the risk management behavior of the organization that is simple enough to be communicated and understood both within and outside of the organization. Before proposing a

“benchmark” framework it is useful to consider some of the dimensions and characteristics of risk management decision-making.

The dimensions considered here are in terms of the functionality of the organization and do not reflect the size of the organization. Clearly an organization with two risk

management employees will be organized differently than one with 200 risk managers.

However, the functionality of the two organizations should be similar.

The characteristics of the risk management functionality of an organization discussed below are applicable at the level of components within an organization that are self- sufficient in terms of mandate, role and responsibility. The complexities of “integrating”

risk management activities for a complex organization is beyond the scope of the proposed benchmark framework. If the framework is applied to a self sufficient, sub- component of a large organization then the integration within the larger organization will require additional functions, criteria and capacities.

Time Horizon – The time horizon will determine if the goal of risk management is mainly concerned with:

1. Strategic risks – long term corporate level decisions on programs, change in governance, etc. For example, the decision to commit resources to implement ISO 9000 to achieve quality control of operations; or a decision to terminate a dangerous, controversial product.

5

2. Tactical risks – medium term decisions, based on policy analysis, such as changes in monitoring methods, revisions to risk controls, and changes in the ISO 9000 Quality Control plan.

3. Operational risks – short term, often day to day, control of risks in production and operations through supervision, correction, retraining of operators, and other quality control methods. The operation of an ISO 9000 certified plan is an example of operational risk management.

Functional level in the organization – The hierarchical level in the organization will usually be explicitly recognized by distinct levels with different management activities as well as different functional activities. For example, many organizations will be separated into the following four levels:

1. Decision-making – makes all decisions in the organization (many decisions will be assigned to managers at other levels in the organization) and interacts with

stakeholders.

2. Policy Analysis – the review of monitoring and supervisory results, the analysis of operations and programs to develop new programs and operations as well as modifications to meet desired criteria.

3. Supervision and Monitoring – The monitoring of operations to identify the degree of achievement of the organization’s “plan”, SOPs, and other determined

performance criteria. Note that this level may be combined with either the Policy Analysis level or the Operational level.

4. Operations – the production and delivery of goods or services to meet the organization’s mandate and goals. For safety organizations (e.g. public health

agencies) these operations will result in reduction of risk through the implementation of regulations or risk treatment and control measures.

There is a flow of information between all levels of an organization. Generally, the flow of information from the top down is related to decisions on programs to be operated and targets for budgets and production. The flow of information upwards in the organization is generally related to output measures of the operations, cost effectiveness of the program, costs to modify the operations, human resources status reporting, etc. Clearly, in any effective organization there is considerable dialogue and exchange of information between all levels. Successful risk management depends largely on the organization’s ability to collect and disseminate information on the external environment and the performance of internal management systems and operations.

It is important that the benchmark framework provide a consistent approach to risk management that can be adopted at the highest level of the organization and implemented at all functional levels of the organization despite the differences in the responsibilities, activities and nature of the risk at each level.

6

2 PROPOSED BENCHMARK FRAMEWORK

2.1 Overview

The proposed benchmark framework for risk management decision-making, is illustrated in Table 1 and Figure 1. Table 1 is organized into three levels within an organization’s structure reflecting the time frame for decisions and the usual functional structure of activities. These levels may also correspond to separate branches or divisions within an organization. The three levels of risk management functions are:

1. Corporate management - where long-term “strategic” decisions are made and responsibility for all decisions at the other two levels lies. At this level the organization’s risks are balanced – taking enough risks to meet the goals and objectives of the organization on the one hand and on the other hand reducing risk inherent in operations to be acceptable and cost-effective.

2. Policy and Program Planning - where the medium term “tactical” policy and program modification analyses are done. It is at this level that the traditional technical “risk management” activities are carried out in an organization.

3. Operations – where the day to day goods and services of the organization are produced and delivered. The risk balance in the organization is realized through incurring risks to meets its mandate (e.g. for promotion of health and protection of the environment) while at the same time reducing these risks to an acceptable and cost-effective level through quality control, quality

assurance and other risk management options.

For each level of management task, Table 1 identifies the Functions that are carried out, the Criteria that are used to measure the performance of the organization and the

Capacity Requirements for the organization to successfully carry out its mandate. The criteria are used to judge how well the functions are carried out, while the capacity requirements provide a set of conditions that the organization must have in place in order to be able to complete the functions.

The Capacity requirements in Table 1 are applicable to all three task levels, but clearly the nature of the capacity will differ with each task level. For example, consider the capability of an organization to effectively communicate and consult about risk management functions:

• at the corporate level this might involve the identification of risk issues through dialogue with key stakeholders and partners; the communication of priorities set by the organization as to which risk issues will be analyzed;

dialogue on the allocation of scarce resources among programs, analysis and capacity building in the organization, expected benefits of activities, etc.

7

• at the policy and program planning level, risk communication will likely be focused on the validity of the data used, the applicability of the models used, the identification of stakeholder benefits and concerns, evaluation of treatment options, likely biases in the analysis, etc.

• at the operations level, risk communication will be focused on

implementation details for new programs, regulations, etc.; the ongoing effort to modify behaviour (e.g. promotion of healthy lifestyles); explanations of how to use products to best advantage, comply with regulations, etc.

Figure 1 illustrates the flow of risk issues through the functional elements of the

framework. Based on Monitoring and Quality Control information, at periodic intervals (or for crisis situations) a large number of risk issues will be identified. The Decision- Maker will select a limited number of risk issues for Preliminary Analysis

(Identification) (the remainder will be set aside for a number of reasons including low risk, no feasible treatment, few benefits, lack of jurisdiction, etc.).

After obtaining further information on risks in the Preliminary Analysis (Identification) the Decision-Maker will establish the Context for the risk issues that have not been set aside. If necessary the Decision-Maker will return to Preliminary Analysis

(Identification) for additional information and this iterative cycle may be repeated until there is sufficient information to make a decision to commit resources.

Once each risk issue that has been selected for further consideration has an acceptable Context, the issue then proceeds through Risk Analysis, Risk Treatment Options, and Evaluation. The Decision-Maker may set the risk issue aside, go back for more analysis or decide to select a treatment option for Implementation. Following implementation the whole system returns to the ongoing Monitoring and Quality Control, which will generate new risk issues and new opportunities. Stakeholder Relations (e.g. risk communication and public involvement) are ongoing throughout the process, at a level determined by the decision-maker depending on the urgency of the issue and the resources available.

Functions in Figure 1 may be either eliminated or carried out in an ad hoc manner at the discretion of the decision-maker. For example, if following preliminary analysis, a risk issue is found to be a well-defined problem with a standard solution (e.g. ban an activity if regulatory limits are exceeded; or take advantage of a “golden opportunity” if all risks are manageable and acceptable) then the decision-maker may decide to proceed directly to implement the chosen option with all other functions limited to risk communication.

Similarly, if an unexpected crisis situation strains the resources of an organization the decision-maker may delay or cancel initiatives addressing a number of lower priority risk issues.

8

Table 1 : Proposed Benchmark Framework for Risk Management Decision- Making ^1,2

MANAGEMENT TASK

FUNCTIONS³ CRITERIA CAPACITY REQUIREMENTS

CORPORATE MANAGEMENT

“STRATEGIC”

• Decision-making

• Monitoring

• Stakeholder Relations

• Context

• Corporate Objectives

• Capacity

• Trust of Stakeholders

• Transparency

• Flexible-Consistency

• Budget

POLICY &

PROGRAM PLANNING

“TACTICAL”

• Preliminary Analysis (Identification)

• Risk Analysis

• Risk Treatment Options

• Evaluate Risk and Risk Treatments

• Cost-Effective

• Stakeholder Acceptance

• Uncertainty Explicit

• Reasonable Relationship

• Precautionary Principle

• Comprehensive OPERATIONS

“OPERATIONAL”

• Implement

• Quality Control

• Programs and Products to Reduce Risk

• Achieve Operational Plan

• Correct Failures

• Continuous Improvement

• Customer Satisfaction

•

Risk Communication and Consultation

• Documentation

• Best “Practical”

Practise

• Partners

• Staff

NOTES: 1. Figure 1 is also a part of the Framework.

2. See Section 2.2 for details of the components of the framework

3. The flow of risk issues through the functions of the decision-making process are illustrated in Figure 1.

Each capacity applies to all three function levels

9

Figure 1: Flow of Risk Issues Through Framework

Monitoring Quality Control

(INITIAL RISK ISSUES)

(most issues) (END)

STAKEHOLDER RELATIONS {Risk Communication, etc.}

(Priority RISK ISSUES) D-M

D-M

D-M

D-M D-M

D-M

Preliminary Analysis (Identification)

Context

Risk Analysis ITERATIVE

LOOP *

Risk Treatment Options

Evaluate Risk and Risk Treatments

Implement Programs or Products

Monitor Quality Control

Legend

Function from Table 1

D-M Decision Making Function from Table 1.

* Process is iterative at all stages.

(END of Risk Issue) (WATCHING BRIEF)

(END)

10

For any existing risk management activity or concept there should be only one location in the Framework where it would be assigned. In the following sections the details of the Functions, Criteria, and Capacity Requirements are defined and their concepts are developed.

2.2 Elements of the Benchmark Framework: Functions

The Functions in the benchmark framework are defined below in the order they appear in Table 1. In each case the subtasks associated with each function are identified, followed by a

discussion of the concepts.

2.2.1 Corporate Management or “Strategic” Functions Decision-Making

• Budget allocation

• Performance auditing

• Maintain programs

• Maintain capacities

• Priority setting

• Assign roles and responsibility for risk issue

• Select treatment options and strategy

The Decision-Making function of the Corporate Management task focuses on determining which risks and risk issues are a priority for expending time and resources in the analysis, planning and design of risk treatments. In public organizations, the decision-making function is shared

between a legislature and civil servants. The decision-making function is illustrated in Figure 1 as a decision diamond (D-M) between each step of the risk management process.

The Decision-Making function is always associated with the decision to undertake (or cancel) an activity. The activity has associated and inextricably integrated risks, costs, benefits, and

opportunities. The activity may also be mandated by legislation or conventions. A decision about an activity generally involves “risk management” if the associated risks are significant and greater than some threshold level.

Monitoring

• Outlook analysis

• Surveillance

• Performance measuring

Monitoring includes evaluation of the organization’s performance in meeting its objectives, including the implementation of programs and activities and their risks and benefits. Outlook analysis can identify possible future changes in the regulatory environment, the exposure, the hazard, risk acceptability, stakeholders or technology. Surveillance at this level focuses on monitoring trends in risks and risk controls in other similar jurisdictions.

11

Stakeholder Relations

• Engage stakeholders

• Seek partners

The organization should have a process in place for identifying, communicating and consulting with stakeholders. Stakeholders can include decision-makers; individuals who are or perceive themselves to be directly affected by a decision or activity, individuals inside the organization, partners in the decision (especially financial), regulators and other government organizations that have authority over activities, politicians, non-government organizations, the media and other interested individuals or groups. The stakeholder consultation process should be continuous and included as an integral part of the risk communication process. The process of seeking partners at the corporate level is related both to increasing opportunities and to risk reduction.

Context

• Review jurisdiction and mandate

• Strategic context

• Assign criteria

• Establish scope of analysis

Effective risk management programs are those that deliver cost effective risk outcomes and reflect the strategic and operational context of the organization. The context includes the

financial, operational, competitive, political (public perceptions/image), social, cultural and legal aspects of the organization’s functions. It is necessary to understand the objectives and mandate of the organization and its capabilities when making decisions about risk. This helps to define the criteria by which risks are evaluated and forms the basis of controls and management options.

The scope and depth of the review of the risk is defined in this step to consider whether the review will entail organization wide issues or be limited to a specific program or process.

2.2.2 Policy and Program Planning or “Tactical” Functions Preliminary Analysis (Identification)

• Hazard identification

• Risk identification

• Begin stakeholder analysis

Preliminary analysis is a screening-level analysis to define the basic dimensions of the risk problem and guide decisions on the need for further analysis of the potential risk. After the preliminary analysis it is often possible to take action and implement a risk management control without further analysis. For example, if it is determined that either a more detailed risk

assessment would not be possible because of a lack of data, or alternatively that the stakeholder reactions will dominate and further risk assessment will not change the ultimate decision (e.g.

vaccination for meningitis). There may be several iterations between the decisions on priority risk issues and the preliminary analysis to provide a more clear and concise definition of the issue. Hazard identification involves identifying known sources of the hazard or initiating events.

This is usually done by reviewing past accidents and losses, developing risk scenarios to identify how a hazard might lead to a risk and by carrying out a preliminary assignment of frequency and

12

consequence to the risk scenario. This is useful for selecting those scenarios to be analyzed further in risk estimation and those scenarios to be set aside. The stakeholder analysis involves hypothesizing and documenting each stakeholder group’s needs, issues and concerns, the current level of knowledge about the risks and other relevant information. This stakeholder information is verified and expanded upon throughout the risk management process.

Risk Analysis

• Risk estimation

• Stakeholder dialogue

• Benefit-Cost analysis

• Socio-Economic analysis

This step seeks to determine the likelihood and consequences of a risk in order to establish the level of risk. Existing information sources need to be accessed and, where necessary, new data sources developed. Some risks will not lend themselves to objective analysis or observation and the cost of collecting all data might be too great considering the benefits to be gained. The objective of risk analysis is to separate the minor acceptable risks from the major risks and to provide data to assist in the evaluation and treatment of risks. There are three categories of methods that can be used to determine the level of risk: qualitative, semi-qualitative and quantitative. A range of analytical techniques can be used to suit the issues being examined.

Techniques range from tapping into the experiences and views of staff and other stakeholders to the use of computer spreadsheet models and simulations. Which techniques should be used will often be determined by the nature of the work; the level of expertise of the staff and the nature of the risks involved. Prior to making any judgement on the acceptability of the risks, the benefits of the activity and any operational costs (other than risk) should also be considered. All benefits and costs (including risks) are evaluated in terms of the needs, issues and concerns of the stakeholders. It is important that more than the obvious, hard, financial benefits and costs are considered. Other indirect benefits and costs to be considered include effects on public anxiety, trust between decision-makers and other stakeholders, employment, quality of life, ecosystem impacts, etc.

Risk Treatment Options

• Generate options

• Risk analysis of options

• Optimization of strategies and options

• Stakeholder acceptance

Risk treatment is the selection and implementation of appropriate control options for reducing the risk. Options for risk treatment may include accepting the risk if the likelihood and

consequence of that risk is consistent with the established criteria; avoiding the risk by deciding either not to proceed with the activity that contains an unacceptable risk, choosing an alternative more acceptable activity, which meets the objectives and goals of the organization, or choosing an alternative less risky method or process within the activity; reducing the likelihood or the consequences of the risk, or both; or transferring the risk, in full or in part, to another party; or retention of either residual risks following completion of risk reduction measures or of those risks which for political, moral or constitutional reasons are required to be retained. A number of

13

factors may be considered when identifying potential treatment options including legislative authority, policies and commitments and how quickly the risk must be addressed. A range of potential options, including both regulatory (direct regulation, self-regulation, issuing of permits or approvals) and non-regulatory options (advisory, voluntary compliance, economic, and technological measures) should be considered unless the nature of the risk issues makes it unwise, unnecessary or impossible to do so. Stakeholders can play an important role in all facets of identifying and analyzing risk treatments.

Evaluate Risk and Risk Treatments Considers the following:

• Risk estimate

• Stakeholder concerns (individual, social, cultural, environmental, etc.)

• Benefit-cost analysis

Alternative strategies for controlling risk are evaluated in terms of their effectiveness in reducing losses, the cost to implement the treatment option(s), and the impacts of control measures on other stakeholder objectives, including the introduction of new risks or other issues. In general, the best risk control options are those that cost the least, effect the greatest reduction in losses, and create the least adverse side-effects. Control options can also introduce beneficial side- effects which should be considered in the evaluation. Stakeholder acceptance of the proposed actions should be assessed through consultation with interested and affected parties to the extent possible and through the use of appropriate methods for the given situation.

2.2.3 Operations or “Operational” Functions Implementation

• Detailed implementation plan

• Procurement

• Commissioning

• Stakeholder dialogue

• Monitor implementation

In this step the selected risk treatment option is planned and implemented, risk communication strategies with stakeholders are carried out and the monitoring program is established. The implementation plan is the basis for carrying out the selected strategy and monitoring and evaluating the results. The plan should document the specific tasks and timeframes for completion; the roles, responsibilities and accountabilities of participants; plans for

communication and engagement of interested and affected parties and the criteria to be used for monitoring and evaluation. The plan should include “show stopping” criteria that will stop the implementation and return to Policy and Program Planning functions to find a solution.

Quality Control

• Supervision of operations

• Training

• Analysis of operational failures

14

• Continuous improvement

Quality control processes including supervision of operating processes, staff training, and systems for tracking operational failures are an important part of maintaining effective implementation of risk management treatment plans. The availability of regular performance information can assist in identifying improvements to the action plan for implementing risk treatments. Possible methods of review include internal and external audit, internal check programs, physical inspections and program evaluations.

Programs and Products

• Register clients

• Track products and services

• Invoice

• Follow up

The establishment of internal systems for tracking clients, products and services provide an infrastructure for maintaining customer satisfaction and public confidence in the risk management capabilities of the organization.

2.3 Elements of the Benchmark Framework: Criteria

It is important that the criteria against which risks are to be evaluated are determined at the outset. Decisions concerning risk acceptability and risk treatment may be based on operational, technical, financial, legal, social, humanitarian or other criteria. The criteria described below are presented in the order listed in Table 1. A conceptual description of the criteria is given first followed by typical components of the criteria. The list of criteria is intended to be

comprehensive and generally mutually exclusive.

Criteria are concepts in the spirit of the words of a recent OECD document:

“not intended to direct, still less to prescribe, a particular ..approach …(but) to help stakeholders see more clearly the range of possibilities and to assist them in decisions which only they can make. For this reason, it was proposed that a prescriptive guideline or a formal “standard” is neither desirable nor realistic, but rather a “template”

Criteria are identified as possible criteria that may be included or not by the decision-maker for particular risk issues or operations, or as corporate wide “ethics.” For example, the criteria for specific risk issues are determined in the Context function.

2.3.1 Possible Criteria for Corporate Management

Corporate Objectives – The strategic corporate objectives, ethics, internal policies, and the interests of stakeholders that are used to guide the development of overall policy and direction.

Corporate objectives would play a key role in determining the Context of a risk issue.

• Mission

• Corporate objectives and ethics

• Program objectives

• Legal mandates

15

• Key stakeholder objectives

Capacity – The existing capacities of the organization (other than financial): for operation of risk treatment controls; for building additional capacity in the organization; or for analysis of risk issues. These capacities are generic in nature and are developed in different ways and at different levels of detail for each of the three functional levels.

• Risk communication and consultation

• Documentation

• Best “practical” practice

• Partners

• Staff

• training, education, and qualifications

• experience

• administrative support

“Practical” in best “practical” practice indicates that organizations have limits in the level of performance that they can be expected to achieve because of limits in staff, limits in budget, or limits in the state of the art. The level of performance typically changes over time and often the minimum level of performance is defined by industry wide best practice “standards.”

Organizations may improve their level of performance by frequent audits, participation in industry task forces, and other learning and development activities.

Trust of Stakeholders – Maintain and increase the trust and respect of stakeholders in the overall organization.

• Acceptance of operations, programs, decisions, and analysis

• Satisfied with risk communication and consultation efforts

• Acceptance of residual risk

Transparency – The decision-making, analysis, and operations are timely, clear, and

understandable to stakeholders. The organization is considered to be “open” and “responsive.”

• Documentation useful and available

• Good risk communication documents

• Committed to a consultative process

Flexible-Consistent – The balance, defined by an organization, between consistency of decision-making on the one hand and flexibility of decision-making on the other. This is a balance between the rights of the decision-makers to be able to make decisions and the rights of the stakeholders to expect consistent approaches to decisions and to be able to influence

decisions.

• Policy is defined and communicated

• Decisions conform to stated policy

Budget – The organization’s activities are compatible with the available budget with a reasonable balance between quality of operations and the comprehensiveness of operations.

• Expenditures controlled to budget

• Extraordinary activities are provided with additional budget