Engineers must be familiar with the risk management process and its application, as the assessment and management of risk are integral components of engineering. In consultation with stakeholders, engineers select analysis techniques, determine input requirements and identify potential hazards to assess their likelihood of occurrence and potential consequences.
All engineering activity involves risk and can have a critical impact on the health and safety of the public and the environment. Engineers make decisions and provide advice that protect both the public and the environment, while preserving organizational and project objectives and reducing the risk of undesirable outcomes.
This Engineers Canada guideline provides an overview of a risk management framework for engineers, and outlines the principles and processes that allow engineers to manage risk and support decision making. It also provides information to engineers on how to exercise due diligence by adhering to and advancing best practices in the assessment and management of risk. Furthermore, it supports engineers in meeting their ethical obligation1 to hold paramount the safety, health and welfare of the public2.
2. Definition of hazard and risk
This section defines the terms hazard and risk (and provides an example of their application) as an introduction to a detailed overview of the risk management process. A glossary of other relevant terms relating to risk management can be found in Appendix A.
Although often used interchangeably, the terms hazard and risk are distinct concepts and must be treated as such. Functional definitions are as follows3:
- Hazard: The potential of a machine, equipment, process, material, or physical factor to cause harm to people, the environment, assets or production.
- Risk: The possibility of injury, loss, or environmental incident created by a hazard. The level of risk is a function of the probability of an unwanted incident and the severity of its consequences.
Risk arises from hazards. As such, comprehensive hazard identification is key to effective risk management. The risk arising from a hazard that has not been identified cannot be managed.
In considering hazards and risk, it is important to recognize that harm or damage can occur in four broad areas:
- natural environment,
- assets (e.g. equipment and property), and
- production (e.g. process loss or business interruption).
These categories inform an integrated approach to risk management that encompasses potential losses. Integrated risk management also encompasses a variety of engineering activities, including analysis of potential hazards throughout the life cycle of a product or process.
There are two aspects to risk—probability (sometimes referred to as likelihood or frequency of occurrence) and severity of consequences. Probability-related risk reduction measures seek to reduce the likelihood of an undesired incident, whereas consequence-related measures try to address the severity of consequences and mitigate the extent of a potential undesired incident. Engineers must analyze hazards relevant to their work for the risk that these hazards pose for causing injuries or death, environmental damage, destruction of property and business interruption.
When driving, ice on a highway is a hazard because it has the potential to cause harm or damage. The risk is the probability of encountering ice and the severity of consequences if that were to occur, including potential injury to the driver or others and damage to property or environment. Engineering design seeks to reduce the severity of the consequences through speed limits, suitable tire requirements, traction control and antilock braking systems.
In this example, risk involves the acceptance of icy conditions and use of the highway while implementing precautions and adhering to regulations, helped by onboard vehicle systems. Driver caution and design safety systems reduce both probability and severity of any consequences.
3. Risk management process
Risk management4 is the process of identifying risks, understanding them, assessing them, and making decisions to mitigate them through effective risk controls. Risk management begins with identifying possible hazards, determining the attendant risk, classifying those risks as acceptable or unacceptable, and managing those risks.
Risk analysis enables risk assessment, which in turn enables risk management5. Engineers analyze risk for probability and consequences, so they can assess risk (with respect to acceptability in comparison with societal and industrial thresholds for acceptance), so they can ultimately manage risk. In the assessment phase of the risk management process, engineers use risk management standards and guidelines. A listing of risk management standards and guidelines can be found in Appendix C, section C.1.
Figure 1: A generic risk management process
Each of the steps set out in Figure 1 will now be explained.
1 Planned review
The objective is to proactively collect data and analyze trends to help fulfill obligations to protect the public, monitor operations and develop new project designs. In addition to regularly collected data on business operations and maintenance activities, other types of data can be gathered including incident investigations, insurance company reviews and regulatory activities (e.g. pressure-vessel inspections, environmental reporting, asset-renewal needs, changes to laws and code updates). This planned review informs hazard identification (step 2) by establishing the physical scope (system boundary) and analytical scope (hazard types) for the risk management exercise.
2 Identification of hazards
Use of qualitative and quantitative risk assessment techniques help engineers demonstrate due diligence. Many qualitative and quantitative tools are available for hazard identification purposes. Some risk scenarios can be effectively managed by a qualitative approach. Other situations require the use of quantitative calculations that attempt to numerically estimate the components of risk. Engineers must understand which methods apply to their situation, how to interpret the results, and recognize their limits. Hazard identification techniques include:
- Event tree,
- Failure mode and effects analysis (FMEA), and
- Fault tree.
Engineers must also act ethically when incorporating the results of their risk assessment and management activities in the products and processes they design and operate. Engineers must consider unique hazard scenarios when establishing physical and analytical scopes. Typical elements included in these scenarios may include:
- domino events,
- security issues, and
- natural hazard triggering technological disasters.
Domino effects occur as a sequence of events in which a primary event or accident, such as a fire or an explosion, triggers a chain of events/accidents with an overall escalation of the consequences of the event6. Information requirements for incorporating domino effects in risk management include7:
- primary accident scenarios,
- escalation mechanisms (e.g. thermal radiation or overpressure),
- escalation and damage probabilities,
- secondary event or accident scenarios, and
- available safety measures.
Management of engineering risks necessitates consideration of domino effects in terms of their causation, prevention and mitigation8.
Security issues: security is a term typically used for measures that mitigate deliberate attempts to cause harm. Security assurance requires a management system approach similar to that described later in this section. In the case of security concerns, the foundation of the management system is a security vulnerability assessment (SVA) to identify weaknesses (e.g. vulnerabilities) from threats across a spectrum of physical and/or cyber security concerns, ranging from vandalism to terrorism.
Natural hazard triggering technological disasters (often referred to as “Natech” events) are incidents where systems fail not as a result of internal pressures on the system (e.g. overfilling or over-pressuring a storage tank9), but as a result of a naturally occurring and typically severe event (e.g. an earthquake, hurricane or flood). While the initiating natural event is beyond an engineer’s control, Natech risks must still be managed, including those related to climate change10.
3 Risk analysis/assessment
Engineers can use a number of tools to assist them with risk analysis and assessment. Regardless of the tool used, it must measure two components:
- Probability of risk: The likelihood of failure of systems, humans, and system elements such as equipment and safety devices. Some data is available generically, although pertinent data is often found in maintenance records, operational logs and incident investigation reports.
- Consequence and severity of risk: A number of methods exist to quantify the consequences of the hazards that may be encountered in engineering practice (e.g. fires [thermal radiation and smoke], explosions [blast wave overpressures], toxic cloud dispersion, toxic exposures, lethality, noise, water pollution and electrical shock). Once the probability and severity of consequences are identified and the risk estimated, engineers conduct a risk assessment and compare outcomes with societal, industrial and regulatory thresholds to determine whether the risk is acceptable or not. When making this decision, engineers must also take into consideration uncertainties in probability data (e.g. event likelihood) and consequence model parameters.
The term black swan signifies a highly improbable and unexpected event that can be explained only once it has happened. The term originates from the assumption that black swans were non-existent because only white swans had been observed in the Western world. Such black swan events have also been labeled in some applications as unknown unknowns. Rather than resigning to the inevitability of such occurrences, engineers must actively search for potential danger signals that could lead to low probability/high consequence events and be aware of the significant learning opportunities that they provide, including for incidents with no identifiable consequences, which are often called near misses. Additional information regarding these types of events is available in Appendix C, section C.4.
4 Is the risk acceptable?
Organizations typically have a risk matrix describing what is a low-level risk (acceptable), medium-level risk (acceptable with certain conditions) and high-level risk (unacceptable). The expectation is that high-level risk will be mitigated to an acceptable level or the project/activity must be abandoned. These matrices clarify the required course of action based on risk acceptability.
Governments establish acceptable risk in the protection of the public. In the highway example, the level of risk associated with icy conditions is a balance between 100% safety (e.g. closure of the highway) and accepting some level of risk that still allows for the movement of people, goods and services during winter conditions. In this example, the Government must consider the road conditions to determine if it is safe to keep the roads open (e.g. is the level of risk acceptable).
As part of their ethical and professional obligations, engineers must hold paramount the safety, health and welfare of the public and the protection of the environment. Engineers must also seek to ensure that their audience understands risk probabilities and severity of potential consequences, even for technical risks.
If the risk is found to be acceptable, then the next step is to manage the residual risk.
5 Manage the residual risk
Once a risk is determined to be acceptable, it must be managed, because the residual (remaining) risk does not go away. Conditions, engineering, and operations must be actively and appropriately monitored on an ongoing basis for concerns, and proactive actions must be taken to correct or mitigate potential problems. This is arguably the most important step in the process and has thus been emphasized in Figure 1. Responsibility has now been taken for assuming the risk and preventing any undesirable incident from occurring. A key engineering tool employed in this stage is a management system appropriate for the risks being managed (e.g. health, occupational safety, process safety, equipment reliability, etc.).
Safety management systems are recognized and accepted worldwide as best-practice methods for managing risk through the continuous improvement scheme known as Plan/Do/Check/Act. They typically consist of 10 to 20 program elements that must be carried out to manage the risks in an acceptable way. For example, the management of process safety hazards (fire, explosion and toxic release) and the ensuing risks can be effectively accomplished when viewed as an integral component of a process risk management framework.
It is also important to note that a safety management system without the accompanying ability and will of the organization to effectively operate the system is of little use. The concept of safety culture and its related features have been well-articulated by both engineers and practitioners in other fields, such as sociology.
Closely aligned with safety culture, conduct of operations and operational discipline involve displaying behaviours and actions within a system of checks and balances that help ensure things are done correctly and consistently. Competency is central to the mission of ensuring that risk is effectively managed. The ethical obligations of professional engineers are also highly relevant in this regard.
6 Can the risk be reduced?
Engineers must seek ways to reduce an unacceptable risk to an acceptable level or abandon the activity. Reducing the risk includes eliminating or addressing underlying hazards. In addition, further controls, management systems and protective features may be used to reduce risk to an acceptable level.
Risk reduction can be accomplished by any of four general means that work together within a hierarchy of controls. In order of greatest to least effectiveness, this hierarchy consists of12:
- inherent safety (or inherently safer design, or ISD),
- passive engineered safety (e.g. safety devices that do not require detection and actuation of moving parts other than as caused by the upset condition), recognizing that the requirement for a fail-safe back-up should be assessed,
- active engineered safety (e.g. safety devices requiring detection and actuation of moving parts), and
- procedural safety (also called administrative controls).
The ISD principles of elimination, minimization, substitution, moderation and simplification are highly effective at reducing risk, especially if considered early in the design life cycle13. The hierarchy of controls offers a barrier-based view of risk management in the form of the Swiss cheese model of accident causation14, which terms is defined in Appendix A.
7 Reduce the risk
If the proposed risk reduction measures are viable, engineers must make necessary changes to configurations, equipment, procedures, hazardous inventories and other risk contributors. Once a change is made, the risk management cycle must be revisited to evaluate possible new hazards and risks. Changes in conditions or engineering processes can create additional problems that can unintentionally (and sometimes unknowingly) lead to increased (operational) risk. A management system approach employing a sound management of change (MOC) protocol is desirable. All changes except replacement-in-kind require MOC treatment.
Engineers must maintain awareness and be vigilant for the reappearance of hazards and risks, or the introduction of new concerns. Even if engineers are not involved in a specific project on an ongoing basis, they must adequately communicate to owners and operators what hazards and risks might be expected to reappear.
The reduction of risk to an acceptable level and the management of residual risk is an ongoing process. A list of several hazard identification tools are provided in Appendix C, section C.3.
If the level of risk is found to be unacceptable, the activity must be discontinued (see step 8).
8 Discontinue the activity
Engineers must recognize where the level of risk is unacceptable. In such cases, the engineers must discontinue the activity or recommend that the activity be discontinued, regardless of personal and professional consequences.
4. Lessons learned on risk management
A review of incidents that have occurred over the years highlight the following key lessons regarding risk management:
- the management of risk should not be viewed as an overly complicated process conducted only by experts or engineers. Hazards should be continuously monitored and all individuals should be aware of risks and implement timely and effective risk mitigation actions. Engineers play an important role in educating and providing information to non-experts.
- engineers must consider life cycle in managing risks.
- as part of an integrated approach to risk management, engineers must consider environmental impact as well as losses to people (including loss of serviceability and enjoyment), assets and production.
- numerous techniques exist to identify hazards related to configurations, equipment, processes and systems. Numerous tools are available to support the effective management of risk – including safety management systems and the hierarchy of controls (inherently safer design, passive engineered safety, active engineered safety and procedural safety).
- engineers must carefully vet transferability of safety measures from one application to another, as the effectiveness of risk reduction measures will vary.
- communication regarding risk should consider the audience.
Risk management is enhanced by considering case histories involving successes as well as those involving failures. Incident investigation, when coupled with a culture of learning from loss-producing events, can be a powerful force in improving risk management.
Significant lessons for all engineers can be found in case histories of extreme risk management failure including those drawn from specific industrial sectors (examples are provided in Appendix B). A few notable examples include the Lac Mégantic and Westray incidents (see discussion below). Following a major industrial accident, focus should be on whether hazards were thoroughly identified, risks were competently assessed and whether management control systems were appropriate.
Lac Mégantic incident
The July 6, 2013 derailment of a freight train carrying crude oil, and the subsequent fires and explosions, at Lac Mégantic, Québec caused enormous personal loss (47 deaths) and idespread property damage. A short time afterward, the following questions were being asked (Creedy, 2013):
- How well did the organizations having control understand the risks they were managing?
- How sound were the systems they had in place to control those risks?
- The Westray coal mine explosion occurred in Plymouth, Nova Scotia on May 9, 1992, killing 26 miners. The incident was the subject of a public inquiry (Richard, 1997) and was highlighted in an article on four major Canadian process-related incidents (Di Menna, 2012). Westray is a glaring example of virtually non-existent risk management practices.
Engineers must be aware of and manage risks. While this guideline provides a general model and information to illustrate core concepts for managing risks, engineers must seek additional information pertaining to their own circumstances and ensure that they consider climate change and keep themselves informed about new developments in their areas of practice. They must also keep themselves apprised of, and abide by, their local industry-specific regulations.
Additional information is available in the appendices of this Guideline, including a glossary of risk management terminology (Appendix A); examples of risk management and key lessons (Appendix B); a listing of risk management resource materials (Appendix C) and a comprehensive set of reference citations (Appendix D).
Appendix A: Glossary
Active engineered safety
Safety measure (device) that requires event detection and actuation of moving parts
Definition of the nature of the hazards to be identified as an initial step in risk management
Black swan event
A so-called unknown unknown; an event thought to be highly improbable and identifiable only after its occurrence
Bowtie (BT) analysis
Graphical technique (qualitative or quantitative) for hazard identification; combination of a fault tree (FT) and an event tree (ET)
Checklist (CL) analysis
Tabular technique (qualitative) for hazard identification; relies on the use of pre-determined questions for checking the state of a process or system
Sequence of a primary incident causing a secondary incident by means of escalation vectors
Event tree (ET) analysis
Graphical technique (qualitative or quantitative) for hazard identification; shows the outcome of actuation of safety measures (success or failure) following an undesired event
Failure modes and effects analysis (FMEA)
Tabular technique (qualitative) for hazard identification; uses a structured approach to identify the causes and consequences of equipment or system failures
Fault tree (FT) analysis
Graphical technique (qualitative or quantitative) for hazard identification; shows the causation factors leading to an undesired event
The potential of a machine, equipment, process, material, or physical factor to cause harm to people, environment, assets, or production (e.g. a chemical or physical condition having the potential to cause loss)
Hazard and operability (HAZOP) study
Tabular technique (qualitative, or semi-quantitative with use of risk matrix) for hazard identification; uses a structured combination of guide words and process parameters to identify the causes and consequences of undesired events
Process of identifying issues of concern (e.g. hazards) by means of a technique appropriate for the circumstances (factors such as life cycle stage and availability of information); a key initial step in the risk management process
Hierarchy of controls
Hierarchical arrangement of safety measures ordered from most to least effective: inherently safer design (ISD), passive engineered safety, active engineered safety and procedural safety
Inherently safer design (ISD)
Approach to risk reduction that attempts to remove or modify hazards at their source without the use of engineered (add-on) devices or human intervention; relies on key principles such as elimination, minimization, substitution, moderation and simplification
Integrated risk management
Approach to risk management that considers all possible loss receptors: people, the natural environment, business assets (equipment and property), and production (e.g. business interruption or loss to process)
Management of change (MOC)
Integral component of a safety management system that attempts to reduce the risk brought about by hazards introduced during equipment, process, and organizational changes
Risk reduction efforts aimed at lessening the severity of consequences arising from an undesired event; also known as protection
Natural hazard triggering technological disasters (Natech) event
Undesired event such as storage tank rupture or building collapse brought about by a naturally occurring and severe disturbance; Natech event initiators include earthquakes, floods, tsunamis and hurricanes
Incident in which no actual loss was experienced
Passive engineered safety
Safety measure (device) that does not require event detection and actuation of moving parts other than as caused by the upset condition
Reminder that risk acceptability and tolerance levels are not the same for all individuals or all societal groups
Definition of the system boundary as an initial step in risk management
Risk reduction efforts aimed at lowering the likelihood of occurrence of an undesired event
Likelihood of occurrence of an undesired event; one of the two components of risk – the other being consequence severity
Safety measure (e.g. safe work procedure) requiring human involvement; also includes administrative controls such as hazard identification and safety management systems
Qualitative risk assessment
Risk assessment done without detailed numerical calculations of probability, consequence severity and risk acceptability criteria
Quantitative risk assessment (QRA)
Risk assessment done using detailed numerical calculations of probability, consequence severity and risk acceptability criteria; semi-quantitative risk assessment involves numbers but not detailed probability and consequence severity modeling – for example, by using a risk matrix
Risk remaining after the overall level of risk has been determined to be acceptable; this reflects the fact that zero risk does not exist
Possibility of injury, loss, or environmental incident created by a hazard; the two components of risk are the probability (likelihood) of an undesired event and the severity of its consequences
Process of estimating the probability and consequence severity of an undesired event
Process of comparing the results of risk analysis against established risk tolerance criteria to determine whether the risk is acceptable
Process of hazard identification, risk analysis, risk assessment, risk reduction and management of residual risk
Graphical representation that considers the components of risk in terms of a number of categories of both likelihood and consequence severity; used during the risk assessment process
Process of lowering risk likelihood and consequence severity by means of appropriate safety measures
Collective safety beliefs and values of an organization and its people; the concept of safety culture is closely aligned with the principles of collective mindfulness and risk awareness
Safety management system (SMS)
System for managing risk which consists of a number of program elements (typically 10 to 20) designed to address various requirements such as asset integrity, training, incident investigation, knowledge enhancement and management of change
Concern brought about by a deliberate act intended to cause harm; differentiated from safety issues which arise from random rather than deliberate actions
Impact of the consequences arising from an undesired event; one of the two components of risk – the other being probability
Swiss cheese model
Graphical representation of accident causation which depicts alignment of failures in the safety barriers between an initiating event and the people or assets to be protected; each hole in a slice of Swiss cheese (safety barrier) represents a barrier failure such that when the holes align, an accident occurs
What-if (WI) analysis
Tabular technique (qualitative) for hazard identification; uses a brainstorming approach to identify the causes and consequences of undesired events (e.g. what could go wrong if an initiating event occurred)
Appendix B: Risk management examples and key lessons
Risk management is an overarching engineering activity that is not restricted to only those industries typically regarded as being highly hazardous. To illustrate this point, this appendix provides several examples of the need for effective risk management in a variety of engineering applications drawn from a wide range of scenarios. A key risk management lesson is given in each case.
The examples and the corresponding discussion in this appendix are not meant to be interpreted as guidance on the practice of engineering and are shared for informational purposes only.
B.1 Industrial odours
The article Something in the Air? by Ahluwalia (2006) states that proper management of industrial odours can improve neighbour relations and minimize environmental impact. The flowchart given for odour management essentially describes a risk management process: Identify odour sources / Quantify/sample odour sources / Analyze odour sources / Dispersion modeling / Impact assessment / Find a solution15.
Key Lesson: An integrated approach to risk management considers environmental impact as well as loss to people (including loss of enjoyment in this example), to assets, and to production.
B.2 Mass transit by railway
The plenary conference presentation by Andrew McCusker, in his position as Operations Director for MTR (Mass Transit Railway) Corporation in Hong Kong, is titled Risk Management – An Essential Strategy for Business Success16. He argues strongly in favour of pragmatic risk management approaches that directly support the business decision-making process.
He also describes a success story that occurred in December 2004 when 10-year old Tilly Smith was credited with saving over 100 lives on a resort beach in Thailand. Tilly recognized the warning signs of an impending tsunami by observing the receding sea. She then warned her parents and the hotel staff who evacuated hotel guests to a safer location. Both the alertness of Tilly and the decisive actions of those who believed her, exemplify what a proactive business risk mindset is about – keen awareness of individuals of the presence of risks, and the timely implementation of effective risk mitigation actions17.
Key Lesson: Learning lessons in the field of risk management can benefit from examination of case histories involving success as well as failure.
B.3 Emergency services for heavy industrial developments
Strathcona County Emergency Services (SCES) in the province of Alberta reviews all heavy industrial projects within its jurisdiction for bylaw and code compliance18. Risk assessment at the development stage and risk management at the occupancy stage are key.
Key Lesson: Life cycle considerations play a key role in the management of engineering risk.
The aptly titled article, What Asbestos Taught Me About Managing Risk, describes what the author calls one of the most colossal corporate blunders of the twentieth century19. The blunder was denial; managers throughout the author’s company are described as being either unwilling or unable to believe that the known hazards of asbestos could pose long-term consequences.
Key Lesson: Risk denial is the antithesis of a safety culture that embodies the principles of risk awareness.
B.5 Mount Polley
The environmental disaster that began on August 4, 2014, with the breach of a tailings pond at the Mount Polley site in British Columbia has similarly been the subject of intense public interest and media attention. In its 2015 report, an independent review panel recommended that future permit applications for a new tailings storage facility should contain a detailed evaluation of all potential failure modes and a management scheme for all residual risk20.
Key Lesson: Numerous techniques exist to identify hazards related to ambient or constructed configurations, equipment, processes and systems; similarly, numerous tools are available to aid in the effective management of risk – including safety management systems and the hierarchy of controls (inherently safer design, passive engineered safety, active engineered safety and procedural safety).
B.6 Hydrocarbon industry
The industry-standard Marsh report21 describes the 100 largest property damage losses in the hydrocarbon industry over the almost 40-year period from 1978 to 2017. The introductory sections of the report cover topics such as the use of risk engineering surveys to evaluate risks, how the insurance market identifies critical risk topics and learning from loss history as a means of improving safety performance.
Key Lesson: The past can be a helpful guide to the future; this is undoubtedly the case in the field of risk management.
B.7 Dust explosions
Explosions of combustible powders in industry are sometimes viewed as incidents that can be adequately addressed by occupational safety measures alone. There is much current evidence, however, that dust explosions are most effectively prevented and mitigated by application of process safety and process risk management principles.22
Key Lesson: Not all risk reduction measures are equal in effectiveness; transferability of safety measures from one application to another requires careful vetting.
B.8 Software and security issues
The need to manage risk arising from cyber security concerns was identified in section III of the guideline. Moreno et al. (2018) give examples from both internal and external perspectives – respectively: (i) An ex-employee of a pharmaceutical firm had secretly installed a piece of software on the company’s server. He later gained access to the server and used the unauthorized program to attack the computer network, leading to operational downtime and financial losses for the company. (ii) Unauthorized computer users disabled communications, shut off alarms and over-pressured the crude oil in a pipeline. The pipeline was destroyed and significant revenue was lost.
Key Lesson: Risk management must be viewed as a critical requirement for both software and hardware system components.
B.9 Grenfell Tower
The June 14, 2017 Grenfell Tower structural fire in London, UK, resulted in 72 fatalities and a similar number of people being injured. In response, the UK Institution of Civil Engineers (ICE) undertook a wide-ranging review aimed at examination of the risks of catastrophic failures in economic infrastructure assets on a general basis.23 Quoting from the report:24 Society rightly expects buildings and infrastructure to be planned, designed, constructed, operated and maintained in such a manner as to present an extremely low risk of failure, and to cause negligible hazard to occupiers, users and the public.
Key Lesson: Competent risk management rests largely on the fundamental professional engineering tenet of holding paramount the safety, health and welfare of the public.
B.10 Life-saving rules
In 2009, Shell Global introduced its 12 life-saving rules to address the most critical safety hazards causing loss of life in the company’s past activities. Their rules25 offer sound advice for the management of risk in any company or organization: (i) Work with a valid work permit when required, (ii) Conduct gas tests when required, (iii) Verify isolation before work begins and use the specified life protecting equipment, (iv) Obtain authorisation before entering a confined space, (v) Obtain authorisation before overriding or disabling safety critical equipment, (vi) Protect yourself against a fall when working at height, (vii) Do not walk under a suspended load, (viii) Do not smoke outside designated smoking areas, (ix) No alcohol or drugs while working or driving, (x) While driving, do not use your phone and do not exceed speed limits, (xi) Wear your seat belt, and (xii) Follow prescribed journey management plan.
Key Lesson: Incident investigation, when coupled with a culture of learning from loss-producing events, can be a powerful force for risk management enhancement.
B.11 Field level risk assessment
It is not uncommon in industry nowadays for employees to undertake field level risk assessments done in a timely, controlled manner. The basic idea is to think through potential issues before starting a job task, or when there has been a period of time away from the task long enough that changes to equipment or ambient conditions might have occurred. An essential component of such assessments is the identification of both hazards and the adequacy of control measures.
Key Lesson: The management of risk should not be viewed as an overly complicated process conducted only by experts.
B.12 Hotel fire safety
One might be tempted to relegate consideration of hazards and risks solely to the technical world of engineering. This is ill-advised, however, given that risk – and hence risk management – are integral parts of daily life. This can be clearly seen in the final four life-saving rules given in section B.10 of this appendix, as well as in a booklet written for BP company employees traveling on business.26 After reviewing some of the common system deficiencies in fire risk management at the hotel corporate and facility levels, BP (2005) states: However, the main culprit is probably the customer, who very rarely asks about fire safety, either when booking or when checking in.
Key Lesson: Risk management is both a personal and a professional matter.
Appendix C: Risk management resources
The following is a list of various risk management resources available in both print and electronic formats:
The following standards and guidelines provide helpful guidance on various aspects of risk management:
- Canadian Standards Association (CSA)
- CAN/CSA-Q850-97 (R2009): Risk Management: Guideline for Decision Makers (CSA, 1997)
- CAN/CSA-Z767-17: Process Safety Management (CSA, 2017a)
- CSA-Z1000:14: Occupational Health and Safety Management (CSA, 2014)
- Z763-96 (R2006): Introduction to Environmental Risk Assessment Studies (CSA, 1996)
- Z731-03 (R2014): Emergency Preparedness and Response (CSA, 2003)
- Z1600-17: Emergency and Continuity Management Program (CSA, 2017b)
- UL (formerly Underwriters Laboratories)
- CAN/UL 2984:2019: Standard for Management of Public Risks – Principles and Guidelines (UL, 2019)
- International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC)
- ISO 31000:2018: Risk Management – Guidelines (ISO, 2018)
- IEC 31010:2009: Risk Management – Risk Assessment Techniques (IEC, 2009)
- ISO Guide 73:2009: Risk Management – Vocabulary (ISO, 2009)
C.2 Reference books
Risk Management Guide for Major Technological Accidents (CRAIM, 2017) is a Canadian-produced book that is comprehensive in both scope and depth of coverage. It is available in both English and French. Relevant features include many of the concepts discussed in section III of the guideline, as well as treatments of land-use planning, emergency response, incident investigation, business continuity, communication, and risk perception.
Engineering Risk Management (Meyer & Reniers, 2016) is, as the title explicitly states, focussed on the management of engineering risk. The authors are engineers residing in Europe, where – in general – there is typically more widespread use of risk management techniques than in North America. Relevant features include many of the concepts discussed in section III of the guideline, as well as treatments of hazard identification from a life-cycle perspective and the long-term impacts of risk management decisions with respect to sustainable development.
As the title implies, Risk Assessment. Theory, Methods, and Applications (Rausand, 2011) focuses more on the analysis and assessment of risk rather than its management. Nevertheless, individual chapters on risk management and job safety analysis are relevant to the current subject matter. Chapter 17 in particular describes the development and application – or lack thereof – of risk assessment and management techniques. Examples are given for: (i) the defense, nuclear power, process, offshore oil and gas, space, and aviation industries, (ii) railway and marine transport, (iii) machinery systems, (iv) the natural environment, (v) critical infrastructures, and (vi) municipalities. Rausand (2011) also offers a good entry point to some of the mathematical treatments of risk assessment.
Perry’s Chemical Engineers’ Handbook (Green & Southard, 2019) is a reference that is well-known to engineers working in the process industries (in particular, chemical and environmental engineers). The section on process safety analysis and risk analysis is succinct and is complementary to CRAIM (2017), Meyer & Reniers (2016), and Rausand (2011).
C.3 Hazard identification methods and references
Qualitative hazard identification methods include: Checklist (CL) Analysis, What-If (WI) Analysis, Failure Modes and Effects Analysis (FMEA), and Hazard and Operability (HAZOP) Study. These and other techniques require differing levels of detail with respect to product or process information, and so are most effective at specific life cycle stages (Crawley & Tyler, 2003). Each method therefore has limitations that must be recognized by the engineer or team involved in hazard identification (Casey, 2019).
Hazard identification techniques such as Bowtie (BT) Analysis (a combination of Fault Tree (FT) Analysis and Event Tree (ET) Analysis) have become increasingly popular in recent years and can be used either qualitatively or quantitatively (de Ruijter & Guldenmund, 2016; CCPS/EI, 2018).
C.4 Minerva Canada Safety Management Education Inc.
Twenty-six teaching modules on various aspects of health and safety are available on the web site of Minerva Canada Safety Management Education Inc. (Minerva, 2019a). A number of these modules (PowerPoint presentations) deal with risk management and associated concepts: (i) Risk Management, (ii) Risk Communications, (iii) Hazard and Risk Identification, (iv) Hazard and Risk Identification – Part II, (v) Quantitative Risk Assessment, (vi) Biorisk, (vii) Safety Management Systems and Leadership, and (viii) Process Safety Management.
The Risk Management module emphasizes that most accidents are smaller than the large (major) accidents that dominate the headlines. There are thus far more minor (high probability/low consequence) accidents than major (low probability/high consequence) accidents in which risk management inadequacies play a role. These less severe incidents afford multiple learning opportunities for engineering practitioners.
Case studies on health and safety in various businesses are also available (Minerva, 2019b). One such product suitable for risk management training purposes is titled Risk Management for Cyanide Handling.
C.5 Canadian Centre for Occupational Health and Safety
The Canadian Centre for Occupational Health and Safety (CCOHS) has developed extensive guidance documentation on hazard identification and risk assessment. For example, CCOHS (2019a) categorizes hazards as: (i) chemicals, (ii) ergonomic, (iii) health, (iv) physical, (v) psychosocial, (vi) safety, and (vii) workplace. CCOHS (2019b) gives fact sheets on topics such as: (i) Hazard and Risk, (ii) Hazard Control, (iii) Risk Assessment, and (iv) Simple Risk Assessment Form.
C.6 Canadian Society for Chemical Engineering
The Canadian Society for Chemical Engineering (CSChE) is active in process safety and risk management through its Process Safety Management Division (PSMD). Examples here include a process safety management (PSM) guide (CSChE, 2012a) and standard (CSChE, 2012b). The CSChE PSM guide and standard have formed the basis for the Canadian Standards Association (CSA) PSM standard (CSA, 2017a) referenced in section C.1 of this appendix
C.7 The Royal Society for the Prevention of Accidents
The Royal Society for the Prevention of Accidents (RoSPA) in the UK has presented a five-step guide to risk assessment (RoSPA, 2019a):
- Identify the hazards.
- Decide who might be harmed and how.
- Evaluate the risks and decide on control measures.
- Record your findings and implement them.
- Review your assessment and update if necessary.
In its Advice Pack for Smaller Firms, RoSPA provides an information sheet entitled Risk Assessment at its Very Simplest (RoSPA, 2019b).
C.8 Institution of Chemical Engineers
The UK Institution of Chemical Engineers (IChemE) offers a variety of risk management training packages and courses. Relevant papers can also be found in the proceedings of IChemE symposia – e.g., Clarke (2000), who advocates for an integrated approach to risk management incorporating employee safety, business continuity, environmental concerns, process safety and product quality.
C.9 UK Health and Safety Executive
The UK Health and Safety Executive (HSE) is an authoritative source of health, safety and risk management information. For example, the five-step process described previously in section C.7 of this appendix is actually an HSE product (HSE, 2019a). Representative risk assessments for food preparation, factories, warehouses, and office cleaning can be found in HSE (2019b). The ALARP (as low as reasonably practicable) principle – a mainstay of risk management practice in the UK and elsewhere – is explained in HSE (2019c).
C.10 Institute of Risk Management
The Institute of Risk Management (IRM), headquartered in the UK, has produced a Risk Management Standard that is available for free download (IRM, 2002).
C.11 US Chemical Safety Board
US Chemical Safety Board (CSB) investigation reports and case studies are a valuable source of lessons learned with respect to hazards and risk in high-hazard industries. A recent CSB Safety Spotlight (CSB, 2019) illustrates the importance of industry safety guidelines, codes and standards (as also discussed in section III of the guideline and section C.1 of this appendix). CSB (2019) cites the American Petroleum Institute (API) and its work on identifying human fatigue as a risk factor.
C.12 Center for Chemical Process Safety
The Center for Chemical Process Safety (CCPS) of the American Institute of Chemical Engineers (AIChE) is a world-leader in the provision of resource material on process hazards and the ensuing risks. Examples include CCPS 2009, 2011, 2012 and 2015.
C.13 US Environmental Protection Agency
From health and environmental perspectives, the US Environmental Protection Agency (EPA) provides guidance in areas such as: (i) Risk Management, (ii) Risk Communication, (iii) Human Health Risk Assessments, and (iv) Ecological Risk Assessments (EPA, 2019).
C.14 Safety Science
The technical journal Safety Science is a rich source of risk management applications. The following examples demonstrate that risk management is central to effective, competent practice in virtually all engineering endeavours: (i) project management (Badri et al., 2012), (ii) high-risk domains (Grote, 2012), (iii) electrical transmission and distribution line construction (Albert & Hallowell, 2013), (iv) airport surface operations (Wilke et al., 2014), (v) risk perceptions in construction (Zhao et al., 2016), (vi) extreme and rare events in asset management (Komljenovic et al., 2016), (vii) industrial biological processes (Moreno & Cozzani, 2018), (viii) managing construction projects (Yiu et al., 2019), and (ix) offshore wind turbine maintenance (Mentes & Turan, 2019).
Appendix D: References
Ahluwalia, J. (2006). Something in the air? Canadian Chemical News, 58 (9), 20-21.
Albert, A. and Hallowell, M.R. (2013). Safety risk management for electrical transmission and distribution line construction. Safety Science, 51, 118-126.
Alp, E. (2008). Process safety management in Canada today. What it is…and what it is not. Canadian Chemical News, 60 (5), 24-26.
Amyotte, P. and Khan, F. (Editors) (2019). Dust Explosions. Methods in Chemical Process Safety, Volume 3, Cambridge, MA: Elsevier/Academic Press.
Amyotte, P.R. and Lupien, C.S. (2017). Elements of process safety management. Chapter 3 in Methods in Chemical Process Safety, Volume 1, F. Khan (Editor), Cambridge, MA: Elsevier/Academic Press, pp. 87-148.
APEGA (Association of Professional Engineers and Geoscientists of Alberta) (2006). Guideline for Management of Risk in Professional Practice, V1.0, Edmonton, AB: Association of Professional Engineers and Geoscientists of Alberta. Last accessed May 31, 2019 at: https://www.apega.ca/assets/PDFs/risk.pdf
Badri, A., Gbodossou, A. and Nadeau, S. (2012). Occupational health and safety risks: Towards the integration into project management. Safety Science, 50, 190-198.
BP (2005). Fire Safety Booklet. Hotel Fire Safety, 2nd edition. BP Process Safety Series, Rugby, UK: Institution of Chemical Engineers.
Bird, F.E. and Germain, G.L. (1996). Practical Loss Control Leadership. Loganville, GA: Det Norske Veritas.
Casey, R. (2019). Limitations and misuse of LOPA. Loss Prevention Bulletin. 265, 13-16.
CCOHS (Canadian Centre for Occupational Health and Safety( (2019a). Hazards. Hamilton, ON: Canadian Centre for Occupational Health and Safety. Last accessed July 29, 2020 at: https://www.ccohs.ca/topics/hazards/
CCOHS (Canadian Centre for Occupational Health and Safety( (2019b). Hazard ID, Risk Assessment. Hamilton, ON: Canadian Centre for Occupational Health and Safety. Last accessed July 29, 2020 at: https://www.ccohs.ca/topics/programs/programs/hazards/
CCPS (Center for Chemical Process Safety) (2009). Inherently Safer Chemical Processes. A Life Cycle Approach, 2nd edition. Hoboken, NJ: John Wiley & Sons, Inc.
CCPS (Center for Chemical Process Safety) (2011). Conduct of Operations and Operational Discipline for Improving Process Safety in Industry. Hoboken, NJ: John Wiley & Sons, Inc.
CCPS (Center for Chemical Process Safety) (2012). Recognizing Catastrophic Incident Warning Signs in the Process Industries. Hoboken, NJ: John Wiley & Sons, Inc.
CCPS (Center for Chemical Process Safety) (2015). Guidelines for Defining Process Safety Competency Requirements. Hoboken, NJ: John Wiley & Sons, Inc.
CCPS/EI (Center for Chemical Process Safety/Energy Institute) (2018). Bow Ties in Risk Management. A Concept Book for Process Safety. Hoboken, NJ: John Wiley & Sons, Inc.
Clarke, I. (2000). Integrated Risk Management. Proceedings of Symposium Series No. 147, Rugby, UK: Institution of Chemical Engineers.
CRAIM (Conseil pour la Réduction des Accidents Industriels Majeurs) (2017). Risk Management Guide for Major Technological Accidents, 7th edition. Laval, QU: Conseil pour la Réduction des Accidents Industriels Majeurs.
Crawley, F. and Tyler, B. (2003). Hazard Identification Methods. European Process Safety Centre, Rugby, UK: Institution of Chemical Engineers.
Creedy, G. (2013). Identifying system failures at Lac Mégantic. Canadian Chemical News, 65 (4), 13.
CSA (Canadian Standards Association) (1996). Introduction to Environmental Risk Assessment Studies. Z763-96 (R2006), Toronto, ON: CSA Group.
CSA (Canadian Standards Association) (1997). Risk Management: Guideline for Decision-Makers. CAN/CSA-Q850-97 (R2009), Toronto, ON: CSA Group.
CSA (Canadian Standards Association) (2003). Emergency Preparedness and Response. Z731-03 (R2014), Toronto, ON: CSA Group.
CSA (Canadian Standards Association) (2014). Occupational Health and Safety Management. CSA-Z1000:14, Toronto, ON: CSA Group.
CSA (Canadian Standards Association) (2017a). Process Safety Management. CAN/CSA-Z767-17, Toronto, ON: CSA Group.
CSA (Canadian Standards Association) (2017b). Emergency and Continuity Management Program. Z1600-17, Toronto, ON: CSA Group.
CSB (Chemical Safety Board) (2019). CSB Safety Spotlight: The Importance of Industry Safety Guidelines, Codes, and Standards. Washington, DC: US Chemical Safety Board. Last accessed July 29, 2020 at: https://www.csb.gov/assets/1/17/csb_safety_spotlight_sdos_1.pdf?16458
CSChE (Canadian Society for Chemical Engineering) (2012a). Process Safety Management Guide, 4th edition. Ottawa, ON: Canadian Society for Chemical Engineering. Last accessed July 29, 2020 at: https://www.cheminst.ca/wp-content/uploads/2019/04/PSM20Guide204th20Edition-1.pdf
CSChE (Canadian Society for Chemical Engineering) (2012b). Process Safety Management Standard, 1st edition. Ottawa, ON: Canadian Society for Chemical Engineering. Last accessed July 29, 2020 at:
de Ruijter, A. and Guldenmund, F. (2016). The bowtie method: A review. Safety Science, 88, 211-218.
Di Menna, J. (2012). Safety Haven. Canadian Chemical News, 64 (9), 24-27.
Dove, K. (2017). Westray, 25 years later. What we’ve learned. The Engineer, Engineers Nova Scotia, 30 (4), 7-8. Last accessed July 29, 2020 at: https://engineersnovascotia.ca/files/publications/107/file/Winter2017ElectronicLowRes.pdf
Engineers Canada (2016). Public Guideline on the Code of Ethics. Ottawa, ON: Engineers Canada. Last accessed July 29, 2020 at: https://engineerscanada.ca/publications/public-guideline-on-the-code-of-ethics
EPA (Environmental Protection Agency) (2019). Risk Assessment. Washington, DC: US Environmental Protection Agency. Last accessed July 29, 2020 at: https://www.epa.gov/risk
Green, D.W. and Southard, M.Z. (Editors) (2019). Process safety analysis. In Perry’s Chemical Engineers’ Handbook, 9th edition, New York, NY: McGraw-Hill, pp. 23-31 – 23-40.
Grote, G. (2012). Safety management in high-risk domains – All the same? Safety Science, 50, 1983-1992.
Hopkins, A. (2000). Lessons from Longford. Sydney, Australia: CCH Australia Limited.
Hopkins, A. (2005). Safety, Culture and Risk. The Organisational Causes of Disasters. Sydney, Australia: CCH Australia Limited.
Hopkins, A. (2009a). Failure to Learn. The BP Texas City Refinery Disaster. Sydney, Australia: CCH Australia Limited.
Hopkins, A. (Editor) (2009b). Learning from High Reliability Organisations. Sydney, Australia: CCH Australia Limited.
HSE (Health and Safety Executive) (2019a). Risk – Controlling the Risks in the Workplace. London, UK: Health and Safety Executive. Last accessed July 29, 2020 at: http://www.hse.gov.uk/risk/controlling-risks.htm
HSE (Health and Safety Executive) (2019b). Risk Management. London, UK: Health and Safety Executive. Last accessed July 29, 2020 at: http://www.hse.gov.uk/risk/
HSE (Health and Safety Executive) (2019c). ALARP “at a Glance”. London, UK: Health and Safety Executive. Last accessed July 29, 2020 at: http://www.hse.gov.uk/risk/theory/alarpglance.htm
ICE (Institution of Civil Engineers) (2018). In Plain Sight: Assuring the Whole-Life Safety of Infrastructure. London, UK: Institution of Civil Engineers. Last accessed July 29, 2020 at: https://www.ice.org.uk/getattachment/news-and-insight/policy/in-plain-sight/In-Plain-Sight.pdf.aspx
IEC (International Electrotechnical Commission) (2009). Risk Management – Risk Assessment Techniques. IEC 31010:2009, Geneva, Switzerland: International Electrotechnical Commission.
IEEIRP (Independent Expert Engineering Investigation and Review Panel) (2015). Report on Mount Polley Tailings Storage Facility Breach. Victoria, BC: Province of British Columbia. Last accessed July 29, 2020 at: https://www.mountpolleyreviewpanel.ca/final-report
IRM (Institute of Risk Management) (2002). A Risk Management Standard. London, UK: Institute of Risk Management. Last accessed July 29, 2020 at: https://www.theirm.org/media/4709/arms_2002_irm.pdf
ISO (International Organization for Standardization) (2009). Risk Management – Vocabulary. ISO Guide 73:2009, Geneva, Switzerland: International Organization for Standardization.
ISO (International Organization for Standardization) (2018). Risk Management – Guidelines. ISO 31000:2018, Geneva, Switzerland: International Organization for Standardization.
Kletz, T. and Amyotte, P. (2010). Process Plants. A Handbook for Inherently Safer Design, 2nd edition. Boca Raton, FL: CRC Press/Taylor & Francis Group.
Komljenovic, D., Gaha, M., Abdul-Nour, G., Langheit, C. and Bourgeois, M. (2016). Risk of rare and extreme events in Asset Management. Safety Science, 88, 129-145.
Li, J., Reniers, G., Cozzani, V. and Khan, F. (2017). A bibliometric analysis of peer-reviewed publications on domino effects in the process industry. Journal of Loss Prevention in the Process Industries, 49, 103-110.
Liserio, F.F. and Mahan, P.W. (2019). Manage the risks of severe wind and flood events. Chemical Engineering Progress, 115 (4), 42-49.
Marsh (2018). The 100 Largest Losses – 1978-2017. Large Property Damage Losses in the Hydrocarbon Industry, 25th edition. Houston, TX: Marsh, Ltd.
McCusker, A. (2008). Risk Management – An Essential Strategy for Business Success.
Plenary lecture presented at PSAM (Probabilistic Safety Assessment and Management) 9, Hong Kong, May 18-23, 2008.
Mentes, A. and Turan, O. (2019). A new risk management model for Offshore Wind Turbine maintenance. Safety Science, in press.
Meyer, T. and Reniers, G. (2016). Engineering Risk Management, 2nd edition. Berlin, Germany: De Gruyter.
Minerva (2019a). Teaching Modules. Mississauga, ON: Minerva Safety Management Education. Last accessed July 29, 2020 at: https://safetymanagementeducation.com/teaching-resources/teaching-modules/
Minerva (2019b). Case Studies. Mississauga, ON: Minerva Safety Management Education. Last accessed July 29, 2020 at: https://safetymanagementeducation.com/teaching-resources/case-studies-instructor-notes/
Moreno, V.C. and Cozzani, V. (2018). Integrated hazard identification within the risk management of industrial biological processes. Safety Science, 103, 340-351.
Moreno, V.C., Reniers, R., Salzano, E. and Cozzani, V. (2018). Analysis of physical and cyber security-related events in the chemical and process industry. Process Safety and Environmental Protection, 116, 621-631.
Rausand, M. (2011). Risk Assessment. Theory, Methods, and Applications. Hoboken, NJ: John Wiley & Sons, Inc.
Reason, J.T. (1997). Managing the Risks of Organizational Accidents. Aldershot, UK: Ashgate Publishing Limited.
Reniers, G. and Cozzani, V. (Editors) (2013). Domino Effects in the Process Industries. Modeling, Prevention and Managing. Oxford, UK: Elsevier.
Richard, K.P., Justice (1997). The Westray Story – A Predictable Path to Disaster. Report of the Westray Mine Public Inquiry. Halifax, NS: Province of Nova Scotia. Last accessed July 29, 2020 at: https://novascotia.ca/lae/pubs/westray/
Roney, C. (2015), The Elliot Lake Inquiry. Lessons Learned. Presentation to the Engineers Canada February 2015 Board Meeting, Engineers Canada, Ottawa, ON. Last accessed July 29, 2020 at: https://engineerscanada.ca/sites/default/files/C-Roney-Elliot-Lake-Presentation-February-2015.pdf
RoSPA (The Royal Society for the Prevention of Accidents) (2019a). The Five Step Guide to Risk Assessment. Birmingham, UK: The Royal Society for the Prevention of Accidents. Last accessed July 29, 2020 at: https://rospaworkplacesafety.com/2013/01/21/what-is-a-risk-assessment/
RoSPA (The Royal Society for the Prevention of Accidents) (2019b). Advice Pack for Smaller Firms. Sheet 7: Risk Assessment at its Very Simplest. Birmingham, UK: The Royal Society for the Prevention of Accidents. Last accessed July 29, 2020 at: https://www.rospa.com/occupational-safety/advice/small-firms/pack/risk-assessment/
SCES (Strathcona County Emergency Services) (2019). Requirements for Heavy Industrial Developments. Sherwood Park, AB: Strathcona County Emergency Services.
Sells, B. (1994). What asbestos taught me about managing risk. Harvard Business Review, 72 (2), 76-90.
Shell (2019). Our 12 Life Saving Rules. The Hague, Netherlands: Shell Global. Last accessed July 29, 2020 at: https://www.shell.com/sustainability/safety/personal-safety.html
UL (2019). Standard for Management of Public Risks – Principles and Guidelines. CAN/UL 2984:2019, Northbrook, IL: UL.
Wilke, S., Majumdar, A. and Ochieng, W.Y. (2014). Airport surface operations: A holistic framework for operations modeling and risk management. Safety Science, 63, 18-33.
Wilson, L. and McCutcheon, D. (2003). Industrial Safety and Risk Management. Edmonton, AB: University of Alberta Press.
Yiu, N.S.N., Chan, D.W.M., Shan, M. and Sze, N.N. (2019). Implementation of safety management system in managing construction projects: Benefits and obstacles. Safety Science, 117, 23-32.
Zhao, D., McCoy, A.P., Kleiner, B.M., Mills, T.H. and Lingard, H. (2016). Stakeholder perceptions of risk in construction. Safety Science, 82, 111-119.
- Engineers Canada, 2016
- Roney, 2015
- adapted from Wilson & McCutcheon, 2003
- adapted from Wilson & McCutcheon, 2003
- Bird & Germain, 1996
- Li et al., 2017
- Li et al., 2017
- Reniers & Cozzani, 2013
- Rausand, 2011
- Liserio & Mahan, 2019
- Amyotte & Lupien, 2017
- CCPS, 2009
- Kletz & Amyotte, 2010
- Reason, 1997
- Ahluwalia, 2006
- McCusker, 2008
- McCusker, 2008
- SCES, 2019
- Sells, 1994
- IEEIRP, 2015
- Marsh, 2018
- Amyotte & Khan, 2019
- ICE, 2018
- ICE, 2018
- Shell, 2019
- BP, 2005