SEARCH

As a Compliance Engineer, I'm focused on developing robust methodologies for emerging compliance challenges. A recent IEEE webinar that I attended on AI Safety for Automotive provided valuable insights into the upcoming I SO PAS 8800 standard, introducing a pragmatic approach to AI safety assurance that I believe warrants sharing. Requirements Isolation Strategy: A Systems Engineering Approach The webinar presented what I'll call the "Requirements Isolation Strategy" - a methodical approach to AI safety compliance. Rather than treating AI as a complete system overhaul, this strategy focuses on isolating specific safety requirements that are allocated to AI functionality. By precisely identifying these requirements, we can develop targeted assurance processes for just these elements. This builds on established practices from other industries such as the medical device industry, where requirements traceability, verification, and validation are paramount. At the same time, this approach acknowledges that the fundamental requirements for automotive safety haven’t changed with the integration of AI. Instead, we’re confronted with additional uncertainty surrounding specific requirements that necessitate structured assurance and risk measures. Critical Distinction: Assurance vs. Risk Management The webinar did not address, but is crucially important, the critical distinction between assurance and risk management activities in the context of safety. Assurance processes are not sufficient to handle risk Assurance entails the provision of quantifiable evidence demonstrating the fulfillment of requirements and the system’s effectiveness. In contrast, Risk Management systematically addresses uncertainty through: Methodical reduction of controllable risks, and Establishing engineering margins for unavoidable or irreducible risk This distinction is crucial for implementing effective management processes, technical controls, and risk measures to achieve the outcome of safety. Applications Beyond Automotive The Requirements Isolation Strategy used in ISO PAS 8800 has broad application for other compliance domains, including: Security requirements Sustainability commitments Quality expectations Regulatory compliance Ethical conduct and others The methodology remains the same: Identify and isolate requirements allocated to the AI system Establish specific assurance protocols for these requirements, and Implement appropriate risk controls and measures. This targeted approach significantly reduces the complexity of managing AI-related risks across a variety of compliance objectives. Looking Forward This requirements-based approach offers a structured path forward as organizations integrate AI systems into their operations. By isolating AI-specific requirements and their associated assurance and risk needs, we can maintain robust compliance without creating unnecessary complexity in our existing systems. This allows for clear traceability between requirements, verification methods, and assurance evidence. What do you think of this approach? What strategies are you using to advance AI Safety within your operations and systems?

Innovation and creativity are often thought of as the cornerstones of success in business. Organizations are continually pushing boundaries to come up with new products, services, and ways of doing things that will set them apart from their competitors. However, the drive to be different can sometimes come at a cost, and that cost is compliance. Compliance has often been seen as negative, holding back innovation and creativity. It is viewed as a set of rules and regulations that stifle creativity and prevent organizations from achieving their full potential. But compliance has evolved, and its role has changed. It is no longer just about compliance with rules or conformance to standards; it is about aligning with organizational values associated with safety, security, sustainability, quality, and stakeholder obligations. Compliance is not a hindrance to innovation, it is a necessary constraint to keep us from harm. Compliance is a necessary constraint that keeps us between ethical lines and ahead of risk. As an engineer, we view constraints as our friends. They present a challenge requiring creativity and innovation to come up with engineered solutions that are aligned with business, stakeholder, and societal values. Compliance is essential to maintaining a level playing field, where businesses can compete fairly and ethically. It ensures that organizations are held accountable for their actions and that they operate within legal and regulatory boundaries. Compliance also protects the interests of stakeholders, such as customers, employees, and shareholders, by ensuring that their rights and expectations are met. Innovation and creativity are important, but they must be balanced with responsibility and accountability. Compliance is not a barrier to innovation, but rather a necessary aspect of responsible innovation. Compliance ensures that innovation is aligned with organizational values and societal expectations. Compliance drives innovation by presenting challenges that require creativity and innovation to find solutions that meet compliance requirements while achieving business goals. Innovation can be risky, and compliance helps manage that risk. Compliance provides a framework for identifying and mitigating the effects of uncertainty, ensuring that organizations operate in a safe and sustainable manner. Compliance also helps to build trust with stakeholders, such as customers and investors, by demonstrating that the organization is committed to ethical and responsible behaviour. Compliance is not just a set of rules and regulations; it is a mindset. Compliance is a way of thinking about innovation and creativity that recognizes the importance of responsibility and accountability. Compliance should be embraced as a necessary constraint that helps drive innovation and ensures that organizations operate in a safe and sustainable manner. Compliance is not the enemy of innovation; it is the friend you never knew you needed.

When it comes to compliance, labelling everything as AI might be a bad idea. For example, engineering has traditionally relied on algorithms, statistical analysis, models, and prediction, and this practice should continue without any confusion with AI. However, AI does have unique characteristics that, if not understood, could pose significant risks to the designs and intended outcomes of the solutions developed. Nevertheless, labelling all of this as AI might unnecessarily create regulatory uncertainty and complexity with obligations that are already handled by existing practice guidelines and standards. The need for defining AI is indeed crucial, not only to separate the boundaries of where new risks not currently addressed are being introduced, but also to ensure that you are not inadvertently creating legal, regulatory, or ethical exposure for yourself.

Predictive analytics is a topic of much discussion these days and is considered by some to be a proactive measure against safety, quality, environmental, and regulatory failure. Predictive analytics can help to prevent a total failure if controls can respond fast enough and if the failure mode is predictive in the first place. However, when uncertainty (the root cause of risk) is connected with natural variation (aleatory uncertainty) we cannot predict outcomes. Also, when uncertainty is due to a lack of knowledge (epistemic uncertainty) prediction is limited based on the strength of our models, experimentation, and the study of cause and effect. Predictive analytics is not a substitute for effective risk management. To properly contend with risk we must be proactive rather than only predictive. We need to estimate uncertainty (both aleatory and epistemic), its impacts, and the effectiveness of the controls we have put in place either to guard against failure (margins) or reduce its likelihood and severity (risk buy-down).

Machines, organizations, and communities include and are themselves part of systems.  Russell L. Ackoff a pioneer in systems thinking defined a system not as a sum of its parts but as the product of its interactions of those parts. ".. the essential properties that define any system are the properties of the whole which none of the parts have." The example he gives is that of a car.  The essential property of car is to take us from one place to another.  This is something that only a car as a whole can do.  The engine by itself cannot do this. Neither can the wheels, the seats, the frame, and so on. Ackoff continues: "In systems thinking, increases in understanding are believed to be obtainable by expanding the systems to be understood, not by reducing them to their elements. Understanding proceeds from the whole to its parts, not from the parts to the whole as knowledge does." A system is a whole which is defined by its function in a larger system of which it's a part. For a system to perform its function it has essential parts: Essential parts are necessary for the system to perform its function but not sufficient Implies that an essential property of a system is that it can not be divided into independent parts. Its properties derive out of the interaction of its parts and not the actions of its parts taken separately. When you apply analysis (reductionism) to a system you take it apart and it loses all its essential properties, and so do the parts.  This gives you knowledge (know how) on how the part works but not what they are for. To understand what parts are for you need synthesis (holism) which considers the role the part has with the whole. Why is this important and what has this to do with quality, safety, environmental and regulatory objectives? The answer is that when it comes to management systems we often take a reductionist approach to implementation.  We divide systems into constituent parts and focus our implementation and improvement at the component level.  This according to Ackoff is necessary, but not sufficient for the system to perform. We only need to look at current discussions with respect to compliance to understand that the problem with performance is not only the performance of the parts themselves, but rather failures in the links (i.e. dependencies) the parts have with each other.  Todd Conklin (Senior Advisor to the Associate Director, at Los Alamos National Laboratory) calls this "between and among" the nodes.  To solve these problems you cannot optimize the system by optimizing the parts making each one better.  You must consider the system as a whole – you must consider dependencies. However, this is not how most compliance systems are implemented and improved.  Instead, the parts of systems are implemented in silos that seldom or ever communicate with each other.  Coordination and governance is also often lacking to properly establish purpose, goals, and objectives for the system. In practice, optimization mostly happens at the nodes and not the dependencies.  It is this lack of systems attention that contributes to poor performance.  No wonder we often hear of companies who have implemented all the "parts" of a particular management system and yet fail to receive any of the benefits from doing so.  For them it has only been a cost without any return. However, by applying Systems Thinking you can achieve a better outcome.  "One can survive without understanding, but not thrive. Without understanding one cannot control causes; only treat effect, suppress symptoms. With understanding one can design and create the future ... people in an age of accelerating change, increasing uncertainty, and growing complexity often respond by acquiring more information and knowledge, but not understanding." -- Russell Ackoff. For those looking for a deeper dive the following video (90 minutes) provides an excellent survey of systems thinking by Russell L. Ackoff a pioneer in the area of systems improvement working along side others such as W. Edward Deming.

Change can be a significant source of risk. That is why compliance programs include a risk-based process for managing planned changes. This process is commonly referred to in highly-regulated, high-risk industries as, Management of Change or MOC. This blog takes a look at MOC across a variety of regulations and standards that are used to help buy down risk. What is Management of Change? MOC is a critical process used to ensure that no unintended consequences occur as a result of planned changes. It is required by EMP-RMP, OSHA 1910.119, NEB, API RP 1173, CSA Z767-17, ICH, and now part of ISO 45001 Safety Standard. An effective MOC process will help to plan, implement, and manage change to prevent or mitigate unintended consequences that affect the safety of workers, public, or the environment. Although MOC processes may look different based on the industry or compliance system involved, the purpose remains the same, which is, to avoid unnecessary risk. MOC differs from change management which refers to the people side of change (Kotter, PROSCI, etc) and focuses on changing mindsets, attitudes, and behaviours needed to effect a change. This is often confused with management of change which refers to the technical side of change and focuses on risk management. However, depending on the type of change both these practices may be necessary. An MOC process provides a structured approach to capture a change, identify and mitigate risks, assess impacts (organization, procedures, behaviours, documentation, training, etc.), define work plans to effect change safely, engage stakeholders, obtain necessary approvals, and update effected documentation. By following such a process risk can be adequately ameliorated which perhaps is the most important measure of MOC effectiveness. While managing risk for individual changes is of value, companies with advanced MOC capabilities are able to measure the total level of risk proposed or currently being introduced across a facility, process, or product line. This information is used to ensure that overall risk is handled within existing risk controls. When to Use MOC The applicability of an MOC process is determined by identifying proposed changes that have the possibility of high unintended consequences. These are called differently by each standard or regulation. Here is a list of examples: covered processes covered pipeline segments high consequence areas safety critical roles or positions safety critical procedures safety critical equipment or assets and so on When changes are made to any of the above then an MOC is required. However, there is an increasing trend towards using a single MOC process to manage all changes even if not required by a given standard or regulation. This has become viable through the introduction of computer automation and adaptive workflows that can adjust the level of rigour commensurate with the level of risk. When Managing Change Hinders Innovation Innovation is necessary for growth and often requires that risks are taken. However, a common sentiment is that compliance is getting in the way of product or process innovation. The pharmaceutical sector is one of the most regulated in industrialized countries. FDA has strict requirements for verification and validation of products and services. The risks to patients are many so it makes sense to scrutinize every aspect from design to delivery of new products. Changes made during the product life-cycle can lead to re-validation and conducting more clinical trials all of which introduce delays to the introduction of the new drug or medical device. In 2005, the Quality Risk Management program ICH-Q9 was introduced to bring a risk based approach to this industry and parallels the risk-based approach introduced by the Center for Chemical Process Safety. ICH-Q9 was extended to the medical device sector by the introduction of the ISO14971 Risk management standard. These were done to partially address the question of risk management and innovation and so was welcomed by the industry and FDA. This risk based approach leverages the ICH-Q8 standard which introduced, among other things, the concept of design space . A design space establishes parameters that have been demonstrated to provide quality assurance. Once a design space is approved, changes within the design space boundaries are not considered a change from a regulatory point of view. This creates a space for innovation to occur. Replacement in Kind Now, let's consider the process sector where a similar concept to design spaces is used known as, "Replacement in Kind" or RIK. Replacement in Kind uses the idea that when changes are made to the "design basis" a Management of Change (MOC) process must be followed to manage risk. Otherwise, the change is considered a "replacement" and not a change from a regulatory point of view. In many ways, RIK has the same effect that design space has in the Pharma/Med Device sectors. They both define boundaries that allow certain changes to occur that will produce a certain design outcome. Unfortunately, one notable difference between the two approaches is how design basis is currently managed in the process sector. Design information tends not to be as controlled or managed as well as it is in the Pharma/Med Device industry. In fact, it is common in older facilities to find that the design basis for a process or equipment is no longer known and engineers and maintenance crews resort to using the manufacturer's specifications for the equipment, parts, or material substitutions. This has the effect of reducing the options and innovations that might otherwise be available. In a fashion, improving the management of design basis could allow for more innovation in the process sector. More changes could be considered as RIK without increasing risk. This would result in fewer MOCs and fewer resources being spent redoing hazard analysis, risk assessments and implementing unnecessary risk measures. What the Standards and Regulations Say For those who would like to explore the topic of MOC further, the following MOC requirements from selected standards and regulations are provided below. It is worth noting that the details of "how" to follow the guidelines are left to each organization to determine based on their business and level of risk. Title 40 CFR Part 68 – EMP RMP Program §68.75 Management of change. (a) The owner or operator shall establish and implement written procedures to manage changes (except for “replacements in kind”) to process chemicals, technology, equipment, and procedures; and, changes to stationary sources that affect a covered process. (b) The procedures shall assure that the following considerations are addressed prior to any change: The technical basis for the proposed change; Impact of change on safety and health; Modifications to operating procedures; Necessary time period for the change; and, Authorization requirements for the proposed change. (c) Employees involved in operating a process and maintenance and contract employees whose job tasks will be affected by a change in the process shall be informed of, and trained in, the change prior to start-up of the process or affected part of the process. (d) If a change covered by this paragraph results in a change in the process safety information required by §68.65 of this part, such information shall be updated accordingly. (e) If a change covered by this paragraph results in a change in the operating procedures or practices required by §68.69, such procedures or practices shall be updated accordingly. OSHA 1910.119(l) – Process Safety Management 1910.119(l) Management of change. 1910.119(l)(1) The employer shall establish and implement written procedures to manage changes (except for "replacements in kind") to process chemicals, technology, equipment, and procedures; and, changes to facilities that affect a covered process. 1910.119(l)(2) The procedures shall assure that the following considerations are addressed prior to any change: 1910.119(l)(2)(i) The technical basis for the proposed change; 1910.119(l)(2)(ii) Impact of change on safety and health; 1910.119(l)(2)(iii) Modifications to operating procedures; 1910.119(l)(2)(iv) Necessary time period for the change; and, 1910.119(l)(2)(v) Authorization requirements for the proposed change. 1910.119(l)(3) Employees involved in operating a process and maintenance and contract employees whose job tasks will be affected by a change in the process shall be informed of, and trained in, the change prior to start-up of the process or affected part of the process. 1910.119(l)(4) If a change covered by this paragraph results in a change in the process safety information required by paragraph (d) of this section, such information shall be updated accordingly. 1910.119(l)(5) If a change covered by this paragraph results in a change in the operating procedures or practices required by paragraph (f) of this section, such procedures or practices shall be updated accordingly. API Recommended Practice 1173 – Pipeline Safety Management 8.4 Management of Change (MOC) 8.4.1 General The pipeline operator shall maintain a procedure for management of change (MOC). For the MOC, the pipeline operator shall identify the potential risks associated with the change and any required approvals prior to the introduction of such changes. 8.4.2 Types of Changes: The type of changes that MOC address shall include: Technical, Physical, Procedural, and Organizational. Changes to the system shall include permanent or temporary. The process shall incorporate planning for each of these situations and consider the unique circumstances of each. 8.4.3 Elements of MOC Process: A MOC process shall include the following: Reason for change, Authority of approving changes, Analysis of implications Acquisitions of required work permits, Documentation (of change process and the outcome of the changes), Communication of changes to affected parties, Time limitations, Qualification and training of staff. CSA Z767-17 7.2 Management of change 7.2.1 The PSM system shall include a MOC system. The primary focus of MOC shall be to manage risks related to design changes and modifications to equipment, procedures, and organization. The MOC system shall: a) define what constitutes a change (such as temporary, emergency) and what constitutes replacement in kind which is not subject to MOC; b) include changes in and deviations from operating procedures or safe operating limits; c) include changes in organizational structure and staffing levels; d) define the review processes and thresholds for approval of changes, based on scope or magnitude of the change; e) require an assessment of hazards and risks associated with the change consistent with Clause 6.3; f) ensure that the change is communicated to affected stakeholders prior to the change, and that any required training is provided before the change is implemented; g) provide procedures for emergency changes including a means to contact appropriate personnel if a change is needed on short notice; and h) define the documentation requirements (such as a description of the proposed change, the authorization for the change, the training requirements, the updated drawings, and the verification that the change was completed as designed). ICH Pharmaceutical Quality System Q10 The change management system ensures continual improvement is undertaken in a timely and effective manner. It should provide a high degree of assurance there are no unintended consequences of the change. The change management system should include the following, as appropriate for the stage of the lifecycle: (a) Quality risk management should be utilised to evaluate proposed changes. The level of effort and formality of the evaluation should be commensurate with the level of risk; (b) Proposed changes should be evaluated relative to the marketing authorisation, including design space, where established, and/or current product and process understanding. There should be an assessment to determine whether a change to the regulatory filing is required under regional requirements. As stated in ICH Q8, working within the design space is not considered a change (from a regulatory filing perspective). However, from a pharmaceutical quality system standpoint, all changes should be evaluated by a company’s change management system; (c) Proposed changes should be evaluated by expert teams contributing the appropriate expertise and knowledge from relevant areas (e.g., Pharmaceutical Development, Manufacturing, Quality, Regulatory Affairs and Medical), to ensure the change is technically justified. Prospective evaluation criteria for a proposed change should be set; (d) After implementation, an evaluation of the change should be undertaken to confirm the change objectives were achieved and that there was no deleterious impact on product quality.

In the world of ethical, regulatory, and stakeholder obligations, understanding the underlying theories that drive compliance is key to achieving both compliance and mission success. Compliance isn't just about following rules; it's about employing strategic principles that not only ensure adherence but also deliver the benefits from always staying between the lines and head of risk. In this article, we will delve into the power of Management Theory (ISO 37301), Promise Theory, Systems Theory, Risk Theory, and Lean Management Theory, exploring how these theories when put into practice can elevate your compliance game. Management Theory (ISO 37301): The Blueprint for Compliance Excellence ISO 37301 (Compliance Management System) standard is rooted in management theory and serves as a comprehensive guide to how to effectively manage obligations. It goes beyond mere rule-following and focuses on proactive strategies for meeting obligations efficiently. Key Takeaway: ISO 37301 provides a structured approach to compliance, emphasizing the importance of proactive planning and performance. Promise Theory: A Culture of Trust through Compliance Promise Theory, introduced by computer scientist Mark Burgess, emphasizes that compliance is not merely a checklist; it's a collection of promises (policies) made to stakeholders. When these promises align with obligations, compliance becomes a part of an organization's culture. Key Takeaway: Promise Theory transforms compliance into a living culture of trust, where commitments to stakeholders are honoured and upheld. Systems Theory: Compliance as an Interconnected Symphony Systems Theory underscores that compliance is not achieved in isolation. Instead, it's a symphony of interconnected components and processes within an organization that must work together seamlessly. Compliance is more than the sum of its parts. Key Takeaway: Systems Theory highlights that Minimum Viable Compliance (MVC) is achieved when essential functions, behaviours and interactions are performing together at levels sufficient to produce compliance outcomes. Risk Theory: Navigating Compliance in Uncertain Waters Risk Theory acknowledges that compliance is not just about meeting expectations under ideal conditions. It recognizes that businesses must be resilient and adaptable in the face of uncertainty and risk. Key Takeaway: Risk Theory encourages organizations to build effective risk measures to improve the probability that compliance outcomes will be achieved in the presence of uncertainty. Lean Theory: Efficiency and Continuous Improvement Lean Management is a philosophy that focuses on efficiency, waste reduction, and continuous improvement. When applied to compliance, it streamlines processes and eliminates inefficiencies. Key Takeaway: Lean Management principles can be harnessed to optimize compliance processes, making them more efficient and adaptable. This frees up resources to be more proactive with compliance delivering compounding benefits over time. Harnessing the Power Understanding the theories behind compliance is crucial for success in ethical and regulatory matters. This article explored five powerful theories: Management Theory (ISO 37301), Promise Theory, Systems Theory, Risk Theory, and Lean Theory, and their potential to transform compliance. ISO 37301 offers a structured approach, emphasizing proactive planning. Promise Theory fosters a culture of trust by aligning commitments with obligations. Systems Theory stresses the interconnected nature of compliance components. Risk Theory focuses on resilience and adaptability. Lean Management improves efficiency. In summary, compliance is about more than just rules; it's about using these theories to thrive in a competitive business world. Applying them can help navigate uncertainty, build trust, streamline processes, and achieve compliance excellence improving the probability of long term mission success.

Compliance is often viewed as a cost of doing business. However, instead of viewing compliance only as an expense and something to reduce, what if it was seen as part of the overall business value proposition. In Geoffrey A. Moore's book entitled, " Zone to Win" he introduces a framework for understanding how change in the form of disruption can be introduced and managed to help organizations successfully compete. This is a very useful model not only to understand how companies can best prioritize their efforts but also to understand where and how compliance fits in. The Four Zones Moore describes 4 zones that define different areas of the business each having distinct goals, focus, and attention during disruption. The following diagram presents these zones with compliance added in RED : Performance Zone – The focus of this zone is to execute the business model and creating revenue. Productivity Zone – This zone focuses on efficiency, effectiveness and meeting compliance. This is the home of shared services, programs, systems and where cost is managed. Incubation Zone – This zone is looking 3-5 years out to position the company to catch the next wave of growth. Transformation Zone – this is where a disruptive business model scales and is introduced to the performance zone. The productivity zone according to Moore is responsible for delivering the following value propositions: Regulatory compliance – meeting obligations Improved efficiency – doing things right Improved effectiveness – doing the right things This is also the primary place where LEAN is applied and where compliance improvements are made. Moving Compliance to the Performance Zone While compliance is enabled by the productivity zone it is manifested in the performance zone. This creates a number of tensions including that between production and compliance objectives such as: safety, quality, environmental, regulatory, and so on. When you view compliance only as a cost you want to spend as little on it as possible. This can often lead to reducing the effort altogether instead of investing in compliance maturity. When it comes to safety, quality, and regulatory compliance this can create significant risk. Moore is correct in saying that as is the case of quality you cannot inspect compliance in; you have to design it in. I would argue that you need to go further and say that you don't have a business without compliance. Therefore, it is not simply a choice between whether to inspect or design in quality; you need to do more. LEAN talks about value in terms of activity that directly contributes to building the product the customer is purchasing. For example, inspections are seen as necessary but not value added. If the product is built correctly in the first place you would not need to do inspections. The customer does not want to pay for the cost of rework. Now, this line of thinking can (inappropriately) also be made regarding safety. If only workers acted in a safe manner we would not need safety systems. Safety systems only exist because of unsafe behaviors and the customer should not have to pay for that. The question that is really being asked is, "what is the value of compliance and is that something that customers are willing to pay for?" This same question was asked during the early days of quality. We now know the answer: quality adds value, reduces cost, and is something that customers are willing to pay for. This is now were compliance is at. Compliance is more than a cost or just necessary to obtain a regulatory license. Compliance contributes directly to and is part of a company's: business value proposition, social license to operate (legitimacy, credibility, trust), regulatory license to operate (quality, safety, environmental, integrity), and customers will not only pay for it; they will demand it. It's time to make compliance a full citizen of the performance zone and not just a visitor.

Cybersecurity is a constantly evolving field, with new threats emerging every day. As such, it is essential for organizations to take a proactive approach to managing cybersecurity risks. The Annual Loss Expectancy (ALE) formula is a crucial tool in this process. In this article, we will explore the history of ALE, provide examples of its application, and explain how it is used to evaluate cybersecurity risks for inherent and treated risks and their effects. History of ALE The history of ALE dates back to the 1970s, when it was first introduced in the field of insurance. ALE was used to calculate the potential financial losses associated with property damage or loss due to natural disasters, theft, or other unexpected events. Over time, ALE was adapted for use in cybersecurity risk management. Today, ALE is widely used in the cybersecurity industry as a standard method for evaluating the financial impact of cyber threats. The formula for calculating ALE is relatively simple, but the data required to input into the formula can be complex. How is ALE Calculated? ALE is a risk management formula used to calculate the expected monetary loss from a security incident over a year. The formula is calculated by multiplying the Annual Rate of Occurrence (ARO) with the Single Loss Expectancy (SLE). ARO is the estimated number of times a security incident is expected to occur in a year, and SLE is the estimated monetary value of a single incident. ALE = ARO x SLE For example, if a business estimates that it will experience a security breach once a year, and the cost of the breach is estimated to be $50,000, then the ALE would be: ALE = 1 x $50,000 = $50,000 This means that the business can expect to lose $50,000 per year from this particular security incident. How is ALE used to Manage Risk? ALE is a critical tool in managing cybersecurity risks. The ALE formula can be used to calculate both inherent and treated cybersecurity risks. Inherent risk refers to the level of risk that exists without any mitigating controls in place, while treated risk refers to the level of risk that remains after implementing mitigating controls. This information can then be used to prioritize risk effort. To illustrate the use of ALE in cybersecurity risk management, consider the following table: Risk ARO SLE Inherent Risk ALE Treated RIsk ALE Effect of Treatment Phishing 1 in 100 $10,000 $100 $10 90% reduction Ransomware 1 in 500 $50,000 $100 $10 90% reduction Inside Threat 1 in 1,000 $100,000 $100 $20 80% reduction Advanced Persistent Threat 1 in 10,000 $1,000,000 $100 $50 95% reduction In this scenario, a company has a database containing sensitive information that is accessible to all employees. Inherent risk is calculated by determining the potential financial loss if an attacker gains access to the database. If the estimated (example highlighted in yellow) SLE is $100,000 and the ARO is 1 in 1,000, then the inherent risk ALE would be $100. Treated risk, on the other hand, takes into account the effectiveness of mitigating controls. Suppose the company implements access controls to restrict access to the database to only authorized personnel. The treated risk ALE would be recalculated using the same ARO but a lower SLE. If the estimated SLE is now $20,000, then the treated risk ALE would be $20. The effect of treatment column shows the percentage reduction in ALE after implementing mitigative controls. Using ALE to Prioritize Risk Management Efforts By using ALE, organizations can identify potential financial losses, prioritize their cybersecurity efforts, and allocate resources more effectively. ALE can be used to compare different risks and determine which risks are the most significant and which ones require immediate attention. The risks with the highest ALE values are the ones that pose the greatest financial threat to the organization and require the most attention. Based on the previous example, the organization can see that the APT risk poses the greatest financial threat, with an inherent risk ALE of $100 and a treated risk ALE of $50. The organization should prioritize their efforts on mitigating this risk, such as implementing advanced security measures and training employees on how to identify and report suspicious activity. Mitigating controls, such as data loss prevention programs, access and identity management, and cyber safety training, can significantly reduce the SLE and the ALE. The cost and effectiveness of the countermeasures should be factored into the evaluation of treated risk. It is crucial to ensure that the cost of implementing the countermeasures does not exceed the potential financial loss. Organizations must also consider the potential impact on business operations and the overall risk management strategy. Conclusion ALE is a crucial tool in managing cybersecurity risks. It enables organizations to identify potential financial losses, prioritize their cybersecurity efforts, and allocate resources more effectively. ALE is calculated by multiplying the ARO by the SLE and can be used to evaluate both inherent and treated cybersecurity risks. Mitigating controls, such as anti-virus software or employee training, can significantly reduce the SLE and the ALE. However, organizations must also consider the potential impact on business operations and the overall risk management strategy. By using ALE, organizations can take a proactive approach to managing cybersecurity risks, reducing the likelihood of security incidents, and minimizing the potential financial losses associated with such incidents. While no security measure can guarantee complete protection against cyber threats, ALE provides a useful framework for evaluating risks and making informed decisions to best direct risk efforts. Cybersecurity and Infrastructure Security Agency (CISA). (2021). Cybersecurity Framework. Retrieved from https://www.cisa.gov/cybersecurity-framework Federal Financial Institutions Examination Council (FFIEC). (2019). Information Security Booklet. Retrieved from https://www.ffiec.gov/press/pr011719.htm Information Technology Laboratory. (2012). NIST SP 800-30 Rev. 1 Guide for Conducting Risk Assessments. Retrieved from https://csrc.nist.gov/publications/detail/sp/800-30/rev-1/final ISACA. (2012). Risk IT Framework. Retrieved from https://www.isaca.org/resources/risk-it-framework National Institute of Standards and Technology (NIST). (2020). Guide for Conducting Risk Assessments. Retrieved from https://csrc.nist.gov/publications/detail/sp/800-30/rev-1/final United States Department of Defense (DoD). (2014). Risk Management Guide for DoD Acquisition (6th ed.). Retrieved from https://www.acq.osd.mil/se/docs/Risk_Management_Guide_for_DoD_Acquisition_6th_Edition.pdf

A Guide to Evaluating Risk Performance and Effectiveness Introduction To proactively contend with risks associated with meeting obligations, companies rely on Key Performance Indicators (KPIs), Key Risk Indicators (KRIs), and Key Control Indicators (KCIs). Integrating these essential metrics into the Bowtie Risk Model offers a powerful framework for evaluating their performance and effectiveness. This article will delve into the process of mapping KPIs, KRIs, and KCIs to the Bowtie Risk Model to optimize risk management strategies and enhance overall performance. Understanding the Bowtie Risk Model The Bowtie Risk Model is a visual and qualitative risk analysis tool that provides a clear and comprehensive representation of risk scenarios. It consists of several key components: Hazard : The potential source of harm or adverse event that may lead to unwanted consequences. Threats : Specific events or circumstances that can trigger the hazard and escalate the risk. Top Event : The central risk event that occurs when the hazard is triggered by a threat. Consequences: The potential outcomes and impacts resulting from the top event. Preventative Barriers : Measures in place to prevent the hazard from being triggered. Mitigative Barriers : Measures aimed at reducing the severity of consequences if the top event occurs. Mapping KPI, KRI, and KCI to the Bowtie Risk Model Identify Relevant Metrics : Start by identifying the most relevant KPIs, KRIs, and KCIs for the specific risk scenario. These indicators should align with the organization's objectives, risk appetite, and regulatory requirements. Align KPIs with Consequences : Map the KPIs to the potential consequences of the top event. For example, if one of the consequences is a financial loss, the relevant financial KPIs could include impacts to revenue growth, cost control, or profitability. Map KRIs to Threats : Associate the KRIs with the identified threats in the Bowtie Risk Model. KRIs should act as early warning signals to detect potential threats before they escalate into top events. For instance, if one of the threats is a cybersecurity breach, relevant KRIs could include the number of unauthorized access attempts or malware detection rate. Connect KCIs to Barriers : Link the KCIs to the preventative and mitigative barriers in the Bowtie Risk Model. KCIs serve as indicators of the effectiveness of the control measures put in place to prevent and mitigate risks. If one of the preventative barriers involves employee training, relevant KCIs could include the percentage of employees who have completed the training or the number of observed near misses. Evaluating Performance and Effectiveness Once the mapping of KPIs, KRIs, and KCIs to the Bowtie Risk Model is complete, organizations can evaluate their performance and effectiveness in risk management through the following steps: Data Collection : Gather relevant data for each indicator from various sources such as performance reports, risk assessments, incident logs, and compliance audits. Data Analysis : Analyze the collected data to assess the performance of KPIs, the trends in KRIs, and the effectiveness of KCIs in meeting the objectives and mitigating risks. Benchmarking : Compare the performance of KPIs, KRIs, and KCIs against established benchmarks or industry standards to gain insights into how well the organization is managing risks relative to its peers. Continuous Improvement: Identify areas where KPIs, KRIs, and KCIs fall short of expectations and use this information to develop targeted improvement strategies. Regularly update and refine the Bowtie Risk Model and associated indicators to stay aligned with changing business conditions and risk profiles. Conclusion Integrating Key Performance Indicators (KPIs), Key Risk Indicators (KRIs), and Key Control Indicators (KCIs) into the Bowtie Risk Model presents organizations with a robust framework to evaluate risk management performance and effectiveness. By mapping these indicators to the relevant components of the Bowtie model, companies can gain valuable insights into their risk landscape, identify potential weaknesses, and enhance their risk management strategies. This proactive approach ensures that organizations are well-prepared to navigate uncertainties, minimize threats, and achieve sustainable success in an ever-evolving business environment.

A simple yet powerful way to discover causes to process problems is using the LEAN 5M+E (or 6M Six Sigma) analysis technique. This approach considers 6 categories that can contribute to problems: Man (human related issues) Machine (computer related) Materials (documents, drawings, standards, specifications) Methods (techniques, approaches, procedures) Measurement (data, units, metrics, KPIs) Environment (mother nature) A combination of using a Fishbone Diagram and 5 Whys can be used to brainstorm through these categories to produce a list of possible causes for the effect (problem) . To get started, a process walk through using the 5M+E as a guide can produce a preliminary list that can serve as a road map for further investigation and process improvement. An Example In highly regulated, high-risk industries organizations must use a Management of Change (MOC) process to assess and mitigate risk due to planned changes. This process typically follows a multi-step procedure which often bottlenecks during the design activities when engineers look to solutions to effect change while maintaining design integrity of the process, assets, and the plant: The following chart is an example of a process review using the 5M+E model looking into why there is a bottleneck in engineering when processing design changes for MOCs (Management of Change): Using this approach, probable causes were identified which were addressed by the process improvement team to reduce the bottleneck. What to look out for What is important and often most difficult when solving problems is coming up with a good problem statement. As some have said, A problem well stated is half solved. In addition, the better the problem is described the better the solutions. Defining the problem with those who are experiencing the problem also yields better results. Those who are most familiar with the process will know best what the issues are and have ideas on how they can be improved. Here are a few questions you might consider when using 5M+E in your process analysis: What processes could benefit from using this technique? Which set of problems generate the greatest impact (you may need to do a Pareto Analysis)? What steps can you take today to start using this approach in your organization?

In the domain of Corporate Ethics and Compliance, one hammer is used to establish ethical behaviour – training. In many cases, this training consists of the transfer of knowledge with respect to rules, regulations, and code of conduct. While this is important, it will not be enough to handle concerns associated with Artificial Intelligence (AI) systems and its use.   What is missing from many organizations are the skills to make ethical decisions which have more to do with values and meaning rather than measurements and metrics – qualitative rather than quantitative decision making.   These skills are no longer valued by some in favour of algorithmic and numerical decision making otherwise known as machine-based decisions.   How will organizations keep humans in-the-loop when the loop no longer involves human decision making?   This is where compliance comes in.   Compliance at its core is an ethical endeavour to stay between the lines and ahead of risk.   When decisions are made to proceed with a course of action (for example, to use or not use AI, in the presence of uncertainty and the possibility of loss or harm, we are making an ethical choice. The commitments that are then made reflect the values we have prioritized.   Unfortunately, all too often, top management goes straight from setting a course of action to specifying boots-on-the-ground tasks. It's no wonder why corporate and ethical compliance are struggling. Ethical dilemmas are not considered, risk is not evaluated, and operational capabilities are not adjusted to stay between the lines and ahead of risk.   Frankly, the message now seems to be, "attend the training and do the tasks."   In the not too recent past, middle management used to do the translation work between top management and boots-on-the-ground activity. But no longer as many have removed these roles to flatten their organizations.   The skill of making ethical decisions now resides with those who directly own the obligations and compliance teams.   That's why I believe our upcoming micro program, "Ethical Decision Making Involving AI" is so important.   You will learn how to make ethical choices supporting responsible and safe AI along with your other compliance obligations.   Can you help us out?   I am looking for compliance practitioners who are interested in trying out this micro program (2 hours / week for four weeks) and provide us with feedback. The number will be limited. Message me to let me know if you would like to participate ( ray.laqua@leancompliance.ca )