Are You Auditing What Really Matters?


When discussing compliance you will quickly hear the word audit. In fact, for many, it means the same thing, the compliance function is synonymous with the audit function.


Audits have been used for many years to confirm the integrity of financial statements and that proper accounting procedures have been used.


You could say that auditing has provided both a quality control along with a quality assurance function usually conducted by third parties.


The American Society for Quality (ASQ) defines these functions as follows:


Quality Control – can be defined as "part of quality management focused on fulfilling quality requirements." While quality assurance relates to how a process is performed or how a product is made, quality control is more the inspection aspect of quality management. An alternate definition is "the operational techniques and activities used to fulfill requirements for quality."


Quality Assurance – can be defined as "part of quality management focused on providing confidence that quality requirements will be fulfilled." The confidence provided by quality assurance is twofold—internally to management and externally to customers, government agencies, regulators, certifiers, and third parties. An alternate definition is "all the planned and systematic activities implemented within the quality system that can be demonstrated to provide confidence that a product or service will fulfill requirements for quality."


In recent decades auditing has also become table stakes not only for quality, but also safety, environmental and regulatory management systems. As with accountancy, the auditing function does not evaluate the effectiveness of your financial system, nor does it do so for quality, safety, or environmental systems. Auditing only confirms that you are following acceptable practices (usually defined by a standard) and the outputs of the system have not been interfered or tampered with.


The evaluation and auditing of system effectiveness is not part of the auditing or compliance function, so which function is it a part of and what should it be auditing?


A Shift that Shouldn't be Ignored


Before we look at the answer to these questions, we first need to recognize a shift that is happening with respect to regulatory designs.


Increasingly, regulatory and standards bodies are transforming their operations, taking on a more risk-based approach focused on outcomes and continuous improvement. This has resulted in the introduction of regulations and standards that are moving away from prescriptive to performance and risk-based requirements.


Organizations are expected to establish their own means (the how) by which they will achieve targeted goals and objectives. This affords greater latitude for organizations to better address complex and systemic problems. It also holds them accountable for the outcomes of their systems, where in the past they have only been responsible for the outputs of prescriptive requirements.


This shift has in many cases come with much confusion. It is not uncommon to find performance-based frameworks including prescriptive "shall" statements related to "how" it should be done. After years under the tutelage of prescriptive regulation the pull towards having something to audit is very strong which while understandable creates confusion for those adopting new and updated regulations and standards.


What Then Should Be Audited?


Increasingly, obligations that arise from regulation along with industry standards are requiring that organizations make progress towards what is often called, Vision Zero targets. These include zero harm, zero fatalities, zero incidents, zero emissions, zero violations and so on. Advancing these goals requires risk-based approaches and the continuous improvement of capabilities to generate appropriate levels of performance for progress to be made.


When we now think about compliance we should be considering the goals that are being targeted. An important distinction that can made is between "terminal" and "instrumental" goals.

Terminal goals are the highest level objective that we want to reach. They define the "ends" of our compliance programs, for example: zero defects, zero fatalities, zero violations, zero releases, zero fines, and others.

Instrumental goals are intermediate outcomes or results that are critical or that must occur in order to achieve the higher-level outcome. These are often used to define Measures of Effectiveness (MoE) for compliance programs as they provide clear indication of progress towards terminal goals. Measures of Effectiveness can be used to validate compliance programs to ensure that they are fit for the purpose of advancing outcomes.


Performance-based Compliance Management Framework

The following are Measures of Success for compliance frameworks that support performance and outcome-based obligations (see previous figure):


  • Measures of Effectiveness (MoE) – critical to program success, independent of any technical implementation (i.e. the how). Focuses on the ends not the means.

  • Measures of Performance (MoP) – measures that relate to the operations of the compliance program, systems, and processes. These are the measures of capabilities needed to be effective.

  • Measures of Conformance (MoC) – critical to compliance, where failure maybe cause for reassessment of the program. These tend to be prescriptive legal requirements but may include voluntary practices.


Auditing has traditionally been helpful to verify Measures of Conformance but now needs to support Measures of Performance and Effectiveness. The latter is the task of governance and program management. Together they identify the destination and then steer the organization towards it. To be effective they need compelling answers to these questions:


  • Where should we be heading?

  • How will we get there? What is our strategy?

  • What capabilities and resources do we need to get there?

  • What obstacles are in the way?

  • How will we measure our progress?


The audit function now forms a validation function connected with progress towards targeted outcomes rather than only conformance to shall statements. Those in the pharma and medical device industry will recognize this distinction between verification and validation. You can build a pacemaker that meets all design specifications (which you can verify) and yet fails to keep your heart pumping. This is precisely the shift that is happening with safety, environmental and regulatory objectives. You can build a system that conforms to all the standards and yet fails to make any progress on outcomes. This is why compliance now should audit outcomes over outputs.



Additional Reading:


https://www.leancompliance.ca/post/four-misuses-of-audits

https://www.leancompliance.ca/post/four-steps-to-proactive-compliance


Lean Compliance helps companies adopt and improve compliance systems to better meet performance and outcome-based obligations.

We offer specialized programs and training tailored to fit each company's size and capabilities. 

Schedule a call with us today to find out which programs are best for you.  You can book your appointment here.

© 2020 Lean Compliance™

All rights reserved.