top of page

BLOG POST

Are AI-Enhanced KPIs Smarter?

Using Key Performance Indicators (KPIs) to regulate and drive operational functions is table stakes for effective organizations and for those that want to elevate their compliance.


In a recent report by MIT Sloan Management Review and Boston Consulting Group (BCG), “The Future of Strategic Management: Enhancing KPIs with AI” the authors provide the results of a global survey of more than 3,000 managers and interviews with 17 executives to examine how managers and leaders use AI to enhance strategic measurement to advance strategic outcomes. More specifically, their study explores how these organizations have adopted KPIs and created new ones using AI.


Are AI-Enhanced KPIs Smarter?
Are AI-Enhanced KPIs Smarter?

In this report the authors categorize AI-enhanced KPIs in the following way:


  • Smart Descriptive KPIs: synthesize historical and current data to deliver insights into what happened or what is happening.

  • Smart Predictive KPIs: anticipate future performance, producing reliable leading indicators and providing visibility into potential outcomes.

  • Smart Prescriptive KPIs: use AI to recommend actions that optimize performance.


Furthermore, the report identifies that developing smart KPIs requires categorizing variables into three distinct types:


  1. Strategic Outcome Variables: well-known overarching targets, such as revenue or profit.

  2. Operational Drivers Variables: that might impact the strategic outcome, such as pricing, consumer reviews, or website traffic.

  3. Contextual Factors: external factors beyond a company’s control, typically measured or tracked through external data such as consumer spending forecasts, inter-country freight, or government regulation.


While there is some evidence that KPIs can be enhanced, the report suggests the need for a shift in mindset and practice with respect to the category of KPIs:


  • From Performance Tracking to Redefining Performance

  • From Static Benchmarks to Dynamic Predictors

  • From Judgment-FIrst to Algorithmically Defined Strategic Metrics

  • From KPI Management to Smart KPI Governance and Oversight

  • From Keeping an Eye on KPIs to KPI Dialogues and Discussion

  • From Strategy with KPIs to Strategy for and with KPIs


To facilitate these transitions (or disruptions) the authors of the report provide several recommendations:


  1. Realign Data Governance to Enable Measurable Smarter KIPs

  2. Establish KPI Governance Systems

  3. Use Digital Twins to Enhance Key Performance Metrics

  4. Prioritize Cultural Readiness and People-Centric Approaches

  5. Strategical Alignment with Smart KPIs


My Thoughts


In general, Key Performance Indicators (KPIs) should by definition have predictive utility which separates them from set of metrics that one might otherwise measure.


The three categories for KPIs outlined in the report suggest how KPIs might be used given their predictive quality. KPIs with low correlation might help describe what's happening but are not good candidates for a KPI compared with those with significant correlation. However, even good KPIs cannot suggest how to effect performance changes.


Making systems changes relies on knowledge of what measures of effectiveness, performance, conformance, and assurance are targeted along with understanding of the underlying concept of operations.


Notwithstanding, the use of AI does hold promise to help with lagging indicators to find new and different correlations. However, leading indicators is a different story. Leading indicator are the holy grail of operational performance and require knowledge of what should be rather than only what once was. Data describing this knowledge seldom appears in operational records or logs and would need to be integrated with an AI System.


Without controlled experiments causation should always be treated with a grain of salt. We need to be mindful that the future is not as deterministic as some may believe. When there is human agency involved the future is open, not closed or bound to AI predictions.


It's helpful to remember that there are other forces at work:


  1. You can’t turn lagging indicators into leading indicators. (Risk Theory)

  2. You can’t turn an “is”, description of what is, into an “ought”, a prescription of what should be. (Hume’s Law)

  3. A system will always regulate away from outcomes you don’t specify. (Ashby’s Cybernetics Law of Ethical Inadequacy)

  4. When a measure becomes a target, it ceases to be a good measure. (Goodhart’s Law)


What steps should be followed when using AI for KPIs?


Instead of considering AI as a solution looking for a problem, first identify the problem that is in need of solving.

Do you have a problem with:


  • Decision making?

  • Execution or follow-through?

  • Conformance or regulation?

  • Lack of understanding of operational systems, processes, and behaviours?

  • Uncertainty and risk?

  • Insufficient or untapped performance?


When the problem is a lack of quality KPIs then one might consider establishing a Smarter KPI Program. The report by MIT-BCG makes an important point that is worth repeating. What they suggest is not so much about creating better KPI's as it is about establishing an on-going set of processes, practices and mindset to use algorithmically defined metrics. This requires more than following a procedure.


The following questions will help define the context for such a program:


  1. What do better KPI’s look like?

  2. What strategy should we follow to achieve that?

  3. What capabilities do we need to support this strategy?

  4. What obstacles or opportunities need to be negotiated or exploited?

  5. What measures will be used to define success?


21 views

Comments


Become a Member

Lean Compliance Member

$30

30

Every month

Access to Exclusive Resources and Programs

Valid until canceled

Access to Recorded Webinars

Access to Exclusive Content (worksheets, templates, etc.)

Access to Exclusive Articles

Access to Exclusive Resources

Access to Elevate Compliance Huddle Worksheets and Content

50% Off First Compliance Consultation ($225 value)

Elevate Compliance Huddle

Mondays @ Noon on Zoom (weekly)

Elevate Compliance Huddle / Free Online Session

bottom of page