Using Key Performance Indicators (KPIs) to regulate and drive operational functions is table stakes for effective organizations and for those that want to elevate their compliance.
In a recent report by MIT Sloan Management Review and Boston Consulting Group (BCG), “The Future of Strategic Management: Enhancing KPIs with AI” the authors provide the results of a global survey of more than 3,000 managers and interviews with 17 executives to examine how managers and leaders use AI to enhance strategic measurement to advance strategic outcomes. More specifically, their study explores how these organizations have adopted KPIs and created new ones using AI.
In this report the authors categorize AI-enhanced KPIs in the following way:
Smart Descriptive KPIs: synthesize historical and current data to deliver insights into what happened or what is happening.
Smart Predictive KPIs: anticipate future performance, producing reliable leading indicators and providing visibility into potential outcomes.
Smart Prescriptive KPIs: use AI to recommend actions that optimize performance.
Furthermore, the report identifies that developing smart KPIs requires categorizing variables into three distinct types:
Strategic Outcome Variables: well-known overarching targets, such as revenue or profit.
Operational Drivers Variables: that might impact the strategic outcome, such as pricing, consumer reviews, or website traffic.
Contextual Factors: external factors beyond a company’s control, typically measured or tracked through external data such as consumer spending forecasts, inter-country freight, or government regulation.
While there is some evidence that KPIs can be enhanced, the report suggests the need for a shift in mindset and practice with respect to the category of KPIs:
From Performance Tracking to Redefining Performance
From Static Benchmarks to Dynamic Predictors
From Judgment-FIrst to Algorithmically Defined Strategic Metrics
From KPI Management to Smart KPI Governance and Oversight
From Keeping an Eye on KPIs to KPI Dialogues and Discussion
From Strategy with KPIs to Strategy for and with KPIs
To facilitate these transitions (or disruptions) the authors of the report provide several recommendations:
Realign Data Governance to Enable Measurable Smarter KIPs
Establish KPI Governance Systems
Use Digital Twins to Enhance Key Performance Metrics
Prioritize Cultural Readiness and People-Centric Approaches
Strategical Alignment with Smart KPIs
My Thoughts
In general, Key Performance Indicators (KPIs) should by definition have predictive utility which separates them from set of metrics that one might otherwise measure.
The three categories for KPIs outlined in the report suggest how KPIs might be used given their predictive quality. KPIs with low correlation might help describe what's happening but are not good candidates for a KPI compared with those with significant correlation. However, even good KPIs cannot suggest how to effect performance changes.
Making systems changes relies on knowledge of what measures of effectiveness, performance, conformance, and assurance are targeted along with understanding of the underlying concept of operations.
Notwithstanding, the use of AI does hold promise to help with lagging indicators to find new and different correlations. However, leading indicators is a different story. Leading indicator are the holy grail of operational performance and require knowledge of what should be rather than only what once was. Data describing this knowledge seldom appears in operational records or logs and would need to be integrated with an AI System.
Without controlled experiments causation should always be treated with a grain of salt. We need to be mindful that the future is not as deterministic as some may believe. When there is human agency involved the future is open, not closed or bound to AI predictions.
It's helpful to remember that there are other forces at work:
You can’t turn lagging indicators into leading indicators. (Risk Theory)
You can’t turn an “is”, description of what is, into an “ought”, a prescription of what should be. (Hume’s Law)
A system will always regulate away from outcomes you don’t specify. (Ashby’s Cybernetics Law of Ethical Inadequacy)
When a measure becomes a target, it ceases to be a good measure. (Goodhart’s Law)
What steps should be followed when using AI for KPIs?
Instead of considering AI as a solution looking for a problem, first identify the problem that is in need of solving.
Do you have a problem with:
Decision making?
Execution or follow-through?
Conformance or regulation?
Lack of understanding of operational systems, processes, and behaviours?
Uncertainty and risk?
Insufficient or untapped performance?
When the problem is a lack of quality KPIs then one might consider establishing a Smarter KPI Program. The report by MIT-BCG makes an important point that is worth repeating. What they suggest is not so much about creating better KPI's as it is about establishing an on-going set of processes, practices and mindset to use algorithmically defined metrics. This requires more than following a procedure.
The following questions will help define the context for such a program:
What do better KPI’s look like?
What strategy should we follow to achieve that?
What capabilities do we need to support this strategy?
What obstacles or opportunities need to be negotiated or exploited?
What measures will be used to define success?
