The Flight Recorder Problem
Why governed records will not make AI safe
Every commercial aircraft carries a flight data recorder. It is one of the most carefully governed records in any industry. Its design is certified, its contents are protected, and its chain of custody is controlled from the moment it is recovered.
And yet, a flight recorder has never prevented a crash.
That was never its purpose. The recorder helps us learn after something catastrophic has gone wrong. Keeping an aircraft safely in the air is the work of engineering design, maintenance, flight operations, and the crew who fly it.
Flight recorders for AI
Much of the AI governance being proposed today looks like a flight recorder: inventories of AI systems, model cards, risk registers, audit trails, and monitoring dashboards. These records are useful, and they are governed with care. But they are consulted after the fact. Adding more of them only confirms what we already know: many AI systems are not safe, secure, or beneficial so as to create value.
This is evidence compliance presented as governance.
It answers the question “Can we show what happened?” It leaves unanswered the more important question:
“Can we keep the AI system operating within our governance?”
The missing middle
To answer that question we need to understand that compliance has two streams.
Operational compliance creates compliance outcomes, such as better safety, security, quality, sustainability, and legal adherence. It does this by operationalizing obligations as promises at each level of the organization: Governance, Programs, Systems, and Processes.
Procedural compliance creates the evidence that those promises were kept.
Records belong to the second stream.
Record-based AI governance jumps from Governance straight to Processes. Policies are written at the top, and procedures, checklists, and logs at the bottom. Missing are the management and technical programs and systems that regulate AI behaviour, actions, and output in real time.
The Conant-Ashby theorem tells us that every good regulator of a system must be a model of that system. A log only tells us what an AI system has already done. To regulate it, we need a model of how it should behave and the means to keep it there.
What the middle looks like
Consider an organization with AI systems and agents across its operations. Its institutional governance already exists: its values, policies, commitments, obligations, contracts, standards, guidelines, and procedures. That governance is the standard its AI is measured against. An AI Compliance Governance System is a real-time system of regulation that keeps its AI operating within it.
Management programs direct how this system of regulation operates. They operationalize obligations into promises, each with an accountable owner. They determine the capabilities needed to regulate AI to keep their promises in real time: how much drift to contend with, how fast to respond, and which principles to apply. Performance targets, expressed as rates, follow from these. Using feedforward, they act ahead of changes in governance, AI capabilities, and operating conditions. Using feedback, they introduce change when performance falls short.
Technical programs and systems do the regulating. A digital twin of institutional governance, connected to each AI system through digital threads, regulates every dimension:
What each AI system is permitted to do, as set by the policies, contracts, and obligations that apply to it
How well it does it, measured against the standards and guidelines for that work
Where it can do it, limited to the processes, data, and jurisdictions it has been qualified for
The value it contributes, measured against the commitments the business has made
When an AI system drifts along any of these dimensions, it is brought back within governance. Effectiveness is measured by integrity: are the promises designed into each AI system being kept?
Most AI governance programs report something else: AI systems inventoried, use cases assessed, and audit logs collected. These count activity and tell us little about whether AI is operating within our governance. This is the Compliance 1 trap.
Fit to fly
An aircraft is made airworthy before it flies and kept airworthy while it flies. The recorder is there to help us learn when something goes wrong anyway. AI deserves the same order of work: engineered to operate within governance, then regulated to stay there.
That work is unlikely to come from the AI technology vendors. It will need to be done by the engineers who design and build AI systems for the enterprise, most often within the organizations that use AI in their business. This is how trust in AI is earned, and trust is what allows an organization to rely on AI to do real work.
Keep the flight recorder. First, make sure your AI is fit to fly.
At Lean Compliance, this is the purpose of Forward Assurance: regulating AI in real time so it stays within your organization’s existing institutional governance.
You can learn more at leancompliance.ca/forward-assurance.
