Regulating AI with Institutional Knowledge

Today many organizations use AI that is general. It was trained on the public record, not on any one sector or business. It does not know your mission, your values, your goals, your processes, your protocols, or your standard operating procedures. When you ask it a question, it answers from what is common across everyone, not from how that knowledge applies to the particulars of what you do.

In a regulated, high-risk domain this is a problem.

The general model gives you the most common answer, which may not be a standard, and is rarely yours. It reverts to common knowledge when common practice is the wrong practice for your plant, your patient, or your portfolio. And it does this without warning. It is as confident when it is guessing as when it is correct. An advisor that cannot tell you when to doubt it is not an advisor.

The answer is not a smarter model. The answer is to govern the model with your knowledge. Curation of knowledge — including institutional knowledge — becomes an essential part of governing AI.

This is not a new problem. The medical profession has already faced the challenge of using AI where a wrong answer carries real consequences, and what the medical community is doing transfers to any sector.

What medicine did

The clinical field did not build a model to replace doctors. According to a 2025 study in Bioengineering (Pingua et al.), trying to train medical knowledge directly into the AI worked worse than keeping that knowledge in a separate, trusted source the AI looks up when answering.

The approach comes down to three things:

keep the knowledge separate from the AI, organize it and rank it by authority, and ground every answer in it with a qualified person accountable for the decision.

The model provides the reasoning. The organization provides the knowledge. Trust comes from keeping the intelligence under the organization's control, not the model's.

Building the regulator

Medicine showed three of these parts: keep the knowledge separate, rank it by authority, and put a qualified person in charge. An engineered regulator adds a fourth — a check that catches the model when it deviates from institutional knowledge — which is the part most deployments, medical ones included, still leave out. Together these form a structure that gives the model an authoritative reference to work from, checks its answers against that reference, and keeps a person accountable.

The method does not depend on the industry, but the particulars will. Replace the medical content with your own and the structure applies — for engineering, finance, energy, aviation, pharmaceuticals, or government.

A regulator has four parts.

1. The reference: your authoritative knowledge

This is what the model must conform to — your specialized knowledge together with your obligations, values, and goals. It is the standard the advisor is measured against, so it has to be built deliberately.

• Set the order of authority first. Decide what the advisor is for, and which sources take precedence when they disagree: your current procedures and standards first, your approved records next, then sector guidance, then general knowledge last. This order is the governing rule. It stops general knowledge from overriding your specific obligations.

• Curate the knowledge, including institutional knowledge. Gather the authoritative material — mission, values, processes, procedures, standards, and the specialized knowledge of how the work is actually done — and tag each source with its authority, its owner, and its version. Do not dump documents into the system. Ungraded knowledge makes the advisor less reliable, not more. The curation is most of the work and most of the value.

• Tie the reference to your change process. Obligations, values, and goals change. When they do, the curated sources are updated through the same controlled process that governs every other change in the organization. This keeps the advisor current, and keeps the reference visible and owned rather than buried in the model.

2. The grounded model: answers drawn from the reference

The model supplies the reasoning, but it must reason from your reference, not from the public average.

• Ground answers in your knowledge, not in the model. Have the advisor answer from the curated sources and cite them. Keep the knowledge external and current, never built into the model, where it would freeze and fall out of date.

• Adapt the model for communication, not for rules. Adjust it to use the terms and format of your field. The rules, limits, and procedures stay in the reference. If something would change tomorrow, it belongs in the reference, not the model.

3. The comparator: detecting deviation

This is the part most systems leave out, and the part that makes a regulator a regulator. A comparator is simply a check that measures one thing against another. Here, the model's answer is compared against the reference to detect when it has deviated — substantively wrong against your knowledge, or in breach of an obligation.

• Check the output independently. The check cannot be the same model grading its own answer; it shares the same blind spots. Deviation is detected by a separate step — rules for the firm limits, comparison against the reference for everything else.

• Require citations, and make refusal the default. When there is no authoritative source for a question, the advisor says so. It does not improvise. A missing source is itself a signal that the answer is outside the reference.

• Calibrate, and keep it calibrated. Confirm that the advisor's confidence matches its actual reliability — certain where it is grounded in an authoritative source, hesitant or silent where it is reaching. Then maintain that match as the organization and the questions change. Validation is a gate you pass once. Calibration is a discipline you keep.

4. The accountable person

A regulator needs an owner — in fact, two kinds.

• A qualified person owns every decision. The advisor advises. A competent person makes the call and is accountable for it. The model gets them to the right answer faster; it does not decide.

• A qualified person owns the reference. Someone must own what counts as authoritative knowledge and what counts as deviation, and keep it current as the organization changes. The reference does not maintain itself.

Path forward

A general AI cannot be trusted in a high-stakes domain because it is general. It does not know you, and it does not know when it is wrong relative to your situation. Closing that gap does not require a better model. It requires building a regulator around the one you have — an authoritative reference of your own knowledge, answers grounded in that reference, an independent check for deviation, and an accountable person who owns it.

Start with the reference. The knowledge, the judgment, and the accountability are already yours; the work is making them explicit enough for the model to be measured against them. The model supplies general reasoning. The trust comes from the regulator you build around it.