Which Uses of AI Need Engineering Discipline in 2027?

AI needs engineering discipline wherever it is used inside safety-critical or business-critical operations. In those uses, responsible AI requires four things: the intended behaviour is specified, the use is bounded, the system is verified and validated before anyone relies on it, and a qualified person owns all three. ISO/IEC 5338 provides the life cycle backbone for this work, supported by standards for functional safety, security and operating envelopes.
Why is AI treated as something to study instead of something to engineer?
At The Curve 2026, Helen Toner of Georgetown's Center for Security and Emerging Technology proposed "minimum viable superintelligence." Her aim was to make superintelligence concrete by extrapolating from today's systems: more modalities, longer time horizons, continual learning, more speed, more memory and many copies working together. It is careful work. It is also a forecast. Systems arrive in her account, and the task is to watch for the signs of their coming.
That stance treats AI development as a natural phenomenon to observe. Yet every capability results from decisions made by labs, investors, engineers and governments about what to build, how fast and with what limits. Forecasts also shape what they predict. When credible people say superintelligence is coming, capital and policy move to make it so. In September 2026, Executive Order 14434 directed U.S. federal agencies to use "Super Intelligence" in place of "Artificial Intelligence," while defining it by the existing statutory definition of AI. Superintelligence now carries three meanings: Nick Bostrom's system that outperforms humans at everything, Toner's extrapolation from today's systems, and a legal label for all AI. None comes with criteria a system could be tested against.
The Hugging Face incident in July 2026 shows where this leads. According to the published post-mortems, agents in an internal OpenAI training run found a way to the internet and attacked Hugging Face's production systems while trying to game a benchmark. OpenAI had detected the agents exploiting its own internal systems weeks earlier. It was an experiment without containment, and it was understood only afterward.
Incidents like this are often explained away as the unpredictable nature of AI, as though its behaviour were beyond anyone's understanding. That explanation does not hold up. The mechanisms of AI are known. Many describe AI as probabilistic, and that word bundles several things together. A trained model is a deterministic function, one that computes a probability distribution over possible outputs, and for the same input it computes the same distribution every time. Variation comes from sampling from that distribution, a design choice set by parameters such as temperature. Small differences can also arise from the order in which GPUs carry out floating-point calculations, an implementation matter that can be engineered out. The rest is our inability to predict the outcome in advance, which we describe with statistics. None of this is chance in nature.
What is missing is a specification of what the system should do. Without one, testing becomes observation and monitoring has no reference. That reference can only come from people. In every loop that governs AI, from a person refining a prompt to a monitor checking a deployed system, a human defines what the output should be and judges whether it was delivered. Humans are the system of regulation for AI. The question for every organization is where that regulation must be engineered, and the answer depends on how each use of AI is classified.
How should organizations classify their uses of AI?
Sort technology along two lines. The first is what it works with. IT handles data and information. I²T, or intelligence technology, handles knowledge and intelligence. The second is how it is used. General use supports productivity. Operational use supports safety-critical and business-critical work.

The general row can be governed through management practice, with a person checking each output. The operational row must be engineered. Use decides the row. The same model can serve as a productivity assistant in one place and inside a control system in another.
What does responsible AI require in operational use?
Anything in the operational row must be engineered. For AI, that comes down to four requirements:
Specified. The intended behaviour, acceptance criteria and tolerances are stated, including any variability designed in through sampling.
Bounded. Use is limited to an operating envelope where that behaviour can be verified, with constrained interfaces, a defined range of action and, where possible, a model reduced to the scope of work.
Verified and validated. Outputs are checked against the specification before anyone relies on them, and the system is shown to serve its intended use in its operating environment. Where certainty is required, AI should not generate new outputs in operation. Its role is limited to design time, where it can help produce an artifact such as code or a configuration. Certainty comes from verifying, validating and freezing that artifact, so the system in operation runs a known, tested result.
Owned. A qualified person is accountable for the specification, the envelope and the verification and validation record.
Each requirement follows from how these systems work. Without a specification there is no test oracle, so a test can only report what happened. Exhaustive verification is out of reach: verifying even simple properties of neural networks is NP-complete (Katz et al., 2017), and the behaviour of agentic systems is undecidable in general under Rice's theorem. Bounding the use, and the model, to the scope of work is what makes verification and validation feasible. Finally, a system cannot answer for its own outputs, so a qualified person must own the evidence and answer for the outcome.
Cybernetics gives the same answer. The Conant-Ashby good regulator theorem says a regulator must contain a model of what it regulates. For AI, that model is the intended behaviour, written down. Ashby's law of requisite variety says a regulator needs at least as much variety as the system it must control. A general-purpose model produces almost unlimited variety, so the operating envelope must reduce it to something people can regulate.
When iteration happens also matters. Using AI to generate code or a design many times over is useful when a person verifies the result before it is used. At run time, an agent that searches for solutions by trial and error is experimenting on a live system. A control system regulates a live system in real time using a control law designed and proven in advance. Operational I²T must behave like a control system.
Which standards apply to operational I²T?
ISO/IEC 5338:2023 provides a strong backbone for operational I²T. It defines life cycle processes for AI systems, built on the systems and software engineering standards ISO/IEC/IEEE 15288 and 12207, with AI-specific processes for data engineering and continuous validation. It supports specification through its stakeholder needs and requirements definition processes, and verification through its verification, validation and continuous validation processes. ISO/IEC 42001 complements it by governing AI through a management system, which suits the general row. For high-risk uses under the EU AI Act, EN 18286:2026 adds the quality management system. Its citation in the Official Journal in October 2026 gives providers a presumption of conformity with the Article 17 requirements it covers. It is built around the AI system life cycle, so it works alongside ISO/IEC 5338.
ISO/IEC 5338 defines process. Operational I²T also needs:
An operating envelope, following the approach ISO 34503 takes for the operational design domain of automated driving
Functional safety where it applies, using ISO/IEC TR 5469 and IEC 61508
Security obligations carried over from OT through IEC 62443
Risk input to the design basis from ISO/IEC 23894
A quality management system for high-risk uses, using EN 18286:2026
An accountable professional who owns the specification, the envelope and the verification and validation record
Continuous validation should connect to management of change. Management of change is the practice, long established in process safety and other regulated industries, of reviewing any change to an operating system for its risks, approving it and documenting it before it is made. Retraining a model in operation is a design change, and it deserves the same review as any other.
What should compliance and engineering leaders in regulated industries do next?
Start by sorting your AI uses into the four categories. Most organizations are likely to find their assistants and copilots in the general row, and a smaller set of uses already drifting into operations. That drift is the immediate risk. Moving AI from general to operational use is a change in obligations. Route it through engineering practice and management of change with the same care you would give a modification to a plant.
For each operational use, name the owner, write down the intended behaviour, define the envelope and decide how outputs will be verified before reliance. Engineering creates certainty where it is needed and contends with the risk that remains.
Regulation is moving the same way:
EU AI Act: obligations for high-risk AI apply from December 2, 2027 for stand-alone systems, and from August 2, 2028 for AI embedded in regulated products.
EU Product Liability Directive: the revised directive brings software, including AI systems, within product liability, with member states due to apply it from December 2026.
Organizations that already specify, bound, verify, validate and own their operational AI will be ready. The discipline that delivers these outcomes is engineering practice, and it is the practice that earns trust.




