What an AI execution runtime does that a workflow engine doesn't

The claim waited 36 hours. The rules didn't.

An insurer uses AI to propose property claim settlements. A claim arrives, policy and loss data are pulled, a model recommends a figure, rules calculate limits, and an adjuster approves a payment. The workflow then waits for the payment service.

One claim sits paused for 36 hours. In that time the prompt template changes, the model route is updated, a policy threshold moves and a payment mapping is republished. When the case resumes, the screen still shows an approval, but the runtime picks up the latest definitions. The proposal that was approved and the one that executes are no longer provably the same.

The bank accepts the payment, but the connection drops before the response arrives. The workflow retries. With no stable operation ID, the bank accepts the second request too. The model answered, the approval was recorded, the retry returned success. The claim was paid twice.

This isn't a bad prompt, a model error or an outage. It's a runtime that didn't keep one version from proposal to payment, didn't treat approval as durable state over a fixed proposal, and didn't classify the payment before retrying it.

Isn't "workflow" enough?

Not as a label. "Workflow" covers everything from a four-step automation to an agent loop, and it doesn't tell you whether identity, policy, approvals, versions and side effects are actually enforced.

Workflow technology isn't primitive. Process standards such as BPMN and decades of workflow research cover branching, parallelism, exceptions, timers, human tasks, events, compensation and long-running state. Durable workflow engines survive crashes, infrastructure failures and long human delays.

The problem is that saying an application "has a workflow" tells you nothing about whether model output is typed, whether identity survives each hop, whether policy can stop an action, whether approvals are tied to a fixed proposal, whether replay can avoid live side effects, or which version a paused case resumes on. Those answers may exist, spread across code, gateways, policy engines, queues, prompt stores and audit tables. The label doesn't promise they're held in one place.

That's the gap AI widens. When a step decides its own path at runtime, the order of steps tells you less, and the guarantees around each step tell you more.

What is an AI execution runtime?

It's the system that runs one published, versioned workflow and keeps its guarantees across time and failure: authority, policy, typed boundaries, approvals, side-effect rules and evidence, from the first step to a confirmed result.

The visual workflow, the stored definition, any compiled form and the run evidence can all look different. They must all resolve to the same versioned meaning. If the diagram changes and the running logic doesn't, the diagram is documentation. If generated code can't be traced back to the workflow, the workflow is a sketch.

A published workflow version holds, directly or by fixed reference: typed steps and their contracts; connections with declared data, control, error and compensation behavior; mappings and defaults; identity, tenant and policy requirements; wait, timeout and approval behavior; side-effect classes and retry rules; pinned versions of prompts, model routes, API specs, policies and sub-workflows; and the evidence each consequential step must leave.

Four things that share one name

Many lifecycle bugs start when teams call four different things "the workflow."

Replay and retry are different operations with different authority. A retry tries to finish the same action. A replay looks back and should never pay anyone.

Steps are capability boundaries

A step is more than a function name. The same box on a canvas might be a calculation, a query, a document search, a model call, a policy decision, an API write, a human task, an event wait or a sub-workflow. Each needs different checks and leaves different evidence.

The runtime doesn't need to know how each capability works inside. It does need to know enough about the boundary to validate the request, enforce policy, schedule the work, classify failure, keep state and attach evidence.

Reusable sub-workflows follow the same rule: they're versioned capabilities with a declared boundary, not boxes copied between canvases. If a new sub-workflow version adds data access, a different model provider, a broader credential or an irreversible write, that changes what the parent must govern. And a published parent should pin the exact sub-workflow version. "Latest" is fine in a draft and indefensible in a paused case.

Connections carry data, authority and conditions

A line between a model and a payment API shouldn't mean "send whatever the last step returned." It should mean: validate this proposed action against this schema, derive these fields through these mappings, keep these classifications, check these policy facts, require this approval, call this versioned API under these side-effect rules, and record this evidence. JSON Schema, OpenAPI, AsyncAPI and CloudEvents give you mature building blocks for each part.

Authority crosses the line with the data. The runtime doesn't need to forward one broad user token everywhere. It can use narrower, audience-bound credentials and delegation records while keeping a verifiable chain back to the person whose authority is being used, for what purpose, over which resource. (See AI agent identity and delegated authority.)

Policy attaches to both steps and connections. A policy step decides whether a settlement is allowed. A rule on a connection decides whether certain fields may reach an external model. The policy engine can stay a separate system. The workflow defines where the decision belongs, which facts it gets and how its result changes what happens next.

Can typed contracts make model output safe?

They make it checkable, not correct. A schema-valid payload can still be wrong or unauthorized, but typed boundaries let the runtime tell invalid structure from denied policy, missing evidence or an expired approval.

Putting a model inside a workflow doesn't make it deterministic. The point is to put deterministic checks around probabilistic output. Before a model's recommendation becomes trusted state or reaches an API, the runtime requires structural validation, semantic checks, policy, thresholds and, where the rules say so, a person.

Without those distinctions, every defect collapses into "the AI failed." With them, you know which boundary stopped the work and why. Contract changes matter too: if a sub-workflow still emits version 2 of a claim object while the next step expects version 3, publishing should catch it, not production.

State has to survive time, failure and people

Enterprise work rarely finishes in one call. It waits for documents, callbacks, approvals, fraud reviews and customers. The runtime has to keep state without holding a thread open or relying on a process that stays alive.

A paused case should keep its published version, its current step and state, pending correlation keys, identity and policy context, the proposal awaiting approval and its evidence, deadlines and the transitions allowed on resume. Keep business state (claim status, approved amount) separate from runtime state (pending event, attempt count, committed effect), so the runtime can check that an incoming event matches the expected case and transition instead of waking up because a familiar ID arrived.

Retries aren't side-effect rules

A remote system can commit while the response is lost. A worker can crash after a write but before recording it. An event can arrive twice. Retrying everything isn't resilience; for writes that can't be safely repeated, it's duplication. HTTP itself warns against automatically retrying non-idempotent requests without knowing they're safe.

So classify each external effect: no external change; read-only; safe to repeat; protected by a stable operation ID; reversible by a defined counter-action; or irreversible and manual. The class decides retries, replay, approvals, timeouts and escalation. A refund doesn't erase a payment; it's a new governed action. And "exactly once" across arbitrary systems is a promise few can keep. What you can keep is explicit behavior under uncertainty: stable IDs, reconciliation before retry, and a person when no safe automation exists. (See retries and idempotency.)

Publishing is an architectural boundary

Long-running cases expose a version problem short requests can ignore. Publishing should freeze or pin everything needed to reproduce a run: steps, contracts, mappings, prompts, model routes, policies, sub-workflows, retry and timeout rules, effect classes and evidence requirements.

A running case stays on its published version unless someone migrates it, and a migration is a governed operation with its own compatibility checks, policy implications, approval consequences and rollback. Silently adopting the latest definition isn't evolution. It's losing track of what ran. (See versioning AI workflows.)

Publishing can also compile the workflow into an efficient runtime plan: precompiled mappings, resolved references, scheduling metadata. The rule is that optimization never removes a policy check, an evidence requirement or an effect class, and every runtime event still maps back to a step people can read.

What stays outside the runtime

The runtime doesn't absorb everything. Algorithms, connectors, specialized models, policy engines, databases, message brokers, identity providers and enterprise systems stay independently engineered. The workflow references them through contracts and versions. It doesn't reimplement the policy engine; it says where the decision belongs, feeds it the facts, enforces the result and records it. Lineage tools keep their job and run records; the workflow connects them to decisions, approvals and effects. The workflow owns what the behavior means. Components own how they do their part.

Common anti-patterns

Ten rules for a governed AI runtime

1. Run the same versioned definition people reviewed.
2. Keep draft, published version, run and replay separate.
3. Declare a contract for every step, including model calls.
4. Carry identity and purpose across every connection.
5. Put policy where the decision belongs, and record that it was enforced.
6. Bind approvals to a fixed proposal.
7. Keep paused cases on the version that started them.
8. Classify each external effect before deciding how to retry.
9. Pin sub-workflow versions at publish.
10. Never let replay repeat a committed effect.

The result: a runtime that can prove what it did

The difference between a workflow and a governed runtime isn't the number of steps, the presence of a model or a visual builder. It's whether there's one authoritative, versioned definition whose guarantees hold across time and system boundaries. A mature workflow engine supplies much of the foundation. Agent frameworks coordinate models and tools. Policy, identity, lineage and observability tools each do their part.

The runtime ties them together so you can answer, for any case: which definition ran, what crossed each step, whose authority was used, which policy could have stopped it, what was approved, what happened on timeout, which effects committed and what evidence remains. If the runtime can't prove those, it's orchestration, not governance.