Why source code is no longer the full record of an AI application

The diff was empty. The application had changed.

An insurer uses an AI-assisted app to decide whether a low-value claim can be settled automatically or needs a specialist. A customer challenges a decision, and engineering is asked the usual question: what changed?

The repository shows nothing relevant. The same commit has been in production for six weeks. Tests and pipeline are green. The branch that sends claims to manual review is exactly where it was.

Yet the behavior changed. In those six weeks, a model alias moved to a newer provider release. A prompt fragment in a separate registry was revised. The search index was rebuilt from an updated claims manual. A feature definition changed how prior incidents count. A policy bundle raised the threshold for specialist review. A mapping started defaulting a missing field differently. One tenant moved to a different inference region.

None of it appeared in the diff. An engineer can show the line that calls the classifier and the branch that follows it. Nobody can show, from the repository, which model, prompt, evidence, feature values, policy revision, mapping and route governed this claim.

The problem isn't that code stopped mattering. It's that the team was using one artifact to answer a question that now spans several.

Was source code ever the whole system?

Not literally, but for deterministic apps the important behavior sat in code and nearby configuration. AI moves much of it into prompts, models, policies and context that change on their own schedules.

Production systems always depended on databases, infrastructure, secrets and operations. But for a deterministic app, the repository, the build and the database could explain most of what happened. Cloud-native practice already loosened that: declarative infrastructure separates desired state from live state, and build provenance separates the source revision from the artifact actually built.

AI widens the gap. An AI application's behavior depends on things that are versioned separately, deployed separately, resolved dynamically or created only at runtime:

• model, provider release, route and fallback;
• prompts, instructions, tool descriptions and output contracts;
• search indexes, ranking, context rules and freshness;
• feature definitions and the values on the day;
• policy revisions, decision inputs and enforcement;
• mappings, queries and schemas;
• tenant, purpose, residency and delegated authority;
• waits, approvals, timers, retries and events;
• region, environment and endpoint;
• the actual inputs, outputs, writes and evidence.

That's what the shift to deciding steps does. When a step chooses its path at runtime, the things that steer it (context, prompt, model, policy) become part of the application whether or not they live in the repo.

What can a repository prove, and what can't it?

A repository proves how each capability is implemented, reviewed and tested. It can't prove what was deployed, what resolved at runtime, what entered the context, what was approved or what changed in other systems.

A well-run repository establishes the algorithm, its review history, its tests, its declared dependencies, its types and its internal error handling. Those are foundational controls.

It doesn't establish which build was deployed, which model alias resolved, which policy was active, which prompt or mapping was used, which records went into the context, which identity reached the API, which proposal an approver saw, what happened after a timeout, or which external write committed.

Software supply-chain frameworks such as SLSA make the same point at build time: provenance exists because a source revision alone isn't the full history of an artifact. AI applications need that discipline at the application level.

Policy engines show it plainly. Policy and data bundles can update without redeploying the app, and decision logs record the revision used. The code stays the same while the rule and the outcome change.

Three linked records, not one source of truth

The usual reaction is to ask which artifact should become the single source of truth. That's the wrong goal. A repository shouldn't become a live event store. A run history shouldn't implement algorithms. A visual workflow shouldn't duplicate every line of custom code. A policy engine shouldn't own control flow.

The goal is an explicit authority model: each important claim has one authoritative owner, and the owners are linked. For an AI application, three records matter most.

The definition doesn't copy every external artifact into one giant file. It binds them by stable, versioned identity: code by digest, policy by revision, schema by ID and version, prompt by revision, model by approved deployment and region, mapping or query by version, context rules by version, plus retry, approval and evidence requirements. Where it's stored matters less than its properties: immutable once published, addressable, reviewable, testable and tied to every case it starts.

When does a visual workflow count as the definition?

Only when it's the thing that runs. A diagram engineers implement elsewhere drifts; a builder that generates opaque code and then stops being the source becomes a historical sketch.

Visual tools can shrink the distance between design, review and operation, showing steps, contracts, policies, approvals, identity and writes in a form many roles can read. But a visual builder isn't authoritative because it looks complete.

Two failure modes are common. In the first, the diagram is documentation and the real system lives elsewhere. In the second, the builder emits code and the generated code becomes the real application. Either way, what people review and what runs come apart.

The fix is fidelity: the visual view, the stored definition, the published version, any compiled form and the run evidence stay traceably connected. Custom code still has a place; it enters the workflow as a capability with a declared contract, artifact identity, authority requirements, effect class and evidence obligations. The workflow doesn't replace code. It stops code from being the only place behavior can hide.

"Everything as code" helps, but doesn't finish the job

Storing prompts, policies, schemas, workflows and deployment declarations in Git is often a good idea. GitOps adds declarative desired state, immutable history and continuous reconciliation. But "stored in Git" and "complete record" are different claims:

1. Git holds declared state; the system has live state. They can differ.
2. Git doesn't prove what was deployed. Build and publish provenance has to connect source, configuration, dependencies and output.
3. Runtime inputs can't be committed. Retrieved records, feature values, approvals, model responses and external writes exist only during a run.
4. Git versions text; it doesn't model relationships. The runtime still has to interpret the definition, enforce its contracts, hold durable state and emit evidence.

Everything-as-code can be part of the architecture. It can't stand in for it.

An application version is a manifest, not a label

Many teams can name a service version but not an application version. A label such as claims-app-4.12 is only useful if it resolves to everything that shaped behavior: the published workflow and its digest, code artifacts, schemas and mappings, policy revisions, prompts and model routes, context and search rules, feature definitions and taxonomy, review screens where human approval matters, region and provider constraints, and the evidence schema.

Not everything must be bundled. It must be resolvable and, where behavior depends on it, pinned. Some dependencies are dynamic on purpose: a query uses the latest eligible records; a policy must apply the rule in force when the case resumes; a router picks among approved deployments. Dynamic doesn't mean unrecorded. The definition states the resolution rule; the run record captures what actually resolved. For long-running cases this matters most: you need an explicit rule for what stays bound, what's re-evaluated and what change invalidates an earlier approval. (See versioning AI workflows.)

Code review becomes behavior-change review

A pull request is still the right gate for code. It's not enough for every change that alters behavior.

This doesn't need a giant committee. It needs impact-aware routing: a schema change to application and data owners, a policy change to security and the domain, a model route change with evaluation evidence, a side-effect change to operations and finance. If the definition knows its dependencies, the platform can show the impact before anyone publishes. Testing the new version against recorded cases (see behavior replay) turns that review from opinion into evidence.

Common anti-patterns

Ten rules for a complete record of an AI application

1. Name which artifact is authoritative for code, definition, deployment, business data and each run.
2. Publish immutable application definitions; drafts can change, published versions can't.
3. Pin or record every dependency that shapes behavior.
4. Keep desired, deployed and realized state separate.
5. Keep the visual view, stored definition, compiled form and evidence connected.
6. Give paused cases explicit rules for resume, migration and re-approval.
7. Record what every dynamic reference resolved to in each run.
8. Review every behavior-bearing change by its impact, not just pull requests.
9. Link trace IDs, decision IDs, operation IDs, approvals and external confirmations to one run.
10. Keep secrets out of the definition, but keep the authority they carry visible.

The result: authority with a boundary

"Single source of truth" helps when it stops two systems owning the same fact. It misleads when it implies one repository can be authoritative for every fact in a distributed application. The better question is: which artifact is authoritative for this claim, and how is it linked to the rest?

For implementation, it's source code and its build provenance. For composition and governance, it's the published application definition. For business facts, it's the system of record. For what happened in one case, it's the run record. Source code hasn't lost its authority. Its authority now has a boundary, and the architecture's job is to keep the links across that boundary intact.