Deterministic vs probabilistic AI steps: combining rules and models safely

The contract didn't change. The decision did.

A global enterprise uses an AI-assisted procurement workflow to renew software and data-service agreements. At onboarding, it analyzed a signed vendor contract, a data-processing addendum and a security annex. A model classified the vendor as a regulated data processor, which triggered privacy and security review before the purchase order.

Twelve months later, the same signed agreement came up for renewal. Same supplier, same text, same purpose, same service. The workflow now called a newer model and context release. This time it classified the vendor as a standard software subscription, and the renewal went down the automated path.

Both answers were valid JSON. Both used an allowed category. Both scored above the generic confidence threshold. Neither call failed, and the policy code downstream did exactly what it was written to do. Privacy later found the service still processed regulated customer identifiers.

The incident review blamed model nondeterminism first. That was only part of it. The real failure was that a learned classification became a business fact without keeping what made it uncertain: the evidence behind it, the context it saw, how its score was calibrated, and what had to be checked before it could steer an irreversible path.

Is "is the model deterministic?" the right question?

No. Repeatability, correctness and permission to act are three separate questions. A repeatable answer can be wrong, a correct answer can be unauthorized, and only the third question decides whether anything happens.

A model can be computationally deterministic with fixed inputs, artifacts, hardware, seeds and settings. It's still probabilistic in the sense that matters: its output is a learned estimate, and its correctness is established by testing, not guaranteed by the code. ML frameworks say so directly: deterministic modes need fixed environments, often cost performance, and can still differ across releases or hardware.

The reverse holds too. A rule engine can give the same answer every time while applying a stale policy, the wrong currency or the wrong legal-entity mapping. Repeatability is valuable. It isn't validity.

The business still owes deterministic obligations

A model can interpret a contract, rank evidence, extract fields, classify intent or draft a proposal. The application around it still owes obligations that plausibility can't satisfy. A payment must meet currency, balance, limit and reconciliation rules. An authorization must name the right person, tenant, resource, purpose and policy version. A state change must go from a valid state to an allowed next one. A tax or pricing rule must apply exactly once its facts are known. An external change needs a stable operation ID and proof of what happened. An approval must bind to the exact proposal that will run.

That doesn't mean every business decision is objective. Plenty need judgment. The narrower requirement is that uncertainty stays visible until a governed checkpoint decides what the system may do. The model supplies judgment; it can't quietly redefine the obligation.

Where does uncertainty get lost?

At the interface. The model returns a clean category, a schema confirms it's an allowed value, and the next step branches as if it were a verified fact. By the time policy code runs, the uncertainty is gone.

Structural validation is necessary. It can't establish factual correctness, policy applicability, source eligibility, calibration or authority. A schema-valid payload can be confidently wrong. So status has to travel with the value: proposed, structurally valid, domain-validated, independently verified, policy-eligible, approved, committed. Collapsing those into one field is how probability turns into false certainty. Typed contracts for AI workflows covers how to carry that status between steps.

Three kinds of step

Describe each step by the contract it offers, not the technology behind it.

These aren't permanent labels for technologies. A hand-coded classifier with an exact contract is deterministic. A model endpoint with deterministic serving is still probabilistic in validity. A person following a checklist still exercises judgment on an ambiguous case. The point is to stop a probabilistic step passing as deterministic just because both return JSON.

Wrap every model in deterministic checks

The envelope doesn't make the model correct. It limits when uncertainty can become action:

1. Eligibility: may this person, tenant and purpose run this step on this resource?
2. Context: select required sources under permission and freshness rules, and record where they came from.
3. Proposal: the model returns a typed proposal, not an unqualified fact.
4. Validation: check shape and type, then domain rules, references, units and allowed values.
5. Thresholds: is the result inside the range where automation was validated?
6. Policy and people: resolve obligations that can't be met automatically.
7. Point of change: enforce authority, operation ID, retry rules and evidence.

Scale it to consequence. A low-risk text suggestion may need little beyond a label and an edit option. A classification that changes a payment, entitlement or regulated workflow needs much more. Deciding which cases a rule can settle before a model is called at all is its own design choice, covered in deterministic-first routing.

What should a model's output contract carry?

Enough for the next step to know what's claimed and how sure it is: the value and task, its status, alternatives, the basis for any confidence, the calibration profile, evidence, versions, scope and abstain reason, and remaining obligations.

Not every task has a defensible confidence value. A contract that says "no calibrated probability available" is more trustworthy than invented precision.

Confidence isn't a control

A score is useful only once its meaning is tested. Neural networks can be accurate and still poorly calibrated: their confidence doesn't match how often they're right. A model saying "94 percent confident" isn't a probability. A token likelihood isn't a business-risk measure. Two models agreeing aren't necessarily independent. A verifier isn't right just because it's separate.

Thresholds belong to a decision policy, not to the model. They depend on the task, class, segment, consequence, the cost of a false yes and a false no, review capacity and the quality of the evaluation behind them. Version the calibration profile with the threshold. When the model, prompt, context or data shifts, the threshold may stop meaning what it meant.

Let the system abstain

Many designs assume every request must get an answer. That turns uncertainty into forced automation. Research on selective prediction formalizes the alternative: trade coverage for lower risk by declining cases the system can't handle reliably. The question becomes which share of the work it can cover at the required risk.

Abstaining isn't a failure. It's a valid outcome with routes: apply a rule or lookup, fetch required evidence under the same permissions, call a stronger model allowed by cost and residency, run an independent check, send the exact proposal and evidence to an authorized person, or stop and record why. Choose the least costly path that meets the risk, evidence and authority needs of this decision.

Test behavior, not just answers

Unit tests still matter, but they can't fully specify a probabilistic step. NIST's AI Risk Management Framework stresses testing, evaluation, verification and validation in the context of use.

And a model change is an application change. Prompts, retrieval rules, context order, features, calibration, thresholds, routing, regions and post-processing all shift which cases get automated, escalated or committed. Publish them together as one release, and compare a proposed release against recorded cases before it goes live.

Common anti-patterns

Ten rules for combining deterministic and probabilistic steps

1. Mark each step as deterministic, probabilistic or judgment.
2. Treat model output as a proposal unless the contract says otherwise.
3. Carry alternatives, evidence, scope and confidence basis through typed contracts.
4. Enforce authority, policy, invariants and approvals outside the prompt.
5. Validate in layers: structure, domain, facts, policy, approval.
6. Calibrate before you set a threshold, and set it per decision.
7. Design safe abstain and escalation paths.
8. Release model, prompt, context, calibration, threshold and policy together.
9. Test exactly where you can, statistically and by segment where you can't.
10. Record what the system knew, proposed, checked, escalated and did.

The result: deterministic responsibility, not deterministic AI

Back in the procurement case, the model's category should never have been the final word. The contract and annexes enter under explicit source rules. The model returns a proposed classification with passages, alternatives and a versioned confidence basis. Domain checks confirm the supplier and service scope. A deterministic policy asks whether the evidence is enough for auto-renewal, and any doubt about regulated data sends it to privacy review whatever the top category says.

None of that guarantees the model is never wrong. It stops the system from hiding uncertainty and acting as if no judgment took place. Probabilistic capability isn't a defect to engineer out; it's why the application can read language and spot patterns at all. The architecture's job is to keep uncertainty visible until enough evidence, validation, authority and policy exist to turn a proposal into action.