When not to call an LLM: deterministic-first routing for enterprise AI

The model calculated a discount the system already knew

A manufacturer's accounts-payable application decides whether supplier invoices qualify for an early-payment discount. The supplier agreement is clear: deduct 2% if payment lands within 10 calendar days of accepted receipt, and leave freight and tax out of the discount base.

Every fact needed is already in structured systems: the receipt date, invoice date, goods amount, freight, tax, currency, supplier status and the versioned contract rule. One query and one calculation settle the amount and the deadline.

Instead, the application sends the agreement text and invoice fields to a general reasoning model and asks it to work out eligibility and the discount. The model returns valid JSON, a plausible rationale and a 2% discount. It used the invoice date instead of the receipt date, and it applied the discount to freight and tax.

A policy check confirms the payment is under the approver's threshold. It doesn't recompute the discount base. The payment service commits the amount.

Every component can defend itself. Retrieval found the right agreement. The model produced well-formed output. The policy check applied the facts it was given. The payment API did what it was told. The business still paid the wrong amount.

When should you not call an LLM?

Don't call a model when an exact lookup, calculation, validation, policy rule or state transition can already satisfy what the decision needs. Call one for the part that's genuinely ambiguous: interpreting language, extracting from varied documents, classifying intent, synthesizing sources, drafting.

This isn't anti-AI. Models earn their place exactly where exhaustive rules are impractical. The question is where uncertainty enters the application. If the answer can be established exactly, adding inference buys variability, cost, latency, evaluation work and harder-to-read evidence, without adding business value.

Deterministic doesn't mean correct. A stale rule, a wrong query or a mis-scoped policy fails the same way every time. The deterministic path still has to be typed, versioned, tested, authorized and bound to current state. The claim is narrower: don't buy uncertainty when you can already know.

Start from the decision, not the model catalog

A model catalog starts with available endpoints and asks which one should take the request. A governed application starts a level higher: what has to be established, under whose authority, and with what tolerance for uncertainty, cost, delay and consequence?

That turns model selection into a requirements question. Some decisions need an exact value. Some accept a bounded approximation. Some need interpretation but no action. Some need a person.

A route fits only if it meets the whole contract for this case. A large model can be powerful and still fail because the data can't leave a region, the output can't be checked, or the result must be exact.

A capability ladder beats a model leaderboard

Applications rarely need one universal model ranking. They need a ladder for each task, ordered by how much uncertainty each step adds.

The ladder isn't a fixed hierarchy. A rule is the wrong tool for a genuinely ambiguous question. A narrow model can beat a general one on a bounded task. A reviewer may be unavailable or under-informed. The ladder follows the decision's requirements, not an industry ranking.

Split the task before you route it

Applications often send a whole business problem to one model because the request arrives as natural language. But one request usually holds several separable jobs. The invoice case splits into six:

1. Resolve the supplier, invoice, purchase order and receipt by exact ID.
2. Fetch the typed discount rule and current supplier status.
3. Calculate the base, deadline and amount.
4. Check policy and authorization against that exact result.
5. Call a model only if a clause is ambiguous, an amendment is unstructured, or an exception needs interpreting.
6. Send unresolved, high-stakes ambiguity to a qualified reviewer.

This keeps models for what they're good at and stops them recreating facts the application already owns. It also makes testing easier: exact parts get exact test oracles, model parts get evaluated on the narrow judgment they contribute, and the whole route is judged on the business outcome. For a deeper treatment of which steps belong on which side, see deterministic vs probabilistic AI steps.

Which models are you even allowed to use?

Filter by identity, authority, policy, privacy and residency first; only then optimize for quality, cost and speed among the options left. A cheaper model outside the permitted region isn't a candidate, however good it looks.

What decides whether a capability may compete for a step:

• the person or agent who started the work, and the authority delegated to it;
• tenant, legal entity, resource and business purpose;
• data classification, residency, provider and retention rules;
• the current state of the case, open obligations and approval status;
• the capability's version, test coverage and deployment availability.

A fast cached answer resolved under stale authority isn't a candidate. A stronger model with no evaluation for this kind of case isn't automatically one either. Failover is the classic trap: when the primary provider is down, the fallback quietly sends regulated data to another region. Availability never justifies widening where data may go.

When should the route stop or escalate?

Stop when a result meets the full declared requirement; escalate, verify or abstain when it doesn't, and say why in a typed outcome. Without an explicit stopping rule, applications either call a model after the answer is already known or accept a deterministic result that's stale or incomplete.

Useful typed outcomes include: requirement met, evidence insufficient, ambiguous, stale or unavailable, out of scope, conflicting sources, model needed, human needed, and abstain or block. Each one leads somewhere specific, which is the point.

When the route does escalate, don't start over. Sending the original raw request to a bigger model throws away what's already settled and invites the model to overwrite it with plausible prose. Pass forward:

• exact IDs and current record state;
• validated calculations and passed invariants;
• the evidence used, with its provenance;
• known conflicts and missing facts;
• policy obligations and prohibited actions;
• why the previous step wasn't enough.

The model then answers a narrower question with the facts fixed as constraints. How to decide when uncertainty warrants more evidence, a stronger model or a person is covered in confidence escalation.

Is a confidence score enough to route on?

No. A confidence score is one input, not a policy; it isn't a probability that the business answer is right. Its meaning depends on the task, the population, the evaluation behind it and what's at stake. Two models agreeing isn't independent verification either.

Before accepting a model result automatically, also check:

• Is all required context present and current?
• Did the deterministic invariants pass?
• Is this case inside the range the model was tested on?
• Is contradictory evidence still unresolved?
• Can the output be checked independently?
• Does the consequence allow automatic acceptance at all?

NIST's guidance stresses that trustworthiness is context-dependent and that test, evaluation and validation run across the lifecycle. Bind routing thresholds to task-specific evaluation, not a generic benchmark rank.

Measure the route, not the model call

A model call costs more than its token price. Context assembly, retrieval, queueing, retries, fallback, evaluation and evidence all add to it. OWASP lists unbounded consumption as a top LLM risk because uncontrolled inference creates cost, availability and security exposure at once. Research on model cascades, such as FrugalGPT, shows routing choices materially change cost-quality tradeoffs, though no single cascade or threshold carries over to every enterprise task.

Deterministic paths aren't free either: complex queries, external services and human review all cost something. Measure the whole route:

Evaluate the route as a whole, too. Was the right rung chosen? Was an exact path skipped? Did escalation happen at the right time? Did the final effect match the accepted decision? Replaying recorded cases against a proposed routing policy, without live side effects, answers these before a change goes live.

Record why the route was chosen

Logging which model ran isn't enough. For each routing decision, keep the requirement it was judged against, the capabilities that were allowed and those ruled out (with reasons), the one selected and its version, region and evaluation, the deterministic results and model outputs, why it stopped or escalated, the cost and latency, and the confirmed business effect. That record is what lets a risk owner explain why a model touched the case at all, and lets finance see cost per outcome instead of cost per call.

Common anti-patterns

Ten rules for deterministic-first routing

1. Start from what the decision needs, not from the model catalog.
2. Split each request into exact, rule-based, model and human parts.
3. Apply identity, policy, privacy and residency limits before optimizing.
4. Try lookups, calculations, validations and rules first when they can meet the requirement.
5. Use explicit, typed stop, escalate and abstain outcomes.
6. Pass settled facts forward as constraints; never ask a model to redo them.
7. Tie every model step to an evaluation, stated limits and a way to check its output.
8. Measure the whole route's cost and time, including retries and review.
9. Never let failover widen where data may go.
10. Record why each route was chosen and what effect it produced.

The result: models where judgment is needed, facts where facts exist

The goal isn't the fewest model calls. It's putting each part of a decision where it can be satisfied with the least uncertainty. Lookups stay exact. Calculations stay typed and testable. Policies stay enforceable. Models contribute judgment where there's real ambiguity, and people step in where the stakes and the uncertainty justify it.

That's often cheaper and faster, but economics is the smaller benefit. Deterministic-first routing narrows what you ask a model, sends it less data, makes each part testable and leaves a record that explains why a model was involved at all. For setting the cost and quality policy behind those choices, see LLM model routing.