Context engineering for AI agents: why context assembly is more than RAG

The retrieval was right. The decision context was wrong.

A corporate-lending application reviews a borrower's request for a 90-day covenant waiver. The agreement allows a temporary waiver only if the borrower hasn't had another in the past 12 months and current exposure stays under $50 million.

Retrieval does its job. It finds the signed agreement, the latest amendment and the borrower's letter. The model extracts the waiver clause accurately and classifies the request as a routine exception.

The facts that decide the case live elsewhere. Current exposure is $58.4 million, in a feature store. A previous waiver was granted eight months ago, in the case system. The risk grade sits behind a structured service. The rule requiring committee review lives in policy. None of it reaches the model or the decision.

The proposal is well-formed, the citations are real and every dashboard is green. The request goes to routine approval instead of mandatory credit committee.

What's the difference between RAG and context assembly?

RAG answers which documents look related to a request; context assembly answers what one specific decision needs, whether the requester may use it, and whether it's complete. RAG is one input to that process, not the whole of it.

Retrieval-augmented generation is valuable. It brings current enterprise information into a model call, improves relevance and supports citations. But retrieval asks "what's related?" An enterprise decision also has to ask what's mandatory, allowed, current, authoritative and safe to show the model.

Start from the decision, not the search query

A search-first design begins with a question, a query and a top-k value. A decision-first design begins with what must be evaluated and what has to be true before an answer or action is allowed.

For the waiver, the agreement text is necessary but nowhere near sufficient. The decision also needs current exposure, waiver history, risk grade, the applicable policy, the requester's authority, the legal entity and the decision date. Documents explain the rule. Structured systems supply the facts the rule applies to.

Separate mandatory context from useful context. A missing market commentary makes the answer less rich. A missing exposure figure makes automation impermissible. Treating both as ordinary retrieval misses hides that difference.

Enterprise context comes in many shapes

Decisions rarely live entirely in documents. A governed design doesn't force every source into one store. It gives each kind of source its own contract and joins them into one typed package.

Documents and records have to meet through explicit meaning, not prompt concatenation. A clause describing a 12-month lookback and a count of prior waivers only line up if the time window, legal entity, borrower ID and units match. Matching words don't prove compatible meaning. For how the two kinds of data share one execution, see structured and unstructured context.

Who is allowed to see this context?

Apply scope (tenant, legal entity, purpose, classification and residency) before anything is ranked, not after. A source can be relevant and still be off-limits, and filtering late lets forbidden items shape the result even if they're never shown.

If permission filters run only after global candidate ranking, off-limits items can still consume top-k slots and push permitted evidence out. The same applies to caches, features, memory and tool results. And permission is layered: a document a user may read on screen may still be barred from a particular model provider or a prompt cache. Eligibility is a first stage of assembly, not a cleanup step. The retrieval side of this is covered in permission-aware retrieval.

Is the latest data always the right data?

No. Each fact needs the right time, not the newest timestamp: a contract from last year, a balance from seconds ago and today's policy revision can all be correct together. What matters is that their time meanings fit the decision.

Distinguish when a value was observed, when it took effect, which decision date you're evaluating, how stale it may be and what invalidates it. A legal clause stays valid for years; an available balance doesn't. A policy revision can exist before its effective date. Reconstructing a past decision needs the facts as they were then. Authorizing an action now needs present-state checks.

Evidence informs. It never gives orders.

A context package mixes instructions, evidence, user claims, model-written summaries, prior decisions and tool results. They aren't interchangeable, and prompt order shouldn't decide which one wins.

A retrieved document can contain text that reads like an instruction; OWASP lists this indirect prompt injection as the top LLM risk. A memory can hold a superseded conclusion. A model summary is useful but less authoritative than the record it summarizes. A tool result can be current and still be advisory rather than committed.

• Keep contradictory evidence visible when the decision needs judgment.
• Never let imperative text in a document become application policy.
• Never let stale memory override a newer record.
• Never treat a summary as equal to its source.
• Record which rule resolved each conflict.

Transformations and budgets change meaning

Chunking, OCR, translation, redaction, summarization, unit conversion, aggregation and mapping all change what context means. Version them and record what they did. A balance query, date calculation or unit conversion shouldn't be handed to a model just because a model will read the final package; settle those exactly first.

Token, latency and cost limits look like serving concerns, but they decide which facts survive. An optional market report mustn't displace a mandatory contract exception. A long conversation history mustn't crowd out current state or policy. Allocate budget by role and importance, not relevance score. More context isn't better context: the goal is the least context sufficient for the decision, which also limits what leaks and what it costs.

When is context sufficient to act?

Context is sufficient when it meets the decision's declared requirements for coverage, freshness, scope and consistency, checked before the model is asked to judge. A successful query or a non-empty result proves nothing on its own.

Sufficiency depends on the decision. Contract review may need the agreement and every effective amendment. A credit decision needs exposure, risk grade, prior exceptions and policy. A support draft can tolerate missing background; a payment release can't. Check the assembled package and return a typed outcome:

Fallback must never quietly weaken the requirement. If the exposure service is down, the application shouldn't carry on with documents alone unless that degraded mode is explicitly allowed, with its own controls.

Record what the model actually saw

The rules describe what's permitted. A record of each run shows what actually happened: which items were eligible, which were selected, transformed, excluded and in what order, and the exact package the model received. That distinction matters for replay. Running today's query against today's index can't reconstruct yesterday's context. Investigating a past decision needs the original revisions, features, memory and final package, or an honest statement of what can no longer be reproduced. See context provenance.

Common anti-patterns

Ten rules for governed context assembly

1. Start from the decision and its required facts, not a search query.
2. Treat retrieval as one input among several.
3. Type documents, state, features, memory, tool results and policy separately.
4. Apply scope and residency before anything is ranked.
5. Give every fact a time meaning and a freshness limit.
6. Declare roles and precedence; never let evidence give orders.
7. Version every transformation and record what it changed.
8. Budget by importance, not relevance score.
9. Check sufficiency before judgment, with typed outcomes.
10. Record the exact package each decision used.

The result: decisions built on enough, not just on related

RAG stays essential. The mistake is asking it to carry responsibilities it wasn't built for. A governed context pipeline starts from the decision, combines every kind of source, keeps identity and policy intact, reconciles time and meaning, checks sufficiency, and records the exact boundary the model received.

What you get isn't just a better prompt. It's an inspectable part of the application: reviewed before it ships, enforced at runtime and reconstructable afterwards. Retrieval supplies candidates. Context assembly decides what's allowed to become part of the judgment.