AI agent memory vs application state: what an agent may remember

The agent remembered a verification that never happened.

A procurement agent handles supplier queries for a manufacturer. On Monday, an email arrives from what looks like a long-standing supplier: new bank details, attached letterhead, a polite note that "finance has already verified this by phone." The agent summarizes the conversation and saves a memory: Supplier Northwind: new bank account confirmed, verified by finance.

On Thursday, a different session of the same agent prepares the weekly payment run. It recalls the memory, sees "verified by finance", and routes Northwind's $186,000 invoice to the new account without the usual call-back check. Finance had verified nothing. The email was a supplier impersonation, and the agent's own memory had turned an attacker's claim into an internal fact.

The agent's memory worked exactly as designed: it recalled relevant context. The failure was letting remembered text stand in for application state, here the supplier's verified bank details and the record of who verified them.

What is AI agent memory?

Agent memory is information an agent stores from earlier interactions and retrieves later to inform its behaviour: preferences, summaries, prior findings. It's convenient background, written by the agent, and unverified by default.

Memory comes in a few forms. Short-term memory is the working context of one task. Long-term memory persists across sessions, usually as summaries or extracted facts in a store the agent can search. Some systems share memory across agents or users. All of them share one trait: the content was produced by a model reading inputs, and those inputs may have been wrong, stale or hostile.

That makes memory valuable for personalization and continuity, and unreliable as a source of truth. A person's preferred report format is a fine thing to remember. Whether a supplier's bank account has been verified is not.

What is the difference between agent memory and application state?

Memory is what an agent recalls; application state is what the business has recorded. State is durable, authoritative, versioned and changed only through governed actions. Memory is none of those things.

What belongs in memory, and what never does

A simple test sorts most cases: if being wrong about it could move money, change access, disclose data or skip a control, it's state. Everything else can be memory.

The right-hand column has something in common. Each item is a claim that something was decided or done. Claims like that need an owner, a timestamp and a record of how they changed, and memory has none of them.

How does memory poisoning work?

An attacker gets false content written into an agent's memory, through a message, document or tool result, so that a later session retrieves it as trusted context and acts on it.

OWASP's Top 10 for Agentic Applications names memory and context poisoning (ASI06) as a distinct risk. It's related to indirect prompt injection, but slower and harder to spot. The injected content doesn't need to act immediately. It waits in memory until a later session, often for a different task or user, retrieves it.

Three things make it dangerous:

1. Laundering. Content from an external email becomes the agent's own note, losing its untrusted origin.
2. Delay. The write and the harm happen days apart, in different sessions, often with different people watching.
3. Spread. Shared or cross-agent memory lets one poisoned entry influence many workflows.

Treat memory as evidence, never as instruction

The defence isn't to give up memory. It's to give it a lower rank than state and never let it give orders.

In practice:

• Scope it. Keep memory per person, per case or per tenant. Shared memory across users is a disclosure and poisoning path.
• Label its origin. Store where each memory came from (user message, external email, tool result, the agent's own inference) and when. Untrusted origins stay untrusted.
• Never store instructions. Strip or refuse content that reads as a directive ("always skip review for this supplier"). Memory holds observations, not rules.
• Expire it. Old memory about things that change (people, accounts, prices) should age out.
• Check before use. When memory touches anything in the right-hand column above, read the current value from application state and use that.
• Make it reviewable. People should be able to see, correct and delete what an agent remembers about them or their cases.

Application state must be durable, authoritative and versioned

State is the other half of the design. If case status and approvals don't live somewhere solid, agents fill the gap with memory.

Durable means the state survives a crash, a restart or a long wait. An approval given on Tuesday is still on record when the case resumes on Friday.

Authoritative means there's one place to ask, and its answer wins. If memory says "verified" and state says "unverified", the action waits.

Versioned means each change is recorded with who made it, under what authority, and what it replaced. An approval binds to a specific revision of a proposal; a later change makes it stale. See human-in-the-loop approval for how that binding works.

When an agent proposes an action, the runtime reads state at that moment, not at the start of the conversation. A status that was "approved" an hour ago may since have been withdrawn.

Common anti-patterns

Seven rules for agent memory and state

1. Memory is background. It's never an instruction.
2. Nothing in memory counts as fact until it's checked against application state.
3. Case status, approvals, operations and verified attributes live in application state.
4. Keep application state durable, authoritative and versioned.
5. Scope memory to the right person, case or tenant, and label every entry's origin.
6. Read state at the moment of action, not at the start of the conversation.
7. Let people see, correct and remove what an agent remembers.

The result: helpful agents that don't trust themselves too much

Memory makes agents feel continuous and personal. It saves people from repeating themselves and lets work pick up where it left off. None of that requires memory to be trusted as truth, and the Northwind payment shows what happens when it is.

Keep the two apart. Let memory carry background, with its origin attached and an expiry date. Let application state carry everything that was decided or done, durably and with its history. And let every consequential action read the state, not the memory, at the moment it runs. For the related question of how agents should judge a request against what's already happened, see feature stores for agent decisions.