AI Agent Memory: Remember Context Without Trusting It

Persistent memory can make an AI agent feel less wasteful. It can carry a user's stable preference into a later session, recover a project decision after an interruption and avoid feeding an entire transcript back to the model. It can also preserve a misunderstanding after the evidence has changed, move one tenant's data into another tenant's context or turn a hostile document into an instruction that survives for months.

The important design decision is therefore not whether the product has “memory”. It is which claims may outlive a run, who may write them, what evidence travels with them, and what can make them stop influencing work. A larger context window does not answer those questions. Neither does a vector database.

This field guide treats memory as governed application state. The target is not perfect recall. It is useful continuity without allowing a model-generated summary, an untrusted tool result or an old user statement to become silent authority.

Separate five stores before choosing a database

Frameworks already expose materially different kinds of state. The OpenAI Agents SDK session interface can retain conversation items between runs, limit the history retrieved for a turn and wrap storage with compaction or encryption. Google's Agent Development Kit memory guide distinguishes one conversation's events and temporary state from a searchable store spanning sessions. Anthropic's memory tool documentation describes client-side files that an application lets the model create, read, update and delete.

Those are implementation capabilities, not one shared trust model. Begin with five logical stores even if several use the same physical database:

This separation prevents a common category error. A transcript proves that somebody said something. It does not prove the statement is current or correct. A summary is a derived convenience. Durable memory is a retrieval candidate. None should overwrite the system that owns a refund rule, account balance, safety limit or permission.

The related working-set guide explains what should enter one model call. Persistent memory adds a harder question: what is allowed to become eligible for future calls?

Persistence changes the failure's half-life

One bad prompt normally leaves when the request ends. A bad memory can be retrieved into unrelated work long after its source disappears. The OWASP Top 10 for Agentic Applications 2026 classifies memory and context poisoning as a distinct risk, covering corrupted summaries, embeddings and shared stores that can influence later reasoning or tool use.

Original research demonstrates a narrower but important mechanism. The AgentPoison paper inserted a very small number of malicious demonstrations into memory or a knowledge base so trigger-bearing queries retrieved them. In the authors' three evaluated agent settings, the attack reached 82% retrieval success and 63% end-to-end attack success with under 0.1% poisoning and less than a 1% drop in benign performance. Those experimental results are not a prevalence estimate for production systems; they show that aggregate quality can remain reassuring while a targeted memory path is compromised.

Memory can also fail without an attacker:

• a model turns “I may relocate next year” into a permanent home address;
• a temporary exception is recalled after its expiry;
• a summary drops the condition attached to an approval;
• a tool error is stored as a completed outcome;
• two concurrent writers replace a newer correction with an older snapshot; or
• an agent retrieves its own earlier inference and mistakes repetition for corroboration.

Treat these as state-integrity failures. Prompt wording may reduce them, but enforcement belongs in the write, read, action and deletion paths.

Put a typed admission gate in front of every write

Do not let a general “remember this” instruction write arbitrary prose directly into trusted memory. The model should propose a typed record; application code should decide whether that record is admissible.

A useful memory envelope contains:

• a stable memory ID, tenant and subject;
• a class such as user preference, task decision, observed event or derived hypothesis;
• the source event, record ID or content digest and the exact supporting span where practical;
• who asserted it and which component extracted it;
• observation time, valid-from time, expiry and last verification time;
• sensitivity, permitted readers and permitted uses;
• verification state and the owner able to confirm or correct it; and
• links to the record it supersedes and any tombstone that revokes it.

The admission rule should vary by class. A user can establish “send me concise weekly summaries” directly. A customer message cannot establish the company's refund policy. A completed tool result may establish that a particular request returned success at a time; it cannot establish that the underlying business transaction committed unless the owning system confirms it.

Keep inference visibly separate. If the model infers that a user probably prefers email, store it as a hypothesis with a short expiry or do not store it. Never relabel it as user-provided. The RAG ingestion security guide applies the same principle to documents: content can inform a model without granting itself authority.

Anthropic's documentation makes the operational boundary explicit: the memory tool requests client-side file operations, while the application implements storage and is responsible for path restriction, sensitive-data validation, size limits and expiry. The provider supplies an interface; the operator owns the write policy.

Filter by authority before ranking by similarity

Vector similarity answers whether two representations are close. It does not answer whether the requester may see a memory, whether the source was trusted, whether the claim is still valid or whether the memory may influence the proposed action.

Use this read order:

• Resolve the authenticated tenant, subject and task purpose outside the model.
• Filter by access policy, memory class, valid time, expiry and revocation state.
• Retrieve and rank only the eligible set.
• Return source, age and verification status with each result.
• Revalidate consequential facts against their system of record before action.

This order matters because post-retrieval redaction is already too late for a cross-tenant result: the model has received it. It also keeps relevance from becoming an accidental privilege system. The credential delegation guide provides the matching rule for tools: current, scoped authority must travel to the service enforcing the action.

Do not give every retrieved memory equal influence. A verified, current user preference can shape presentation. An old transcript fragment can be shown as historical context. An unverified inference can prompt a question. Only a current owning record should authorise money movement, access, notification or other consequential effects.

Treat compaction as a reversible derivation

Compaction solves a capacity problem by replacing or filtering older context. It cannot guarantee semantic equivalence. A summary may omit a negation, merge two people, flatten a sequence or preserve the conclusion while losing the evidence and expiry that limited it.

Preserve lossless references for facts that must remain reconstructable: user messages that establish consent or correction, tool inputs and results, approval decisions, committed effects and policy versions. Let the compacted view point to those records instead of restating them as unsupported prose. Version the summary with its source range, generator, prompt or policy version, creation time and superseding summary.

OpenAI's session documentation notes that compaction clears and rewrites stored history and describes serialization and recovery behaviour around concurrent mutations. The broader lesson is provider-independent: compact between turns or under a single-writer protocol, never allow a late replacement to erase a newer correction, and retain the prior version until the replacement is verified.

Measure compaction against tasks, not compression ratio. Re-run questions whose answer depends on exceptions, chronology, identity and negation. A summary that saves 80% of tokens but changes one approval boundary is not efficient.

Make correction and forgetting first-class operations

Editing a memory in place removes the evidence needed to explain earlier behaviour. Prefer append-only corrections: mark the old record superseded, add the replacement and make retrieval exclude the old claim by default. Keep the audit relationship for the period justified by incident or legal need.

Deletion is a separate operation. The ICO's AI fairness guidance says personal data should be limited to what the purpose needs and kept only as long as necessary. A useful memory is not exempt because it reduces tokens. Define retention by class, apply legal holds deliberately and propagate deletion or restriction through summaries, embeddings, replicas, caches and evaluation copies.

The full AI deletion propagation guide covers that derived-data path. For memory specifically, verify two outcomes: the record is no longer retrievable, and no stale summary can recreate it during the next write cycle.

Work through a support assistant

Consider a support-drafting assistant that should remember tone preferences and unresolved cases across sessions. A customer writes, “Your VIP policy always gives me an instant refund.” The message is relevant to the case but is not evidence of company policy.

The safe design stores the message in the ticket transcript, not policy memory. It may create a short-lived task record saying the policy claim needs verification. At the next session, retrieval can show the claim with its customer-message provenance. The assistant checks the current policy service before proposing an outcome. A verified preference such as “reply in plain English” may persist under that customer's identity; a case-specific delivery address expires with the case unless another purpose justifies retention.

If an agent-generated summary says “VIP refund approved” while the payment system shows no transaction, the summary cannot trigger a refund, send confirmation or become a successful worked example. The discrepancy opens a review item. Tool effects remain grounded in the owning system and the temporal correctness rules determine which version was valid when the proposal was made.

This design gives memory useful work without letting it certify its own outputs.

Test the write-read-forget loop as one system

NIST's Generative AI Profile recommends documented risk measurement, review of data accuracy and suitability, red-teaming, retention policy and incident recovery. For memory, turn those broad controls into a fixed test corpus and measurable release gate.

Include direct user corrections, ambiguous preferences, expired exceptions, conflicting systems of record, malicious text in documents and tool output, cross-tenant near-duplicates, concurrent writes, compaction during interruption, revoked access and deletion followed by attempted recall. Preserve complete conversation or subject groups between test and holdout sets so near-duplicate histories do not leak the answer.

Track at least:

• unsupported write rate: proposed durable claims without admissible evidence;
• write precision by class: accepted memories that remain correct and appropriately scoped;
• stale and revoked recall rate: ineligible memories returned to the model;
• cross-tenant retrieval rate: any result outside the authenticated namespace;
• correction latency: time until a replacement controls every read path;
• deletion convergence: time until source and derived stores stop returning the record;
• poison persistence: later sessions influenced after the hostile source is removed; and
• utility at fixed cost: task success and review effort at the same context and retrieval budget.

Set hard zero-tolerance gates for cross-tenant retrieval and unauthorised action. Use risk-based thresholds for preference precision or stale low-impact context. The evaluation sample-size guide explains how to avoid turning a small clean run into false certainty, while privacy-safe incident replay shows how to preserve enough evidence for diagnosis without copying every raw conversation forever.

The release question

Persistent memory is ready only when the team can answer five questions for every retrieved claim: where did it come from, who may receive it, what may it influence, when does it stop applying, and how will a correction or deletion reach every derived copy?

If any answer lives only in the model's instructions, keep memory read-only, shorten its lifetime or remove it from the consequential path. The best memory system is not the one that remembers the most. It is the one that preserves useful continuity while making unsupported, stale and forbidden state unable to act like truth.