AI Cache Safety: Reuse Compute Without Reusing Authority

An AI cache can save money and latency while returning a perfectly plausible answer to the wrong person. The failure is easy to miss because the cached object may be accurate, safe in isolation and recently created. What changed is the authority under which it is reused.

The decision is therefore not simply whether to cache. It is which work may be treated as equivalent, for which principal, against which model, prompt, policy and evidence state, and until what event breaks that equivalence. A high hit rate is useful only after those boundaries are explicit.

The standards and provider documentation linked below supply the source facts. The reuse contract, cache classes, invalidation ledger and worked example are AIEngine's engineering synthesis. They are a design method, not a claim that one cache product or key format is safe for every system.

A cache hit is an equivalence claim

HTTP caching already has a precise vocabulary. RFC 9111 distinguishes private caches from shared caches, defines a cache key as the information used to select a response and restricts reuse of authenticated responses unless explicit controls permit it. Its security section also warns about stored sensitive information, cache poisoning and timing information.

AI systems add more hidden state to that familiar problem. Two requests can have identical text but different users, object permissions, retrieval corpora, regions, safety policies or tool entitlements. Conversely, two prompts can look semantically similar while asking about different accounts. Text equality is not authority equality; vector proximity is not policy equality.

Treat every hit as the answer to four questions:

• Content equivalence: would the same model and prompt produce an acceptable object?
• Authority equivalence: may this caller see and use the object and every fact behind it?
• Evidence equivalence: are the source data, tool results and safety checks still current?
• Lifecycle equivalence: does reuse comply with location, retention, deletion and review obligations?

If any answer is uncertain, the safe outcome is a miss, a narrower cache or no cache at all.

Separate compute reuse from answer reuse

“AI caching” describes several mechanisms with very different consequences. Provider prompt caching usually reuses internal computation for an exact prefix; it does not normally return a previous completion. OpenAI's current prompt-caching guide describes exact-prefix matching and recommends placing static instructions and examples before variable content. Anthropic likewise documents prefix-based caching, with tools, system content and messages evaluated in order.

Application response caching is different: it can return a prior output without another model call. Semantic response caching widens the match from identical input to similar meaning. Retrieval and tool caches may reuse documents, search results, permissions or API responses before generation. Idempotency keys prevent one operation from being executed twice; they should not become a general answer cache.

Provider controls also differ. OpenAI's API data-controls table describes prompt-caching application state separately from abuse-monitoring and endpoint retention. Anthropic documents its own cache lifetime, isolation and retention behaviour. Those facts must be verified for the exact provider, plan and region; “the provider caches prompts” is not a complete data-flow answer. The AI data-residency execution-path guide shows why contractual region labels are not enough on their own.

Write the reuse contract before the key

A cache key is an implementation of a reuse contract. If the contract is vague, hashing more fields merely makes the mistake harder to inspect. Write a human-readable record for each layer before choosing Redis keys, vector indexes or gateway settings.

At minimum, record:

• the cached object and whether it contains input, output, embeddings, documents or tool state;
• the allowed sharing scope: public, organisation, tenant, principal, role or one operation;
• the authority version, including object permissions and policy decision inputs;
• the model, system prompt, tool schema, safety policy and response-schema versions;
• the source collection, retrieval filter, index build or upstream record version;
• the permitted region, encryption boundary, retention limit and deletion route;
• the freshness budget plus events that invalidate immediately; and
• the evidence required before a hit may trigger a consequential action.

This follows the resource-centred principle in NIST SP 800-207: trust is not granted because of network location, and authentication and authorisation are discrete checks before access to a resource. It also addresses the object-level failure described by OWASP API1:2023: an endpoint receiving an object identifier must enforce authorisation for that object. A tenant ID in the key cannot replace an object permission check.

Choose public, private or forbidden reuse

Classify the workload before tuning time-to-live. Three classes are usually enough to expose disagreement.

“Private” does not mean one global cache with a tenant filter applied after lookup. Partition the storage or include the complete authority context in selection, then re-authorise the underlying objects before delivery when permissions can change faster than the entry expires. For generated content that may cause an action, reuse the draft only; repeat the output validation and action gate against current policy.

Treat semantic similarity as a candidate, not permission

Semantic caching is attractive because people ask the same general question in many forms. Microsoft's current Azure API Management semantic-caching policy can return a stored response for an identical or semantically similar prompt; its example varies the cache by subscription ID and exposes a similarity-score threshold.

That is a retrieval mechanism, not a security judgement. “Summarise my latest invoice” and “summarise the latest invoice” may sit close in embedding space while referring to different objects. “Can contractors export records?” may need a different answer for a privileged administrator, a contractor and a customer. Raising the similarity threshold reduces some false hits but does not encode object authority, policy version or temporal meaning.

Use semantic matching only after hard partition filters. A safe sequence is:

• authenticate the caller and resolve the current principal, tenant and region;
• select only entries already inside that storage and authority partition;
• retrieve semantically similar candidates;
• reject candidates whose prompt, model, policy, tools or evidence versions differ;
• re-authorise referenced objects when permissions are mutable; and
• return the hit with provenance, age and cache status available to telemetry.

For sensitive support, finance, health, employment or legal workflows, exact private caching—or no answer cache—is usually easier to reason about than a broad semantic cache.

Invalidate on events, not only elapsed time

A time-to-live limits staleness; it does not describe why an entry became invalid. A five-minute answer can already be unsafe after a permission revocation, document correction, account closure or policy change. Long TTLs can be correct for immutable public material, while seconds can be too long for a revoked credential.

Maintain an invalidation ledger that maps state changes to affected cache namespaces. Events should include:

• user, role, group, consent or object-permission changes;
• source-record updates, deletions, legal holds and index rebuilds;
• model, system-prompt, tool-schema, classifier or safety-policy releases;
• region, provider, encryption-key or retention-policy changes; and
• security incidents, poisoning reports and manual quarantine decisions.

Make deletion observable end to end. Purging an answer while leaving embeddings, retrieved chunks or provider state unmanaged is not complete erasure. The AI deletion propagation playbook provides the adjacent lifecycle controls; the cache ledger should emit evidence into the same deletion record.

Work a support-RAG example

Consider a support assistant that answers from public manuals, tenant runbooks and live account data. A single semantic cache for all answers would be cheap and indefensible. Split it into three paths.

Public-manual answers may use a shared exact or semantic response cache keyed by manual release, locale, system-prompt version and model family. Tenant-runbook answers require a tenant partition, document-set version and current access to every cited chunk. Live account answers should normally skip response caching; individual tool reads may have a short private cache keyed by principal, account object, permission version and upstream record version.

Suppose an administrator asks, “Why did the renewal fail?” The assistant retrieves a tenant procedure and a live billing status, then drafts an explanation. A later viewer asks the same words. The text matches, but the correct reuse decision depends on account authority, whether the billing record changed, whether the runbook was revised and whether the viewer may see the failure reason. Reusing only the public procedure may be valid. Reusing the assembled answer is not.

The cache trace should therefore show a public-manual hit, a tenant-document miss after re-authorisation and a fresh account lookup. That trace is also useful during incident reconstruction; the privacy-safe AI replay guide explains how to preserve decisions without logging every sensitive payload.

Test the boundaries a hit can cross

Ordinary unit tests confirm that a cache hits. Safety tests try to make it hit when it must not. Build a matrix that changes one dimension at a time:

• same prompt, different tenant, principal, role and object permission;
• same user, changed consent, group membership or account ownership;
• same wording, different document, date, locale or region;
• same source, changed model, prompt, tool or safety-policy version;
• near-duplicate prompt with a different entity, negation or time reference;
• poisoned candidate inserted by an untrusted or lower-authority source;
• deletion or revocation arriving immediately before and during lookup; and
• a cached draft presented to an action gate under changed policy.

Run concurrency tests too. Invalidation and lookup can race: a process may read an entry just before a revocation event removes it. Versioned authority tokens or monotonic epochs let the delivery path reject an old hit even if physical eviction is still propagating. Record this as a tested failure mode, not an assumption about queue speed.

Measure useful reuse, not hit rate

Raw hit rate rewards over-sharing and long-lived staleness. Operate a small scorecard by cache layer and risk class:

Review the contract when a provider's retention changes, a new tool or region is introduced, a policy version changes, or rejection rates suggest the proposed partition is too broad. The finishing question is not “How much did the cache hit?” It is “Which equivalence claim authorised each hit, and can we prove when that claim stopped being true?”