AI Agent Memory Poisoning: Threat Model, Detection, and Recovery

Memory poisoning occurs when untrusted or misleading input becomes persistent state that influences later agent decisions. Unlike a one-turn prompt injection, a poisoned memory can survive the original session, gain apparent legitimacy through repetition, and affect users or tools that never saw the source. Defend the memory write path as carefully as a database mutation.

The core controls are provenance, typed memory classes, source-dependent write policy, quarantine, contradiction handling, least-privilege retrieval, and a recovery process that can remove every derived copy.

Map the attack path

A typical chain is:

untrusted content
  → model reads instruction as data
  → agent proposes memory
  → weak writer persists it
  → later retrieval treats memory as trusted context
  → model changes a decision or invokes a tool

OWASP’s 2026 discussion of agent memory describes the central issue: persistent context carries untrusted influence forward and turns memory into an attack surface. Its example shows how an ordinary repository workflow can create persistent prompt-injection risk. See OWASP, “Memory Is a Feature. It Is Also an Attack Surface”.

NIST classifies both data poisoning and indirect prompt injection in its adversarial machine-learning taxonomy. The taxonomy is useful for consistent names, but NIST also notes that mitigations have limitations; no single filter establishes safety. See NIST AI 100-2e2025.

Separate observations from trusted state

An agent may record that a document contained text without accepting the text as a fact or instruction. Use explicit classes:

Model-visible documents must never create or modify controlled policy. Repeated untrusted statements do not become verified through frequency.

Put a deterministic gate before persistence

The model may propose a memory, but application code decides whether the class is writable:

type MemoryProposal = {
  kind: "observation" | "preference" | "verified_fact";
  subjectId: string;
  statement: string;
  sourceIds: string[];
  requestedTtlSeconds: number;
};

function mayPersist(proposal: MemoryProposal, context: WriteContext) {
  if (!context.authenticatedPrincipal) return false;
  if (proposal.sourceIds.length === 0) return false;
  if (proposal.requestedTtlSeconds > context.maxTtlSeconds) return false;
  if (proposal.kind === "verified_fact" && !context.hasAuthoritativeToolEvidence) {
    return false;
  }
  return context.allowedKinds.includes(proposal.kind);
}

This example omits sensitive-data detection and domain policy. Its important property is that model prose cannot grant itself a trusted class or longer lifetime.

Limit size, count, subject scope, lifetime, and write rate. Bind the memory to user, tenant, task, source version, and writer version. Store an immutable audit event before publishing the memory to retrieval.

Quarantine risky proposals

Quarantine when a proposal:

• originates in retrieved web, email, repository, or uploaded content;
• contains instructions about credentials, permissions, or future tool use;
• attempts to override system or organizational policy;
• refers to another user or tenant;
• contradicts active high-confidence state;
• requests unusually broad or long retention; or
• cannot resolve its source provenance.

Quarantined content can remain available to a security reviewer without entering normal context assembly. Do not ask the same potentially compromised agent to approve its own memory.

Detect contradictions without trusting similarity

Retrieve candidate memories about the same subject and predicate, then compare polarity, time, authority, and scope. Embedding similarity alone cannot distinguish “allow production writes” from “do not allow production writes.”

Create alerts for:

• rapid changes to stable preferences or policies;
• one source causing many memory writes;
• memories retrieved across unusual tasks;
• an untrusted source superseding an authoritative one;
• instructions that mention the memory system itself; and
• a memory that repeatedly precedes denied tool calls.

Behavioral signals help investigation but do not replace preventative write rules.

Constrain retrieval and use

Apply tenant, principal, purpose, memory class, expiration, and trust filters before semantic ranking. Label every retrieved memory with provenance and class in the model context. Treat observation text as quoted data.

Even a legitimate memory should not directly authorize an irreversible action. Recheck current state through an authoritative tool and require approval according to risk. Memory can suggest which account the user usually chooses; it should not prove the user authorized a payment today.

Build recovery before the incident

Recovery needs a provenance graph from raw source to observations, summaries, embeddings, caches, and outputs. When poisoning is confirmed:

1. disable the implicated memory from retrieval;
2. identify every derived record and affected principal;
3. revoke credentials or actions influenced by it when applicable;
4. delete or quarantine derived vectors and summaries;
5. replay affected decisions against clean state;
6. notify owners according to incident policy; and
7. add the attack path to regression tests.

Deleting one visible memory row is insufficient if a consolidated summary, prompt cache, or evaluation corpus still contains its instruction.

Red-team the lifecycle

Test delayed attacks: plant an instruction in one session and trigger it later from an unrelated task. Test authority laundering, where several untrusted sources repeat one claim. Test cross-tenant identifiers, encoded instructions, contradiction floods, deletion followed by re-consolidation, and memory imported from backups.

Record both attack success and legitimate-memory utility. A defense that disables all useful memory may be safe but does not satisfy the product goal; the no-memory design remains a valid alternative when benefits are marginal.