Prompt injection · AI agents · Security

Prompt Injection Defense in Depth for Tool-Using Agents

Contain direct and indirect prompt injection with provenance, privilege separation, constrained tools, output validation, egress controls, and bound approvals.

Published
Updated
Code examples
Illustrative
Reading time
6 minutes

Prompt injection cannot be reduced to finding phrases such as “ignore previous instructions.” A defensible agent assumes some hostile content will reach the model and prevents that content from acquiring authority: untrusted data stays labeled, tools enforce least privilege, sensitive outputs are validated, egress is constrained, memory writes are quarantined, and high-impact actions require an approval bound to the exact effect.

OWASP defines direct injection through user input and indirect injection through documents, pages, email, code, tool results, or other external content. It recommends layered controls and explicitly treats filters and guardrail models as components rather than complete solutions (OWASP LLM Prompt Injection Prevention Cheat Sheet).

Model the injection path as authority flow

attacker-controlled page
        │
        ▼
browser/retrieval tool ──► model context ──► proposed tool call
                                                │
                                                ▼
                                         privileged service

The vulnerability is not merely that the model followed bad text. Harm occurs when data crosses into instructions and then reaches authority. Break that chain at multiple independent points.

A realistic security invariant is:

No text obtained from users, retrieval, browser pages, email, files,
memory, or other agents can independently expand tool authority or
authorize an external effect.

Layer 1: minimize and label context

Retrieve only the fields and spans needed for the task. Attach machine-readable provenance:

type ContextItem = Readonly<{
  source: "user" | "ticket" | "web" | "tool" | "memory";
  sourceId: string;
  trust: "trusted_instruction" | "untrusted_data";
  content: string;
  fetchedAt: string;
}>;

Only application-owned policy belongs in trusted_instruction. A page saying “SYSTEM” remains untrusted data. Delimiters and structured messages help the model understand that distinction, but they are not an access-control boundary.

Strip active rendering features that are not needed, including scripts and remote resources. Normalize encodings for inspection, bound bytes and tokens, and preserve the original content hash for investigation. Do not silently rewrite source evidence so heavily that reviewers cannot reproduce what the model saw.

Layer 2: detect and route suspicious content

Use deterministic checks for known markers, hidden Unicode, unexpected markup, encoded payloads, or data-exfiltration URLs. A classifier can add coverage for semantic attacks. Route suspicious content to quarantine, reduced-capability processing, or human review.

Treat detection as a signal:

type InjectionAssessment = Readonly<{
  disposition: "allow" | "constrain" | "quarantine";
  reasons: readonly string[];
  detectorVersion: string;
}>;

Do not make “not detected” equivalent to safe. Attackers can vary wording and distribute instructions across turns. Rate limits slow systematic probing but do not establish semantic safety.

If a guardrail model judges inputs or actions, isolate its prompt and data, log its version and decision, and keep deterministic enforcement behind it. The guardrail is also a model and can fail or be attacked.

Layer 3: split reading from acting

Place untrusted-content processing in a low-authority component with no write tools or secrets. It returns a bounded structured extraction, not executable instructions:

{
  "documentId": "doc_81f",
  "facts": [
    {
      "claim": "The maintenance window begins at 02:00 UTC.",
      "sourceSpan": { "start": 381, "end": 430 }
    }
  ],
  "suspiciousContent": true
}

A privileged planner may consume the extraction only after schema and provenance validation. It should not receive hidden document layers or raw markup unless the task requires them.

This separation reduces the route from hostile text to authority, but the structured extractor can still encode malicious content in fields. Constrain field semantics, lengths, URLs, and permitted values; do not pass a free-form “recommended command” field into an executor.

Layer 4: authorize tools outside the model

Every proposed call passes through a policy decision using verified principal, tenant, tool, object, requested effect, and current approval—not the model's claim that the user approved.

type ToolDecisionInput = Readonly<{
  principalId: string;
  tenantId: string;
  tool: string;
  resourceIds: readonly string[];
  effect: "read" | "preview" | "commit" | "delete";
  capabilityId: string;
  approvalId?: string;
}>;

Use outcome-specific tools. Replace http_request(url, body) with get_public_advisory(advisoryId). Replace send_email(to, body) with create_reply_preview(ticketId) and send_approved_reply(previewId, approvalId).

The model cannot expand an allowlist, choose credentials, select a tenant, or add an arbitrary destination. Validate schemas, object ownership, state transitions, idempotency, rate, cost, and deadlines. OWASP's agent guidance recommends minimum tool sets, per-tool scopes, explicit authorization for sensitive operations, and external validation (OWASP AI Agent Security Cheat Sheet).

Layer 5: constrain network and output channels

Injection often needs an exfiltration channel. Block arbitrary outbound requests, URL schemes, private addresses, redirects, and DNS-rebinding paths. Allowlist known services where possible and send requests through a controlled egress proxy. The model should not set authorization headers.

Validate model output for its destination:

  • JSON must satisfy the exact schema and semantic constraints;
  • SQL uses parameterized application queries, not model text;
  • shell execution uses an allowlisted operation API, not a command string;
  • rendered Markdown/HTML is sanitized and cannot auto-load remote resources;
  • URLs are parsed, resolved, and policy-checked; and
  • secrets and cross-tenant identifiers are redacted before display or telemetry.

Output validation is not about whether prose sounds reasonable. It ensures the value is safe for the next interpreter.

Layer 6: bind approval to the exact action

For external messages, payments, deployments, deletion, or permission changes, show the operator the real target and effect. Store a digest over canonical action fields:

action type + tenant + target IDs + recipient + content hash + amount
+ currency + policy version + expiry

The commit endpoint re-computes and compares the digest. If anything changed, approval is invalid. Do not ask the same model that proposed the action to decide whether approval is needed.

Approval is valuable only when the reviewer has enough context and time. Repeated low-value prompts produce automation bias and approval fatigue.

Layer 7: quarantine memory and delegation

Never write raw tool output or retrieved content into durable memory automatically. Store source, trust, tenant, author, creation time, expiry, and validation status. Separate user preferences from factual claims and executable workflow state. Read memory under the same tenant and purpose constraints used to write it.

Treat another agent's messages and artifacts as untrusted input. Delegation does not transfer all authority. Give the peer a narrow task token and validate its artifact before it influences a tool.

Test outcomes, not blocked phrases

Create attack cases across direct, indirect, stored, encoded, multimodal, multi-turn, tool-result, and delegated-agent inputs. Vary wording and placement. Assert security invariants:

  • no unauthorized tool executes;
  • no sensitive data leaves an allowed sink;
  • no cross-tenant object is read;
  • no untrusted item becomes durable trusted memory;
  • no high-impact commit occurs without a matching approval;
  • budgets stop repeated attempts; and
  • telemetry contains enough provenance to reconstruct the decision.

A test that passes because the model refused one famous jailbreak is not a security regression suite. Run cases across model and prompt updates because behavior changes.

Incident response and recovery

If injection succeeds, revoke exposed credentials, disable affected tools or egress, quarantine poisoned memory, preserve prompts and tool transcripts under controlled access, identify every action and tenant touched, and rotate filters only after architectural containment. A kill switch should stop new commits without destroying evidence or blocking safe read-only investigation.

Defense checklist

  • Mark all external content as untrusted data with provenance.
  • Minimize content and strip unnecessary active features.
  • Use detection for routing, not as proof of safety.
  • Separate untrusted-content readers from privileged actors.
  • Enforce authorization, schemas, and budgets outside the model.
  • Replace generic tools with bounded outcomes.
  • Constrain egress and validate every interpreter boundary.
  • Bind approval to immutable action details.
  • Quarantine memory writes and distrust peer-agent outputs.
  • Test invariants across attack variants and model changes.

For browser-specific isolation and URL controls, see the secure Playwright agent guide. For persistent context, use the memory-poisoning defenses.