When Should an AI Agent Ask for Help?

An AI agent should ask for help when proceeding would require a material assumption, exceed its authority, rely on insufficient evidence, risk a consequential side effect, or consume resources beyond the approved budget. Escalation should be triggered by observable conditions and policy—not by a model's self-reported confidence alone.

This guide turns that principle into a decision method and a typed escalation contract. “Help” can mean asking the user a focused question, requesting approval, routing to a domain expert, or stopping with a diagnostic.

Separate four kinds of escalation

Different blockers need different responses:

Do not ask a random user to resolve an infrastructure incident, and do not ask an operator to guess the user's business preference. Define the recipient and response authority for every escalation class.

NIST AI RMF 1.0 says human roles and responsibilities in AI decision-making and oversight should be clearly defined and differentiated; it also notes that appropriate human-AI configurations vary by context. NIST AI RMF 1.0, Appendix C

Escalate on policy signals, not model mood

Start with deterministic triggers:

The goal is materially ambiguous

Ask when two plausible interpretations would produce different external effects, costs, audiences, or irreversible choices. Do not ask about details that can be safely represented as defaults and clearly disclosed.

Example: “Clean up old customer records” is ambiguous about age, tenant, retention policy, and whether cleanup means archive or delete. A prose formatting preference is usually not material.

Required authority is absent

Stop before an operation whose tool, target, scope, or risk class is not in the current capability grant. Never allow the model to grant itself a broader permission or choose its own approver.

Evidence is insufficient or conflicting

Escalate when a required claim lacks a source, two authoritative sources conflict, the information is stale for the decision, or the evidence cannot be accessed. The agent should name the missing evidence instead of filling the gap with plausible prose.

A side effect has an unknown outcome

If a payment, deployment, message, or deletion times out after submission, do not ask the model whether it “probably failed.” Reconcile through an operation ID or route to an operator. The durable-agent guide covers this state in detail.

Risk or budget crosses a boundary

High-impact actions require approval even when the model appears confident. Budget exhaustion requires an explicit larger grant or an honest partial result. The budget-controller guide provides the accounting pattern.

OWASP recommends explicit approval for high-impact or irreversible actions, action previews, risk-based autonomy boundaries, audit trails, and the ability to interrupt or roll back operations. OWASP AI Agent Security Cheat Sheet

Use a typed decision instead of free-form uncertainty

type EscalationKind =
  | "clarification"
  | "authorization"
  | "expert_review"
  | "operational_intervention";

interface Escalation {
  readonly id: string;
  readonly kind: EscalationKind;
  readonly runId: string;
  readonly reasonCode:
    | "material_ambiguity"
    | "missing_permission"
    | "insufficient_evidence"
    | "conflicting_evidence"
    | "outcome_unknown"
    | "risk_threshold"
    | "budget_exhausted";
  readonly blockedStepId: string;
  readonly question: string;
  readonly choices?: readonly {
    id: string;
    label: string;
    consequence: string;
  }[];
  readonly evidenceIds: readonly string[];
  readonly expiresAt: string;
}

The model may propose an escalation, but code should create it only after checking the state and policy. Conversely, code should force escalation for non-negotiable triggers even when the model proposes proceeding.

Do not use a single numeric “confidence” as permission. The meaning of a model-generated score is not stable across prompts, models, or task classes unless it has been calibrated on representative labeled data. Use confidence as one signal in an evaluated policy, never as the sole gate for a high-impact action.

Ask the smallest answerable question

A good clarification contains:

1. the decision that blocks progress;
2. why it matters;
3. mutually exclusive options where appropriate;
4. the consequence of each option;
5. a safe default only if one exists; and
6. what work can continue without the answer.

Bad:

What would you like me to do?

Better:

“Old records” could mean records older than 30 or 90 days. Archiving is reversible; deletion is not. Which age threshold and action should apply to tenant X?

Do not reveal secrets or copy an entire hostile document into the question. Summarize the decision using trusted application labels and show exact consequential parameters through a protected preview.

Clarification is not approval

Answering a missing field does not necessarily authorize an action. A user can specify a payment recipient without approving the transfer. Keep these events distinct:

type HumanResponse =
  | {
      readonly type: "clarification";
      readonly escalationId: string;
      readonly answer: unknown;
    }
  | {
      readonly type: "approval";
      readonly actionDigest: string;
      readonly decision: "approve" | "reject";
      readonly expiresAt: string;
    };

For protocol-based tools, MCP 2025-11-25 elicitation provides structured and URL modes for requesting user interaction. Its URL mode allows sensitive flows to occur out of band, and an accept response means the user consented to the interaction—not that the out-of-band operation finished. Preserve that distinction in your own state machine. MCP elicitation specification, 2025-11-25

MCP behavior is protocol-specific; you do not need MCP to implement the escalation contract above.

Resume from recorded state

An escalation suspends one step; it should not force the model to reconstruct the entire workflow from conversation history. Persist:

• the blocked step and validated inputs;
• the exact reason and required response type;
• evidence already collected;
• the policy and artifact versions;
• an expiry;
• which actor or role may answer; and
• whether unaffected work may continue.

On response, authorize the actor, validate the response shape, check expiry, and re-evaluate policy. If source facts or action parameters changed while waiting, invalidate the response and create a new escalation.

Avoid escalation fatigue

Too many prompts cause users to approve mechanically or abandon the workflow. Reduce avoidable questions by:

• collecting related low-risk preferences together;
• remembering preferences only with explicit scope and expiry;
• using reversible, visible defaults for low-impact decisions;
• improving tools and evidence retrieval;
• asking once after gathering all currently knowable facts;
• measuring repeat escalations by reason code.

Do not batch unrelated high-impact approvals into one “continue” button. The goal is fewer unnecessary interruptions, not weaker or ambiguous approval.

NIST's Generative AI Profile recommends documenting knowledge limits and how outputs are overseen, and assessing output accuracy and authenticity using known ground truth, human oversight, automated evaluation, and review of content inputs as appropriate. NIST AI 600-1

Test both over-escalation and under-escalation

Build paired evaluation cases:

• same request, but one has a safe default and one has a material ambiguity;
• same action, but one is read-only and one is irreversible;
• same claim, but one has sufficient primary evidence and one has conflict;
• same timeout, but one occurred before dispatch and one after submission;
• same user response, but one comes from the authorized approver and one does not.

Score:

• required escalations that occurred;
• unnecessary escalations;
• unsafe actions attempted without escalation;
• question answerability;
• correct recipient and response type;
• successful resume without losing state;
• stale or replayed responses rejected.

Never optimize only for fewer human interventions. A system can achieve zero interventions by acting unsafely.

Escalation decision checklist

Before proceeding, ask:

1. Would a different plausible interpretation materially change the result?
2. Is the exact action within the current user's and agent's authority?
3. Does every required factual claim have adequate, current evidence?
4. Is the previous side effect's outcome known?
5. Is the action reversible, reviewable, and within the risk policy?
6. Can the required work fit inside the remaining budget and deadline?

If any answer is “no,” choose the matching escalation type. If the answer is unknown, do not silently treat it as “yes.”