Supervisor vs. Handoffs vs. Agent Graphs: A Reproducible Comparison

A supervisor, a handoff system, and an explicit agent graph can all coordinate specialists. None is generally best. The right choice is the smallest topology that meets your task-success and safety requirements under a fixed model, tool, and budget envelope.

This article defines the three designs and gives you a reproducible benchmark protocol. It intentionally reports no winner: the experiment has not been run for your workload, and architecture claims without controlled evidence would be misleading.

Define the topologies precisely

Names vary across frameworks, so compare behavior rather than class names.

Supervisor

A central coordinator owns the user interaction and global state. It delegates bounded tasks to workers, inspects their outputs, and decides the next action.

user ↔ supervisor ─┬→ researcher ─┐
                   ├→ analyst ────┼→ supervisor → result
                   └→ reviewer ───┘

The supervisor is a natural place for budgets, synthesis, and approvals. It is also a bottleneck and a single point where a plausible but incorrect synthesis can hide worker failures.

Handoffs

The active agent transfers control and selected state to another specialist. The receiver becomes responsible for the next turn until it completes or hands off again.

user ↔ triage → specialist A → specialist B → result

Handoffs fit conversations whose ownership genuinely changes, such as triage followed by a domain specialist. They make global invariants harder to enforce if each specialist can reinterpret the task or silently drop state.

Explicit graph

Nodes perform typed operations; edges and conditions in application code determine allowed transitions.

intake → gather ─┬→ verify → synthesize → approve → execute
                 └→ insufficient_evidence → ask_user

A graph makes dependencies and stop states reviewable. It costs more engineering and becomes awkward when every task needs a different path.

Anthropic's December 2024 agent architecture note describes routing, parallelization, orchestrator-workers, and evaluator-optimizer as composable workflow patterns, while advising teams to start with the simplest workable design. That is practitioner guidance, not proof that one pattern wins across tasks. Anthropic, Building Effective AI Agents

Hold the important variables constant

An architecture comparison is invalid if the supervisor gets a stronger model, more context, or more attempts. Build one harness that injects the same:

• model snapshot and inference settings;
• tool schemas and tool implementations;
• source corpus and retrieval index;
• maximum input, output, and total tokens;
• wall-clock and tool-call limits;
• task set and hidden reference answers;
• approval and authorization policy; and
• retry rules and infrastructure.

Use the same specialist instructions where roles are equivalent. The topology may change who calls the specialist, but not what capability it receives.

Record every model call, transition, tool request, tool result, validation error, and terminal reason. Redact sensitive content before telemetry leaves the application. OpenTelemetry's GenAI attributes warn that tool arguments, tool results, and instructions may contain sensitive information; content capture should therefore be an explicit policy choice. OpenTelemetry GenAI attribute registry

Construct a task matrix, not one demo

Use tasks that stress different structural properties:

For each class, create enough independently authored cases to reveal variance. Do not tune prompts on the held-out test cases. Maintain a development split for iteration and a sealed test split for final comparison.

The agent-evals-in-CI guide explains versioned cases, rubrics, repeated trials, and release gates. Reuse that harness so topology is the experimental treatment rather than a new evaluator.

Define outcomes before running the systems

The primary metric should be task success under a rubric that does not reward verbosity. Add architecture-sensitive secondary metrics:

interface TrialResult {
  readonly caseId: string;
  readonly topology: "supervisor" | "handoff" | "graph";
  readonly success: boolean;
  readonly rubricScore: number;
  readonly unsupportedClaims: number;
  readonly unsafeActionAttempts: number;
  readonly humanInterventions: number;
  readonly modelCalls: number;
  readonly toolCalls: number;
  readonly inputTokens: number;
  readonly outputTokens: number;
  readonly latencyMs: number;
  readonly terminalReason:
    | "completed"
    | "budget_exhausted"
    | "blocked"
    | "failed";
}

Measure at least:

1. Success: Did the final artifact satisfy the hidden criteria?
2. Safety: Were unauthorized or approval-bypassing actions attempted or executed?
3. Coverage: Were required subtasks completed and surfaced when missing?
4. Efficiency: How many calls, tokens, tool operations, and seconds were consumed?
5. Reliability: How variable were repeated trials and how often did runs terminate cleanly?
6. Operability: Can an operator identify the owner, state, and failure from the trace?

Report medians and tail latency, plus uncertainty across cases and repeated trials. Pair comparisons by case and seed where the provider exposes deterministic controls, but do not call model inference deterministic merely because a seed is supplied.

Implement one topology interface

Make each candidate conform to the same runner contract:

interface TrialLimits {
  readonly maxModelCalls: number;
  readonly maxToolCalls: number;
  readonly maxTotalTokens: number;
  readonly deadlineMs: number;
}

interface ArchitectureRunner {
  readonly name: "supervisor" | "handoff" | "graph";
  run(
    task: Readonly<{ id: string; prompt: string }>,
    limits: TrialLimits,
    signal: AbortSignal,
  ): Promise<Readonly<{
    artifact: unknown;
    traceId: string;
    terminalReason: TrialResult["terminalReason"];
  }>>;
}

Put limit enforcement in the shared harness. If a graph checks budgets but a handoff prompt merely asks agents to conserve tokens, you are testing enforcement quality as well as topology.

For remote agents, A2A's published specification provides useful common nouns: a Task is stateful work, Message handles communication, and Artifact carries deliverables. The specification says messages should not be used as reliable delivery for critical information and task outputs should be returned as artifacts. Even if you do not implement A2A, model your benchmark records with the same separation. A2A protocol specification

Predict where each design should fail

Write hypotheses before examining results:

Supervisor failure modes

• coordinator context grows with every worker result;
• all work waits on one routing or synthesis decision;
• a compromised worker can influence the supervisor through untrusted output;
• repeated delegation obscures whether the original assignment was completed.

Handoff failure modes

• important constraints disappear at transfer time;
• ownership cycles between specialists;
• the user cannot tell which agent is authorized to act;
• each agent applies a different stop or approval policy.

Graph failure modes

• the predefined graph cannot represent an unanticipated recovery path;
• node and edge proliferation makes changes risky;
• developers encode brittle routing conditions from a small development set;
• graph state is trusted even when a model generated invalid fields.

OWASP recommends testing recursive tool abuse, approval bypass, and multi-agent chaining, with enforced chain depth, token, retry, and cost limits. Include those as executable benchmark cases rather than prose-only concerns. OWASP AI Agent Security Cheat Sheet

Decide with a constrained frontier

Do not collapse quality, cost, latency, and safety into one arbitrary score unless stakeholders have agreed to the weights. First remove candidates that violate hard constraints, such as any unauthorized side effect or a tail-latency requirement. Among the remainder, inspect the quality-cost-latency frontier.

A defensible decision might read:

On evaluation set E version 3, architecture A met the safety gate and had the highest paired task-success estimate among candidates below the specified cost and latency ceilings. Results apply to the pinned models, prompts, tools, and limits in the run manifest.

That statement is narrower—and far more useful—than “graphs are better than supervisors.”

Reproducibility checklist

• Publish or internally version every case and hidden criterion.
• Pin models where providers allow it; otherwise record exact returned model identifiers.
• Share specialist instructions and tools across candidates.
• Enforce identical budgets in code.
• Separate development and final test cases.
• Run repeated, paired trials in randomized topology order.
• Preserve failed runs and terminal reasons.
• Review traces for evaluator leakage and unsafe attempts.
• Report uncertainty and raw per-case outcomes.
• Re-run after model, prompt, tool, or policy changes.