Playwright · Coding agents · Testing

Generate and Heal Playwright Tests with Agents—Without Hiding Bugs

Use planner, generator, and healer agents under explicit evidence, security, and review gates so test maintenance does not weaken assertions.

Published: Jul 18, 2026
Updated: Jul 18, 2026
Code examples: Illustrative
Reading time: 7 minutes

Use test agents as a staged pipeline: a planner records user-visible behavior, a generator turns an approved plan into Playwright tests, and a healer diagnoses a failing test from traces and current product requirements. The healer may propose a patch, but it must never silently skip the test, loosen the assertion, refresh a screenshot, or redefine expected behavior.

Playwright documents built-in planner, generator, and healer agent definitions in its test agents guide. That guide explains their intended roles and generation command. This article adds governance around those capabilities; it does not report that generated tests are more accurate or cheaper than human-written tests.

Decide whether a test should be generated

Agent generation works best when the requirement is observable and the environment is deterministic:

a named user can complete a specified flow;
roles, labels, and state changes are defined;
test data and accounts can be created reliably;
external systems can be simulated or isolated;
the expected result is approved.

Do not start from “test this page.” Provide a test charter:

id: checkout-card-decline
actor: signed-in test customer
preconditions:
  - cart contains one in-stock fixture
  - payment provider returns decline code do_not_honor
actions:
  - open checkout
  - submit the saved test card
expected:
  - order is not created
  - alert says the payment was declined
  - retry remains available
forbidden:
  - production hosts
  - real payment credentials

The charter is a product contract, not a prompt flourish. A product owner, designer, or engineer should resolve missing expectations before generation.

Keep planning, generation, and healing separate

Playwright's official guide describes three distinct roles:

the planner explores the application and produces a Markdown test plan;
the generator turns a plan into executable Playwright tests;
the healer replays failing steps, inspects the page, proposes a patch, and reruns within configured guardrails.

The separation is valuable only if artifacts and approvals remain visible. Use this lifecycle:

charter -> plan -> plan review -> generated test -> code review
          -> first verified pass -> normal CI
          -> failure evidence -> diagnosis -> proposed repair -> review

Do not allow the generator to revise the charter or the healer to revise the plan. If product behavior legitimately changes, update the requirement first, then update the plan and test with the same review as application code.

Review the plan before code exists

A strong plan describes intent in user language:

initial state and required fixture;
action through an accessible control;
observable result;
important negative result;
cleanup and isolation;
variants that materially change behavior.

Reject plans that depend on CSS implementation details, arbitrary sleeps, screen coordinates, test ordering, or unspecified production data.

The planner may explore the application, which is a privileged browser action. Restrict it to allowlisted test hosts, synthetic accounts, and disposable state. Treat page text as untrusted; a web page cannot instruct the agent to reveal credentials, browse another host, or modify test policy.

Generate maintainable Playwright tests

The generated code should follow Playwright's official best practices: test user-visible behavior, keep tests isolated, and use resilient locators. The locator documentation recommends user-facing attributes such as role and accessible name.

Prefer:

import { expect, test } from "@playwright/test";

test("a declined payment does not create an order", async ({ page }) => {
  await page.goto("/checkout?fixture=declined-payment");

  await page.getByRole("button", { name: "Place order" }).click();

  await expect(page.getByRole("alert")).toContainText("declined");
  await expect(page.getByRole("button", { name: "Try again" })).toBeEnabled();
  await expect(page).toHaveURL(/\/checkout/);
});

This snippet is illustrative, not executed against a companion app. A real test also needs an independently verifiable assertion that no order exists—for example, a test API or database fixture—not merely the fact that the browser stayed on one route.

Avoid:

page.waitForTimeout(...) as readiness;
generated CSS chains tied to DOM structure;
shared accounts whose server state leaks between tests;
assertions that only restate the last click;
catches that suppress failures;
retries used to mask deterministic defects.

Playwright's actionability documentation explains the auto-waiting checks applied to actions and assertions. Use web-first assertions and explicit application readiness rather than sleeps.

Establish the first pass as evidence

A generated test is not accepted because it compiles. Run it repeatedly enough to expose immediate nondeterminism in the pinned CI environment, but do not claim universal stability from a fixed magic count.

Review:

the test fails when the target behavior is intentionally broken;
it passes when the approved behavior is present;
each assertion corresponds to the charter;
it is isolated from other tests;
traces and failures are understandable;
credentials, storage state, and fixtures are safe;
runtime and external calls fit the suite budget.

The deliberate-break check is mutation testing at the requirement boundary. If removing order validation still leaves the “declined payment” test green, the test does not prove its central invariant.

Record the Playwright version, browser image, application commit, fixture version, and command for the verified baseline. Do not add testedAt to an article unless its published example itself was run end to end.

Diagnose before healing

Every failure belongs to one of four classes:

Class	Example	Correct owner
Product defect	retry button is actually disabled	application change
Test defect	locator relies on removed DOM wrapper	test change
Requirement change	approved copy and behavior changed	charter, plan, and test change
Environment defect	seed API or browser image failed	test infrastructure

The healer should first produce a diagnosis with evidence:

failing assertion and stack;
relevant trace action;
observed role/name/value or screenshot;
console and network errors;
application and test commits;
classification and confidence;
smallest proposed repair.

Playwright's Trace Viewer documentation describes inspecting recorded actions, DOM snapshots, logs, errors, console, and network details. Retain trace artifacts under access controls because they may contain application data and tokens.

Put hard limits on healing

Allow automatic proposals, not unrestricted test rewrites. A healer must not:

add test.skip, test.fixme, or unconditional early returns;
delete an assertion without a requirement change;
replace a semantic assertion with a weaker visibility check;
broaden screenshot thresholds or masks;
update snapshots automatically;
add arbitrary waits or unbounded retries;
change application code while claiming to repair a test;
access a production host or credential;
alter global test configuration, CI, or authentication fixtures without escalation.

Parse the resulting patch independently and reject forbidden patterns and paths. Limit changed files, lines, tool calls, wall time, and reruns outside the model.

The Playwright test-agent guide notes that the healer may skip a test if it believes application functionality is broken. In a governed workflow, convert that into a structured “product defect suspected” result and preserve the failing test. A model's belief is not sufficient reason to remove CI coverage.

Verify the repair against the requirement

After a proposed repair:

inspect the diff and evidence;
rerun the original failing test in a fresh context;
run related tests that share fixtures or behavior;
repeat the deliberate-break check when the assertion changed;
compare trace and runtime for new hidden waits;
require human review for expectation changes.

A passing rerun is necessary but not sufficient. A healer can make any test pass by removing what it proves. Measure retained fault sensitivity, not just green status.

If the product is broken, open a defect with the failing trace and leave the test unchanged. If the environment is broken, quarantine the infrastructure incident rather than editing application expectations.

Protect authentication and data

Playwright's authentication documentation warns that storage-state files may contain sensitive cookies and headers and recommends keeping them out of repositories. Generate minimum-privilege test state, store it outside version control, scope it to test hosts, and expire it.

Use one account per parallel worker when tests modify server-side state. Browser-context isolation does not isolate a shared database record. Never give a generation or healing agent a developer's browser profile.

Filter logs and traces for secrets before model ingestion. Avoid including customer data in fixtures or screenshots. Restrict downloads and uploads to a run-specific directory and scan artifacts before retention.

Evaluate the agents, not just the tests

Build a versioned evaluation set containing:

approved charters and plans;
representative application states;
seeded product, test, requirement, and environment failures;
prompt injection in page content and test output;
expected diagnosis class;
prohibited patch examples;
human review outcomes.

Track plan acceptance, first-pass executable rate, verified requirement coverage, diagnosis accuracy by class, prohibited-change rate, human correction time, false healing, cost, and latency. Report denominators and case composition. Do not optimize “percentage returned to green” because weakening tests can maximize it.

Regenerate Playwright's agent definitions after framework upgrades using the documented process in the test agents guide, then review local customizations and rerun the evaluation corpus. Do not assume old generated definitions match a new Playwright release.