One run across lanes

One logical run across several Playwright invocations

If you run playwright test once — with any number of workers — you already get one report and can skip this section. Worker parallelism inside a single invocation needs no setup: the workers observe, the one reporter writes, and one run lands on disk. That is the default, and most suites never need more.

You need this section only when one suite execution makes several playwright test calls. In practice that means one thing: Playwright's workers setting is global, so a suite that needs a serial lane (tests that write shared state) and a parallel lane (read-only sweeps at --workers=4) has to run as two invocations. Without help, each invocation writes its own run, and one suite execution looks like two half-runs.

Setsquare cannot detect this by itself — nothing on the machine distinguishes "one script making two passes" from "you running your suite twice", and merging two unrelated runs would fabricate a run that never happened. So the script that makes the calls declares it, with three variables: mint one id per execution and export it to every invocation:

$ export SETSQUARE_RUN_ID="run-$(date +%s)-$$"   # one id per suite execution$ export SETSQUARE_RUN_PARTS=2                   # how many invocations you will make $ SETSQUARE_LANE=functional SETSQUARE_AUTOSCAN=0 npx playwright test --project=functional --workers=1$ SETSQUARE_LANE=a11y                       npx playwright test --project=a11y --workers=4

SETSQUARE_AUTOSCAN=0 on the functional lane is the cost lever: axe is the bulk of the SDK's per-test overhead, and a lane whose surfaces the a11y lane scans anyway would pay it twice for evidence the merge deduplicates. The lane still contributes everything else — test outcomes, navigation observations (so the visited-but-never-scanned accounting stays union-aware), and explicit setsquare.scan() calls, which always run. Leave it unset on any lane that is your only scanner of some surface.

All invocations sharing the id merge into one run at .setsquare/runs/group-<id>/ — one run.json, one report.html, one screens.html spanning every lane — with every accounting computed over the union: a surface visited in pass 1 and scanned in pass 2 is covered, not "visited but never scanned". Concurrent invocations are safe; so are the parallel workers inside each one.

The merged run.json carries the group facts where a consumer can query them: source.runGroup holds { id, coverage, completed, expectedParts } — where coverage is complete only against a declared census, partial when evidence is known absent, and unknown otherwise — and every result row carries its lane. Test ids are additionally prefixed (a11y :: …) so the same test run in two lanes stays two rows. The sidecar meta.json holds the full per-invocation breakdown.

The variables:

• SETSQUARE_RUN_ID — the group identity. Mint a fresh id per suite execution (a CI build id is ideal; a branch name is not — reusing an id across executions gets the stale parts excluded with a warning). When the variable is absent, nothing changes: a solo playwright test behaves exactly as before.
• SETSQUARE_LANE (optional) — a label for this invocation. It prefixes the merged run's test ids (a11y :: suite > spec > test), groups the report's provenance table, and keeps the same test's results from two lanes apart.
• SETSQUARE_RUN_PARTS (optional but recommended) — how many invocations this execution will make. This is the only way the merged artifact can say "complete": an invocation that crashes before it starts is invisible, so without a declared count the honest verdict is "unknown". A wrong count fails safe — it reads as partial, never as falsely complete.

If an invocation crashes mid-run, the merged run still exists, is marked partial, and says which invocation's evidence is absent — absent, never clean. Sharding works the same way: SETSQUARE_RUN_ID shared across shards, SETSQUARE_LANE=shard-1 and so on.

Two cautions. Every invocation must resolve the same .setsquare directory (same config directory and outputDir), or the group silently splits. And truly concurrent invocations should give Playwright itself distinct outputDir / report folders — Playwright's own artifacts are not merge-aware; setsquare's are.