Configuration

Configuration

Everything is optional. An empty options object produces a useful run.

export default withSetsquare(defineConfig({/* … */}), {
  wcag: 'AAA', // default 'AA'
  bestPractice: true, // axe's advisory rules; not WCAG failures
  personas: { 'a11y-admin': 'recruiter' },
  themes: { 'dark-suite': 'dark' },
  surfaces: { '/jobs/:id/edit': 'job-editor' },
  mask: ['.candidate-name'],
  outputDir: '.setsquare',
  quietMs: 500, // how still counts as settled
  settleTimeoutMs: 5000, // how long we wait for it
  settleByRoute: { '/pages/editor': { settleTimeoutMs: 15000 } }, // for one route
  loadingSelectors: ['.animate-pulse'], // your own skeletons, beside aria-busy
  verbose: false, // or SETSQUARE_VERBOSE=1 — the whole summary in the terminal
});

What the terminal prints, and what the report is for

The summary ends at the criteria table and points at the report. Everything below that — which rules failed, on which surfaces, with their selectors, and the full criteria × surfaces grid — is in report.html, because it is the part you read by searching rather than by scrolling.

Anything that qualifies the figures stays in the terminal either way: surfaces that could not be checked, scans the settle ceiling cut short. A shorter summary that was also a more confident one would be the wrong trade.

SETSQUARE_VERBOSE=1 prints all of it in the terminal instead, for a CI log where the scrollback is the artifact and there is no report to open.

Rule and surface counts are the stable pair. Element counts follow how much data is on the page, so the same code over a table with fewer rows reports fewer failing elements. Quote the rule count.

When a page gets scanned, and the knobs that decide it

Every scan waits for the page to settle: automatic scans fire when it does, and setsquare.scan() and the scan at the end of each test wait for it before reading the page. A page is settled when all of these hold:

• document.readyState is complete, and neither the URL nor the DOM has changed for quietMs. Any change restarts that window, which is what makes the scanner wait for late content instead of auditing a skeleton.
• Nothing visible still says it is loading: no aria-busy="true", no progress bar without a value (role="progressbar" with no aria-valuenow, or a <progress> with no value), and nothing matching your loadingSelectors. A skeleton waiting on a slow fetch changes nothing in the DOM, so stillness alone would call it settled.
• No animation or transition that will end is still running on a visible element. Text fading in, or a ring sliding into place, has a colour and a position that belong to no state the page is left in.

Two things never hold a page up, or a page could be held for ever: anything invisible (component libraries keep skeletons mounted inside closed panels), and animations that loop for ever, such as a skeleton's pulse or a spinner.

• quietMs (default 500) — how long a page must be still. Shorter is not better: measured against a route that renders a conformant shell and injects its real content a second later, 250ms lands inside that gap and reports a clean page. 500ms does not.
• settleTimeoutMs (default 5000) — how long we wait for a page to settle before scanning anyway. It is a trigger, not a give-up. A page that never goes still — a poller, a carousel, a progress animation — is extremely common, and declining to scan it would produce exactly the "no findings" that reads as "no problems". It runs from when the page last stopped being settled: its navigation, or the first change after it had settled. So a page that never settles is waited on once per navigation, not once for every scan() call and again when the test ends.
• loadingSelectors — your own loading placeholders, added to the ones above. A component library's skeleton is often only a class, Tailwind's .animate-pulse among them, so name yours: loadingSelectors: ['.animate-pulse'].

What it costs. On a page that has already settled, nothing. A test that ends within a quiet window of its last change waits out the rest of that window before its final scan. A page still loading waits for as long as it takes, up to settleTimeoutMs, but never for more than half the time the test has left, or a third of the time Playwright gives it to tear down: waiting for a page is never why a test times out. A scan whose wait was cut short that way says so, because "the page ran out of window" and "we stopped so your test could finish" want different fixes.

What happens to a page that settles late. If it is still on screen when the test ends, the final scan waits for it, up to its ceiling, so the evidence describes the finished page. What cannot be recovered is a surface the test navigated away from while it was still rendering: it is gone before anything can look at it, and the run reports it as visited-but-never-scanned rather than quietly dropping it. Raising settleTimeoutMs does not help there — the limit is the test's pace, not ours.

Every scan says how it got there. Its record carries how long it waited, whether the page settled, and, when it had not, what was still loading, counted by kind: elements marked busy, progress bars, matches for your selectors, running animations, and how long the DOM had been still. The summary counts the scans that read a page that had not settled and says what held them.

The report tells you which problem you have. A run with scans cut short names the longest wait and what held the pages: the number of DOM mutations still arriving, a loading marker still showing, or an animation still running. A page a few mutations from quiet will settle if you give it longer; a page with thousands never will, and no value of settleTimeoutMs changes that. Nor does it help a loading marker that is never removed.

Tuning one route rather than the whole suite

One window for a whole application is usually the wrong shape. A page builder and a login form are not the same kind of page, and the two numbers above pull in opposite directions on them: raising the ceiling for the run makes every permanently-animating page hold every test that touches it for longer, and lowering it scans fast-but-busy pages half-built.

settleByRoute: {
  '/pages/editor': { settleTimeoutMs: 15000 },  // a builder; give it longer
  '/dashboard/*': { quietMs: 250 },             // polls forever; ask for less
  '/blueprint/fields/:id': { settleTimeoutMs: 2000 },
}

Keys are route patterns matched against the page's own path, segment by segment: :name matches exactly one segment of anything, a trailing * matches the rest, and everything else matches itself. Where several patterns match, the one with the most literal segments wins, so /pages/editor/new beats /pages/editor/:id.

Either half may be set on its own, and whatever a route does not mention keeps the run's value. The report names the routes that ran out of window, so the list you need is the one it already printed.

One difference from surfaces, which takes the same syntax: there, :id means "a segment we judged to be an identifier", because that runs after the URL has been normalised. Here it means "any one segment", because this runs inside the page, which knows its URL and not our judgement about it. So /blueprint/fields/:id covers /blueprint/fields/new as well as /blueprint/fields/42.

Every run ends with one line stating what it is: "14 rules failing across 28 surfaces, this run only — local runs keep no history, no baselines and no comparison with a previous run, by design." That is not switchable, because it is a limit on the artefact you are holding rather than a preference.

2.5.5 measures the region that accepts a pointer, not the one that is painted. A control drawn 24×24 whose hit area is extended to 44×44 — by a pseudo-element, by padding, or by the control itself being the larger box — meets the criterion, and the finding says which of those it found. The hit test only runs when the painted box is already too small, so a control that is genuinely 44×44 costs nothing.

Contrast has three tiers, and the rule picks between them

WCAG defines text as a sequence "expressing something in human language". Every glyph is a text node, so a contrast check has to answer that first — and the answer changes the bar by more than double:

A ✕ in a close button is a text node and is not human language, so 3:1 applies to it. Each finding says which criterion it was held to and why, in appliedCriterion and appliedBecause, and which signal decided that, in textKindBecause. expectedContrastRatio is always the bar the row was held to.

The bar only ever moves down on positive evidence — a private-use codepoint, an icon ligature detected by pixel comparison, or a run with no letters or digits in any script. A single letter could be an avatar initial or a one-letter label, and nothing in the DOM separates them, so it keeps the stricter bar. Guessing in the other direction would excuse a real failure by reclassifying it, which is far harder to notice than a missing finding.

aria-hidden is not evidence. Both contrast criteria are about what a person sees, and their only hidden-text exception is text "not visible to anyone". Visible words under aria-hidden="true" are text, and are held to the text bar.

Why AA by default. It is what conformance means in practice, and what EN 301 549 and procurement ask for. AAA is a deliberate commitment — WCAG itself does not recommend it as a general policy — so opting in is the honest default.

personas, themes and surfaces map your names to ours, and the direction is the same in all three: the key is what is in your repository, the value is the label you want in the report. So personas: { 'a11y-admin': 'recruiter' } means the Playwright project named a11y-admin is reported as the persona recruiter; themes: { 'dark-suite': 'dark' } reads a project name and reports a theme; and surfaces maps a URL pattern to a surface name. Keys are Playwright project names in the first two and URL patterns in the third — nothing here maps the other way round.

Themes are also read from data-theme on the live page when your suite toggles them in-test, so a suite whose project names carry no theme still gets one context per theme without configuring anything.