Testing accessibility: layers that catch what axe cannot
getByRole makes every test an a11y smoke test; axe catches roughly a third of WCAG issues — focus order, name quality, announcement timing need a manual screen-reader pass. CI: jsx-a11y lint, role queries, axe on key pages; every a11y bug gets a failing test first.
The quarterly business review slide said “Accessibility: 100% of components covered by automated checks, zero violations.” Six weeks later, the government client’s conformance audit — the kind procurement requires before signature — failed the product in the first session, and the deal slipped a quarter. Every finding had sailed through the green pipeline. Closing the export dialog dumped focus onto the page footer: axe passed, because axe inspects a rendered tree, not a focus journey. Twelve portfolio links were all named “View details”: axe passed, because each link had a name — the rule checks presence, not sense. The success toast appeared and vanished without a live region, so screen-reader users never heard their transfer confirm: axe passed, because nothing in the static tree was invalid. And the custom date picker trapped keyboard focus like a roach motel: axe passed, because axe does not press keys. None of this means the tooling was bad; it means the team had quietly replaced a question — “can a screen-reader user complete the flow?” — with a different, automatable question, and then reported the second answer as the first. Deque, who build axe-core, market it as catching about 57% of issues “by volume”; independent evaluations, like the UK government’s accessibility-tool audit, put automated detection closer to a third of WCAG failures. Plan on the lower number. The team had automated a third and declared a whole.
getByRole is the cheapest gate you already own
Before adding any tool, notice the one already in the build: React Testing Library’s query priority. getByRole("button", { name: "Save" }) is not just a selector — it is an assertion that the accessibility tree contains a button-role node whose accessible name is “Save”. Convert the clickable div from lesson one and the test cannot find it: no role, no match, red test. Strip the icon button’s label and the name option stops matching. Wrap the option list in a div that breaks list semantics, and getAllByRole("listitem") comes back short. Every component test written this way is an accessibility smoke test that costs nothing extra and runs on every PR — which is precisely why RTL’s documentation ranks queries by how well they reflect what AT perceives: role first, then label text, then text, with data-testid as the explicit last resort. The anti-pattern is reaching for testid by default: it pins tests to invisible attributes, removing the pressure that makes semantic regressions fail loudly. A useful team rule: testid requires a comment justifying why no role-based query works.
The same priority shapes component APIs. A test that needs getByRole("dialog", { name: "Confirm deletion" }) forces the dialog component to accept and wire an accessible title; a test that asserts toHaveAccessibleName() on the icon button forces the aria-label prop through. Tests written in the vocabulary of the accessibility tree push the components toward exposing one — the cheap virtuous cycle.
A teammate refactors a Button primitive from a native button to a styled div for design reasons. Which existing test layer fails first, assuming the team queries by role?
What axe automates, honestly
jest-axe wires axe-core into component tests: render, run axe(container), assert no violations. What axe-core actually checks is statically computable facts about a rendered tree: images without alt, form controls without labels, ARIA attributes that are invalid for the role, missing landmark structure, duplicate ids that break aria-labelledby, color contrast. Within that territory it is excellent — fast, deterministic, with a low false-positive rate as a design goal. The boundary is everything that requires judgment or interaction: whether the focus order makes sense, whether “View details” is a good name for the twelfth identical link (name presence is checkable; name quality is not), whether the toast’s announcement timing lets a screen reader finish the sentence, whether Escape closes the picker, whether the alt text actually describes the chart. Put numbers on the boundary and keep them conservative: Deque’s own by-volume claim is around 57%; the UK government’s audit of automated tools and most independent studies land near a third of WCAG failures detected. Either way, a green axe run bounds less than half of the problem — treat it as a floor, never a certificate.
One jsdom-specific honesty note for jest-axe: color-contrast checking needs real layout and rendering, which jsdom does not do — the rule is unreliable there and typically disabled. The serious deployment is axe in a real browser — @axe-core/playwright against your running app — where the full rule set applies to real pages with real CSS. Component-level jest-axe still earns its keep as a fast regression net for ARIA validity and labeling, but the browser run is the one that counts.
Your dialog component has a green jest-axe test. A user reports that after closing the dialog, focus lands on the page footer and NVDA reads the copyright line. Why did the test not catch this, and which layer should have?
The manual pass, the CI gates, and the regression rule
The layer automation cannot reach is covered by a ritual, not heroics: a scripted screen-reader pass over the critical flows — checkout, signup, the money-moving form — before each release. Scripted means a written checklist per flow: can you reach every control with Tab and arrows; does each announce a sensible role, name, and state; does the route change announce; does the dialog restore focus; does the error message get read. Run it with VoiceOver plus Safari on macOS and NVDA plus Chrome or Firefox on Windows — the pairings real users run — and a pass takes twenty to thirty minutes per flow once the script exists. Cadence beats coverage: three critical flows every release outperforms a heroic full-app audit once a year, because regressions cluster where code churns.
In CI, the layers sort by latency and scope. eslint-plugin-jsx-a11y runs at the editor and in lint CI: static JSX analysis — interactive elements without keyboard handlers, missing alt props, ARIA attributes invalid for the role. It is fast and shallow; it sees only literal JSX, not computed values or runtime composition. RTL role queries run with unit tests on every PR. Browser axe runs against key pages on PR or nightly, gating on zero new violations — a ratchet, not a cleanup mandate: existing debt is inventoried, new debt is blocked. The manual pass gates the release itself.
The regression rule ties the system together: an accessibility bug gets a failing test first, exactly like any other defect. Focus-restoration bug — an interaction test asserting document.activeElement after close. Lying aria-expanded — an RTL assertion on the attribute against the open state. Unannounced toast — a test asserting the live-region’s text content changes. Treating a11y findings as untestable folklore is how the same bug ships three times; treating them as defects with reproductions is how the manual pass gets cheaper every quarter, because everything it has ever caught is now pinned by a machine.
▸Why this works
Why is automated coverage capped near a third instead of climbing toward full coverage as tools mature? Because the cap is semantic, not technical. A large share of WCAG criteria are judgment claims — alt text must be equivalent, labels must describe, focus order must preserve meaning — and judgment requires knowing intent, which a DOM does not carry. Another share are interaction properties spread over time: traps, restoration, announcement timing. Tools keep improving inside the computable territory (contrast math, ARIA validity, name presence), and LLM-assisted checkers are pushing at name quality, but the conformance question — can a person using AT complete this task — is a usability claim. The honest posture: automate the floor relentlessly, and budget human verification for the ceiling as a recurring cost, like on-call.
- 01Name what axe-core can and cannot check, with the honest coverage numbers, and the jsdom caveat for jest-axe.
- 02Lay out the four-layer testing strategy with each layer's latency and blind spot, plus the regression rule.
Accessibility testing fails when one automatable question silently replaces the real one. The real question — can a person using assistive technology complete the flow — decomposes into layers, each catching what the previous structurally cannot. React Testing Library’s role queries are the free first layer: getByRole asserts that the accessibility tree exposes the right role and name, so a div-ified button or a delabeled icon fails ordinary component tests on every PR, and testids are demoted to justified exceptions. axe-core automates the second layer — statically computable facts like missing labels, invalid ARIA, broken id references, and contrast — and its honest coverage is roughly a third of WCAG failures (the vendor’s own by-volume claim peaks near 57%); jest-axe brings a fast subset to components, but jsdom cannot do contrast, so the scan that counts runs in a real browser via @axe-core/playwright, gating PRs on zero new violations. The uncoverable two thirds — focus journeys, announcement timing, whether names and alt text mean anything — belongs to a scripted screen-reader ritual over the critical flows before each release, with VoiceOver and NVDA in the pairings users actually run, twenty to thirty minutes per flow. And the regression rule closes the loop: every accessibility bug becomes a failing automated test before its fix — activeElement assertions for focus, attribute assertions for lying state, live-region content for silent announcements — so each manual discovery permanently shrinks the manual surface. Automate the floor; budget humans for the ceiling; never report the floor as the ceiling. Now when you see a green axe run in CI, you know exactly what it guarantees — and what two thirds of the problem it is still leaving for the scripted screen-reader pass and the regression tests you write after every manual finding.
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.
Apply this
Put this lesson to work on a real build.