Writing
Your axe scan passed. Here's what it didn't check.
We audited three demo sites by hand. Automated checks fully caught 6 of 47 barriers, and none of the 7 that stopped people completely.
· Doorwide · 3 min read
Automated accessibility checks are worth running. They are fast, cheap and catch real problems, and every team should have one in CI. The trouble starts when a green scan gets read as "we're accessible".
To see how big that gap is, we built three demo sites that copy patterns we often see on real Danish sites: a homeware webshop, a ferry booking flow and a savings app sign-up. Then we audited them the way we audit client work, by keyboard, at 400% zoom and against every WCAG 2.2 AA criterion. We also ran axe-core, the engine behind most automated checkers, on every page.
The numbers
Across the three sites we found 47 barriers. axe-core fully flagged 6 of them and partly flagged 4 more. Seven of the barriers were blockers, meaning a person could not finish the journey at all. axe fully flagged none of the seven.
That fits the wider research. Deque's coverage report found that automated tools can flag issues on only 16 of the 50 WCAG 2.1 AA success criteria. Deque also found that, counted by volume, automated tools catch more than half of all issues, because the most common problems, like low contrast, are the easiest to detect. Our share is lower because we count each barrier once, however many times it appears.
What the scan missed
A colour picker made of divs. On the webshop, the colour swatches were <div> elements with click handlers. They passed every rule, because axe has nothing to check on an element with no role. No keyboard user could choose a colour, so no keyboard user could add the product to the basket.
A terms checkbox you couldn't tick. The same pattern at checkout: a styled <div> standing in for a checkbox. The order can't be placed without ticking it.
A date picker and a drag-only slider. The ferry booking's calendar only responded to mouse clicks, and vehicle length could only be set by dragging. Both passed the scan, and both stopped the booking. WCAG 2.2 added a criterion, 2.5.7 Dragging Movements, for exactly this.
An image CAPTCHA. axe did flag the savings app's CAPTCHA, but for the wrong reason: the image had no alt text. Adding alt text would make the scan pass and give away the answer. The CAPTCHA itself is the barrier, and WCAG 2.2's 3.3.8 Accessible Authentication says so.
Things that only happen when you use the page. Errors shown only as a red border. A booking timer that expired silently and wiped the booking. Dialogs that didn't take focus or close with Escape. A scan looks at the page as it is, not at what happens when someone tries to use it.
A 10-minute check before every release
You don't need a full audit to catch most of this. Put the mouse away and run your most important journey, like checkout or sign-up, like this:
- Tab through the whole journey. Can you reach and operate every control, including custom ones like swatches, date pickers and steppers?
- Watch the focus. Can you always see where you are? Does a sticky header, cookie banner or chat widget ever cover the focused element?
- Open every dialog and menu. Does focus move in, does Escape close it, and does focus return to where you were?
- Submit each form empty. Does each error say in words which field is wrong and how to fix it?
- Zoom to 400%, or make the window 320 pixels wide. Can you still use everything without scrolling sideways?
- Turn on a screen reader and tab through. VoiceOver is built into every Mac (Cmd+F5), and NVDA for Windows is free. Does every button and field have a name that makes sense? Are states like "checked" or "selected" announced?
- Look for anything that needs dragging. Is there a way to do it with a click or a key instead?
If something fails, you've found a barrier your scan can't see.
Keep the scan
None of this is an argument against automated testing. Run axe or a similar tool in CI and fail the build on new violations. It's the cheapest accessibility work you'll ever do. Just read a green result as "the easy part is done", not as a pass.
All three demo audits are on our case studies page, with the journeys we tested, what the scan saw and the code we'd use to fix each blocker.