An AI agent that opens a web page does not see the page the way you do. It sees a data structure called the accessibility tree, a filtered, semantic version of the Document Object Model (DOM) that browsers build to describe what is on screen to software rather than to eyes. When that tree lags behind what has actually been painted, an agent can be handed a page that looks finished to a human and reads as empty, half built, or silent to a machine. This is the actual mechanism behind a familiar complaint from anyone building browser automation today: the agent clicked nothing, waited on a state that had already resolved, or missed a dialog a human would have closed without thinking. The paint and the semantics are two separate races, and contemporary front-end frameworks are extremely good at winning the first one while neglecting the second entirely.
What an agent actually reads when a page loads
Every browser maintains at least three representations of a page at once: the DOM, which is the tree of elements and attributes parsed from HTML and mutated by JavaScript; the render tree, which is the DOM filtered down to what will actually be painted, with layout and style resolved; and the accessibility tree, which is built from the DOM specifically to answer questions like what kind of control this is, what its current state is, and what it should be called. The Mozilla Developer Network (MDN) accessibility-tree glossary describes this third structure and how assistive technologies use it rather than interpreting the visual interface directly.
Vision-based agents that reason over screenshots are a partial exception, but even they typically hand off the actual clicking and typing to DOM-level automation, because pixel coordinates drift across viewports and optical text recognition is slow. The dominant pattern in practice, whether the agent is a testing script, a computer-use model, or a scraping pipeline, is to query the accessibility tree or the raw DOM for elements, roles, and states. If a visual change never touches the accessibility tree, meaning no new role, no updated attribute, no text node an assistive technology application programming interface (API) can read, the agent inherits the exact blind spot a screen reader user would.
Four visual states that look done and read as empty
The gap between paint and semantics shows up in a small number of recurring patterns. The table below lines up what a sighted user perceives against what actually lands in the accessibility tree for each one.
| Pattern | What a sighted user sees | What the accessibility tree contains | What closes the gap |
|---|---|---|---|
| Skeleton loading screens | Gray placeholder blocks fade smoothly into real content | Often nothing: no busy state, no announcement of the swap | Set aria-busy="true" during load and clear it, or wrap the region in a live region |
| Client-side hydration gap | Static markup appears instantly, the interactive version mounts moments later | A button or form exists visually before it exists in the interactive tree | Server-render the real semantic elements rather than placeholders, so state matches from first paint |
| Icon-only controls and toasts | A recognizable icon or a message that fades in and out | A control with no accessible name, or a message with no exposed text at all | Give every icon a text alternative and route transient messages through aria-live="polite" |
| Custom modal dialogs | A layered panel that visually blocks the rest of the page | No dialog role, no focus moved, background content remains reachable | Use the native dialog element or apply a dialog role, mark it modal, and move focus programmatically |
Why screenshot-based testing does not catch this
Visual regression tools compare pixels between builds, which is exactly why they miss this category of bug: a skeleton loader that fades into real content and a skeleton loader that fades into a decorative element with no semantic content behind it can render as visually identical screenshots while differing completely in what an agent or assistive technology can extract. Nielsen Norman Group's research on skeleton screens focuses on perceived performance for sighted users and says nothing about what those placeholders expose to the accessibility tree, which is itself instructive: the pattern was designed and studied for one audience and then applied everywhere without anyone checking whether it survived translation to the other.
My suspicion is that most teams respond to agent failures by patching automation scripts with longer wait calls rather than asking why the interface never announced a state change it clearly experienced. That is a reasonable short-term fix and a poor long-term one, since it treats a missing signal as a timing problem instead of what it actually is: a missing signal.
Making visual completion legible to agents and assistive technology together
The fix is the same whether the reader is a screen reader user or a browser automation agent, which is the more useful way to think about this problem. Both are asking the DOM to describe itself, and both are satisfied by the same set of disciplines.
- Reach for semantic HTML elements such as
button,dialog,nav, andtablebefore reaching for a styled generic container, because these elements carry default roles and states that ARIA (Accessible Rich Internet Applications) attributes would otherwise have to reconstruct by hand. - Expose asynchronous state changes explicitly with attributes such as
aria-busy,aria-expanded, andaria-selected, and route transient content such as toasts and validation messages through a live region so their arrival is announced rather than merely rendered. - Follow the patterns in the ARIA Authoring Practices Guide for interactive widgets like dialogs, tabs, and comboboxes rather than improvising role and state combinations, since small deviations are exactly what silently break both automation and assistive technology.
- Treat the accessibility tree as a build artifact worth inspecting directly. Browser developer tools expose it, and modern automation frameworks include accessibility snapshot commands that let a team assert on tree content the same way it already asserts on visual output.
None of this requires abandoning modern rendering techniques. It requires accepting that a page is not finished when it looks finished. It is finished when its DOM and accessibility tree describe the same page the screen is showing, a check that has to run alongside visual review rather than after it, per the guidance in the ARIA Authoring Practices Guide and WebAIM's ARIA implementation techniques.
Key takeaways
- Browser automation agents and screen readers both query the accessibility tree rather than reading pixels, so a page can be visually complete while remaining semantically empty.
- Skeleton loaders, hydration gaps, icon-only controls, and custom modals are the four most common places where visual completion and semantic completion diverge.
- Screenshot-based visual regression testing cannot detect this class of bug because it compares rendered pixels, not accessibility tree content.
- Closing the gap means exposing state through semantic HTML and attributes such as
aria-busy,aria-expanded, and live regions, not adding longer wait times to automation scripts. - The same fix that makes a page legible to an AI agent makes it legible to assistive technology, because both are reading the same underlying tree.
Questions readers ask
Do AI browsing agents read screenshots or the DOM?
Most rely primarily on the DOM and the accessibility tree for locating and interacting with elements, even when a vision model is used for higher-level reasoning, because coordinate-based clicking on screenshots is slower and less reliable than querying elements directly.
What is the accessibility tree, exactly?
It is a structure the browser derives from the DOM specifically to describe roles, names, and states to software such as screen readers and automation tools, and it is distinct from both the raw DOM and the visual render tree.
Can visual regression testing catch pages that fail this way?
No. Visual regression tools compare rendered pixels between builds, and a semantically broken element can be pixel-identical to a working one, which is why this class of failure needs an accessibility tree check rather than another screenshot diff.
Does fixing this for AI agents also help screen reader users?
Yes, because both are consuming the same accessibility tree built from the same DOM, and a fix that exposes state correctly for one audience exposes it correctly for the other.
What is the fastest way to check whether a page has this problem?
Inspect the accessibility tree directly through browser developer tools or an automation framework's accessibility snapshot output, and compare what it contains against what the screen shows during and after a loading or interaction state, rather than trusting a visual pass alone.