Skill Tracer
Skills › gstack › browse › PLAN-snapshot-dropdown-interactive.md

PLAN-snapshot-dropdown-interactive.md

102 lines · browse/PLAN-snapshot-dropdown-interactive.md at 3f79d81

Open on GitHub
1# Plan: Snapshot Dropdown/Autocomplete Interactive Element Detection
2

Plan: Snapshot Dropdown/Autocomplete Interactive Element Detection

Raw lines 1 to 2
3## Problem
4
5`snapshot -i` misses dropdown/autocomplete items on modern web apps. These elements:
61. Are often `<div>`/`<li>` with click handlers but no semantic ARIA roles
72. Live inside dynamically-created portals/popovers (floating containers)
83. Don't appear in Playwright's accessibility tree (`ariaSnapshot()`)
9
10The `-C` flag (cursor-interactive scan) was designed for this but:
11- Requires separate flag — agents using `-i` don't get it automatically
12- Skips elements that HAVE an ARIA role (even if the ARIA tree missed them)
13- Doesn't prioritize popover/portal containers where dropdown items live

Problem

snapshot -i misses dropdown/autocomplete items on modern web apps. These elements:

  1. Are often <div>/<li> with click handlers but no semantic ARIA roles
  2. Live inside dynamically-created portals/popovers (floating containers)
  3. Don't appear in Playwright's accessibility tree (ariaSnapshot())

The -C flag (cursor-interactive scan) was designed for this but:

  • Requires separate flag — agents using -i don't get it automatically
  • Skips elements that HAVE an ARIA role (even if the ARIA tree missed them)
  • Doesn't prioritize popover/portal containers where dropdown items live
Raw lines 3 to 14
15## Root Cause
17Playwright's `ariaSnapshot()` builds from the browser's accessibility tree. Dynamically-rendered popovers (React portals, Radix Popover, etc.) may not be in the accessibility tree if:
18- The component doesn't set ARIA roles
19- The portal renders outside the scoped `body` locator's subtree timing
20- The browser hasn't updated the accessibility tree yet after DOM mutation

Root Cause

Playwright's ariaSnapshot() builds from the browser's accessibility tree. Dynamically-rendered popovers (React portals, Radix Popover, etc.) may not be in the accessibility tree if:

  • The component doesn't set ARIA roles
  • The portal renders outside the scoped body locator's subtree timing
  • The browser hasn't updated the accessibility tree yet after DOM mutation
Raw lines 15 to 21
22## Changes
24### 1. Auto-enable cursor-interactive scan with `-i` flag
26**File:** `browse/src/snapshot.ts`
28When `-i` (interactive) is passed, automatically include the cursor-interactive scan. This means agents always see clickable non-ARIA elements when they ask for interactive elements.
30The `-C` flag remains as a standalone option for non-interactive snapshots.
32```
33if (opts.interactive) {
34 opts.cursorInteractive = true;
36```

1. Auto-enable cursor-interactive scan with -i flag

File: browse/src/snapshot.ts

When -i (interactive) is passed, automatically include the cursor-interactive scan. This means agents always see clickable non-ARIA elements when they ask for interactive elements.

The -C flag remains as a standalone option for non-interactive snapshots.

if (opts.interactive) {
  opts.cursorInteractive = true;
}
Raw lines 24 to 37
38### 2. Add popover/portal priority scanning
40**File:** `browse/src/snapshot.ts` (inside cursor-interactive evaluate block)
42Before the general cursor:pointer scan, specifically scan for visible floating containers (popovers, dropdowns, menus) and include ALL their direct children as interactive:
44Detection heuristics for floating containers:
45- `position: fixed` or `position: absolute` with `z-index >= 10`
46- Has `role="listbox"`, `role="menu"`, `role="dialog"`, `role="tooltip"`, `[data-radix-popper-content-wrapper]`, `[data-floating-ui-portal]`, etc.
47- Appeared recently in the DOM (not in initial page load)
48- Is visible (`offsetParent !== null` or `position: fixed`)
50For each floating container, include child elements that:
51- Have text content
52- Are visible
53- Have cursor:pointer OR onclick OR role="option" OR role="menuitem"
54- Tag with reason `popover-child` for clarity

2. Add popover/portal priority scanning

File: browse/src/snapshot.ts (inside cursor-interactive evaluate block)

Before the general cursor:pointer scan, specifically scan for visible floating containers (popovers, dropdowns, menus) and include ALL their direct children as interactive:

Detection heuristics for floating containers:

  • position: fixed or position: absolute with z-index >= 10
  • Has role="listbox", role="menu", role="dialog", role="tooltip", [data-radix-popper-content-wrapper], [data-floating-ui-portal], etc.
  • Appeared recently in the DOM (not in initial page load)
  • Is visible (offsetParent !== null or position: fixed)

For each floating container, include child elements that:

  • Have text content
  • Are visible
  • Have cursor:pointer OR onclick OR role="option" OR role="menuitem"
  • Tag with reason popover-child for clarity
Raw lines 38 to 55
56### 3. Remove the `hasRole` skip in cursor-interactive scan
58**File:** `browse/src/snapshot.ts`
60Currently: `if (hasRole) continue;` — skips any element with an ARIA role, assuming the ARIA tree already captured it.
62Problem: if the ARIA tree MISSED the element (timing, portal, bad DOM structure), it falls through both systems.
64Fix: Only skip if the element's role is in `INTERACTIVE_ROLES` AND it was actually captured in the main refMap. Otherwise include it.
66Since we can't easily check the refMap from inside `page.evaluate()`, the simpler fix: remove the `hasRole` skip entirely for elements inside detected floating containers. For elements outside floating containers, keep the `hasRole` skip as-is (to avoid duplicates in normal page content).

3. Remove the hasRole skip in cursor-interactive scan

File: browse/src/snapshot.ts

Currently: if (hasRole) continue; — skips any element with an ARIA role, assuming the ARIA tree already captured it.

Problem: if the ARIA tree MISSED the element (timing, portal, bad DOM structure), it falls through both systems.

Fix: Only skip if the element's role is in INTERACTIVE_ROLES AND it was actually captured in the main refMap. Otherwise include it.

Since we can't easily check the refMap from inside page.evaluate(), the simpler fix: remove the hasRole skip entirely for elements inside detected floating containers. For elements outside floating containers, keep the hasRole skip as-is (to avoid duplicates in normal page content).

Raw lines 56 to 67
68### 4. Add dropdown test fixture and tests
70**File:** `browse/test/fixtures/dropdown.html`
72HTML page with:
73- A combobox input that shows a dropdown on focus/type
74- Dropdown items as `<div>` with click handlers (no ARIA roles)
75- Dropdown items as `<li>` with `role="option"`
76- A React-portal-style container (`position: fixed`, high z-index)
78**File:** `browse/test/snapshot.test.ts`
80New test cases:
81- `snapshot -i` on dropdown page finds dropdown items via cursor scan
82- `snapshot -i` on dropdown page includes popover-child elements
83- `@c` refs from dropdown scan are clickable
84- Elements inside floating containers with ARIA roles are captured even when ARIA tree misses them

4. Add dropdown test fixture and tests

File: browse/test/fixtures/dropdown.html

HTML page with:

  • A combobox input that shows a dropdown on focus/type
  • Dropdown items as <div> with click handlers (no ARIA roles)
  • Dropdown items as <li> with role="option"
  • A React-portal-style container (position: fixed, high z-index)

File: browse/test/snapshot.test.ts

New test cases:

  • snapshot -i on dropdown page finds dropdown items via cursor scan
  • snapshot -i on dropdown page includes popover-child elements
  • @c refs from dropdown scan are clickable
  • Elements inside floating containers with ARIA roles are captured even when ARIA tree misses them
Raw lines 68 to 85
86## Rollout Risk
88**Low.** The `-C` scan is additive — it only adds `@c` refs, never removes `@e` refs. The change to auto-enable it with `-i` increases output size but agents already handle mixed ref types.
90**One concern:** The `-C` scan queries ALL elements (`document.querySelectorAll('*')`) which can be slow on heavy pages. For the popover-specific scan, we limit to elements inside detected floating containers, which is fast (small subtree).

Rollout Risk

Low. The -C scan is additive — it only adds @c refs, never removes @e refs. The change to auto-enable it with -i increases output size but agents already handle mixed ref types.

One concern: The -C scan queries ALL elements (document.querySelectorAll('*')) which can be slow on heavy pages. For the popover-specific scan, we limit to elements inside detected floating containers, which is fast (small subtree).

Raw lines 86 to 91
92## Testing
94```bash
95cd /data/gstack/browse && bun test snapshot
96```

Testing

cd /data/gstack/browse && bun test snapshot
Raw lines 92 to 97
98## Files Changed
1001. `browse/src/snapshot.ts` — auto-enable -C with -i, popover scanning, remove hasRole skip in floating containers
1012. `browse/test/fixtures/dropdown.html` — new test fixture
1023. `browse/test/snapshot.test.ts` — new dropdown/popover test cases

Files Changed

  1. browse/src/snapshot.ts — auto-enable -C with -i, popover scanning, remove hasRole skip in floating containers
  2. browse/test/fixtures/dropdown.html — new test fixture
  3. browse/test/snapshot.test.ts — new dropdown/popover test cases
Raw lines 98 to 102