docs(testing): propose keyless browser e2e lane for the web GUI

Design study for a deterministic, keyless browser e2e lane over the real
assembled web chain (chromium -> SSE/HTTP wire -> apiproxy -> agent loop ->
persistence), replayed through dsh-llm-replay from recorded session-log
fixtures, with aria-tree goldens plus in-process world-state assertions.

Synthesized from an OSS prior-art survey (LibreChat, ai-chatbot, lobe-chat,
OpenHands, cline, aimock...), a repo seam deep-dive, and three adversarial
critiques (doctrine, flakiness, YAGNI). Records the settled shape (no new
package, no suite factory, seed via the real persistence API, whenIdle
barrier stack, no transient-DOM assertions) and the open questions (LLM
seam, Loader-izing dsh web, header pin, golden breadth, settled signal).

.agents/notes/proposed/testing/2026-07-24-web-gui-browser-e2e-lane.md

@@ -0,0 +1,92 @@
+# Agent Note: Keyless browser e2e lane for the web GUI
+
+Status: proposed
+
+## Problem
+
+The web GUI ships as a real assembled chain — chromium page → nine client plugin bundles → HTTP unary RPC + two SSE streams → `toFetchHandler`/apiproxy → `bootHost`'s agent loop, tools, and JSONL persistence — and no test exercises that chain keylessly and deterministically. The [GUI testing system](../../implemented/process/2026-07-20-gui-testing-system.md) covers tier 1 (wire isomorphism in node), tier 2 (object-layer state machines), and a tier-3 smoke pair, but the keyless smoke (`apps/web/tests/smoke-fixture.e2e.ts`) drives `FixtureApiClient` behind `?fixture` — no host, no wire, no agent loop — while the full-chain smoke (`smoke-real.e2e.ts`) needs `DEEPSEEK_API_KEY` and a live model, so it is nondeterministic and self-skips in keyless CI. The snapshot philosophy of [docs/testing.md](../../../../docs/testing.md) — record once with a key, replay forever keyless, refresh on format churn — already covers the ACP, headless `stream-json`, and TUI transcript surfaces; the web surface is the one assembled product shape without it. The gap is exactly where the two confirmed GUI P0s hid: the wire carriage chain the fixture client short-circuits.
+
+## Proposal
+
+Add a keyless, deterministic browser e2e lane under `apps/web/tests/`, driven by recorded session-log fixtures replayed through `@deepseek-ai/dsh-llm-replay`, asserting the rendered accessibility tree plus in-process world state. No new package; no product-code change except (open question 1) an LLM composition knob.
+
+### Harness: `apps/web/tests/harness.ts`
+
+A plain shared-fixture module (the [testing-policy sanctioned shape](../../../../docs/testing.md)), not a package: the gate-worthy logic this lane needs — replay derivation, session parsing, log scrubbing, persistence — already lives in gated packages (`dsh-llm-replay`, `dsh-acp-snapshot`, `dsh-session-persistence-jsonl`); what remains is boot wiring and browser glue, and chromium-driving code cannot hold per-file 100% coverage on the browserless coverage runners.
+
+`launchWebHarness()` boots the real web assembly in-process from the exported production functions — `startHost({ boot: { persistenceRoot: <tmp>, workspaceContext: false, cwd: <tmp workspace> } })`, `installLlmReplay(host.ctx, { file, childFiles, providers })`, `mountWebPlugins(host.ctx)`, `createHostWebPluginRegistry`, `startWebServer({ port: 0, distIndex, apiHandler: host.handler, webPlugins })` — and returns `{ baseUrl, host, workspaceCwd, close }`. This is the web analog of the TUI suite mounting the production bundle in-process ([TUI snapshots](../../implemented/testing/2026-07-18-tui-terminal-state-snapshots.md)): the real entry boundary (`dsh web` bin arg-parsing and dist resolution) stays held by the existing keyless CLI smokes in `smoke-real.e2e.ts`, and the web surface has no `cordis.yml` to bypass — assembly is written in the app per the [GUI layering decision](../../implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.md). Replay runs in providers-catalog mode with a `contextWindow` (the TUI suite's `PROVIDERS` shape), never catch-all: with no adapter registered, catch-all mode would make `compact-basic`'s `resolveModelContext` throw into its post-step catch every step, spamming warnings and silently disabling the pressure path instead of proving it inert.
+
+`seedSession(host, fixtureText)` seeds cold sessions through the real persistence API — a throwaway `Context` mounting `SessionStore` + `SessionPersistenceJsonl` against the host's root, `create()` + `append()`, one `utimes` for deterministic sidebar order (the precedent is `examples/acp-agent/tests/semantic-checkpoint.snapshot.ts`) — never raw file writes, so the seeder needs no knowledge of bucket hashing, filename encoding, or compression. Seeds are validated at seed time (parseable, `seq`-contiguous, ending in `turn/end`) so fixture drift fails loud at the earliest resolvable point rather than as silently dropped frames in the client; a seed not ending in `turn/end` would be mutated by resume's crash repair.
+
+### Determinism rules
+
+The barrier stack, in order, for a prompted turn: (1) host-side `await agent.whenIdle()` under a timeout — the idle flip happens after both the `turn/end` append and the persistence flush, so one await covers turn completion and durability; (2) browser settled poll — streaming node detached, composer restored, final text visible; (3) log harvest only after `host.dispose()`. An in-process `turn/end` listener alone is a wrong barrier (it fires before the SSE frame reaches the browser and before the fsync), and file polling is banned (slow on NFS, superseded by `whenIdle`). For history-open scenarios the barrier is a poll for the last expected message's text, then a poll-until-equal aria capture. `networkidle` is banned outright — it never resolves while an SSE stream is open.
+
+No single-shot transient-DOM assertions: every hop from replay yield to React commit can coalesce chunks, so sampling `[data-streaming]` is a race by construction. Streaming incrementality is asserted from the persisted `assistant/chunk` events (model-visible ⟺ logged makes the log the authoritative proof), optionally corroborated by an in-page MutationObserver latch armed before send — observers cannot miss a commit; polls can. `dsh-llm-replay` gains an opt-in `paceMs` config field (default absent = today's instant yield) as a realism knob so the browser observes genuinely incremental SSE; correctness never leans on the pace.
+
+Every scenario fails on any pageerror and on the client's connection-loss/gap-repair console warnings: the reconnect machine plus history resync would otherwise self-heal a dead SSE path and the suite would certify a broken wire. Harness `close()` asserts every replay script was fully consumed (all scripts bound, every cursor at end), converting silent underruns and shifted bindings into crisp diagnostics; this is a small additive stats handle on `installLlmReplay`. No vitest retry on the lane — a retried-green race is a flake deferred, and only the chromium launch itself may retry inside the harness, logged. One chromium per file, fresh browser context per scenario, one host per scenario; viewport pinned; selectors anchor on roles, `data-*` attributes, and visible text only.
+
+### Expected outputs
+
+One committed golden per scenario: a normalized `ariaSnapshot()` of the conversation region only (`ui.expected.md`) — uuid/cwd/duration tokens normalized, sidebar and other time-bearing chrome structurally excluded, captured poll-until-equal at the settled milestone. The accessibility tree is the mechanization of the client rule "assert what the user would see, never class names": it survives CSS-module hash churn, styling rewrites, and DOM restructuring, and a wholesale component rewrite refreshes it keylessly. Alongside the golden, three or four targeted role/text anchor assertions (heading level, `pre code` content, tool-row accessible name) keep green anchors under a semantics-preserving rewrite so a churned golden diff is reviewable against surviving anchors. World-state assertions ride `host.ctx` session events inline (which tools ran, `turn/end` completed, no error) instead of a second committed log golden: the persisted-log surface is already pinned by the ACP/headless/TUI suites through the same loop and persistence plugins, and re-pinning it here would double refresh cost for no new regression class. `playwright` gets pinned exactly in `apps/web/package.json` — the aria format is Playwright-owned, the one snapshot format in this repo we do not own, so version bumps must be deliberate bump-and-refresh commits.
+
+### Modes and fixtures
+
+`DSH_SNAPSHOT` selects replay (default, keyless), record (with key), or refresh (keyless), as inline branches in the specs — the TUI suite's shape, not a suite-factory: at two scenarios the acp-snapshot factory machinery (scenario tables, pinning classes, Windows sidecars, packed-row stabilization) has no owner, and the genuinely shared parts are already exported (`scrubRequestHeaders`, `normalizeSessionLog`, `parseSessionLog`, `installLlmReplay`). Each scenario script splits into drive steps (type, send, `whenIdle`-generic waits — run in all modes, never waiting on model-content selectors) and interaction/assertion steps (expand reasoning, aria capture — replay/refresh only), so record mode cannot hang on a live model answering with a different tool count. Record = drive + harvest the in-memory `session.header` + `session.events` (the TUI `rawSessionLog` shape — no file decompression, so `bootHost` needs no compression knob) + scrub via `scrubRequestHeaders` + a mandatory keyless refresh to regenerate `ui.expected.md`. Web fixtures scrub headers everywhere and pin nowhere, matching the TUI precedent; whether the web surface must instead own a header-class pin per the [pinned-header discipline](../../implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md) is open question 3. Seeds are recorded fixtures under the same inventory and refresh discipline as replay fixtures, never hand-authored one-offs, so `DSH_SNAPSHOT=refresh` heals every committed surface after intentional shape churn and only `assistant/chunk`-shape churn escalates to re-record. A TUI-style `afterAll` fixture guard holds the inventory closed (expected files present, every fixture scrub-fixed-point, no orphan directories).
+
+### Demo scenarios
+
+1. **`fresh-round-trip`** — new session, prompt, replay streams reasoning + markdown + a `bash` tool call that really executes (`echo` in the temp workspace) + final text. Asserts settled markdown semantics (heading, code block), the tool card row, composer restore, the aria golden, and inline world state (bash `tool/call` + completed `turn/end` in the session events). The keyless version of the with-key W5 flows.
+2. **`seeded-history`** — a recorded session seeded cold; the sidebar lists it, opening it renders tool cards and collapsible reasoning purely from the log. This exercises the implicit cold-resume attach (`session.history` resumes via `agentFor`), cold summaries, history pagination views, and the client fold of historical events — the surface nothing else covers — with zero model calls, so no replay-binding constraints at all. A follow-up-prompt-after-resume scenario is deliberately deferred until the history/live stitch path changes or regresses.
+
+### Lane wiring and CI stance
+
+The lane rides `vitest.web.config.ts` (`pnpm run test:web`, serial), which stays gate-exempt exactly as its header comment records. Adding chromium to CI would reverse the "no browser infrastructure in CI" premise recorded in the [GUI testing system note](../../implemented/process/2026-07-20-gui-testing-system.md) and therefore requires its own Agent Note cross-linked from that note, staged as: non-required CI job first, promotion criteria measured (consecutive green runs, wall time, zero-retry flake budget, browser cache strategy on the enterprise runners, whether the runner images carry the chromium system libraries). Deferred out of this proposal; a `TODO(ci-browser)` marks the seam. Scenarios are `posixOnly` initially. Docs updated in the same implementation PR: the [testing policy](../../../../docs/testing.md) names `apps/web/tests/snapshots/` as the web surface's snapshot home with its divergent `DSH_SNAPSHOT=… pnpm run test:web` commands, the GUI testing note's tier map gains the lane (and drops its stale references to the deleted `missions/scripts/verify-*` files), `packages/client/AGENTS.md`'s check ladder mentions it, and the `dsh-acp-snapshot` README's "ACP-specific by design" sentence is corrected — this lane is the third consumer of its normalizers.
+
+### Open questions
+
+1. **LLM seam.** Two viable shapes. (a) `BootHostOptions.llm?: 'deepseek' | false` — an assembly toggle in the one module that owns assembly, matching the existing `workspaceContext: Config | false` shape and the reserved-knob sentence in `start.ts`; `llm: false` mounts no adapter, the harness fills the open seam on `host.ctx`, misuse fails loud at the first stream with `NO_ADAPTER`, and keyless boots stop needing any key. (b) Zero product change — a placeholder `DEEPSEEK_API_KEY` env var satisfies `llm-deepseek`'s load-time presence check (twice-precedented in-tree) and replay intercepts ahead of the mounted adapter. (a) is cleaner semantics and honest keylessness at the cost of a test-motivated product field; (b) is free but satisfies a fail-loud check with a lie and leaves a dead adapter mounted. Recommendation: (a), shaped minimal.
+2. **Loader-izing `dsh web`.** Making the web host `cordis.yml`-driven like every example would give the ACP-style `cordis.snapshot.yml` replay overlay for free and align with "everything is a plugin", but it reverses the settled "assembly is written in the app" ruling and is a product-architecture decision on its own merits — its own proposal if wanted; this lane does not need it.
+3. **Header-class pin.** Strict reading of the pinned-header discipline wants one web scenario pinning `bootHost`'s composed prompt + tool schemas (a header class no ACP scenario covers); the TUI precedent scrubs everywhere and pins nowhere. Cheap middle: pin sidecars on `fresh-round-trip` at record time. Recommendation: follow TUI now (scrub-only), revisit when the web assembly's header diverges further from the repl composition it mirrors.
+4. **Golden breadth.** Full conversation-region aria golden (adopted above) versus targeted assertions only. The golden is the "assembled transcript" duty for user-visible changes; the cost is a keyless refresh on every component rewrite. Recommendation: keep the golden + anchors.
+5. **Client settled signal.** A `data-dsh-busy` attribute derived from the object layer's pending-RPC/active-stream state would replace multi-condition settled polls with one selector. Presentation-plane observability, no session-log leak — but the current polls suffice for two scenarios. Recommendation: defer until a settled-poll flake actually appears.
+
+## Prior art
+
+Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot + AI SDK, lobe-chat, open-webui, OpenHands, Chainlit, continue, cline, langfuse, gradio/streamlit; Playwright HAR/route, MSW, Polly/nock, WireMock, aimock). The dominant proven architecture for apps that own their backend is an in-process fake/replay model behind the real backend seam with everything downstream real (LibreChat's `LIBRECHAT_TEST_RUN_HOOK` fake model; ai-chatbot's `MockLanguageModelV3` + `simulateReadableStream`; continue's scripted mock provider classes) — which is what `dsh-llm-replay` already is. Browser-level SSE interception cannot exercise incremental rendering (`route.fulfill` delivers the whole body at once; playwright#33564) and leaves the server SSE stack untested, so projects use it only for edge cases. Chunk pacing as a fixture parameter recurs everywhere (LibreChat 10ms default with slow profiles; ai-chatbot 500ms); real models in CI rot (open-webui's suite grew 120-second timeouts, was disabled, then deleted); sessions are seeded at the persistence layer with controlled timestamps (LibreChat inserts backdated Mongo documents; langfuse seeds its DB). No surveyed project replays a recorded agent-event log through the real backend for UI tests — the closest are provider-level recorded fixtures (aimock) and frontend-level socket history emission (OpenHands MSW) — so the session-log-as-fixture design goes one step beyond prior art along the axis this repo's model-visible ⟺ logged invariant makes natural.
+
+## Alternatives considered
+
+**Browser-network SSE interception (`page.route`).** Rejected: `route.fulfill` cannot stream, so incremental token rendering is unexercisable and the server-side SSE/backpressure/close path — where both confirmed P0s hid — goes untested.
+
+**Mock HTTP provider at `DEEPSEEK_BASE_URL`.** Rejected as the lane's mechanism (kept for the one existing workspace-probe smoke): fixtures become hand-authored OpenAI SSE byte scripts, a second fixture format that drifts from the session-log format the rest of the repo records and replays; the adapter's real HTTP path is with-key e2e's job.
+
+**Growing the `?fixture` client.** Rejected: tier separation — `FixtureApiClient` exists to test the client shell without a server; everything below the client API seam stays untested by construction.
+
+**A `packages/support/web-snapshot` package with a `defineWebSnapshotSuite` factory.** Rejected for now: chromium-driving source cannot honestly hold per-file 100% coverage on browserless coverage runners, and at two scenarios the factory generalizes from one consumer while the genuinely shared logic already lives exported in `dsh-llm-replay`/`dsh-acp-snapshot`. Re-entry trigger: a second web-shaped consumer or ≥6 scenarios with demonstrably drifting inline branches; the package boundary would then be drawn browser-free.
+
+**A committed normalized-session-log golden as a second expected surface.** Rejected: the log surface is pinned by the ACP/headless/TUI suites through the same loop and persistence; here it would double refresh cost and re-test lower tiers, against the tier discipline. Inline world-state assertions on `host.ctx` events keep the world-verification duty.
+
+**Spawning the `dsh web` bin with a `DSH_SNAPSHOT` replay branch.** Rejected for now: it needs a test-mode branch plus env plumbing in the product bin where the in-process route uses exported production functions with zero product change; the bin's thin glue is covered by the keyless CLI smokes. Becomes free if the web host is ever Loader-ized (open question 2).
+
+**Changing the wire protocol for testability.** Rejected: the contract already has a first-class keyless isomorphic seam (`InProcessApiClient(toFetchHandler(api))`), the per-event unbatched SSE is exactly what makes replay observable in a browser, and testing a wire we no longer ship would invert the tier's purpose.
+
+**Real-model browser tests as the keyless lane.** Rejected: nondeterministic by construction; the surveyed cautionary case (open-webui) grew unbounded timeouts and was deleted. The with-key W5 smoke stays as the live-model complement.
+
+## Acceptance criteria
+
+- `pnpm run test:web` keyless passes the two scenarios deterministically (no vitest retry), alongside the existing smoke pair, on a checkout with built client bundles and the frontend dist.
+- Replay asserts: aria golden equality at the settled milestone, anchor role/text assertions, inline world-state event assertions, zero pageerrors, zero connection-loss/gap-repair console warnings, all replay scripts fully consumed at teardown.
+- `DSH_SNAPSHOT=record` with a key re-records `fresh-round-trip` (drive steps only), rewrites its `session.jsonl` scrubbed, and a follow-up `DSH_SNAPSHOT=refresh` regenerates `ui.expected.md` keylessly; `refresh` alone heals goldens after intentional non-chunk shape churn.
+- The seeded scenario renders history through the real cold-resume path with zero model calls and leaves the seed fixture byte-identical (closedness validated at seed time).
+- Fixture guard holds the snapshot inventory closed; failure produces a bundle under `.artifacts/` (screenshot, console, pageerrors, persistence copy, actual-vs-expected aria).
+- Docs land in the same PR: testing.md web-lane entry, GUI testing note tier map + stale verify-script cleanup, client AGENTS.md ladder, acp-snapshot README correction, this note moved to `implemented/` rewritten in present tense.
+
+## Risks
+
+- **The aria format is Playwright-owned** — the one committed snapshot format the repo does not control; a version bump can churn every golden. Mitigated by an exact version pin in `apps/web` and a documented bump-and-refresh procedure; residual risk accepted.
+- **Replay's first-call-order binding** stays fragile under concurrent browser-driven sessions; the lane constrains scenarios to one prompting session each (the seeded scenario prompts none), and the teardown consumption assertion turns violations into diagnostics rather than surreal transcripts.
+- **`compact-basic` shares the session's replay cursor** — a pressure-triggered summarize would consume a script entry; inert for small fixtures under the 128k catalog window, and the consumption assertion catches it if a fixture ever grows past the threshold.
+- **CI remains browserless for now**, so the lane guards regressions only where it is run (locally and in any future non-required job) until the CI reversal is separately decided; the runner images' chromium-library situation is unverified.
+- **Record-mode nondeterminism** is contained but not eliminated by the drive/assert split: a live model may still produce a transcript whose replay violates a scenario's assertions, requiring prompt tuning at record time (bounded by terse prompts and a chunk-count warning in record mode).
+- **jsdom-lane overlap**: component-level rendering is already covered per-plugin; this lane must stay at assembled-transcript altitude (whole-region golden + anchors) or it starts re-testing tier 2 and paying double maintenance.