test(web): split long conversation performance cases

This commit is contained in:
kingwl
2026-07-31 19:37:50 +08:00
committed by imccyu
parent 9784d6c932
commit 1cc59ae78e
5 changed files with 740 additions and 244 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md
2026-07-24-web-gui-browser-e2e-lane.md: 51277846f7e796430081501a06c3164fc3e2cdd0
2026-07-24-web-gui-browser-e2e-lane.zh.md: 0a625ca98257df813e3413c90cc47c3be563f3be
2026-07-24-web-gui-browser-e2e-lane.md: e9559aa1f6fba89a99d91e79026552511b337793
2026-07-24-web-gui-browser-e2e-lane.zh.md: 5f2047125e38f6e1eecf4b8b81d6f5da16686afa

View File

@@ -48,7 +48,7 @@ The lane covers three behavior families. Live-turn scenarios pin ordinary tool e
The lane is a required compare-only gate for Linux pull requests under the [browser snapshot CI decision](2026-07-30-web-browser-snapshot-ci-gate.md). The `node 24 / snapshots and artifacts` consumer job owns the [single Linux build](../process/2026-07-30-independent-ci-consumer-build.md), installs the lockfile-selected Chromium, restores its OS-and-lockfile-keyed cache, and runs the lane with `DSH_SNAPSHOT=replay`. This is an intentional plane split: the host and specs use the [tsx source-launch contract](../architecture/2026-07-29-dsh-source-launch-tsx-esm.md), while the browser consumes `apps/web/dist` and package `lib/client.js` artifacts, so the gate depends on `built-package-invariants` for those client artifacts. The hosted and self-hosted default-branch Linux serial jobs run the same gate; the hosted job produces the browser cache consumed by pull requests, while the persistent self-hosted pool needs no hosted cache. CI never records or refreshes goldens. Scenarios remain POSIX-oriented and stay outside the Windows and macOS matrices.
High-cardinality performance diagnostics use the separate opt-in `apps/web/tests/**/*.perf.ts` inventory selected only by `vitest.web.perf.config.ts`. The `complex-history.perf.ts` case reuses the real scaffold, seeds 1,000 compact sessions plus one 500-turn history containing 500 tool calls, and reports Chromium main-thread, DOM, listener, heap, paging, search, and Trajectory measurements. It then opens a blank session, fills a deterministic mixed-language prompt through the real composer, clicks Send, and replays 120 paced text deltas through the real loop and SSE wire; the report includes click-to-user-echo, click-to-first-chunk, stream completion, browser mutation, and persisted-chunk measurements. It carries structural assertions for the intended load and stream shapes but no timing thresholds because machine speed is not a correctness contract. The required `vitest.web.config.ts` inventory remains limited to `*.e2e.ts` and `*.snapshot.ts`, so neither `test:web:built` nor its CI gate collects performance cases.
High-cardinality performance diagnostics use the separate opt-in `apps/web/tests/**/*.perf.ts` inventory selected only by `vitest.web.perf.config.ts`. The isolated `complex-history.perf.ts` cases reuse the real scaffold: the workspace case seeds 1,000 compact sessions plus one 500-turn history containing 500 tool calls, exhausts and remounts that history in Chat, and reports Chromium main-thread, DOM, listener, heap, paging, search, and Trajectory measurements. Two continuation cases seed the same long history but compare the default 24-turn Chat window with all 500 turns expanded before each continues eight identical turns through the real composer, agent loop, SSE wire, tools, and persistence; two turns execute a real `bash` call and assert its durable result, while the final turn fills an 8,232-character mixed-language prompt and replays 120 paced text deltas. A separate soak case starts from a blank session, drives 100 consecutive real composer turns with a `bash` call and result every tenth turn, forces GC every ten turns, and reports ten-turn latency windows plus retained browser state. Per-turn diagnostics cover composer fill, click-to-user-echo, click-to-first-chunk, completion, browser mutations, persisted chunks, and tool events; the synthetic replay model has enough context capacity to keep fixture cardinality stable instead of consuming scripted calls through compaction. Structural assertions pin the intended load, stream, and tool shapes, but timing remains threshold-free because machine speed is not a correctness contract. The required `vitest.web.config.ts` inventory remains limited to `*.e2e.ts` and `*.snapshot.ts`, so neither `test:web:built` nor its CI gate collects performance cases.
## Prior art

View File

@@ -48,7 +48,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
根据[浏览器快照 CI 决策](2026-07-30-web-browser-snapshot-ci-gate.md),该车道是 Linux 拉取请求必需的只比较门禁。`node 24 / snapshots and artifacts` 消费方任务在[消费方独立构建](../process/2026-07-30-independent-ci-consumer-build.md)中负责唯一一次 Linux 构建,安装锁文件选定的 Chromium恢复以操作系统和锁文件为键的缓存并用 `DSH_SNAPSHOT=replay` 运行该车道。这是有意的平面切分host 与 spec 使用 [tsx 源码启动契约](../architecture/2026-07-29-dsh-source-launch-tsx-esm.md),浏览器则消费 `apps/web/dist` 和包的 `lib/client.js` 产物,因此门禁依赖 `built-package-invariants` 提供这些客户端产物。托管和自托管的默认分支 Linux 串行任务运行同一门禁;托管任务生成供 PR 消费的浏览器缓存持久化自托管池则不需要托管侧缓存。CI 从不录制或刷新预期输出。场景仍面向 POSIX并继续置于 Windows 和 macOS 矩阵之外。
高基数性能诊断使用单独按需启用的 `apps/web/tests/**/*.perf.ts` 清单,并且只由 `vitest.web.perf.config.ts` 选中。`complex-history.perf.ts` 用例复用真实 scaffold播种 1,000 个紧凑会话以及一份包含 500 次工具调用的 500 轮次历史,并报告 Chromium 主线程、DOM、监听器、堆内存、分页、搜索和 Trajectory 测量结果。随后它打开一个空白会话,通过真实输入框填入一条确定性的混合语言提示词,点击 Send并通过真实循环和 SSE wire 回放 120 个带节奏的文本增量;报告包括点击到用户消息回显、点击到首个分片、完成、浏览器变更持久化分片测量结果。它对预期负载和流形状设有结构性断言,但不设时间阈值,因为机器速度不属于正确性契约。必需的 `vitest.web.config.ts` 清单仍仅限 `*.e2e.ts``*.snapshot.ts`,因此 `test:web:built` 及其 CI 门禁都不会收集性能用例。
高基数性能诊断使用单独按需启用的 `apps/web/tests/**/*.perf.ts` 清单,并且只由 `vitest.web.perf.config.ts` 选中。`complex-history.perf.ts` 的隔离用例复用真实 scaffold:工作区用例播种 1,000 个紧凑会话以及一份包含 500 次工具调用的 500 轮次历史,在 Chat 中穷尽并重新挂载该历史,并报告 Chromium 主线程、DOM、监听器、堆内存、分页、搜索和 Trajectory 测量结果。两个续聊用例播种同一份长历史,但比较默认的 24 轮次 Chat 窗口与展开全部 500 轮次的状态然后各自通过真实输入框、agent loop、SSE wire、工具和持久化继续进行 8 个相同轮次;其中两轮执行真实 `bash` 调用并断言其持久化结果,最后一轮则填入一条包含 8,232 个字符的混合语言提示词,并回放 120 个带节奏的文本增量。一个单独的 soak 用例从空白会话开始,通过真实输入框连续驱动 100 轮,每第 10 轮执行一次 `bash` 调用并产生结果,每 10 轮强制执行一次 GC并报告每 10 轮的延迟窗口及保留的浏览器状态。逐轮诊断涵盖输入框填入、点击到用户消息回显、点击到首个分片、完成、浏览器变更持久化分片和工具事件;合成回放模型拥有足够的上下文容量,可使 fixture 基数保持稳定而不会因压缩compaction消耗脚本化调用。结构性断言钉住预期负载、流和工具形状,但时间仍不设阈值,因为机器速度不属于正确性契约。必需的 `vitest.web.config.ts` 清单仍仅限 `*.e2e.ts``*.snapshot.ts`,因此 `test:web:built` 及其 CI 门禁都不会收集性能用例。
## 业界先例

File diff suppressed because it is too large Load Diff

View File

@@ -85,6 +85,14 @@ const REPLAY_PROVIDERS = [{
models: [{ id: 'deepseek-v4-flash', name: 'DeepSeek-V4-Flash', contextWindow: 128_000 }],
}]
function replayProviders(contextWindow: number | undefined): typeof REPLAY_PROVIDERS {
if (contextWindow === undefined) return REPLAY_PROVIDERS
return REPLAY_PROVIDERS.map(provider => ({
...provider,
models: provider.models.map(model => ({ ...model, contextWindow })),
}))
}
/** A booted web scaffold: real composition, mode-selected model backend, temp world. */
export interface WebScaffold {
/** The active snapshot mode this scaffold booted under. */
@@ -134,6 +142,8 @@ export interface LaunchOptions {
replayOverride?: string
/** Per-chunk replay pacing (ms) so the browser observes genuinely incremental SSE; replay/refresh only. */
paceMs?: number
/** Synthetic model capacity for UI scenarios whose seeded history must remain uncompacted. */
replayContextWindow?: number
/**
* Tool presentation mode patched onto the shipped `tools` row (`code`
* collapses the wire to run_code + the SDK prompt section). Omit for the
@@ -344,7 +354,7 @@ export async function launchWebScaffold(options: LaunchOptions = {}): Promise<We
if (mode !== 'record' && options.replayFixture !== undefined) {
replayHandle = installLlmReplay(ctx, {
file: options.replayFixture,
providers: REPLAY_PROVIDERS,
providers: replayProviders(options.replayContextWindow),
...(options.replayOverride === undefined ? {} : { overrideFile: options.replayOverride }),
...(options.replayChildFixtures === undefined ? {} : { childFiles: options.replayChildFixtures }),
...(options.paceMs === undefined ? {} : { paceMs: options.paceMs }),