mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
Merge remote-tracking branch 'origin/master' into worktree/llm-reasoning-effort
# Conflicts: # .agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.i18n.yaml # docs/config-catalog.md # docs/core-data-structures/session.i18n.yaml # scripts/type-equiv.manifest.json
This commit is contained in:
Yichen Jiang
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-06-19-acp-snapshot-tests.md: b4cda8f32fe7a84a977bcbdbe5db0671cb9a7083
|
||||
2026-06-19-acp-snapshot-tests.zh.md: 5337c3852b524af4e8c556e93ec80084b30a6d0b
|
||||
2026-06-19-acp-snapshot-tests.md: 430441e633af1e487f19272900360a8ed2f595c9
|
||||
2026-06-19-acp-snapshot-tests.zh.md: 243e431567b45d69e1fe16dda5d07ec058403b7c
|
||||
|
||||
@@ -20,7 +20,7 @@ A snapshot test boots the real ACP example, drives its stdio protocol from a det
|
||||
|
||||
Each scenario's `session.jsonl` is harvested from a real run. `assistant/chunk` events reproduce the model streams; tool, message, and boundary events capture the harness behavior. One ordinary session artifact therefore serves as both replay source and behavioral expected output.
|
||||
|
||||
When a scenario pins an alternative physical storage layout, its fixture is mechanically derived from a real unpacked counterpart. The scenario test requires every intended storage-row kind and exact event-for-event equality after decoding before the ordinary replay and log comparison proves that the assembled process consumes and reproduces that layout.
|
||||
Every committed session-format fixture uses the canonical packed physical layout. The all-row-kinds scenario is mechanically derived from an independent real recording; its test requires every packed storage-row kind and exact event-for-event equality after both fixtures decode, then ordinary replay and log comparison prove that the assembled process consumes and reproduces the layout.
|
||||
|
||||
### Replay derives the model script from the log
|
||||
|
||||
@@ -44,16 +44,16 @@ Replay is positional and therefore permits only one in-flight model stream per s
|
||||
|
||||
### Recording harvests the log; keyless replay needs a providerless config
|
||||
|
||||
Recording runs the scenario with the real `llm-deepseek` adapter and the JSONL persistence backend configured with `persistenceCompression: 'none'`, then copies the produced `.jsonl` into the scenario dir. The explicit raw mode keeps committed replay fixtures line-readable while ordinary deployments use the backend's compressed default. Per-event appends are durable, but the harness shuts the subprocess down gracefully (close stdin → `await ctx.dispose()`) before harvesting so the final events are flushed. `llm-replay` itself does no recording — it is replay-only.
|
||||
Recording runs the scenario with the real `llm-deepseek` adapter and the JSONL persistence backend configured with `persistenceCompression: 'none'`, then copies the produced `.jsonl` into the scenario dir. The explicit raw mode keeps committed replay fixtures line-readable while ordinary deployments use the backend's compressed default; eligible chunk runs still use the default packed storage rows. Per-event appends are durable, but the harness shuts the subprocess down gracefully (close stdin → `await ctx.dispose()`) before harvesting so the final events are flushed. `llm-replay` itself does no recording — it is replay-only.
|
||||
|
||||
Replay uses a `cordis.snapshot.yml` overlay that replaces the real adapter with `llm-replay` while retaining the live composition. Recording uses the ordinary config and a harness-supplied persistence root. Replay mode skips `.env` loading, so a stray API key cannot trigger a live call. See the [single-source config Agent Note](2026-07-04-single-source-acp-replay-config.md).
|
||||
Replay uses a `cordis.snapshot.yml` overlay that replaces the real adapter with `llm-replay` while retaining the live composition. Recording uses the ordinary config and a harness-supplied persistence root. Replay mode skips `.env` loading, so a stray API key cannot trigger a live call. See the [single-source config Agent Note](../../archived/testing/2026-07-04-single-source-acp-replay-config.md).
|
||||
|
||||
### Two surfaces: normalize, then compare
|
||||
|
||||
A snapshot run asserts **two** normalized surfaces, because the harness's external surfaces are distinct:
|
||||
|
||||
1. The **stdout transcript** — the framed ACP JSON-RPC responses and committed-message updates an automation client receives. It catches regressions in the transport contract and is compared against a committed `stdout.expected.jsonl`.
|
||||
2. The **re-persisted session JSONL**, normalized and compared with `session.jsonl`. The same fixture is both replay source and expected log. Prompt text is scrubbed; one scenario per header class pins readable prompt and tool content as described in the [header-pinning Agent Note](2026-07-06-pin-request-header-content-in-one-scenario.md). Override scenarios derive model behavior solely from their sidecar.
|
||||
2. The **re-persisted session JSONL**, normalized and compared with `session.jsonl`. The same fixture is both replay source and expected log. Prompt text is scrubbed; one scenario per header class pins readable prompt and tool content as described in the [header-pinning Agent Note](../../archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md). Override scenarios derive model behavior solely from their sidecar.
|
||||
|
||||
The surfaces are complementary: stdout covers the minimal automation wire, while JSONL covers loop, tool, and boundary structure that the wire intentionally omits.
|
||||
|
||||
@@ -69,7 +69,7 @@ Tool determinism comes from a generated cwd, scrubbed environment, fresh non-log
|
||||
|
||||
### Two subcommands, replay in the default gate
|
||||
|
||||
`pnpm run test:snapshot` replays committed fixtures keylessly; `test:snapshot:record` uses the real API and rewrites the harvested session log and stdout expected output. Missing fixtures fail loud. Every scenario carries `input.json`, `stdout.expected.jsonl`, and `session.jsonl`; no-model cases use a header-only log. `replay.override.json` is required only for scenarios marked `overridden`, because its presence replaces derived replay. Fixture guards reject missing, mismatched, and orphaned files. Both commands accept scenario filters.
|
||||
`pnpm run test:snapshot` replays committed fixtures keylessly; `test:snapshot:record` uses the real API and rewrites the harvested session log and stdout expected output. The same keyless gate discovers repository JSONL by its `session` header and rejects any fixture that differs from the shared codec's canonical packed representation. Missing fixtures fail loud. Every scenario carries `input.json`, `stdout.expected.jsonl`, and `session.jsonl`; no-model cases use a header-only log. `replay.override.json` is required only for scenarios marked `overridden`, because its presence replaces derived replay. Fixture guards reject missing, mismatched, and orphaned files. Both commands accept scenario filters.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ Status: implemented
|
||||
|
||||
每个场景的 `session.jsonl` 都从真实运行中采集。`assistant/chunk` 事件复现模型流；工具、消息和边界事件捕获 harness 行为。因此，一份普通会话产物同时充当重放来源和行为预期输出。
|
||||
|
||||
当场景固定另一种物理存储布局时，其 fixture 会从真实的未打包对应项机械派生。场景测试要求包含每一种预期存储行类型，并在解码后逐事件精确相等；随后，普通重放与日志比较才会证明组合后的进程能够消费并复现该布局。
|
||||
每个签入仓库的会话格式 fixture 都使用规范的打包物理布局。覆盖所有行类型的场景从一份独立的真实录制机械派生；测试要求它包含每一种打包存储行类型，并在两份 fixture 解码后逐事件精确相等；随后，普通重放与日志比较会证明组装后的进程能够消费并复现该布局。
|
||||
|
||||
### 回放从日志推导模型脚本
|
||||
|
||||
@@ -44,16 +44,16 @@ Status: implemented
|
||||
|
||||
### 录制采集日志；无密钥回放需要无提供方的配置
|
||||
|
||||
记录模式使用真实 `llm-deepseek` 适配器和配置为 `persistenceCompression: 'none'` 的 JSONL 持久化后端运行场景，再把生成的 `.jsonl` 复制到场景目录。显式 raw 模式让已提交重放 fixture 保持逐行可读，而普通部署使用后端的压缩默认值。逐事件追加具有持久性，但 harness 会在采集前优雅关闭子进程（关闭 stdin → `await ctx.dispose()`），以确保最终事件已刷出。`llm-replay` 本身不执行记录——它只负责重放。
|
||||
记录模式使用真实 `llm-deepseek` 适配器和配置为 `persistenceCompression: 'none'` 的 JSONL 持久化后端运行场景，再把生成的 `.jsonl` 复制到场景目录。显式 raw 模式让已提交重放 fixture 保持逐行可读，而普通部署使用后端的压缩默认值；符合条件的分片连续段仍使用默认的打包存储行。逐事件追加具有持久性，但 harness 会在采集前优雅关闭子进程（关闭 stdin → `await ctx.dispose()`），以确保最终事件已刷出。`llm-replay` 本身不执行记录——它只负责重放。
|
||||
|
||||
重放使用 `cordis.snapshot.yml` overlay，以 `llm-replay` 替换真实适配器，同时保留实时组合。记录使用普通配置和由 harness 提供的持久化根目录。重放模式跳过 `.env` 加载，因此意外存在的 API 密钥不会触发实时调用。参见[单一来源配置 Agent Note](2026-07-04-single-source-acp-replay-config.md)。
|
||||
重放使用 `cordis.snapshot.yml` overlay，以 `llm-replay` 替换真实适配器，同时保留实时组合。记录使用普通配置和由 harness 提供的持久化根目录。重放模式跳过 `.env` 加载，因此意外存在的 API 密钥不会触发实时调用。参见[单一来源配置 Agent Note](../../archived/testing/2026-07-04-single-source-acp-replay-config.md)。
|
||||
|
||||
### 两个表面：归一化后比对
|
||||
|
||||
快照运行断言**两个**归一化后的表面，因为 harness 的外部表面是不同的：
|
||||
|
||||
1. **stdout transcript**——自动化客户端收到的、经过 framing 的 ACP JSON-RPC 响应与已提交的消息更新。它捕获传输契约的回归，与已提交的 `stdout.expected.jsonl` 比较。
|
||||
2. **重新持久化的会话 JSONL**，经过规范化后与 `session.jsonl` 比较。同一 fixture 同时作为重放来源和预期日志。提示词文本会被清理；按照[请求头固定 Agent Note](2026-07-06-pin-request-header-content-in-one-scenario.md)所述，每种请求头类别由一个场景固定可读提示词与工具内容。Override 场景仅从其 sidecar 派生模型行为。
|
||||
2. **重新持久化的会话 JSONL**，经过规范化后与 `session.jsonl` 比较。同一 fixture 同时作为重放来源和预期日志。提示词文本会被清理；按照[请求头固定 Agent Note](../../archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)所述，每种请求头类别由一个场景固定可读提示词与工具内容。Override 场景仅从其 sidecar 派生模型行为。
|
||||
|
||||
两个表面互补：stdout 覆盖精简的自动化线协议，JSONL 覆盖线协议有意省略的 loop、工具和 boundary 结构。
|
||||
|
||||
@@ -69,7 +69,7 @@ Status: implemented
|
||||
|
||||
### 两个子命令，回放在默认门禁中
|
||||
|
||||
`pnpm run test:snapshot` 无需密钥即可重放已提交 fixture；`test:snapshot:record` 使用真实 API，并重写采集的会话日志与 stdout 预期输出。缺少 fixture 时会响亮失败。每个场景都包含 `input.json`、`stdout.expected.jsonl` 和 `session.jsonl`；不调用模型的情况使用仅有请求头的日志。只有标记为 `overridden` 的场景才需要 `replay.override.json`，因为它一旦存在就会取代派生重放。Fixture 守卫会拒绝缺失、不匹配和孤立文件。两个命令都接受场景过滤器。
|
||||
`pnpm run test:snapshot` 无需密钥即可重放已提交 fixture；`test:snapshot:record` 使用真实 API，并重写采集的会话日志与 stdout 预期输出。同一无密钥门禁会通过 `session` header 发现仓库中的 JSONL，并拒绝与共享编解码器的规范打包表示不同的任何 fixture。缺少 fixture 时会响亮失败。每个场景都包含 `input.json`、`stdout.expected.jsonl` 和 `session.jsonl`；不调用模型的情况使用仅有请求头的日志。只有标记为 `overridden` 的场景才需要 `replay.override.json`，因为它一旦存在就会取代派生重放。Fixture 守卫会拒绝缺失、不匹配和孤立文件。两个命令都接受场景过滤器。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-06-20-remove-redundant-snapshot-log-expected-output.md: c2452f971d3cb76dceb766072dbc0a5c81465e78
|
||||
2026-06-20-remove-redundant-snapshot-log-expected-output.zh.md: e6175181589eabae064c34044f353efae537961b
|
||||
@@ -1,37 +0,0 @@
|
||||
# Agent Note: Use `session.jsonl` as the only snapshot session-log artifact
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-06-20-remove-redundant-snapshot-log-expected-output.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Model-driving ACP snapshot scenarios ship both `session.jsonl` and `session.expected.jsonl`. For normal recorded scenarios, `session.jsonl` is the replay fixture harvested from a real run, and the replay test normalizes the newly persisted log and compares it to `session.expected.jsonl`. In the current fixtures, the two normalized logs are identical for ordinary recorded scenarios.
|
||||
|
||||
Authored override scenarios (`error-finish`, `cancel`) currently use `replay.override.json` to drive model behavior and keep `session.jsonl` as a minimal dummy fixture, while `session.expected.jsonl` holds the expected persisted log. The override file is a JSON array of `ReplayEntry` objects: `{ "kind": "chunks", "chunks": StreamChunk[] }`, `{ "kind": "throw", "chunks": StreamChunk[], "message": string, "code": string }`, or `{ "kind": "hang" }`. That split is also unnecessary: when an override sidecar exists, `llm-replay` replaces the derived script and does not need `session.jsonl` for model chunks, so `session.jsonl` can still be the expected session-log artifact for the scenario.
|
||||
|
||||
## Decision
|
||||
|
||||
The `session.expected.jsonl` concept is removed entirely. Every scenario has at most one committed session-log artifact, `session.jsonl`:
|
||||
|
||||
- For recorded scenarios, `session.jsonl` remains the raw harvested log. Replay still derives model chunks from it, and the snapshot test compares the replay run's normalized persisted log against normalized `session.jsonl`.
|
||||
- For authored override scenarios, `replay.override.json` drives model behavior and `session.jsonl` holds the expected produced session log. The replay adapter ignores the fixture for model chunks when the override exists, so the same file can be the expected log without affecting replay behavior.
|
||||
- For no-model scenarios, `session.jsonl` can stay as the minimal fixture needed to boot `llm-replay`; no session-log comparison is needed unless the scenario creates a persisted session.
|
||||
|
||||
Stdout expected outputs remain unchanged; they are the editor-facing projection and are not redundant with the session fixture.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Normalizing both sides against a shared (replay-run) context** — rejected: `normalizeSessionLog` scrubs cwd by exact string match, so the fixture's recorded cwd would survive unscrubbed and every compare would fail. Each side normalizes against its own header-derived context — the implementation note below carries the mechanics.
|
||||
|
||||
## Verification
|
||||
|
||||
`session.expected.jsonl` appears nowhere in the snapshot harness, fixtures, orphan guards, or docs; the snapshot test derives the expected session log from `session.jsonl` for every model scenario; authored sidecar scenarios commit their expected produced log as `session.jsonl` with `replay.override.json` as the model-behavior override; and the orphan-fixture guards know which files each scenario kind requires. The [ACP snapshot tests Agent Note](2026-06-19-acp-snapshot-tests.md) describes the reduced fixture set.
|
||||
|
||||
## Consequences
|
||||
|
||||
Reviewers lose one artifact name that made the expected persisted log visually separate from the replay fixture. The stdout expected output still protects the editor transcript, and comparing replay output to `session.jsonl` preserves the loop/persistence regression check without duplicating files.
|
||||
|
||||
## Implementation note
|
||||
|
||||
Each side is normalized against its own header values because recording and replay have different ids, paths, and timestamps. `fixtureContext()` derives the fixture context from its header, making already-normalized fixtures idempotent. Session logs use plain equality rather than file-snapshot updates, so comparison never rewrites fixtures.
|
||||
@@ -1,37 +0,0 @@
|
||||
# Agent Note: 使用 `session.jsonl` 作为唯一的快照会话日志产物
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-06-20-remove-redundant-snapshot-log-expected-output.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
驱动模型的 ACP（Agent Client Protocol）快照场景同时包含 `session.jsonl` 和 `session.expected.jsonl`。对于普通记录场景，`session.jsonl` 是从真实运行采集的重放 fixture（测试前置数据）；重放测试会规范化新持久化的日志，并将其与 `session.expected.jsonl` 比较。在当前 fixture 中，普通记录场景的两份规范化日志完全相同。
|
||||
|
||||
手工编写的 override 场景（`error-finish`、`cancel`）目前使用 `replay.override.json` 驱动模型行为，并把 `session.jsonl` 保留为最小 dummy fixture，而 `session.expected.jsonl` 存放预期的持久化日志。override 文件是由 `ReplayEntry` 对象组成的 JSON 数组：`{ "kind": "chunks", "chunks": StreamChunk[] }`、`{ "kind": "throw", "chunks": StreamChunk[], "message": string, "code": string }` 或 `{ "kind": "hang" }`。这种拆分同样没有必要：override sidecar 存在时，`llm-replay` 会替换派生脚本，不需要从 `session.jsonl` 取得模型分片，因此 `session.jsonl` 仍可作为场景的预期会话日志产物。
|
||||
|
||||
## 决策
|
||||
|
||||
彻底移除 `session.expected.jsonl` 概念。每个场景最多只有一个已提交会话日志产物，即 `session.jsonl`：
|
||||
|
||||
- 对于录制场景，`session.jsonl` 仍是原始采集的日志。回放仍从中派生模型分片，快照测试将回放运行归一化后的持久化日志与归一化后的 `session.jsonl` 进行比较。
|
||||
- 对于手工编写的覆盖场景，`replay.override.json` 驱动模型行为，`session.jsonl` 存放预期产出的会话日志。当覆盖文件存在时，回放适配器不从 fixture 获取模型分片，因此同一个文件既可作为预期日志，又不影响回放行为。
|
||||
- 对于无模型场景，`session.jsonl` 可保留为引导 `llm-replay` 所需的最小 fixture；除非场景创建了持久化会话，否则无需进行会话日志比较。
|
||||
|
||||
Stdout 预期输出保持不变；它们是面向编辑器的投影，与会话 fixture 并不重复。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**对两侧基于共享的（回放运行）上下文做归一化**：否决。`normalizeSessionLog` 通过精确字符串匹配擦除 cwd，因此 fixture 中录制的 cwd 不会被擦除，每次比较都会失败。两侧各自基于自身 header 派生的上下文做归一化——下方的实现说明描述了具体机制。
|
||||
|
||||
## 验证
|
||||
|
||||
快照 harness、fixture、孤立项守卫和文档中都不再出现 `session.expected.jsonl`；对于每个模型场景，快照测试都从 `session.jsonl` 派生预期会话日志；手工编写 sidecar 的场景把预期生成日志提交为 `session.jsonl`，并以 `replay.override.json` 覆盖模型行为；孤立 fixture 守卫知道每种场景类型所需的文件。[ACP 快照测试 Agent Note（agent 决策记录）](2026-06-19-acp-snapshot-tests.md)描述了精简后的 fixture 集合。
|
||||
|
||||
## 后果
|
||||
|
||||
评审者失去了一个能在视觉上区分预期持久化日志与重放 fixture 的产物名。stdout 预期输出仍然保护编辑器 transcript（文本记录），而将重放输出与 `session.jsonl` 比较，无需复制文件即可保留循环/持久化回归检查。
|
||||
|
||||
## 实现说明
|
||||
|
||||
两侧各自基于自身 header 值做归一化，因为录制与回放具有不同的 id、路径和时间戳。`fixtureContext()` 从 fixture 的 header 派生上下文，使已归一化的 fixture 具有幂等性。会话日志使用普通相等比较而非文件快照更新，因此比较过程不会改写 fixture。
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-06-22-fork-child-replay-seed-boundary.md: d3cbbb1dae1d64a10973bd5895ccc47d877eba28
|
||||
2026-06-22-fork-child-replay-seed-boundary.zh.md: a944538b9dcb74eb15142593089c7905efc3f565
|
||||
2026-06-22-fork-child-replay-seed-boundary.md: ed3ec095bc14128f5ebc0a9188bc022ef97b1c8b
|
||||
2026-06-22-fork-child-replay-seed-boundary.zh.md: 84cd56ccea69aab0246582512908b4a73ce3c36a
|
||||
|
||||
@@ -35,7 +35,7 @@ The SQLite layout containing `seed_length`, `source_event_seqs`, and `surface_op
|
||||
|
||||
`dsh-llm-replay`'s `parseSessionHeader` now also reads `seedLength` (absent ⇒ 0), and `loadSessionScripts` derives a child's entries from `parseSessionLog(text).slice(seedLength)` — the events at or after the boundary, i.e. the child's own model calls. For a spawn child `seedLength` is 0 and this is a no-op, so spawn scenarios are byte-for-byte unchanged.
|
||||
|
||||
This closes the routing correctness gap, and two recorded fork scenarios exercise it end to end — see [Record fork and mixed spawn+fork snapshot scenarios](2026-06-22-fork-snapshot-scenarios.md).
|
||||
This closes the routing correctness gap, and two recorded fork scenarios exercise it end to end — see [Record fork and mixed spawn+fork snapshot scenarios](../../archived/testing/2026-06-22-fork-snapshot-scenarios.md).
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
@@ -35,7 +35,7 @@ subagent 脚本由 [`deriveReplayScript`](../../../../packages/support/llm-repla
|
||||
|
||||
`dsh-llm-replay` 的 `parseSessionHeader` 现在也读取 `seedLength`（缺失则为 0），`loadSessionScripts` 从 `parseSessionLog(text).slice(seedLength)` 推导子会话条目——即边界及之后的事件，也就是子会话自身的模型调用。对 spawn 子会话而言 `seedLength` 为 0，此操作是空操作，spawn 场景逐字节不变。
|
||||
|
||||
这关闭了路由正确性的缺口，两个已录制的 fork 场景对其进行端到端验证——见[记录 fork 与混合 spawn+fork 快照场景](2026-06-22-fork-snapshot-scenarios.md)。
|
||||
这关闭了路由正确性的缺口，两个已录制的 fork 场景对其进行端到端验证——见[记录 fork 与混合 spawn+fork 快照场景](../../archived/testing/2026-06-22-fork-snapshot-scenarios.md)。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-06-22-fork-snapshot-scenarios.md: 46c688a4095a1d8af32b3b99887929f71a1526ce
|
||||
2026-06-22-fork-snapshot-scenarios.zh.md: 8823611ec02f269e5a21a8065c9ed3cc3911f749
|
||||
@@ -1,31 +0,0 @@
|
||||
# Agent Note: Record fork and mixed spawn+fork snapshot scenarios
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-06-22-fork-snapshot-scenarios.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The [seed-boundary Agent Note](2026-06-22-fork-child-replay-seed-boundary.md) made fork-child replay route correctly: `dsh-llm-replay` derives a child's script from the events at or after its persisted `seedLength` boundary, so a fork child's inherited parent prefix is not replayed as the child's own model calls. But it shipped with **no recorded fork scenario** — the slice was exercised only by `llm-replay`'s unit tests (a synthetic child fixture) and a persistence round-trip test. The full-transcript snapshot tier, the one net that boots the real `acp-agent` and replays an end-to-end nested transcript, had only spawn children (`subagent-spawn`, `subagent-multi`). A fork-routing regression that left the unit tests green would still have escaped the tier built to catch transcript regressions.
|
||||
|
||||
The snapshot infrastructure to express a fork scenario was already in place — both in-process backends are wired into `cordis.yml` / `cordis.snapshot.yml` as two model-facing tools (`subagent` → spawn, `subagent_fork` → fork), the harness harvests every child log, and replay forwards per-child fixtures keyed by `seedLength`. What was missing was a *recorded scenario* that drives a fork child through it.
|
||||
|
||||
## Decision
|
||||
|
||||
Record two scenarios against the real API, both replayed keyless in the default gate:
|
||||
|
||||
- **`subagent-fork`** — the parent completes a turn that establishes a fact, then delegates one subtask via `subagent_fork`. The fork child inherits the conversation (its log carries a non-zero `seedLength`), so it can answer from the parent's context. This is the focused regression: the child fixture's `seedLength` is the boundary the replay slice depends on, recorded from a real fork rather than hand-synthesized.
|
||||
- **`subagent-mixed`** — the parent completes a turn, then delegates once via `subagent` (a fresh spawn child, `seedLength` 0) and once via `subagent_fork` (a fork child, non-zero `seedLength`) in one transcript. This is the mixed spawn+fork scenario the seed-boundary and per-session-replay Agent Notes both named as a future addition: one transcript exercises both transports and both branches of the slice (`seedLength` 0 = no-op, `seedLength > 0` = trim the inherited prefix), with the two children ordered spawn-then-fork by `createdAt`.
|
||||
|
||||
### Why a completed turn-1 is required
|
||||
|
||||
The fork backend seeds the child with the parent's **balanced completed-turn prefix**. A parent that forks on its very first turn has no completed turn to inherit, so the seed is empty (≡ a fresh spawn, `seedLength` 0) — which would NOT exercise the slice. Both scenarios therefore use a two-prompt input: the first prompt completes a turn (establishing a codeword the child is later asked to recall), the second delegates the fork. The recalled codeword in the child's transcript is incidental to the model's behavior; the load-bearing artifact is the child fixture's recorded `seedLength`, which the replay slice consumes.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The fork-routing slice is now guarded at the full-transcript tier, not just by unit tests. Removing the `slice(seedLength)` (replaying the whole child log) turns **both** new scenarios red — the fork child receives the parent's recorded chunks instead of its own — proving the guard bites (verified red→green when the scenarios landed).
|
||||
- `subagent-mixed` is the first snapshot scenario to drive two *different* subagent backends in one transcript, exercising the per-session replay keying across a spawn and a fork child simultaneously.
|
||||
- Out-of-process (ACP) subagent replay remains a different shape (each child is its own process with its own replay) and is still tracked as `TODO(acp-subagent-replay)` — these scenarios are in-process only.
|
||||
- Re-recording (`pnpm run test:snapshot:record`) regenerates all four fork/spawn fixtures from the live API; the two new scenarios self-skip without a key like every recorded scenario.
|
||||
|
||||
<!-- agent-note-format: alternatives-not-recorded (pre-format Agent Note) -->
|
||||
@@ -1,31 +0,0 @@
|
||||
# Agent Note: 记录 fork 与混合 spawn+fork 快照场景
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-06-22-fork-snapshot-scenarios.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
[seed 边界 Agent Note（agent 决策记录）](2026-06-22-fork-child-replay-seed-boundary.md)让 fork 子项重放能够正确路由：`dsh-llm-replay` 根据持久化 `seedLength` 边界处及其后的事件派生子项脚本，因此 fork 子项继承的父前缀不会作为子项自身的模型调用重放。但落地时**没有记录式 fork 场景**——slice 只由 `llm-replay` 单元测试（合成子项 fixture（测试前置数据））和持久化往返测试覆盖。完整 transcript（文本记录）快照层——会启动真实 `acp-agent` 并重放端到端嵌套 transcript 的那张网——只有 spawn 子项（`subagent-spawn`、`subagent-multi`）。如果 fork 路由回归没有让单元测试变红，它仍会逃过专为捕获 transcript 回归而构建的这一层。
|
||||
|
||||
表达 fork 场景所需的快照基础设施已经就位：两个进程内后端都在 `cordis.yml` / `cordis.snapshot.yml` 中以两个面向模型的工具接入（`subagent` → spawn、`subagent_fork` → fork），harness 会收集每个子会话的日志，回放按 `seedLength` 为键转发各子会话的 fixture。缺少的是一个*已记录的场景*来驱动 fork 子会话走完这条路径。
|
||||
|
||||
## 决策
|
||||
|
||||
针对真实 API 记录两个场景，均在默认门禁中以无密钥方式回放：
|
||||
|
||||
- **`subagent-fork`**：父会话完成一个轮次以建立一个事实，然后通过 `subagent_fork` 委派一个子任务。fork 子会话继承对话（其日志携带非零 `seedLength`），因此可以从父会话的上下文中作答。这是聚焦的回归守卫：子会话 fixture 的 `seedLength` 就是回放切片所依赖的边界，来自真实 fork 的记录而非手工合成。
|
||||
- **`subagent-mixed`**——父项完成一个轮次，随后在同一 transcript 中通过 `subagent` 委托一次（全新 spawn 子项，`seedLength` 为 0），再通过 `subagent_fork` 委托一次（fork 子项，`seedLength` 非零）。这是 seed 边界与逐会话重放 Agent Note 都点名作为未来新增项的 spawn+fork 混合场景：一份 transcript 覆盖两种传输方式和 slice 的两个分支（`seedLength` 为 0 = 无操作，`seedLength > 0` = 裁剪继承前缀），两个子项按 `createdAt` 排列为先 spawn、后 fork。
|
||||
|
||||
### 为什么需要一个已完成的第一轮次
|
||||
|
||||
fork 后端使用父项的**已配平完整轮次前缀**为子项提供 seed。父项若在第一个轮次就执行 fork，没有已完成轮次可供继承，因此 seed 为空（≡ 全新 spawn，`seedLength` 为 0）——这不会覆盖 slice。因此，两个场景都使用双提示词输入：第一个提示词完成一个轮次（建立稍后要求子项回忆的 codeword），第二个提示词委托 fork。子项 transcript 中回忆出的 codeword 只是模型行为的附带结果；承载关键约束的产物是子项 fixture 中记录、由重放 slice 消费的 `seedLength`。
|
||||
|
||||
## 后果
|
||||
|
||||
- fork 路由切片现在由全 transcript 层守卫，而不仅仅是单元测试。移除 `slice(seedLength)`（回放整个子会话日志）会让**两个**新场景变红——fork 子会话收到的是父会话记录的分片而非自己的——证明守卫确实生效（场景落地时已验证红→绿）。
|
||||
- `subagent-mixed` 是第一个在同一个 transcript 中驱动两种*不同* subagent 后端的快照场景，同时覆盖了跨 spawn 和 fork 子会话的逐会话回放键控。
|
||||
- 进程外（ACP（Agent Client Protocol））subagent 回放形态不同（每个子会话是独立进程、有自己的回放），仍以 `TODO(acp-subagent-replay)` 跟踪——本文场景仅限进程内。
|
||||
- 重新录制（`pnpm run test:snapshot:record`）会从真实 API 重新生成全部四个 fork/spawn fixture；两个新场景在无密钥时自动跳过，与所有已录制场景一致。
|
||||
|
||||
<!-- agent-note-format: alternatives-not-recorded (pre-format Agent Note) -->
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-06-22-subagent-snapshot-replay.md: 8cd7bc86e07af9ed274c18574b575b9070854e88
|
||||
2026-06-22-subagent-snapshot-replay.zh.md: eae78129405fedd03c2c579845c07c6e5694cc30
|
||||
2026-06-22-subagent-snapshot-replay.md: b8fefce5ff27b0cd3cfa2920b137e78cda0d696d
|
||||
2026-06-22-subagent-snapshot-replay.zh.md: a673d6e5dd124986b827fcc6708db447090173c7
|
||||
|
||||
@@ -54,5 +54,5 @@ Both replay keyless in the default gate.
|
||||
|
||||
- The `TODO(subagent-snapshots)` deferral is resolved: nested-agent transcripts are now a first-class snapshot shape.
|
||||
- `GenerateOptions.sessionId` is a small, honest core-seam addition useful beyond replay (telemetry, request routing).
|
||||
- The `subagent` tool is bound to a single provider, so both children in `subagent-multi` are spawn (fresh). The keying routes by session, not by backend, so it is already correct for fork. The script *derivation* was not: a fork child's log begins with the seeded parent prefix (the parent's `assistant/chunk` events), so deriving its script from the whole log would replay the parent's responses as the child's. That correctness gap is closed by persisting a seed boundary — see [Persist the seed boundary so fork-child replay routes correctly](2026-06-22-fork-child-replay-seed-boundary.md) — and recorded fork + mixed spawn+fork scenarios now exercise both transports through one transcript (see [Record fork and mixed spawn+fork snapshot scenarios](2026-06-22-fork-snapshot-scenarios.md)).
|
||||
- The `subagent` tool is bound to a single provider, so both children in `subagent-multi` are spawn (fresh). The keying routes by session, not by backend, so it is already correct for fork. The script *derivation* was not: a fork child's log begins with the seeded parent prefix (the parent's `assistant/chunk` events), so deriving its script from the whole log would replay the parent's responses as the child's. That correctness gap is closed by persisting a seed boundary — see [Persist the seed boundary so fork-child replay routes correctly](2026-06-22-fork-child-replay-seed-boundary.md) — and recorded fork + mixed spawn+fork scenarios now exercise both transports through one transcript (see [Record fork and mixed spawn+fork snapshot scenarios](../../archived/testing/2026-06-22-fork-snapshot-scenarios.md)).
|
||||
- Out-of-process (ACP) subagents are a different replay shape entirely (each child is its own PROCESS with its own replay), tracked as `TODO(acp-subagent-replay)` in the PR3 plan.
|
||||
|
||||
@@ -54,5 +54,5 @@ Status: implemented
|
||||
|
||||
- `TODO(subagent-snapshots)` 延期项已解决：嵌套 agent 的 transcript 现在是快照层的一等形态。
|
||||
- `GenerateOptions.sessionId` 是一个小而诚实的 core-seam 新增，在回放之外同样有用（遥测、请求路由）。
|
||||
- `subagent` 工具绑定到单一提供方，因此 `subagent-multi` 中的两个子 agent 都是 spawn（全新创建）。键控按会话路由而非按后端路由，因此对 fork 同样正确。但脚本*派生*逻辑此前不正确：fork 子会话的日志以种子化的父前缀（父会话的 `assistant/chunk` 事件）开头，如果从完整日志派生脚本，就会把父 agent 的响应当作子 agent 的来回放。这一正确性缺口通过持久化种子边界来弥合——见[持久化种子边界，使 fork 子项重放能够正确路由](2026-06-22-fork-child-replay-seed-boundary.md)——录制的 fork 与混合 spawn+fork 场景现在通过一份 transcript 同时验证两种传输方式（见[记录 fork 与混合 spawn+fork 快照场景](2026-06-22-fork-snapshot-scenarios.md)）。
|
||||
- `subagent` 工具绑定到单一提供方，因此 `subagent-multi` 中的两个子 agent 都是 spawn（全新创建）。键控按会话路由而非按后端路由，因此对 fork 同样正确。但脚本*派生*逻辑此前不正确：fork 子会话的日志以种子化的父前缀（父会话的 `assistant/chunk` 事件）开头，如果从完整日志派生脚本，就会把父 agent 的响应当作子 agent 的来回放。这一正确性缺口通过持久化种子边界来弥合——见[持久化种子边界，使 fork 子项重放能够正确路由](2026-06-22-fork-child-replay-seed-boundary.md)——录制的 fork 与混合 spawn+fork 场景现在通过一份 transcript 同时验证两种传输方式（见[记录 fork 与混合 spawn+fork 快照场景](../../archived/testing/2026-06-22-fork-snapshot-scenarios.md)）。
|
||||
- 进程外（ACP（Agent Client Protocol））subagent 是完全不同的回放形态（每个子 agent 是自己的进程、有自己的回放），作为 `TODO(acp-subagent-replay)` 记录在 PR3 计划中。
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-04-hook-snapshot-matrix.md: 98f9db27fd8afaaa99c9985dff4d148ef4eb926b
|
||||
2026-07-04-hook-snapshot-matrix.zh.md: 40b9ee84ad7232ac806096375e8da42bd02bfddf
|
||||
@@ -1,52 +0,0 @@
|
||||
# Agent Note: Hook snapshot matrix — end-to-end expected outputs for both bridges
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-04-hook-snapshot-matrix.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The hook bridges — [`dsh-hooks-claude`](../../../../packages/hooks/hooks-claude) (7 Claude Code hook points) and [`dsh-hooks-codex`](../../../../packages/hooks/hooks-codex) (5 Codex points) — map external hook commands onto the harness interception seams. They carry deep unit and coverage-spec coverage (every decision arm, every payload dialect, driven against a mocked seam) plus one key-gated e2e (`hooks.e2e.ts`, a live `PreToolUse` block). But the full-transcript snapshot tier — the one net that boots the real `acp-agent` subprocess, replays a recorded session keyless, and diffs the normalized ACP stdout + re-persisted log against committed expected outputs — covered exactly ONE hook: a Claude `UserPromptSubmit` block (`hook-cc-promptsubmit-block`).
|
||||
|
||||
That is the tier a mocked unit test structurally cannot be: it exercises the REAL bridge translating a REAL hook process's outcome into the REAL seam decision, then the REAL loop's reaction through the automation wire and persisted log. A bridge-translation or loop-structure regression that left every unit green would still escape it for every hook point but one — and for the Codex bridge, the ACP example did not even LOAD it, so no Codex hook could fire end-to-end at all.
|
||||
|
||||
## Decision
|
||||
|
||||
The implementation has two coupled parts:
|
||||
|
||||
### 1. The ACP example ships BOTH hook bridges
|
||||
|
||||
`examples/acp-agent/cordis.yml` and `cordis.snapshot.yml` now load `dsh-hooks-codex` alongside `dsh-hooks-claude`, each pointed at its own config file (`./hooks.json` for Claude, `./codex-hooks.json` for Codex — the two dialects cannot share one file). This is a genuine product-surface change, not test-only wiring: the shipped ACP server (and the `demo:acp` front door) now carries both bridges.
|
||||
|
||||
It is safe because a bridge whose config file is absent is a **silent no-op**: `apply()` catches the read failure, logs through `ctx.logger`, and registers nothing — zero listeners, zero session events. The `acp-agent` app ships no stdout logger, so the warning cannot reach the ACP JSON-RPC channel. A scenario (or a real project) that wants only Claude hooks ships only `hooks.json`; the Codex bridge sees no `codex-hooks.json` and vanishes. This was verified empirically: with both bridges loaded, all pre-existing snapshots (none of which ship a `codex-hooks.json`) are byte-identical.
|
||||
|
||||
Loading both is the minimum that lets the snapshot tier exercise each dialect against the same real app the product ships. Recording (which boots `cordis.yml`) loads both by construction, and replay inherits them the same way: `cordis.snapshot.yml` is an include-overlay of `cordis.yml` that swaps only the llm entry (see [single-source the acp-agent replay config](2026-07-04-single-source-acp-replay-config.md)), so a bridge added to the live tree is in the replay tree with no second edit.
|
||||
|
||||
### 2. A snapshot scenario per hook point × its headline outcome, both dialects
|
||||
|
||||
Thirteen scenarios under `examples/acp-agent/tests/snapshots/`, naming `hook-<dialect>-<point>-<outcome>`:
|
||||
|
||||
- **Authored, no model turn** (keyless, no sidecar — the derived replay script is empty; the `rejected` turn carrying `hook/*` events is compared): `hook-cc-promptsubmit-block`, `hook-codex-promptsubmit-block`.
|
||||
- **Recorded against the real API, hook active during recording** (the model's reaction to the decision is part of the captured transcript, replayed keyless thereafter): `hook-{cc,codex}-promptsubmit-context` (allow + additionalContext fold), `hook-cc-pretool-deny` / `hook-codex-pretool-block` (deny → `isError` tool result), `hook-cc-pretool-ask` (ask → degrades to deny with the approval-required reason), `hook-{cc,codex}-posttool-block` (block with feedback), `hook-{cc,codex}-posttool-context` (accept + additionalContext), `hook-{cc,codex}-stop-continue` (a blocking Stop hook forces one extra step via steering).
|
||||
|
||||
Each hook command emits only FIXED LITERAL strings (no timestamps/pids/`$RANDOM`/cwd echoes); the snapshot normalizer scrubs the one volatile field a `hook/result` carries (`durationMs`). The `Stop` scenarios self-limit with a marker file (`.stop_fired`) so the force-continue does not loop — the `stop_hook_active` loop-guard is still a bridge `TODO`, so an unconditional Stop hook would force-continue every step.
|
||||
|
||||
The `PostToolUse` block scenarios self-limit at the mechanism they prove. The Claude hook persists a workspace marker after its first rejection, so one recovery call is allowed; the Codex prompt makes one call and reports the injected result. Each expected output pins one blocked call without repeated block/retry cycles.
|
||||
|
||||
### Three hook points are deliberately NOT snapshotted
|
||||
|
||||
Discovered while building the matrix, and documented here because the omission is a decision, not an oversight:
|
||||
|
||||
- **`SessionStart` and `SubagentStart`** inject context through a detached, best-effort `void runPoint(...).then(agent.inject())` with NO turn binding. The resulting `context/message` races the work it precedes (the first model request / the child's first turn) and lands at a nondeterministic log position. A recorded expected output does not even reproduce on its own replay — a 10× replay stability check failed 10/10 for both. They stay on the bridges' unit coverage, which drives the seam directly without the timing race. (If the injection is ever made turn-bound and deterministic — the direction the `TODO(session-start-gating)` points — these become snapshottable.)
|
||||
- **`SubagentStop`** is observe-only: its `subagent/end` handler passes no turn (so no `hook/*` log events) and does no injection. It writes NOTHING to the transcript, so an expected output would be byte-identical to the no-hook run and could never be proven to fail — a guard that cannot bite. It stays on unit coverage (`bridge.spec.ts` already asserts the observe-only call).
|
||||
|
||||
The matrix therefore covers every hook point that has a DETERMINISTIC, OBSERVABLE transcript footprint, for both dialects.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Every bridge seam mapping with an observable transcript is now guarded at the full-transcript tier, in the real app, for both dialects — including the Codex bridge, which had no end-to-end coverage at all. Recorded expected outputs capture the model's real reaction to a denied/blocked/force-continued turn, which a hand-authored transcript could only guess at.
|
||||
- The `UserPromptSubmit` block scenarios are authored keylessly (no model turn); the rest replay keylessly from recorded fixtures. `pnpm run test:snapshot:record` regenerates the recorded fixtures from the live API and self-skips without a key like every recorded scenario.
|
||||
- The prove-red discipline holds: tampering a hook config's output (e.g. changing a deny reason) turns its scenario red on replay — the hook process runs FOR REAL during replay (only the model is replayed), so the expected output guards the actual hook→seam→loop path, not a mock of it.
|
||||
- The `acp-agent` demo now loads a Codex bridge it will usually no-op (no `codex-hooks.json` in a typical project), which is the intended fail-soft behavior, not a cost.
|
||||
|
||||
<!-- agent-note-format: alternatives-not-recorded (pre-format Agent Note) -->
|
||||
@@ -1,52 +0,0 @@
|
||||
# Agent Note: 钩子快照矩阵——覆盖两种 bridge 的端到端预期输出测试
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-04-hook-snapshot-matrix.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
钩子 bridge——[`dsh-hooks-claude`](../../../../packages/hooks/hooks-claude)（7 个 Claude Code 钩子点）和 [`dsh-hooks-codex`](../../../../packages/hooks/hooks-codex)（5 个 Codex 点）——把外部钩子命令映射到 harness 拦截 seam。它们有深入的单元与覆盖率规格覆盖（每个决策分支、每种 payload dialect，针对 mock seam 驱动），外加一个受密钥门控的 e2e（`hooks.e2e.ts`，实时 `PreToolUse` 阻止）。但完整 transcript（文本记录）快照层——会启动真实 `acp-agent` 子进程、无需密钥重放已记录会话，并将规范化 ACP（Agent Client Protocol）stdout + 重新持久化日志与已提交预期输出进行 diff 的那张网——只覆盖了一个钩子：Claude `UserPromptSubmit` 阻止（`hook-cc-promptsubmit-block`）。
|
||||
|
||||
这正是 mock 单元测试在结构上无法替代的层级：它验证的是真实 bridge 将真实钩子进程的结果翻译到真实 seam 决策，再经由自动化线协议和持久化日志检验真实 agent loop（智能体循环）的反应。一个 bridge 翻译或 loop 结构的回归，即使让所有单元测试保持绿色，也会在除那一个钩子点之外的所有点上逃逸；而对于 Codex bridge，ACP 示例甚至没有加载它，因此没有任何 Codex 钩子能端到端触发。
|
||||
|
||||
## 决策
|
||||
|
||||
实现由两个耦合部分组成：
|
||||
|
||||
### 1. ACP 示例同时加载两种钩子 bridge
|
||||
|
||||
`examples/acp-agent/cordis.yml` 和 `cordis.snapshot.yml` 现在同时加载 `dsh-hooks-codex` 与 `dsh-hooks-claude`，各自指向自己的配置文件（Claude 用 `./hooks.json`，Codex 用 `./codex-hooks.json`——两种方言无法共用一个文件）。这是一个真正的产品接口变更，而非仅用于测试的接线：交付的 ACP 服务器（以及 `demo:acp` 入口）现在同时携带两种 bridge。
|
||||
|
||||
这是安全的，因为配置文件不存在时 bridge 是**静默无操作**的：`apply()` 捕获读取失败、通过 `ctx.logger` 记录日志、不注册任何东西——零监听器、零会话事件。`acp-agent` 应用不附带 stdout logger，因此警告不会到达 ACP JSON-RPC 通道。只需要 Claude 钩子的场景（或真实项目）只提供 `hooks.json`；Codex bridge 找不到 `codex-hooks.json` 便自动消失。这已通过实验验证：在两种 bridge 同时加载的情况下，所有既有快照（均不附带 `codex-hooks.json`）逐字节一致。
|
||||
|
||||
同时加载是让快照层能够在产品交付的同一个真实应用上验证每种方言的最低要求。录制（启动 `cordis.yml`）天然加载两者，回放以同样方式继承：`cordis.snapshot.yml` 是 `cordis.yml` 的 include-overlay，只替换 llm 入口（见[单一来源 acp-agent 回放配置](2026-07-04-single-source-acp-replay-config.md)），因此添加到运行时树的 bridge 无需第二次编辑即出现在回放树中。
|
||||
|
||||
### 2. 每个钩子点 × 其主要结果各一个快照场景，覆盖两种方言
|
||||
|
||||
`examples/acp-agent/tests/snapshots/` 下共 13 个场景，命名为 `hook-<dialect>-<point>-<outcome>`：
|
||||
|
||||
- **手工编写、无模型轮次**（无密钥、无 sidecar——派生的回放脚本为空；比对的是携带 `hook/*` 事件的 `rejected` 轮次）：`hook-cc-promptsubmit-block`、`hook-codex-promptsubmit-block`。
|
||||
- **对真实 API 录制、录制期间钩子活跃**（模型对决策的反应是捕获的 transcript 的一部分，此后无密钥回放）：`hook-{cc,codex}-promptsubmit-context`（allow + additionalContext 折叠）、`hook-cc-pretool-deny` / `hook-codex-pretool-block`（deny → `isError` 工具结果）、`hook-cc-pretool-ask`（ask → 降级为 deny 并附带 approval-required 原因）、`hook-{cc,codex}-posttool-block`（阻止并附带反馈）、`hook-{cc,codex}-posttool-context`（accept + additionalContext）、`hook-{cc,codex}-stop-continue`（阻塞性 Stop 钩子通过 steering（中途引导）强制多走一步）。
|
||||
|
||||
每个钩子命令只输出固定字面量字符串（无时间戳/pid/`$RANDOM`/cwd 回显）；快照规范化器擦除 `hook/result` 携带的唯一不稳定字段（`durationMs`）。`Stop` 场景通过标记文件（`.stop_fired`）自限，使 force-continue 不会循环——`stop_hook_active` 循环守卫仍是 bridge 的一个 `TODO`，因此无条件的 Stop 钩子会在每一步都 force-continue。
|
||||
|
||||
`PostToolUse` 阻止场景会在其证明的机制处自行限制。Claude 钩子在首次拒绝后持久化一个 workspace 标记，因此允许一次恢复调用；Codex 提示词发起一次调用并报告注入结果。每份预期输出固定一次遭阻止调用，不会重复阻止/重试循环。
|
||||
|
||||
### 三个钩子点被有意排除在快照之外
|
||||
|
||||
在构建矩阵过程中发现，记录于此是因为这些遗漏是决策而非疏忽：
|
||||
|
||||
- **`SessionStart` 和 `SubagentStart`** 通过脱离且尽力而为的 `void runPoint(...).then(agent.inject())` 注入上下文，没有轮次绑定。由此产生的 `context/message` 会与它应先于的工作（首次模型请求 / 子项的第一个轮次）竞速，并落在不确定的日志位置。记录的预期输出甚至无法在自己的重放中复现——对两者执行 10 次重放稳定性检查，结果均为 10/10 次失败。它们继续留在 bridge 的单元覆盖率中，那里会直接驱动 seam 而不存在时序竞速。（如果注入未来改为绑定轮次且具备确定性——`TODO(session-start-gating)` 所指方向——它们就能接受快照测试。）
|
||||
- **`SubagentStop`** 只观察：其 `subagent/end` handler 不传递轮次（因此没有 `hook/*` 日志事件），也不执行注入。它不会向 transcript 写入任何内容，因此预期输出会与无钩子运行逐字节相同，永远无法证明失败——一道咬不住问题的守卫。它继续由单元覆盖率负责（`bridge.spec.ts` 已断言仅观察调用）。
|
||||
|
||||
因此，该矩阵覆盖了所有具有确定性、可观测 transcript 足迹的钩子点，涵盖两种方言。
|
||||
|
||||
## 后果
|
||||
|
||||
- 现在，两种 dialect 中每个具有可观察 transcript 的 bridge seam 映射，都在真实应用的完整 transcript 层受到守护——包括此前完全没有端到端覆盖的 Codex bridge。记录的预期输出捕获模型对遭拒绝/遭阻止/强制继续轮次的真实反应，而手工编写的 transcript 只能猜测这种反应。
|
||||
- `UserPromptSubmit` 阻止场景无需密钥即可编写（没有模型轮次）；其余场景从已记录 fixture（测试前置数据）无需密钥重放。`pnpm run test:snapshot:record` 从实时 API 重新生成记录式 fixture，并像所有记录场景一样在缺少密钥时自行跳过。
|
||||
- 证明会变红的准则仍成立：篡改钩子配置输出（例如改变拒绝理由）会让相应场景在重放时变红——钩子进程在重放期间真实运行（只有模型被重放），因此预期输出守护的是实际钩子→seam→循环路径，而非其 mock。
|
||||
- `acp-agent` 演示现在加载了一个通常会无操作的 Codex bridge（典型项目中没有 `codex-hooks.json`），这正是预期的柔性失败行为，而非代价。
|
||||
|
||||
<!-- agent-note-format: alternatives-not-recorded (pre-format Agent Note) -->
|
||||
@@ -1,6 +0,0 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-04-single-source-acp-replay-config.md: f270d70feca184217503c472c1cb7c536187a249
|
||||
2026-07-04-single-source-acp-replay-config.zh.md: 86eb2d3e15c4939d1de3a78150cd0536d46e10f6
|
||||
@@ -1,27 +0,0 @@
|
||||
# Agent Note: Single-source the acp-agent replay config
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-04-single-source-acp-replay-config.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
`examples/acp-agent` shipped two hand-maintained configs: `cordis.yml` (the live tree) and a `cordis.snapshot.yml` that mirrored it entry-for-entry with only the llm backend swapped — stripped of comments, the entire difference was the eight-line `llm-deepseek` stanza versus the two-line `llm-replay` stanza. Every app-shape change had to be made twice, and nothing gated the symmetry: if the copies drifted, the snapshot tier would silently exercise a different app than the one that ships — the ["green units, broken product" class of gap](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md) the snapshot tier exists to close, reintroduced one level up, with reviewer vigilance as the only defense.
|
||||
|
||||
## Decision
|
||||
|
||||
`cordis.snapshot.yml` includes the live config, disables the named DeepSeek adapter by id and name, and inserts the replay adapter. Every other entry therefore comes from the shipping tree. Replay selects the overlay; recording still boots `cordis.yml`, and the load guard permits the intentionally disabled entry.
|
||||
|
||||
One vendored-plugin fact the overlay depends on, deliberately: the include applies `patches` when it loads the file — its `refresh()`/`internal/update` paths re-read without re-patching — which is exactly enough for a one-shot replay boot (the replay app loads no `hmr` and nothing rewrites the config mid-run). The snapshot suite is the proof: all scenarios pass unchanged on the overlay, byte-identical expected outputs included.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
### Why not the alternatives?
|
||||
|
||||
Keeping the full twin with a symmetry verify-gate was the recorded fallback — it would have removed the silent-drift class but kept a 125-line near-copy whose only content was one entry's difference, growing with every plugin the app gains. A bin-side swap (parse the config, replace the entry, delete the file) would have put YAML surgery inside a published artifact and moved the replay delta out of sight; the overlay keeps the delta declarative, readable, and next to the base config — the teaching value the twin's defenders actually wanted.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A plugin added to `cordis.yml` is in the replay tree with no second edit; the drift class is structurally gone rather than gated.
|
||||
- The overlay depends on entries carrying stable `id:`s. The `name` assertion on the disable patch guards mis-targeting (a reused id skips the patch instead of disabling the wrong plugin). An id RENAME degrades the patch to a skip whose warning needs a logger the replay app deliberately lacks — the observable result is a futile keyless `llm-deepseek` entry alongside `llm-replay`, with replay output still correct (`llm-replay` owns the stream short-circuit); config rot for review to catch, not wrong snapshots. A top-level insert whose id collides with an existing entry resolves last-wins through the loader's id map — the current config has no collision, and a new patch line is where one would be introduced.
|
||||
- If a future replay tree needs a second divergence (another backend swapped), it is one more patch line, not a second fork of the file.
|
||||
@@ -1,27 +0,0 @@
|
||||
# Agent Note: 将 acp-agent 回放配置改为单一来源
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-04-single-source-acp-replay-config.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
`examples/acp-agent` 发布了两份手工维护的配置：`cordis.yml`（实时树）和逐条镜像它、只替换 llm 后端的 `cordis.snapshot.yml`——去除注释后，两者的全部差异就是八行 `llm-deepseek` stanza 与两行 `llm-replay` stanza。每次应用形状变化都必须修改两遍，也没有任何机制约束对称性：如果副本发生漂移，快照层会悄然覆盖与已发布应用不同的应用——快照层本就是为了弥合[“单元测试绿色，产品损坏”这类缺口](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md)，如今同类缺口在上一层重新出现，只能依靠评审者警惕。
|
||||
|
||||
## 决策
|
||||
|
||||
`cordis.snapshot.yml` include 正式配置，通过 id 和 name 禁用指定的 DeepSeek 适配器，并插入回放适配器。其余所有条目因此来自正式运行树。回放时选择 overlay；录制仍然启动 `cordis.yml`，加载守卫允许被有意禁用的条目。
|
||||
|
||||
overlay 有意依赖一项 vendored 插件事实：include 加载文件时会应用 `patches`，而其 `refresh()`/`internal/update` 路径会重新读取但不重新打补丁——这恰好足以满足一次性重放启动（重放应用不加载 `hmr`，运行中也没有内容重写配置）。快照套件就是证明：所有场景都能在 overlay 上原样通过，包括逐字节相同的预期输出。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
### 为何不采用这些替代方案？
|
||||
|
||||
保留完整的双副本并加一道对称性校验门禁是记录在案的退路——它能消除静默漂移这一类问题，但仍保留一份 125 行的近乎复制品，其全部内容只是一个条目的差异，且随应用每增加一个插件而增长。在 bin 侧做替换（解析配置、替换条目、删除文件）则会把 YAML 手术放进发布产物，并把回放差异藏到视线之外；overlay 让差异保持声明式、可读，且紧邻基础配置——这正是双副本支持者真正看重的教学价值。
|
||||
|
||||
## 后果
|
||||
|
||||
- 向 `cordis.yml` 添加插件即自动进入回放树，无需第二次编辑；漂移这一类问题从结构上消失，而非靠门禁拦截。
|
||||
- overlay 依赖条目携带稳定的 `id:`。禁用补丁上的 `name` 断言防止误定位（id 被复用时补丁跳过而非禁用错误的插件）。如果 id 被重命名，补丁退化为跳过，其警告需要一个回放应用有意不具备的 logger——可观测结果是一条无效的无密钥 `llm-deepseek` 条目与 `llm-replay` 并存，回放输出仍然正确（`llm-replay` 拥有流的短路权）；这属于配置腐烂，留给评审发现，不会产生错误的快照。顶层插入一个 id 与既有条目冲突的新条目时，loader 的 id map 以后者为准；当前配置无冲突，新增补丁行才是引入冲突的场所。
|
||||
- 如果未来回放树需要第二处差异（另一个后端被替换），只需多加一行补丁，而非再 fork 一份文件。
|
||||
@@ -1,6 +0,0 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-06-pin-request-header-content-in-one-scenario.md: bca6d9eb943e758d68efaf3a76ec367179cd15fd
|
||||
2026-07-06-pin-request-header-content-in-one-scenario.zh.md: e01c84c63583e2fe14b0b4fe0381a18b209d2346
|
||||
@@ -1,35 +0,0 @@
|
||||
# Agent Note: Pin request-header content in one snapshot scenario
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-06-pin-request-header-content-in-one-scenario.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
An ACP snapshot suite needs to prove the exact composed system prompt and tool-schema list sent in each `request/header`, but duplicating that content inside every `session.jsonl` makes a prompt or schema edit rewrite dozens of giant one-line JSON records. Keeping one raw header avoids the duplication but still makes prompt review poor: prose is JSON-escaped onto one line and mixed with thousands of characters of tool schemas.
|
||||
|
||||
## Decision
|
||||
|
||||
Exactly one scenario per header-composition class is flagged `pinsHeader`. Its directory splits the pin by review format: `system-prompt.expected.md` contains the normalized full prompt sequence as ordinary Markdown, `tool-schemas.expected.json` contains the corresponding complete schema sequence as structured JSON, and `session.jsonl` retains config, reason, and any model-visible prefix while storing `header.system` and `header.tools` as `"{{system}}"` / `"{{tools}}"`. Every other JSONL uses the same prompt and tool tokens and also tokenizes session-prefix content. The pin mechanics live in [`dsh-acp-snapshot`](../../../../packages/support/acp-snapshot/README.md), whose suite factory enforces one pin per class.
|
||||
|
||||
The pure `scrubSystemPrompts` and `scrubToolSchemas` normalizers independently tokenize every stored full header. `scrubRequestHeaders` also tokenizes session-prefix content for non-pinning scenarios while retaining header count, field presence, config, reason, and prefix message count. Record and refresh write-back apply the appropriate scrub before writing JSONL and regenerate both sidecars from the normalized live full-header sequence, so neither path can reintroduce prompt/schema bulk into JSONL or leave a review artifact stale.
|
||||
|
||||
Guards make the split self-enforcing. On disk, every `session*.jsonl` is a fixed point of both prompt and schema scrubbers, only non-pinning fixtures must be fixed points of the full header scrub, both sidecars exist exactly beside pinning fixtures in canonical newline-terminated formats, and each class has one pin. Live, every `request/header` produced by a parent, spawn child, fork child, initial request, resume, or in-instance change must match the reconstructed class sequence after volatile-value normalization. A header without a string prompt, without an array-valued tool list, or beyond the pin's declared changed-header count fails loud.
|
||||
|
||||
One pin covers the whole suite because every session — parent, spawn child, fork child — composes the identical tool list and the identical prompt modulo cwd, and the uniformity guard fails the suite the moment that stops holding. If header composition ever becomes session-dependent by design (a restricted subagent toolset, say), the divergent shape gets its own pinning scenario.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Re-record or hand-edit every fixture per change** — preserves exact headers but buries behavioral diffs under duplicated prompt and schema content.
|
||||
- **Scrub at compare time only, keeping fixtures raw** — lets compares pass while committed fixtures retain stale duplicate content and rewrite wholesale on the next recording. Stored tokens state honestly what each JSONL does not pin.
|
||||
- **Scrub everywhere, pin nowhere** — loses the only end-to-end record of the composed header as actually sent (prompt assembly, registered-tool order, full schemas). The generated tool catalog documents each tool in isolation; only a real fixture pins the composed set.
|
||||
- **Keep the one full pin entirely in JSONL** — removes suite-wide duplication but leaves prompt and schema changes as one escaped line. Markdown and structured JSON give each surface its natural review format without weakening the reconstructed-header assertion.
|
||||
- **Slim the session log itself (log a content digest, store the header elsewhere)** — violates the reconstructability contract: the product log must reproduce each request bit-for-bit ([reconstructable-requests Agent Note](../architecture/2026-07-05-reconstructable-requests.md)). Header bulk is a test-artifact concern, solved in test normalization; the live log is untouched.
|
||||
|
||||
## Verification
|
||||
|
||||
The suite replays every scenario against the split pins. Unit coverage exercises the independent and full scrubbers, both full-header sidecar formats, record/refresh regeneration, normalized prompt/schema extraction, fixed-point enforcement, required-file symmetry, reconstructed-header uniformity, and changed-header count rejection.
|
||||
|
||||
## Consequences
|
||||
|
||||
A system-prompt change produces a line-oriented Markdown diff in one file per affected composition class; a tool-description change produces a structured JSON diff in one file per class; ordinary behavioral fixtures remain untouched. Session fixtures display tokens for omitted content, and the live uniformity guard makes each split pin authoritative for every session in its class. Each pinning scenario carries two generated, newline-canonicalized sidecars.
|
||||
@@ -1,35 +0,0 @@
|
||||
# Agent Note: 在单个快照场景中固定请求头内容
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-06-pin-request-header-content-in-one-scenario.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
一个 ACP（Agent Client Protocol）快照测试套件需要证明每个 `request/header` 中实际发送的组合系统提示词与工具 schema 列表，但如果在每个 `session.jsonl` 中重复这些内容，一次提示词或 schema 编辑就会改写数十条巨大的单行 JSON 记录。保留一份原始 header 可以避免重复，但提示词的评审体验仍然很差：行文被 JSON 转义到一行中，与数千字符的工具 schema 混在一起。
|
||||
|
||||
## 决策
|
||||
|
||||
每种请求头组合类别恰好有一个场景标记为 `pinsHeader`。其目录按评审格式拆分固定内容：`system-prompt.expected.md` 以普通 Markdown 包含规范化的完整提示词序列；`tool-schemas.expected.json` 以结构化 JSON 包含对应的完整 schema 序列；`session.jsonl` 保留 config、reason 和所有模型可见前缀，同时将 `header.system` 与 `header.tools` 存为 `"{{system}}"` / `"{{tools}}"`。其他每份 JSONL 都使用相同的提示词与工具 token，并同样将会话前缀内容 token 化。固定机制位于 [`dsh-acp-snapshot`](../../../../packages/support/acp-snapshot/README.md)，其套件 factory 强制每种类别恰好有一个固定场景。
|
||||
|
||||
纯 `scrubSystemPrompts` 和 `scrubToolSchemas` 规范化器会分别将每个已存储完整请求头 token 化。`scrubRequestHeaders` 还会为非固定场景把会话前缀内容 token 化，同时保留请求头数量、字段存在性、config、reason 和前缀消息数量。记录与刷新写回会在写入 JSONL 前应用适当清理，并根据规范化的实时完整请求头序列重新生成两个 sidecar，因此两条路径都无法把大段提示词/schema 重新引入 JSONL，也不会留下陈旧的评审产物。
|
||||
|
||||
守卫使这种拆分能够自我强制。在磁盘上，每个 `session*.jsonl` 都是提示词和 schema 清理器的固定点；只有非固定 fixture（测试前置数据）必须是完整请求头清理的固定点；两个 sidecar 恰好位于固定 fixture 旁，并采用规范、以换行符结尾的格式；每种类别都有一个固定场景。在实时运行中，由父项、spawn 子项、fork 子项、初始请求、恢复或实例内变化产生的每个 `request/header`，都必须在易变值规范化后与重建的类别序列匹配。请求头若没有字符串提示词、没有数组值工具列表，或超过固定场景声明的变更请求头数量，就会响亮失败。
|
||||
|
||||
一个固定场景覆盖整个套件，因为每个会话（parent、spawn 子会话、fork 子会话）组合出的工具列表完全相同、提示词除 cwd 外完全相同，而一致性守卫会在这一前提不再成立时立即使套件失败。如果 header 组合将来在设计上变为会话相关的（例如受限的 subagent 工具集），那么分歧的形态将获得自己的固定场景。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
- **每次变更重新录制或手动编辑所有 fixture**：保留了精确的 header，但行为差异被重复的提示词和 schema 内容淹没。
|
||||
- **仅在比较时 scrub，fixture 保持原始内容**：比较能通过，但已提交的 fixture 保留着陈旧的重复内容，下次录制时会整体重写。存储 token 诚实地表明每个 JSONL 没有固定什么。
|
||||
- **全部 scrub，不做任何固定**：丢失了组合 header 实际发送内容（提示词组装、已注册工具顺序、完整 schema）的唯一端到端记录。生成的工具目录只孤立地记录每个工具；只有真实 fixture 才能固定组合后的完整集合。
|
||||
- **将完整固定内容全部保留在 JSONL 中**：消除了套件范围的重复，但提示词和 schema 变更仍然是一行转义文本。Markdown 和结构化 JSON 为每种内容提供其自然的评审格式，同时不削弱重建 header 的断言。
|
||||
- **收窄会话日志本身（记录内容 digest，把请求头存到其他位置）**——违反可重建性契约：产品日志必须逐 bit 复现每个请求（[可重建请求 Agent Note（agent 决策记录）](../architecture/2026-07-05-reconstructable-requests.md)）。请求头体积是测试产物问题，应在测试规范化中解决；实时日志保持不变。
|
||||
|
||||
## 验证
|
||||
|
||||
该套件针对拆分后的固定内容回放每个场景。单元覆盖率会覆盖独立与完整清理器、两种完整请求头 sidecar 格式、记录/刷新重新生成、规范化提示词/schema 提取、固定点强制、必需文件对称性、重建请求头一致性，以及变更请求头数量拒绝。
|
||||
|
||||
## 后果
|
||||
|
||||
系统提示词变更在每个受影响的组合类别中产生一个面向行的 Markdown diff；工具描述变更在每个类别中产生一个结构化 JSON diff；普通行为 fixture 不受影响。会话 fixture 对省略的内容显示 token，运行时一致性守卫使每个拆分固定场景对其类别内的所有会话具有权威性。每个固定场景携带两个生成的、换行规范化的 sidecar 文件。
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-08-shared-acp-snapshot-package.md: 3e5a2b12114d535490a17361128862f6d1c09a73
|
||||
2026-07-08-shared-acp-snapshot-package.zh.md: 072ef692702cb769c428f8b6aa3863a3b77d3d59
|
||||
2026-07-08-shared-acp-snapshot-package.md: dc86bf020b159a1c4af26bbc49725ce2b7de8180
|
||||
2026-07-08-shared-acp-snapshot-package.zh.md: 19eb070bbc5aa1a0b72c0cc874225064e92632b8
|
||||
|
||||
@@ -12,7 +12,7 @@ A second ACP example wanting snapshot coverage — the sandbox/approval composit
|
||||
|
||||
## Decision
|
||||
|
||||
The machinery lives in [`packages/support/acp-snapshot`](../../../../packages/support/acp-snapshot/README.md) (`@deepseek-ai/dsh-acp-snapshot`); an example's `*.snapshot.ts` is its scenario table, its agent paths, and one factory call, over its own `snapshots/` fixtures and `cordis.snapshot.yml` overlay ([single-source replay config](2026-07-04-single-source-acp-replay-config.md)). Reading `DSH_SNAPSHOT` stays at that edge — the library takes a resolved `mode`.
|
||||
The machinery lives in [`packages/support/acp-snapshot`](../../../../packages/support/acp-snapshot/README.md) (`@deepseek-ai/dsh-acp-snapshot`); an example's `*.snapshot.ts` is its scenario table, its agent paths, and one factory call, over its own `snapshots/` fixtures and `cordis.snapshot.yml` overlay ([single-source replay config](../../archived/testing/2026-07-04-single-source-acp-replay-config.md)). Reading `DSH_SNAPSHOT` stays at that edge — the library takes a resolved `mode`.
|
||||
|
||||
**`src/launcher.ts`** — `launchAcpTestAgent` owns the common unbuilt-process boundary: absolute tsx loader resolution, `TSX_TSCONFIG_PATH`, isolated harness homes, stdio wiring, a raw-byte stdout tee, stderr and update capture, fail-closed permission fallback, update waiters, and graceful or signalled shutdown. Snapshot scenarios and ordinary e2e suites supply the same `AgentUnderTest` (`binScript`, `configPath`, `tsconfigPath`); a test that plays a user supplies only its permission handler. The ACP and hook e2e suites plus the sandbox/approval e2e suite use this launcher instead of rebuilding the SDK client boundary.
|
||||
|
||||
@@ -20,7 +20,7 @@ The machinery lives in [`packages/support/acp-snapshot`](../../../../packages/su
|
||||
|
||||
**`src/normalize.ts`** — the pure normalizers, hook-free by policy: when a future event carries a new volatile field (an approval duration, say), the shared normalizer learns it in the same change, keeping one home for what "normalized" means rather than per-suite scrub extensions.
|
||||
|
||||
**`src/suite.ts`** — the `Scenario` type and `defineAcpSnapshotSuite(options)`, registering the per-scenario compares, record/refresh fixture write-back, the header pin with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin per class, every JSONL a `scrubSystemPrompts` fixed point, non-pinning fixtures also `scrubRequestHeaders` fixed points). Refresh expands packed timing envelopes before aligning existing volatile event times, so switching between packed and unpacked layouts cannot shift later records; fresh chunk-fragment arrays remain authoritative because their boundaries are replay behavior. A scenario directory's `session.jsonl` plus contiguous `session.<n>.jsonl` siblings are its ordered primary/child inventory, so the scenario table declares policy without duplicating a child count. The pinned-header contract ([pinned-header Agent Note](2026-07-06-pin-request-header-content-in-one-scenario.md)) is per-suite: each header class flags exactly one `pinsHeader` scenario, whose `system-prompt.expected.md` and JSONL tool list split the composed header into reviewable artifacts; the uniformity guard compares both against every live header in that class. A pinning scenario declares any legitimate changed-header count, and its Markdown artifact records every full changed prompt. The pure helpers (`sessionFixtureNames`, `fixtureContext`, `normalizedHeaders`, `normalizedSystemPrompts`, `formatSystemPromptSnapshot`, `headerChangeCount`) are exported from the module for direct unit coverage.
|
||||
**`src/suite.ts`** — the `Scenario` type and `defineAcpSnapshotSuite(options)`, registering the per-scenario compares, record/refresh fixture write-back, the header pin with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin per class, every JSONL a `scrubSystemPrompts` fixed point, non-pinning fixtures also `scrubRequestHeaders` fixed points). Refresh expands packed timing envelopes before aligning existing volatile event times, so switching between packed and unpacked layouts cannot shift later records; fresh chunk-fragment arrays remain authoritative because their boundaries are replay behavior. A scenario directory's `session.jsonl` plus contiguous `session.<n>.jsonl` siblings are its ordered primary/child inventory, so the scenario table declares policy without duplicating a child count. The pinned-header contract ([pinned-header Agent Note](../../archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)) is per-suite: each header class flags exactly one `pinsHeader` scenario, whose `system-prompt.expected.md` and JSONL tool list split the composed header into reviewable artifacts; the uniformity guard compares both against every live header in that class. A pinning scenario declares any legitimate changed-header count, and its Markdown artifact records every full changed prompt. The pure helpers (`sessionFixtureNames`, `fixtureContext`, `normalizedHeaders`, `normalizedSystemPrompts`, `formatSystemPromptSnapshot`, `headerChangeCount`) are exported from the module for direct unit coverage.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ ACP（Agent Client Protocol）快照层（[快照 Agent Note（agent 决策记
|
||||
|
||||
## 决策
|
||||
|
||||
这些机制位于 [`packages/support/acp-snapshot`](../../../../packages/support/acp-snapshot/README.md)（`@deepseek-ai/dsh-acp-snapshot`）；示例的 `*.snapshot.ts` 只包含场景表、agent 路径和一次工厂调用，依赖自己的 `snapshots/` fixture 与 `cordis.snapshot.yml` overlay（[单源回放配置](2026-07-04-single-source-acp-replay-config.md)）。读取 `DSH_SNAPSHOT` 留在边缘层——库接收的是已解析的 `mode`。
|
||||
这些机制位于 [`packages/support/acp-snapshot`](../../../../packages/support/acp-snapshot/README.md)（`@deepseek-ai/dsh-acp-snapshot`）；示例的 `*.snapshot.ts` 只包含场景表、agent 路径和一次工厂调用，依赖自己的 `snapshots/` fixture 与 `cordis.snapshot.yml` overlay（[单源回放配置](../../archived/testing/2026-07-04-single-source-acp-replay-config.md)）。读取 `DSH_SNAPSHOT` 留在边缘层——库接收的是已解析的 `mode`。
|
||||
|
||||
**`src/launcher.ts`**——`launchAcpTestAgent` 拥有通用的未构建进程边界：绝对 tsx loader 解析、`TSX_TSCONFIG_PATH`、隔离的 harness home、stdio 接线、原始字节 stdout tee、stderr 与更新捕获、失败关闭的权限后备、更新 waiter，以及优雅或信号式关闭。快照场景和普通 e2e 套件提供相同的 `AgentUnderTest`（`binScript`、`configPath`、`tsconfigPath`）；扮演用户的测试只提供其权限 handler。ACP 与钩子 e2e 套件以及沙箱/approval e2e 套件都使用该 launcher，而不再重新构建 SDK client 边界。
|
||||
|
||||
@@ -20,7 +20,7 @@ ACP（Agent Client Protocol）快照层（[快照 Agent Note（agent 决策记
|
||||
|
||||
**`src/normalize.ts`** 是纯规范化器，按策略不含钩子：当未来某个事件携带新的易变字段（例如审批耗时），共享规范化器在同一个变更中学会它，保持「规范化」的含义只有一个归属，而非各套件各自扩展清洗逻辑。
|
||||
|
||||
**`src/suite.ts`**——包含 `Scenario` 类型和 `defineAcpSnapshotSuite(options)`，注册各场景比较、记录/刷新 fixture 写回、带实时一致性守卫的请求头固定项，以及 fixture 守卫块（没有孤立场景目录、必需文件存在、每种类别恰好一个固定项、每份 JSONL 都是 `scrubSystemPrompts` 固定点、非固定 fixture 同时也是 `scrubRequestHeaders` 固定点）。刷新会先展开打包的计时信封，再对齐现有易变事件时间，因此在打包与未打包布局之间切换不会移动后续记录；全新的分片片段数组仍为权威，因为其边界属于回放行为。场景目录中的 `session.jsonl` 加连续的 `session.<n>.jsonl` 同级文件构成有序主项/子项清单，因此场景表可以声明策略而不重复子项数量。固定请求头契约（[固定请求头 Agent Note](2026-07-06-pin-request-header-content-in-one-scenario.md)）按套件生效：每种请求头类别恰好标记一个 `pinsHeader` 场景，其 `system-prompt.expected.md` 和 JSONL 工具列表把组合请求头拆成可评审产物；一致性守卫会将两者与该类别的每个实时请求头比较。固定场景可以声明任何合法的变更请求头数量，其 Markdown 产物记录每个完整的已变提示词。纯辅助函数（`sessionFixtureNames`、`fixtureContext`、`normalizedHeaders`、`normalizedSystemPrompts`、`formatSystemPromptSnapshot`、`headerChangeCount`）从模块导出，以便直接进行单元覆盖。
|
||||
**`src/suite.ts`**——包含 `Scenario` 类型和 `defineAcpSnapshotSuite(options)`，注册各场景比较、记录/刷新 fixture 写回、带实时一致性守卫的请求头固定项，以及 fixture 守卫块（没有孤立场景目录、必需文件存在、每种类别恰好一个固定项、每份 JSONL 都是 `scrubSystemPrompts` 固定点、非固定 fixture 同时也是 `scrubRequestHeaders` 固定点）。刷新会先展开打包的计时信封，再对齐现有易变事件时间，因此在打包与未打包布局之间切换不会移动后续记录；全新的分片片段数组仍为权威，因为其边界属于回放行为。场景目录中的 `session.jsonl` 加连续的 `session.<n>.jsonl` 同级文件构成有序主项/子项清单，因此场景表可以声明策略而不重复子项数量。固定请求头契约（[固定请求头 Agent Note](../../archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)）按套件生效：每种请求头类别恰好标记一个 `pinsHeader` 场景，其 `system-prompt.expected.md` 和 JSONL 工具列表把组合请求头拆成可评审产物；一致性守卫会将两者与该类别的每个实时请求头比较。固定场景可以声明任何合法的变更请求头数量，其 Markdown 产物记录每个完整的已变提示词。纯辅助函数（`sessionFixtureNames`、`fixtureContext`、`normalizedHeaders`、`normalizedSystemPrompts`、`formatSystemPromptSnapshot`、`headerChangeCount`）从模块导出，以便直接进行单元覆盖。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-24-web-gui-browser-e2e-lane.md: f7f5d6382e6b7d4a7f2949cc43e0be1778ab9516
|
||||
2026-07-24-web-gui-browser-e2e-lane.zh.md: f3ded2085eb383dbbfb847e2166533328a6e32a8
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md
|
||||
2026-07-24-web-gui-browser-e2e-lane.md: d9e0a9660ecd6aeb75e835e68f92c0a268423872
|
||||
2026-07-24-web-gui-browser-e2e-lane.zh.md: e8c7d1c4596f20d88bd08423549fb6a9f7b0654b
|
||||
|
||||
@@ -10,7 +10,7 @@ The web GUI ships as a real assembled chain — chromium page → client plugin
|
||||
|
||||
## Decision
|
||||
|
||||
`pnpm run test:web` carries a keyless, deterministic browser e2e lane under `apps/web/tests/`: recorded session-log fixtures replayed through `@deepseek-ai/dsh-llm-replay` against the real in-process web composition, asserting a normalized conversation aria golden plus in-process world state. No new package; the product deltas are two additive `dsh-llm-replay` surfaces (`paceMs`, `ReplayHandle`).
|
||||
`pnpm run test:web` carries a keyless, deterministic browser e2e lane under `apps/web/tests/`: recorded session-log fixtures replay through `@deepseek-ai/dsh-llm-replay` against the real in-process web composition, with normalized aria goldens for user-visible states and in-process assertions for durable world state. The supporting product contracts are `dsh-llm-replay` pacing, consumption checks, and validated indexed override patches; cross-package `dsh-llm` failures retain validated provider facts through own data properties; and the shipped web composition mounts `llm-retry` for transient model failures.
|
||||
|
||||
### Scaffold: `apps/web/tests/scaffold.ts`
|
||||
|
||||
@@ -32,18 +32,17 @@ Every scenario fails on any pageerror and on the client's connection-loss/gap-re
|
||||
|
||||
### Expected outputs
|
||||
|
||||
One committed golden per scenario: a normalized `ariaSnapshot()` of the conversation region (`ui.expected.md`) — uuid/cwd/workspace-basename/duration tokens normalized, captured poll-until-equal at the settled milestone — plus a few role/text anchor assertions that stay green under a semantics-preserving component rewrite while the golden churns reviewably. The aria tree is the mechanization of the client rule "assert what the user would see, never class names". World-state assertions ride root-context session events inline (which tool call produced which durable result, whether `turn/end` completed) instead of a second committed log golden: the persisted-log surface is pinned by the ACP/headless/TUI suites through the same loop and persistence, and re-pinning it here would double refresh cost against the tier discipline. `refresh` is the sole golden writer — a missing golden in replay mode fails with the healing command rather than self-bootstrapping.
|
||||
Scenarios with a stable owning region commit a normalized `ariaSnapshot()` for each distinct user-visible state; cross-region workspace-management states instead use semantic DOM assertions plus authoritative host-state checks. UUID, cwd, workspace basename, and duration volatility collapse to stable tokens; captures poll until consecutive normalized reads agree. Role and text anchors remain semantic guards around the reviewable goldens and own cross-region states directly. World-state assertions use root-context session events rather than a second committed log golden because the ACP, headless, and TUI suites already pin the persisted-log surface through the same loop and persistence. `refresh` is the sole golden writer; a missing replay golden fails with the regeneration command.
|
||||
|
||||
The typecheck plane split is structural: the three files that boot the host spine (`scaffold`, `replay-round-trip.e2e`, and `seeded-history.e2e`) are excluded from the client-registered `apps/web` project. Those files and their shared `support.ts` are included file-by-file in `tsconfig.host.json` — one program cannot hold both sides of the cordis `Context` merges.
|
||||
The typecheck plane split is structural: the host scaffold, its support module, and every web spec that boots or inspects the host composition are excluded from the client-registered `apps/web` project and included file-by-file in `tsconfig.host.json`. One program cannot hold both sides of the Cordis `Context` merges.
|
||||
|
||||
### Modes and fixtures
|
||||
|
||||
`DSH_SNAPSHOT` selects replay (default, keyless), record (with key), or refresh (keyless) as inline spec branches — the TUI shape, not a suite factory: at two scenarios the acp-snapshot factory machinery has no owner, and the genuinely shared parts are already exported (`scrubRequestHeaders`, `parseSessionLog`, `installLlmReplay`). Each spec splits into drive steps (type, send, `whenTurnSettled` — run in all modes, never waiting on model-content selectors, so record cannot hang on a live model answering differently) and assertion steps (replay/refresh only). Record = drive live through the real composer + harvest the in-memory `session.header`/`session.events` (the TUI `rawSessionLog` shape — no file decompression) + `scrubRequestHeaders` + `{{sessionId}}`/`{{cwd}}`/`{{rpcId}}` tokenization; a follow-up keyless refresh regenerates `ui.expected.md`. Both scenarios' fixtures were recorded against this assembly through this flow. A drift guard ties each spec's drive prompt to the fixture's recorded `user/message`. A fixture-inventory guard holds each scenario directory closed (exact file set, every JSONL a scrub fixed-point with no run-local `rpcId`). Web fixtures scrub headers everywhere and pin no header class, following the TUI precedent over the strict [pinned-header](2026-07-06-pin-request-header-content-in-one-scenario.md) reading — see Deferred.
|
||||
`DSH_SNAPSHOT` selects replay (default, keyless), record (with key), or refresh (keyless). Prompting specs separate drive steps shared by all modes from replay/refresh assertions; record mode drives the live composer, harvests the in-memory session header and events, scrubs request headers, and tokenizes run-local session, cwd, and RPC identities. A follow-up keyless refresh regenerates aria goldens. Each prompt is checked against its fixture's recorded `user/message`, and each scenario directory has a closed inventory whose JSONL files are scrub fixed points. Web fixtures scrub headers everywhere and pin no header class; see Deferred.
|
||||
|
||||
### Scenarios
|
||||
### Coverage contract
|
||||
|
||||
1. **`replay-round-trip`** — new session, prompt through the real composer, replay streams reasoning + a `bash` tool call that really executes in the temp workspace + final text (paced 15ms). Asserts settled markdown, the aria golden, and inline world state (the bash call's durable result is exactly `WEB_E2E_OK\n`, completed `turn/end`, >10 chunk events).
|
||||
2. **`seeded-history`** — a recorded session seeded cold; the sidebar lists it (group row → session row, collapsed by default), opening renders tool cards and text purely from the log through the implicit cold-resume attach inside `session.history` — zero model calls in replay, so no binding constraints; record mode drives the same turn live (real `read` tool against seeded workspace files) to produce the seed.
|
||||
The lane covers three behavior families. Live-turn scenarios pin ordinary tool execution, cancellation, non-retryable failure, transient retry, resident questions, and mid-turn steering; synchronization uses durable events, `whenIdle()`, or an explicit replay marker rather than delays. Cold-history scenarios seed through the real persistence API and cover history rendering, sidebar search, trajectory and waterfall views, and tool details without model calls. Browser-lifecycle scenarios cover first-send workspace materialization, reload recovery, layout persistence, theme and locale preferences, and workspace create/rename/view operations. Each family asserts the browser surface and the authoritative host state; a stray model call or under-consumed fixture fails teardown.
|
||||
|
||||
### CI stance
|
||||
|
||||
@@ -63,7 +62,7 @@ Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot
|
||||
|
||||
**Placeholder `DEEPSEEK_API_KEY` + replay interception instead of disabling the adapter row.** Rejected despite zero composition change and two in-tree precedents: it satisfies `llm-deepseek`'s fail-loud key check with a lie and leaves a dead adapter mounted-but-intercepted; the disabled row (the ACP overlay's move) is honest keylessness and fails loud at the earliest resolvable point.
|
||||
|
||||
**A `packages/support/web-snapshot` package with a `defineWebSnapshotSuite` factory.** Rejected: chromium-driving source cannot honestly hold per-file 100% coverage on browserless coverage runners, and at two scenarios a factory generalizes from one consumer while the genuinely shared logic is already exported from gated packages. Re-entry trigger: a second web-shaped consumer or ≥6 scenarios with demonstrably drifting inline branches; the package boundary would then be drawn browser-free.
|
||||
**A `packages/support/web-snapshot` package with a `defineWebSnapshotSuite` factory.** Rejected: chromium-driving source cannot honestly hold per-file 100% coverage on browserless coverage runners, and the scenario-specific interactions have not produced a stable browser-free contract beyond the helpers already exported from gated packages and the local scaffold. Reconsider when a second web-shaped consumer or demonstrably repeated lifecycle code establishes that contract.
|
||||
|
||||
**A committed normalized-session-log golden as a second expected surface.** Rejected: the log surface is pinned by the ACP/headless/TUI suites through the same loop and persistence; here it would double refresh cost and re-test lower tiers. Inline world-state assertions on root-context events keep the world-verification duty.
|
||||
|
||||
@@ -73,17 +72,20 @@ Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot
|
||||
|
||||
**Real-model browser tests as the keyless lane.** Rejected: nondeterministic by construction; the surveyed cautionary case (open-webui) grew unbounded timeouts and was deleted. The with-key W5 smoke stays as the live-model complement.
|
||||
|
||||
**A client `data-dsh-busy` settled signal.** Deferred: the multi-condition settled polls proved sufficient at two scenarios and the host-side `whenIdle` barrier does the heavy lifting. Re-entry trigger: the first settled-poll flake, or a scenario needing a state the DOM does not expose.
|
||||
**A client `data-dsh-busy` settled signal.** Deferred: the host-side `whenIdle` barrier plus stable DOM polls cover the current scenarios. Reconsider after the first settled-poll flake or when a required state is not observable in the DOM.
|
||||
|
||||
## Testing
|
||||
|
||||
The lane itself: `pnpm run test:web` runs both scenarios keylessly alongside the existing smoke pair; `DSH_SNAPSHOT=record pnpm exec vitest run --config vitest.web.config.ts apps/web/tests/<spec>` re-records a scenario's fixture against the live model; `DSH_SNAPSHOT=refresh` rewrites both aria goldens keylessly. `paceMs` validation, pacing floor, abort-during-pace, and both `assertConsumed` failure shapes are pinned in `packages/support/llm-replay/tests/llm-replay.spec.ts`.
|
||||
`pnpm run test:web` runs the lane keylessly. `DSH_SNAPSHOT=record pnpm exec vitest run --config vitest.web.config.ts apps/web/tests/<spec>` records a prompting scenario against the live model, and `DSH_SNAPSHOT=refresh` rewrites aria goldens keylessly. `dsh-llm-replay` unit coverage pins pacing, cancellation, consumption diagnostics, sidecar validation, indexed replacement, and the single append position.
|
||||
|
||||
## Deferred
|
||||
|
||||
- **Web header-class pin**: web fixtures tokenize `{{system}}`/`{{tools}}` everywhere and no scenario pins the web composition's prompt/tool schemas (`TODO(web-header-pin)` — the scaffold `recordFixture` JSDoc marks it). Following the TUI scrub-everywhere precedent; revisit when the web assembly's header diverges from the repl composition it mirrors.
|
||||
- **CI browser provisioning**: reversal of the no-browser-in-CI ruling, staged criteria above (`TODO(ci-browser)`).
|
||||
- **Follow-up-prompt-after-resume scenario**: the history/live stitch path over the real wire; add as its own scenario when that code changes or regresses.
|
||||
- **Web error surface**: the client consumes no `agent/error` frames and a pre-chunk failure freezes no partial, so a non-retryable provider failure renders no error copy — the user sees the send simply stop. The AUTH scenario pins the current contract (no crash, composer recovers, turn logged `error`) and `FIXME(web-error-surface)` marks where visible error text gets asserted once the UI grows an error rendering.
|
||||
- **Composer steering gesture**: the input locks while running (stop-or-wait), so the steering scenario steers over the wire from the page; `TODO(web-steer-composer)` upgrades the drive step to a real composer gesture when the product grows one.
|
||||
- **Drag session reorder**: `workspace.insertSessionBefore` has no browser scenario; it needs two sessions materialized in one workspace plus synthesized HTML5 drag events. Add it when that surface changes or regresses. The inert session Rename/Fork/Delete and workspace Delete menu rows get scenarios when they gain behavior.
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
## 决策
|
||||
|
||||
`pnpm run test:web` 携带 `apps/web/tests/` 下的无密钥、确定性浏览器 e2e 车道：录制的会话日志 fixture 经 `@deepseek-ai/dsh-llm-replay` 对真实进程内 web 组合回放，断言规范化后的会话区 aria 预期输出加进程内世界状态。不新增包（package）；产品侧增量只有 `dsh-llm-replay` 的两处增量接口（`paceMs`、`ReplayHandle`）。
|
||||
`pnpm run test:web` 携带 `apps/web/tests/` 下的无密钥、确定性浏览器 e2e 车道：录制的会话日志 fixture 经 `@deepseek-ai/dsh-llm-replay` 对真实进程内 web 组合回放；用户可见状态使用规范化的 aria 预期输出，持久世界状态则使用进程内断言。配套的产品契约包括 `dsh-llm-replay` 的节奏控制、消费检查与已校验的索引式覆写 patch；跨包的 `dsh-llm` 失败通过自有数据属性保留经校验的提供方信息；已交付的 web 组合挂载 `llm-retry`，以处理瞬态模型失败。
|
||||
|
||||
### Scaffold：`apps/web/tests/scaffold.ts`
|
||||
|
||||
@@ -32,18 +32,17 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
### 预期输出
|
||||
|
||||
每场景一份提交的预期输出：会话区规范化 `ariaSnapshot()`（`ui.expected.md`）——uuid/cwd/工作区目录名/时长归一为稳定 token，在安定里程碑处轮询至两次相等再采集——外加几条 role/文本锚断言，让保语义的组件重写在预期输出可评审地变动时仍保持绿色锚点。aria 树是 client 规则「断言用户所见，绝不断言类名」的机械化。世界状态断言内联在根上下文的会话事件上（哪次工具调用产生了哪项已持久化的工具结果、`turn/end` 是否完成）而不是第二份提交的日志预期输出：持久化日志表面已由 ACP/headless/TUI 套件经同一循环和持久化钉住，在此重复钉住会违背分层纪律、翻倍刷新成本。`refresh` 是预期输出的唯一写入者——回放模式下预期输出缺失会连同修复命令一起报错，而不是静默自举。
|
||||
具有稳定所属区域的场景会为每个不同的用户可见状态提交一份规范化的 `ariaSnapshot()`；跨区域的工作区管理状态则使用语义 DOM 断言和权威的 host 状态检查。UUID、cwd、工作区目录名与时长等易变内容会归一为稳定 token；采集过程持续轮询，直到连续两次规范化读取结果相同。Role 与文本锚点继续充当可评审预期输出周围的语义防线，并直接覆盖跨区域状态。世界状态断言使用根上下文的会话事件，而不是第二份提交的日志预期输出，因为 ACP、headless 与 TUI 套件已经通过同一循环和持久化钉住持久化日志表面。`refresh` 是预期输出的唯一写入者；回放模式下缺少预期输出时，测试会连同重新生成命令一起失败。
|
||||
|
||||
类型检查平面切分是结构性的：启动 host 主干的三个文件（`scaffold`、`replay-round-trip.e2e` 和 `seeded-history.e2e`）被排除出注册在 client 侧的 `apps/web` 工程。这三个文件及其共享的 `support.ts` 逐文件纳入 `tsconfig.host.json`——一个程序不能同时持有 cordis `Context` 合并的两侧。
|
||||
类型检查平面切分是结构性的：host scaffold、其支持模块，以及每个启动或检查 host 组合的 web spec 都会从注册在 client 侧的 `apps/web` 工程中排除，并逐文件纳入 `tsconfig.host.json`。一个程序不能同时持有 Cordis `Context` 合并的两侧。
|
||||
|
||||
### 模式与 fixture
|
||||
|
||||
`DSH_SNAPSHOT` 以内联 spec 分支选择 replay（默认，无密钥）、record（带密钥）或 refresh（无密钥）——TUI 的形态，不是套件工厂：两个场景撑不起 acp-snapshot 工厂机制，且真正共享的部分已被导出（`scrubRequestHeaders`、`parseSessionLog`、`installLlmReplay`）。每个 spec 切分为驱动步骤（输入、发送、`whenTurnSettled`——所有模式都执行，绝不等待模型内容选择器，因此 record 不会因真实模型答法不同而挂起）与断言步骤（仅 replay/refresh）。Record = 经真实输入框实时驱动 + 采收内存中的 `session.header`/`session.events`（TUI 的 `rawSessionLog` 形态——无需文件解压）+ `scrubRequestHeaders` + `{{sessionId}}`/`{{cwd}}`/`{{rpcId}}` token 化；随后一次无密钥 refresh 重新生成 `ui.expected.md`。两个场景的 fixture 都经此流程对本组装录制。一条漂移防线把每个 spec 的驱动提示词与 fixture 录制的 `user/message` 绑定。fixture 清单防线保持每个场景目录封闭（精确文件集合，每个 JSONL 都是脱敏不动点，不含当次运行的 `rpcId`）。Web fixture 全部脱敏请求头且不钉任何头类别，沿用 TUI 先例而非[钉住请求头](2026-07-06-pin-request-header-content-in-one-scenario.md)的严格读法——见「暂缓」。
|
||||
`DSH_SNAPSHOT` 选择 replay（默认，无密钥）、record（带密钥）或 refresh（无密钥）。发起提示的 spec 将所有模式共用的驱动步骤与仅供 replay/refresh 使用的断言分开；record 模式驱动真实输入框，采收内存中的会话 header 与事件，脱敏请求头，并 token 化当次运行的会话、cwd 与 RPC 标识。随后一次无密钥 refresh 重新生成 aria 预期输出。每条提示词都会与 fixture 中录制的 `user/message` 核对；每个场景目录都采用封闭清单，其中每个 JSONL 都是脱敏不动点。Web fixture 全部脱敏请求头且不钉任何 header 类别；见「暂缓」。
|
||||
|
||||
### 场景
|
||||
### 覆盖契约
|
||||
|
||||
1. **`replay-round-trip`**——新会话，经真实输入框发送提示词，回放流式输出推理（reasoning）+ 一次在临时工作区真实执行的 `bash` 工具调用 + 最终文本（15ms 节奏）。断言安定后的 markdown、aria 预期输出与内联世界状态（这次 bash 调用的已持久化工具结果严格等于 `WEB_E2E_OK\n`、完成的 `turn/end`、>10 个分片事件）。
|
||||
2. **`seeded-history`**——冷播种一份已录会话；侧栏列出它（分组行 → 会话行，默认折叠），打开后纯凭日志经 `session.history` 内的隐式冷恢复挂载渲染工具卡片与文本——replay 下零模型调用，因此没有任何绑定约束；record 模式实时驱动同一轮（真实 `read` 工具读取播种的工作区文件）来产出种子。
|
||||
该车道覆盖三类行为。实时轮次场景钉住普通工具执行、取消、不可重试失败、瞬态重试、常驻提问与轮次中途 steering；同步依赖持久事件、`whenIdle()` 或显式回放标记，而不使用延时。冷历史场景通过真实持久化 API 播种，在不调用模型的情况下覆盖历史渲染、侧栏搜索、Trajectory 与 Waterfall 视图及工具详情。浏览器生命周期场景覆盖首次发送时物化工作区、重新加载恢复、布局持久化、主题与语言偏好，以及工作区的创建、重命名和视图操作。每类场景都断言浏览器表面和权威的 host 状态；离群的模型调用或未耗尽的 fixture 会使拆卸失败。
|
||||
|
||||
### CI 立场
|
||||
|
||||
@@ -63,7 +62,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
**用占位 `DEEPSEEK_API_KEY` + 回放拦截替代禁用适配器行。** 尽管零组合改动且树内有两处先例仍被否决：它用谎言满足 `llm-deepseek` 的快速失败密钥检查，还留下一个挂载却被拦截的死适配器；禁用行（ACP overlay 的同款做法）是诚实的无密钥，并在最早可解析点快速失败。
|
||||
|
||||
**`packages/support/web-snapshot` 包 + `defineWebSnapshotSuite` 工厂。** 已否决：驱动 chromium 的源码在无浏览器的覆盖率 runner 上无法诚实保持逐文件 100%，且两个场景就上工厂是从单一消费方过度泛化，真正共享的逻辑已从受门禁的包中导出。重启条件：出现第二个 web 形态消费方，或 ≥6 个场景的内联分支被证实各自漂移；届时包边界将画在无浏览器一侧。
|
||||
**`packages/support/web-snapshot` 包 + `defineWebSnapshotSuite` 工厂。** 已否决：驱动 chromium 的源码在无浏览器的覆盖率 runner 上无法诚实保持逐文件 100%，且除受门禁的包已导出的辅助工具与本地 scaffold 外，这些场景专用交互尚未形成稳定的无浏览器契约。出现第二个 web 形态消费方，或被证实重复的生命周期代码确立该契约后，再重新考虑。
|
||||
|
||||
**第二份提交的规范化会话日志预期输出。** 已否决：日志表面已由 ACP/headless/TUI 套件经同一循环与持久化钉住；在此只会翻倍刷新成本并重复测试下层。内联在根上下文事件上的世界状态断言保住了验证世界的义务。
|
||||
|
||||
@@ -73,17 +72,20 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
**以真实模型浏览器测试充当无密钥车道。** 已否决：按构造即不确定；被调研的前车之鉴（open-webui）长出无界超时后被删除。带密钥的 W5 冒烟仍是真实模型侧的补充。
|
||||
|
||||
**客户端 `data-dsh-busy` 安定信号。** 暂缓：两个场景下多条件安定轮询已经够用，host 侧 `whenIdle` 屏障承担了重活。重启条件：第一次安定轮询抖动，或某场景需要等待 DOM 不暴露的状态。
|
||||
**客户端 `data-dsh-busy` 安定信号。** 暂缓：host 侧 `whenIdle` 屏障配合稳定 DOM 轮询，足以覆盖当前场景。第一次安定轮询抖动，或必要状态在 DOM 中不可观察时，再重新考虑。
|
||||
|
||||
## Testing
|
||||
|
||||
车道自身：`pnpm run test:web` 与既有冒烟对一起无密钥运行两个场景；`DSH_SNAPSHOT=record pnpm exec vitest run --config vitest.web.config.ts apps/web/tests/<spec>` 对真实模型重录某场景的 fixture；`DSH_SNAPSHOT=refresh` 无密钥重写两份 aria 预期输出。`paceMs` 校验、节奏下限、节奏中中止、`assertConsumed` 的两种失败形态钉在 `packages/support/llm-replay/tests/llm-replay.spec.ts`。
|
||||
`pnpm run test:web` 无密钥运行该车道。`DSH_SNAPSHOT=record pnpm exec vitest run --config vitest.web.config.ts apps/web/tests/<spec>` 对真实模型录制一个发起提示的场景，`DSH_SNAPSHOT=refresh` 则无密钥重写 aria 预期输出。`dsh-llm-replay` 单元覆盖率钉住节奏控制、取消、消费诊断、sidecar 校验、按索引替换与唯一的追加位置。
|
||||
|
||||
## 暂缓
|
||||
|
||||
- **Web 头类别钉住**：web fixture 处处 token 化 `{{system}}`/`{{tools}}`，没有场景钉住 web 组合的提示词/工具 schema（`TODO(web-header-pin)`——scaffold 的 `recordFixture` JSDoc 有标记）。沿用 TUI 处处脱敏先例；当 web 组装的请求头与其镜像的 repl 组合进一步分叉时重审。
|
||||
- **CI 浏览器供给**：推翻 CI 无浏览器裁定，分阶段标准见上（`TODO(ci-browser)`）。
|
||||
- **恢复后追问场景**：真实 wire 上的历史/实时缝合路径；当该代码变更或回归时作为独立场景补充。
|
||||
- **Web 错误表面**：客户端不消费任何 `agent/error` 帧，分片前的失败也没有可冻结的部分输出，因此不可重试的提供方失败不渲染任何错误文案——用户看到的只是发送就此停住。AUTH 场景钉住当前契约（不崩溃、输入框恢复可用、轮次记录为 `error`），`FIXME(web-error-surface)` 标记了待 UI 长出错误渲染后断言可见错误文本的位置。
|
||||
- **输入框 steering 手势**：输入在运行期间锁定（只能停止或等待），因此 steering 场景从页面走 wire 做 steer；`TODO(web-steer-composer)` 待产品长出真实的输入框手势后，把驱动步骤升级为该手势。
|
||||
- **拖拽会话重排**：`workspace.insertSessionBefore` 尚无浏览器场景；它需要在同一个工作区里物化两个会话，并合成 HTML5 拖拽事件。当该表面变更或回归时再补充。无行为的会话 Rename/Fork/Delete 和工作区 Delete 菜单行待获得行为后再补充场景。
|
||||
|
||||
## 后果
|
||||
|
||||
|
||||
Reference in New Issue
Block a user