From eb51d4c6696927556ab9ad554d173b2e6beeb8a1 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 15:41:16 +0800 Subject: [PATCH 1/4] fix(acp-snapshot): retain unchanged message ids --- ...table-snapshot-refresh-volatiles.i18n.yaml | 4 +- ...07-27-stable-snapshot-refresh-volatiles.md | 10 +- ...27-stable-snapshot-refresh-volatiles.zh.md | 10 +- .../support/acp-snapshot/README.i18n.yaml | 4 +- packages/support/acp-snapshot/README.md | 2 +- packages/support/acp-snapshot/README.zh.md | 2 +- packages/support/acp-snapshot/src/suite.ts | 112 +++++++++++++-- .../record-suite/rec-child/behavior.json | 6 +- .../record-suite/rec-child/session.1.jsonl | 1 + .../record-suite/rec-child/session.jsonl | 1 + .../support/acp-snapshot/tests/suite.spec.ts | 128 ++++++++++++++++++ 11 files changed, 253 insertions(+), 27 deletions(-) diff --git a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.i18n.yaml index f2d73ddf1f..28e3accc20 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.i18n.yaml +++ b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md -2026-07-27-stable-snapshot-refresh-volatiles.md: e2e951cd9f78b319a701a3e60afba48786633f03 -2026-07-27-stable-snapshot-refresh-volatiles.zh.md: 55302b509e28520f90f6cd820e4962be014318cc +2026-07-27-stable-snapshot-refresh-volatiles.md: 5b513ea026008fc0c4ae9a8045408c534c050bec +2026-07-27-stable-snapshot-refresh-volatiles.zh.md: f3c21d14ae235179ca15ec30d964e02877aa86e9 diff --git a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md index e2e951cd9f..5b513ea026 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md +++ b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md @@ -8,9 +8,13 @@ English | [中文](2026-07-27-stable-snapshot-refresh-volatiles.zh.md) ACP snapshot comparison normalizes generated UUIDs, cwd aliases, spill locators, embedded event times, and omitted-byte counts, but refresh write-back persisted the fresh raw values. A behaviorally unchanged refresh therefore rewrote fixtures with new randomness or host-specific path spellings even though the comparison contract considered both logs equal. +Message identity needs a weaker structural precondition than aligned records: an unrelated log event can break record alignment while an inherited message's identity-free value remains unchanged across parent and child logs. Record mode also begins with freshly minted message UUIDs when it replaces an existing fixture. + ## Decision -Refresh write-back uses `normalizeSessionLog` as its sole volatile-value authority. It normalizes the original harvested records with the fresh run's ids, cwd, and every cwd alias, while normalizing fixture records with the fixture header context; literal replacements affect only the raw values being written. After existing record alignment, it recursively compares fresh and existing leaves through those normalized records: normalized-equivalent leaves retain the existing raw value, while normalized-distinct leaves retain the fresh semantic value. +Before record or refresh writes fixtures, the suite fingerprints every complete surface message with its top-level `id` removed and groups occurrences across all parent/child logs. It reuses an existing UUID only when one fingerprint resolves to exactly one fresh ID and one existing ID, then applies that mapping to every fresh log. Repeated inherited occurrences with the same ID remain one candidate, while new, changed, duplicate-content, malformed, and conflicting messages keep their fresh IDs. + +Refresh write-back uses `normalizeSessionLog` as its volatile-value authority for aligned leaves. It normalizes the original harvested records with the fresh run's ids, cwd, and every cwd alias, while normalizing fixture records with the fixture header context; literal replacements affect only the raw values being written. After existing record alignment, it recursively compares fresh and existing leaves through those normalized records: normalized-equivalent leaves retain the existing raw value, while normalized-distinct leaves retain the fresh semantic value. Before reuse, the complete logical-record layout must align, apart from the existing packed-chunk and inserted-title equivalences. Normalized-equivalent changed strings form a log-wide bijection: one fresh string maps to exactly one existing string and vice versa, so repeated IDs remain correlated across records. An unexplained record mismatch or conflicting mapping disables normalized string reuse for that log. @@ -26,6 +30,6 @@ Object fields align by key. Array elements align only when all corresponding arr ## Consequences -Repeated refreshes no longer rewrite aligned fixture values solely because the normalizer classifies them as volatile, and new volatile categories added to the normalizer automatically inherit the write-back behavior. Structural ambiguity remains conservative: unmatched records, conflicting string mappings, resized arrays, and strings containing both semantic and volatile changes use fresh values rather than risk reusing misaligned data. +Record and refresh no longer rewrite an unchanged unique message UUID solely because another event changed the surrounding record layout. Repeated refreshes also retain aligned fixture values that the normalizer classifies as volatile, and new volatile categories added to the normalizer automatically inherit that write-back behavior. Structural ambiguity remains conservative: unmatched records, conflicting string mappings, resized arrays, strings containing both semantic and volatile changes, and non-unique message fingerprints use fresh values rather than risk reusing misaligned data. -Focused unit coverage pins recursive object/array behavior, correlated IDs, ambiguous-layout fallback, conflicting mappings, fresh cwd aliases, volatile strings, and fresh semantic fields. Keyless refresh coverage proves approval UUIDs, cwd aliases, spill paths, and event-read volatility leave their committed fixtures byte-identical. +Focused unit coverage pins scenario-wide parent/child message correlation, unrelated event insertion, record write-back, new/changed/ambiguous messages, recursive object/array behavior, conflicting mappings, fresh cwd aliases, volatile strings, and fresh semantic fields. Keyless refresh coverage proves approval UUIDs, cwd aliases, spill paths, and event-read volatility leave their committed fixtures byte-identical. diff --git a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.zh.md b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.zh.md index 55302b509e..f3c21d14ae 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.zh.md +++ b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.zh.md @@ -8,9 +8,13 @@ Status: implemented ACP（Agent Client Protocol）快照比较会归一化生成的 UUID、cwd 别名、spill locator、嵌入的事件时间和省略字节数，但刷新写回会持久化本次生成的原始值。因此，即使比较契约将两份日志视为相等，一次行为未发生变化的刷新仍会用新的随机值或宿主特有的路径写法改写 fixture（测试前置数据）。 +消息身份所需的结构前提比记录对齐更弱：无关的日志事件可能破坏记录对齐，但继承而来的消息去除身份后的值在父级和子级日志之间仍保持不变。录制模式在替换现有 fixture 时也会从新生成的消息 UUID 开始。 + ## 决策 -刷新写回以 `normalizeSessionLog` 作为易变值的唯一判定依据。系统使用本次运行的 id、cwd 及全部 cwd 别名归一化原始收集记录，并使用 fixture header 上下文归一化 fixture 记录；字面量替换只影响要写入的原始值。现有记录完成对齐后，系统基于这些归一化记录，递归比较本次生成记录与现有记录的叶节点：归一化后等价的叶节点保留现有原始值，归一化后不同的叶节点则保留本次生成的语义值。 +在录制或刷新写入 fixture 前，套件会移除每条完整 surface 消息的顶层 `id` 并计算指纹，同时将所有父级/子级日志中的出现项分组。仅当一个指纹恰好对应一个本次生成的 ID 和一个现有 ID 时，才会复用现有 UUID，随后将该映射应用到每份本次生成的日志。具有相同 ID、重复出现的继承消息仍算作一个候选项；新增、发生变化、内容重复、格式错误和存在冲突的消息则保留本次生成的 ID。 + +刷新写回以 `normalizeSessionLog` 作为已对齐叶值的易变值判定依据。系统使用本次运行的 id、cwd 及全部 cwd 别名归一化原始收集记录，并使用 fixture header 上下文归一化 fixture 记录；字面量替换只影响要写入的原始值。现有记录完成对齐后，系统基于这些归一化记录，递归比较本次生成记录与现有记录的叶节点：归一化后等价的叶节点保留现有原始值，归一化后不同的叶节点则保留本次生成的语义值。 复用前必须确保完整逻辑记录布局对齐，现有的打包分片与插入标题等价情形除外。归一化后等价但发生变化的字符串在整份日志范围内形成双射：一个本次生成的字符串只映射到一个现有字符串，反向亦然，因此跨记录重复出现的 ID 仍保持关联。出现无法解释的记录不匹配或映射冲突时，该日志会停用规范化字符串复用。 @@ -26,6 +30,6 @@ ACP（Agent Client Protocol）快照比较会归一化生成的 UUID、cwd 别 ## 后果 -重复刷新不再仅仅因为规范化器将已对齐的 fixture 值归类为易变值，就改写这些值；以后加入规范化器的新易变值类别也会自动继承该写回行为。结构有歧义时仍采取保守策略：记录无法匹配、字符串映射冲突、数组尺寸发生变化，或字符串同时包含语义变化与易变变化时，均使用本次生成的值，避免冒险复用未对齐的数据。 +录制和刷新不再仅仅因为另一个事件改变了周边记录布局，就改写未变化且唯一的消息 UUID。重复刷新也会保留规范化器归类为易变值的已对齐 fixture 值；以后加入规范化器的新易变值类别也会自动继承该写回行为。结构有歧义时仍采取保守策略：记录无法匹配、字符串映射冲突、数组尺寸发生变化、字符串同时包含语义变化与易变变化，或消息指纹不唯一时，均使用本次生成的值，避免冒险复用未对齐的数据。 -聚焦的单元测试固定了递归处理对象与数组的行为、关联 ID、有歧义布局时的回退、映射冲突、本次运行的 cwd 别名、易变字符串以及本次生成的语义字段。无密钥刷新测试证明，审批 UUID、cwd 别名、spill 路径和事件读取中的易变值不会改变已提交 fixture 的任何字节。 +聚焦的单元测试固定了场景范围内的父级/子级消息关联、无关事件插入、录制写回、新增/发生变化/有歧义的消息、递归处理对象与数组的行为、映射冲突、本次运行的 cwd 别名、易变字符串以及本次生成的语义字段。无密钥刷新测试证明，审批 UUID、cwd 别名、spill 路径和事件读取中的易变值不会改变已提交 fixture 的任何字节。 diff --git a/packages/support/acp-snapshot/README.i18n.yaml b/packages/support/acp-snapshot/README.i18n.yaml index 363e0f268c..afabe9147a 100644 --- a/packages/support/acp-snapshot/README.i18n.yaml +++ b/packages/support/acp-snapshot/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/support/acp-snapshot/README.md -README.md: 948c33a91977f078d16842c285011bf8f83623bd -README.zh.md: fb86bd4e236be1c79f66dc46fbaac4d7dfbf9977 +README.md: e3752dfb522cd55776f3ef796acdc15037e5a761 +README.zh.md: 6ce531640b5311662e4b958177e4417c8878617f diff --git a/packages/support/acp-snapshot/README.md b/packages/support/acp-snapshot/README.md index 948c33a919..e3752dfb52 100644 --- a/packages/support/acp-snapshot/README.md +++ b/packages/support/acp-snapshot/README.md @@ -9,7 +9,7 @@ Four layers, importable separately: - **`launchAcpTestAgent` (launcher)** — boots a source agent under tsx or a built `lib` agent under plain Node from a supplied cwd, connects the SDK client over a raw-byte stdout tee, collects session updates and stderr, surfaces asynchronous spawn failures through startup, fails closed on unhandled permission requests, and owns graceful or signalled shutdown. Shutdown waits for process exit, inherited stdio closure, and ACP parser exhaustion before resolving or propagating a child error, so captures are complete and callers can remove owned paths after either outcome. When Windows accepts forced termination but publishes its exit marker asynchronously, shutdown gives that marker a bounded grace before treating fallback refusal as a second failure. Snapshot and ordinary e2e suites share this process boundary; a test supplies only agent paths, cwd, environment overrides, and any permission policy. - **`runScenario` (harness)** — drives ACP JSON-RPC stdio from a deterministic `input.json` script through the launcher, tees raw stdout for the expected-output and purity checks, and harvests every persisted raw JSONL session log (parent and subagent children, primary-first) after graceful stdin EOF. `AgentUnderTest` supplies absolute `binScript`, optional `libBinScript`, `configPath`, and `tsconfigPath` paths because the subprocess cwd is outside the repo; `workspaceParent` may move the generated child cwd from the platform temp directory when that grant is itself under test. Startup failures preserve captured agent stderr in the rejected diagnostic. - **Normalizers** — pure functions turning captured surfaces into stable text or portable fixtures: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs and every native/JavaScript filesystem spelling of the generated cwd → tokens, longest-first; cwd-rooted separators selected as canonical `/` or host-native; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept, the same cwd-path policy), `tokenizeSessionFixtureCwd` (the generated workspace and its filesystem aliases → `{{cwd}}`, authored temp paths unchanged), `scrubSystemPrompts` (prompt text → `{{system}}`), `scrubToolSchemas` (schema bulk → `{{tools}}`), and `scrubRequestHeaders` (all header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}` outside each pin, structure kept — [pinned-header Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)). -- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario expected-output and re-persisted-log comparisons, record/refresh fixture write-back, rejection of structured `UNKNOWN_TOOL` results, a tokenized pin per header class composed with independently shared `system-prompt.expected.md` and `tool-schemas.expected.json` sidecars, and a live uniformity guard. Its fixture guards reject orphan scenario dirs, missing files, multiple pins for one class, duplicate sidecar content, unscrubbed JSONL headers, and malformed pinning headers. Refresh evaluates fresh leaves with the harvested run's ids, cwd, and every cwd alias, then reuses normalized-equivalent leaves only when the complete logical-record layout aligns and volatile string replacements form a bijection; ambiguous logs keep fresh strings, and fresh semantic values remain authoritative. It also expands packed timing envelopes before aligning event times, so switching between packed and unpacked layouts cannot shift later records. A newly inserted `session/title` receives its preceding event's time so feature-driven insertions do not churn the remainder of a fixture. Each scenario directory's `session.jsonl` plus contiguous `session..jsonl` siblings are the ordered primary/child inventory; the scenario table does not duplicate their count. Must be called at vitest collection time. +- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario expected-output and re-persisted-log comparisons, record/refresh fixture write-back, rejection of structured `UNKNOWN_TOOL` results, a tokenized pin per header class composed with independently shared `system-prompt.expected.md` and `tool-schemas.expected.json` sidecars, and a live uniformity guard. Its fixture guards reject orphan scenario dirs, missing files, multiple pins for one class, duplicate sidecar content, unscrubbed JSONL headers, and malformed pinning headers. Before record or refresh writes fixtures, an unchanged complete message retains its committed UUID when its identity-free value resolves to exactly one fresh ID and one existing ID across the scenario's parent/child logs; new, changed, and ambiguous messages keep fresh UUIDs. Refresh evaluates fresh leaves with the harvested run's ids, cwd, and every cwd alias, then reuses normalized-equivalent leaves only when the complete logical-record layout aligns and volatile string replacements form a bijection; ambiguous logs keep fresh strings, and fresh semantic values remain authoritative. It also expands packed timing envelopes before aligning event times, so switching between packed and unpacked layouts cannot shift later records. A newly inserted `session/title` receives its preceding event's time so feature-driven insertions do not churn the remainder of a fixture. Each scenario directory's `session.jsonl` plus contiguous `session..jsonl` siblings are the ordered primary/child inventory; the scenario table does not duplicate their count. Must be called at vitest collection time. Committed session fixtures use canonical packed rows. An in-flight branch that merges this contract runs the [temporary repository migrator](../../../scripts/migrate-packed-session-fixtures.ts) with `pnpm run migrate:packed-session-fixtures`; its [removal proposal](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md) owns deletion after affected branches converge. diff --git a/packages/support/acp-snapshot/README.zh.md b/packages/support/acp-snapshot/README.zh.md index fb86bd4e23..6ce531640b 100644 --- a/packages/support/acp-snapshot/README.zh.md +++ b/packages/support/acp-snapshot/README.zh.md @@ -9,7 +9,7 @@ ACP 快照套件工具包：无密钥快照层（`pnpm run test:snapshot`，见[ - **`launchAcpTestAgent`（启动器）**：从指定 cwd 在 tsx 下启动源 agent，或在普通 Node 下启动已构建 `lib` agent；通过原始字节 stdout tee 连接 SDK 客户端，收集会话更新和 stderr，在启动过程中公开异步 spawn 失败，对未处理权限请求快速失败，并负责优雅或带信号关闭。关闭会等待进程退出、继承 stdio 关闭和 ACP parser 耗尽，然后才解析或传播子级错误，使捕获内容完整，且调用方可在任一结果后移除自有路径。当 Windows 接受强制终止但异步发布退出标记时，关闭会给该标记有界宽限，然后才将回退拒绝视为第二次失败。快照和普通 e2e 套件共享该进程边界；测试只需提供 agent 路径、cwd、环境覆盖和任何权限策略。 - **`runScenario`（harness）**：通过启动器从确定性 `input.json` 脚本驱动 ACP JSON-RPC stdio，将原始 stdout tee 给预期输出和纯度检查，并在优雅 stdin EOF 后收集每个持久化原始 JSONL 会话日志（父级和 subagent 子级，主级优先）。`AgentUnderTest` 提供绝对 `binScript`、可选 `libBinScript`、`configPath` 和 `tsconfigPath` 路径，因为子进程 cwd 位于仓库外。当生成子级 cwd 自身位于待测授权中时，`workspaceParent` 可以将它从平台临时目录移出。启动失败会在拒绝诊断中保留已捕获 agent stderr。 - **规范化器**：将已捕获接口转换为稳定文本或可移植 fixture 的纯函数：`normalizeStdout`（JSON-RPC id → 首次出现序列；UUID 以及生成 cwd 的每个原生/JavaScript 文件系统写法 → token，按最长优先；根据 cwd 的分隔符选择规范 `/` 或宿主原生形式；同时作为 stdout 纯度检查）、`normalizeSessionLog`（时间归零、保留 `seq`、使用同一 cwd 路径策略）、`tokenizeSessionFixtureCwd`（生成的 workspace 及其文件系统别名 → `{{cwd}}`，手工编写的临时路径保持不变）、`scrubSystemPrompts`（提示词文本 → `{{system}}`）、`scrubToolSchemas`（schema bulk → `{{tools}}`）和 `scrubRequestHeaders`（每个 pin 之外的所有 header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}`，保留结构；见[header 固定 Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)）。 -- **`defineAcpSnapshotSuite`（工厂）**：为场景表注册完整 describe/it 树：每场景预期输出与重新持久化日志比较、录制/刷新 fixture 回写、拒绝结构化 `UNKNOWN_TOOL` 结果、每个 header 类别一个 token 化 pin（由可独立共享的 `system-prompt.expected.md` 和 `tool-schemas.expected.json` sidecar 组合而成），以及实时一致性保护。其 fixture 保护会拒绝遗留场景目录、缺失文件、一个类别包含多个 pin、重复的 sidecar 内容、未擦除的 JSONL header，以及格式错误的 pin header。刷新会使用收集所得本次运行的 id、cwd 及全部 cwd 别名评估本次生成的叶值；只有完整逻辑记录布局对齐且易变字符串替换形成双射时，才会复用规范化后等价的叶值；有歧义的日志保留本次生成的字符串，而本次生成的语义值仍为权威数据。它还会在对齐事件时间前展开打包时序 envelope，因此切换打包/非打包布局无法移动后续记录。新插入的 `session/title` 使用前一个事件的时间，因此功能驱动的插入不会扰动 fixture 余下部分。每个场景目录的 `session.jsonl` 和连续 `session..jsonl` 同级文件是有序主级/子级清单；场景表不重复其数量。必须在 vitest 收集时调用。 +- **`defineAcpSnapshotSuite`（工厂）**：为场景表注册完整 describe/it 树：每场景预期输出与重新持久化日志比较、录制/刷新 fixture 回写、拒绝结构化 `UNKNOWN_TOOL` 结果、每个 header 类别一个 token 化 pin（由可独立共享的 `system-prompt.expected.md` 和 `tool-schemas.expected.json` sidecar 组合而成），以及实时一致性保护。其 fixture 保护会拒绝遗留场景目录、缺失文件、一个类别包含多个 pin、重复的 sidecar 内容、未擦除的 JSONL header，以及格式错误的 pin header。在录制或刷新写入 fixture 前，如果一条未变化的完整消息去除身份后的值在场景的父级/子级日志中恰好对应一个本次生成的 ID 和一个现有 ID，它就会保留已提交的 UUID；新增、发生变化和有歧义的消息则保留本次生成的 UUID。刷新会使用收集所得本次运行的 id、cwd 及全部 cwd 别名评估本次生成的叶值；只有完整逻辑记录布局对齐且易变字符串替换形成双射时，才会复用规范化后等价的叶值；有歧义的日志保留本次生成的字符串，而本次生成的语义值仍为权威数据。它还会在对齐事件时间前展开打包时序 envelope，因此切换打包/非打包布局无法移动后续记录。新插入的 `session/title` 使用前一个事件的时间，因此功能驱动的插入不会扰动 fixture 余下部分。每个场景目录的 `session.jsonl` 和连续 `session..jsonl` 同级文件是有序主级/子级清单；场景表不重复其数量。必须在 vitest 收集时调用。 签入仓库的会话 fixture 使用规范打包行。合并此契约的在途分支通过 `pnpm run migrate:packed-session-fixtures` 运行[临时仓库迁移器](../../../scripts/migrate-packed-session-fixtures.ts)；待受影响分支收敛后，由其[移除提案](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md)负责删除该迁移器。 diff --git a/packages/support/acp-snapshot/src/suite.ts b/packages/support/acp-snapshot/src/suite.ts index 1a8a49dac5..8a99bc0813 100644 --- a/packages/support/acp-snapshot/src/suite.ts +++ b/packages/support/acp-snapshot/src/suite.ts @@ -48,6 +48,9 @@ const TOOLS_TOKEN = '{{tools}}' const PACKED_CHUNK_ROW_TYPES = new Set(['text-chunks', 'reasoning-chunks', 'tool-call-chunks']) +/** Canonical UUID spelling minted for ordinary message identities. */ +const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i + /** A snapshot scenario and how its fixtures are produced. */ export interface Scenario { name: string @@ -480,9 +483,9 @@ export function headerChangeCount(rawLog: string): number { .length } -/** A literal string replacement used to carry an existing fixture's volatile value into a refreshed log. */ +/** A literal string replacement used to carry an existing fixture value into fresh write-back. */ export interface FixtureReplacement { - /** The fresh replay-run value to replace. */ + /** The fresh run's value to replace. */ from: string /** The existing fixture value to keep. */ to: string @@ -494,6 +497,82 @@ function parseJsonlRecords(text: string): Record[] { .map(line => JSON.parse(line) as Record) } +/** Return the complete identified message carried by one surface event. */ +function eventMessage(record: Record): Record | undefined { + const data = record.data + if (!isRecord(data)) return undefined + const message = record.type === 'user/message' + ? data + : record.type === 'assistant/message' || record.type === 'tool/result' || record.type === 'steering/message' + ? data.message + : undefined + if ( + !isRecord(message) + || typeof message.id !== 'string' + || !UUID_RE.test(message.id) + || typeof message.role !== 'string' + || !Array.isArray(message.content) + || !isRecord(message.source) + ) return undefined + return message +} + +/** Serialize parsed JSON by value rather than insertion order. */ +function canonicalJson(value: unknown): string { + if (Array.isArray(value)) return `[${value.map(canonicalJson).join(',')}]` + if (isRecord(value)) { + return `{${Object.keys(value).sort().map(key => `${JSON.stringify(key)}:${canonicalJson(value[key])}`).join(',')}}` + } + return JSON.stringify(value) +} + +/** Index each unambiguous identity-free message value by its sole message id. */ +function uniqueMessageIds(logs: readonly string[]): Map { + const fingerprintsById = new Map() + for (const log of logs) { + for (const record of parseJsonlRecords(log)) { + const message = eventMessage(record) + if (message === undefined) continue + const { id, ...withoutId } = message + const messageId = id as string + const fingerprint = canonicalJson(withoutId) + if (!fingerprintsById.has(messageId)) fingerprintsById.set(messageId, fingerprint) + else if (fingerprintsById.get(messageId) !== fingerprint) fingerprintsById.set(messageId, undefined) + } + } + + const idsByFingerprint = new Map() + for (const [id, fingerprint] of fingerprintsById) { + if (fingerprint === undefined) continue + if (!idsByFingerprint.has(fingerprint)) idsByFingerprint.set(fingerprint, id) + else idsByFingerprint.set(fingerprint, undefined) + } + return idsByFingerprint +} + +/** + * Match unchanged complete messages across a scenario's fresh and existing logs. + * New, changed, repeated, or otherwise ambiguous messages keep their fresh ids. + */ +function fixtureMessageIdReplacements(logs: HarvestedLog[], fixtures: string[]): FixtureReplacement[] { + const freshIds = uniqueMessageIds(logs.map(log => log.content)) + const existingIds = uniqueMessageIds(fixtures) + const replacements: FixtureReplacement[] = [] + for (const [fingerprint, fresh] of freshIds) { + const existing = existingIds.get(fingerprint) + if (fresh === undefined || existing === undefined || fresh === existing) continue + replacements.push({ from: fresh, to: existing }) + } + return replacements +} + +/** Apply literal fixture replacements without changing any other fresh value. */ +function applyFixtureReplacements(content: string, replacements: readonly FixtureReplacement[]): string { + let stable = content + for (const { from, to } of replacements) stable = stable.split(from).join(to) + return stable +} + /** One packed row's member times, or `undefined` for an ordinary record. */ function packedTimes(record: Record): number[] | undefined { if (!PACKED_CHUNK_ROW_TYPES.has(record.type as string)) return undefined @@ -539,14 +618,15 @@ export function unknownToolCallIds(rawLog: string): string[] { } /** - * Build the cross-log id/cwd/spill-path replacements used by refresh write-back. + * Build refresh write-back replacements: scenario-wide unchanged message ids, + * plus per-log session ids, cwd values, and spill paths. * * @param logs The freshly harvested logs, in fixture order. * @param fixtures The existing fixture contents, in matching order. - * @returns Literal replacements from fresh volatile values to the fixture's old values. + * @returns Literal replacements from fresh values to the fixture's existing values. */ export function refreshFixtureReplacements(logs: HarvestedLog[], fixtures: string[]): FixtureReplacement[] { - const replacements: FixtureReplacement[] = [] + const replacements = fixtureMessageIdReplacements(logs, fixtures) for (let i = 0; i < logs.length; i++) { const fresh = parseJsonlRecords((logs[i] as HarvestedLog).content)[0] const existing = parseJsonlRecords(fixtures[i] ?? '')[0] @@ -823,8 +903,7 @@ export function stabilizeRefreshLog( freshContext: NormalizeContext, ): string { const freshRecords = parseJsonlRecords(fresh) - let stable = fresh - for (const { from, to } of replacements) stable = stable.split(from).join(to) + const stable = applyFixtureReplacements(fresh, replacements) const existingRecords = logicalRecords(parseJsonlRecords(existing)) const records = parseJsonlRecords(stable) const existingContext = fixtureContext(existing) @@ -1016,10 +1095,6 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { const portableFixture = scenario.workspaceParent === undefined ? tokenizeSessionFixtureCwd : (log: string): string => log - const existingFixtures = REFRESHING - ? await Promise.all(fixtureFiles.map(file => readFile(join(dir, file), 'utf8'))) - : [] - const replacements = REFRESHING ? refreshFixtureReplacements(result.sessionLogs, existingFixtures) : [] const writesSessionFixtures = (RECORDING && scenario.recorded && scenario.hasModelTurn) || (REFRESHING && comparesLog) if (writesSessionFixtures) { @@ -1032,14 +1107,25 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { 'session.jsonl', ...Array.from({ length: result.sessionLogs.length - 1 }, (_, i) => `session.${i + 1}.jsonl`), ] + const existingFixtures = await Promise.all(outputFixtureFiles.map(async (file) => { + const path = join(dir, file) + return existsSync(path) ? readFile(path, 'utf8') : '' + })) + const replacements = REFRESHING + ? refreshFixtureReplacements(result.sessionLogs, existingFixtures) + : fixtureMessageIdReplacements(result.sessionLogs, existingFixtures) const primary = (result.sessionLogs[0] as HarvestedLog).content await writeFile(join(dir, outputFixtureFiles[0] as string), scrub(portableFixture( - REFRESHING ? stabilizeRefreshLog(primary, existingFixtures[0] as string, replacements, ctx) : primary, + REFRESHING + ? stabilizeRefreshLog(primary, existingFixtures[0] as string, replacements, ctx) + : applyFixtureReplacements(primary, replacements), ))) for (let i = 1; i < result.sessionLogs.length; i++) { const child = (result.sessionLogs[i] as HarvestedLog).content await writeFile(join(dir, outputFixtureFiles[i] as string), scrub(portableFixture( - REFRESHING ? stabilizeRefreshLog(child, existingFixtures[i] as string, replacements, ctx) : child, + REFRESHING + ? stabilizeRefreshLog(child, existingFixtures[i] as string, replacements, ctx) + : applyFixtureReplacements(child, replacements), ))) } if (RECORDING) { diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/behavior.json b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/behavior.json index d98afb4865..971006c139 100644 --- a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/behavior.json +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/behavior.json @@ -3,11 +3,13 @@ "logs": [ { "file": "b/parent/session.jsonl", "lines": [ { "type": "session", "id": "{{SID}}", "createdAt": 700, "cwd": "{{CWD}}", "delegationDepth": 0 }, - { "type": "request/header", "seq": 0, "time": 3, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } } + { "type": "request/header", "seq": 0, "time": 3, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, + { "type": "user/message", "seq": 1, "time": 5, "data": { "role": "user", "content": [{ "type": "text", "text": "same inherited message" }], "source": { "kind": "user" }, "id": "11111111-1111-4111-8111-111111111111" }, "surfaceOp": "append" } ]}, { "file": "b/child/session.jsonl", "lines": [ { "type": "session", "id": "abababab-cdcd-4efe-8ada-badabadabada", "createdAt": 800, "cwd": "{{CWD}}", "parentSession": "{{SID}}", "delegationDepth": 1 }, - { "type": "request/header", "seq": 0, "time": 2, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } } + { "type": "request/header", "seq": 0, "time": 2, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, + { "type": "user/message", "seq": 1, "time": 5, "data": { "role": "user", "content": [{ "type": "text", "text": "same inherited message" }], "source": { "kind": "user" }, "id": "11111111-1111-4111-8111-111111111111" }, "surfaceOp": "append" } ]} ] } diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl index 4fa81014ae..384b3954cf 100644 --- a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl @@ -1,2 +1,3 @@ {"type":"session","id":"abababab-cdcd-4efe-8ada-badabadabada","createdAt":800,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW","parentSession":"f6fa7fcf-dd9c-4b39-8815-b25ddcebfd88","delegationDepth":1} {"type":"request/header","seq":0,"time":2,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"user/message","seq":1,"time":5,"data":{"role":"user","content":[{"type":"text","text":"same inherited message"}],"source":{"kind":"user"},"id":"22222222-2222-4222-8222-222222222222"},"surfaceOp":"append"} diff --git a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl index e972a78d8e..0ffe7f9f5f 100644 --- a/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl +++ b/packages/support/acp-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl @@ -1,2 +1,3 @@ {"type":"session","id":"f6fa7fcf-dd9c-4b39-8815-b25ddcebfd88","createdAt":700,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW","delegationDepth":0} {"type":"request/header","seq":0,"time":3,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"user/message","seq":1,"time":5,"data":{"role":"user","content":[{"type":"text","text":"same inherited message"}],"source":{"kind":"user"},"id":"22222222-2222-4222-8222-222222222222"},"surfaceOp":"append"} diff --git a/packages/support/acp-snapshot/tests/suite.spec.ts b/packages/support/acp-snapshot/tests/suite.spec.ts index da069905ec..2cd665f540 100644 --- a/packages/support/acp-snapshot/tests/suite.spec.ts +++ b/packages/support/acp-snapshot/tests/suite.spec.ts @@ -193,6 +193,18 @@ describe('defineAcpSnapshotSuite: record inventory write-back', () => { expect(fixture).toContain('"cwd":"{{cwd}}"') expect(() => readFileSync(join(recordDir, 'rec-child', 'session.2.jsonl'), 'utf8')).toThrow() }) + + it('retains an unchanged message id across the recorded parent and child fixtures', () => { + const existingMessageId = '22222222-2222-4222-8222-222222222222' + const freshMessageId = '11111111-1111-4111-8111-111111111111' + const fixtures = ['session.jsonl', 'session.1.jsonl'] + .map(file => readFileSync(join(recordDir, 'rec-child', file), 'utf8')) + + for (const fixture of fixtures) { + expect(fixture).toContain(`"id":"${existingMessageId}"`) + expect(fixture).not.toContain(freshMessageId) + } + }) }) describe('defineAcpSnapshotSuite: registration contract', () => { @@ -663,6 +675,78 @@ describe('refreshFixtureReplacements', () => { { from: freshBash, to: oldBash }, ]) }) + + it('maps one inherited message id across parent and child logs', () => { + const freshMessageId = '11111111-1111-4111-8111-111111111111' + const existingMessageId = '22222222-2222-4222-8222-222222222222' + const content = [{ type: 'text', text: 'inherited' }] + const log = (sessionId: string, messageId: string): string => [ + JSON.stringify({ type: 'session', id: sessionId, cwd: '/same' }), + JSON.stringify({ + type: 'user/message', + data: { role: 'user', content, source: { kind: 'user' }, id: messageId }, + }), + '', + ].join('\n') + const harvested = (content: string): HarvestedLog => ({ id: 'diagnostic', createdAt: 1, content }) + + const replacements = refreshFixtureReplacements( + [harvested(log('fresh-parent', freshMessageId)), harvested(log('fresh-child', freshMessageId))], + [log('old-parent', existingMessageId), log('old-child', existingMessageId)], + ) + + expect(replacements.filter(replacement => replacement.from === freshMessageId)).toEqual([ + { from: freshMessageId, to: existingMessageId }, + ]) + }) + + it('keeps fresh ids for new, changed, and ambiguous messages', () => { + const ids = { + new: '11111111-1111-4111-8111-111111111111', + changed: '22222222-2222-4222-8222-222222222222', + ambiguousA: '33333333-3333-4333-8333-333333333333', + ambiguousB: '44444444-4444-4444-8444-444444444444', + oldChanged: '55555555-5555-4555-8555-555555555555', + oldAmbiguous: '66666666-6666-4666-8666-666666666666', + stable: '77777777-7777-4777-8777-777777777777', + } as const + const message = (id: string, text: string): Record => ({ + type: 'user/message', + data: { role: 'user', content: [{ type: 'text', text }], source: { kind: 'user' }, id }, + }) + const log = (messages: Record[]): string => [ + JSON.stringify({ type: 'session', id: 'same', cwd: '/same' }), + ...messages.map(record => JSON.stringify(record)), + '', + ].join('\n') + const fresh = log([ + message(ids.new, 'new'), + message(ids.changed, 'changed'), + message(ids.changed, 'changed again'), + message(ids.ambiguousA, 'duplicate'), + message(ids.ambiguousB, 'duplicate'), + message(ids.stable, 'stable'), + ]) + const existing = log([ + message(ids.oldChanged, 'before'), + message(ids.oldAmbiguous, 'duplicate'), + message(ids.stable, 'stable'), + ]) + + const replacements = refreshFixtureReplacements( + [{ id: 'diagnostic', createdAt: 1, content: fresh }], + [existing], + ) + + const replacedIds = replacements.map(replacement => replacement.from) + for (const id of [ + ids.new, + ids.changed, + ids.ambiguousA, + ids.ambiguousB, + ids.stable, + ]) expect(replacedIds).not.toContain(id) + }) }) describe('stabilizeRefreshLog', () => { @@ -765,6 +849,50 @@ describe('stabilizeRefreshLog', () => { ].join('\n')) }) + it('retains unchanged message ids across an unrelated inserted event', () => { + const freshUserId = '11111111-1111-4111-8111-111111111111' + const existingUserId = '22222222-2222-4222-8222-222222222222' + const freshAssistantId = '33333333-3333-4333-8333-333333333333' + const existingAssistantId = '44444444-4444-4444-8444-444444444444' + const user = (id: string): Record => ({ + type: 'user/message', + data: { role: 'user', content: [{ type: 'text', text: 'same user' }], source: { kind: 'user' }, id }, + }) + const assistant = (id: string): Record => ({ + type: 'assistant/message', + data: { + turn: 1, + step: 1, + message: { + role: 'assistant', + content: [{ type: 'text', text: 'same assistant' }], + source: { kind: 'model', provider: 'fake', model: 'fake' }, + id, + }, + }, + }) + const lines = (records: Record[]): string => [ + JSON.stringify({ type: 'session', id: 'same', createdAt: 1, cwd: '/same' }), + ...records.map(record => JSON.stringify(record)), + '', + ].join('\n') + const fresh = lines([ + user(freshUserId), + { type: 'session/inherited', data: {} }, + assistant(freshAssistantId), + ]) + const existing = lines([user(existingUserId), assistant(existingAssistantId)]) + const replacements = refreshFixtureReplacements( + [{ id: 'diagnostic', createdAt: 1, content: fresh }], + [existing], + ) + const output = stabilize(fresh, existing, replacements).trim().split('\n') + .map(line => JSON.parse(line) as Record) + + expect((output[1]?.data as { id: string }).id).toBe(existingUserId) + expect(((output[3]?.data as { message: { id: string } }).message).id).toBe(existingAssistantId) + }) + it('keeps volatile fixture fields while preserving fresh meaningful payloads', () => { const fresh = [ '{"type":"session","id":"new-child","createdAt":200,"cwd":"/new","parentSession":"new-parent","seedLength":1}', From 466a2f12c3196c532ea6fde0918805daf67d87a8 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 17:51:07 +0800 Subject: [PATCH 2/4] fix(snapshot): stabilize all recorder message ids --- ...table-snapshot-refresh-volatiles.i18n.yaml | 4 +- ...07-27-stable-snapshot-refresh-volatiles.md | 4 +- ...27-stable-snapshot-refresh-volatiles.zh.md | 4 +- apps/web/tests/scaffold.ts | 9 ++-- examples/jsonrpc-agent/tests/sdk.snapshot.ts | 15 ++++-- examples/tui-agent/tests/tui.snapshot.ts | 25 +++++----- .../support/acp-snapshot/README.i18n.yaml | 4 +- packages/support/acp-snapshot/README.md | 4 +- packages/support/acp-snapshot/README.zh.md | 4 +- packages/support/acp-snapshot/src/index.ts | 1 + packages/support/acp-snapshot/src/suite.ts | 49 ++++++++++++------- .../support/acp-snapshot/tests/suite.spec.ts | 37 +++++++++++++- 12 files changed, 110 insertions(+), 50 deletions(-) diff --git a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.i18n.yaml index 28e3accc20..3a474e377b 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.i18n.yaml +++ b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md -2026-07-27-stable-snapshot-refresh-volatiles.md: 5b513ea026008fc0c4ae9a8045408c534c050bec -2026-07-27-stable-snapshot-refresh-volatiles.zh.md: f3c21d14ae235179ca15ec30d964e02877aa86e9 +2026-07-27-stable-snapshot-refresh-volatiles.md: cd806c929ba956098f532d19159ff2dc3e782325 +2026-07-27-stable-snapshot-refresh-volatiles.zh.md: 3144bcb45aa8524e29fae3d07479cb917987ea79 diff --git a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md index 5b513ea026..cd806c929b 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md +++ b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md @@ -12,7 +12,7 @@ Message identity needs a weaker structural precondition than aligned records: an ## Decision -Before record or refresh writes fixtures, the suite fingerprints every complete surface message with its top-level `id` removed and groups occurrences across all parent/child logs. It reuses an existing UUID only when one fingerprint resolves to exactly one fresh ID and one existing ID, then applies that mapping to every fresh log. Repeated inherited occurrences with the same ID remain one candidate, while new, changed, duplicate-content, malformed, and conflicting messages keep their fresh IDs. +Before record or refresh writes session fixtures, the shared snapshot support fingerprints every complete surface message with its top-level `id` removed and groups occurrences across all parent/child logs. It reuses an existing UUID only when one fingerprint resolves to exactly one fresh ID and one existing ID, then applies that mapping to every fresh log. Repeated inherited occurrences with the same ID remain one candidate, while new, changed, duplicate-content, malformed, and conflicting messages keep their fresh IDs. ACP, JSON-RPC, TUI, and web recorders pass fixture-ready logs through the same helper before writing. Refresh write-back uses `normalizeSessionLog` as its volatile-value authority for aligned leaves. It normalizes the original harvested records with the fresh run's ids, cwd, and every cwd alias, while normalizing fixture records with the fixture header context; literal replacements affect only the raw values being written. After existing record alignment, it recursively compares fresh and existing leaves through those normalized records: normalized-equivalent leaves retain the existing raw value, while normalized-distinct leaves retain the fresh semantic value. @@ -30,6 +30,6 @@ Object fields align by key. Array elements align only when all corresponding arr ## Consequences -Record and refresh no longer rewrite an unchanged unique message UUID solely because another event changed the surrounding record layout. Repeated refreshes also retain aligned fixture values that the normalizer classifies as volatile, and new volatile categories added to the normalizer automatically inherit that write-back behavior. Structural ambiguity remains conservative: unmatched records, conflicting string mappings, resized arrays, strings containing both semantic and volatile changes, and non-unique message fingerprints use fresh values rather than risk reusing misaligned data. +Record and refresh no longer rewrite an unchanged unique message UUID solely because another event changed the surrounding record layout, regardless of whether ACP, JSON-RPC, TUI, or web owns the recording. Repeated refreshes also retain aligned fixture values that the normalizer classifies as volatile, and new volatile categories added to the normalizer automatically inherit that write-back behavior. Structural ambiguity remains conservative: unmatched records, conflicting string mappings, resized arrays, strings containing both semantic and volatile changes, and non-unique message fingerprints use fresh values rather than risk reusing misaligned data. Focused unit coverage pins scenario-wide parent/child message correlation, unrelated event insertion, record write-back, new/changed/ambiguous messages, recursive object/array behavior, conflicting mappings, fresh cwd aliases, volatile strings, and fresh semantic fields. Keyless refresh coverage proves approval UUIDs, cwd aliases, spill paths, and event-read volatility leave their committed fixtures byte-identical. diff --git a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.zh.md b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.zh.md index f3c21d14ae..3144bcb45a 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.zh.md +++ b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.zh.md @@ -12,7 +12,7 @@ ACP（Agent Client Protocol）快照比较会归一化生成的 UUID、cwd 别 ## 决策 -在录制或刷新写入 fixture 前，套件会移除每条完整 surface 消息的顶层 `id` 并计算指纹，同时将所有父级/子级日志中的出现项分组。仅当一个指纹恰好对应一个本次生成的 ID 和一个现有 ID 时，才会复用现有 UUID，随后将该映射应用到每份本次生成的日志。具有相同 ID、重复出现的继承消息仍算作一个候选项；新增、发生变化、内容重复、格式错误和存在冲突的消息则保留本次生成的 ID。 +在录制或刷新写入会话 fixture 前，共享快照支持层会移除每条完整 surface 消息的顶层 `id` 并计算指纹，同时将所有父级/子级日志中的出现项分组。仅当一个指纹恰好对应一个本次生成的 ID 和一个现有 ID 时，才会复用现有 UUID，随后将该映射应用到每份本次生成的日志。具有相同 ID、重复出现的继承消息仍算作一个候选项；新增、发生变化、内容重复、格式错误和存在冲突的消息则保留本次生成的 ID。ACP、JSON-RPC、TUI 和 web 录制器都会先让可写入 fixture 的日志经过同一个辅助函数，再执行写入。 刷新写回以 `normalizeSessionLog` 作为已对齐叶值的易变值判定依据。系统使用本次运行的 id、cwd 及全部 cwd 别名归一化原始收集记录，并使用 fixture header 上下文归一化 fixture 记录；字面量替换只影响要写入的原始值。现有记录完成对齐后，系统基于这些归一化记录，递归比较本次生成记录与现有记录的叶节点：归一化后等价的叶节点保留现有原始值，归一化后不同的叶节点则保留本次生成的语义值。 @@ -30,6 +30,6 @@ ACP（Agent Client Protocol）快照比较会归一化生成的 UUID、cwd 别 ## 后果 -录制和刷新不再仅仅因为另一个事件改变了周边记录布局，就改写未变化且唯一的消息 UUID。重复刷新也会保留规范化器归类为易变值的已对齐 fixture 值；以后加入规范化器的新易变值类别也会自动继承该写回行为。结构有歧义时仍采取保守策略：记录无法匹配、字符串映射冲突、数组尺寸发生变化、字符串同时包含语义变化与易变变化，或消息指纹不唯一时，均使用本次生成的值，避免冒险复用未对齐的数据。 +录制和刷新不再仅仅因为另一个事件改变了周边记录布局，就改写未变化且唯一的消息 UUID，无论该录制由 ACP、JSON-RPC、TUI 还是 web 负责。重复刷新也会保留规范化器归类为易变值的已对齐 fixture 值；以后加入规范化器的新易变值类别也会自动继承该写回行为。结构有歧义时仍采取保守策略：记录无法匹配、字符串映射冲突、数组尺寸发生变化、字符串同时包含语义变化与易变变化，或消息指纹不唯一时，均使用本次生成的值，避免冒险复用未对齐的数据。 聚焦的单元测试固定了场景范围内的父级/子级消息关联、无关事件插入、录制写回、新增/发生变化/有歧义的消息、递归处理对象与数组的行为、映射冲突、本次运行的 cwd 别名、易变字符串以及本次生成的语义字段。无密钥刷新测试证明，审批 UUID、cwd 别名、spill 路径和事件读取中的易变值不会改变已提交 fixture 的任何字节。 diff --git a/apps/web/tests/scaffold.ts b/apps/web/tests/scaffold.ts index 753cdc2953..d11604c075 100644 --- a/apps/web/tests/scaffold.ts +++ b/apps/web/tests/scaffold.ts @@ -27,7 +27,7 @@ import { expect } from 'vitest' import { Context } from 'cordis' import Loader from '@cordisjs/plugin-loader' import Include, { type PatchOptions } from '@cordisjs/plugin-include' -import { scrubRequestHeaders } from '@deepseek-ai/dsh-acp-snapshot' +import { scrubRequestHeaders, stabilizeFixtureMessageIds } from '@deepseek-ai/dsh-acp-snapshot' import { assertEntriesLoaded } from '@deepseek-ai/dsh-app-boot' import type { ReplayHandle } from '@deepseek-ai/dsh-llm-replay' import { installLlmReplay, parseSessionLog } from '@deepseek-ai/dsh-llm-replay' @@ -302,11 +302,14 @@ function rawSessionLog(session: Session): string { export async function recordFixture(scaffold: WebScaffold, sessionId: SessionId, fixturePath: string): Promise { const agent = scaffold.ctx.agents.get(sessionId) if (agent === undefined) throw new Error(`record harvest: no live agent for ${sessionId}`) - const tokenized = scrubRequestHeaders(rawSessionLog(agent.session)) + const fresh = scrubRequestHeaders(rawSessionLog(agent.session)) .split(sessionId).join('{{sessionId}}') .split(scaffold.workspaceCwd).join('{{cwd}}') .replace(/"rpcId":"[^"]+"/g, '"rpcId":"{{rpcId}}"') - await writeFile(fixturePath, tokenized) + const existing = existsSync(fixturePath) ? await readFile(fixturePath, 'utf8') : '' + const stable = stabilizeFixtureMessageIds([fresh], [existing])[0] + if (stable === undefined) throw new Error('record harvest: no stabilized fixture') + await writeFile(fixturePath, stable) } /** diff --git a/examples/jsonrpc-agent/tests/sdk.snapshot.ts b/examples/jsonrpc-agent/tests/sdk.snapshot.ts index c54812e3e5..3f0edac743 100644 --- a/examples/jsonrpc-agent/tests/sdk.snapshot.ts +++ b/examples/jsonrpc-agent/tests/sdk.snapshot.ts @@ -9,6 +9,7 @@ * fixtures and rewrites expected outputs. */ +import { existsSync } from 'node:fs' import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { delimiter, join } from 'node:path' @@ -19,6 +20,7 @@ import { normalizeStdout, refreshFixtureReplacements, scrubRequestHeaders, + stabilizeFixtureMessageIds, stabilizeRefreshLog, tokenizeSessionFixtureCwd, type HarvestedLog, @@ -230,20 +232,25 @@ describe('TypeScript SDK snapshots over the jsonrpc runtime', () => { const { result, notifications, logs, cwd } = await runScenario(scenario) const ordered = orderLogs(logs, scenario) const actualContext = contextOf(ordered, cwd) + const files = fixtureFiles(scenario) if (recording) { // Fixtures carry tokenized request headers; llm-replay reads only // assistant output and tool traffic, so scrubbing keeps prompts and // schemas out of the corpus without affecting replay. await mkdir(scenarioDir, { recursive: true }) - await Promise.all(ordered.map(async (log, index) => { - const file = fixtureFiles(scenario)[index] + const existing = await Promise.all(files.map(async file => existsSync(file) ? readFile(file, 'utf8') : '')) + const fixtures = stabilizeFixtureMessageIds( + ordered.map(log => scrubRequestHeaders(tokenizeSessionFixtureCwd(log.content))), + existing, + ) + await Promise.all(fixtures.map(async (fixture, index) => { + const file = files[index] if (file === undefined) throw new Error(`no fixture path for persisted log ${index}`) - await writeFile(file, scrubRequestHeaders(tokenizeSessionFixtureCwd(log.content))) + await writeFile(file, fixture) })) } - const files = fixtureFiles(scenario) let expectedContents = await Promise.all(files.map(file => readFile(file, 'utf8'))) if (refreshing) { diff --git a/examples/tui-agent/tests/tui.snapshot.ts b/examples/tui-agent/tests/tui.snapshot.ts index 95b960e3b6..934621b037 100644 --- a/examples/tui-agent/tests/tui.snapshot.ts +++ b/examples/tui-agent/tests/tui.snapshot.ts @@ -1,10 +1,15 @@ +import { existsSync } from 'node:fs' import { cp, mkdir, mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { basename, dirname, isAbsolute, join, relative, sep } from 'node:path' import { fileURLToPath } from 'node:url' import { afterAll, describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' -import { scrubRequestHeaders, tokenizeSessionFixtureCwd } from '@deepseek-ai/dsh-acp-snapshot' +import { + scrubRequestHeaders, + stabilizeFixtureMessageIds, + tokenizeSessionFixtureCwd, +} from '@deepseek-ai/dsh-acp-snapshot' import type { Agent } from '@deepseek-ai/dsh-agent' import * as AgentCore from '@deepseek-ai/dsh-agent-spine-demo' import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local' @@ -484,17 +489,15 @@ async function runScenario(scenario: Scenario): Promise { async function writeRecording(scenario: Scenario, result: ScenarioResult): Promise { const dir = scenarioDir(scenario) await mkdir(dir, { recursive: true }) - await writeFile( - join(dir, 'session.jsonl'), - scrubRequestHeaders(tokenizeSessionFixtureCwd(rawSessionLog(result.parent))), - ) expect(result.children).toHaveLength(scenario.childSessions ?? 0) - for (const [index, child] of result.children.entries()) { - await writeFile( - join(dir, `session.${index + 1}.jsonl`), - scrubRequestHeaders(tokenizeSessionFixtureCwd(rawSessionLog(child))), - ) - } + const files = [join(dir, 'session.jsonl'), ...childFixturePaths(scenario)] + const existing = await Promise.all(files.map(async file => existsSync(file) ? readFile(file, 'utf8') : '')) + const fixtures = stabilizeFixtureMessageIds( + [result.parent, ...result.children] + .map(session => scrubRequestHeaders(tokenizeSessionFixtureCwd(rawSessionLog(session)))), + existing, + ) + await Promise.all(fixtures.map((fixture, index) => writeFile(files[index] as string, fixture))) } describe('TUI recorded-session terminal snapshots', () => { diff --git a/packages/support/acp-snapshot/README.i18n.yaml b/packages/support/acp-snapshot/README.i18n.yaml index afabe9147a..f90ce7ada8 100644 --- a/packages/support/acp-snapshot/README.i18n.yaml +++ b/packages/support/acp-snapshot/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/support/acp-snapshot/README.md -README.md: e3752dfb522cd55776f3ef796acdc15037e5a761 -README.zh.md: 6ce531640b5311662e4b958177e4417c8878617f +README.md: c5a1a07b9f85e1a91c52fe102be17e4172be112d +README.zh.md: 2454df9b3d8e67b4728d6582279ba21798d2ba9e diff --git a/packages/support/acp-snapshot/README.md b/packages/support/acp-snapshot/README.md index e3752dfb52..c5a1a07b9f 100644 --- a/packages/support/acp-snapshot/README.md +++ b/packages/support/acp-snapshot/README.md @@ -8,7 +8,7 @@ Four layers, importable separately: - **`launchAcpTestAgent` (launcher)** — boots a source agent under tsx or a built `lib` agent under plain Node from a supplied cwd, connects the SDK client over a raw-byte stdout tee, collects session updates and stderr, surfaces asynchronous spawn failures through startup, fails closed on unhandled permission requests, and owns graceful or signalled shutdown. Shutdown waits for process exit, inherited stdio closure, and ACP parser exhaustion before resolving or propagating a child error, so captures are complete and callers can remove owned paths after either outcome. When Windows accepts forced termination but publishes its exit marker asynchronously, shutdown gives that marker a bounded grace before treating fallback refusal as a second failure. Snapshot and ordinary e2e suites share this process boundary; a test supplies only agent paths, cwd, environment overrides, and any permission policy. - **`runScenario` (harness)** — drives ACP JSON-RPC stdio from a deterministic `input.json` script through the launcher, tees raw stdout for the expected-output and purity checks, and harvests every persisted raw JSONL session log (parent and subagent children, primary-first) after graceful stdin EOF. `AgentUnderTest` supplies absolute `binScript`, optional `libBinScript`, `configPath`, and `tsconfigPath` paths because the subprocess cwd is outside the repo; `workspaceParent` may move the generated child cwd from the platform temp directory when that grant is itself under test. Startup failures preserve captured agent stderr in the rejected diagnostic. -- **Normalizers** — pure functions turning captured surfaces into stable text or portable fixtures: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs and every native/JavaScript filesystem spelling of the generated cwd → tokens, longest-first; cwd-rooted separators selected as canonical `/` or host-native; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept, the same cwd-path policy), `tokenizeSessionFixtureCwd` (the generated workspace and its filesystem aliases → `{{cwd}}`, authored temp paths unchanged), `scrubSystemPrompts` (prompt text → `{{system}}`), `scrubToolSchemas` (schema bulk → `{{tools}}`), and `scrubRequestHeaders` (all header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}` outside each pin, structure kept — [pinned-header Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)). +- **Normalizers** — pure functions turning captured surfaces into stable text or portable fixtures: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs and every native/JavaScript filesystem spelling of the generated cwd → tokens, longest-first; cwd-rooted separators selected as canonical `/` or host-native; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept, the same cwd-path policy), `tokenizeSessionFixtureCwd` (the generated workspace and its filesystem aliases → `{{cwd}}`, authored temp paths unchanged), `scrubSystemPrompts` (prompt text → `{{system}}`), `scrubToolSchemas` (schema bulk → `{{tools}}`), `scrubRequestHeaders` (all header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}` outside each pin, structure kept — [pinned-header Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)), and `stabilizeFixtureMessageIds` (committed UUIDs carried into unchanged, unambiguous messages across any recorder's fixture-ready parent/child logs). - **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario expected-output and re-persisted-log comparisons, record/refresh fixture write-back, rejection of structured `UNKNOWN_TOOL` results, a tokenized pin per header class composed with independently shared `system-prompt.expected.md` and `tool-schemas.expected.json` sidecars, and a live uniformity guard. Its fixture guards reject orphan scenario dirs, missing files, multiple pins for one class, duplicate sidecar content, unscrubbed JSONL headers, and malformed pinning headers. Before record or refresh writes fixtures, an unchanged complete message retains its committed UUID when its identity-free value resolves to exactly one fresh ID and one existing ID across the scenario's parent/child logs; new, changed, and ambiguous messages keep fresh UUIDs. Refresh evaluates fresh leaves with the harvested run's ids, cwd, and every cwd alias, then reuses normalized-equivalent leaves only when the complete logical-record layout aligns and volatile string replacements form a bijection; ambiguous logs keep fresh strings, and fresh semantic values remain authoritative. It also expands packed timing envelopes before aligning event times, so switching between packed and unpacked layouts cannot shift later records. A newly inserted `session/title` receives its preceding event's time so feature-driven insertions do not churn the remainder of a fixture. Each scenario directory's `session.jsonl` plus contiguous `session..jsonl` siblings are the ordered primary/child inventory; the scenario table does not duplicate their count. Must be called at vitest collection time. Committed session fixtures use canonical packed rows. An in-flight branch that merges this contract runs the [temporary repository migrator](../../../scripts/migrate-packed-session-fixtures.ts) with `pnpm run migrate:packed-session-fixtures`; its [removal proposal](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md) owns deletion after affected branches converge. @@ -59,7 +59,7 @@ Every scenario compares `stdout.expected.jsonl` with cwd-rooted separators canon The example also ships a `cordis.snapshot.yml` replay overlay next to its `cordis.yml` (the bin swaps them under `DSH_SNAPSHOT=replay` — [single-source replay config Agent Note](../../../.agents/notes/archived/testing/2026-07-04-single-source-acp-replay-config.md)); replay fixtures are served by [`dsh-llm-replay`](../llm-replay/README.md), which this package points at via the `DSH_SNAPSHOT_*` env vars it sets on the child. `pnpm run test:snapshot:record` calls the live LLM and rewrites the recorded scenarios' model fixtures; `pnpm run test:snapshot:refresh` stays keyless, runs the replay overlay, and rewrites stdout, comparable session-log expected outputs, and owned prompt and tool-schema sidecars from the committed model scripts. Fixture roles, record/replay/refresh semantics, and scenario-table fields are documented on `Scenario` and in the [snapshot Agent Note](../../../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md). -Constraints: `suite.ts` and `harness.ts` import vitest (the harness polls its durable-boundary waits through `vi.waitFor`), so the package entry is importable only inside a vitest run (the launcher and normalizers have no such dependency but ship from the same entry). The launcher and suite factory are ACP-specific by design — the launcher speaks the SDK's `ClientSideConnection` — while the normalizers are transport-neutral session-log/text helpers also consumed by the TUI snapshot suite and the web browser e2e lane. Input scripts cover initialization, fresh-session creation, text prompting, cancellation, expected RPC failures, and durable turn-boundary waits. Permission round-trips are a FIFO queue of option-kind selections (`allow_once`, `reject_once`, …) mapped to the agent-issued `optionId`; an absent or exhausted queue answers `cancelled`, and an unoffered kind rejects the run. +Constraints: `suite.ts` and `harness.ts` import vitest (the harness polls its durable-boundary waits through `vi.waitFor`), so the package entry is importable only inside a vitest run (the launcher and normalizers have no such dependency but ship from the same entry). The launcher and suite factory are ACP-specific by design — the launcher speaks the SDK's `ClientSideConnection` — while the normalizers are transport-neutral session-log/text helpers also consumed by the JSON-RPC, TUI, and web snapshot recorders. Input scripts cover initialization, fresh-session creation, text prompting, cancellation, expected RPC failures, and durable turn-boundary waits. Permission round-trips are a FIFO queue of option-kind selections (`allow_once`, `reject_once`, …) mapped to the agent-issued `optionId`; an absent or exhausted queue answers `cancelled`, and an unoffered kind rejects the run. ## Model Experience diff --git a/packages/support/acp-snapshot/README.zh.md b/packages/support/acp-snapshot/README.zh.md index 6ce531640b..2454df9b3d 100644 --- a/packages/support/acp-snapshot/README.zh.md +++ b/packages/support/acp-snapshot/README.zh.md @@ -8,7 +8,7 @@ ACP 快照套件工具包：无密钥快照层（`pnpm run test:snapshot`，见[ - **`launchAcpTestAgent`（启动器）**：从指定 cwd 在 tsx 下启动源 agent，或在普通 Node 下启动已构建 `lib` agent；通过原始字节 stdout tee 连接 SDK 客户端，收集会话更新和 stderr，在启动过程中公开异步 spawn 失败，对未处理权限请求快速失败，并负责优雅或带信号关闭。关闭会等待进程退出、继承 stdio 关闭和 ACP parser 耗尽，然后才解析或传播子级错误，使捕获内容完整，且调用方可在任一结果后移除自有路径。当 Windows 接受强制终止但异步发布退出标记时，关闭会给该标记有界宽限，然后才将回退拒绝视为第二次失败。快照和普通 e2e 套件共享该进程边界；测试只需提供 agent 路径、cwd、环境覆盖和任何权限策略。 - **`runScenario`（harness）**：通过启动器从确定性 `input.json` 脚本驱动 ACP JSON-RPC stdio，将原始 stdout tee 给预期输出和纯度检查，并在优雅 stdin EOF 后收集每个持久化原始 JSONL 会话日志（父级和 subagent 子级，主级优先）。`AgentUnderTest` 提供绝对 `binScript`、可选 `libBinScript`、`configPath` 和 `tsconfigPath` 路径，因为子进程 cwd 位于仓库外。当生成子级 cwd 自身位于待测授权中时，`workspaceParent` 可以将它从平台临时目录移出。启动失败会在拒绝诊断中保留已捕获 agent stderr。 -- **规范化器**：将已捕获接口转换为稳定文本或可移植 fixture 的纯函数：`normalizeStdout`（JSON-RPC id → 首次出现序列；UUID 以及生成 cwd 的每个原生/JavaScript 文件系统写法 → token，按最长优先；根据 cwd 的分隔符选择规范 `/` 或宿主原生形式；同时作为 stdout 纯度检查）、`normalizeSessionLog`（时间归零、保留 `seq`、使用同一 cwd 路径策略）、`tokenizeSessionFixtureCwd`（生成的 workspace 及其文件系统别名 → `{{cwd}}`，手工编写的临时路径保持不变）、`scrubSystemPrompts`（提示词文本 → `{{system}}`）、`scrubToolSchemas`（schema bulk → `{{tools}}`）和 `scrubRequestHeaders`（每个 pin 之外的所有 header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}`，保留结构；见[header 固定 Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)）。 +- **规范化器**：将已捕获接口转换为稳定文本或可移植 fixture 的纯函数：`normalizeStdout`（JSON-RPC id → 首次出现序列；UUID 以及生成 cwd 的每个原生/JavaScript 文件系统写法 → token，按最长优先；根据 cwd 的分隔符选择规范 `/` 或宿主原生形式；同时作为 stdout 纯度检查）、`normalizeSessionLog`（时间归零、保留 `seq`、使用同一 cwd 路径策略）、`tokenizeSessionFixtureCwd`（生成的 workspace 及其文件系统别名 → `{{cwd}}`，手工编写的临时路径保持不变）、`scrubSystemPrompts`（提示词文本 → `{{system}}`）、`scrubToolSchemas`（schema bulk → `{{tools}}`）、`scrubRequestHeaders`（每个 pin 之外的所有 header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}`，保留结构；见[header 固定 Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)）和 `stabilizeFixtureMessageIds`（针对任意录制器已准备写入 fixture 的父级/子级日志，将已提交 UUID 带入未变化且无歧义的消息）。 - **`defineAcpSnapshotSuite`（工厂）**：为场景表注册完整 describe/it 树：每场景预期输出与重新持久化日志比较、录制/刷新 fixture 回写、拒绝结构化 `UNKNOWN_TOOL` 结果、每个 header 类别一个 token 化 pin（由可独立共享的 `system-prompt.expected.md` 和 `tool-schemas.expected.json` sidecar 组合而成），以及实时一致性保护。其 fixture 保护会拒绝遗留场景目录、缺失文件、一个类别包含多个 pin、重复的 sidecar 内容、未擦除的 JSONL header，以及格式错误的 pin header。在录制或刷新写入 fixture 前，如果一条未变化的完整消息去除身份后的值在场景的父级/子级日志中恰好对应一个本次生成的 ID 和一个现有 ID，它就会保留已提交的 UUID；新增、发生变化和有歧义的消息则保留本次生成的 UUID。刷新会使用收集所得本次运行的 id、cwd 及全部 cwd 别名评估本次生成的叶值；只有完整逻辑记录布局对齐且易变字符串替换形成双射时，才会复用规范化后等价的叶值；有歧义的日志保留本次生成的字符串，而本次生成的语义值仍为权威数据。它还会在对齐事件时间前展开打包时序 envelope，因此切换打包/非打包布局无法移动后续记录。新插入的 `session/title` 使用前一个事件的时间，因此功能驱动的插入不会扰动 fixture 余下部分。每个场景目录的 `session.jsonl` 和连续 `session..jsonl` 同级文件是有序主级/子级清单；场景表不重复其数量。必须在 vitest 收集时调用。 签入仓库的会话 fixture 使用规范打包行。合并此契约的在途分支通过 `pnpm run migrate:packed-session-fixtures` 运行[临时仓库迁移器](../../../scripts/migrate-packed-session-fixtures.ts)；待受影响分支收敛后，由其[移除提案](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md)负责删除该迁移器。 @@ -59,7 +59,7 @@ defineAcpSnapshotSuite({ 示例还发布 `cordis.snapshot.yml` 回放 overlay，位于 `cordis.yml` 旁边（bin 在 `DSH_SNAPSHOT=replay` 下交换它们，见[单源回放配置 Agent Note](../../../.agents/notes/archived/testing/2026-07-04-single-source-acp-replay-config.md)）；回放 fixture 由 [`dsh-llm-replay`](../llm-replay/README.md) 提供，该包通过对子级设置的 `DSH_SNAPSHOT_*` env var 指向它。`pnpm run test:snapshot:record` 调用实时 LLM，并重写已记录场景的模型 fixture；`pnpm run test:snapshot:refresh` 保持无密钥，运行回放 overlay，并从已提交模型脚本重写 stdout、可比较会话日志预期输出，以及各 pin 自有的提示词与工具 schema sidecar。Fixture 角色、录制/回放/刷新语义和场景表字段记录在 `Scenario` 以及[快照 Agent Note](../../../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md) 中。 -约束：`suite.ts` 与 `harness.ts` 导入 vitest（harness 通过 `vi.waitFor` 轮询其持久边界等待），因此包入口只能在 vitest 运行中导入（启动器和规范化器没有此依赖，但从同一入口发布）。启动器和套件工厂按设计专用于 ACP，启动器使用 SDK 的 `ClientSideConnection`；规范化器是与传输无关的会话日志/文本辅助工具，还由 TUI 快照套件和 web 浏览器 e2e lane 消费。输入脚本覆盖初始化、新建会话、文本提示、取消、预期 RPC 失败和持久轮次边界等待。权限往返是选项类别选择（`allow_once`、`reject_once`等）的 FIFO 队列，映射到 agent 发出的 `optionId`；缺少或耗尽的队列回答 `cancelled`，未提供类别会拒绝运行。 +约束：`suite.ts` 与 `harness.ts` 导入 vitest（harness 通过 `vi.waitFor` 轮询其持久边界等待），因此包入口只能在 vitest 运行中导入（启动器和规范化器没有此依赖，但从同一入口发布）。启动器和套件工厂按设计专用于 ACP，启动器使用 SDK 的 `ClientSideConnection`；规范化器是与传输无关的会话日志/文本辅助工具，还由 JSON-RPC、TUI 和 web 快照录制器消费。输入脚本覆盖初始化、新建会话、文本提示、取消、预期 RPC 失败和持久轮次边界等待。权限往返是选项类别选择（`allow_once`、`reject_once`等）的 FIFO 队列，映射到 agent 发出的 `optionId`；缺少或耗尽的队列回答 `cancelled`，未提供类别会拒绝运行。 ## 模型体验 diff --git a/packages/support/acp-snapshot/src/index.ts b/packages/support/acp-snapshot/src/index.ts index 09d05031c7..6d8f5c0953 100644 --- a/packages/support/acp-snapshot/src/index.ts +++ b/packages/support/acp-snapshot/src/index.ts @@ -46,6 +46,7 @@ export { export { defineAcpSnapshotSuite, refreshFixtureReplacements, + stabilizeFixtureMessageIds, stabilizeRefreshLog, type Scenario, type SnapshotSuiteOptions, diff --git a/packages/support/acp-snapshot/src/suite.ts b/packages/support/acp-snapshot/src/suite.ts index 8a99bc0813..315095cc10 100644 --- a/packages/support/acp-snapshot/src/suite.ts +++ b/packages/support/acp-snapshot/src/suite.ts @@ -554,8 +554,8 @@ function uniqueMessageIds(logs: readonly string[]): Map log.content)) +function fixtureMessageIdReplacements(logs: readonly string[], fixtures: readonly string[]): FixtureReplacement[] { + const freshIds = uniqueMessageIds(logs) const existingIds = uniqueMessageIds(fixtures) const replacements: FixtureReplacement[] = [] for (const [fingerprint, fresh] of freshIds) { @@ -573,6 +573,18 @@ function applyFixtureReplacements(content: string, replacements: readonly Fixtur return stable } +/** + * Carry committed UUIDs into unchanged, unambiguous messages in fresh session fixtures. + * + * @param logs Fresh fixture-ready session JSONL contents for one scenario. + * @param fixtures Existing fixture contents in matching order; missing fixtures may be empty strings. + * @returns The fresh contents with only reusable message UUIDs replaced. + */ +export function stabilizeFixtureMessageIds(logs: readonly string[], fixtures: readonly string[]): string[] { + const replacements = fixtureMessageIdReplacements(logs, fixtures) + return logs.map(log => applyFixtureReplacements(log, replacements)) +} + /** One packed row's member times, or `undefined` for an ordinary record. */ function packedTimes(record: Record): number[] | undefined { if (!PACKED_CHUNK_ROW_TYPES.has(record.type as string)) return undefined @@ -626,7 +638,7 @@ export function unknownToolCallIds(rawLog: string): string[] { * @returns Literal replacements from fresh values to the fixture's existing values. */ export function refreshFixtureReplacements(logs: HarvestedLog[], fixtures: string[]): FixtureReplacement[] { - const replacements = fixtureMessageIdReplacements(logs, fixtures) + const replacements = fixtureMessageIdReplacements(logs.map(log => log.content), fixtures) for (let i = 0; i < logs.length; i++) { const fresh = parseJsonlRecords((logs[i] as HarvestedLog).content)[0] const existing = parseJsonlRecords(fixtures[i] ?? '')[0] @@ -1111,23 +1123,22 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { const path = join(dir, file) return existsSync(path) ? readFile(path, 'utf8') : '' })) - const replacements = REFRESHING + const refreshReplacements = REFRESHING ? refreshFixtureReplacements(result.sessionLogs, existingFixtures) - : fixtureMessageIdReplacements(result.sessionLogs, existingFixtures) - const primary = (result.sessionLogs[0] as HarvestedLog).content - await writeFile(join(dir, outputFixtureFiles[0] as string), scrub(portableFixture( - REFRESHING - ? stabilizeRefreshLog(primary, existingFixtures[0] as string, replacements, ctx) - : applyFixtureReplacements(primary, replacements), - ))) - for (let i = 1; i < result.sessionLogs.length; i++) { - const child = (result.sessionLogs[i] as HarvestedLog).content - await writeFile(join(dir, outputFixtureFiles[i] as string), scrub(portableFixture( - REFRESHING - ? stabilizeRefreshLog(child, existingFixtures[i] as string, replacements, ctx) - : applyFixtureReplacements(child, replacements), - ))) - } + : [] + const outputFixtures = REFRESHING + ? result.sessionLogs.map((log, index) => scrub(portableFixture(stabilizeRefreshLog( + log.content, + existingFixtures[index] as string, + refreshReplacements, + ctx, + )))) + : stabilizeFixtureMessageIds( + result.sessionLogs.map(log => scrub(portableFixture(log.content))), + existingFixtures, + ) + await Promise.all(outputFixtures.map((fixture, index) => + writeFile(join(dir, outputFixtureFiles[index] as string), fixture))) if (RECORDING) { const outputNames = new Set(outputFixtureFiles) const entries = await readdir(dir, { withFileTypes: true }) diff --git a/packages/support/acp-snapshot/tests/suite.spec.ts b/packages/support/acp-snapshot/tests/suite.spec.ts index 2cd665f540..e2e4129a5b 100644 --- a/packages/support/acp-snapshot/tests/suite.spec.ts +++ b/packages/support/acp-snapshot/tests/suite.spec.ts @@ -4,7 +4,12 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { fileURLToPath } from 'node:url' import { afterAll, describe, expect, it } from 'vitest' -import { defineAcpSnapshotSuite, type HarvestedLog, type Scenario } from '../src/index.ts' +import { + defineAcpSnapshotSuite, + stabilizeFixtureMessageIds, + type HarvestedLog, + type Scenario, +} from '../src/index.ts' import { assertUniqueSnapshotContents, claimSharedSnapshot, @@ -635,6 +640,36 @@ describe('unknownToolCallIds', () => { }) }) +describe('stabilizeFixtureMessageIds', () => { + it('reuses one committed message UUID across fixture-ready parent and child logs', () => { + const freshId = '11111111-1111-4111-8111-111111111111' + const existingId = '22222222-2222-4222-8222-222222222222' + const log = (session: string, id: string): string => [ + JSON.stringify({ type: 'session', id: session, cwd: '{{cwd}}' }), + JSON.stringify({ + type: 'user/message', + data: { role: 'user', content: [{ type: 'text', text: 'same' }], source: { kind: 'user' }, id }, + }), + '', + ].join('\n') + const fresh = [log('fresh-parent', freshId), log('fresh-child', freshId)] + const existing = [log('old-parent', existingId), log('old-child', existingId)] + + const stable = stabilizeFixtureMessageIds(fresh, existing) + + expect(stable).toHaveLength(2) + for (const fixture of stable) { + expect(fixture).toContain(`"id":"${existingId}"`) + expect(fixture).not.toContain(freshId) + } + }) + + it('leaves fresh fixtures unchanged when no committed counterpart exists', () => { + const fresh = '{"type":"session","id":"new"}\n' + expect(stabilizeFixtureMessageIds([fresh], [''])).toEqual([fresh]) + }) +}) + describe('refreshFixtureReplacements', () => { it('maps fresh ids and cwd values to the existing fixture values, skipping non-replacements', () => { const log = (content: string): HarvestedLog => ({ id: 'diagnostic', createdAt: 1, content }) From 8982d714cb2bc362af06cd1274afc1acb667c891 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 8 Aug 2026 02:15:44 +0800 Subject: [PATCH 3/4] fix(snapshot): harden message id retention --- ...table-snapshot-refresh-volatiles.i18n.yaml | 4 +- ...07-27-stable-snapshot-refresh-volatiles.md | 8 +- ...27-stable-snapshot-refresh-volatiles.zh.md | 8 +- examples/jsonrpc-agent/tests/sdk.snapshot.ts | 13 +- .../support/acp-snapshot/README.i18n.yaml | 4 +- packages/support/acp-snapshot/README.md | 4 +- packages/support/acp-snapshot/README.zh.md | 4 +- packages/support/acp-snapshot/package.json | 2 + packages/support/acp-snapshot/src/suite.ts | 162 +++++++++----- .../support/acp-snapshot/tests/suite.spec.ts | 199 ++++++++++++------ packages/support/acp-snapshot/tsconfig.json | 3 + pnpm-lock.yaml | 3 + 12 files changed, 283 insertions(+), 131 deletions(-) diff --git a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.i18n.yaml index e322fba1dd..0820c01b7f 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.i18n.yaml +++ b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md -2026-07-27-stable-snapshot-refresh-volatiles.md: a0613357c698934f91598bdf53da983b1dd53f08 -2026-07-27-stable-snapshot-refresh-volatiles.zh.md: 303a4d6a4cc9f2d45448359c0e48677228a0c1f9 +2026-07-27-stable-snapshot-refresh-volatiles.md: c3eeeca01a7820b5f410bd895de998e944e58eb2 +2026-07-27-stable-snapshot-refresh-volatiles.zh.md: 388b67c67052074fa7423eae294e00b4122b2fe8 diff --git a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md index a0613357c6..c3eeeca01a 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md +++ b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.md @@ -12,9 +12,9 @@ Message identity needs a weaker structural precondition than aligned records: an ## Decision -Before record or refresh writes session fixtures, the shared snapshot support fingerprints every complete surface message with its top-level `id` removed and groups occurrences across all parent/child logs. It reuses an existing UUID only when one fingerprint resolves to exactly one fresh ID and one existing ID, then applies that mapping to every fresh log. Repeated inherited occurrences with the same ID remain one candidate, while new, changed, duplicate-content, malformed, and conflicting messages keep their fresh IDs. ACP, JSON-RPC, and Web recorders pass fixture-ready logs through the same helper before writing. +Before record or refresh writes session fixtures, the shared snapshot support passes fixture-ready logs to one structural message-ID owner. It recognizes surface carriers through the session package's authoritative surface-type predicate and the correlated queued copies in `agent/inbox/spliced`, fingerprints every complete message with its top-level `id` removed, and records every ID-to-fingerprint edge across all parent/child logs. It reuses an existing UUID only when both its ID and fingerprint have degree one in the fresh and existing graphs, then rewrites only validated message `id` fields in those carriers. Repeated inherited occurrences with the same ID remain one candidate, while new, changed, duplicate-content, malformed, and conflicting messages keep their fresh IDs. ACP, JSON-RPC, and Web recorders run this pass after header scrubbing and cwd tokenization, so fixture spellings rather than raw host paths determine identity. -Refresh write-back uses `normalizeSessionLog` as its volatile-value authority for aligned leaves. It normalizes the original harvested records with the fresh run's ids, cwd, and every cwd alias, while normalizing fixture records with the fixture header context; literal replacements affect only the raw values being written. After existing record alignment, it recursively compares fresh and existing leaves through those normalized records: normalized-equivalent leaves retain the existing raw value, while normalized-distinct leaves retain the fresh semantic value. +Refresh write-back uses `normalizeSessionLog` as its volatile-value authority for aligned leaves. It normalizes the original harvested records with the fresh run's ids, cwd, and every cwd alias, while normalizing fixture records with the fixture header context; literal replacements are limited to fresh-run session IDs, cwd values, and spill paths. After existing record alignment, it recursively compares fresh and existing leaves through those normalized records: normalized-equivalent leaves retain the existing raw value, while normalized-distinct leaves retain the fresh semantic value. Complete message IDs in surface or inbox carriers are excluded from this path so positional reuse and structural reuse cannot assign the same committed UUID independently. Before reuse, the complete logical-record layout must align, apart from the existing packed-chunk and inserted-title equivalences. Normalized-equivalent changed strings form a log-wide bijection: one fresh string maps to exactly one existing string and vice versa, so repeated IDs remain correlated across records. An unexplained record mismatch or conflicting mapping disables normalized string reuse for that log. @@ -30,6 +30,6 @@ Object fields align by key. Array elements align only when all corresponding arr ## Consequences -Record and refresh no longer rewrite an unchanged unique message UUID solely because another event changed the surrounding record layout, regardless of whether ACP, JSON-RPC, or Web owns the recording. Repeated refreshes also retain aligned fixture values that the normalizer classifies as volatile, and new volatile categories added to the normalizer automatically inherit that write-back behavior. Structural ambiguity remains conservative: unmatched records, conflicting string mappings, resized arrays, strings containing both semantic and volatile changes, and non-unique message fingerprints use fresh values rather than risk reusing misaligned data. +Record and refresh no longer rewrite an unchanged unique message UUID solely because another event changed the surrounding record layout, regardless of whether ACP, JSON-RPC, or Web owns the recording. Repeated refreshes also retain aligned fixture values that the normalizer classifies as volatile, and new volatile categories added to the normalizer automatically inherit that write-back behavior. Structural ambiguity remains conservative: unmatched records, conflicting string mappings, resized arrays, strings containing both semantic and volatile changes, malformed messages, and any message graph with a non-unique ID or fingerprint use fresh values rather than risk reusing misaligned data. -Focused unit coverage pins scenario-wide parent/child message correlation, unrelated event insertion, record write-back, new/changed/ambiguous messages, recursive object/array behavior, conflicting mappings, fresh cwd aliases, volatile strings, and fresh semantic fields. Keyless refresh coverage proves approval UUIDs, cwd aliases, spill paths, and event-read volatility leave their committed fixtures byte-identical. +Focused unit coverage pins all authoritative surface-message shapes, durable inbox/surface correlation, scenario-wide parent/child correlation, cwd-bearing fixture-ready matching, unrelated event insertion, malformed-message isolation, both-axis graph ambiguity, single-owner write-back, recursive object/array behavior, conflicting mappings, fresh cwd aliases, volatile strings, and fresh semantic fields. Keyless refresh coverage proves approval UUIDs, cwd aliases, spill paths, and event-read volatility leave their committed fixtures byte-identical. diff --git a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.zh.md b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.zh.md index 303a4d6a4c..388b67c670 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.zh.md +++ b/.agents/notes/implemented/bug-fix/2026-07-27-stable-snapshot-refresh-volatiles.zh.md @@ -12,9 +12,9 @@ ACP（Agent Client Protocol）快照比较会归一化生成的 UUID、cwd 别 ## 决策 -在录制或刷新写入会话 fixture 前，共享快照支持层会移除每条完整 surface 消息的顶层 `id` 并计算指纹，同时将所有父级/子级日志中的出现项分组。仅当一个指纹恰好对应一个本次生成的 ID 和一个现有 ID 时，才会复用现有 UUID，随后将该映射应用到每份本次生成的日志。具有相同 ID、重复出现的继承消息仍算作一个候选项；新增、发生变化、内容重复、格式错误和存在冲突的消息则保留本次生成的 ID。ACP、JSON-RPC 和 Web 录制器都会先让可写入 fixture 的日志经过同一个辅助函数，再执行写入。 +在录制或刷新写入会话 fixture 前，共享快照支持层会将可写入 fixture 的日志交给一个负责结构化处理消息 ID 的组件。该组件通过会话包的权威 surface 类型谓词识别 surface 载体，并识别 `agent/inbox/spliced` 中与这些载体关联的已排队消息副本；随后移除每条完整消息的顶层 `id` 并计算指纹，同时记录所有父级/子级日志中每条 ID 与指纹之间的关联边。仅当该 ID 与指纹在本次生成图和现有图中的度均为 1 时，才会复用现有 UUID，随后仅改写这些载体中通过验证的消息 `id` 字段。具有相同 ID、重复出现的继承消息仍算作一个候选项；新增、发生变化、内容重复、格式错误和存在冲突的消息则保留本次生成的 ID。ACP、JSON-RPC 和 Web 录制器会在擦除 header 并对 cwd 进行 token 化后执行这一步，因此消息身份取决于 fixture 中的写法，而非宿主机原始路径。 -刷新写回以 `normalizeSessionLog` 作为已对齐叶值的易变值判定依据。系统使用本次运行的 id、cwd 及全部 cwd 别名归一化原始收集记录，并使用 fixture 头部上下文归一化 fixture 记录；字面量替换只影响要写入的原始值。现有记录完成对齐后，系统基于这些归一化记录，递归比较本次生成记录与现有记录的叶节点：归一化后等价的叶节点保留现有原始值，归一化后不同的叶节点则保留本次生成的语义值。 +刷新写回以 `normalizeSessionLog` 作为已对齐叶值的易变值判定依据。系统使用本次运行的 id、cwd 及全部 cwd 别名归一化原始收集记录，并使用 fixture 头部上下文归一化 fixture 记录；字面量替换仅限于本次运行生成的会话 ID、cwd 值和 spill 路径。现有记录完成对齐后，系统基于这些归一化记录，递归比较本次生成记录与现有记录的叶节点：归一化后等价的叶节点保留现有原始值，归一化后不同的叶节点则保留本次生成的语义值。surface 或 inbox 载体中的完整消息 ID 不参与这一路径，以免按位置复用与结构复用各自独立分配同一个已提交 UUID。 复用前必须确保完整逻辑记录布局对齐，现有的打包分片与插入标题等价情形除外。归一化后等价但发生变化的字符串在整份日志范围内形成双射：一个本次生成的字符串只映射到一个现有字符串，反向亦然，因此跨记录重复出现的 ID 仍保持关联。出现无法解释的记录不匹配或映射冲突时，该日志会停用归一化字符串复用。 @@ -30,6 +30,6 @@ ACP（Agent Client Protocol）快照比较会归一化生成的 UUID、cwd 别 ## 后果 -录制和刷新不再仅仅因为另一个事件改变了周边记录布局，就改写未变化且唯一的消息 UUID，无论该录制由 ACP、JSON-RPC 还是 Web 负责。重复刷新也会保留规范化器归类为易变值的已对齐 fixture 值；以后加入规范化器的新易变值类别也会自动继承该写回行为。结构有歧义时仍采取保守策略：记录无法匹配、字符串映射冲突、数组尺寸发生变化、字符串同时包含语义变化与易变变化，或消息指纹不唯一时，均使用本次生成的值，避免冒险复用未对齐的数据。 +录制和刷新不再仅仅因为另一个事件改变了周边记录布局，就改写未变化且唯一的消息 UUID，无论该录制由 ACP、JSON-RPC 还是 Web 负责。重复刷新也会保留规范化器归类为易变值的已对齐 fixture 值；以后加入规范化器的新易变值类别也会自动继承该写回行为。结构有歧义时仍采取保守策略：记录无法匹配、字符串映射冲突、数组尺寸发生变化、字符串同时包含语义变化与易变变化、消息格式错误，或消息图中的 ID 或指纹不唯一时，均使用本次生成的值，避免冒险复用未对齐的数据。 -聚焦的单元测试固定了场景范围内的父级/子级消息关联、无关事件插入、录制写回、新增/发生变化/有歧义的消息、递归处理对象与数组的行为、映射冲突、本次运行的 cwd 别名、易变字符串以及本次生成的语义字段。无密钥刷新测试证明，审批 UUID、cwd 别名、spill 路径和事件读取中的易变值不会改变已提交 fixture 的任何字节。 +聚焦的单元测试固定了会话包权威谓词识别的所有 surface 消息形态、持久 inbox/surface 关联、场景范围内的父级/子级消息关联、带 cwd 的可写入 fixture 消息匹配、无关事件插入、格式错误消息隔离、消息图在 ID 与指纹两条轴上的歧义、由单一处理方负责的写回、递归处理对象与数组的行为、映射冲突、本次运行的 cwd 别名、易变字符串以及本次生成的语义字段。无密钥刷新测试证明，审批 UUID、cwd 别名、spill 路径和事件读取中的易变值不会改变已提交 fixture 的任何字节。 diff --git a/examples/jsonrpc-agent/tests/sdk.snapshot.ts b/examples/jsonrpc-agent/tests/sdk.snapshot.ts index 1f4335402e..4a29d6eb6a 100644 --- a/examples/jsonrpc-agent/tests/sdk.snapshot.ts +++ b/examples/jsonrpc-agent/tests/sdk.snapshot.ts @@ -350,15 +350,18 @@ describe('TypeScript SDK snapshots over the jsonrpc runtime', () => { content: log.content, })) const replacements = refreshFixtureReplacements(harvested, expectedContents) - expectedContents = await Promise.all(ordered.map(async (log, index) => { + const refreshed = ordered.map((log, index) => { const existing = expectedContents[index] - const file = files[index] - if (existing === undefined || file === undefined) throw new Error(`no fixture for persisted log ${index}`) - const stable = scrubRequestHeaders(tokenizeSessionFixtureCwd( + if (existing === undefined) throw new Error(`no fixture for persisted log ${index}`) + return scrubRequestHeaders(tokenizeSessionFixtureCwd( stabilizeRefreshLog(log.content, existing, replacements, actualContext), )) + }) + expectedContents = stabilizeFixtureMessageIds(refreshed, expectedContents) + await Promise.all(expectedContents.map(async (stable, index) => { + const file = files[index] + if (file === undefined) throw new Error(`no fixture for persisted log ${index}`) await writeFile(file, stable) - return stable })) } diff --git a/packages/support/acp-snapshot/README.i18n.yaml b/packages/support/acp-snapshot/README.i18n.yaml index bed064e909..25b54630b2 100644 --- a/packages/support/acp-snapshot/README.i18n.yaml +++ b/packages/support/acp-snapshot/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/support/acp-snapshot/README.md -README.md: 9d142dc964e60f9508b6c137525eb916cdbd969f -README.zh.md: 2e88e7c5bb8acb0cd99a35b5fe0fbfc15d6101e3 +README.md: 0b935ef60c33fd24660d8ecf2497f5506157c724 +README.zh.md: 91be3c97bcb67ce10c61513113f683e741bc762f diff --git a/packages/support/acp-snapshot/README.md b/packages/support/acp-snapshot/README.md index 9d142dc964..0b935ef60c 100644 --- a/packages/support/acp-snapshot/README.md +++ b/packages/support/acp-snapshot/README.md @@ -8,8 +8,8 @@ Four layers, importable separately: - **`launchAcpTestAgent` (launcher)** — boots a source agent under tsx or a built `lib` agent under plain Node from a supplied cwd, connects the SDK client over a raw-byte stdout tee, collects session updates and stderr, surfaces asynchronous spawn failures through startup, fails closed on unhandled permission requests, and owns graceful or signalled shutdown. Shutdown waits for process exit, inherited stdio closure, and ACP parser exhaustion before resolving or propagating a child error, so captures are complete and callers can remove owned paths after either outcome. When Windows accepts forced termination but publishes its exit marker asynchronously, shutdown gives that marker a bounded grace before treating fallback refusal as a second failure. Snapshot and ordinary e2e suites share this process boundary; a test supplies only agent paths, cwd, environment overrides, and any permission policy. - **`runScenario` (harness)** — drives ACP JSON-RPC stdio from a deterministic `input.json` script through the launcher, tees raw stdout for the expected-output and purity checks, and harvests every persisted raw JSONL session log (parent and subagent children, primary-first) after graceful stdin EOF. `AgentUnderTest` supplies absolute `binScript`, optional `libBinScript`, `configPath`, and `tsconfigPath` paths because the subprocess cwd is outside the repo; `workspaceParent` may move the generated child cwd from the platform temp directory when that grant is itself under test. Startup failures preserve captured agent stderr in the rejected diagnostic. -- **Normalizers** — pure functions turning captured surfaces into stable text or portable fixtures: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs and every native/JavaScript filesystem spelling of the generated cwd → tokens, longest-first; cwd-rooted separators selected as canonical `/` or host-native; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept, the same cwd-path policy), `tokenizeSessionFixtureCwd` (the generated workspace and its filesystem aliases → one canonical `{{cwd}}`, including an already-tokenized macOS `/private` alias; authored temp paths unchanged), `scrubSystemPrompts` (prompt text → `{{system}}`), `scrubToolSchemas` (schema bulk → `{{tools}}`), `scrubRequestHeaders` (all header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}` outside each pin, structure kept — [pinned-header Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)), and `stabilizeFixtureMessageIds` (committed UUIDs carried into unchanged, unambiguous messages across any recorder's fixture-ready parent/child logs). -- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario expected-output and re-persisted-log comparisons, record/refresh fixture write-back, rejection of structured `UNKNOWN_TOOL` results, a tokenized pin per header class composed with independently shared `system-prompt.expected.md` and `tool-schemas.expected.json` sidecars, and a live uniformity guard. Its fixture guards reject orphan scenario dirs, missing files, multiple pins for one class, duplicate sidecar content, noncanonical macOS-prefixed cwd tokens, unscrubbed JSONL headers, and malformed pinning headers. Before record or refresh writes fixtures, an unchanged complete message retains its committed UUID when its identity-free value resolves to exactly one fresh ID and one existing ID across the scenario's parent/child logs; new, changed, and ambiguous messages keep fresh UUIDs. Refresh evaluates fresh leaves with the harvested run's ids, cwd, and every cwd alias, then reuses normalized-equivalent leaves only when the complete logical-record layout aligns and volatile string replacements form a bijection; ambiguous logs keep fresh strings, and fresh semantic values remain authoritative. It also expands packed timing envelopes before aligning event times, so switching between packed and unpacked layouts cannot shift later records. A newly inserted `session/title` receives its preceding event's time so feature-driven insertions do not churn the remainder of a fixture. Each scenario directory's `session.jsonl` plus contiguous `session..jsonl` siblings are the ordered primary/child inventory; the scenario table does not duplicate their count. Must be called at vitest collection time. +- **Normalizers** — pure functions turning captured surfaces into stable text or portable fixtures: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs and every native/JavaScript filesystem spelling of the generated cwd → tokens, longest-first; cwd-rooted separators selected as canonical `/` or host-native; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept, the same cwd-path policy), `tokenizeSessionFixtureCwd` (the generated workspace and its filesystem aliases → one canonical `{{cwd}}`, including an already-tokenized macOS `/private` alias; authored temp paths unchanged), `scrubSystemPrompts` (prompt text → `{{system}}`), `scrubToolSchemas` (schema bulk → `{{tools}}`), `scrubRequestHeaders` (all header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}` outside each pin, structure kept — [pinned-header Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)), and `stabilizeFixtureMessageIds` (committed UUIDs carried into unchanged, mutually unique messages by structurally rewriting only complete surface and durable-inbox message ID fields across any recorder's fixture-ready parent/child logs). +- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario expected-output and re-persisted-log comparisons, record/refresh fixture write-back, rejection of structured `UNKNOWN_TOOL` results, a tokenized pin per header class composed with independently shared `system-prompt.expected.md` and `tool-schemas.expected.json` sidecars, and a live uniformity guard. Its fixture guards reject orphan scenario dirs, missing files, multiple pins for one class, duplicate sidecar content, noncanonical macOS-prefixed cwd tokens, unscrubbed JSONL headers, and malformed pinning headers. Before record or refresh writes fixtures, an unchanged complete message retains its committed UUID only when both its ID and identity-free fingerprint are unique across the scenario's fixture-ready parent/child logs; the session package's authoritative surface-type predicate selects surface carriers, correlated `agent/inbox/spliced` copies join the same mapping, and only validated `id` fields in those carriers are rewritten. New, changed, malformed, and graph-ambiguous messages keep fresh UUIDs. Refresh evaluates fresh leaves with the harvested run's ids, cwd, and every cwd alias, then reuses normalized-equivalent leaves only when the complete logical-record layout aligns and volatile string replacements form a bijection; complete message IDs in surface or inbox carriers are excluded because the later structural pass owns them, ambiguous logs keep fresh strings, and fresh semantic values remain authoritative. It also expands packed timing envelopes before aligning event times, so switching between packed and unpacked layouts cannot shift later records. A newly inserted `session/title` receives its preceding event's time so feature-driven insertions do not churn the remainder of a fixture. Each scenario directory's `session.jsonl` plus contiguous `session..jsonl` siblings are the ordered primary/child inventory; the scenario table does not duplicate their count. Must be called at vitest collection time. Committed session fixtures use canonical packed rows. An in-flight branch that merges this contract runs the [temporary repository migrator](../../../scripts/migrate-packed-session-fixtures.ts) with `pnpm run migrate:packed-session-fixtures`; its [removal proposal](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md) owns deletion after affected branches converge. diff --git a/packages/support/acp-snapshot/README.zh.md b/packages/support/acp-snapshot/README.zh.md index 2e88e7c5bb..91be3c97bc 100644 --- a/packages/support/acp-snapshot/README.zh.md +++ b/packages/support/acp-snapshot/README.zh.md @@ -8,8 +8,8 @@ ACP 快照套件工具包：无密钥快照层（`pnpm run test:snapshot`，见[ - **`launchAcpTestAgent`（启动器）**：从指定 cwd 在 tsx 下启动源 agent，或在普通 Node 下启动已构建 `lib` agent；通过原始字节 stdout tee 连接 SDK 客户端，收集会话更新和 stderr，在启动过程中公开异步 spawn 失败，对未处理权限请求快速失败，并负责优雅或带信号关闭。关闭会等待进程退出、继承 stdio 关闭和 ACP parser 耗尽，然后才解析或传播子级错误，使捕获内容完整，且调用方可在任一结果后移除自有路径。当 Windows 接受强制终止但异步发布退出标记时，关闭会给该标记有界宽限，然后才将回退拒绝视为第二次失败。快照和普通 e2e 套件共享该进程边界；测试只需提供 agent 路径、cwd、环境覆盖和任何权限策略。 - **`runScenario`（harness）**：通过启动器从确定性 `input.json` 脚本驱动 ACP JSON-RPC stdio，将原始 stdout tee 给预期输出和纯度检查，并在优雅 stdin EOF 后收集每个持久化原始 JSONL 会话日志（父级和 subagent 子级，主级优先）。`AgentUnderTest` 提供绝对 `binScript`、可选 `libBinScript`、`configPath` 和 `tsconfigPath` 路径，因为子进程 cwd 位于仓库外。当生成子级 cwd 自身位于待测授权中时，`workspaceParent` 可以将它从平台临时目录移出。启动失败会在拒绝诊断中保留已捕获 agent stderr。 -- **规范化器**：将已捕获接口转换为稳定文本或可移植 fixture 的纯函数：`normalizeStdout`（JSON-RPC id → 首次出现序列；UUID 以及生成 cwd 的每个原生/JavaScript 文件系统写法 → token，按最长优先；根据 cwd 的分隔符选择规范 `/` 或宿主原生形式；同时作为 stdout 纯度检查）、`normalizeSessionLog`（时间归零、保留 `seq`、使用同一 cwd 路径策略）、`tokenizeSessionFixtureCwd`（生成的 workspace 及其文件系统别名，包括已 token 化的 macOS `/private` 别名 → 单一规范 `{{cwd}}`；手工编写的临时路径保持不变）、`scrubSystemPrompts`（提示词文本 → `{{system}}`）、`scrubToolSchemas`（schema bulk → `{{tools}}`）、`scrubRequestHeaders`（每个 pin 之外的所有 header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}`，保留结构；见[header 固定 Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)）和 `stabilizeFixtureMessageIds`（针对任意录制器已准备写入 fixture 的父级/子级日志，将已提交 UUID 带入未变化且无歧义的消息）。 -- **`defineAcpSnapshotSuite`（工厂）**：为场景表注册完整 describe/it 树：每场景预期输出与重新持久化日志比较、录制/刷新 fixture 回写、拒绝结构化 `UNKNOWN_TOOL` 结果、每个 header 类别一个 token 化 pin（由可独立共享的 `system-prompt.expected.md` 和 `tool-schemas.expected.json` sidecar 组合而成），以及实时一致性保护。其 fixture 保护会拒绝遗留场景目录、缺失文件、一个类别包含多个 pin、重复的 sidecar 内容、带非规范 macOS 前缀的 cwd token、未擦除的 JSONL header，以及格式错误的 pin header。在录制或刷新写入 fixture 前，如果一条未变化的完整消息去除身份后的值在场景的父级/子级日志中恰好对应一个本次生成的 ID 和一个现有 ID，它就会保留已提交的 UUID；新增、发生变化和有歧义的消息则保留本次生成的 UUID。刷新会使用收集所得本次运行的 id、cwd 及全部 cwd 别名评估本次生成的叶值；只有完整逻辑记录布局对齐且易变字符串替换形成双射时，才会复用规范化后等价的叶值；有歧义的日志保留本次生成的字符串，而本次生成的语义值仍为权威数据。它还会在对齐事件时间前展开打包时序 envelope，因此切换打包/非打包布局无法移动后续记录。新插入的 `session/title` 使用前一个事件的时间，因此功能驱动的插入不会扰动 fixture 余下部分。每个场景目录的 `session.jsonl` 和连续 `session..jsonl` 同级文件是有序主级/子级清单；场景表不重复其数量。必须在 vitest 收集时调用。 +- **规范化器**：将已捕获接口转换为稳定文本或可移植 fixture 的纯函数：`normalizeStdout`（JSON-RPC id → 首次出现序列；UUID 以及生成 cwd 的每个原生/JavaScript 文件系统写法 → token，按最长优先；根据 cwd 的分隔符选择规范 `/` 或宿主原生形式；同时作为 stdout 纯度检查）、`normalizeSessionLog`（时间归零、保留 `seq`、使用同一 cwd 路径策略）、`tokenizeSessionFixtureCwd`（生成的 workspace 及其文件系统别名，包括已 token 化的 macOS `/private` 别名 → 单一规范 `{{cwd}}`；手工编写的临时路径保持不变）、`scrubSystemPrompts`（提示词文本 → `{{system}}`）、`scrubToolSchemas`（schema bulk → `{{tools}}`）、`scrubRequestHeaders`（每个 pin 之外的所有 header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}`，保留结构；见[header 固定 Agent Note](../../../.agents/notes/archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)）和 `stabilizeFixtureMessageIds`（针对任意录制器已准备写入 fixture 的父级/子级日志，通过结构化方式仅改写 surface 和持久 inbox 中完整消息的 ID 字段，将已提交 UUID 带入未变化且双向唯一匹配的消息）。 +- **`defineAcpSnapshotSuite`（工厂）**：为场景表注册完整 describe/it 树：每场景预期输出与重新持久化日志比较、录制/刷新 fixture 回写、拒绝结构化 `UNKNOWN_TOOL` 结果、每个 header 类别一个 token 化 pin（由可独立共享的 `system-prompt.expected.md` 和 `tool-schemas.expected.json` sidecar 组合而成），以及实时一致性保护。其 fixture 保护会拒绝遗留场景目录、缺失文件、一个类别包含多个 pin、重复的 sidecar 内容、带非规范 macOS 前缀的 cwd token、未擦除的 JSONL header，以及格式错误的 pin header。在录制或刷新写入 fixture 前，仅当一条未变化完整消息的 ID 及其去除身份后的指纹在场景可写入 fixture 的父级/子级日志中均唯一时，该消息才会保留已提交的 UUID；会话包的权威 surface 类型谓词负责选择 surface 载体，与其关联的 `agent/inbox/spliced` 副本也纳入同一映射，且仅改写这些载体中通过验证的 `id` 字段。新增、发生变化、格式错误以及图关系存在歧义的消息保留本次生成的 UUID。刷新会使用收集所得本次运行的 id、cwd 及全部 cwd 别名评估本次生成的叶值；只有完整逻辑记录布局对齐且易变字符串替换形成双射时，才会复用归一化后等价的叶值；surface 或 inbox 载体中的完整消息 ID 不参与此路径，因为后续结构化处理负责这些 ID；有歧义的日志保留本次生成的字符串，而本次生成的语义值仍为权威数据。它还会在对齐事件时间前展开打包时序 envelope，因此切换打包/非打包布局无法移动后续记录。新插入的 `session/title` 使用前一个事件的时间，因此功能驱动的插入不会扰动 fixture 余下部分。每个场景目录的 `session.jsonl` 和连续 `session..jsonl` 同级文件是有序主级/子级清单；场景表不重复其数量。必须在 vitest 收集时调用。 签入仓库的会话 fixture 使用规范打包行。合并此契约的在途分支通过 `pnpm run migrate:packed-session-fixtures` 运行[临时仓库迁移器](../../../scripts/migrate-packed-session-fixtures.ts)；待受影响分支收敛后，由其[移除提案](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md)负责删除该迁移器。 diff --git a/packages/support/acp-snapshot/package.json b/packages/support/acp-snapshot/package.json index c231591103..b504cbe50d 100644 --- a/packages/support/acp-snapshot/package.json +++ b/packages/support/acp-snapshot/package.json @@ -31,10 +31,12 @@ }, "peerDependencies": { "@deepseek-ai/dsh-invariants": "^0.0.1", + "@deepseek-ai/dsh-session": "^0.0.1", "cordis": "^4.0.0-rc.7" }, "devDependencies": { "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", "cordis": "^4.0.0-rc.7" } } diff --git a/packages/support/acp-snapshot/src/suite.ts b/packages/support/acp-snapshot/src/suite.ts index 993aee254f..c0cfc56e24 100644 --- a/packages/support/acp-snapshot/src/suite.ts +++ b/packages/support/acp-snapshot/src/suite.ts @@ -20,6 +20,7 @@ import { readFile, readdir, rm, writeFile } from 'node:fs/promises' import { existsSync } from 'node:fs' import { join } from 'node:path' +import { isSurfaceEligibleType } from '@deepseek-ai/dsh-session/surface' import { describe, expect, it } from 'vitest' import { type AgentUnderTest, type HarvestedLog, type InputScript, runScenario } from './harness.ts' import { @@ -512,11 +513,11 @@ export function headerChangeCount(rawLog: string): number { .length } -/** A literal string replacement used to carry an existing fixture value into fresh write-back. */ +/** A literal replacement from a fresh replay-run volatile to its existing fixture value. */ export interface FixtureReplacement { - /** The fresh run's value to replace. */ + /** The fresh replay run's volatile value. */ from: string - /** The existing fixture value to keep. */ + /** The existing fixture value retained during write-back. */ to: string } @@ -526,24 +527,51 @@ function parseJsonlRecords(text: string): Record[] { .map(line => JSON.parse(line) as Record) } +/** Narrow one parsed value to the complete identified-message shape retained by fixtures. */ +function completeMessage(value: unknown): Record | undefined { + if ( + !isRecord(value) + || typeof value.id !== 'string' + || !UUID_RE.test(value.id) + || typeof value.role !== 'string' + || !Array.isArray(value.content) + || !isRecord(value.source) + ) return undefined + return value +} + /** Return the complete identified message carried by one surface event. */ -function eventMessage(record: Record): Record | undefined { +function surfaceEventMessage(record: Record): Record | undefined { + const type = record.type + if (typeof type !== 'string' || !isSurfaceEligibleType(type)) return undefined const data = record.data if (!isRecord(data)) return undefined - const message = record.type === 'user/message' - ? data - : record.type === 'assistant/message' || record.type === 'tool/result' || record.type === 'steering/message' - ? data.message - : undefined - if ( - !isRecord(message) - || typeof message.id !== 'string' - || !UUID_RE.test(message.id) - || typeof message.role !== 'string' - || !Array.isArray(message.content) - || !isRecord(message.source) - ) return undefined - return message + let message: unknown + switch (type) { + case 'user/message': + message = data + break + case 'assistant/message': + case 'tool/result': + message = data.message + break + /* v8 ignore next -- the authoritative predicate must fail loud when a new surface shape lands. */ + default: throw new Error(`acp-snapshot: unsupported surface event type "${type}"`) + } + return completeMessage(message) +} + +/** Return complete message identities structurally owned by one durable record. */ +function recordMessages(record: Record): Record[] { + const surfaceMessage = surfaceEventMessage(record) + if (surfaceMessage !== undefined) return [surfaceMessage] + if (record.type !== 'agent/inbox/spliced' || !isRecord(record.data) || !Array.isArray(record.data.inserted)) { + return [] + } + return record.data.inserted.flatMap((value) => { + const message = completeMessage(value) + return message === undefined ? [] : [message] + }) } /** Serialize parsed JSON by value rather than insertion order. */ @@ -555,42 +583,48 @@ function canonicalJson(value: unknown): string { return JSON.stringify(value) } -/** Index each unambiguous identity-free message value by its sole message id. */ -function uniqueMessageIds(logs: readonly string[]): Map { - const fingerprintsById = new Map() +/** Index identity-free message values whose ID and fingerprint are mutually unique. */ +function uniqueMessageIds(logs: readonly string[]): Map { + const fingerprintsById = new Map>() + const idsByFingerprint = new Map>() for (const log of logs) { for (const record of parseJsonlRecords(log)) { - const message = eventMessage(record) - if (message === undefined) continue - const { id, ...withoutId } = message - const messageId = id as string - const fingerprint = canonicalJson(withoutId) - if (!fingerprintsById.has(messageId)) fingerprintsById.set(messageId, fingerprint) - else if (fingerprintsById.get(messageId) !== fingerprint) fingerprintsById.set(messageId, undefined) + for (const message of recordMessages(record)) { + const { id, ...withoutId } = message + const messageId = id as string + const fingerprint = canonicalJson(withoutId) + const fingerprints = fingerprintsById.get(messageId) + if (fingerprints === undefined) fingerprintsById.set(messageId, new Set([fingerprint])) + else fingerprints.add(fingerprint) + const ids = idsByFingerprint.get(fingerprint) + if (ids === undefined) idsByFingerprint.set(fingerprint, new Set([messageId])) + else ids.add(messageId) + } } } - const idsByFingerprint = new Map() - for (const [id, fingerprint] of fingerprintsById) { - if (fingerprint === undefined) continue - if (!idsByFingerprint.has(fingerprint)) idsByFingerprint.set(fingerprint, id) - else idsByFingerprint.set(fingerprint, undefined) + const unique = new Map() + for (const [id, fingerprints] of fingerprintsById) { + if (fingerprints.size !== 1) continue + const fingerprint = fingerprints.values().next().value as string + if (idsByFingerprint.get(fingerprint)?.size !== 1) continue + unique.set(fingerprint, id) } - return idsByFingerprint + return unique } /** * Match unchanged complete messages across a scenario's fresh and existing logs. - * New, changed, repeated, or otherwise ambiguous messages keep their fresh ids. + * New, changed, duplicate-content, or otherwise ambiguous messages keep their fresh ids. */ -function fixtureMessageIdReplacements(logs: readonly string[], fixtures: readonly string[]): FixtureReplacement[] { +function fixtureMessageIdReplacements(logs: readonly string[], fixtures: readonly string[]): Map { const freshIds = uniqueMessageIds(logs) const existingIds = uniqueMessageIds(fixtures) - const replacements: FixtureReplacement[] = [] + const replacements = new Map() for (const [fingerprint, fresh] of freshIds) { const existing = existingIds.get(fingerprint) - if (fresh === undefined || existing === undefined || fresh === existing) continue - replacements.push({ from: fresh, to: existing }) + if (existing === undefined || fresh === existing) continue + replacements.set(fresh, existing) } return replacements } @@ -602,6 +636,22 @@ function applyFixtureReplacements(content: string, replacements: readonly Fixtur return stable } +/** Rewrite only validated durable-message ID fields, leaving every other occurrence untouched. */ +function applyFixtureMessageIds(content: string, replacements: ReadonlyMap): string { + return content.split('\n').map((line) => { + if (line.trim().length === 0) return line + const record = JSON.parse(line) as Record + let changed = false + for (const message of recordMessages(record)) { + const replacement = replacements.get(message.id as string) + if (replacement === undefined) continue + message.id = replacement + changed = true + } + return changed ? JSON.stringify(record) : line + }).join('\n') +} + /** * Carry committed UUIDs into unchanged, unambiguous messages in fresh session fixtures. * @@ -611,7 +661,7 @@ function applyFixtureReplacements(content: string, replacements: readonly Fixtur */ export function stabilizeFixtureMessageIds(logs: readonly string[], fixtures: readonly string[]): string[] { const replacements = fixtureMessageIdReplacements(logs, fixtures) - return logs.map(log => applyFixtureReplacements(log, replacements)) + return logs.map(log => applyFixtureMessageIds(log, replacements)) } /** One packed row's member times, or `undefined` for an ordinary record. */ @@ -659,15 +709,15 @@ export function unknownToolCallIds(rawLog: string): string[] { } /** - * Build refresh write-back replacements: scenario-wide unchanged message ids, - * plus per-log session ids, cwd values, and spill paths. + * Build refresh write-back replacements for per-log session ids, cwd values, + * and spill paths. Durable message ids have a later structural owner. * * @param logs The freshly harvested logs, in fixture order. * @param fixtures The existing fixture contents, in matching order. * @returns Literal replacements from fresh values to the fixture's existing values. */ export function refreshFixtureReplacements(logs: HarvestedLog[], fixtures: string[]): FixtureReplacement[] { - const replacements = fixtureMessageIdReplacements(logs.map(log => log.content), fixtures) + const replacements: FixtureReplacement[] = [] for (let i = 0; i < logs.length; i++) { const fresh = parseJsonlRecords((logs[i] as HarvestedLog).content)[0] const existing = parseJsonlRecords(fixtures[i] ?? '')[0] @@ -818,6 +868,7 @@ function collectNormalizedStringMappings( existing: unknown, normalizedFresh: unknown, normalizedExisting: unknown, + excludedStrings: ReadonlySet, forward: Map, reverse: Map, ): boolean { @@ -837,6 +888,7 @@ function collectNormalizedStringMappings( existing[index], normalizedFresh[index], normalizedExisting[index], + excludedStrings, forward, reverse, )) @@ -856,6 +908,7 @@ function collectNormalizedStringMappings( existing[key], normalizedFresh[key], normalizedExisting[key], + excludedStrings, forward, reverse, )) @@ -866,6 +919,8 @@ function collectNormalizedStringMappings( || typeof normalizedFresh !== 'string' || normalizedFresh !== normalizedExisting || fresh === existing + || excludedStrings.has(fresh) + || excludedStrings.has(existing) ) return true const freshKey = JSON.stringify([normalizedFresh, fresh]) const existingKey = JSON.stringify([normalizedFresh, existing]) @@ -891,6 +946,10 @@ function normalizedStringMappings( freshContext: NormalizeContext, existingContext: NormalizeContext, ): Map | undefined { + const excludedStrings = new Set() + for (const record of [...freshRecords, ...existingRecords]) { + for (const message of recordMessages(record)) excludedStrings.add(message.id as string) + } const forward = new Map() const reverse = new Map() let existingIndex = 0 @@ -912,6 +971,7 @@ function normalizedStringMappings( existingRecord, normalizedRefreshRecord(freshRecords[recordIndex] as Record, freshContext), normalizedRefreshRecord(existingRecord, existingContext), + excludedStrings, forward, reverse, )) return undefined @@ -924,11 +984,13 @@ function normalizedStringMappings( /** * Rewrite a fresh replay-produced log so repeated refreshes do not churn * volatile fixture fields. Meaningful event payloads come from `fresh`; the - * existing fixture lends normalized-equivalent values, including ids, paths, + * existing fixture lends normalized-equivalent values, including non-message ids, paths, * creation/event times, spill locators, and hook durations, only when the * complete record layout aligns and volatile strings form a consistent - * bijection. Ambiguous layouts or mappings keep fresh strings. Packed timing - * envelopes expand for alignment, so packing does not shift later records; + * bijection. Complete durable-message ids are excluded because the later + * fixture-ready structural pass owns them. Ambiguous layouts or mappings + * keep fresh strings. Packed timing envelopes expand for alignment, so + * packing does not shift later records; * fresh semantic values and fragment arrays remain authoritative. * * @param fresh The newly harvested session JSONL. @@ -1158,17 +1220,15 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { const refreshReplacements = REFRESHING ? refreshFixtureReplacements(result.sessionLogs, existingFixtures) : [] - const outputFixtures = REFRESHING + const freshFixtures = REFRESHING ? result.sessionLogs.map((log, index) => scrub(portableFixture(stabilizeRefreshLog( log.content, existingFixtures[index] as string, refreshReplacements, ctx, )))) - : stabilizeFixtureMessageIds( - result.sessionLogs.map(log => scrub(portableFixture(log.content))), - existingFixtures, - ) + : result.sessionLogs.map(log => scrub(portableFixture(log.content))) + const outputFixtures = stabilizeFixtureMessageIds(freshFixtures, existingFixtures) await Promise.all(outputFixtures.map((fixture, index) => writeFile(join(dir, outputFixtureFiles[index] as string), fixture))) if (RECORDING) { diff --git a/packages/support/acp-snapshot/tests/suite.spec.ts b/packages/support/acp-snapshot/tests/suite.spec.ts index a915590b5d..3cfd1c0f0a 100644 --- a/packages/support/acp-snapshot/tests/suite.spec.ts +++ b/packages/support/acp-snapshot/tests/suite.spec.ts @@ -7,6 +7,7 @@ import { afterAll, describe, expect, it } from 'vitest' import { defineAcpSnapshotSuite, stabilizeFixtureMessageIds, + tokenizeSessionFixtureCwd, type HarvestedLog, type Scenario, } from '../src/index.ts' @@ -679,6 +680,109 @@ describe('stabilizeFixtureMessageIds', () => { } }) + it('rewrites only complete messages carried by surface events or durable inbox splices', () => { + const ids = { + freshUser: '11111111-1111-4111-8111-111111111111', + oldUser: '22222222-2222-4222-8222-222222222222', + freshAssistant: '33333333-3333-4333-8333-333333333333', + oldAssistant: '44444444-4444-4444-8444-444444444444', + freshTool: '55555555-5555-4555-8555-555555555555', + oldTool: '66666666-6666-4666-8666-666666666666', + oldMalformed: '77777777-7777-4777-8777-777777777777', + } as const + const message = (id: string, role: string, text: string): Record => ({ + id, + role, + content: [{ type: 'text', text }], + source: { kind: role === 'user' ? 'user' : 'model' }, + }) + const log = (userId: string, assistantId: string, toolId: string, malformedId: string): string => [ + JSON.stringify({ type: 'session', id: 'same', cwd: '{{cwd}}' }), + JSON.stringify({ + type: 'agent/inbox/spliced', + data: { + inserted: [ + message(userId, 'user', 'user'), + { ...message(userId, 'user', 'malformed inbox'), source: null }, + ], + }, + }), + JSON.stringify({ type: 'user/message', data: message(userId, 'user', 'user') }), + JSON.stringify({ type: 'assistant/message', data: { message: message(assistantId, 'assistant', 'assistant') } }), + JSON.stringify({ type: 'tool/result', data: { message: message(toolId, 'tool', 'tool') } }), + JSON.stringify({ type: 'turn/start', data: { id: userId } }), + JSON.stringify({ type: 'steering/message', data: message(userId, 'user', 'obsolete') }), + JSON.stringify({ type: 'user/message', data: { ...message(userId, 'user', 'malformed'), source: null } }), + JSON.stringify({ type: 'user/message', data: message(malformedId, 'user', 'non-UUID') }), + JSON.stringify({ type: 'assistant/message', data: null }), + JSON.stringify({ type: 42, data: message(userId, 'user', 'non-string type') }), + '', + ].join('\n') + + const stable = stabilizeFixtureMessageIds( + [log(ids.freshUser, ids.freshAssistant, ids.freshTool, 'not-a-uuid')], + [log(ids.oldUser, ids.oldAssistant, ids.oldTool, ids.oldMalformed)], + )[0] as string + const records = stable.trim().split('\n').map(line => JSON.parse(line) as Record) + + const inserted = ((records[1]?.data as { inserted: Array<{ id: string }> }).inserted) + expect(inserted[0]?.id).toBe(ids.oldUser) + expect(inserted[1]?.id).toBe(ids.freshUser) + expect((records[2]?.data as { id: string }).id).toBe(ids.oldUser) + expect((records[3]?.data as { message: { id: string } }).message.id).toBe(ids.oldAssistant) + expect((records[4]?.data as { message: { id: string } }).message.id).toBe(ids.oldTool) + expect((records[5]?.data as { id: string }).id).toBe(ids.freshUser) + expect((records[6]?.data as { id: string }).id).toBe(ids.freshUser) + expect((records[7]?.data as { id: string }).id).toBe(ids.freshUser) + expect((records[8]?.data as { id: string }).id).toBe('not-a-uuid') + }) + + it('matches cwd-bearing messages only after the fresh log reaches fixture-ready form', () => { + const freshId = '11111111-1111-4111-8111-111111111111' + const existingId = '22222222-2222-4222-8222-222222222222' + const freshCwd = '/tmp/acp-snapshot-fresh-cwd' + const message = (id: string, path: string): Record => ({ + type: 'user/message', + data: { + id, + role: 'user', + content: [{ type: 'text', text: `read ${path}/input.txt` }], + source: { kind: 'user' }, + }, + }) + const fresh = tokenizeSessionFixtureCwd([ + JSON.stringify({ type: 'session', id: 'fresh', cwd: freshCwd }), + JSON.stringify(message(freshId, freshCwd)), + '', + ].join('\n')) + const existing = [ + JSON.stringify({ type: 'session', id: 'old', cwd: '{{cwd}}' }), + JSON.stringify(message(existingId, '{{cwd}}')), + '', + ].join('\n') + + expect(stabilizeFixtureMessageIds([fresh], [existing])[0]).toContain(`"id":"${existingId}"`) + }) + + it('rejects a fingerprint connected to an id that also identifies different content', () => { + const freshId = '11111111-1111-4111-8111-111111111111' + const conflictingId = '22222222-2222-4222-8222-222222222222' + const competingId = '33333333-3333-4333-8333-333333333333' + const message = (id: string, text: string): string => JSON.stringify({ + type: 'user/message', + data: { id, role: 'user', content: [{ type: 'text', text }], source: { kind: 'user' } }, + }) + const fresh = `${message(freshId, 'shared')}\n` + const existing = [ + message(conflictingId, 'shared'), + message(conflictingId, 'different'), + message(competingId, 'shared'), + '', + ].join('\n') + + expect(stabilizeFixtureMessageIds([fresh], [existing])).toEqual([fresh]) + }) + it('leaves fresh fixtures unchanged when no committed counterpart exists', () => { const fresh = '{"type":"session","id":"new"}\n' expect(stabilizeFixtureMessageIds([fresh], [''])).toEqual([fresh]) @@ -726,76 +830,28 @@ describe('refreshFixtureReplacements', () => { ]) }) - it('maps one inherited message id across parent and child logs', () => { + it('leaves complete message ids out of the literal refresh replacement list', () => { const freshMessageId = '11111111-1111-4111-8111-111111111111' const existingMessageId = '22222222-2222-4222-8222-222222222222' - const content = [{ type: 'text', text: 'inherited' }] const log = (sessionId: string, messageId: string): string => [ JSON.stringify({ type: 'session', id: sessionId, cwd: '/same' }), JSON.stringify({ type: 'user/message', - data: { role: 'user', content, source: { kind: 'user' }, id: messageId }, + data: { + id: messageId, + role: 'user', + content: [{ type: 'text', text: 'same' }], + source: { kind: 'user' }, + }, }), '', ].join('\n') - const harvested = (content: string): HarvestedLog => ({ id: 'diagnostic', createdAt: 1, content }) - const replacements = refreshFixtureReplacements( - [harvested(log('fresh-parent', freshMessageId)), harvested(log('fresh-child', freshMessageId))], - [log('old-parent', existingMessageId), log('old-child', existingMessageId)], + [{ id: 'diagnostic', createdAt: 1, content: log('fresh', freshMessageId) }], + [log('old', existingMessageId)], ) - expect(replacements.filter(replacement => replacement.from === freshMessageId)).toEqual([ - { from: freshMessageId, to: existingMessageId }, - ]) - }) - - it('keeps fresh ids for new, changed, and ambiguous messages', () => { - const ids = { - new: '11111111-1111-4111-8111-111111111111', - changed: '22222222-2222-4222-8222-222222222222', - ambiguousA: '33333333-3333-4333-8333-333333333333', - ambiguousB: '44444444-4444-4444-8444-444444444444', - oldChanged: '55555555-5555-4555-8555-555555555555', - oldAmbiguous: '66666666-6666-4666-8666-666666666666', - stable: '77777777-7777-4777-8777-777777777777', - } as const - const message = (id: string, text: string): Record => ({ - type: 'user/message', - data: { role: 'user', content: [{ type: 'text', text }], source: { kind: 'user' }, id }, - }) - const log = (messages: Record[]): string => [ - JSON.stringify({ type: 'session', id: 'same', cwd: '/same' }), - ...messages.map(record => JSON.stringify(record)), - '', - ].join('\n') - const fresh = log([ - message(ids.new, 'new'), - message(ids.changed, 'changed'), - message(ids.changed, 'changed again'), - message(ids.ambiguousA, 'duplicate'), - message(ids.ambiguousB, 'duplicate'), - message(ids.stable, 'stable'), - ]) - const existing = log([ - message(ids.oldChanged, 'before'), - message(ids.oldAmbiguous, 'duplicate'), - message(ids.stable, 'stable'), - ]) - - const replacements = refreshFixtureReplacements( - [{ id: 'diagnostic', createdAt: 1, content: fresh }], - [existing], - ) - - const replacedIds = replacements.map(replacement => replacement.from) - for (const id of [ - ids.new, - ids.changed, - ids.ambiguousA, - ids.ambiguousB, - ids.stable, - ]) expect(replacedIds).not.toContain(id) + expect(replacements).toEqual([{ from: 'fresh', to: 'old' }]) }) }) @@ -936,13 +992,38 @@ describe('stabilizeRefreshLog', () => { [{ id: 'diagnostic', createdAt: 1, content: fresh }], [existing], ) - const output = stabilize(fresh, existing, replacements).trim().split('\n') + const refreshed = stabilize(fresh, existing, replacements) + const intermediate = refreshed.trim().split('\n') + .map(line => JSON.parse(line) as Record) + expect((intermediate[1]?.data as { id: string }).id).toBe(freshUserId) + expect(((intermediate[3]?.data as { message: { id: string } }).message).id).toBe(freshAssistantId) + + const output = (stabilizeFixtureMessageIds([refreshed], [existing])[0] as string).trim().split('\n') .map(line => JSON.parse(line) as Record) expect((output[1]?.data as { id: string }).id).toBe(existingUserId) expect(((output[3]?.data as { message: { id: string } }).message).id).toBe(existingAssistantId) }) + it('leaves an aligned complete message id to the fixture-ready structural pass', () => { + const freshId = '11111111-1111-4111-8111-111111111111' + const existingId = '22222222-2222-4222-8222-222222222222' + const log = (id: string): string => [ + JSON.stringify({ type: 'session', id: 'same', createdAt: 1, cwd: '/same' }), + JSON.stringify({ + type: 'user/message', + data: { id, role: 'user', content: [{ type: 'text', text: 'same' }], source: { kind: 'user' } }, + }), + '', + ].join('\n') + const fresh = log(freshId) + const existing = log(existingId) + const refreshed = stabilize(fresh, existing) + + expect(refreshed).toContain(`"id":"${freshId}"`) + expect(stabilizeFixtureMessageIds([refreshed], [existing])[0]).toContain(`"id":"${existingId}"`) + }) + it('keeps volatile fixture fields while preserving fresh meaningful payloads', () => { const fresh = [ '{"type":"session","id":"new-child","createdAt":200,"cwd":"/new","parentSession":"new-parent","seedLength":1}', diff --git a/packages/support/acp-snapshot/tsconfig.json b/packages/support/acp-snapshot/tsconfig.json index 893282ce51..9d85ccaf1f 100644 --- a/packages/support/acp-snapshot/tsconfig.json +++ b/packages/support/acp-snapshot/tsconfig.json @@ -13,6 +13,9 @@ }, { "path": "../../support/invariants" + }, + { + "path": "../../core/session" } ] } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 078f775ecf..757da9b662 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -5907,6 +5907,9 @@ importers: '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../invariants + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session cordis: specifier: ^4.0.0-rc.7 version: link:../../../vendor/cordis From de93253b148c737355f0d700843211b5916ec3fd Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 8 Aug 2026 02:18:40 +0800 Subject: [PATCH 4/4] docs: refresh module graph --- docs/module-graph.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/module-graph.md b/docs/module-graph.md index d963273363..bd9bcbf070 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -300,7 +300,6 @@ flowchart TD pkg_timeout --> pkg_invariants pkg_scope --> pkg_invariants pkg_skill --> pkg_invariants - pkg_acp_snapshot --> pkg_invariants pkg_llm_mock_server --> pkg_invariants pkg_loader_smoke --> pkg_invariants pkg_base --> pkg_invariants @@ -422,6 +421,8 @@ flowchart TD pkg_session_persistence --> pkg_brand pkg_session_persistence --> pkg_invariants pkg_session_persistence --> pkg_session + pkg_acp_snapshot --> pkg_invariants + pkg_acp_snapshot --> pkg_session pkg_app_boot --> pkg_environment pkg_app_boot --> pkg_invariants pkg_app_boot --> pkg_paths @@ -1168,7 +1169,6 @@ flowchart TD | [`timeout`](../packages/util/timeout) | `util` | [`invariants`](../packages/support/invariants) | | [`scope`](../packages/core/scope) | `core` | [`invariants`](../packages/support/invariants) | | [`skill`](../packages/skill/skill) | `skill` | [`invariants`](../packages/support/invariants) | -| [`acp-snapshot`](../packages/support/acp-snapshot) | `support` | [`invariants`](../packages/support/invariants) | | [`llm-mock-server`](../packages/support/llm-mock-server) | `support` | [`invariants`](../packages/support/invariants) | | [`loader-smoke`](../packages/support/loader-smoke) | `support` | [`invariants`](../packages/support/invariants) | | [`base`](../packages/bundle/base) | `bundle` | [`invariants`](../packages/support/invariants) | @@ -1220,6 +1220,7 @@ flowchart TD | [`web-search-perplexity`](../packages/web/web-search-perplexity) | `web` | [`environment`](../packages/util/environment), [`invariants`](../packages/support/invariants), [`web`](../packages/web/web) | | [`spill`](../packages/spill/spill) | `spill` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session) | | [`session-persistence`](../packages/session-persistence/session-persistence) | `session-persistence` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`session`](../packages/core/session) | +| [`acp-snapshot`](../packages/support/acp-snapshot) | `support` | [`invariants`](../packages/support/invariants), [`session`](../packages/core/session) | | [`app-boot`](../packages/ui/app-boot) | `ui` | [`environment`](../packages/util/environment), [`invariants`](../packages/support/invariants), [`paths`](../packages/util/paths), [`system-prompt`](../packages/core/system-prompt) | | [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime), [`invariants`](../packages/support/invariants), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) | | [`lsp-local`](../packages/lsp/lsp-local) | `lsp` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`lsp`](../packages/lsp/lsp), [`subprocess`](../packages/subprocess/subprocess), [`timeout`](../packages/util/timeout) |