Record fork and mixed spawn+fork snapshot scenarios

The seed-boundary change made fork-child replay route correctly but shipped
with no recorded fork scenario — the seedLength slice was exercised only by
llm-replay unit tests and a persistence round-trip, never by the full-transcript
snapshot tier. Add two recorded scenarios that drive a real fork child through
it:

- subagent-fork: parent completes a turn, then forks one child (child fixture
  carries a non-zero seedLength, the boundary the replay slice consumes).
- subagent-mixed: parent completes a turn, then delegates once via spawn
  (seedLength 0) and once via fork (non-zero seedLength) in one transcript —
  the first scenario to drive two subagent backends at once, exercising both
  branches of the slice.

Both need a completed turn-1 so the fork seed is a non-empty completed-turn
prefix (a turn-1 fork seeds empty = spawn, which would not exercise the slice).
Removing the slice turns both scenarios red (the fork child receives the
parent's recorded chunks), proving the guard bites.

ACP (out-of-process) subagent replay remains a different shape, still tracked
as TODO(acp-subagent-replay).
This commit is contained in:
Tianyi Cui
2026-06-22 21:20:54 +08:00
parent b3d40d427e
commit 78d60366ec
14 changed files with 1147 additions and 2 deletions

View File

@@ -143,6 +143,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
| [Use `session.jsonl` as the only snapshot session-log artifact](implemented/testing/2026-06-20-remove-redundant-snapshot-log-goldens.md) | 2026-06-20 |
| [Per-session snapshot replay for nested agents](implemented/testing/2026-06-22-subagent-snapshot-replay.md) | 2026-06-22 |
| [Persist the seed boundary so fork-child replay routes correctly](implemented/testing/2026-06-22-fork-child-replay-seed-boundary.md) | 2026-06-22 |
| [Record fork and mixed spawn+fork snapshot scenarios](implemented/testing/2026-06-22-fork-snapshot-scenarios.md) | 2026-06-22 |
## Rejected

View File

@@ -33,7 +33,7 @@ The SQLite change is a breaking table-layout change, so `SCHEMA_VERSION` bumps *
`dsh-llm-replay`'s `parseSessionHeader` now also reads `seedLength` (absent ⇒ 0), and `loadSessionScripts` derives a child's entries from `parseSessionLog(text).slice(seedLength)` — the events at or after the boundary, i.e. the child's own model calls. For a spawn child `seedLength` is 0 and this is a no-op, so spawn scenarios are byte-for-byte unchanged.
This closes the routing correctness gap; an actual fork *scenario* (a recorded `subagent-multi`-style fixture with a fork child) is still a future addition, but it can now be recorded and replayed correctly rather than mis-routing.
This closes the routing correctness gap, and two recorded fork scenarios exercise it end to end — see [Record fork and mixed spawn+fork snapshot scenarios](2026-06-22-fork-snapshot-scenarios.md).
## Alternatives considered

View File

@@ -0,0 +1,27 @@
# RFC: Record fork and mixed spawn+fork snapshot scenarios
Status: implemented
## Problem
The [seed-boundary RFC](2026-06-22-fork-child-replay-seed-boundary.md) made fork-child replay route correctly: `dsh-llm-replay` derives a child's script from the events at or after its persisted `seedLength` boundary, so a fork child's inherited parent prefix is not replayed as the child's own model calls. But it shipped with **no recorded fork scenario** — the slice was exercised only by `llm-replay`'s unit tests (a synthetic child fixture) and a persistence round-trip test. The full-transcript snapshot tier, the one net that boots the real `acp-agent` and replays an end-to-end nested transcript, had only spawn children (`subagent-spawn`, `subagent-multi`). A fork-routing regression that left the unit tests green would still have escaped the tier built to catch transcript regressions.
The snapshot infrastructure to express a fork scenario was already in place — both in-process backends are wired into `cordis.yml` / `cordis.snapshot.yml` as two model-facing tools (`subagent` → spawn, `subagent_fork` → fork), the harness harvests every child log, and replay forwards per-child fixtures keyed by `seedLength`. What was missing was a *recorded scenario* that drives a fork child through it.
## Decision
Record two scenarios against the real API, both replayed keyless in the default gate:
- **`subagent-fork`** — the parent completes a turn that establishes a fact, then delegates one subtask via `subagent_fork`. The fork child inherits the conversation (its log carries a non-zero `seedLength`), so it can answer from the parent's context. This is the focused regression: the child fixture's `seedLength` is the boundary the replay slice depends on, recorded from a real fork rather than hand-synthesized.
- **`subagent-mixed`** — the parent completes a turn, then delegates once via `subagent` (a fresh spawn child, `seedLength` 0) and once via `subagent_fork` (a fork child, non-zero `seedLength`) in one transcript. This is the mixed spawn+fork scenario the seed-boundary and per-session-replay RFCs both named as a future addition: one transcript exercises both transports and both branches of the slice (`seedLength` 0 = no-op, `seedLength > 0` = trim the inherited prefix), with the two children ordered spawn-then-fork by `createdAt`.
### Why a completed turn-1 is required
The fork backend seeds the child with the parent's **balanced completed-turn prefix** ([`completedTurnPrefix`](../../../../packages/subagent/subagent-fork)). A parent that forks on its very first turn has no completed turn to inherit, so the seed is empty (≡ a fresh spawn, `seedLength` 0) — which would NOT exercise the slice. Both scenarios therefore use a two-prompt input: the first prompt completes a turn (establishing a codeword the child is later asked to recall), the second delegates the fork. The recalled codeword in the child's transcript is incidental to the model's behavior; the load-bearing artifact is the child fixture's recorded `seedLength`, which the replay slice consumes.
## Consequences
- The fork-routing slice is now guarded at the full-transcript tier, not just by unit tests. Removing the `slice(seedLength)` (replaying the whole child log) turns **both** new scenarios red — the fork child receives the parent's recorded chunks instead of its own — proving the guard bites (verified red→green when the scenarios landed).
- `subagent-mixed` is the first snapshot scenario to drive two *different* subagent backends in one transcript, exercising the per-session replay keying across a spawn and a fork child simultaneously.
- Out-of-process (ACP) subagent replay remains a different shape (each child is its own process with its own replay) and is still tracked as `TODO(acp-subagent-replay)` — these scenarios are in-process only.
- Re-recording (`pnpm run test:snapshot:record`) regenerates all four fork/spawn fixtures from the live API; the two new scenarios self-skip without a key like every recorded scenario.

View File

@@ -50,5 +50,5 @@ Both replay keyless in the default gate.
- The `TODO(subagent-snapshots)` deferral is resolved: nested-agent transcripts are now a first-class snapshot shape.
- `GenerateOptions.sessionId` is a small, honest core-seam addition useful beyond replay (telemetry, request routing).
- The `subagent` tool is bound to a single provider, so both children in `subagent-multi` are spawn (fresh). The keying routes by session, not by backend, so it is already correct for fork. The script *derivation* was not: a fork child's log begins with the seeded parent prefix (the parent's `assistant/chunk` events), so deriving its script from the whole log would replay the parent's responses as the child's. That correctness gap is closed by persisting a seed boundary — see [Persist the seed boundary so fork-child replay routes correctly](2026-06-22-fork-child-replay-seed-boundary.md). A recorded mixed spawn+fork *scenario* (a second tool instance bound to `fork`, pure config) remains a future addition, but a fork child now derives correctly.
- The `subagent` tool is bound to a single provider, so both children in `subagent-multi` are spawn (fresh). The keying routes by session, not by backend, so it is already correct for fork. The script *derivation* was not: a fork child's log begins with the seeded parent prefix (the parent's `assistant/chunk` events), so deriving its script from the whole log would replay the parent's responses as the child's. That correctness gap is closed by persisting a seed boundary — see [Persist the seed boundary so fork-child replay routes correctly](2026-06-22-fork-child-replay-seed-boundary.md) — and recorded fork + mixed spawn+fork scenarios now exercise both transports through one transcript (see [Record fork and mixed spawn+fork snapshot scenarios](2026-06-22-fork-snapshot-scenarios.md)).
- Out-of-process (ACP) subagents are a different replay shape entirely (each child is its own PROCESS with its own replay), tracked as `TODO(acp-subagent-replay)` in the PR3 plan.