Commit Graph

7 Commits

Author SHA1 Message Date
Tianyi Cui
515d04339b Address review round 2: preserve delta insert arity; reject unpinned header-deltas
Residuals from the Codex re-review:

1. A system delta's insert was flattened to one token, so deltas differing
   only in inserted-line count compared equal. Now one {{system}} token per
   inserted line — position AND extent survive, content does not.

2. The live uniformity guard folded only request/header snapshots, so a
   mid-run header CHANGE (request/header-delta) could diverge from the pin
   invisibly. Non-pinning runs now assert zero header-delta events: a
   scenario that legitimately changes its header mid-run exists to show
   that change, so it must pin (fail-loud until it does).
2026-07-07 01:26:25 +08:00
Tianyi Cui
a0d8f33b29 Address review: structure-preserving delta scrub, live header-uniformity guard, RFC style
Codex review findings on the pinned-header change:

1. scrubRequestHeaders flattened a request/header-delta's whole
   system/tools payload to one token, so two meaningfully different deltas
   compared equal. Now the structural facts survive — keepStart/keepEnd
   line positions, added/removed/changed tool NAMES — and only the bulk
   (inserted prompt lines, schema bodies) is tokenized.

2. The one-pin design rested on an unasserted premise (all sessions
   compose the same header). Every non-pinning scenario now asserts, live,
   that each request/header its run produces equals the pinned fixture's
   header (both sides normalized against their own volatile values), so a
   session-dependent header fails loud until it gets its own pin.
   Verified the guard bites: perturbing the pinned fixture's prompt fails
   a non-pinned scenario with the intended message.

3. RFC de-slopped per docs/AGENTS.md: no PR reference, no SHOULD
   spec-speak; Decision/Verification/Consequences updated for 1 and 2.
2026-07-07 00:51:55 +08:00
Tianyi Cui
1fd9e2a9ab Pin request-header content in one snapshot scenario
Every session.jsonl fixture embedded the full composed system prompt and
complete tool-schema list in its request/header event (~8 KB on one line,
identical across the suite), so any prompt or tool-schema edit forced a
re-record or hand-edit of every fixture — see the dynamic-workflows PR for
the churn pattern this removes.

Now exactly one scenario (text-turn, flagged pinsHeader) commits and
compares that content verbatim; every other fixture stores and compares it
as {{system}}/{{tools}} tokens via the new pure scrubRequestHeaders
normalizer (applied to both compare sides and to record-mode writes, so a
re-record cannot reintroduce the content). request/header-delta payloads
are scrubbed the same way; config/reason stay verbatim — a model swap
SHOULD churn every fixture, a prompt edit should not. Replay is unaffected:
script derivation reads only assistant/chunk events.

Fixture meta-guards enforce the split: non-pinning fixtures must be fixed
points of the scrub, the pinning fixture must not be, and exactly one
scenario pins. Committed fixtures migrated through the same function.

Docs: pinned-header RFC (implemented/testing), base snapshot RFC + testing
policy + llm-replay module doc/README updated.
2026-07-06 23:41:20 +08:00
Tianyi Cui
8adcbceeed feat(hooks): dsh-hooks-claude + dsh-hooks-codex bridges (hooks stack PR-F)
The two bridge plugins that run a user's existing Claude Code / Codex hook
config on the harness's typed interception seams, built on the shared
dsh-hook-protocol library. A bridge is a faithfulness adapter, not a power
tool: anything it does a native cordis plugin does more powerfully — the
bridge exists only to run UNMODIFIED external hooks.

- dsh-hooks-claude: CC dialect. Seven hook points (SessionStart,
  UserPromptSubmit, PreToolUse, PostToolUse, Stop, SubagentStart,
  SubagentStop), CC per-event stdin payloads, env + ${CLAUDE_PLUGIN_ROOT}/
  ${CLAUDE_PROJECT_DIR} substitution, literal-or-regex matcher.
- dsh-hooks-codex: Codex dialect — a deliberate subset. Five hook points,
  always-regex matcher, snake_case payloads (turn_id/model, no trailing
  newline), no env/substitution, block-only decisions.

Both map the neutral merged outcome onto the seam's typed Decision and stamp
an explicit {kind:'plugin'} source on injected context (so it is never
mislabeled as a user prompt). Config parse-failure is contained; only command
hooks run. updatedInput is logged+warned (input rewrite deferred); the Stop
loop-guard is deferred (TODO).

Tests: per-file 100% — config-parse unit branches + per-seam mappings
end-to-end through the REAL loop + REAL bash + REAL shell scripts (scripted
mock model only) + a real-Loader export-shape guard. A keyless ACP snapshot
scenario (hook-prompt-block) proves a UserPromptSubmit hook blocks a prompt
end-to-end (rejected turn -> ACP cancelled, hook/* events in the log); a
with-key e2e (hooks.e2e.ts) proves a PreToolUse hook blocks real bash
(verified on disk). The snapshot normalizer now scrubs hook/result.durationMs.

RFC: docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
2026-07-01 04:23:49 +08:00
Tianyi Cui
605587e79c docs(rfc): classify RFCs by kind via path-encoded subdirectories
Add a second axis to every RFC — its class (feature, bug-fix,
simplification, architecture, process, testing) — encoded in the path
as docs/rfc/{lifecycle}/{class}/file.md. The folder is the label, so
the closed set is enforced by structure rather than a parsed field.

Two new doc-sync gates back it:
- verify-rfc-classification: every RFC sits in a valid class folder and
  the README index lists it under the matching lifecycle→class heading.
- verify-doc-refs: every docs/*.md path cited in a packages|examples TS
  comment resolves — closes a drift class verify-md-links can't see, and
  catches the four comment refs this reorg moved.

The README gains a Classification section explaining the taxonomy and
per-class index sub-sections. A self-referential process RFC records why
the scheme is path-encoded and gated.
2026-06-20 22:29:45 +08:00
Tianyi Cui
ec73e11c9b refactor(acp-example): snapshot goldens are JSONL, not pretty-printed .txt
The goldens now mirror the shape of the surfaces they capture — one compact
JSON record per line — matching the wire (NDJSON stdout) and disk (JSONL
session log) formats, renamed *.golden.jsonl. They stay grep/jq-able and
faithful to what the agent emits, where the prior pretty-printed .txt was a
reformatted representation. Both normalizers drop the 2-space indent; the
normalizer spec asserts the compact form. All 11 goldens regenerated; replay
remains deterministic (8/8 across runs).
2026-06-19 09:29:41 +08:00
Tianyi Cui
81d434896d feat(acp-example): snapshot harness, normalizers, wiring, and handshake scenario
Adds the snapshot-test harness and the keyless replay pipeline end-to-end.

- snapshot-harness.ts: boots the real acp-agent subprocess via the cordis
  Loader (preserving TSX_TSCONFIG_PATH so unbuilt dsh-* imports resolve from a
  temp cwd), tees raw stdout into an SDK ClientSideConnection, interprets a
  per-scenario input.json DSL (initialize / newSession capturing the random
  sessionId / prompt / cancel), closes stdin to trigger graceful shutdown, and
  harvests the persisted session.jsonl. Failure-safe: a finally block SIGKILLs
  a live child, awaits its exit, and removes both temp dirs even on a thrown
  step or harvest. Raw bytes are buffered and decoded once (no multibyte split).
- snapshot-normalize.ts (+ spec): two pure normalizers (stdout frames + session
  JSONL) scrub cwd, session ids / UUIDs, and JSON-RPC ids, and zero time /
  createdAt — but keep `seq` (deterministic by contract). normalizeStdout throws
  on a non-JSON line (the stdout-purity check).
- start.ts: selects cordis.snapshot.yml (replay, providerless) or
  cordis.snapshot-record.yml (record, real adapter) from DSH_SNAPSHOT, skips
  .env in replay, and disposes the ctx on stdin end so persistence flushes
  before exit (harvest-after-flush, not on the prompt response).
- acp.snapshot.ts: asserts the normalized stdout golden (and, for model
  scenarios, the re-persisted JSONL golden) via toMatchFileSnapshot; record mode
  writes the harvested log back to the scenario fixture; an orphan-fixture guard
  fails on an unregistered scenario dir.
- handshake scenario: initialize + session/new (no model call; a header-only
  session.jsonl, since session/new persists no events).
- vitest.snapshot.config.ts, test:snapshot / test:snapshot:record scripts, a
  pre-push snapshot job, and the knip entry.

Incorporates Codex review: record-fixture writeback, failure-safe teardown,
seq-not-scrubbed, harvest-after-flush. Per docs/rfc/implemented/2026-06-19.
2026-06-19 03:36:12 +08:00