mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
c182543dd5
Per a design revision, the per-scenario snapshot fixture becomes EXACTLY the persisted session JSONL (<scenario>/session.jsonl) rather than a hand-authored llm.json. The log already holds all LLM behavior (assistant/chunk carries every StreamChunk) AND all harness behavior (tool/call, tool/result, turn/*, usage), so one artifact drives replay and doubles as a behavioral golden. llm-replay becomes replay-only (the record-tee is removed; recording is now "run the real agent once and harvest the .jsonl", done by the harness in a later commit). deriveReplayScript(events) groups assistant/chunk by (turn,step) in log order — exact because the loop makes one ctx.llm.stream() call per step and tags each chunk with the current (turn,step). The two failure modes the log can't express (a thrown stream — no terminal finish; cancel/hang — timing) use an optional replay.override.json sidecar. Hardens against a Codex review finding: a derived group is only valid if it ends in a `finish` chunk. A group without one is the fingerprint of a thrown stream() and is NOT silently replayed as a clean stop — deriveReplayScript throws, naming the (turn,step), so a missing sidecar override fails loud. Updates the unit tests (parse/derive/load helpers, sidecar override, finish- terminated grouping, HMR), the example README, and the RFC prose to the JSONL format. Two goldens (stdout transcript + re-persisted JSONL) and the harness wiring land in the next commit.