mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
bef9386591
Records the decision to add a third test tier: snapshot tests that boot the real acp-agent subprocess over ACP stdio, record the LLM's streamed responses once against the real API, then replay them deterministically so the full stdout transcript can be diffed against a committed golden — keyless in CI. Captures the design choices hardened in a Codex (xhigh) review: record at the provider-neutral llm/stream waterfall; a discriminated fixture entry schema (chunks/throw/hang) that honors both LLM failure branches; positional replay with a one-in-flight-stream constraint; per-stream atomic fixture flush (the subprocess is SIGKILLed, so dispose-time flush would never run); a providerless replay config; normalize-then-snapshot parsed frames; normalization over an OS sandbox now with the rootless bwrap/sandbox-exec tier reserved via the BashExecutor capability seam. Cross-links the proposed determinism RFC (complementary: internal history invariant vs external protocol contract).