mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
81d434896d
Adds the snapshot-test harness and the keyless replay pipeline end-to-end. - snapshot-harness.ts: boots the real acp-agent subprocess via the cordis Loader (preserving TSX_TSCONFIG_PATH so unbuilt dsh-* imports resolve from a temp cwd), tees raw stdout into an SDK ClientSideConnection, interprets a per-scenario input.json DSL (initialize / newSession capturing the random sessionId / prompt / cancel), closes stdin to trigger graceful shutdown, and harvests the persisted session.jsonl. Failure-safe: a finally block SIGKILLs a live child, awaits its exit, and removes both temp dirs even on a thrown step or harvest. Raw bytes are buffered and decoded once (no multibyte split). - snapshot-normalize.ts (+ spec): two pure normalizers (stdout frames + session JSONL) scrub cwd, session ids / UUIDs, and JSON-RPC ids, and zero time / createdAt — but keep `seq` (deterministic by contract). normalizeStdout throws on a non-JSON line (the stdout-purity check). - start.ts: selects cordis.snapshot.yml (replay, providerless) or cordis.snapshot-record.yml (record, real adapter) from DSH_SNAPSHOT, skips .env in replay, and disposes the ctx on stdin end so persistence flushes before exit (harvest-after-flush, not on the prompt response). - acp.snapshot.ts: asserts the normalized stdout golden (and, for model scenarios, the re-persisted JSONL golden) via toMatchFileSnapshot; record mode writes the harvested log back to the scenario fixture; an orphan-fixture guard fails on an unregistered scenario dir. - handshake scenario: initialize + session/new (no model call; a header-only session.jsonl, since session/new persists no events). - vitest.snapshot.config.ts, test:snapshot / test:snapshot:record scripts, a pre-push snapshot job, and the knip entry. Incorporates Codex review: record-fixture writeback, failure-safe teardown, seq-not-scrubbed, harvest-after-flush. Per docs/rfc/implemented/2026-06-19.
33 lines
1.4 KiB
TypeScript
33 lines
1.4 KiB
TypeScript
import tsconfigPaths from 'vite-tsconfig-paths'
|
|
import { defineConfig } from 'vitest/config'
|
|
|
|
// Snapshot tests: `pnpm run test:snapshot`, file pattern *.snapshot.ts.
|
|
// REPLAY by default — they boot the real acp-agent subprocess against a
|
|
// recorded session JSONL fixture (no API key, no network) and diff the
|
|
// normalized stdout transcript + re-persisted log against committed goldens.
|
|
// `pnpm run test:snapshot:record` (DSH_SNAPSHOT=record + -u) re-records the
|
|
// fixtures against the real API and refreshes the goldens.
|
|
//
|
|
// Replay loads no .env; record reads DEEPSEEK_API_KEY from the env or a
|
|
// gitignored repo-root .env (loaded here, mirroring the e2e config), so a
|
|
// contributor with a key only in .env can still record.
|
|
try {
|
|
process.loadEnvFile(new URL('.env', import.meta.url).pathname)
|
|
} catch {
|
|
// No .env — fine; replay needs no key and record reads it from the env.
|
|
}
|
|
|
|
export default defineConfig({
|
|
// Same resolution note as vitest.config.ts: bare workspace names resolve
|
|
// through the root tsconfig paths map; the native option cannot do this.
|
|
plugins: [tsconfigPaths({ projects: ['./tsconfig.test.json'] })],
|
|
test: {
|
|
include: ['examples/*/tests/**/*.snapshot.ts'],
|
|
// Each test boots a subprocess; give it room, and run files one at a time
|
|
// (a record run hits the live API, and replay subprocess boot is heavy).
|
|
testTimeout: 120_000,
|
|
hookTimeout: 30_000,
|
|
fileParallelism: false,
|
|
},
|
|
})
|