diff --git a/docs/core-data-structures/core.md b/docs/core-data-structures/core.md index 6d20900c93..ee4f06e14a 100644 --- a/docs/core-data-structures/core.md +++ b/docs/core-data-structures/core.md @@ -205,7 +205,7 @@ type SessionEvent = { /** * Seq numbers of events that are provenance sources of this event * (e.g. the `assistant/chunk` seqs that built an `assistant/message`, - * or the surface nodes shadowed by a compaction marker). + * or the surface nodes shadowed by a compaction replace node). */ sourceEventSeqs?: number[] /** How this event entered the surface; absent for non-surface events. */ diff --git a/docs/core-data-structures/session.md b/docs/core-data-structures/session.md index 8f8ef4a800..890a3a0dee 100644 --- a/docs/core-data-structures/session.md +++ b/docs/core-data-structures/session.md @@ -55,7 +55,7 @@ type SessionEvent = { /** * Seq numbers of events that are provenance sources of this event * (e.g. the `assistant/chunk` seqs that built an `assistant/message`, - * or the surface nodes shadowed by a compaction marker). + * or the surface nodes shadowed by a compaction replace node). */ sourceEventSeqs?: number[] /** How this event entered the surface; absent for non-surface events. */ diff --git a/examples/AGENTS.md b/examples/AGENTS.md index 67ea68ae6f..2c2b2a44f1 100644 --- a/examples/AGENTS.md +++ b/examples/AGENTS.md @@ -20,7 +20,7 @@ A keyless smoke that spawns the example from a temp cwd must set `TSX_TSCONFIG_P | Example | Keyless smoke | With-key smoke | |---|---|---| | `echo-agent` | `tests/echo.e2e.ts` — boots the real `cordis.yml`, drives the echo tool round-trip and the direct canned reply | **N/A — keyless by nature** (the `mock-echo` model has no real provider) | -| `coding-agent` | `tests/keyless-smoke.e2e.ts` — boots the full real tree (dummy key, no prompt → no model call), asserts banner + clean exit | `tests/{full-loop,coding-task,resume}.e2e.ts` — real model + real bash, world-verified | +| `coding-agent` | `tests/keyless-smoke.e2e.ts` — boots the full real tree (dummy key, no prompt → no model call), asserts banner + clean exit | `tests/{full-loop,coding-task,resume,compaction}.e2e.ts` — real model + real bash, world-verified | | `acp-agent` | `pnpm run test:snapshot` — boots the real ACP subprocess and replays a recorded session keyless; `tests/acp.e2e.ts` also asserts stdout purity without a key | `tests/acp.e2e.ts` — real ACP prompt, verifies a file the agent wrote | See [the root AGENTS.md](../AGENTS.md) for repo-wide conventions and [docs/architecture.md](../docs/architecture.md) for the design. diff --git a/examples/coding-agent/README.md b/examples/coding-agent/README.md index 7585129382..76fbb7a237 100644 --- a/examples/coding-agent/README.md +++ b/examples/coding-agent/README.md @@ -48,5 +48,6 @@ This example is a thin leaf `cordis.yml`: it picks the swappable backends and lo - `tests/full-loop.e2e.ts` — the canary: real model runs `echo e2e-ok` through the real bash tool; asserts `tool/call`/`tool/result` session events and the final answer. - `tests/coding-task.e2e.ts` — the swebench-style smoke: a temp dir holds `add.js` (with `a - b` where `a + b` belongs) and a failing `add.test.js`; the agent must fix the bug and verify. The test re-runs `node add.test.js` ITSELF and inspects the files — agent claims are not trusted. - `tests/resume.e2e.ts` — durable continuity across processes: run 1 tells the real model a secret code and persists the turn to a temp JSONL root, then the whole context is disposed; run 2 is a fresh context over the same root that RESUMES the session id and asks the model to recall the code. The recall can only come from the rehydrated log. +- `tests/compaction.e2e.ts` — the compaction smoke: a real multi-step bash task runs with a deliberately tiny context window so the auto-compaction listener fires MID-SESSION. Verifies the WORLD — a `compact/start…end` pair landed in the real log, the surface shrank (a replace node shadowed older nodes), and the agent still produced a correct final answer after compaction. -Both self-skip without `DEEPSEEK_API_KEY`. +All four self-skip without `DEEPSEEK_API_KEY`. The keyless boot smoke is `tests/keyless-smoke.e2e.ts` (boots the full real tree with a dummy key and no prompt, so no model call), which runs in the default e2e gate. diff --git a/examples/coding-agent/tests/compaction.e2e.ts b/examples/coding-agent/tests/compaction.e2e.ts index fefcd579d0..2c9814c79d 100644 --- a/examples/coding-agent/tests/compaction.e2e.ts +++ b/examples/coding-agent/tests/compaction.e2e.ts @@ -33,21 +33,23 @@ afterEach(async () => { describe.skipIf(!process.env.DEEPSEEK_API_KEY)('compaction: a long session compacts mid-flight and keeps running', () => { it('summarizes older history into a checkpoint without breaking the task', async () => { workdir = await mkdtemp(join(tmpdir(), 'dsh-compaction-')) - // A few files for the model to read, so multiple bash steps accumulate - // surface nodes (tool calls + results) and grow the history. - for (let i = 1; i <= 4; i++) { + // A handful of files for the model to read, so multiple bash steps + // accumulate surface nodes (tool calls + results) and grow the history past + // the (deliberately tiny) window. + for (let i = 1; i <= 6; i++) { await writeFile(join(workdir, `file${i}.txt`), `This is file number ${i}. `.repeat(40)) } - // Tiny window so a handful of steps crosses the threshold. The convergence - // invariant requires summarizationMaxTokens + retainTokens <= window * - // ratio = floor(8000 * 0.5) = 4000; 1500 + 2000 = 3500 <= 4000. + // Tiny window so a couple of steps crosses the threshold. The convergence + // invariant requires summarizationMaxTokens + retainTokens to be strictly + // BELOW the threshold = floor(contextWindow * thresholdRatio) = + // floor(2400 * 0.5) = 1200; 300 + 500 = 800 < 1200. ctx = await codingHarness(workdir, { compact: { - contextWindow: 8000, + contextWindow: 2400, thresholdRatio: 0.5, - retainTokens: 2000, - summarizationMaxTokens: 1500, + retainTokens: 500, + summarizationMaxTokens: 300, }, }) const agent = ctx.agentLoop.create(AgentId('e2e-compaction'), { @@ -57,9 +59,9 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('compaction: a long session compa agent.send([{ type: 'text', - text: 'Read file1.txt, file2.txt, file3.txt, and file4.txt one at a time using cat ' - + '(a separate bash command for each). After reading all four, tell me how many ' - + 'files you read and the number mentioned in file1.txt.', + text: 'Read file1.txt, file2.txt, file3.txt, file4.txt, file5.txt, and file6.txt one at a ' + + 'time using cat (a separate bash command for each). After reading all six, tell me how ' + + 'many files you read and the number mentioned in file1.txt.', }]) await waitForIdle(ctx, agent) @@ -87,9 +89,9 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('compaction: a long session compa expect(summaryData.shadowedSeqs.length).toBeGreaterThan(0) // The conversation survived compaction: the agent produced a final answer - // that reflects the work (it read four files). + // that reflects the work (it read six files). const answer = finalText(events).toLowerCase() expect(answer.length).toBeGreaterThan(0) - expect(answer).toMatch(/\b(4|four)\b/) + expect(answer).toMatch(/\b(6|six)\b/) }, 240_000) }) diff --git a/packages/core/session/README.md b/packages/core/session/README.md index 2a90d0f792..cde29d091a 100644 --- a/packages/core/session/README.md +++ b/packages/core/session/README.md @@ -51,13 +51,13 @@ Plain class (not a Cordis Service). Create via `ctx.sessions.create()`. The append-only log: `turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `assistant/message`, `assistant/chunk`, `tool/call`, `tool/result`, `steering/message`, `context/message`. Token usage rides on `assistant/message.usage`; an operational error's step is on `turn/end.reason` for `kind: 'error'`. -Merge-extensible via `SessionEventMap` — a compaction plugin adds `compaction/marker`, etc. +Merge-extensible via `SessionEventMap` — the compaction plugin (`dsh-compact-basic`) adds `compact/start`, `compact/summary`, `compact/end`, etc. Also defines `TurnTriggerMap` and `TurnEndReasonMap` (merge-extensible sum types for typed turn boundaries — `kind`-tagged instead of strings). Every `SessionEvent` carries two optional top-level fields (structural metadata): -- `sourceEventSeqs?: number[]` — seq numbers of provenance sources (e.g., the `assistant/chunk` seqs behind an `assistant/message`, or the shadowed nodes behind a compaction marker). +- `sourceEventSeqs?: number[]` — seq numbers of provenance sources (e.g., the `assistant/chunk` seqs behind an `assistant/message`, or the shadowed nodes behind a compaction replace node). - `surfaceOp?: SurfaceOp` — how this event entered the surface. Absent for non-surface events (boundaries, chunks, usage, errors). ### Metadata types (`types.ts`) @@ -68,7 +68,7 @@ Every `SessionEvent` carries two optional top-level fields (structural metadata) - Persistence plugins: subscribe to `session/event` (write-behind) and drain on `session/flush` (awaited) and fiber dispose. A durable backend reads the log and reloads it into a live session; the metadata seam (`SessionHeader`, `session.header`) is what such a backend stores beside the log. - Replay/fork: `ctx.sessions.create(id, { seed })` seeds a new session with an existing event log. The surface rebuilds deterministically from `surfaceOp` markers in the seeded events. The seed is validated to the SAME invariants `append` enforces — including that every surface-eligible event (`SurfaceEventType`) carries a `surfaceOp` marker — so a marker-less message event is rejected at construction rather than silently vanishing from `deriveMessages()` (the surface is the sole derivation path) on resume. -- Compaction: a future plugin appends a new event with `surfaceOp: { op: 'replace', start, end }` to shadow old surface nodes. +- Compaction: the `dsh-compact-basic` plugin appends a `user/message` with `surfaceOp: { op: 'replace', start, end }` to shadow old surface nodes behind a summary checkpoint. ### What is NOT here (TODO) diff --git a/packages/core/session/src/types.ts b/packages/core/session/src/types.ts index 41f878892f..2a32c7b3f0 100644 --- a/packages/core/session/src/types.ts +++ b/packages/core/session/src/types.ts @@ -156,7 +156,8 @@ export type TurnEndReason = TurnEndReasonMap[keyof TurnEndReasonMap] * same events; trace/telemetry = subscribe to the log. * * Merge-extensible: plugins declare extra event types via declaration merging - * (e.g. a compaction plugin adds `'compaction/marker'`). + * (e.g. the compaction plugin adds `'compact/start'`, `'compact/summary'`, + * `'compact/end'`). * * Durability contract (what a persistence backend relies on): the durable log * persists every event verbatim, INCLUDING `assistant/chunk` — `seq` must stay @@ -279,7 +280,7 @@ export type SessionEvent = { /** * Seq numbers of events that are provenance sources of this event * (e.g. the `assistant/chunk` seqs that built an `assistant/message`, - * or the surface nodes shadowed by a compaction marker). + * or the surface nodes shadowed by a compaction replace node). */ sourceEventSeqs?: number[] /** How this event entered the surface; absent for non-surface events. */