DeepSeek/deepseek-harness
1
0
Fork 0
mirror of https://github.com/deepseek-ai/deepseek-harness synced 2026-08-15 21:04:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #66 from deepseek-ai/worktree-drop-session-summary

Browse Source 
refactor(session): drop the dead mutable SessionSummary
This commit is contained in:
Tianyi Cui
2026-06-20 14:11:37 +08:00
committed by GitHub
parent 0561fb47b6 38cc62b644
commit 1017ebbc74
21 changed files with 169 additions and 523 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
AGENTS.md
View File

@@ -6,6 +6,16 @@ This is the monorepo for the DeepSeek Harness group. It currently hosts the code
**This applies only while the harness is unreleased — remove this section at the first tagged/published release.** There are no external consumers yet, so optimize for the *correct foundation*, not for a small diff. When the right structure means moving a file across package boundaries, renaming a public symbol, or repackaging a plugin, do it — and update every reference in the same change. Do **not** add backward-compat shims, deprecation aliases, re-export stubs, or "keep it where it is to avoid churn" hedges; those are debts you take on to protect callers you do not have. Churn now is cheap; a wrong foundation set in stone is not. (Once released, this inverts — backward compatibility becomes a real constraint and this section comes out.)
This extends to **on-disk formats, schemas, and stored data**: while unreleased there is no persisted user data to preserve, so a format/schema/contract change needs **no migration path**. Bump the version and reject (don't migrate) anything not at the current version — e.g. the SQLite backend's `SCHEMA_VERSION` bump that drops columns simply rejects any non-current `user_version` on open, with no v1→v2 migration. A migration written now is a shim for data that does not exist.
## Tests document behavior, not golden truth
A passing test pins the behavior the code **currently** has — not necessarily the behavior it **should** have. Existing tests faithfully document existing behavior, but existing behavior is not automatically golden: it can be the residue of a past compromise, a half-built feature, or a limitation that no longer applies. So when a refactor or review makes you ask "can I change this?", a green test is **not** the answer — the question is whether the behavior the test pins is actually correct.
Before you preserve a behavior solely to keep a test green, ask: is this behavior load-bearing (a real consumer depends on it, a contract promises it, a user observes it), or is it an artifact? If it's an artifact, **change the behavior AND its test together, in the same change, and say why in the PR** — do not contort new code to keep an obsolete assertion passing, and do not treat "but the test expects X" as a reason X must stay. Conversely, do not delete a test just because it is inconvenient: the discipline cuts both ways — you must show the *behavior* is dead, not merely that the test is in your way.
The worked example is [Drop the mutable session summary](docs/rfc/implemented/2026-06-19-drop-mutable-session-summary.md): an entire `SessionSummary` type, a `SessionPersistence.update()` method, a JSONL sidecar, and SQLite columns existed and were exercised by their own contract test — yet **nothing in production CONSUMED any of it, and `update()` had no production caller**. (The backends did *write* summary state — JSONL touched the sidecar after a durable append, SQLite bumped `updated_at` in the append transaction — but those writes fed only reads that nothing performed.) The tests documented the behavior perfectly; the behavior was dead. Deleting the behavior and its tests together removed ~400 lines and erased a durability divergence the next refactor would have had to model. (This is the test-tier echo of "verify the world, not a synthetic stand-in" in § Defensive patterns: a test agrees with whatever it was written to assert; only a real consumer proves the behavior matters.)
## Architecture
This codebase is based on the **Cordis** framework, built microkernel-style: **everything is a plugin**. All necessary Cordis dependencies are copied into this monorepo as vendored source (under `vendor/`) instead of being depended on via npm.
@@ -166,7 +176,7 @@ In the **core** packages (`packages/llm`, `packages/tools`, `packages/agent`, `p
Verbose documentation is fine **as long as docs and code stay strictly in sync**. Out-of-sync docs are worse than no docs. **When you change code, update its docs in the SAME change** — grep the package README and the module/JSDoc comments for the old behavior (config keys, defaults, error codes, wire field names, event names) and fix every hit. CI runs `pnpm run doc-sync` (`doc-typecheck` + `verify-event-taxonomy` + `verify-md-wrap` + `verify-md-links`), which typechecks every fenced `ts` block in `README.md`, `docs/**/*.md`, and `packages/*/README.md`, verifies the event-taxonomy table against source, asserts no hard-wrapped prose paragraphs, and checks that every relative Markdown cross-link resolves — across those files plus `AGENTS.md` / `packages/AGENTS.md` — but that scope does NOT catch prose drift in `AGENTS.md` / `packages/AGENTS.md` / `packages/README.md` (config keys, defaults, error codes), so keeping those in sync remains on the author. Every module has a module-level doc comment explaining its role. Every exported class, interface, type, function, and non-obvious method has a JSDoc that explains semantics (not just the name) — contracts (what events fire when), disposal behavior, error behavior, and extension intent. Internal helpers get docs only where non-obvious. Prefer one-liners when one line suffices.
**Write an RFC when — and only when — a PR makes a decision that is durable, contested, and surprising.** RFCs (`docs/rfc/`, grouped into `proposed/` / `implemented/` / `rejected/`) record the *why* behind choices a future reader would otherwise re-litigate (the vendoring policy, event-sourcing, the schema DSL are the existing examples). A PR that introduces such a decision — a new third-party runtime dependency over the vendoring default, a cross-package contract, a security/isolation model, a deviation from a documented architecture rule — writes the RFC in `implemented/` **in the same PR**, and links it from the relevant code. A proposal for future work not yet built goes in `proposed/`. A PR whose changes are mechanical, self-evident, or already covered by an existing RFC needs none — do not manufacture an RFC for a routine change. When unsure, the test is: would a competent maintainer six months from now ask "why was it done this way?" and be unable to answer from the code alone? If yes, write it. See [docs/rfc/README.md](docs/rfc/README.md) for the naming scheme and [docs/AGENTS.md](docs/AGENTS.md) for the cross-link convention.
**Document the CURRENT state — the "what" and "why" — never the PROCESS or HISTORY of how it got there.** A comment, JSDoc, or doc paragraph describes what the code *is* and why it is that way, as if it had always been so. Do NOT narrate the change that produced it: no "previously X, now Y", "changed from", "used to", "this replaces", "the old map", "renamed", "moved here", "as of this PR", or "(was …)". **In particular, NEVER name the change unit a reader cannot see — the PR, commit, or stack position that introduced the code — in a comment, JSDoc, OR a test name/description.** A `// (PR D's per-agent teardown)` aside, a `* Tests for the cancel primitive (PR C).` module doc, or an `it('… identity no longer matters')` title that only makes sense relative to a prior design are all the same violation: the reader of the current tree has no "PR D" or "old design" to anchor against, and the reference rots the moment the stack merges. Name the *mechanism* (`the session's AgentHandle teardown`), not the PR. Such phrasing rots the instant the next change lands, and a reader of the current code does not need the diff narrated in prose — that belongs in the commit message, the PR description, or an RFC (the durable home for "why we moved away from X"). Write "the owner token lives on the task in the executor" — not "ownership *now* lives on the executor instead of a plugin-local map". When a contrast genuinely aids understanding (a non-obvious choice between live alternatives), frame it against the alternative as a standing fact ("stored on the executor, NOT the tool plugin, so it survives an HMR reload"), not against the codebase's past. The same rule governs review-fix commits: the *commit message* records what the review caught; the *code comment* it touches states only the resulting truth. RFCs (`docs/rfc/`, grouped into `proposed/` / `implemented/` / `rejected/`) record the *why* behind choices a future reader would otherwise re-litigate (the vendoring policy, event-sourcing, the schema DSL are the existing examples). A PR that introduces such a decision — a new third-party runtime dependency over the vendoring default, a cross-package contract, a security/isolation model, a deviation from a documented architecture rule — writes the RFC in `implemented/` **in the same PR**, and links it from the relevant code. A proposal for future work not yet built goes in `proposed/`. A PR whose changes are mechanical, self-evident, or already covered by an existing RFC needs none — do not manufacture an RFC for a routine change. When unsure, the test is: would a competent maintainer six months from now ask "why was it done this way?" and be unable to answer from the code alone? If yes, write it. See [docs/rfc/README.md](docs/rfc/README.md) for the naming scheme and [docs/AGENTS.md](docs/AGENTS.md) for the cross-link convention.
**Markdown is not hard-wrapped**: write one line per paragraph and let the editor soft-wrap. Hard line breaks mid-paragraph make docs harder to edit and diff — a one-word change reflows and re-diffs the whole paragraph. This applies to prose only: leave fenced code blocks, tables, and list structure intact (a wrapped list item folds to one line per bullet). Code comments / JSDoc are exempt — they stay under the linter's column limit. `pnpm run verify-md-wrap` (part of `doc-sync`) enforces this across `README.md`, `docs/**/*.md`, `packages/*/README.md`, and `AGENTS.md` / `packages/AGENTS.md`; `pnpm run verify-md-links` (also part of `doc-sync`) checks that every relative cross-link in those files resolves.

4
docs/architecture.md
View File

@@ -45,7 +45,7 @@ Dependency rule: plugins depend on interface packages, never on `dsh-agent-loop`
|---|---|---|---|
| `ctx.llm` | `LlmService` | dsh-llm | adapter registry; `stream()` / `streamBlocks()` / `generate()` |
| `ctx.sessions` | `SessionStore` | dsh-session | creates/holds event-sourced `Session`s |
| `ctx.sessionPersistence` | `SessionPersistence` (abstract) | dsh-session-persistence | durable persistence seam: create/append/load/list/update sessions |
| `ctx.sessionPersistence` | `SessionPersistence` (abstract) | dsh-session-persistence | durable persistence seam: create/append/load/list sessions |
| `ctx.systemPrompt` | `SystemPrompt` | dsh-system-prompt | ordered sections + tool schemas → `assemble()` |
| `ctx.tools` | `ToolRegistry` | dsh-tools | tool definitions; `execute()` through waterfall |
| `ctx.agents` | `AgentRegistry` | dsh-agent | live `Agent` handles + the create/resume factory seam |
@@ -85,7 +85,7 @@ A `Session` is an append-only log of typed `SessionEvent`s — the single source
Replay/fork = `ctx.sessions.create(id, { seed: seedEvents })`. Trace/telemetry = listen to `session/event`.
**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list/update over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, crash recovery that PRESERVES an interrupted turn (closing it with a synthetic `turn/end {interrupted}` rather than truncating — a turn can be huge), and a read/replay path. Session metadata (format version, cwd, lineage) travels separately as `SessionMeta`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent` — the row shape `(session_id, seq, type, time, data)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic.
**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, crash recovery that PRESERVES an interrupted turn (closing it with a synthetic `turn/end {interrupted}` rather than truncating — a turn can be huge), and a read/replay path. Session metadata (format version, cwd, lineage) travels separately as `SessionHeader`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent` — the row shape `(session_id, seq, type, time, data)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic.
## Prompt assembly (dsh-system-prompt)

1
docs/rfc/README.md
View File

@@ -60,6 +60,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
| [Rich ACP bash rendering — the terminal card (`_meta`) and command classification](implemented/2026-06-18-acp-terminal-and-tool-rendering.md) | 2026-06-18 |
| [ACP snapshot tests — record-once / replay-deterministic](implemented/2026-06-19-acp-snapshot-tests.md) | 2026-06-19 |
| [Real-API e2e in CI against the external DeepSeek API](implemented/2026-06-19-real-api-e2e-ci.md) | 2026-06-19 |
| [Drop the mutable session summary](implemented/2026-06-19-drop-mutable-session-summary.md) | 2026-06-19 |
## Rejected

6
docs/rfc/implemented/2026-06-14-session-persistence.md
View File

@@ -16,15 +16,15 @@ The [event-sourced model](2026-06-11-event-sourced-sessions.md) makes the append
Persistence is an abstract **capability seam** ([capability seams](2026-06-13-capability-seams.md), the `dsh-bash` template), not loop or core logic:
1. **Interface** (`dsh-session-persistence`, `ctx.sessionPersistence`) — an abstract `SessionPersistence` service: `create`/`append`/`load`/`list`/`has`/`delete`/`update`. Its persisted unit IS the existing `SessionEvent` (`{ type, seq, time, data }`), reused verbatim — no conversion type.
2. **Implementation** (`dsh-session-persistence-jsonl`) — an append-only JSONL log per session (a `SessionHeader` line then one `SessionEvent` per line, verbatim **including `assistant/chunk`**) plus an atomic `.summary.json` sidecar for the mutable `SessionSummary`.
1. **Interface** (`dsh-session-persistence`, `ctx.sessionPersistence`) — an abstract `SessionPersistence` service: `create`/`append`/`load`/`list`/`has`/`delete`. Its persisted unit IS the existing `SessionEvent` (`{ type, seq, time, data }`), reused verbatim — no conversion type.
2. **Implementation** (`dsh-session-persistence-jsonl`) — an append-only JSONL log per session (a `SessionHeader` line then one `SessionEvent` per line, verbatim **including `assistant/chunk`**).
Key choices recorded here because they are durable, contested, and surprising:
- **The canonical durable log persists every `SessionEvent` verbatim, including `assistant/chunk`.** `deriveMessages()` skips chunks, and a chunk-filtered rollout (Codex's `policy.rs`) is tempting — but `seq = log.length` and the load-validation `events[i].seq === i` require a *contiguous* log; filtering chunks out would leave holes and break both the contract and resume. A chunk-filtered projection is possible later as a derived view with its own renumbering, but it is NOT the canonical log.
- **Append-only; a crashed turn is closed, never truncated.** Committed events — those at or below a flushed `turn/end` — are never rewritten. The loop only flushes at `turn/end`, so a crash can leave a durable log whose final turn never closed: real, fully-written events sit after the last `turn/end`. **A single turn can be huge in a long-horizon task** (many steps, large tool output spanning a long autonomous run), so discarding the interrupted turn would silently destroy a large amount of real work — truncating a turn is wrong. Instead, on reload `load` PRESERVES those events and CLOSES the orphaned turn by durably appending the minimal synthetic boundary events: an error `tool/result` for every `tool-call` the crash left unanswered, then a `step/end` if a step was still open, then a `turn/end` carrying the merge-extensible `{ kind: 'interrupted' }` reason (a marker that records the turn was cut short by a crash, not completed by the model — no loop ever emits it). The synthetic tool results matter for resume correctness: the loop logs the `assistant/message` (carrying the `tool-call` blocks) BEFORE running the tools, so a crash mid-tool leaves calls without results; `deriveMessages()` would then replay a dangling assistant tool-call, which every provider rejects as an invalid transcript on the next request. Answering each orphaned call with an error result keeps the rehydrated history valid. `load` returns the balanced log, so a resumed session is immediately usable. The ONLY thing discarded is a never-fully-written **torn tail fragment** — a final record whose bytes (JSONL) or row were never completely flushed; that fragment is not a valid event and is dropped before the synthetic closers are written. A parse error or `seq` gap in the COMMITTED region (at or before the last real `turn/end`) is genuine corruption and makes the session unloadable.
- **File backend canonical, DB backend a proven drop-in.** `SessionEvent` maps 1:1 onto a row `(session_id, seq, type, time, data)` — `append` is INSERT (in a transaction asserting the contiguous-seq contract), `load` is SELECT … ORDER BY seq. `dsh-session-persistence-sqlite` is exactly this: a `SessionPersistence` subclass with no interface change (opencode runs this exact shape on SQLite/WAL), and it passes the same `runPersistenceContract` suite as the JSONL backend — so the contract holds both backends to identical semantics (lazy materialization, interrupted-turn close on load, contiguous-seq), expressed once over file bytes and once over rows.
- **Metadata is out-of-log.** Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a `SessionMeta` (`SessionHeader & SessionSummary`) owned by `dsh-session` and attached to a `Session` via a new readonly `session.header` — never in `SessionEventMap`, never reaching `deriveMessages()`. The alternative (a merge-extensible `session/meta` event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header seam is the cleaner cost.
- **Metadata is out-of-log.** Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a `SessionHeader` owned by `dsh-session` and attached to a `Session` via a new readonly `session.header` — never in `SessionEventMap`, never reaching `deriveMessages()`. The alternative (a merge-extensible `session/meta` event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header seam is the cleaner cost. (The header was originally split into an immutable `SessionHeader` plus a mutable `SessionSummary` whose union was `SessionMeta`; the mutable summary was later removed as dead state — see [Drop the mutable session summary](2026-06-19-drop-mutable-session-summary.md).)
- **Resume is an async factory, not a change to synchronous create.** `ctx.agents.resume({ resumeSessionId })` awaits `ctx.sessionPersistence.load`, recreates the live session with the loaded events (so `lastTurnNumber`/`deriveMessages` continue), and starts a fresh agent on the resumed id (NOT `${agentId}-session`). The agent-loop does NOT hard-inject `sessionPersistence` (that would pend non-persistent demos forever); `resume` rejects with a clear error when it is absent.
Format versioning: the header carries a `version`; `load` rejects an unknown version (no v1 migration). Stated honestly: append-only + flush is robust to partial trailing writes (tolerated on load) but not to fsync-less power loss mid-line; a DB/WAL backend is the stronger option later.

31
docs/rfc/implemented/2026-06-19-drop-mutable-session-summary.md Normal file
View File

@@ -0,0 +1,31 @@
# RFC: Drop the mutable session summary
Status: implemented (proposed and accepted 2026-06-19)
## Context
The [session-persistence seam](2026-06-14-session-persistence.md) split a session's out-of-log metadata into two types owned by `dsh-session`: an immutable `SessionHeader` (`version`, `id`, `createdAt`, `cwd?`, `parentSession?`) written once at creation, and a mutable `SessionSummary` (`updatedAt`, `title?`, `firstPrompt?`) "updateable without touching the append-only log". Their union was `SessionMeta = SessionHeader & SessionSummary`, and the abstract `SessionPersistence` service carried a seventh method — `update(id, summary)` — for rewriting the summary. Each backend implemented the mutable store its own way: JSONL wrote a separate atomic `.summary.json` **sidecar** beside the log (temp-write + rename, best-effort), SQLite kept `updated_at`/`title`/`first_prompt` **columns** bumped inside the append transaction.
The summary was designed for a future session picker (recency ordering via `updatedAt`, a `title`/`firstPrompt` preview). That picker was never built. An audit of the whole repo found the entire `SessionSummary` surface is **dead state**:
- `SessionPersistence.update()` has **zero production callers** (every `.update(` hit is `createHash().update()` or a test).
- `firstPrompt` is **never read** anywhere in production.
- `title` *is* read in the ACP bridge — but from a tool-call **presenter** (`present.title`), never from stored session metadata.
- `updatedAt` has **no consumer**: the only production caller of `list()` reads `meta.cwd` (a `SessionHeader` field) to validate a workspace on `session/load`; resume reads `createdAt`/`cwd`/`parentSession` — all header fields.
- Decisively: the live `Session.header` was already typed `SessionHeader`, not `SessionMeta` — the summary never existed on the live session object; it lived only in the persistence layer, written and read by nothing but its own contract test.
## Decision
Delete the mutable session summary entirely. `SessionSummary` and the `SessionMeta` name are removed; the metadata a backend stores and returns is just `SessionHeader`. `SessionPersistence.update()` is removed from the abstract service and every backend. JSONL loses the whole sidecar machinery (`writeSidecar`/`readSidecar`/`touchSummary`/`removeSidecars`/`sidecarPath` and the load/list overlays); SQLite drops the `updated_at`/`title`/`first_prompt` columns and the per-append `updated_at` bump, and its `SCHEMA_VERSION` goes `1 → 2`.
Anything the summary was meant to provide is **derivable from the append-only log** when a consumer actually needs it (`firstPrompt` = first `user/message`; recency = the last event's `time` or the file mtime) or already lives in the immutable header (`createdAt`, `cwd`). The one thing *not* derivable — a user-*edited* title — had no implementation and is pure YAGNI; it can return as its own log event or header field if a real feature ever needs it.
This is recorded as a decision because it is **durable** (it narrows a public service contract and an on-disk format across two backends), **contested** (the summary was a deliberate forward-looking design, not an accident), and **surprising** (a future reader finding `SessionHeader` where the original RFC describes `SessionMeta` would otherwise ask why the summary vanished). It also unblocks the [shared persistence write coordinator](../proposed/2026-06-18-shared-persistence-write-coordinator.md): with no mutable summary, the coordinator's hook interface needs no `updateSummary` hook and the JSONL-sidecar-vs-SQLite-column durability divergence disappears, so the two backends' write paths converge.
## No migration
This is unreleased software (see [root AGENTS.md](../../../AGENTS.md) § "Pre-release stance: foundation over blast radius"), so there are no on-disk databases or logs to preserve. SQLite does not migrate a v1 database: the `openDatabase` guard now rejects any non-current on-disk `user_version` (`onDisk !== 0 && onDisk !== SCHEMA_VERSION`) — older *or* newer — so a stale v1 DB is cleanly rejected rather than half-read against the new column set. A fresh database stamps the current version; that is the only path that needs to work.
## What we gave up
A future session picker now has to derive its preview/ordering from the log (or reintroduce a typed field) rather than reading a ready-made summary row. That is the correct cost: a cache for a feature that does not exist is dead weight that every backend pays to maintain and every contract test pays to assert. The principle — **a passing test pins current behavior, not necessarily correct behavior; behavior can be an artifact of a past compromise** — is now recorded as a standalone convention in [root AGENTS.md](../../../AGENTS.md), with this change as its worked example.

4
docs/rfc/proposed/2026-06-18-shared-persistence-write-coordinator.md
View File

@@ -8,7 +8,7 @@ Status: proposed
## Proposal
Extract a backend-agnostic coordinator into `dsh-session-persistence`. The coordinator owns live-session adoption, buffering, cursor filtering, per-id serialization, and disposal quiescence. Concrete backends provide small hooks for durable operations: create lazy state, find/load stored prefix, append a contiguous batch, update summary, delete, and list.
Extract a backend-agnostic coordinator into `dsh-session-persistence`. The coordinator owns live-session adoption, buffering, cursor filtering, per-id serialization, and disposal quiescence. Concrete backends provide small hooks for durable operations: create lazy state, find/load stored prefix, append a contiguous batch, delete, and list.
The public `SessionPersistence` service shape can stay the same. The coordinator can be an internal exported helper or protected base class used by first-party backends; third-party backends may still implement the abstract service directly if their write path is different.
@@ -16,7 +16,7 @@ The public `SessionPersistence` service shape can stay the same. The coordinator
- JSONL and SQLite keep passing the existing shared `runPersistenceContract`.
- HMR/adoption/collision tests move to a shared coordinator test suite and run once for each backend through hook-driven fixtures.
- Backend-specific tests focus on storage mechanics only: JSONL path safety/fsync/sidecar behavior and SQLite schema/WAL/transaction behavior.
- Backend-specific tests focus on storage mechanics only: JSONL path safety/fsync behavior and SQLite schema/WAL/transaction behavior.
- A future backend does not need to copy the current `session/event` → buffer → flush orchestration.
## Risks

4
packages/acp/tests/load.spec.ts
View File

@@ -166,7 +166,7 @@ describe('acp bridge — session/load replay', () => {
loader = await makeBridgeHarness({ storageDir, script: [] })
const otherCwd = '/some/other/workspace'
await loader.ctx.sessionPersistence.create({
version: 1, id: SessionId('elsewhere'), createdAt: 1, cwd: otherCwd, updatedAt: 1,
version: 1, id: SessionId('elsewhere'), createdAt: 1, cwd: otherCwd,
})
await loader.ctx.sessionPersistence.append(SessionId('elsewhere'), [
{ type: 'turn/start', seq: 0, time: 0, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } },
@@ -203,7 +203,7 @@ describe('acp bridge — session/load replay', () => {
// to the server's launch dir (the request cwd does not override the header).
loader = await makeBridgeHarness({ storageDir, script: [] })
await loader.ctx.sessionPersistence.create({
version: 1, id: SessionId('legacy'), createdAt: 1, updatedAt: 1, // no cwd
version: 1, id: SessionId('legacy'), createdAt: 1, // no cwd
})
await loader.ctx.sessionPersistence.append(SessionId('legacy'), [
{ type: 'turn/start', seq: 0, time: 0, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } },

5
packages/session-persistence-jsonl/README.md
View File

@@ -1,6 +1,6 @@
# @deepseek-ai/dsh-session-persistence-jsonl
The JSONL durable session-persistence backend — a concrete `SessionPersistence` (the `dsh-session-persistence` seam). One append-only `.jsonl` event log per session plus a small atomic `.summary.json` sidecar for mutable metadata.
The JSONL durable session-persistence backend — a concrete `SessionPersistence` (the `dsh-session-persistence` seam). One append-only `.jsonl` event log per session.
## On-disk layout
@@ -8,7 +8,6 @@ The JSONL durable session-persistence backend — a concrete `SessionPersistence
<root>/
cwd-<sha256(cwd)[:12]>/ # per-project bucket (or _no-cwd/ when no cwd)
<encoded-id>.jsonl # header line + one SessionEvent per line (verbatim)
<encoded-id>.summary.json # mutable SessionSummary (atomic temp-write + rename)
```
- The first `.jsonl` line is the immutable `SessionHeader` tagged `{ type: 'session', version, id, cwd?, createdAt, parentSession? }`; every subsequent line is one `SessionEvent` JSON, **verbatim including `assistant/chunk`** so `seq` stays contiguous (`events[i].seq === i`).
@@ -26,7 +25,7 @@ The JSONL durable session-persistence backend — a concrete `SessionPersistence
- **Append-only.** Committed events (at or below a flushed `turn/end`) are never rewritten. Subsequent appends are line appends at EOF + `fsync`.
- **Crash recovery — close, don't truncate.** A crash can leave a log whose final turn never closed (real events after the last `turn/end`). `load` PRESERVES those events (a turn can be huge — they are real work) and closes the orphaned turn by durably appending synthetic boundary events: an error `tool/result` for every `tool-call` the crash left unanswered (the loop logs the assistant message before running the tools, so a mid-tool crash leaves dangling calls — and `deriveMessages()` would replay an assistant tool-call with no result, which providers reject), then a `step/end` if a step was open, then `turn/end {kind:'interrupted'}`, returning a balanced log. Only a never-fully-written **torn tail fragment** (a final line with no newline / unparseable) is `ftruncate`d away before the closers are written. See [session persistence](../../docs/rfc/implemented/2026-06-14-session-persistence.md).
- **Contiguous-seq.** `load` rejects a mid-log parse error or `seq` gap (unloadable); `append` rejects a batch whose first `seq` does not continue the stored log, and rejects non-JSON-serializable `event.data` naming the offending event type.
- **Format version.** Only v1 is supported; `load` rejects an unknown version. A future format change requires a version bump + migration.
- **Format version.** Only v1 is supported; `load` rejects an unknown version. While the harness is unreleased a format change bumps the version and rejects non-current logs — there is no migration (no persisted user data to preserve).
## Write path

33
packages/session-persistence-jsonl/src/format.ts
View File

@@ -2,15 +2,15 @@
* On-disk format helpers for the JSONL session-persistence backend: path
* sanitization (a {@link SessionId} is an unvalidated branded string, so it
* MUST be encoded before use in a path — no traversal, no collision), the
* per-cwd directory layout, header-line (de)serialization, the atomic sidecar
* for mutable summary fields, and the truncation-repair offset computation.
* per-cwd directory layout, header-line (de)serialization, and the
* truncation-repair offset computation.
*
* @module dsh-session-persistence-jsonl/format
*/
import { createHash } from 'node:crypto'
import { join } from 'node:path'
import type { SessionEvent, SessionHeader, SessionId, SessionMeta } from '@deepseek-ai/dsh-session'
import type { SessionEvent, SessionHeader, SessionId } from '@deepseek-ai/dsh-session'
/**
* The first line of a session's `.jsonl` file: the immutable
@@ -109,11 +109,6 @@ export function logPath(root: string, cwd: string | undefined, id: SessionId): s
return join(sessionDir(root, cwd), `${encodeSegment(id)}.jsonl`)
}
/** The mutable-summary sidecar path for a session (beside its log). */
export function sidecarPath(root: string, cwd: string | undefined, id: SessionId): string {
return join(sessionDir(root, cwd), `${encodeSegment(id)}.summary.json`)
}
/** Serialize one event as a JSONL line (no trailing newline). */
export function eventLine(event: SessionEvent): string {
return JSON.stringify(event)
@@ -138,7 +133,7 @@ export function eventLine(event: SessionEvent): string {
* (`Session.append` enforces it): only the final turn can be open, so the
* preserved tail is at most one unclosed turn.
*/
export function scanLog(buffer: Buffer): { meta: SessionMeta; events: SessionEvent[]; committedBytes: number } {
export function scanLog(buffer: Buffer): { meta: SessionHeader; events: SessionEvent[]; committedBytes: number } {
const text = buffer.toString('utf8')
// Split into complete (newline-terminated) lines, tracking the byte offset of
// each line's end so the truncation point is exact (multi-byte chars make the
@@ -233,26 +228,16 @@ export function scanLog(buffer: Buffer): { meta: SessionMeta; events: SessionEve
// synthetic closers + new events.
const lastPreserved = parsed[preserved.length - 1]
const committedBytes = preserved.length > 0 && lastPreserved ? lastPreserved.endByte : headerEntry.endByte
return { meta: metaFrom(headerLine), events: preserved, committedBytes }
}
/** Build the load-time {@link SessionMeta} from a header line (summary overlaid later). */
function metaFrom(headerLine: HeaderLine): SessionMeta {
return {
...fromHeaderLine(headerLine),
updatedAt: headerLine.createdAt, // overlaid by the sidecar in load()
}
return { meta: fromHeaderLine(headerLine), events: preserved, committedBytes }
}
/**
* Parse just the header line of a log into load-time {@link SessionMeta}, or
* Parse just the header line of a log into a {@link SessionHeader}, or
* `undefined` if it is missing/not a header. Used by `list()` to read session
* metadata WITHOUT parsing the whole log: a session picker scales with the
* number of sessions, not the total size of every conversation. The summary
* sidecar is overlaid by the caller; `updatedAt` here mirrors `createdAt` until
* then (same as {@link scanLog}'s load-time meta).
* number of sessions, not the total size of every conversation.
*/
export function parseHeaderMeta(firstLine: string): SessionMeta | undefined {
export function parseHeaderMeta(firstLine: string): SessionHeader | undefined {
let parsed: unknown
try {
parsed = JSON.parse(firstLine)
@@ -260,5 +245,5 @@ export function parseHeaderMeta(firstLine: string): SessionMeta | undefined {
return undefined
}
if (!isHeaderLine(parsed)) return undefined
return metaFrom(parsed)
return fromHeaderLine(parsed)
}

159
packages/session-persistence-jsonl/src/index.ts
View File

@@ -5,8 +5,7 @@
*
* 1. **The backend** — a concrete {@link SessionPersistence}: one append-only
* `.jsonl` event log per session (a header line then one `SessionEvent` per
* line, verbatim including `assistant/chunk` so `seq` stays contiguous) plus
* a small atomic `.summary.json` sidecar for the mutable `SessionSummary`.
* line, verbatim including `assistant/chunk` so `seq` stays contiguous).
* Lazy materialization (no file until the first `append`), atomic first
* write, and load-time repair of a never-committed crash tail.
*
@@ -22,16 +21,16 @@
import { Context } from 'cordis'
import z from 'schemastery'
import { open, mkdir, readFile, readdir, rename, link, rm, truncate } from 'node:fs/promises'
import { open, mkdir, readFile, readdir, link, rm, truncate } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { randomBytes } from 'node:crypto'
import {
SessionPersistence, assertSerializable, seedCoversPrefix,
} from '@deepseek-ai/dsh-session-persistence'
import { interruptedTurnClosers } from '@deepseek-ai/dsh-session'
import type { Session, SessionEvent, SessionId, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session'
import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
import {
encodeSegment, eventLine, logPath, parseHeaderMeta, scanLog, sessionDir, sidecarPath, toHeaderLine,
encodeSegment, eventLine, logPath, parseHeaderMeta, scanLog, sessionDir, toHeaderLine,
} from './format.ts'
export interface Config {
@@ -45,7 +44,7 @@ export interface Config {
/** Per-session write state held by the backend's in-memory bookkeeping. */
interface SessionState {
meta: SessionMeta
meta: SessionHeader
/** The next seq the backend expects to append (the stored log length). */
cursor: number
/** Whether the `.jsonl` file has been physically materialized. */
@@ -130,17 +129,17 @@ export class SessionPersistenceJsonl extends SessionPersistence {
// --- SessionPersistence backend surface (all serialized per session id) ---
create(meta: SessionMeta): Promise<void> {
create(meta: SessionHeader): Promise<void> {
// Snapshot the metadata at call time: the op runs later (behind the
// per-session chain) and the snapshot is also stored as the lazy state, so
// keeping the caller's object by reference would let a later mutation of
// `id`/`cwd` register under one key but materialize under a different
// path/header. A shallow copy is enough — SessionMeta is a flat record.
const snapshot: SessionMeta = { ...meta }
// path/header. A shallow copy is enough — SessionHeader is a flat record.
const snapshot: SessionHeader = { ...meta }
return this.serialize(snapshot.id, () => this.createCore(snapshot))
}
private async createCore(meta: SessionMeta): Promise<void> {
private async createCore(meta: SessionHeader): Promise<void> {
// Do NOT clobber an existing session. If we already track it, or a log
// exists on disk under this id, refuse — the SessionId IS the identity, and
// silently resetting state (cursor 0, materialized false) over committed
@@ -218,18 +217,15 @@ export class SessionPersistenceJsonl extends SessionPersistence {
await this.appendLines(state, events)
}
// The durable event log is the transaction: advance the cursor as soon as
// the log write commits. The sidecar (mutable summary) is best-effort here
// — a failed sidecar write must NOT reject an append whose log already
// landed (that would desync the cursor and let a retry duplicate seqs).
// the log write commits.
state.cursor += events.length
await this.touchSummary(state).catch(() => { /* sidecar is recoverable metadata; log is durable */ })
}
load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> {
load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
return this.serialize(id, () => this.loadCore(id))
}
private async loadCore(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> {
private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
const cwd = this.states.get(id)?.meta.cwd
const file = await this.findLog(id, cwd)
if (file === undefined) throw new Error(`session "${id}" not found`)
@@ -237,9 +233,6 @@ export class SessionPersistenceJsonl extends SessionPersistence {
const { meta, events, committedBytes } = scanLog(buffer)
this.assertVersion(meta)
const summary = await this.readSidecar(id, meta.cwd)
const fullMeta: SessionMeta = { ...meta, ...summary }
// Crash-recovery: if the log ended mid-turn (an open turn with real,
// preserved events but no closing turn/end), close it durably DURING load so
// disk, the returned log, and the cursor all agree — both append routes then
@@ -253,7 +246,7 @@ export class SessionPersistenceJsonl extends SessionPersistence {
// Set state BEFORE the repair writes so they can resolve the log path.
const needsTorn = committedBytes < buffer.byteLength
const state: SessionState = {
meta: { ...fullMeta },
meta: { ...meta },
cursor: events.length,
materialized: true,
}
@@ -267,15 +260,12 @@ export class SessionPersistenceJsonl extends SessionPersistence {
if (closers.length > 0) {
// Durably append the synthetic closers, then advance the cursor to the
// balanced length. After this, disk == balanced and the next append (live
// or direct) continues cleanly. No sidecar touch here: load is not a
// summary-changing op (the closers carry no new title/firstPrompt), and
// the next real append bumps `updatedAt` — keeping the summary write off
// the recovery path avoids a second best-effort failure mode.
// or direct) continues cleanly.
await this.appendLines(state, closers)
state.cursor = balanced.length
}
return { meta: fullMeta, events: balanced }
return { meta, events: balanced }
}
private async adoptLiveDiskPrefix(
@@ -290,9 +280,8 @@ export class SessionPersistenceJsonl extends SessionPersistence {
throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`)
}
const summary = await this.readSidecar(session.header.id, meta.cwd)
const state: SessionState = {
meta: { ...meta, ...summary },
meta: { ...meta },
cursor: events.length,
materialized: true,
owner: session,
@@ -306,8 +295,8 @@ export class SessionPersistenceJsonl extends SessionPersistence {
if (suffix.length > 0) await this.appendCore(session.header.id, suffix)
}
async list(): Promise<SessionMeta[]> {
const metas: SessionMeta[] = []
async list(): Promise<SessionHeader[]> {
const metas: SessionHeader[] = []
for (const dir of await this.listCwdDirs()) {
for (const name of await this.listJsonl(dir)) {
// Read ONLY the header line, not the whole log: a session picker must
@@ -318,8 +307,7 @@ export class SessionPersistenceJsonl extends SessionPersistence {
if (first === undefined) continue // empty/half-written file
const meta = parseHeaderMeta(first)
if (meta === undefined) continue // not a session header
const summary = await this.readSidecarForList(meta.id, meta.cwd)
metas.push({ ...meta, ...summary })
metas.push(meta)
}
}
return metas
@@ -367,41 +355,9 @@ export class SessionPersistenceJsonl extends SessionPersistence {
const cwd = this.states.get(id)?.meta.cwd
const file = await this.findLog(id, cwd)
if (file) await rm(file.path, { force: true })
// Remove the sidecar too. A lazy session (update() before the first
// append()) has a `.summary.json` sidecar but NO `.jsonl` log, and after a
// restart the in-memory cwd is gone — so keying sidecar removal off the log
// or the in-memory cwd would leak its possibly-sensitive title/firstPrompt.
// Scan every cwd bucket for the sidecar by its (sanitized) filename.
await this.removeSidecars(id)
this.states.delete(id)
}
/** Remove a session's summary sidecar from EVERY cwd bucket (id is unique). */
private async removeSidecars(id: SessionId): Promise<void> {
const target = `${encodeSegment(id)}.summary.json`
for (const dir of await this.listCwdDirs()) {
await rm(`${dir}/${target}`, { force: true })
}
}
update(id: SessionId, summary: Partial<SessionSummary>): Promise<void> {
return this.serialize(id, () => this.updateCore(id, summary))
}
private async updateCore(id: SessionId, summary: Partial<SessionSummary>): Promise<void> {
let state = this.states.get(id)
if (state === undefined) state = await this.adopt(id)
// Build the NEXT meta separately and commit it to in-memory state only AFTER
// the sidecar write succeeds. update's only durable effect is the sidecar,
// so a failure DOES reject (unlike append, whose log is the transaction and
// sidecar is best-effort) — but if we mutated state.meta first, a later
// touchSummary() on a successful append would persist the rejected
// title/firstPrompt, making a failed update durable after the fact.
const nextMeta: SessionMeta = { ...state.meta, ...summary, updatedAt: summary.updatedAt ?? Date.now() }
if (state.materialized) await this.writeSidecar(nextMeta)
state.meta = nextMeta
}
// --- materialization / append / repair ---
/** Atomically write the header line + first batch (temp-write, fsync, rename). */
@@ -521,76 +477,6 @@ export class SessionPersistenceJsonl extends SessionPersistence {
}
}
// --- sidecar (mutable summary) ---
private async touchSummary(state: SessionState): Promise<void> {
state.meta = { ...state.meta, updatedAt: Date.now() }
await this.writeSidecar(state.meta)
}
/**
* Atomic sidecar write (temp-write + rename), summary fields only.
*
* Deliberately NOT directory-fsynced (unlike {@link materialize}): the
* sidecar holds mutable, recoverable summary metadata (updatedAt, title,
* firstPrompt), not source-of-truth log data. The rename is atomic so a
* reader never sees a torn file, but a power loss may lose the most recent
* summary — acceptable because it is re-derivable and the durable log (the
* transaction) is independently synced. Strict crash-durability is reserved
* for the event log.
*/
private async writeSidecar(meta: SessionMeta): Promise<void> {
const dir = sessionDir(this.root, meta.cwd)
await mkdir(dir, { recursive: true, mode: 0o700 })
const path = sidecarPath(this.root, meta.cwd, meta.id)
const summary: SessionSummary = {
updatedAt: meta.updatedAt,
...meta.title !== undefined ? { title: meta.title } : {},
...meta.firstPrompt !== undefined ? { firstPrompt: meta.firstPrompt } : {},
}
const tmp = `${path}.${randomBytes(6).toString('hex')}.tmp`
// Exclusive owner-only create ('wx', 0o600), matching the log-materialization
// temp write: the sidecar can carry user data (title/firstPrompt), so a
// predictable/pre-existing temp path must never be silently truncated and
// followed (symlink race / disclosure). The random suffix already makes a
// collision unlikely; 'wx' makes reuse an error rather than a clobber.
const handle = await open(tmp, 'wx', 0o600)
try {
await handle.writeFile(JSON.stringify(summary))
} finally {
await handle.close()
}
await rename(tmp, path)
}
/**
* Read the mutable-summary sidecar, or `undefined` if it is absent (a session
* that has never been `update()`d). Non-ENOENT failures surface on strict
* load/adopt paths so corrupt metadata does not masquerade as a clean default.
*/
private async readSidecar(id: SessionId, cwd: string | undefined): Promise<SessionSummary | undefined> {
try {
const raw = await readFile(sidecarPath(this.root, cwd, id), 'utf8')
return JSON.parse(raw) as SessionSummary
} catch (error) {
if (isENOENT(error)) return undefined
throw error
}
}
/**
* Best-effort summary read for list(): a corrupt sidecar should degrade one
* row to header metadata, not hide every session from a picker.
*/
private async readSidecarForList(id: SessionId, cwd: string | undefined): Promise<SessionSummary | undefined> {
try {
return await this.readSidecar(id, cwd)
} catch (error: unknown) {
this.ctx.logger.warn(`session-persistence-jsonl: ignoring unreadable summary for session "${id}" while listing: ${String(error)}`)
return undefined
}
}
// --- discovery helpers ---
/** Find a session's log file across cwd buckets (when cwd is unknown). */
@@ -604,7 +490,8 @@ export class SessionPersistenceJsonl extends SessionPersistence {
for (const dir of await this.listCwdDirs()) {
const path = `${dir}/${target}`
if (await this.exists(path)) {
// Recover cwd from the header for accurate sidecar pathing.
// Recover the cwd from the header so the caller has the session's
// bucket location (which `findLog` was given an unknown cwd for).
const { meta } = scanLog(await readFile(path))
return { path, cwd: meta.cwd }
}
@@ -657,7 +544,7 @@ export class SessionPersistenceJsonl extends SessionPersistence {
return state
}
private assertVersion(meta: SessionMeta): void {
private assertVersion(meta: SessionHeader): void {
if (meta.version !== 1) {
throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`)
}
@@ -827,7 +714,7 @@ export class SessionPersistenceJsonl extends SessionPersistence {
// case 4: a genuinely new session. Register its meta (lazy), then persist
// its seed (events present at creation time) once.
const meta: SessionMeta = { ...session.header, updatedAt: Date.now() }
const meta: SessionHeader = { ...session.header }
await this.create(meta)
// Bind this state to the live session so a later DIFFERENT session reusing
// the id is detected as a collision (case 1) rather than silently no-opped.

150
packages/session-persistence-jsonl/tests/jsonl.spec.ts
View File

@@ -4,9 +4,9 @@ import { appendFile, mkdtemp, mkdir, rm, readFile, writeFile, readdir, stat } fr
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
import type { Session, SessionEvent, SessionMeta } from '@deepseek-ai/dsh-session'
import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
import { encodeSegment, logPath, scanLog, sessionDir, sidecarPath } from '../src/format.ts'
import { encodeSegment, logPath, scanLog, sessionDir } from '../src/format.ts'
import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts'
let root: string
@@ -274,7 +274,7 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
it('path-traversal session ids are neutralized (no escape from root)', async () => {
const evil = SessionId('../../etc/pwn')
const m = { version: 1, id: evil, createdAt: 1, updatedAt: 1 }
const m = { version: 1, id: evil, createdAt: 1 }
await ctx.sessionPersistence.create(m)
await ctx.sessionPersistence.append(evil, oneTurnLog())
// The file lives UNDER root, not at ../../etc.
@@ -581,23 +581,6 @@ describe('SessionPersistenceJsonl: edge cases', () => {
expect(await ctx.sessionPersistence.has(m.id)).toBe(false)
})
it('append resolves even when the best-effort sidecar write fails (log is the transaction)', async () => {
const m = meta('sidecar-fail')
await ctx.sessionPersistence.create(m)
// Force the sidecar write to reject AFTER the durable log append commits.
// The append must still resolve and advance the cursor — a failed sidecar
// is recoverable metadata and must never desync the log (which would let a
// retry duplicate seqs). This exercises the `.catch()` on touchSummary.
const backend = ctx.sessionPersistence as unknown as { writeSidecar: (state: unknown) => Promise<void> }
const original = backend.writeSidecar.bind(backend)
backend.writeSidecar = () => Promise.reject(new Error('disk full'))
await expect(ctx.sessionPersistence.append(m.id, oneTurnLog())).resolves.toBeUndefined()
backend.writeSidecar = original
// The durable log landed in full despite the sidecar failure.
const loaded = await ctx.sessionPersistence.load(m.id)
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5])
})
it('append rejects non-JSON-serializable undefined-producing data', async () => {
const m = meta('undef')
await ctx.sessionPersistence.create(m)
@@ -610,87 +593,6 @@ describe('SessionPersistenceJsonl: edge cases', () => {
await expect(ctx.sessionPersistence.delete(SessionId('ghost'))).resolves.toBeUndefined()
})
it('update adopts a session that exists only on disk', async () => {
const m = meta('disk-only')
await ctx.sessionPersistence.create(m)
await ctx.sessionPersistence.append(m.id, oneTurnLog())
// A fresh backend has no in-memory state → update must adopt from disk.
const ctx2 = new Context()
await ctx2.plugin(SessionStore)
await ctx2.plugin(SessionPersistenceJsonl, { root })
await ctx2.sessionPersistence.update(m.id, { title: 'adopted' })
const loaded = await ctx2.sessionPersistence.load(m.id)
expect(loaded.meta.title).toBe('adopted')
await ctx2.fiber.dispose()
})
it('a failed update does not become durable via a later append', async () => {
const m = meta('update-fail')
await ctx.sessionPersistence.create(m)
await ctx.sessionPersistence.append(m.id, oneTurnLog())
// Force the sidecar write to fail for the update.
const backend = ctx.sessionPersistence as unknown as { writeSidecar: (meta: unknown) => Promise<void> }
const original = backend.writeSidecar.bind(backend)
backend.writeSidecar = () => Promise.reject(new Error('disk full'))
await expect(ctx.sessionPersistence.update(m.id, { title: 'rejected-title' })).rejects.toThrow(/disk full/)
backend.writeSidecar = original
// A later successful append's touchSummary must NOT persist the rejected
// title (it was never committed to in-memory state).
await ctx.sessionPersistence.append(m.id, [
{ type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
{ type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } },
] as SessionEvent[])
const loaded = await ctx.sessionPersistence.load(m.id)
expect(loaded.meta.title).toBeUndefined()
})
it('update before the first append keeps summary in memory and writes no orphan sidecar', async () => {
const m = meta('lazy-update', '/a')
await ctx.sessionPersistence.create(m)
await ctx.sessionPersistence.update(m.id, { title: 'secret', firstPrompt: 'sensitive' })
const sidecar = sidecarPath(root, '/a', m.id)
await expect(stat(sidecar)).rejects.toThrow()
await expect(stat(logPath(root, '/a', m.id))).rejects.toThrow()
await ctx.sessionPersistence.append(m.id, oneTurnLog())
const loaded = await ctx.sessionPersistence.load(m.id)
expect(loaded.meta.title).toBe('secret')
expect(loaded.meta.firstPrompt).toBe('sensitive')
expect((await stat(sidecar)).isFile()).toBe(true)
})
it('a lazy update leaves no sidecar that can leak into a future same-id session after restart', async () => {
await ctx.sessionPersistence.create(meta('restart-lazy', '/a'))
await ctx.sessionPersistence.update(SessionId('restart-lazy'), { title: 'secret' })
await expect(stat(sidecarPath(root, '/a', SessionId('restart-lazy')))).rejects.toThrow()
const ctx2 = new Context()
await ctx2.plugin(SessionStore)
await ctx2.plugin(SessionPersistenceJsonl, { root })
const m2 = meta('restart-lazy', '/a')
await ctx2.sessionPersistence.create(m2)
await ctx2.sessionPersistence.append(m2.id, oneTurnLog())
const loaded = await ctx2.sessionPersistence.load(m2.id)
expect(loaded.meta.title).toBeUndefined()
await ctx2.fiber.dispose()
})
it('delete removes a materialized cwd-bucket sidecar after a restart', async () => {
await ctx.sessionPersistence.create(meta('restart-del', '/a'))
await ctx.sessionPersistence.append(SessionId('restart-del'), oneTurnLog())
await ctx.sessionPersistence.update(SessionId('restart-del'), { title: 'secret' })
const sidecar = sidecarPath(root, '/a', SessionId('restart-del'))
expect((await stat(sidecar)).isFile()).toBe(true)
const ctx2 = new Context()
await ctx2.plugin(SessionStore)
await ctx2.plugin(SessionPersistenceJsonl, { root })
await ctx2.sessionPersistence.delete(SessionId('restart-del'))
await expect(stat(sidecar)).rejects.toThrow()
await expect(stat(logPath(root, '/a', SessionId('restart-del')))).rejects.toThrow()
await ctx2.fiber.dispose()
})
it('an abandoned lazy session (never materialized) releases its id for reuse', async () => {
// A live session is created then disposed BEFORE its first append: cursor 0,
// never materialized, nothing on disk. A new live session reusing the id
@@ -765,27 +667,6 @@ describe('SessionPersistenceJsonl: edge cases', () => {
expect(ids).toEqual(['p1', 'p2', 'p3'])
})
it('list tolerates one corrupt sidecar and still returns other sessions', async () => {
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => undefined)
const bad = meta('bad-list-summary', '/proj')
await ctx.sessionPersistence.create(bad)
await ctx.sessionPersistence.append(bad.id, oneTurnLog())
await ctx.sessionPersistence.update(bad.id, { title: 'hidden by corrupt sidecar' })
await writeFile(sidecarPath(root, '/proj', bad.id), '{not json')
const good = meta('good-list-summary', '/proj')
await ctx.sessionPersistence.create(good)
await ctx.sessionPersistence.append(good.id, oneTurnLog())
await ctx.sessionPersistence.update(good.id, { title: 'visible' })
const listed = await ctx.sessionPersistence.list()
const badListed = listed.find(m => m.id === bad.id)
expect(badListed).toMatchObject({ id: bad.id })
expect(badListed).not.toHaveProperty('title')
expect(listed.find(m => m.id === good.id)).toMatchObject({ id: good.id, title: 'visible' })
expect(warn).toHaveBeenCalledWith(expect.stringContaining('bad-list-summary'))
})
it('list on an empty root returns nothing', async () => {
expect(await ctx.sessionPersistence.list()).toEqual([])
})
@@ -1062,36 +943,13 @@ describe('SessionPersistenceJsonl: edge cases', () => {
})
it('round-trips a header with parentSession (fork lineage)', async () => {
const m: SessionMeta = { version: 1, id: SessionId('forked-child'), createdAt: 1, updatedAt: 1, parentSession: SessionId('the-parent') }
const m: SessionHeader = { version: 1, id: SessionId('forked-child'), createdAt: 1, parentSession: SessionId('the-parent') }
await ctx.sessionPersistence.create(m)
await ctx.sessionPersistence.append(m.id, oneTurnLog())
const loaded = await ctx.sessionPersistence.load(m.id)
expect(loaded.meta.parentSession).toBe('the-parent')
})
it('loads a log that has no sidecar (default summary)', async () => {
// Hand-write a valid log WITHOUT a sidecar, then load it.
const dir = sessionDir(root, undefined)
await (await import('node:fs/promises')).mkdir(dir, { recursive: true })
const header = JSON.stringify({ type: 'session', version: 1, id: 'no-sidecar', createdAt: 5 })
const body = oneTurnLog().map(e => JSON.stringify(e)).join('\n')
await writeFile(logPath(root, undefined, SessionId('no-sidecar')), header + '\n' + body + '\n')
const loaded = await ctx.sessionPersistence.load(SessionId('no-sidecar'))
expect(loaded.events).toHaveLength(6)
expect(loaded.meta.title).toBeUndefined() // no sidecar → no title
// With no sidecar, updatedAt falls back to the header createdAt (5), NOT 0
// — reporting an active session as updated at the Unix epoch would be wrong.
expect(loaded.meta.updatedAt).toBe(5)
})
it('load rejects a corrupt sidecar instead of treating it as absent', async () => {
const m = meta('bad-sidecar')
await ctx.sessionPersistence.create(m)
await ctx.sessionPersistence.append(m.id, oneTurnLog())
await writeFile(sidecarPath(root, undefined, m.id), '{not json')
await expect(ctx.sessionPersistence.load(m.id)).rejects.toThrow()
})
it('list returns nothing when the root directory does not exist', async () => {
const ctx2 = new Context()
await ctx2.plugin(SessionStore)

4
packages/session-persistence-sqlite/README.md
View File

@@ -6,9 +6,9 @@ A SQLite durable session-persistence backend — a second `SessionPersistence` i
## Storage model
Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). Out-of-log metadata (`SessionMeta`) lives in a `sessions` row, including the mutable `SessionSummary` fields (`updatedAt`, `title`, `firstPrompt`) that `update()` rewrites without touching the event log. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`has`/`list` report exactly the sessions that have a row), so no separate column is needed.
Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). Out-of-log metadata (`SessionHeader`) lives in a `sessions` row. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`has`/`list` report exactly the sessions that have a row), so no separate column is needed.
The repo targets Node ≥ 24 (the root `engines` field), which includes the stable `node:sqlite` module. The database opens with `foreign_keys = ON` (so `ON DELETE CASCADE` drops a session's events with its row) and `journal_mode = WAL`. The table-layout version is stored in `PRAGMA user_version` and checked on open: a fresh database is stamped with the current `SCHEMA_VERSION`; a database written by a newer, incompatible build (higher `user_version`) is rejected rather than opened against an unknown layout.
The repo targets Node ≥ 24 (the root `engines` field), which includes the stable `node:sqlite` module. The database opens with `foreign_keys = ON` (so `ON DELETE CASCADE` drops a session's events with its row) and `journal_mode = WAL`. The table-layout version is stored in `PRAGMA user_version` and checked on open: a fresh database is stamped with the current `SCHEMA_VERSION`; a database written by any other, incompatible build (a non-current `user_version`, older or newer) is rejected rather than opened against an unknown layout — there is no migration (unreleased software).
## Contract semantics over rows

66
packages/session-persistence-sqlite/src/index.ts
View File

@@ -7,8 +7,7 @@
* / interrupted-turn-close-on-load semantics the JSONL backend expresses over
* file bytes, expressed here over `node:sqlite` rows. Each `SessionEvent` maps
* 1:1 onto a row `(session_id, seq, type, time, data)`; `append` is an INSERT
* inside a transaction that asserts the contiguous-seq contract; the mutable
* `SessionSummary` lives in the `sessions` metadata row.
* inside a transaction that asserts the contiguous-seq contract.
*
* Like the JSONL backend it is also the write-path plugin: it installs the
* `session/event` → buffer → `session/flush` drain, persists a fork's seed once
@@ -28,7 +27,7 @@ import {
SessionPersistence, assertSerializable, seedCoversPrefix,
} from '@deepseek-ai/dsh-session-persistence'
import { interruptedTurnClosers } from '@deepseek-ai/dsh-session'
import type { Session, SessionEvent, SessionId, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session'
import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
import {
openDatabase, rowToMeta, scanRows, type EventRow, type SessionRow,
} from './schema.ts'
@@ -47,7 +46,7 @@ export interface Config {
/** Backend bookkeeping for a session id (NOT the live Session object). */
interface SessionState {
meta: SessionMeta
meta: SessionHeader
/** Next seq to write — equals the number of committed events. */
cursor: number
/** Whether the session has at least one persisted event (materialized). */
@@ -108,12 +107,12 @@ export class SessionPersistenceSqlite extends SessionPersistence {
// --- SessionPersistence backend surface (all serialized per session id) ---
create(meta: SessionMeta): Promise<void> {
const snapshot: SessionMeta = { ...meta }
create(meta: SessionHeader): Promise<void> {
const snapshot: SessionHeader = { ...meta }
return this.serialize(snapshot.id, () => this.createCore(snapshot))
}
private async createCore(meta: SessionMeta): Promise<void> {
private async createCore(meta: SessionHeader): Promise<void> {
await this.ready
if (this.states.has(meta.id)) {
throw new Error(`session "${meta.id}" already exists in this backend`)
@@ -173,11 +172,7 @@ export class SessionPersistenceSqlite extends SessionPersistence {
for (const event of events) {
insertEvent.run(id, event.seq, event.type, event.time, JSON.stringify(event.data))
}
// Bump updatedAt on every append (the mutable summary lives in the row).
const updatedAt = Date.now()
this.db.prepare('UPDATE sessions SET updated_at = ? WHERE id = ?').run(updatedAt, id)
this.db.exec('COMMIT')
state.meta = { ...state.meta, updatedAt }
} catch (error) {
this.db.exec('ROLLBACK')
throw error
@@ -186,11 +181,11 @@ export class SessionPersistenceSqlite extends SessionPersistence {
state.cursor += events.length
}
load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> {
load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
return this.serialize(id, () => this.loadCore(id))
}
private async loadCore(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> {
private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
await this.ready
const row = this.rowFor(id)
if (row === undefined) throw new Error(`session "${id}" not found`)
@@ -290,7 +285,7 @@ export class SessionPersistenceSqlite extends SessionPersistence {
if (suffix.length > 0) await this.appendCore(session.header.id, suffix)
}
async list(): Promise<SessionMeta[]> {
async list(): Promise<SessionHeader[]> {
await this.ready
// Every metadata row is a materialized session: the row is written only by
// the first append (a created-but-never-appended session has no row), so
@@ -320,23 +315,6 @@ export class SessionPersistenceSqlite extends SessionPersistence {
this.states.delete(id)
}
update(id: SessionId, summary: Partial<SessionSummary>): Promise<void> {
return this.serialize(id, () => this.updateCore(id, summary))
}
private async updateCore(id: SessionId, summary: Partial<SessionSummary>): Promise<void> {
await this.ready
let state = this.states.get(id)
if (state === undefined) state = await this.adopt(id)
const nextMeta: SessionMeta = { ...state.meta, ...summary, updatedAt: summary.updatedAt ?? Date.now() }
// update's only durable effect is the summary fields; the event log is
// untouched. If the row is not materialized yet (a lazy session updated
// before its first append) there is nothing to write — keep the pending
// summary in memory so the materializing append carries it.
if (state.materialized) this.writeRow(nextMeta)
state.meta = nextMeta
}
// --- row helpers ---
/** Fetch a session's row, or undefined if absent. */
@@ -346,32 +324,26 @@ export class SessionPersistenceSqlite extends SessionPersistence {
}
/**
* Insert-or-replace a session's metadata row. The only callers are the first
* materializing `append` and a post-materialization `update`, so writing the
* row IS the materialization (its existence is the signal `has`/`list` read);
* a never-appended session has no row at all.
* Insert-or-replace a session's metadata row. The only caller is the first
* materializing `append`, so writing the row IS the materialization (its
* existence is the signal `has`/`list` read); a never-appended session has no
* row at all.
*/
private writeRow(meta: SessionMeta): void {
private writeRow(meta: SessionHeader): void {
this.db.prepare(`
INSERT INTO sessions (id, version, created_at, cwd, parent_session, updated_at, title, first_prompt)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
INSERT INTO sessions (id, version, created_at, cwd, parent_session)
VALUES (?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
version = excluded.version,
created_at = excluded.created_at,
cwd = excluded.cwd,
parent_session = excluded.parent_session,
updated_at = excluded.updated_at,
title = excluded.title,
first_prompt = excluded.first_prompt
parent_session = excluded.parent_session
`).run(
meta.id,
meta.version,
meta.createdAt,
meta.cwd ?? null,
meta.parentSession ?? null,
meta.updatedAt,
meta.title ?? null,
meta.firstPrompt ?? null,
)
}
@@ -384,7 +356,7 @@ export class SessionPersistenceSqlite extends SessionPersistence {
return state
}
private assertVersion(meta: SessionMeta): void {
private assertVersion(meta: SessionHeader): void {
if (meta.version !== 1) {
throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`)
}
@@ -513,7 +485,7 @@ export class SessionPersistenceSqlite extends SessionPersistence {
}
// case 4: a genuinely new session.
const meta: SessionMeta = { ...session.header, updatedAt: Date.now() }
const meta: SessionHeader = { ...session.header }
await this.create(meta)
const created = this.states.get(id)
/* v8 ignore next -- create() always sets the state for the id */

36
packages/session-persistence-sqlite/src/schema.ts
View File

@@ -8,18 +8,18 @@
*/
import { DatabaseSync } from 'node:sqlite'
import type { SessionEvent, SessionId, SessionMeta } from '@deepseek-ai/dsh-session'
import type { SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
/**
* The on-disk schema version. Bumped only on a breaking change to the table
* layout; orthogonal to a session's own `version` (which versions the EVENT
* vocabulary, stored per session in the `sessions` row).
*/
export const SCHEMA_VERSION = 1
export const SCHEMA_VERSION = 2
/**
* A row of the `sessions` table — the out-of-log metadata (`SessionMeta`). The
* row's EXISTENCE is the materialization signal: it is written only by the
* A row of the `sessions` table — the out-of-log metadata ({@link SessionHeader}).
* The row's EXISTENCE is the materialization signal: it is written only by the
* first `append` (lazy materialization), so a created-but-never-appended
* session has no row and is absent from `has`/`list`, mirroring the JSONL
* backend's "no file until first append".
@@ -30,9 +30,6 @@ export interface SessionRow {
created_at: number
cwd: string | null
parent_session: string | null
updated_at: number
title: string | null
first_prompt: string | null
}
/** An `events` table row: one `SessionEvent` mapped 1:1 (`data` is JSON text). */
@@ -51,10 +48,11 @@ export interface EventRow {
*
* The table-layout version is persisted in SQLite's `PRAGMA user_version` and
* checked on open: a fresh database (user_version 0) is stamped with the
* current {@link SCHEMA_VERSION}; an existing database with a NEWER version
* (written by a future, incompatible build) is rejected rather than opened
* against a layout this build does not understand. (An older-but-compatible
* version would be migrated here when migrations exist; v1 has none.)
* current {@link SCHEMA_VERSION}; an existing database whose version is NOT the
* current one (written by a different, incompatible build — older or newer) is
* REJECTED rather than opened against a layout this build does not understand.
* There are no migrations: v1 had a different `sessions` layout and is not
* upgraded in place.
*/
export function openDatabase(path: string): DatabaseSync {
const db = new DatabaseSync(path)
@@ -62,9 +60,9 @@ export function openDatabase(path: string): DatabaseSync {
db.exec('PRAGMA journal_mode = WAL')
// `PRAGMA user_version` always returns exactly one row { user_version }.
const { user_version: onDisk } = db.prepare('PRAGMA user_version').get() as { user_version: number }
if (onDisk > SCHEMA_VERSION) {
if (onDisk !== 0 && onDisk !== SCHEMA_VERSION) {
db.close()
throw new Error(`session database at "${path}" has schema version ${onDisk}, newer than this build supports (${SCHEMA_VERSION})`)
throw new Error(`session database at "${path}" has schema version ${onDisk}, incompatible with this build (${SCHEMA_VERSION})`)
}
if (onDisk === 0) {
// Fresh (or pre-versioning) database: stamp the current layout version.
@@ -78,10 +76,7 @@ export function openDatabase(path: string): DatabaseSync {
version INTEGER NOT NULL,
created_at INTEGER NOT NULL,
cwd TEXT,
parent_session TEXT,
updated_at INTEGER NOT NULL,
title TEXT,
first_prompt TEXT
parent_session TEXT
) STRICT
`)
db.exec(`
@@ -97,17 +92,14 @@ export function openDatabase(path: string): DatabaseSync {
return db
}
/** Reconstruct the full {@link SessionMeta} from a `sessions` row. */
export function rowToMeta(row: SessionRow): SessionMeta {
/** Reconstruct the {@link SessionHeader} from a `sessions` row. */
export function rowToMeta(row: SessionRow): SessionHeader {
return {
version: row.version,
id: row.id as SessionId,
createdAt: row.created_at,
updatedAt: row.updated_at,
...row.cwd !== null ? { cwd: row.cwd } : {},
...row.parent_session !== null ? { parentSession: row.parent_session as SessionId } : {},
...row.title !== null ? { title: row.title } : {},
...row.first_prompt !== null ? { firstPrompt: row.first_prompt } : {},
}
}

63
packages/session-persistence-sqlite/tests/sqlite.spec.ts
View File

@@ -4,7 +4,7 @@ import { mkdtemp, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
import type { Session, SessionEvent, SessionMeta } from '@deepseek-ai/dsh-session'
import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
import SessionPersistenceSqlite, { SCHEMA_VERSION } from '@deepseek-ai/dsh-session-persistence-sqlite'
import { openDatabase, scanRows, type EventRow } from '../src/schema.ts'
import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts'
@@ -232,14 +232,23 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
await b2.dispose()
})
it('rejects opening a database whose schema version is newer than this build', async () => {
it('rejects opening a database whose schema version is not the current build (newer OR older)', async () => {
const path = await freshDbPath()
openDatabase(path).close() // stamp user_version = SCHEMA_VERSION
// Bump user_version past what this build supports.
const db = openDatabase(path)
db.exec(`PRAGMA user_version = ${SCHEMA_VERSION + 1}`)
db.close()
expect(() => openDatabase(path)).toThrow(/newer than this build/)
const dbNewer = openDatabase(path)
dbNewer.exec(`PRAGMA user_version = ${SCHEMA_VERSION + 1}`)
dbNewer.close()
expect(() => openDatabase(path)).toThrow(/incompatible with this build/)
// A stale OLDER version (e.g. a pre-summary-drop v1 DB) is also rejected —
// we do not migrate (unreleased software, no backward-compat).
const olderPath = await freshDbPath()
openDatabase(olderPath).close()
const dbOlder = openDatabase(olderPath)
dbOlder.exec('PRAGMA user_version = 1')
dbOlder.close()
expect(() => openDatabase(olderPath)).toThrow(/incompatible with this build/)
})
it('append snapshots the batch: mutating an event after the call does not corrupt the persisted copy', async () => {
@@ -323,7 +332,6 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path })
await ctx1.sessionPersistence.create(m)
await ctx1.sessionPersistence.append(m.id, oneTurnLog())
await ctx1.sessionPersistence.update(m.id, { title: 'T', firstPrompt: 'hi' })
await fiber1.dispose()
const ctx2 = new Context()
@@ -331,7 +339,7 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path })
expect((await ctx2.sessionPersistence.list()).map(x => x.id)).toContain(m.id)
const loaded = await ctx2.sessionPersistence.load(m.id)
expect(loaded.meta).toMatchObject({ id: m.id, cwd: '/proj', title: 'T', firstPrompt: 'hi' })
expect(loaded.meta).toMatchObject({ id: m.id, cwd: '/proj' })
expect(loaded.events).toEqual(oneTurnLog())
await fiber2.dispose()
})
@@ -340,8 +348,8 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
const path = await freshDbPath()
// Materialize a row with version 2 directly via the real schema.
const db = openDatabase(path)
db.prepare('INSERT INTO sessions (id, version, created_at, updated_at) VALUES (?, ?, ?, ?)')
.run('v2', 2, 1, 1)
db.prepare('INSERT INTO sessions (id, version, created_at) VALUES (?, ?, ?)')
.run('v2', 2, 1)
db.close()
const ctx = new Context()
@@ -372,7 +380,7 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
})
it('exposes the schema version constant', () => {
expect(SCHEMA_VERSION).toBe(1)
expect(SCHEMA_VERSION).toBe(2)
})
})
@@ -468,22 +476,6 @@ describe('SessionPersistenceSqlite: write path (session/event → flush)', () =>
await expect(ctx2.parallel('session/flush', s2)).rejects.toThrow(/id collision/)
await fiber2.dispose()
})
it('update before the first append keeps the summary in memory and the session lazy', async () => {
const ctx = new Context()
await ctx.plugin(SessionStore)
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' })
const m = meta('lazy-update')
await ctx.sessionPersistence.create(m)
await ctx.sessionPersistence.update(m.id, { title: 'pending' })
// Still lazy: no materialized row yet.
expect(await ctx.sessionPersistence.has(m.id)).toBe(false)
// The first append materializes and carries the pending title.
await ctx.sessionPersistence.append(m.id, oneTurnLog())
const loaded = await ctx.sessionPersistence.load(m.id)
expect(loaded.meta.title).toBe('pending')
await fiber.dispose()
})
})
describe('SessionPersistenceSqlite: edge cases', () => {
@@ -529,21 +521,6 @@ describe('SessionPersistenceSqlite: edge cases', () => {
await b2.dispose()
})
it('update adopts a session that exists only in the DB (fresh instance)', async () => {
const path = await freshDbPath()
const m = meta('adopt-update')
const b1 = await backend(path)
await b1.ctx.sessionPersistence.create(m)
await b1.ctx.sessionPersistence.append(m.id, oneTurnLog())
await b1.dispose()
const b2 = await backend(path)
await b2.ctx.sessionPersistence.update(m.id, { title: 'after restart' })
const loaded = await b2.ctx.sessionPersistence.load(m.id)
expect(loaded.meta.title).toBe('after restart')
await b2.dispose()
})
it('append rolls back and rethrows when an event INSERT fails inside the transaction', async () => {
const path = await freshDbPath()
const m = meta('rollback-insert')
@@ -574,7 +551,7 @@ describe('SessionPersistenceSqlite: edge cases', () => {
it('round-trips a header with parentSession (fork lineage)', async () => {
const { ctx, dispose } = await backend()
const m: SessionMeta = { ...meta('child'), parentSession: SessionId('parent') }
const m: SessionHeader = { ...meta('child'), parentSession: SessionId('parent') }
await ctx.sessionPersistence.create(m)
await ctx.sessionPersistence.append(m.id, oneTurnLog())
const loaded = await ctx.sessionPersistence.load(m.id)

9
packages/session-persistence/README.md
View File

@@ -1,8 +1,8 @@
# @deepseek-ai/dsh-session-persistence
The abstract durable session-persistence seam (`ctx.sessionPersistence`). Defines WHAT a persistence backend does — durably store, reload, list, and update sessions — without saying HOW. Mirrors the `dsh-bash` capability-seam template ([capability seams](../../docs/rfc/implemented/2026-06-13-capability-seams.md)): an abstract service here, a concrete implementation in a sibling package, consumers that inject the interface.
The abstract durable session-persistence seam (`ctx.sessionPersistence`). Defines WHAT a persistence backend does — durably store, reload, and list sessions — without saying HOW. Mirrors the `dsh-bash` capability-seam template ([capability seams](../../docs/rfc/implemented/2026-06-13-capability-seams.md)): an abstract service here, a concrete implementation in a sibling package, consumers that inject the interface.
The persisted unit IS the existing `SessionEvent` (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage) travels separately as `SessionMeta`, owned by `dsh-session` and re-exported here.
The persisted unit IS the existing `SessionEvent` (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage) travels separately as `SessionHeader`, owned by `dsh-session` and re-exported here.
## Service API (`ctx.sessionPersistence`)
@@ -11,9 +11,8 @@ The persisted unit IS the existing `SessionEvent` (event-sourced model — the l
| `create(meta): Promise<void>` | Register a new session's metadata. MAY defer the physical write until the first `append` (lazy materialization). |
| `append(id, events): Promise<void>` | Durably persist a batch (from the `session/flush` drain). Append-only; first event `seq` == stored next-seq after any repair; rejects non-JSON-serializable data naming the offending type. |
| `load(id): Promise<{ meta; events }>` | Reload meta + log. Preserves an interrupted (unclosed) final turn and closes it with synthetic closers — an error `tool/result` per unanswered `tool-call`, then `step/end?`+`turn/end {interrupted}` (a turn can be huge — never truncated); only a torn tail fragment is dropped. Events contiguous (`events[i].seq === i`); rejects a committed-region gap/parse error or unknown `version`. |
| `list(): Promise<SessionMeta[]>` | Lightweight listing from metadata, no full-log parse. |
| `list(): Promise<SessionHeader[]>` | Lightweight listing from metadata, no full-log parse. |
| `has(id)` / `delete(id)` | Existence / removal. A zero-event lazily-materialized session is absent from `has`/`list`. |
| `update(id, summary): Promise<void>` | Update mutable `SessionSummary` fields without touching the append-only log. |
## Invariants every backend must honor
@@ -30,4 +29,4 @@ Two backends run this suite: `dsh-session-persistence-jsonl` (append-only file l
## Metadata types
Re-exported from `dsh-session`: `SessionHeader` (immutable: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`), `SessionSummary` (mutable: `updatedAt`, `title?`, `firstPrompt?`), `SessionMeta` (their intersection).
Re-exported from `dsh-session`: `SessionHeader` (immutable session metadata: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`).

24
packages/session-persistence/src/index.ts
View File

@@ -1,7 +1,7 @@
/**
* The durable session-persistence seam (`ctx.sessionPersistence`): an abstract
* service defining WHAT a persistence backend does — durably store, reload,
* list, and update sessions — without saying HOW. Implementations subclass
* and list sessions — without saying HOW. Implementations subclass
* {@link SessionPersistence} and register themselves as the
* `sessionPersistence` service; `@deepseek-ai/dsh-session-persistence-jsonl`
* (an append-only JSONL log per session) is the first and
@@ -15,7 +15,7 @@
* parallel "persisted message" type the log must be converted to and from
* (faithful to the event-sourced model: the log is the single source of
* truth). Metadata that is NOT replayable conversation state (format version,
* cwd, lineage) travels separately as {@link SessionMeta}, which is owned by
* cwd, lineage) travels separately as {@link SessionHeader}, which is owned by
* `dsh-session` and re-exported here.
*
* @module @deepseek-ai/dsh-session-persistence
@@ -23,10 +23,10 @@
import { Context, Service } from 'cordis'
import { isJsonValue } from '@deepseek-ai/dsh-session'
import type { SessionEvent, SessionId, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session'
import type { SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
// Re-export the metadata vocabulary so consumers import it from the seam.
export type { SessionHeader, SessionSummary, SessionMeta } from '@deepseek-ai/dsh-session'
export type { SessionHeader } from '@deepseek-ai/dsh-session'
declare module 'cordis' {
interface Context {
@@ -102,7 +102,7 @@ export abstract class SessionPersistence extends Service {
* created-but-never-appended session is absent from {@link has}/{@link list}
* — abandoned sessions leave nothing behind.
*/
abstract create(meta: SessionMeta): Promise<void>
abstract create(meta: SessionHeader): Promise<void>
/**
* Durably persist a batch of events (called from the write-behind drain at
@@ -114,7 +114,7 @@ export abstract class SessionPersistence extends Service {
abstract append(id: SessionId, events: readonly SessionEvent[]): Promise<void>
/**
* Reload a session: its {@link SessionMeta} plus the event log up to the last
* Reload a session: its {@link SessionHeader} plus the event log up to the last
* durable checkpoint. Returns `meta` AND `events` so the live session is
* reconstructed with its `cwd`/lineage, not just its log.
*
@@ -135,24 +135,16 @@ export abstract class SessionPersistence extends Service {
* unloadable (reject). Rejects an unknown format `version`. See the session-persistence RFC for
* the crash-recovery contract.
*/
abstract load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }>
abstract load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }>
/** Lightweight listing from metadata, without a full-log parse. */
abstract list(): Promise<SessionMeta[]>
abstract list(): Promise<SessionHeader[]>
/** Whether a session is durably present (materialized). */
abstract has(id: SessionId): Promise<boolean>
/** Remove a session and all its persisted artifacts. */
abstract delete(id: SessionId): Promise<void>
/**
* Update mutable metadata ({@link SessionSummary}: `updatedAt`, `title`,
* `firstPrompt`) WITHOUT touching the append-only event log. A backend
* stores the summary beside the log (a sidecar file, a header row) and
* rewrites only it.
*/
abstract update(id: SessionId, summary: Partial<SessionSummary>): Promise<void>
}
export default SessionPersistence

35
packages/session-persistence/tests/contract.ts
View File

@@ -8,9 +8,9 @@
* @module @deepseek-ai/dsh-session-persistence/tests/contract
*/
import { describe, expect, it, vi } from 'vitest'
import { describe, expect, it } from 'vitest'
import { SessionId } from '@deepseek-ai/dsh-session'
import type { SessionEvent, SessionMeta } from '@deepseek-ai/dsh-session'
import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
import { CallId } from '@deepseek-ai/dsh-llm'
import type { SessionPersistence } from '../src/index.ts'
@@ -20,13 +20,12 @@ export interface ContractBackend {
dispose: () => Promise<void>
}
/** Build a minimal {@link SessionMeta} for a session id. */
export function meta(id: string, cwd?: string): SessionMeta {
/** Build a minimal {@link SessionHeader} for a session id. */
export function meta(id: string, cwd?: string): SessionHeader {
return {
version: 1,
id: SessionId(id),
createdAt: 1000,
updatedAt: 1000,
...cwd !== undefined ? { cwd } : {},
}
}
@@ -242,31 +241,5 @@ export function runPersistenceContract(name: string, make: () => Promise<Contrac
await dispose()
}
})
it('update mutates summary fields without touching the event log', async () => {
const { persistence, dispose } = await make()
try {
const m = meta('s7')
const log = oneTurnLog()
await persistence.create(m)
await persistence.append(m.id, log)
const beforeUpdate = (await persistence.load(m.id)).meta.updatedAt
vi.useFakeTimers()
vi.setSystemTime(beforeUpdate + 1_000)
try {
await persistence.update(m.id, { title: 'My session', firstPrompt: 'hi' })
} finally {
vi.useRealTimers()
}
const loaded = await persistence.load(m.id)
expect(loaded.meta.title).toBe('My session')
expect(loaded.meta.firstPrompt).toBe('hi')
expect(loaded.meta.updatedAt).toBe(beforeUpdate + 1_000)
expect(loaded.events).toEqual(log) // log untouched
} finally {
await dispose()
}
})
})
}

17
packages/session-persistence/tests/persistence.spec.ts
View File

@@ -1,7 +1,7 @@
import { describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import { SessionId, isJsonValue, interruptedTurnClosers } from '@deepseek-ai/dsh-session'
import type { SessionEvent, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session'
import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
import { SessionPersistence, assertSerializable, seedCoversPrefix } from '../src/index.ts'
import { runPersistenceContract, meta, oneTurnLog } from './contract.ts'
@@ -12,10 +12,10 @@ import { runPersistenceContract, meta, oneTurnLog } from './contract.ts'
* `@deepseek-ai/dsh-session-persistence-jsonl`.
*/
class MemoryPersistence extends SessionPersistence {
private store = new Map<string, { meta: SessionMeta; events: SessionEvent[] }>()
private pending = new Map<string, SessionMeta>()
private store = new Map<string, { meta: SessionHeader; events: SessionEvent[] }>()
private pending = new Map<string, SessionHeader>()
async create(m: SessionMeta): Promise<void> {
async create(m: SessionHeader): Promise<void> {
// Lazy: record the intended meta, but stay absent from has/list until the
// first append materializes the session.
this.pending.set(m.id, m)
@@ -43,7 +43,7 @@ class MemoryPersistence extends SessionPersistence {
}
}
async load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> {
async load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
const entry = this.store.get(id)
if (!entry) throw new Error(`session "${id}" not found`)
// Honor the crash-recovery contract: if the stored log ends mid-turn, close
@@ -54,7 +54,7 @@ class MemoryPersistence extends SessionPersistence {
return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) }
}
async list(): Promise<SessionMeta[]> {
async list(): Promise<SessionHeader[]> {
return [...this.store.values()].map(e => structuredClone(e.meta))
}
@@ -66,11 +66,6 @@ class MemoryPersistence extends SessionPersistence {
this.store.delete(id)
this.pending.delete(id)
}
async update(id: SessionId, summary: Partial<SessionSummary>): Promise<void> {
const entry = this.store.get(id)
if (entry) Object.assign(entry.meta, summary, { updatedAt: summary.updatedAt ?? Date.now() })
}
}
// Run the shared contract against the in-memory backend.

6
packages/session/README.md
View File

@@ -31,9 +31,7 @@ Plain class (not a Cordis Service). Create via `ctx.sessions.create()`.
### Metadata types (`types.ts`)
- `SessionHeader` — immutable, written once: `{ version, id, createdAt, cwd?, parentSession? }`.
- `SessionSummary` — mutable, updateable without touching the log: `{ updatedAt, title?, firstPrompt? }`.
- `SessionMeta = SessionHeader & SessionSummary` — owned here (beside `SessionId`) because `Session.header` is typed by it; persistence backends re-export these rather than own them (which would force a package cycle).
- `SessionHeader` — immutable session metadata, written once: `{ version, id, createdAt, cwd?, parentSession? }`. Owned here (beside `SessionId`) because `Session.header` is typed by it; persistence backends re-export it rather than own it (which would force a package cycle).
### Session event vocabulary (`types.ts`)
@@ -45,7 +43,7 @@ Also defines `TurnTriggerMap` and `TurnEndReasonMap` (merge-extensible sum types
### Extension points
- Persistence plugins: subscribe to `session/event` (write-behind) and drain on `session/flush` (awaited) and fiber dispose. A durable backend reads the log and reloads it into a live session; the metadata seam (`SessionHeader`/`SessionSummary`/`SessionMeta`, `session.header`) is what such a backend stores beside the log.
- Persistence plugins: subscribe to `session/event` (write-behind) and drain on `session/flush` (awaited) and fiber dispose. A durable backend reads the log and reloads it into a live session; the metadata seam (`SessionHeader`, `session.header`) is what such a backend stores beside the log.
- Replay/fork: `ctx.sessions.create(id, { seed })` seeds a new session with an existing event log.
### What is NOT here (TODO)

23
packages/session/src/types.ts
View File

@@ -30,29 +30,6 @@ export interface SessionHeader {
parentSession?: SessionId
}
/**
* Mutable session metadata — updateable without touching the append-only log.
* A persistence backend stores this beside the log (a sidecar file, a header
* row) and rewrites only it on update.
*/
export interface SessionSummary {
/** Unix epoch milliseconds of the last mutation (event append or update). */
updatedAt: number
/** Human-facing title (derived/edited), if any. */
title?: string
/** The first user prompt, cached for listing previews. */
firstPrompt?: string
}
/**
* Full session metadata: the immutable {@link SessionHeader} merged with the
* mutable {@link SessionSummary}. Owned here in `dsh-session` (beside
* {@link SessionId}) because `Session.header` is typed by it; the persistence
* package imports/re-exports these rather than owning them, which would force
* a package cycle.
*/
export type SessionMeta = SessionHeader & SessionSummary
/**
* Options for creating a {@link Session} via the store. `seed` replays/forks
* an existing event log; `meta` carries the caller-supplied storage fields the