diff --git a/docs/rfc/README.md b/docs/rfc/README.md index bc915f2292..3897020434 100644 --- a/docs/rfc/README.md +++ b/docs/rfc/README.md @@ -32,7 +32,6 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r | [Optional Code Mode — model writes TypeScript against an SDK of all tools](proposed/2026-06-15-optional-code-mode.md) | 2026-06-15 | | [Runtime schemas for the event vocabulary (Zod vs the merge-extensible-map pattern)](proposed/2026-06-16-typed-event-schemas.md) | 2026-06-16 | | [Agent lifecycle and ownership seams](proposed/2026-06-18-agent-lifecycle-and-ownership-seams.md) | 2026-06-18 | -| [Shared persistence write coordinator](proposed/2026-06-18-shared-persistence-write-coordinator.md) | 2026-06-18 | ## Implemented @@ -61,6 +60,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r | [ACP snapshot tests — record-once / replay-deterministic](implemented/2026-06-19-acp-snapshot-tests.md) | 2026-06-19 | | [Real-API e2e in CI against the external DeepSeek API](implemented/2026-06-19-real-api-e2e-ci.md) | 2026-06-19 | | [Drop the mutable session summary](implemented/2026-06-19-drop-mutable-session-summary.md) | 2026-06-19 | +| [Shared persistence write coordinator](implemented/2026-06-18-shared-persistence-write-coordinator.md) | 2026-06-18 | ## Rejected diff --git a/docs/rfc/implemented/2026-06-18-shared-persistence-write-coordinator.md b/docs/rfc/implemented/2026-06-18-shared-persistence-write-coordinator.md new file mode 100644 index 0000000000..59ee9bb7c9 --- /dev/null +++ b/docs/rfc/implemented/2026-06-18-shared-persistence-write-coordinator.md @@ -0,0 +1,37 @@ +# RFC: Shared persistence write coordinator + +Status: implemented (proposed and accepted 2026-06-18, implemented 2026-06-20) + +## Problem + +`dsh-session-persistence-jsonl` and `dsh-session-persistence-sqlite` intentionally prove the same `SessionPersistence` contract over different storage media, but their write-path orchestration was duplicated: per-session state, `session/created` adoption, backend-specific prefix reads, write-behind buffers, serialized flush chains, HMR seeding, and dispose drains. The pure seed-prefix collision and serializability guards had already moved into the seam package; the remaining orchestration was still correctness-heavy and received the same fixes twice. A code-level diff showed the two backends were byte-identical — or same-algorithm — for ALL of it: the four maps (`states`/`buffers`/`chains`/`inits`), `installWritePath`, `initFor`, `onCreated`'s four cases, `flush`, `drain`, `serialize`, `adopt`, `adoptLivePrefix`, `assertVersion`, and the `create`/`append`/`load`/`has`/`delete` skeletons. Only the storage primitives (write bytes vs. INSERT rows) differed. + +## Decision + +Extract a backend-agnostic `PersistenceCoordinator` into `dsh-session-persistence`. The coordinator owns the orchestration once; each first-party backend composes one (`new PersistenceCoordinator(ctx, this)`), implements a small `PersistenceBackend` hook interface, and delegates its six public service methods (`create`/`append`/`load`/`list`/`has`/`delete`) to it. + +Composition, not inheritance. The coordinator is a concrete class the backend holds, not a base class the backend extends. The RFC's risk — "a coordinator must not make unusual backends fight an inheritance hierarchy" — is avoided: a backend exposes only the hooks; it cannot reach the coordinator's private orchestration state, and the public `SessionPersistence` service shape is unchanged, so a third-party backend MAY still implement the abstract service directly without the coordinator at all. + +### The hook interface (`PersistenceBackend`) + +Seven methods (six required + an optional lifecycle hook) — the only seam between the coordinator and storage: + +- `name` — backend label for the dispose-failure `AggregateError`. +- `loadStored(id)` — read a stored prefix by id, scanning ANY storage scope (every JSONL cwd bucket; SQLite's id is globally unique). Used by resume/load and, via `!== undefined`, the create-collision probe and `has`. +- `loadLive(id, cwd)` — read a stored prefix SCOPED to `cwd`. **Deliberately distinct from `loadStored`**: HMR live-adoption must only adopt a persisted log at the SAME cwd as the live session; a same-id log at a different cwd is a collision, not a resume. Collapsing the two reintroduces a cross-cwd adoption bug. SQLite ignores `cwd`. +- `appendBatch(meta, events, isMaterialized)` — durably append a contiguous batch, lazily materializing the session ATOMICALLY when not yet materialized (the materialize-write and the first event batch must commit together — a crash between them must not leave a materialized-but-empty session; this is why there is no separate `materialize` hook). +- `commitRepair(meta, tornMarker, closers)` — make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined`) and append `closers`. **NOT required to be atomic** — JSONL legitimately truncates-then-appends in two fsync'd steps, SQLite does DELETE+INSERT in one transaction. Used by `load` (truncate + synthetic closers) and live-adoption (truncate only, `closers = []`). +- `deleteStored(id)` / `list()` — remove a stored artifact / list all stored metadata. +- `close?()` — optional lifecycle teardown (SQLite closes its db handle; JSONL omits it), awaited in the dispose effect AFTER the quiescence drain so a close failure never masks a drain error. + +### The opaque torn marker + +The single design choice that keeps the seam clean: the crash-repair "where is the torn tail" token is OPAQUE to the coordinator. The coordinator computes the synthetic closers (it owns `interruptedTurnClosers` from `dsh-session`), but it only ever tests `tornMarker !== undefined` and passes the value straight back to `commitRepair` — it never inspects it. Each backend picks its own marker type: JSONL uses the byte offset to truncate to, SQLite the seq to delete from (both happen to be `number`). The JSONL backend folds its `committedBytes < buffer.byteLength` comparison INSIDE the hook so the returned marker is already `number | undefined`; without that fold the coordinator would have to know about byte lengths. + +## Testing + +The shared `runPersistenceContract` (public-API contract) keeps running for every backend. A new `runCoordinatorContract` (`tests/coordinator-contract.ts`) holds the write-path orchestration — adoption, HMR, collision, dispose-drain, crash-tail repair — and runs once per backend through a `CoordinatorFixture` (an in-memory reference + jsonl + sqlite). The per-backend specs shrank to storage mechanics only (JSONL: path safety, fsync rollback, bucket listing; SQLite: schema version, `scanRows`, transaction rollback). A through-coordinator torn-tail→load→`commitRepair` test per real backend (via a `corruptTail` fixture hook) keeps the coordinator's torn-marker repair branch covered under the 100% per-file gate — the contract crash test only produces synthetic closers, never a torn marker, so it could not reach that branch. + +## Risks and what we gave up + +The pre-extraction duplication was verbose but explicit — each backend read top-to-bottom. The coordinator adds one indirection (the hook seam) and one new concept (the opaque torn marker). This clears the bar because the centralized logic is the correctness-heavy part that was already being fixed twice, and the hook set is narrow (seven methods, no inheritance). The hook surface was deliberately held to the minimum: `has` and the create-collision probe are NOT separate hooks — they fold into `loadStored(id) !== undefined`; there is no separate `materialize` hook (folded into `appendBatch` for atomicity); `list()` stays a backend method with no coordinator pass-through (listing needs none of the orchestration). The net effect is a reduction: one orchestration copy instead of two, the backends shrank by ~1200 lines of duplicated churn, and a future backend implements ~7 small primitives instead of copying the entire `session/event` → buffer → flush machinery. diff --git a/docs/rfc/implemented/2026-06-19-drop-mutable-session-summary.md b/docs/rfc/implemented/2026-06-19-drop-mutable-session-summary.md index f2b82fe2f9..6f73664c26 100644 --- a/docs/rfc/implemented/2026-06-19-drop-mutable-session-summary.md +++ b/docs/rfc/implemented/2026-06-19-drop-mutable-session-summary.md @@ -20,7 +20,7 @@ Delete the mutable session summary entirely. `SessionSummary` and the `SessionMe Anything the summary was meant to provide is **derivable from the append-only log** when a consumer actually needs it (`firstPrompt` = first `user/message`; recency = the last event's `time` or the file mtime) or already lives in the immutable header (`createdAt`, `cwd`). The one thing *not* derivable — a user-*edited* title — had no implementation and is pure YAGNI; it can return as its own log event or header field if a real feature ever needs it. -This is recorded as a decision because it is **durable** (it narrows a public service contract and an on-disk format across two backends), **contested** (the summary was a deliberate forward-looking design, not an accident), and **surprising** (a future reader finding `SessionHeader` where the original RFC describes `SessionMeta` would otherwise ask why the summary vanished). It also unblocks the [shared persistence write coordinator](../proposed/2026-06-18-shared-persistence-write-coordinator.md): with no mutable summary, the coordinator's hook interface needs no `updateSummary` hook and the JSONL-sidecar-vs-SQLite-column durability divergence disappears, so the two backends' write paths converge. +This is recorded as a decision because it is **durable** (it narrows a public service contract and an on-disk format across two backends), **contested** (the summary was a deliberate forward-looking design, not an accident), and **surprising** (a future reader finding `SessionHeader` where the original RFC describes `SessionMeta` would otherwise ask why the summary vanished). It also unblocks the [shared persistence write coordinator](2026-06-18-shared-persistence-write-coordinator.md): with no mutable summary, the coordinator's hook interface needs no `updateSummary` hook and the JSONL-sidecar-vs-SQLite-column durability divergence disappears, so the two backends' write paths converge. ## No migration diff --git a/docs/rfc/proposed/2026-06-18-shared-persistence-write-coordinator.md b/docs/rfc/proposed/2026-06-18-shared-persistence-write-coordinator.md deleted file mode 100644 index fbb7defcd0..0000000000 --- a/docs/rfc/proposed/2026-06-18-shared-persistence-write-coordinator.md +++ /dev/null @@ -1,24 +0,0 @@ -# RFC: Shared persistence write coordinator - -Status: proposed - -## Problem - -`dsh-session-persistence-jsonl` and `dsh-session-persistence-sqlite` intentionally prove the same `SessionPersistence` contract over different storage media, but their write-path orchestration is now duplicated: per-session state, `session/created` adoption, backend-specific prefix reads, write-behind buffers, serialized flush chains, HMR seeding, and dispose drains. The pure seed-prefix collision and serializability guards have already moved into the seam package; the remaining orchestration is still correctness-heavy and already receives the same fixes twice. - -## Proposal - -Extract a backend-agnostic coordinator into `dsh-session-persistence`. The coordinator owns live-session adoption, buffering, cursor filtering, per-id serialization, and disposal quiescence. Concrete backends provide small hooks for durable operations: create lazy state, find/load stored prefix, append a contiguous batch, delete, and list. - -The public `SessionPersistence` service shape can stay the same. The coordinator can be an internal exported helper or protected base class used by first-party backends; third-party backends may still implement the abstract service directly if their write path is different. - -## Acceptance Criteria - -- JSONL and SQLite keep passing the existing shared `runPersistenceContract`. -- HMR/adoption/collision tests move to a shared coordinator test suite and run once for each backend through hook-driven fixtures. -- Backend-specific tests focus on storage mechanics only: JSONL path safety/fsync behavior and SQLite schema/WAL/transaction behavior. -- A future backend does not need to copy the current `session/event` → buffer → flush orchestration. - -## Risks - -The current duplication is verbose but explicit. A coordinator must not hide storage-specific durability semantics or make unusual backends fight an inheritance hierarchy. Prefer narrow hooks and contract tests over a large framework. diff --git a/packages/session-persistence-jsonl/src/index.ts b/packages/session-persistence-jsonl/src/index.ts index ef921f2b8f..a469095c5d 100644 --- a/packages/session-persistence-jsonl/src/index.ts +++ b/packages/session-persistence-jsonl/src/index.ts @@ -1,20 +1,18 @@ /** * JSONL durable session-persistence backend (`@deepseek-ai/dsh-session-persistence-jsonl`). * - * Two concerns in one plugin: + * One append-only `.jsonl` event log per session (a header line then one + * `SessionEvent` per line, verbatim including `assistant/chunk` so `seq` stays + * contiguous), with lazy materialization (no file until the first `append`), + * atomic first write, and load-time repair of a never-committed crash tail. * - * 1. **The backend** — a concrete {@link SessionPersistence}: one append-only - * `.jsonl` event log per session (a header line then one `SessionEvent` per - * line, verbatim including `assistant/chunk` so `seq` stays contiguous). - * Lazy materialization (no file until the first `append`), atomic first - * write, and load-time repair of a never-committed crash tail. - * - * 2. **The write path** — the `session/event` → buffer → `session/flush` drain - * that generalizes the example `session-jsonl.ts`: snapshot each event when - * it is buffered (the live `session.events` object is mutable), persist - * forks once on `session/created`, maintain a per-session write cursor so a - * resumed session never re-appends already-stored events, and seed existing - * live sessions on plugin apply (HMR does not replay `session/created`). + * The backend supplies ONLY the file-bytes storage primitives (the + * {@link PersistenceBackend} hooks below); all the write-path orchestration + * (the `session/event` → buffer → `session/flush` drain, per-session + * serialization, write cursors, fork-seed persistence, HMR live-adoption, + * crash-repair sequencing, dispose quiescence) lives in the backend-agnostic + * {@link PersistenceCoordinator} this class composes. The six public + * {@link SessionPersistence} methods delegate to the coordinator. * * @module @deepseek-ai/dsh-session-persistence-jsonl */ @@ -25,9 +23,9 @@ import { open, mkdir, readFile, readdir, link, rm, truncate } from 'node:fs/prom import { dirname, resolve } from 'node:path' import { randomBytes } from 'node:crypto' import { - SessionPersistence, assertSerializable, seedCoversPrefix, + SessionPersistence, PersistenceCoordinator, + type PersistenceBackend, type StoredPrefix, } from '@deepseek-ai/dsh-session-persistence' -import { interruptedTurnClosers } from '@deepseek-ai/dsh-session' import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' import { encodeSegment, eventLine, logPath, parseHeaderMeta, scanLog, sessionDir, toHeaderLine, @@ -42,267 +40,152 @@ export interface Config { root: string } -/** Per-session write state held by the backend's in-memory bookkeeping. */ -interface SessionState { - meta: SessionHeader - /** The next seq the backend expects to append (the stored log length). */ - cursor: number - /** Whether the `.jsonl` file has been physically materialized. */ - materialized: boolean - /** - * The live Session this state was bound to via `onCreated`, if any. Used to - * detect a DIFFERENT live session reusing a tracked id (a collision): state - * created through the public `create()`/`load()` API has no owner, but state - * bound to a live session lets `onCreated` reject a second, unrelated session - * object on the same id instead of silently no-opping (which would leave the - * new session's events to be dropped against the old cursor). - */ - owner?: Session -} - /** * Whether `error` is a "no such file/directory" (`ENOENT`) failure — the ONLY * filesystem error that legitimately means "this session/root is absent" for a * durable backend. Any OTHER error (`EACCES`, `ENOTDIR`, transient I/O) must - * surface rather than be silently reported as absence: masking it would let - * `list()` report no sessions, `load()` report "not found", and collision - * checks proceed under a false absence assumption — all unsafe for durable - * persistence. (A NodeJS filesystem rejection carries a string `code`.) + * surface rather than be silently reported as absence. (A NodeJS filesystem + * rejection carries a string `code`.) */ function isENOENT(error: unknown): boolean { return (error as NodeJS.ErrnoException | null)?.code === 'ENOENT' } -async function settledErrors(promises: Iterable>): Promise { - const settled = await Promise.allSettled([...promises]) - const errors: unknown[] = [] - for (const result of settled) { - if (result.status === 'rejected') errors.push(result.reason) - } - return errors -} - /** * The JSONL persistence backend. Load as a plugin; it registers as - * `ctx.sessionPersistence` and installs the write-path listeners. + * `ctx.sessionPersistence` and (via the coordinator) installs the write-path + * listeners. Its torn-tail marker is the byte offset to truncate the log to. */ -export class SessionPersistenceJsonl extends SessionPersistence { +export class SessionPersistenceJsonl extends SessionPersistence implements PersistenceBackend { static inject = ['sessions'] static Config: z = z.object({ root: z.string().required(), }) + /** + * Backend label for the coordinator's dispose-failure AggregateError and + * effect name. NOTE: this intentionally shadows cordis `Service.name` (which + * the base sets to `'sessionPersistence'`). The service is registered under the + * fixed key the Service constructor captured (`reflect.provide('sessionPersistence', …)`), + * not via `this.name`, so overwriting the instance field with the backend label + * does not affect `ctx.sessionPersistence` resolution — it only relabels the + * dispose diagnostics, which is exactly what {@link PersistenceBackend.name} is for. + */ + override readonly name = 'session-persistence-jsonl' + private root: string - /** Backend bookkeeping keyed by session id (NOT the live Session object). */ - private states = new Map() - /** Write-behind buffers keyed by the live Session (write path). */ - private buffers = new Map() - /** - * Per-session serialization: every backend operation chains onto the prior - * one for the same id, so concurrent flushes / a flush racing onCreated never - * interleave file writes or read a half-built state. Keyed by session id. - */ - private chains = new Map>() - /** - * Per-session init promise (onCreated). Keyed by the LIVE Session OBJECT, not - * its id: a disposed fiber's session can be replaced by a different live - * Session reusing the same id (HMR, an ACP reconnect), and an id-keyed cache - * would hand the new object the old object's init promise — skipping - * onCreated for the new session, so its events start at seq 0 while flush - * filters against the stale cursor and silently drops them. Keying by object - * gives each live Session its own init. flush awaits it before appending. - */ - private inits = new Map>() + private coordinator: PersistenceCoordinator constructor(ctx: Context, public config: Config) { super(ctx) - // Resolve the configured root to an ABSOLUTE path ONCE, here. A relative - // root (the examples use `./.sessions`) would otherwise re-resolve against - // `process.cwd()` at every later readdir/open — so if any plugin or test - // changed cwd between create, append, and load, one session's files could - // split across directories. Pinning it at construction makes all paths - // stable regardless of later cwd changes. + // Resolve the configured root to an ABSOLUTE path ONCE, here. A relative root + // would otherwise re-resolve against `process.cwd()` at every later + // readdir/open — so if any plugin or test changed cwd between create, append, + // and load, one session's files could split across directories. this.root = resolve(config.root) - this.installWritePath() + this.coordinator = new PersistenceCoordinator(this.ctx, this) } - // --- SessionPersistence backend surface (all serialized per session id) --- + // --- SessionPersistence service surface (delegated to the coordinator) --- create(meta: SessionHeader): Promise { - // Snapshot the metadata at call time: the op runs later (behind the - // per-session chain) and the snapshot is also stored as the lazy state, so - // keeping the caller's object by reference would let a later mutation of - // `id`/`cwd` register under one key but materialize under a different - // path/header. A shallow copy is enough — SessionHeader is a flat record. - const snapshot: SessionHeader = { ...meta } - return this.serialize(snapshot.id, () => this.createCore(snapshot)) + return this.coordinator.create(meta) } - private async createCore(meta: SessionHeader): Promise { - // Do NOT clobber an existing session. If we already track it, or a log - // exists on disk under this id, refuse — the SessionId IS the identity, and - // silently resetting state (cursor 0, materialized false) over committed - // data would let the next append rename over the existing log. - if (this.states.has(meta.id)) { - throw new Error(`session "${meta.id}" already exists in this backend`) - } - // Scan ALL cwd buckets (pass undefined), not just meta.cwd's: load/has/adopt - // identify a session by id alone and search every bucket, so an id already - // persisted under a DIFFERENT cwd must still block creation here. Probing - // only meta.cwd's bucket would let two logs share one id and make resume - // (which picks the first matching bucket) nondeterministic. - if (await this.findLog(meta.id, undefined) !== undefined) { - throw new Error(`session "${meta.id}" already has a persisted log on disk; load/resume it instead of creating`) - } - // Pure lazy: record intent only. No file until the first append, so an - // abandoned (never-appended) session leaves nothing on disk and stays - // absent from has()/list(). - this.states.set(meta.id, { meta, cursor: 0, materialized: false }) - } - - /** - * Run `op` after any in-flight operation for the same session id, so writes - * for one session never interleave (two flushes, a flush racing a load, an - * update racing an append). Errors do not poison the chain — the next op - * still runs. NOTE: serialized public methods must NOT call each other (that - * would deadlock on the same chain); they call the unserialized `*Core` - * helpers instead. - */ - private serialize(id: SessionId, op: () => Promise): Promise { - const prior = this.chains.get(id) ?? Promise.resolve() - const next = prior.then(op, op) - // Keep the chain alive but swallow this op's rejection for the NEXT waiter - // (the caller still sees the real rejection via `next`). - this.chains.set(id, next.then(() => undefined, () => undefined)) - return next - } - - // `async` so the synchronous validate/clone below reject (not throw) per the - // Promise contract — callers use `await expect(...).rejects`. - async append(id: SessionId, events: readonly SessionEvent[]): Promise { - // Validate serializability BEFORE cloning, so a bad event surfaces the typed - // "non-JSON-serializable" error rather than an opaque DataCloneError from - // structuredClone below. (In an async method this throw becomes a rejection, - // honoring the Promise contract rather than throwing synchronously.) - assertSerializable(events) - // Deep-snapshot the batch here, BEFORE the op waits behind the per-session - // chain: the op may await before serializing, so a caller that passes a live - // array (e.g. session.events) and mutates it — OR mutates an event object - // inside it — before the op runs would otherwise have those changes - // persisted, or advance the cursor past what was actually written. - // structuredClone covers both the array and the event objects (safe now that - // serializability is checked above). The clone happens synchronously (before - // the first await), so it is taken at call time. - const batch = events.map(e => structuredClone(e)) - return this.serialize(id, () => this.appendCore(id, batch)) - } - - private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise { - if (events.length === 0) return - assertSerializable(events) - let state = this.states.get(id) - if (state === undefined) state = await this.adopt(id) // calls loadCore, not load - - // Contiguity contract: each event's seq must continue the stored log. - for (const [i, event] of events.entries()) { - if (event.seq !== state.cursor + i) { - throw new Error(`append seq mismatch for "${id}": expected ${state.cursor + i} at index ${i}, got ${event.seq}`) - } - } - - if (!state.materialized) { - await this.materialize(state, events) - } else { - await this.appendLines(state, events) - } - // The durable event log is the transaction: advance the cursor as soon as - // the log write commits. - state.cursor += events.length + append(id: SessionId, events: readonly SessionEvent[]): Promise { + return this.coordinator.append(id, events) } load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { - return this.serialize(id, () => this.loadCore(id)) + return this.coordinator.load(id) } - private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { - const cwd = this.states.get(id)?.meta.cwd + has(id: SessionId): Promise { + return this.coordinator.has(id) + } + + delete(id: SessionId): Promise { + return this.coordinator.delete(id) + } + + // `list` is BOTH the public service method and the PersistenceBackend hook — + // one method, the bucket walk below. The coordinator adds no orchestration for + // listing (no per-id serialization, no cursor), so it would just call back into + // this same method; routing it through the coordinator would recurse. Defined + // once, in the "PersistenceBackend hooks" section. + + /** + * The per-session init promises, exposed for white-box tests that await a + * specific session's onCreated (there is no public API to await one init). + */ + get inits(): Map> { + return this.coordinator.inits + } + + // --- PersistenceBackend hooks (the file-bytes storage primitives) --- + + /** Read a stored prefix by id across ALL cwd buckets (cwd unknown). */ + async loadStored(id: SessionId): Promise | undefined> { + return this.readPrefix(id, undefined) + } + + /** Read a stored prefix SCOPED to `cwd` (HMR live-adoption must not cross cwd). */ + async loadLive(id: SessionId, cwd: string | undefined): Promise | undefined> { + return this.readPrefix(id, cwd) + } + + /** + * Read and scan a session's log into a {@link StoredPrefix}. Folds the + * torn-tail comparison HERE so the `tornMarker` is the byte offset to truncate + * to (or `undefined` when nothing is torn) — the coordinator never sees the + * raw byteLength. + */ + private async readPrefix(id: SessionId, cwd: string | undefined): Promise | undefined> { const file = await this.findLog(id, cwd) - if (file === undefined) throw new Error(`session "${id}" not found`) + if (file === undefined) return undefined const buffer = await readFile(file.path) const { meta, events, committedBytes } = scanLog(buffer) - this.assertVersion(meta) - - // Crash-recovery: if the log ended mid-turn (an open turn with real, - // preserved events but no closing turn/end), close it durably DURING load so - // disk, the returned log, and the cursor all agree — both append routes then - // continue with no special-casing. Synthesize the boundary events (a - // step/end if a step was open, then a turn/end {kind:'interrupted'}); the - // interrupted turn's real events are preserved, never truncated (a turn can - // be huge — the session-persistence RFC). - const closers = interruptedTurnClosers(events) - const balanced = [...events, ...closers] - - // Set state BEFORE the repair writes so they can resolve the log path. - const needsTorn = committedBytes < buffer.byteLength - const state: SessionState = { - meta: { ...meta }, - cursor: events.length, - materialized: true, + return { + meta, + events, + ...committedBytes < buffer.byteLength ? { tornMarker: committedBytes } : {}, } - this.states.set(id, state) - - if (needsTorn) { - // Discard the torn trailing fragment (a final line never fully flushed) - // before writing the closers, so the closers land at a clean EOF. - await this.repair(state, committedBytes) - } - if (closers.length > 0) { - // Durably append the synthetic closers, then advance the cursor to the - // balanced length. After this, disk == balanced and the next append (live - // or direct) continues cleanly. - await this.appendLines(state, closers) - state.cursor = balanced.length - } - - return { meta, events: balanced } } - private async adoptLiveDiskPrefix( - session: Session, - seed: readonly SessionEvent[], - file: { path: string; cwd: string | undefined }, - ): Promise { - const buffer = await readFile(file.path) - const { meta, events, committedBytes } = scanLog(buffer) - this.assertVersion(meta) - if (!seedCoversPrefix(seed, events)) { - throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`) + /** Durably append a batch, lazily materializing the file when not yet present. */ + async appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise { + if (isMaterialized) { + await this.appendLines(meta, events) + } else { + await this.materialize(meta, events) } - - const state: SessionState = { - meta: { ...meta }, - cursor: events.length, - materialized: true, - owner: session, - } - this.states.set(session.header.id, state) - - if (committedBytes < buffer.byteLength) { - await this.repair(state, committedBytes) - } - const suffix = seed.slice(events.length) - if (suffix.length > 0) await this.appendCore(session.header.id, suffix) } + /** + * Make a crash repair durable: truncate the torn tail to `tornMarker` bytes (if + * any), then append the synthetic `closers` (if any). Two fsync'd steps — the + * seam does not require this to be atomic. + */ + async commitRepair(meta: SessionHeader, tornMarker: number | undefined, closers: readonly SessionEvent[]): Promise { + if (tornMarker !== undefined) await this.repair(meta, tornMarker) + if (closers.length > 0) await this.appendLines(meta, closers) + } + + /** Remove a session's log file (the coordinator clears its in-memory state). */ + async deleteStored(id: SessionId): Promise { + const file = await this.findLog(id, undefined) + if (file) await rm(file.path, { force: true }) + } + + /** List all stored sessions' metadata (header line only — no full-log parse). */ async list(): Promise { const metas: SessionHeader[] = [] for (const dir of await this.listCwdDirs()) { for (const name of await this.listJsonl(dir)) { // Read ONLY the header line, not the whole log: a session picker must // scale with the number of sessions, not the total size of every - // conversation (the log persists every assistant/chunk verbatim, so a - // full scanLog here would be O(total history)). + // conversation (the log persists every assistant/chunk verbatim). const first = await this.readFirstLine(`${dir}/${name}`) if (first === undefined) continue // empty/half-written file const meta = parseHeaderMeta(first) @@ -313,11 +196,119 @@ export class SessionPersistenceJsonl extends SessionPersistence { return metas } + // --- materialization / append / repair (file mechanics) --- + + /** Atomically write the header line + first batch (temp-write, fsync, rename). */ + private async materialize(meta: SessionHeader, events: readonly SessionEvent[]): Promise { + const dir = sessionDir(this.root, meta.cwd) + await mkdir(this.root, { recursive: true, mode: 0o700 }) + await this.syncDir(dirname(this.root)) + await mkdir(dir, { recursive: true, mode: 0o700 }) + await this.syncDir(this.root) + const finalPath = logPath(this.root, meta.cwd, meta.id) + // Never rename over an existing committed log: materialize is the FIRST write + // of a session the backend believes is new. A file here means a different + // session shares this id on disk — reject loudly. (createCore already guards + // the create path, so this is unreachable-in-practice TOCTOU defense.) + /* v8 ignore next 3 -- createCore guards collisions before materialize; this is a TOCTOU backstop */ + if (await this.exists(finalPath)) { + throw new Error(`refusing to materialize "${meta.id}": a log already exists on disk (load/resume it instead)`) + } + const header = JSON.stringify(toHeaderLine(meta)) + const body = events.map(eventLine).join('\n') + const content = header + '\n' + body + '\n' + + const tmp = `${finalPath}.${randomBytes(6).toString('hex')}.tmp` + const handle = await open(tmp, 'wx', 0o600) + try { + await handle.writeFile(content) + await handle.sync() + } finally { + await handle.close() + } + // Publish via link()+unlink(), NOT rename(): link fails with EEXIST if the + // final path already exists, so two processes materializing the same id + // concurrently cannot clobber each other. rename() would silently overwrite. + let linked = false + try { + await link(tmp, finalPath) + linked = true + } finally { + // If link FAILED, the temp is the only reference and must be removed before + // the original error propagates. If it SUCCEEDED, defer temp cleanup to + // AFTER the publish is durable (below) so a temp-rm failure can never reject + // a session whose log already published. + /* v8 ignore next -- link failure is the TOCTOU/IO race guarded above; not reachable in test */ + if (!linked) await rm(tmp, { force: true }) + } + // link() succeeded — the log is published. fsync the directory so the new + // entry survives a power loss: the new link is not crash-durable until the + // parent directory's metadata is synced. + await this.syncDir(dir) + // Best-effort temp cleanup: the log is already published and durable, so a + // failure to remove the (now-redundant) temp hard link must NOT reject the + // append. Swallow only the rm failure; nothing else of consequence runs here. + try { + await rm(tmp, { force: true }) + } catch { + /* v8 ignore next -- redundant temp link; publish already durable, rm failure is an unreachable IO edge */ + } + } + + /** fsync a directory so a just-created/renamed entry inside it is crash-durable. */ + private async syncDir(dir: string): Promise { + const handle = await open(dir, 'r') + try { + await handle.sync() + } finally { + await handle.close() + } + } + + /** + * Append event lines at EOF and fsync. On a write/sync failure AFTER the kernel + * accepted some bytes (ENOSPC, an fsync error), truncate the file back to its + * pre-append size before rethrowing: the cursor is unchanged, so the batch will + * be retried, and without this rollback the retry would append AFTER the partial + * bytes — producing duplicate seqs that make `scanLog` see a gap. + */ + private async appendLines(meta: SessionHeader, events: readonly SessionEvent[]): Promise { + const path = logPath(this.root, meta.cwd, meta.id) + const handle = await open(path, 'a') + try { + const { size: before } = await handle.stat() + try { + await handle.writeFile(events.map(eventLine).join('\n') + '\n') + await handle.sync() + } catch (error) { + // Roll back whatever bytes landed so a retry starts from a clean EOF. + await handle.truncate(before) + await handle.sync() + throw error + } + } finally { + await handle.close() + } + } + + /** Truncate the log file to `offset` bytes and fsync (discard the crash tail). */ + private async repair(meta: SessionHeader, offset: number): Promise { + const path = logPath(this.root, meta.cwd, meta.id) + await truncate(path, offset) + const handle = await open(path, 'r+') + try { + await handle.sync() + } finally { + await handle.close() + } + } + + // --- discovery helpers --- + /** * Read the first newline-terminated line of a file without loading the whole - * file. Returns undefined if the file is empty or has no complete first line - * (a half-written log). Reads in bounded chunks so a huge log costs only the - * header read. + * file. Returns undefined if the file is empty or has no complete first line. + * Reads in bounded chunks so a huge log costs only the header read. */ private async readFirstLine(path: string): Promise { const handle = await open(path, 'r') @@ -340,145 +331,6 @@ export class SessionPersistenceJsonl extends SessionPersistence { } } - async has(id: SessionId): Promise { - const state = this.states.get(id) - if (state?.materialized) return true - const cwd = state?.meta.cwd - return (await this.findLog(id, cwd)) !== undefined - } - - delete(id: SessionId): Promise { - return this.serialize(id, () => this.deleteCore(id)) - } - - private async deleteCore(id: SessionId): Promise { - const cwd = this.states.get(id)?.meta.cwd - const file = await this.findLog(id, cwd) - if (file) await rm(file.path, { force: true }) - this.states.delete(id) - } - - // --- materialization / append / repair --- - - /** Atomically write the header line + first batch (temp-write, fsync, rename). */ - private async materialize(state: SessionState, events: readonly SessionEvent[]): Promise { - const dir = sessionDir(this.root, state.meta.cwd) - await mkdir(this.root, { recursive: true, mode: 0o700 }) - await this.syncDir(dirname(this.root)) - await mkdir(dir, { recursive: true, mode: 0o700 }) - await this.syncDir(this.root) - const finalPath = logPath(this.root, state.meta.cwd, state.meta.id) - // Never rename over an existing committed log: materialize is the FIRST - // write of a session the backend believes is new. A file here means a - // different session shares this id on disk — reject loudly rather than - // clobber committed data. (createCore already guards the create path before - // this point, so this is unreachable-in-practice defense-in-depth against a - // TOCTOU/fork race; ignored for coverage.) - /* v8 ignore next 3 -- createCore guards collisions before materialize; this is a TOCTOU backstop */ - if (await this.exists(finalPath)) { - throw new Error(`refusing to materialize "${state.meta.id}": a log already exists on disk (load/resume it instead)`) - } - const header = JSON.stringify(toHeaderLine(state.meta)) - const body = events.map(eventLine).join('\n') - const content = header + '\n' + body + '\n' - - const tmp = `${finalPath}.${randomBytes(6).toString('hex')}.tmp` - const handle = await open(tmp, 'wx', 0o600) - try { - await handle.writeFile(content) - await handle.sync() - } finally { - await handle.close() - } - // Publish via link()+unlink(), NOT rename(): link fails with EEXIST if the - // final path already exists, so two processes materializing the same id - // concurrently cannot clobber each other (both could pass the exists() check - // above, but only one link() wins). rename() would silently overwrite the - // log the other process just committed. - let linked = false - try { - await link(tmp, finalPath) - linked = true - } finally { - // If link FAILED (EEXIST on a race, or any I/O error), the temp is the - // only reference and must be removed before the original error propagates. - // If link SUCCEEDED, the temp cleanup is deferred to AFTER the publish is - // durable (below) so a temp-rm failure can never reject a session whose - // log already published — that would leave state.materialized false and - // wedge every retry on the exists() backstop above. - /* v8 ignore next -- link failure is the TOCTOU/IO race guarded above; not reachable in test */ - if (!linked) await rm(tmp, { force: true }) - } - // link() succeeded — the log is published. fsync the directory so the new - // entry survives a power loss: on POSIX filesystems the new link is not - // crash-durable until the parent directory's metadata is synced. The seam - // contract is "append returns once durable", and materialize is the first - // append's write — so the directory entry must be durable before we return. - await this.syncDir(dir) - state.materialized = true - // Best-effort temp cleanup: the log is already published and durable, so a - // failure to remove the (now-redundant) temp hard link must NOT reject the - // append. A leftover `*.tmp` is harmless — it is never read, and the next - // materialize of this id is guarded by exists()/link(). Swallow only the - // rm failure; nothing else of consequence runs in the try. - try { - await rm(tmp, { force: true }) - } catch { - /* v8 ignore next -- redundant temp link; publish already durable, rm failure is an unreachable IO edge */ - } - } - - /** fsync a directory so a just-created/renamed entry inside it is crash-durable. */ - private async syncDir(dir: string): Promise { - const handle = await open(dir, 'r') - try { - await handle.sync() - } finally { - await handle.close() - } - } - - /** - * Append event lines at EOF and fsync. On a write/sync failure AFTER the - * kernel accepted some bytes (ENOSPC, an fsync error), truncate the file back - * to its pre-append size before rethrowing: `cursor` is unchanged, so the - * batch will be retried, and without this rollback the retry would append - * AFTER the partial bytes — producing duplicate seqs that make `scanLog` see a - * gap and render the session unloadable. - */ - private async appendLines(state: SessionState, events: readonly SessionEvent[]): Promise { - const path = logPath(this.root, state.meta.cwd, state.meta.id) - const handle = await open(path, 'a') - try { - const { size: before } = await handle.stat() - try { - await handle.writeFile(events.map(eventLine).join('\n') + '\n') - await handle.sync() - } catch (error) { - // Roll back whatever bytes landed so a retry starts from a clean EOF. - await handle.truncate(before) - await handle.sync() - throw error - } - } finally { - await handle.close() - } - } - - /** Truncate the log file to `offset` bytes and fsync (discard the crash tail). */ - private async repair(state: SessionState, offset: number): Promise { - const path = logPath(this.root, state.meta.cwd, state.meta.id) - await truncate(path, offset) - const handle = await open(path, 'r+') - try { - await handle.sync() - } finally { - await handle.close() - } - } - - // --- discovery helpers --- - /** Find a session's log file across cwd buckets (when cwd is unknown). */ private async findLog(id: SessionId, cwd: string | undefined): Promise<{ path: string; cwd: string | undefined } | undefined> { if (cwd !== undefined) { @@ -490,8 +342,7 @@ export class SessionPersistenceJsonl extends SessionPersistence { for (const dir of await this.listCwdDirs()) { const path = `${dir}/${target}` if (await this.exists(path)) { - // Recover the cwd from the header so the caller has the session's - // bucket location (which `findLog` was given an unknown cwd for). + // Recover the cwd from the header so the caller has the session's bucket. const { meta } = scanLog(await readFile(path)) return { path, cwd: meta.cwd } } @@ -505,10 +356,9 @@ export class SessionPersistenceJsonl extends SessionPersistence { const entries = await readdir(this.root, { withFileTypes: true }) return entries.filter(e => e.isDirectory()).map(e => `${this.root}/${e.name}`) } catch (error) { - // ENOENT = the root has not been created yet → genuinely no sessions. - // Any other error (EACCES, ENOTDIR, transient I/O) must NOT be reported - // as "no sessions" — a durable backend cannot silently pretend persisted - // state is absent on a storage fault. + // ENOENT = the root has not been created yet → genuinely no sessions. Any + // other error (EACCES, ENOTDIR, transient I/O) must NOT be reported as "no + // sessions" — a durable backend cannot silently pretend state is absent. if (isENOENT(error)) return [] throw error } @@ -526,244 +376,12 @@ export class SessionPersistenceJsonl extends SessionPersistence { return true } catch (error) { // Only ENOENT means absent. A permission/I/O error must surface, not be - // collapsed to `false` — otherwise load() reports "not found" and - // collision checks proceed under a false absence assumption. + // collapsed to `false` — otherwise load() reports "not found" and collision + // checks proceed under a false absence assumption. if (isENOENT(error)) return false throw error } } - - /** Build a state for a session discovered on disk but not yet in memory. */ - private async adopt(id: SessionId): Promise { - // loadCore (NOT load) — adopt runs inside an already-serialized op, so - // re-entering the chain via the public load() would deadlock. - await this.loadCore(id) - const state = this.states.get(id) - /* v8 ignore next -- loadCore always sets the state for the id */ - if (!state) throw new Error(`failed to adopt session "${id}"`) - return state - } - - private assertVersion(meta: SessionHeader): void { - if (meta.version !== 1) { - throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`) - } - } - - // --- write path (session/event → flush drain) --- - - private installWritePath(): void { - const ctx = this.ctx - - // Capture the header on creation; persist a fork's seed once. Record the - // init promise so flush/dispose can await it (onCreated is async). - ctx.on('session/created', (session) => { void this.initFor(session) }) - - // Snapshot + buffer every event (the live object is mutable; clone so a - // later in-place mutation of session.events cannot rewrite a buffered - // event). Serializability is guaranteed at the source — `Session.append` - // rejects non-JSON-serializable data before the event ever enters the log - // or this emit — so structuredClone here can never hit a non-cloneable - // value, and the durable log can never diverge from session.events. - ctx.on('session/event', (session, event) => { - let buffer = this.buffers.get(session) - if (!buffer) this.buffers.set(session, buffer = []) - buffer.push(structuredClone(event)) - }) - - // Drain to the backend at the durability checkpoint. - ctx.on('session/flush', session => this.flush(session)) - - // Dispose must reach quiescence: await every session's init + final drain - // BEFORE returning, so no write lands after teardown (orphan rename/ENOENT). - ctx.effect(() => async () => { - const errors = [ - ...await settledErrors(this.inits.values()), - ...await settledErrors([...this.buffers.keys()].map(s => this.flush(s))), - ...await settledErrors(this.chains.values()), - ] - if (errors.length > 0) { - throw new AggregateError(errors, 'session-persistence-jsonl dispose failed') - } - }, 'session-persistence-jsonl write path') - - // HMR: a hot reload does not replay session/created, so seed existing live - // sessions (mirrors dsh-invariants). - for (const session of ctx.sessions.list()) void this.initFor(session) - } - - /** Start (once) the async init for a session and remember its promise. */ - private initFor(session: Session): Promise { - const existing = this.inits.get(session) - if (existing) return existing - // Snapshot the seed SYNCHRONOUSLY here — initFor runs inside the - // `session/created` emit, before any later `append` adds non-seed events. - // A clone freezes it against later mutation of the live event objects. - const seed = session.events.map(e => structuredClone(e)) - const p = this.onCreated(session, seed) - // Attach a no-op rejection handler so a failing init (e.g. an id collision) - // does not surface as an unhandled rejection if no flush observes `p` before - // it rejects. The REAL error is still delivered: flush/dispose await the - // same `p` from the map and see the rejection there. - p.catch(() => { /* observed by flush/dispose via the stored promise */ }) - this.inits.set(session, p) - return p - } - - /** - * Whether a live `session`'s `seed` reproduces the first `cursor` persisted - * events. Reads the on-disk committed prefix and compares. A `cursor` of 0 - * (nothing persisted yet) trivially matches. Used when a live session claims - * ownerless state left by a prior `load()`/`create()` — to reject a fresh, - * unrelated session that reuses the id and would otherwise have its seq - * 0..cursor-1 events filtered as already-written. - */ - private async seedMatchesPersisted(session: Session, seed: readonly SessionEvent[], cursor: number): Promise { - if (cursor === 0) return true - const onDisk = await this.findLog(session.header.id, session.header.cwd) - /* v8 ignore next -- a cursor > 0 means the log was materialized, so it exists */ - if (onDisk === undefined) return false - const { events: diskEvents } = scanLog(await readFile(onDisk.path)) - return seedCoversPrefix(seed, diskEvents.slice(0, cursor)) - } - - /** - * On session/created: sync the backend's in-memory state to a live Session. - * - * Cases, by whether this backend tracks the id and whether a log is on disk: - * 1. Already in `states` (created here, or a prior load/resume) → no-op. - * 2. Not tracked, a log EXISTS on disk, and it is a seq-aligned PREFIX of the - * live session's current events → ADOPT it (HMR/reload): a fresh backend - * instance (empty `states`) meets a live session whose log a previous - * instance materialized; the live object already carries that history (it - * is the source of truth this run), so we continue from the stored length - * instead of re-creating. This keeps persistence alive across hot reload. - * 3. Not tracked, a log EXISTS on disk, but it is NOT a prefix of the live - * session's events → REJECT: a different session collides on the id. The - * SessionId is the identity, so two unrelated sessions sharing one is a - * bug, not a resume — fail loudly rather than clobber committed data. - * 4. Not tracked and NO log on disk → a genuinely new session: register its - * meta (lazy) and persist its `seed` once. - * - * The public `create(meta)` API is stricter still (rejects ANY on-disk id): - * there the caller asserts "brand new", so even a prefix match is a bug. - * - * The seed events were copied into the Session by its constructor WITHOUT - * emitting session/event, so the write-behind buffer never sees them — the - * one explicit `append(seed)` below is the only persistence of the seed. - * Events appended AFTER creation flow through the session/event buffer and - * are persisted by flush (filtered by the write cursor), never here. - */ - private async onCreated(session: Session, seed: readonly SessionEvent[]): Promise { - const id = session.header.id - const tracked = this.states.get(id) - if (tracked !== undefined) { - // case 1: already tracked. - // (owner === session is a defensive same-object guard: initFor dedupes by - // session object, so onCreated never actually runs twice for one session.) - /* v8 ignore next -- initFor dedupes per session object; same-object re-entry can't occur */ - if (tracked.owner === session) return - if (tracked.owner === undefined) { - // Ownerless state was created via the public create()/load() API. The - // FIRST live session to arrive claims it — but ONLY if its seed is the - // already-persisted prefix. A load() for preview leaves cursor at the - // persisted length; a fresh, unrelated session reusing that id has a - // seed shorter than (or not matching) that prefix, so flush would filter - // its seq 0..cursor-1 events as already-written and silently graft the - // new conversation onto the old log. Verify the seed covers the cursor. - if (!await this.seedMatchesPersisted(session, seed, tracked.cursor)) { - throw new Error(`session "${id}" is already persisted with ${tracked.cursor} event(s) that do not match this live session (id collision)`) - } - tracked.owner = session - // Persist the live seed SUFFIX beyond the persisted prefix. Constructor - // seed events (from sessions.create(id, { seed })) never emit - // session/event, so the write-behind buffer never sees them — without - // this they would be lost and a later flush would seq-mismatch. (cursor - // is 0 for a public create(), so this covers the whole seed there.) - const suffix = seed.slice(tracked.cursor) - if (suffix.length > 0) await this.append(id, suffix) - return - } - // The state is owned by a DIFFERENT live session. We may reclaim the id - // ONLY if that owner left nothing behind: never materialized a log (cursor - // 0, not materialized) AND has no write-behind buffer still pending. A - // session that appended events but was disposed before its first flush is - // NOT materialized yet but DOES have buffered events — reclaiming then - // would let that stale buffer drain against the new session's state - // (persisting old events under the new id, or dropping the new session's - // seq-0 events). Such an owner, and any materialized owner, is a real - // collision and rejects; only a truly-abandoned (artifact-free) id is - // freed, honoring lazy materialization's "leaves nothing behind" promise. - const ownerBuffer = this.buffers.get(tracked.owner) - if (!tracked.materialized && !ownerBuffer?.length) { - this.states.delete(id) - } else { - throw new Error(`session "${id}" is already bound to a different live session in this backend (id collision)`) - } - } - - const onDisk = await this.findLog(id, session.header.cwd) - if (onDisk !== undefined) { - // case 2: adopt a LIVE prefix. Do NOT route through loadCore(): loadCore - // crash-repairs open turns as interrupted, which is right for a true load - // after a crash but wrong for HMR while the live Session is still the - // authority and may append the real step/turn end later. - await this.serialize(id, () => this.adoptLiveDiskPrefix(session, seed, onDisk)) - return - } - - // case 4: a genuinely new session. Register its meta (lazy), then persist - // its seed (events present at creation time) once. - const meta: SessionHeader = { ...session.header } - await this.create(meta) - // Bind this state to the live session so a later DIFFERENT session reusing - // the id is detected as a collision (case 1) rather than silently no-opped. - const created = this.states.get(id) - /* v8 ignore next -- create() always sets the state for the id */ - if (created !== undefined) created.owner = session - if (seed.length > 0) { - await this.append(id, seed) - } - } - - private async flush(session: Session): Promise { - // Wait for the session's init (onCreated) to finish so the state/cursor and - // any fork-seed persistence are in place before we drain. Awaiting the same - // promise initFor stored also surfaces an init failure (e.g. an id - // collision) here, where the caller of session/flush observes it. - await this.inits.get(session) - // Serialize the WHOLE drain (read cursor → append → splice) on the - // per-session chain. Two concurrent flushes (e.g. an idle inject()'s - // fire-and-forget flush racing an explicit checkpoint) would otherwise both - // read the same cursor, both compute the same `fresh` slice, and the second - // append would seq-mismatch against the cursor the first already advanced. - await this.serialize(session.header.id, () => this.drain(session)) - } - - /** Drain a session's write buffer to disk. Caller serializes this per id. */ - private async drain(session: Session): Promise { - const buffer = this.buffers.get(session) - if (!buffer?.length) return - // Copy WITHOUT removing: the buffer is the only durable-pending copy of - // these events (session/event does not re-emit). Splicing before the append - // means a failed append (disk error, or a seq mismatch after a dropped bad - // event) permanently loses a completed turn. Drain the buffer only AFTER - // the append commits; events pushed during the await sit past batch.length - // and survive the prefix splice, so a retry/dispose re-drains the rest. - const batch = buffer.slice() - const state = this.states.get(session.header.id) - // Only append events at or beyond the write cursor (a resumed session's - // seed is already on disk; the cursor was set to the loaded length). flush - // awaits the init above, which always sets state, so the `?? 0` fallback is - // a defensive guard that never fires in practice. - /* v8 ignore next -- state is always set by the awaited init before flush */ - const cursor = state?.cursor ?? 0 - const fresh = batch.filter(e => e.seq >= cursor) - // appendCore (NOT the serialized append) — drain already runs inside the - // per-session chain, so re-entering it via append() would deadlock. - if (fresh.length > 0) await this.appendCore(session.header.id, fresh) - buffer.splice(0, batch.length) - } } export default SessionPersistenceJsonl diff --git a/packages/session-persistence-jsonl/tests/jsonl.spec.ts b/packages/session-persistence-jsonl/tests/jsonl.spec.ts index 07cc470c6e..b48bb249d1 100644 --- a/packages/session-persistence-jsonl/tests/jsonl.spec.ts +++ b/packages/session-persistence-jsonl/tests/jsonl.spec.ts @@ -4,10 +4,11 @@ import { appendFile, mkdtemp, mkdir, rm, readFile, writeFile, readdir, stat } fr import { tmpdir } from 'node:os' import { join } from 'node:path' import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' -import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl' import { encodeSegment, logPath, scanLog, sessionDir } from '../src/format.ts' import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts' +import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts' let root: string const dirs: string[] = [] @@ -37,6 +38,29 @@ runPersistenceContract('jsonl', async () => { } }) +// Run the shared coordinator orchestration suite against the real JSONL backend. +// One temp root is the shared storage scope (two mounted instances over the same +// root = HMR/reload). `corruptTail` appends a partial, newline-less fragment to +// the session's .jsonl past the committed region — a never-committed torn tail +// that drives the coordinator's commitRepair-with-tornMarker branch over real +// file bytes. +runCoordinatorContract('jsonl', async (): Promise => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-jsonl-coord-')) + return { + mount: async (ctx) => { + const fiber = await ctx.plugin(SessionPersistenceJsonl, { root: dir }) + return fiber + }, + corruptTail: async (id, cwd) => { + // A half-written record with no trailing newline: scanLog treats it as an + // uncommitted crash fragment and reports committedBytes < byteLength, so + // the coordinator sees a tornMarker to truncate. + await appendFile(logPath(dir, cwd, id), '{"type":"assistant/chunk","seq":8,"ti') + }, + cleanup: async () => { await rm(dir, { recursive: true, force: true }) }, + } +}) + describe('SessionPersistenceJsonl: format helpers', () => { it('encodeSegment neutralizes traversal, separators, and absolute paths', () => { expect(encodeSegment('..')).toBe('~002E~002E') @@ -198,36 +222,6 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => { expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) }) - it('append snapshots its batch: mutating the caller array after the call is ignored', async () => { - const m = meta('snapshot') - await ctx.sessionPersistence.create(m) - const events = oneTurnLog() // seqs 0..5 - const p = ctx.sessionPersistence.append(m.id, events) - // Mutate the caller's array immediately after calling append (before the - // queued op runs). The backend must persist the snapshot taken at call time, - // not the mutated array. - events.push({ type: 'turn/start', seq: 6, time: 99, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }) - await p - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) // not 0..6 - }) - - it('append deep-snapshots event objects: mutating an event after the call is ignored', async () => { - const m = meta('deep-snapshot') - await ctx.sessionPersistence.create(m) - const events = oneTurnLog() - const userMsg = events[1] // the user/message event - const p = ctx.sessionPersistence.append(m.id, events) - // Mutate an event OBJECT (not just the array) after calling append. The deep - // snapshot taken at call time must shield the persisted data. - if (userMsg?.type === 'user/message') userMsg.data.content = [{ type: 'text', text: 'MUTATED' }] - await p - const loaded = await ctx.sessionPersistence.load(m.id) - const persisted = JSON.stringify(loaded.events) - expect(persisted).toContain('hi') // original content - expect(persisted).not.toContain('MUTATED') - }) - it('load returns a meta copy: mutating it does not corrupt backend pathing', async () => { const m = meta('meta-copy', '/proj') await ctx.sessionPersistence.create(m) @@ -246,25 +240,6 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => { expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) }) - it('rejects an unknown format version on load', async () => { - const m = meta('v2') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - // Corrupt the header version on disk. - const path = logPath(root, undefined, m.id) - const lines = (await readFile(path, 'utf8')).split('\n') - const header = JSON.parse(lines[0]!) as { version: number } - header.version = 2 - lines[0] = JSON.stringify(header) - await writeFile(path, lines.join('\n')) - // Fresh backend (no in-memory state) → must reject on load. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - await expect(ctx2.sessionPersistence.load(m.id)).rejects.toThrow(/version/) - await ctx2.fiber.dispose() - }) - it('rejects a re-append of an already-stored seq', async () => { const m = meta('reappend') await ctx.sessionPersistence.create(m) @@ -293,62 +268,6 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => { }) describe('SessionPersistenceJsonl: write path (session/event → flush)', () => { - it('persists a live session driven through the store, surviving reload', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - await ctx.plugin(SessionPersistenceJsonl, { root }) - - const session = ctx.sessions.create('live', { meta: { cwd: '/w' } }) - for (const e of oneTurnLog()) session.append(e.type, e.data) - await ctx.parallel('session/flush', session) - - const loaded = await ctx.sessionPersistence.load(SessionId('live')) - expect(loaded.events).toHaveLength(6) - expect(loaded.meta.cwd).toBe('/w') - await ctx.fiber.dispose() - }) - - it('snapshot-on-buffer: mutating an event after session/event does not corrupt the persisted copy', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - await ctx.plugin(SessionPersistenceJsonl, { root }) - - const session = ctx.sessions.create('mutate') - const ev = session.append('user/message', { content: [{ type: 'text', text: 'original' }], source: { kind: 'user' } }) - // Mutate the live event object AFTER it was buffered. - ;(ev.data as { content: { type: 'text'; text: string }[] }).content[0]!.text = 'HACKED' - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - - const loaded = await ctx.sessionPersistence.load(SessionId('mutate')) - const first = loaded.events[0] - expect(first?.type === 'user/message' && (first.data.content[0] as { text: string }).text).toBe('original') - await ctx.fiber.dispose() - }) - - it('fork: a seeded new session persists its seed once', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - await ctx.plugin(SessionPersistenceJsonl, { root }) - - const seed = oneTurnLog() - // A fork: a brand-new id whose seed came from elsewhere. - const forked = ctx.sessions.create('forked', { seed }) - // onCreated persisted the seed asynchronously; wait a tick. - await new Promise(r => setTimeout(r, 10)) - const loaded = await ctx.sessionPersistence.load(SessionId('forked')) - expect(loaded.events).toEqual(seed) - // A flush with no NEW events must not double-write. - await ctx.parallel('session/flush', forked) - const reloaded = await ctx.sessionPersistence.load(SessionId('forked')) - expect(reloaded.events).toEqual(seed) - await ctx.fiber.dispose() - }) - it('concurrent sessions do not cross buffers', async () => { root = await freshRoot() const ctx = new Context() @@ -373,112 +292,6 @@ describe('SessionPersistenceJsonl: write path (session/event → flush)', () => await ctx.fiber.dispose() }) - it('HMR: applying the plugin seeds existing live sessions', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - // A session exists BEFORE the persistence plugin is applied. - const session = ctx.sessions.create('pre-existing') - session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - - await ctx.plugin(SessionPersistenceJsonl, { root }) - // The plugin seeded it on apply; a subsequent flush persists its events. - await ctx.parallel('session/flush', session) - const loaded = await ctx.sessionPersistence.load(SessionId('pre-existing')) - expect(loaded.events.length).toBeGreaterThanOrEqual(2) - await ctx.fiber.dispose() - }) - - it('HMR: dispose drains remaining buffers', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - let session!: Session - const fiber = await ctx.plugin(SessionPersistenceJsonl, { root }) - const sessFiber = await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create('drain') - }, { inject: ['sessions'] })) - session.append('user/message', { content: [{ type: 'text', text: 'buffered' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - // No explicit flush — dispose must drain. - await fiber.dispose() - await sessFiber.dispose() - - // A fresh backend reads what the disposed one drained. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - const loaded = await ctx2.sessionPersistence.load(SessionId('drain')) - expect(loaded.events.length).toBeGreaterThanOrEqual(2) - await ctx2.fiber.dispose() - }) - - it('HMR: reloading the backend adopts a still-live, already-materialized session', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - // The session lives in its OWN fiber so it survives the backend reload. - let session!: Session - await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create('hmr-adopt') - }, { inject: ['sessions'] })) - - // Backend instance 1 materializes the session on disk. - const backend1 = await ctx.plugin(SessionPersistenceJsonl, { root }) - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - - // Hot-reload the backend: dispose instance 1, plug in instance 2 over the - // SAME root while the session stays live. Instance 2 has an empty states - // map but the log is on disk — it must ADOPT (not reject) so flush keeps - // working. A second turn appended after reload then persists. - await backend1.dispose() - await ctx.plugin(SessionPersistenceJsonl, { root }) - session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('user/message', { content: [{ type: 'text', text: 'again' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) - await expect(ctx.parallel('session/flush', session)).resolves.not.toThrow() - - const loaded = await ctx.sessionPersistence.load(SessionId('hmr-adopt')) - expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2) - await ctx.fiber.dispose() - }) - - it('HMR: adoption persists the live SUFFIX that was ahead of the on-disk prefix', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - let session!: Session - await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create('hmr-suffix') - }, { inject: ['sessions'] })) - - // Instance 1 flushes turn 1 to disk. - const backend1 = await ctx.plugin(SessionPersistenceJsonl, { root }) - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - - // Append turn 2 to the LIVE session, then dispose instance 1 WITHOUT - // flushing turn 2. Turn 2 is now ONLY in the live session's events; the new - // backend never buffered it via session/event. - await backend1.dispose() - session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) - - // Instance 2 adopts the on-disk prefix (turn 1) and MUST also persist the - // live suffix (turn 2) carried in the session's events — otherwise turn 2 is - // lost and a later flush would mismatch. - await ctx.plugin(SessionPersistenceJsonl, { root }) - await ctx.parallel('session/flush', session) - const loaded = await ctx.sessionPersistence.load(SessionId('hmr-suffix')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3]) - expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2) - await ctx.fiber.dispose() - }) }) @@ -570,17 +383,6 @@ describe('SessionPersistenceJsonl: edge cases', () => { }) afterEach(async () => { await ctx.fiber.dispose() }) - it('load rejects a missing session', async () => { - await expect(ctx.sessionPersistence.load(SessionId('nope'))).rejects.toThrow(/not found/) - }) - - it('append of an empty batch is a no-op', async () => { - const m = meta('empty-batch') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, []) - expect(await ctx.sessionPersistence.has(m.id)).toBe(false) - }) - it('append rejects non-JSON-serializable undefined-producing data', async () => { const m = meta('undef') await ctx.sessionPersistence.create(m) @@ -589,60 +391,6 @@ describe('SessionPersistenceJsonl: edge cases', () => { await expect(ctx.sessionPersistence.append(m.id, bad)).rejects.toThrow(/non-JSON-serializable/) }) - it('delete of a non-existent session is a no-op', async () => { - await expect(ctx.sessionPersistence.delete(SessionId('ghost'))).resolves.toBeUndefined() - }) - - it('an abandoned lazy session (never materialized) releases its id for reuse', async () => { - // A live session is created then disposed BEFORE its first append: cursor 0, - // never materialized, nothing on disk. A new live session reusing the id - // must reclaim it (lazy materialization promises no lingering artifact), - // not wedge on an "already bound" collision until restart. - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - let firstSession!: Session - const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { - firstSession = inner.sessions.create('abandoned', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await backend.inits.get(firstSession) // let the lazy create register the state - await firstFiber.dispose() // disposed before any append → never materialized - - let reuse!: Session - await ctx.plugin(Object.assign((inner: Context) => { - reuse = inner.sessions.create('abandoned', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - // The new session claims the id without error and can persist a turn. - await expect(backend.inits.get(reuse)).resolves.toBeUndefined() - reuse.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - reuse.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', reuse) - const loaded = await ctx.sessionPersistence.load(SessionId('abandoned')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1]) - }) - - it('does NOT reclaim an id whose abandoned owner still has buffered (unflushed) events', async () => { - // A session that appended events but was disposed BEFORE its first flush is - // not materialized yet but still holds a write-behind buffer. Reusing the id - // must be rejected (not reclaimed), or the stale buffer would drain against - // the new session — persisting old events under the new id or dropping the - // new session's seq-0 events. - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - let first!: Session - const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { - first = inner.sessions.create('buffered', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await backend.inits.get(first) - // Append a turn but do NOT flush — events sit in the write-behind buffer. - first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - first.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await firstFiber.dispose() // disposed before flush; not materialized, buffer pending - - let reuse!: Session - await ctx.plugin(Object.assign((inner: Context) => { - reuse = inner.sessions.create('buffered', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await expect(backend.inits.get(reuse)).rejects.toThrow(/already bound to a different live session/) - }) - it('create snapshots its meta: mutating the caller object after the call is ignored', async () => { const m = meta('create-snap', '/orig') const p = ctx.sessionPersistence.create(m) @@ -713,81 +461,6 @@ describe('SessionPersistenceJsonl: edge cases', () => { await ctx2.fiber.dispose() }) - it('resume/adopt: a live session whose id is already on disk continues from the stored length', async () => { - // First lifecycle: persist a session through the store. - const s1 = ctx.sessions.create('resumed', { meta: { cwd: '/r' } }) - for (const e of oneTurnLog()) s1.append(e.type, e.data) - await ctx.parallel('session/flush', s1) - - // Second lifecycle: a NEW backend + a session re-created with the same id - // and SEEDED with the loaded events (the resume path). onCreated must adopt - // the on-disk log (not re-persist the seed), and a new turn appends at seq 6. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - const loaded = await ctx2.sessionPersistence.load(SessionId('resumed')) - const s2 = ctx2.sessions.create('resumed', { seed: loaded.events, meta: { cwd: '/r' } }) - await new Promise(r => setTimeout(r, 10)) // let onCreated adopt - // Append a fresh turn through the live session. - s2.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) - s2.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) - await ctx2.parallel('session/flush', s2) - - const reloaded = await ctx2.sessionPersistence.load(SessionId('resumed')) - // 6 original + 2 new, contiguous, no duplicated seed. - expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await ctx2.fiber.dispose() - }) - - it('HMR adoption does not crash-repair an active open turn as interrupted', async () => { - const dir = await freshRoot() - const hmr = new Context() - await hmr.plugin(SessionStore) - const first = await hmr.plugin(SessionPersistenceJsonl, { root: dir }) - const session = hmr.sessions.create('hmr-open', { meta: { cwd: '/hmr' } }) - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('step/start', { turn: 1, step: 1 }) - await hmr.parallel('session/flush', session) - - await first.dispose() - await appendFile(logPath(dir, '/hmr', SessionId('hmr-open')), '{"torn":') - const second = await hmr.plugin(SessionPersistenceJsonl, { root: dir }) - session.append('step/end', { turn: 1, step: 1 }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await hmr.parallel('session/flush', session) - - const loaded = await hmr.sessionPersistence.load(SessionId('hmr-open')) - expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'step/start', 'step/end', 'turn/end']) - expect(loaded.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } }) - await second.dispose() - await hmr.fiber.dispose() - }) - - it('a NEW live session whose id collides with an on-disk log is rejected, not silently adopted', async () => { - // Persist a session on disk. - const s1 = ctx.sessions.create('collide', { meta: { cwd: '/a' } }) - for (const e of oneTurnLog()) s1.append(e.type, e.data) - await ctx.parallel('session/flush', s1) - const before = await readFile(logPath(root, '/a', SessionId('collide')), 'utf8') - - // A FRESH backend + a NEW live session with the same id but NO explicit - // load/resume. onCreated must NOT adopt-from-disk (resume is explicit); it - // treats this as a new session and create() rejects because a log already - // exists on disk. The rejection surfaces via the init promise (flush awaits - // it); the on-disk committed log is left byte-for-byte intact. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - const backend = ctx2.sessionPersistence as unknown as { inits: Map> } - const s2 = ctx2.sessions.create('collide', { meta: { cwd: '/a' } }) - // The init for the new live session rejects (observed via the per-session - // init map and, in production, via flush which awaits the same promise). - await expect(backend.inits.get(s2)).rejects.toThrow(/already has a persisted log on disk/) - // The committed log is untouched (no clobber). - expect(await readFile(logPath(root, '/a', SessionId('collide')), 'utf8')).toBe(before) - await ctx2.fiber.dispose() - }) - it('a DIFFERENT live session object reusing a disposed id gets its own init (no stale cache)', async () => { // Session A materializes a log under id "reuse". const sessFiberA = await ctx.plugin(Object.assign((inner: Context) => { @@ -811,98 +484,6 @@ describe('SessionPersistenceJsonl: edge cases', () => { await expect(backend.inits.get(b)).rejects.toThrow(/already bound to a different live session|already has a persisted log on disk/) }) - it('a live session claims cursor-0 ownerless state created via the public API', async () => { - // create() registers ownerless state with cursor 0 (lazy, nothing persisted - // yet). A live session with that id then arrives and claims it without a - // prefix check (cursor 0 matches trivially), persisting its seed. - await ctx.sessionPersistence.create(meta('lazy-claim', '/a')) - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - let live!: Session - await ctx.plugin(Object.assign((inner: Context) => { - live = inner.sessions.create('lazy-claim', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await expect(backend.inits.get(live)).resolves.toBeUndefined() - live.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - live.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', live) - const loaded = await ctx.sessionPersistence.load(SessionId('lazy-claim')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1]) - }) - - it('a fresh session reusing a previously-loaded id is rejected (ownerless guard)', async () => { - // Materialize a log, then load() it into the backend's state WITHOUT a live - // session — leaving state.owner undefined and cursor at the persisted length - // (the public preview path). - await ctx.sessionPersistence.create(meta('preview', '/a')) - await ctx.sessionPersistence.append(SessionId('preview'), oneTurnLog()) - await ctx.sessionPersistence.load(SessionId('preview')) - - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - // A FRESH (empty-seed) live session reusing that id must be rejected: its - // seq 0..cursor-1 events would otherwise be filtered as already-persisted - // and its conversation grafted onto the old log. - let fresh!: Session - await ctx.plugin(Object.assign((inner: Context) => { - fresh = inner.sessions.create('preview', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await expect(backend.inits.get(fresh)).rejects.toThrow(/do not match this live session|already has a persisted log/) - }) - - it('a session whose seed matches the loaded prefix claims ownerless state', async () => { - // Materialize a log and load it (ownerless state, cursor = 6). - await ctx.sessionPersistence.create(meta('match', '/a')) - await ctx.sessionPersistence.append(SessionId('match'), oneTurnLog()) - await ctx.sessionPersistence.load(SessionId('match')) - - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - // A live session SEEDED with the persisted log legitimately continues it — - // its seed reproduces the loaded prefix, so it claims the ownerless state. - let cont!: Session - await ctx.plugin(Object.assign((inner: Context) => { - cont = inner.sessions.create('match', { seed: oneTurnLog(), meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await expect(backend.inits.get(cont)).resolves.toBeUndefined() - }) - - it('claiming ownerless state persists the seed suffix beyond the prefix', async () => { - // Materialize a one-turn log and load it (ownerless state, cursor = 6). - await ctx.sessionPersistence.create(meta('suffix-claim', '/a')) - await ctx.sessionPersistence.append(SessionId('suffix-claim'), oneTurnLog()) - await ctx.sessionPersistence.load(SessionId('suffix-claim')) - - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - // A live session seeded with the prefix PLUS a second turn (seqs 6,7). The - // suffix (constructor seed, never emits session/event) must be persisted on - // claim, not lost. - const seed = [ - ...oneTurnLog(), - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, - ] as SessionEvent[] - let cont!: Session - await ctx.plugin(Object.assign((inner: Context) => { - cont = inner.sessions.create('suffix-claim', { seed, meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await backend.inits.get(cont) - const loaded = await ctx.sessionPersistence.load(SessionId('suffix-claim')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - }) - - it('claiming cursor-0 ownerless state persists the whole constructor seed', async () => { - // create() registers ownerless state with cursor 0 (lazy, nothing on disk). - await ctx.sessionPersistence.create(meta('lazy-seed', '/a')) - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - // A live session seeded with a full turn claims it; the whole seed (cursor - // is 0) must be persisted. - let cont!: Session - await ctx.plugin(Object.assign((inner: Context) => { - cont = inner.sessions.create('lazy-seed', { seed: oneTurnLog(), meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await backend.inits.get(cont) - const loaded = await ctx.sessionPersistence.load(SessionId('lazy-seed')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) - }) - it('a seed with matching seq/type/time but DIFFERENT data is rejected (deep prefix compare)', async () => { // Materialize and load (ownerless, cursor = 6). await ctx.sessionPersistence.create(meta('divergent', '/a')) @@ -942,14 +523,6 @@ describe('SessionPersistenceJsonl: edge cases', () => { .rejects.toThrow(/already bound to a different live session|already has a persisted log|do not match/) }) - it('round-trips a header with parentSession (fork lineage)', async () => { - const m: SessionHeader = { version: 1, id: SessionId('forked-child'), createdAt: 1, parentSession: SessionId('the-parent') } - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.meta.parentSession).toBe('the-parent') - }) - it('list returns nothing when the root directory does not exist', async () => { const ctx2 = new Context() await ctx2.plugin(SessionStore) @@ -1025,48 +598,6 @@ describe('SessionPersistenceJsonl: edge cases', () => { expect(end.type === 'turn/end' && end.data.reason).toEqual({ kind: 'interrupted' }) }) - it('initFor is idempotent: a re-seeded existing session is not re-initialized', async () => { - const session = ctx.sessions.create('idem', { meta: { cwd: '/i' } }) - session.append('user/message', { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - // Re-emit session/created for the SAME live session (idempotent initFor). - ctx.emit('session/created', session) - await ctx.parallel('session/flush', session) - const loaded = await ctx.sessionPersistence.load(SessionId('idem')) - expect(loaded.events).toHaveLength(2) // not doubled - }) - - - it('flush before init resolves with no state uses cursor 0', async () => { - // Drive a fork (seed) flush where the buffer holds the seed; the fresh - // events filter against cursor. Exercises the state-undefined cursor path. - const session = ctx.sessions.create('flush-nostate') - // Append directly to the live session and flush IMMEDIATELY, before the - // async onCreated init has necessarily set state. - session.append('user/message', { content: [{ type: 'text', text: 'q' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - const loaded = await ctx.sessionPersistence.load(SessionId('flush-nostate')) - expect(loaded.events).toHaveLength(2) - }) - - it('createCore rejects creating an id this backend already tracks', async () => { - await ctx.sessionPersistence.create(meta('dup')) - await expect(ctx.sessionPersistence.create(meta('dup'))).rejects.toThrow(/already exists in this backend/) - }) - - it('createCore rejects creating an id whose log already exists on disk', async () => { - const m = meta('on-disk', '/od') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - // A fresh backend (no in-memory state) must refuse to create over the log. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - await expect(ctx2.sessionPersistence.create(meta('on-disk', '/od'))).rejects.toThrow(/already has a persisted log on disk/) - await ctx2.fiber.dispose() - }) it('createCore rejects an id already on disk under a DIFFERENT cwd bucket', async () => { // Persist the id under cwd A. diff --git a/packages/session-persistence-sqlite/src/index.ts b/packages/session-persistence-sqlite/src/index.ts index 34ed6213da..49cf3882d4 100644 --- a/packages/session-persistence-sqlite/src/index.ts +++ b/packages/session-persistence-sqlite/src/index.ts @@ -1,19 +1,18 @@ /** * SQLite durable session-persistence backend (`@deepseek-ai/dsh-session-persistence-sqlite`). * - * A SECOND {@link SessionPersistence} implementation, built to validate that - * the abstract seam + the shared `runPersistenceContract` suite are genuinely + * A SECOND {@link SessionPersistence} implementation, built to validate that the + * abstract seam + the shared `runPersistenceContract` suite are genuinely * backend-agnostic: the same append-only / contiguous-seq / lazy-materialization * / interrupted-turn-close-on-load semantics the JSONL backend expresses over * file bytes, expressed here over `node:sqlite` rows. Each `SessionEvent` maps - * 1:1 onto a row `(session_id, seq, type, time, data)`; `append` is an INSERT - * inside a transaction that asserts the contiguous-seq contract. + * 1:1 onto a row `(session_id, seq, type, time, data)`. * - * Like the JSONL backend it is also the write-path plugin: it installs the - * `session/event` → buffer → `session/flush` drain, persists a fork's seed once - * on `session/created`, keeps a per-session write cursor so a resumed session - * never re-appends stored events, and seeds existing live sessions on apply - * (HMR does not replay `session/created`). + * Like the JSONL backend it supplies ONLY the storage primitives (the + * {@link PersistenceBackend} hooks below — INSERT/DELETE/SELECT inside + * transactions); all the write-path orchestration lives in the backend-agnostic + * {@link PersistenceCoordinator} this class composes. The six public + * {@link SessionPersistence} methods delegate to the coordinator. * * @module @deepseek-ai/dsh-session-persistence-sqlite */ @@ -24,9 +23,9 @@ import { DatabaseSync } from 'node:sqlite' import { mkdir } from 'node:fs/promises' import { dirname, resolve } from 'node:path' import { - SessionPersistence, assertSerializable, seedCoversPrefix, + SessionPersistence, PersistenceCoordinator, + type PersistenceBackend, type StoredPrefix, } from '@deepseek-ai/dsh-session-persistence' -import { interruptedTurnClosers } from '@deepseek-ai/dsh-session' import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' import { openDatabase, rowToMeta, scanRows, type EventRow, type SessionRow, @@ -44,55 +43,36 @@ export interface Config { path: string } -/** Backend bookkeeping for a session id (NOT the live Session object). */ -interface SessionState { - meta: SessionHeader - /** Next seq to write — equals the number of committed events. */ - cursor: number - /** Whether the session has at least one persisted event (materialized). */ - materialized: boolean - /** The live Session that owns this state (collision detection); see onCreated. */ - owner?: Session -} - -async function settledErrors(promises: Iterable>): Promise { - const settled = await Promise.allSettled([...promises]) - const errors: unknown[] = [] - for (const result of settled) { - if (result.status === 'rejected') errors.push(result.reason) - } - return errors -} - /** * The SQLite persistence backend. Load as a plugin; it registers as - * `ctx.sessionPersistence` and installs the write-path listeners. + * `ctx.sessionPersistence` and (via the coordinator) installs the write-path + * listeners. Its torn-tail marker is the seq to delete from. */ -export class SessionPersistenceSqlite extends SessionPersistence { +export class SessionPersistenceSqlite extends SessionPersistence implements PersistenceBackend { static inject = ['sessions'] static Config: z = z.object({ path: z.string().required(), }) + /** + * Backend label for the coordinator's dispose diagnostics. Intentionally + * shadows cordis `Service.name` (set to `'sessionPersistence'` by the base); + * see the JSONL backend for why this does not affect service resolution. + */ + override readonly name = 'session-persistence-sqlite' + private db!: DatabaseSync private ready: Promise - /** Backend bookkeeping keyed by session id (NOT the live Session object). */ - private states = new Map() - /** Write-behind buffers keyed by the live Session (write path). */ - private buffers = new Map() - /** Per-session serialization chain (keyed by session id). */ - private chains = new Map>() - /** Per-session init promise (onCreated), keyed by the LIVE Session object. */ - private inits = new Map>() + private coordinator: PersistenceCoordinator constructor(ctx: Context, public config: Config) { super(ctx) // Open the database asynchronously (the parent directory may need creating); - // every backend op awaits `ready` first. Opening synchronously in the ctor - // would force a sync mkdir and block plugin apply. + // every hook awaits `ready` first. Opening synchronously would force a sync + // mkdir and block plugin apply. this.ready = this.openDb(config.path) - this.installWritePath() + this.coordinator = new PersistenceCoordinator(this.ctx, this) } private async openDb(path: string): Promise { @@ -105,229 +85,156 @@ export class SessionPersistenceSqlite extends SessionPersistence { } } - // --- SessionPersistence backend surface (all serialized per session id) --- + // --- SessionPersistence service surface (delegated to the coordinator) --- create(meta: SessionHeader): Promise { - const snapshot: SessionHeader = { ...meta } - return this.serialize(snapshot.id, () => this.createCore(snapshot)) + return this.coordinator.create(meta) } - private async createCore(meta: SessionHeader): Promise { + append(id: SessionId, events: readonly SessionEvent[]): Promise { + return this.coordinator.append(id, events) + } + + load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { + return this.coordinator.load(id) + } + + has(id: SessionId): Promise { + return this.coordinator.has(id) + } + + delete(id: SessionId): Promise { + return this.coordinator.delete(id) + } + + // `list` is BOTH the public service method and the PersistenceBackend hook — + // one method (the SELECT below). The coordinator adds no orchestration for + // listing, so routing it through the coordinator would just recurse. Defined + // once, in the "PersistenceBackend hooks" section. + + /** + * The per-session init promises, exposed for white-box tests that await a + * specific session's onCreated (there is no public API to await one init). + */ + get inits(): Map> { + return this.coordinator.inits + } + + // --- PersistenceBackend hooks (the SQLite storage primitives) --- + + /** Read a stored prefix by id (ids are globally unique — no scope to scan). */ + loadStored(id: SessionId): Promise | undefined> { + return this.readPrefix(id) + } + + /** Read a stored prefix; `cwd` is ignored (the id is globally unique in SQLite). */ + loadLive(id: SessionId, _cwd: string | undefined): Promise | undefined> { + return this.readPrefix(id) + } + + /** + * Read a session's row + ordered events into a {@link StoredPrefix}. The + * torn-tail marker is the seq from which a never-committed tail must be deleted + * (`scanRows` already returns it as `number | undefined`). + */ + private async readPrefix(id: SessionId): Promise | undefined> { await this.ready - if (this.states.has(meta.id)) { - throw new Error(`session "${meta.id}" already exists in this backend`) - } - if (this.rowFor(meta.id) !== undefined) { - throw new Error(`session "${meta.id}" already has a persisted row; load/resume it instead of creating`) - } - // Lazy: record intent in memory only. No row until the first append, so an - // abandoned (never-appended) session leaves nothing behind and stays absent - // from has()/list(). - this.states.set(meta.id, { meta, cursor: 0, materialized: false }) + const row = this.rowFor(id) + if (row === undefined) return undefined + const meta = rowToMeta(row) + const eventRows = this.db + .prepare('SELECT seq, type, time, data FROM events WHERE session_id = ? ORDER BY seq') + .all(id) as unknown as EventRow[] + const { preserved, tornFrom } = scanRows(eventRows) + return { meta, events: preserved, ...tornFrom !== undefined ? { tornMarker: tornFrom } : {} } } - // `async` so the synchronous validate/clone below reject (not throw) per the - // Promise contract — callers use `await expect(...).rejects`. - async append(id: SessionId, events: readonly SessionEvent[]): Promise { - // Validate serializability BEFORE cloning so a bad event surfaces the typed - // "non-JSON-serializable" error rather than an opaque DataCloneError from - // structuredClone. Then deep-snapshot the batch HERE, before the op waits - // behind the per-session chain: a caller that passes a live array (e.g. - // session.events) and mutates it — OR mutates an event inside it — before - // the op runs would otherwise have those changes persisted, or advance the - // cursor past what was written. The clone is taken at call time (before the - // first await), matching the JSONL backend. - assertSerializable(events) - const batch = events.map(e => structuredClone(e)) - return this.serialize(id, () => this.appendCore(id, batch)) - } - - private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise { + /** + * Durably append a batch in ONE transaction: materialize the sessions row (if + * lazy) and INSERT every event, or roll back entirely. The transaction is the + * atomicity + durability boundary, so a mid-batch failure (a UNIQUE violation + * on a duplicated seq) leaves the stored log untouched. + */ + async appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise { await this.ready - if (events.length === 0) return - let state = this.states.get(id) - if (state === undefined) state = await this.adopt(id) - - // Contiguity contract: each event's seq must continue the stored log. - for (const [i, event] of events.entries()) { - if (event.seq !== state.cursor + i) { - throw new Error(`append seq mismatch for "${id}": expected ${state.cursor + i} at index ${i}, got ${event.seq}`) - } - } - - // The transaction is the durability + atomicity boundary: materialize the - // sessions row (if lazy) and INSERT every event, or roll back entirely. A - // BEGIN/COMMIT around the batch means a mid-batch failure (a UNIQUE - // violation on a duplicated seq from a concurrent writer) leaves the stored - // log untouched, so the cursor stays truthful and a retry is clean. (A crash - // tail is already gone: load() physically deletes the torn fragment and - // durably closes the interrupted turn before returning, so by the time any - // append runs the stored log is balanced and contiguous.) const insertEvent = this.db.prepare( 'INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)', ) this.db.exec('BEGIN') try { - if (!state.materialized) this.writeRow(state.meta) + if (!isMaterialized) this.writeRow(meta) for (const event of events) { - insertEvent.run(id, event.seq, event.type, event.time, JSON.stringify(event.data)) + insertEvent.run(meta.id, event.seq, event.type, event.time, JSON.stringify(event.data)) } this.db.exec('COMMIT') } catch (error) { this.db.exec('ROLLBACK') throw error } - state.materialized = true - state.cursor += events.length } - load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { - return this.serialize(id, () => this.loadCore(id)) - } - - private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { + /** + * Make a crash repair durable in ONE transaction: DELETE the torn tail (from + * `tornMarker`) and INSERT the synthetic `closers`. After COMMIT the stored rows + * == the balanced log. + */ + async commitRepair(meta: SessionHeader, tornMarker: number | undefined, closers: readonly SessionEvent[]): Promise { await this.ready - const row = this.rowFor(id) - if (row === undefined) throw new Error(`session "${id}" not found`) - const meta = rowToMeta(row) - this.assertVersion(meta) - - // Read every stored row ordered by seq, then scan for the preserved prefix: - // the longest seq-contiguous, parseable run, INCLUDING the real events of an - // interrupted final turn after the last turn/end (a turn can be huge — they - // are never truncated). scanRows works off the seq+type COLUMNS for the - // last-turn/end boundary, so a malformed `data` in a torn tail row is - // discarded (not unloadable); only a parse error / seq gap in the COMMITTED - // region (at or before the last turn/end) throws (genuine corruption). - const eventRows = this.db - .prepare('SELECT seq, type, time, data FROM events WHERE session_id = ? ORDER BY seq') - .all(id) as unknown as EventRow[] - const { preserved, tornFrom } = scanRows(eventRows) - - // Crash-recovery (mutating load, same as the JSONL backend): if the log ended - // mid-turn, close it DURING load so disk, the returned log, and the cursor all - // agree — both append routes then continue with no special-casing. Synthesize - // the boundary events (a step/end if a step was open, then a - // turn/end {kind:'interrupted'}); the interrupted turn's real events are - // preserved, never truncated (the session-persistence RFC). - const closers = interruptedTurnClosers(preserved) - const balanced = [...preserved, ...closers] - - // Physically repair the stored log inside one transaction: DELETE the torn - // tail fragment (if any), then INSERT the synthetic closers. After COMMIT the - // stored rows == balanced, so the cursor is truthful and the next append - // continues cleanly with no deferred repair. The metadata row stays as-is - // even when preserved.length === 0 (an all-tail crash): the session WAS - // materialized by the partial append, so has()/list() still report it — the - // same as the JSONL backend, whose file likewise survives a first append that - // never reached turn/end. - if (tornFrom !== undefined || closers.length > 0) { - this.db.exec('BEGIN') - try { - if (tornFrom !== undefined) { - this.db.prepare('DELETE FROM events WHERE session_id = ? AND seq >= ?').run(id, tornFrom) - } - if (closers.length > 0) { - const insertEvent = this.db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)') - for (const event of closers) { - insertEvent.run(id, event.seq, event.type, event.time, JSON.stringify(event.data)) - } - } - this.db.exec('COMMIT') - } catch (error) { - // The DELETE+INSERT cannot collide (a row at a closer's seq is preserved - // or deleted as torn first); this rolls back a DB-level failure (disk - // full, etc.), unreachable in test. - /* v8 ignore start */ - this.db.exec('ROLLBACK') - throw error - /* v8 ignore stop */ + this.db.exec('BEGIN') + try { + if (tornMarker !== undefined) { + this.db.prepare('DELETE FROM events WHERE session_id = ? AND seq >= ?').run(meta.id, tornMarker) } + if (closers.length > 0) { + const insertEvent = this.db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)') + for (const event of closers) { + insertEvent.run(meta.id, event.seq, event.type, event.time, JSON.stringify(event.data)) + } + } + this.db.exec('COMMIT') + } catch (error) { + // The DELETE+INSERT cannot collide (a row at a closer's seq is preserved or + // deleted as torn first); this rolls back a DB-level failure (disk full, + // etc.), unreachable in test. + /* v8 ignore start */ + this.db.exec('ROLLBACK') + throw error + /* v8 ignore stop */ } - - // Record state at the balanced length. The state keeps its OWN copy of the - // meta; the returned value is separate so a consumer mutating loaded.meta - // cannot corrupt the backend's row metadata. - this.states.set(id, { - meta: { ...meta }, - cursor: balanced.length, - materialized: true, - }) - return { meta, events: balanced } } - private async adoptLiveStoredPrefix(session: Session, seed: readonly SessionEvent[]): Promise { + /** Remove a session's row (ON DELETE CASCADE drops its events). */ + async deleteStored(id: SessionId): Promise { await this.ready - const row = this.rowFor(session.header.id) - /* v8 ignore next -- caller checked row existence */ - if (row === undefined) throw new Error(`session "${session.header.id}" not found`) - const meta = rowToMeta(row) - this.assertVersion(meta) - - const rows = this.db - .prepare('SELECT seq, type, time, data FROM events WHERE session_id = ? ORDER BY seq') - .all(session.header.id) as unknown as EventRow[] - const { preserved, tornFrom } = scanRows(rows) - if (!seedCoversPrefix(seed, preserved)) { - throw new Error(`session "${session.header.id}" already has a persisted log that does not match this live session (id collision)`) - } - - if (tornFrom !== undefined) { - this.db.prepare('DELETE FROM events WHERE session_id = ? AND seq >= ?').run(session.header.id, tornFrom) - } - this.states.set(session.header.id, { - meta: { ...meta }, - cursor: preserved.length, - materialized: true, - owner: session, - }) - const suffix = seed.slice(preserved.length) - if (suffix.length > 0) await this.appendCore(session.header.id, suffix) + this.db.prepare('DELETE FROM sessions WHERE id = ?').run(id) } + /** List all materialized sessions' metadata (every row is a materialized session). */ async list(): Promise { await this.ready - // Every metadata row is a materialized session: the row is written only by - // the first append (a created-but-never-appended session has no row), so - // listing all rows is exactly the materialized set. const rows = this.db .prepare('SELECT * FROM sessions') .all() as unknown as SessionRow[] return rows.map(rowToMeta) } - async has(id: SessionId): Promise { + /** Close the database handle (awaited by the coordinator's dispose, post-drain). */ + async close(): Promise { await this.ready - const state = this.states.get(id) - if (state?.materialized) return true - // A metadata row exists iff the session was materialized by a first append. - return this.rowFor(id) !== undefined - } - - delete(id: SessionId): Promise { - return this.serialize(id, () => this.deleteCore(id)) - } - - private async deleteCore(id: SessionId): Promise { - await this.ready - // ON DELETE CASCADE drops the session's events with its row. - this.db.prepare('DELETE FROM sessions WHERE id = ?').run(id) - this.states.delete(id) + this.db.close() } // --- row helpers --- /** Fetch a session's row, or undefined if absent. */ private rowFor(id: SessionId): SessionRow | undefined { - const row = this.db.prepare('SELECT * FROM sessions WHERE id = ?').get(id) as unknown as SessionRow | undefined - return row + return this.db.prepare('SELECT * FROM sessions WHERE id = ?').get(id) as unknown as SessionRow | undefined } /** * Insert-or-replace a session's metadata row. The only caller is the first - * materializing `append`, so writing the row IS the materialization (its - * existence is the signal `has`/`list` read); a never-appended session has no - * row at all. + * materializing `appendBatch`, so writing the row IS the materialization (its + * existence is the signal `has`/`list` read). */ private writeRow(meta: SessionHeader): void { this.db.prepare(` @@ -346,190 +253,6 @@ export class SessionPersistenceSqlite extends SessionPersistence { meta.parentSession ?? null, ) } - - /** Build a state for a session present in the DB but not yet in memory. */ - private async adopt(id: SessionId): Promise { - await this.loadCore(id) // sets the state; load (serialized) would deadlock - const state = this.states.get(id) - /* v8 ignore next -- loadCore always sets the state for the id */ - if (!state) throw new Error(`failed to adopt session "${id}"`) - return state - } - - private assertVersion(meta: SessionHeader): void { - if (meta.version !== 1) { - throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`) - } - } - - /** - * Run `op` after any in-flight operation for the same session id, so writes - * for one session never interleave. Errors do not poison the chain. NOTE: - * serialized public methods must NOT call each other (deadlock); they call - * the unserialized `*Core` helpers instead. - */ - private serialize(id: SessionId, op: () => Promise): Promise { - const prior = this.chains.get(id) ?? Promise.resolve() - const next = prior.then(op, op) - this.chains.set(id, next.then(() => undefined, () => undefined)) - return next - } - - // --- write path (session/event → flush drain) --- - - private installWritePath(): void { - const ctx = this.ctx - - ctx.on('session/created', (session) => { void this.initFor(session) }) - - // Snapshot + buffer every event (the live object is mutable; clone so a - // later in-place mutation cannot rewrite a buffered event). Serializability - // is guaranteed at the source (Session.append), so structuredClone is safe. - ctx.on('session/event', (session, event) => { - let buffer = this.buffers.get(session) - if (!buffer) this.buffers.set(session, buffer = []) - buffer.push(structuredClone(event)) - }) - - ctx.on('session/flush', session => this.flush(session)) - - // Dispose must reach quiescence: await every init + final drain, then close - // the database, BEFORE returning, so no write lands after teardown. - ctx.effect(() => async () => { - let disposeError: unknown - try { - const errors = [ - ...await settledErrors(this.inits.values()), - ...await settledErrors([...this.buffers.keys()].map(s => this.flush(s))), - ...await settledErrors(this.chains.values()), - ] - if (errors.length > 0) { - throw new AggregateError(errors, 'session-persistence-sqlite dispose failed') - } - } catch (error: unknown) { - disposeError = error - throw error - } finally { - try { - await this.ready - this.db.close() - } catch (error: unknown) { - /* v8 ignore next -- open/close failure racing disposal is a defensive teardown edge */ - if (disposeError === undefined) throw error - // Opening/closing the database can only add teardown context here; keep - // the already-captured init/flush/chain AggregateError as the primary - // disposal failure instead of masking it from callers. - } - } - }, 'session-persistence-sqlite write path') - - // HMR: a hot reload does not replay session/created, so seed existing live - // sessions (mirrors dsh-invariants and the JSONL backend). - for (const session of ctx.sessions.list()) void this.initFor(session) - } - - /** Start (once) the async init for a session and remember its promise. */ - private initFor(session: Session): Promise { - const existing = this.inits.get(session) - if (existing) return existing - const seed = session.events.map(e => structuredClone(e)) - const p = this.onCreated(session, seed) - p.catch(() => { /* observed by flush/dispose via the stored promise */ }) - this.inits.set(session, p) - return p - } - - /** - * On session/created: sync the backend's state to a live Session. Cases - * mirror the JSONL backend: - * 1. Already tracked → no-op (or claim ownerless state if the seed matches). - * 2. A row EXISTS and is a seq-aligned PREFIX of the live events → adopt - * (HMR/resume), persisting any live suffix beyond the stored prefix. - * 3. A row EXISTS but is NOT a prefix → reject (id collision). - * 4. No row → a genuinely new session: register meta (lazy) + persist seed. - */ - private async onCreated(session: Session, seed: readonly SessionEvent[]): Promise { - await this.ready - const id = session.header.id - const tracked = this.states.get(id) - if (tracked !== undefined) { - /* v8 ignore next -- initFor dedupes per session object; same-object re-entry can't occur */ - if (tracked.owner === session) return - if (tracked.owner === undefined) { - // Ownerless state from a public create()/load(). The first live session - // claims it ONLY if its seed reproduces the persisted prefix. - if (!await this.seedMatchesPersisted(id, seed, tracked.cursor)) { - throw new Error(`session "${id}" is already persisted with ${tracked.cursor} event(s) that do not match this live session (id collision)`) - } - tracked.owner = session - const suffix = seed.slice(tracked.cursor) - if (suffix.length > 0) await this.append(id, suffix) - return - } - // Owned by a DIFFERENT live session. Reclaim ONLY a truly-abandoned id - // (never materialized, no pending buffer); else it is a real collision. - const ownerBuffer = this.buffers.get(tracked.owner) - if (!tracked.materialized && !ownerBuffer?.length) { - this.states.delete(id) - } else { - throw new Error(`session "${id}" is already bound to a different live session in this backend (id collision)`) - } - } - - const row = this.rowFor(id) - if (row !== undefined) { - // Adopt a LIVE prefix without crash-repairing an open turn as interrupted; - // HMR may still append the real completion from the live Session. - await this.serialize(id, () => this.adoptLiveStoredPrefix(session, seed)) - return - } - - // case 4: a genuinely new session. - const meta: SessionHeader = { ...session.header } - await this.create(meta) - const created = this.states.get(id) - /* v8 ignore next -- create() always sets the state for the id */ - if (created !== undefined) created.owner = session - if (seed.length > 0) await this.append(id, seed) - } - - /** The preserved events for a session id (torn tail excluded, turn NOT yet closed). */ - private eventsFor(id: SessionId): SessionEvent[] { - const rows = this.db - .prepare('SELECT seq, type, time, data FROM events WHERE session_id = ? ORDER BY seq') - .all(id) as unknown as EventRow[] - // Scan on seq+type columns, parsing `data` only for the preserved prefix (a - // malformed torn tail must not throw here — same as loadCore). Returns the - // preserved events WITHOUT the synthetic closers, so a collision check - // compares a live seed against the real on-disk events, mirroring the JSONL - // backend's scanLog use in onCreated. - return scanRows(rows).preserved - } - - /** Whether a live session's seed reproduces the first `cursor` stored events. */ - private async seedMatchesPersisted(id: SessionId, seed: readonly SessionEvent[], cursor: number): Promise { - await this.ready - if (cursor === 0) return true - return seedCoversPrefix(seed, this.eventsFor(id).slice(0, cursor)) - } - - private async flush(session: Session): Promise { - await this.inits.get(session) - await this.serialize(session.header.id, () => this.drain(session)) - } - - /** Drain a session's write buffer to the database. Caller serializes per id. */ - private async drain(session: Session): Promise { - const buffer = this.buffers.get(session) - if (!buffer?.length) return - const batch = buffer.slice() - const state = this.states.get(session.header.id) - /* v8 ignore next -- state is always set by the awaited init before flush */ - const cursor = state?.cursor ?? 0 - const fresh = batch.filter(e => e.seq >= cursor) - if (fresh.length > 0) await this.appendCore(session.header.id, fresh) - buffer.splice(0, batch.length) - } } export default SessionPersistenceSqlite diff --git a/packages/session-persistence-sqlite/tests/sqlite.spec.ts b/packages/session-persistence-sqlite/tests/sqlite.spec.ts index 6cdc0dcb7a..aecfa5665d 100644 --- a/packages/session-persistence-sqlite/tests/sqlite.spec.ts +++ b/packages/session-persistence-sqlite/tests/sqlite.spec.ts @@ -3,11 +3,12 @@ import { Context } from 'cordis' import { mkdtemp, rm } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' -import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' -import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' +import SessionStore from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' import SessionPersistenceSqlite, { SCHEMA_VERSION } from '@deepseek-ai/dsh-session-persistence-sqlite' import { openDatabase, scanRows, type EventRow } from '../src/schema.ts' import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts' +import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts' const dirs: string[] = [] afterEach(async () => { for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) }) @@ -38,6 +39,31 @@ runPersistenceContract('sqlite', async () => { } }) +// Run the shared coordinator orchestration suite against the real SQLite backend. +// A FILE-backed db (not :memory:) is the shared storage scope so two mounted +// instances see the same rows (HMR/reload). `corruptTail` INSERTs a row past the +// committed seq whose `data` is invalid JSON — a never-committed torn tail that +// drives the coordinator's commitRepair-with-tornMarker branch over real db rows. +runCoordinatorContract('sqlite', async (): Promise => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-sqlite-coord-')) + const path = join(dir, 'sessions.db') + return { + mount: async ctx => ctx.plugin(SessionPersistenceSqlite, { path }), + corruptTail: async (id) => { + // A row past the committed region whose `data` does not parse: scanRows + // bounds the preserved prefix at it and returns its seq as tornFrom, which + // the backend surfaces to the coordinator as the tornMarker to delete from. + const db = openDatabase(path) + const next = (db.prepare('SELECT COALESCE(MAX(seq), -1) + 1 AS n FROM events WHERE session_id = ?') + .get(id) as { n: number }).n + db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)') + .run(id, next, 'assistant/chunk', 99, '{not valid json') + db.close() + }, + cleanup: async () => { await rm(dir, { recursive: true, force: true }) }, + } +}) + describe('scanRows', () => { // scanRows works off EventRows (data is a JSON string column); build them from // SessionEvents so the unit tests read in terms of the event vocabulary. @@ -108,35 +134,6 @@ describe('scanRows', () => { }) }) -describe('SessionPersistenceSqlite: HMR adoption', () => { - it('does not crash-repair an active open turn as interrupted', async () => { - const path = await freshDbPath() - const ctx = new Context() - await ctx.plugin(SessionStore) - const first = await ctx.plugin(SessionPersistenceSqlite, { path }) - const session = ctx.sessions.create('hmr-open', { meta: { cwd: '/hmr' } }) - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('step/start', { turn: 1, step: 1 }) - await ctx.parallel('session/flush', session) - - await first.dispose() - const db = openDatabase(path) - db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)') - .run('hmr-open', 2, 'step/end', 2, '{"torn":') - db.close() - const second = await ctx.plugin(SessionPersistenceSqlite, { path }) - session.append('step/end', { turn: 1, step: 1 }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - - const loaded = await ctx.sessionPersistence.load(SessionId('hmr-open')) - expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'step/start', 'step/end', 'turn/end']) - expect(loaded.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } }) - await second.dispose() - await ctx.fiber.dispose() - }) -}) - describe('SessionPersistenceSqlite: durability and crash semantics', () => { it('an interrupted turn (rows after the last turn/end) is PRESERVED and closed during load', async () => { const path = await freshDbPath() @@ -251,31 +248,6 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => { expect(() => openDatabase(olderPath)).toThrow(/incompatible with this build/) }) - it('append snapshots the batch: mutating an event after the call does not corrupt the persisted copy', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) - const m = meta('snapshot') - await ctx.sessionPersistence.create(m) - const batch: SessionEvent[] = [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'user/message', seq: 1, time: 2, data: { content: [{ type: 'text', text: 'original' }], source: { kind: 'user' } } }, - { type: 'turn/end', seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }, - ] - const p = ctx.sessionPersistence.append(m.id, batch) - // Mutate the live array AND an event's data AFTER the call but before it - // drains behind the per-session chain. The snapshot taken at call time must - // shield the persisted copy. - ;(batch[1]!.data as { content: { type: 'text'; text: string }[] }).content[0]!.text = 'HACKED' - batch.push({ type: 'user/message', seq: 3, time: 4, data: { content: [{ type: 'text', text: 'injected' }], source: { kind: 'user' } } }) - await p - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.events).toHaveLength(3) // the pushed event was not persisted - const um = loaded.events[1] - expect(um?.type === 'user/message' && (um.data.content[0] as { text: string }).text).toBe('original') - await fiber.dispose() - }) - it('a corrupt-JSON row in the uncommitted tail is discarded on load, not unloadable', async () => { const path = await freshDbPath() const m = meta('corrupt-tail') @@ -344,183 +316,12 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => { await fiber2.dispose() }) - it('rejects an unknown format version on load', async () => { - const path = await freshDbPath() - // Materialize a row with version 2 directly via the real schema. - const db = openDatabase(path) - db.prepare('INSERT INTO sessions (id, version, created_at) VALUES (?, ?, ?)') - .run('v2', 2, 1) - db.close() - - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path }) - await expect(ctx.sessionPersistence.load(SessionId('v2'))).rejects.toThrow(/version 2/) - await fiber.dispose() - }) - - it('create rejects a duplicate id (in memory and on a persisted row)', async () => { - const path = await freshDbPath() - const m = meta('dup') - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path }) - await ctx.sessionPersistence.create(m) - // Same in-memory state. - await expect(ctx.sessionPersistence.create(m)).rejects.toThrow(/already exists/) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - await fiber.dispose() - - // A fresh instance over the same file sees the persisted row. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) - await expect(ctx2.sessionPersistence.create(m)).rejects.toThrow(/already has a persisted row/) - await fiber2.dispose() - }) - it('exposes the schema version constant', () => { expect(SCHEMA_VERSION).toBe(2) }) }) -describe('SessionPersistenceSqlite: write path (session/event → flush)', () => { - function send(session: Session, events: SessionEvent[]): void { - for (const e of events) session.append(e.type, e.data) - } - - it('persists a turn appended through the live session on flush', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) - const session = ctx.sessions.create('w1') - send(session, oneTurnLog()) - await ctx.parallel('session/flush', session) - const loaded = await ctx.sessionPersistence.load(SessionId('w1')) - expect(loaded.events.map(e => e.type)).toEqual(oneTurnLog().map(e => e.type)) - await fiber.dispose() - }) - - it('a resumed session does not re-append its seed', async () => { - const path = await freshDbPath() - // Run 1: persist a full turn through the live session. - const ctx1 = new Context() - await ctx1.plugin(SessionStore) - const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path }) - const s1 = ctx1.sessions.create('resume') - for (const e of oneTurnLog()) s1.append(e.type, e.data) - await ctx1.parallel('session/flush', s1) - await fiber1.dispose() - - // Run 2: reconstruct the live session from the loaded log (seed), then add a - // second turn. The seed must NOT be re-appended (no UNIQUE collision), and - // the second turn continues the seq. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) - const { events } = await ctx2.sessionPersistence.load(SessionId('resume')) - const s2 = ctx2.sessions.create('resume', { seed: events }) - s2.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) - s2.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) - await ctx2.parallel('session/flush', s2) - const reloaded = await ctx2.sessionPersistence.load(SessionId('resume')) - expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await fiber2.dispose() - }) - - it('HMR: applying the plugin seeds existing live sessions', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const session = ctx.sessions.create('hmr') - for (const e of oneTurnLog()) session.append(e.type, e.data) - // Plugin applied AFTER the session already has events. - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) - await ctx.parallel('session/flush', session) - expect(await ctx.sessionPersistence.has(SessionId('hmr'))).toBe(true) - await fiber.dispose() - }) - - it('dispose drains a pending buffer before closing the database', async () => { - const path = await freshDbPath() - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path }) - const session = ctx.sessions.create('drain') - for (const e of oneTurnLog()) session.append(e.type, e.data) - // No explicit flush — dispose must drain the buffer. - await fiber.dispose() - - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) - expect(await ctx2.sessionPersistence.has(SessionId('drain'))).toBe(true) - await fiber2.dispose() - }) - - it('rejects a different live session colliding on a persisted id', async () => { - const path = await freshDbPath() - const ctx1 = new Context() - await ctx1.plugin(SessionStore) - const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path }) - const s1 = ctx1.sessions.create('collide') - for (const e of oneTurnLog()) s1.append(e.type, e.data) - await ctx1.parallel('session/flush', s1) - await fiber1.dispose() - - // A fresh, unrelated session reusing the id (no seed) must be rejected. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) - const s2 = ctx2.sessions.create('collide') - s2.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - await expect(ctx2.parallel('session/flush', s2)).rejects.toThrow(/id collision/) - await fiber2.dispose() - }) -}) - describe('SessionPersistenceSqlite: edge cases', () => { - it('append of an empty batch is a no-op', async () => { - const { ctx, dispose } = await backend() - const m = meta('empty-batch') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, []) - expect(await ctx.sessionPersistence.has(m.id)).toBe(false) // still lazy - await dispose() - }) - - it('load rejects a missing session', async () => { - const { ctx, dispose } = await backend() - await expect(ctx.sessionPersistence.load(SessionId('nope'))).rejects.toThrow(/not found/) - await dispose() - }) - - it('delete of a non-existent session is a no-op', async () => { - const { ctx, dispose } = await backend() - await ctx.sessionPersistence.delete(SessionId('ghost')) - expect(await ctx.sessionPersistence.has(SessionId('ghost'))).toBe(false) - await dispose() - }) - - it('append adopts a session that exists only in the DB (fresh instance)', async () => { - const path = await freshDbPath() - const m = meta('adopt-append') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) - await b1.dispose() - - // A fresh instance appends a second turn WITHOUT a prior create/load: append - // must adopt the on-disk row (cursor = stored length) and continue the seq. - const b2 = await backend(path) - await b2.ctx.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, - ]) - const loaded = await b2.ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await b2.dispose() - }) - it('append rolls back and rethrows when an event INSERT fails inside the transaction', async () => { const path = await freshDbPath() const m = meta('rollback-insert') @@ -549,190 +350,6 @@ describe('SessionPersistenceSqlite: edge cases', () => { await b2.dispose() }) - it('round-trips a header with parentSession (fork lineage)', async () => { - const { ctx, dispose } = await backend() - const m: SessionHeader = { ...meta('child'), parentSession: SessionId('parent') } - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.meta.parentSession).toBe(SessionId('parent')) - await dispose() - }) - - it('a fresh live session reusing a previously-loaded id is rejected (ownerless guard)', async () => { - const path = await freshDbPath() - const m = meta('ownerless') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) - await b1.dispose() - - const b2 = await backend(path) - // load() leaves ownerless state with cursor 6. - await b2.ctx.sessionPersistence.load(m.id) - // A fresh, unrelated live session reusing the id has a shorter/non-matching - // seed → its onCreated must reject rather than graft onto the loaded prefix. - const s = b2.ctx.sessions.create('ownerless') - s.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - await expect(b2.ctx.parallel('session/flush', s)).rejects.toThrow(/id collision/) - await b2.dispose() - }) - - it('a live session whose seed matches the loaded prefix claims ownerless state and persists the suffix', async () => { - const path = await freshDbPath() - const m = meta('claim') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) - await b1.dispose() - - const b2 = await backend(path) - const { events } = await b2.ctx.sessionPersistence.load(m.id) // ownerless, cursor 6 - // A live session seeded with the loaded log PLUS a new turn claims the state - // and persists only the suffix. - const s = b2.ctx.sessions.create('claim', { seed: [ - ...events, - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, - ] }) - await b2.ctx.parallel('session/flush', s) - const loaded = await b2.ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await b2.dispose() - }) - - it('an abandoned lazy session (never materialized) releases its id for reuse', async () => { - const { ctx, dispose } = await backend() - const inits = (ctx.sessionPersistence as unknown as { inits: Map> }).inits - let first!: Session - const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { - first = inner.sessions.create('reuse') - }, { inject: ['sessions'] })) - await inits.get(first) // let the lazy create register the state - await firstFiber.dispose() // disposed before any append → never materialized - - let reuse!: Session - await ctx.plugin(Object.assign((inner: Context) => { - reuse = inner.sessions.create('reuse') - }, { inject: ['sessions'] })) - await expect(inits.get(reuse)).resolves.toBeUndefined() - reuse.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - reuse.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', reuse) - expect(await ctx.sessionPersistence.has(SessionId('reuse'))).toBe(true) - await dispose() - }) - - it('does NOT reclaim an id whose abandoned owner still has buffered (unflushed) events', async () => { - const { ctx, dispose } = await backend() - const inits = (ctx.sessionPersistence as unknown as { inits: Map> }).inits - let first!: Session - const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { - first = inner.sessions.create('buffered') - }, { inject: ['sessions'] })) - await inits.get(first) - // Append a turn but do NOT flush — events sit in the write-behind buffer. - first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - first.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await firstFiber.dispose() // disposed before flush; not materialized, buffer pending - - let reuse!: Session - await ctx.plugin(Object.assign((inner: Context) => { - reuse = inner.sessions.create('buffered') - }, { inject: ['sessions'] })) - await expect(inits.get(reuse)).rejects.toThrow(/already bound to a different live session/) - await dispose() - }) - - it('initFor is idempotent: re-emitting session/created does not re-initialize', async () => { - const { ctx, dispose } = await backend() - const session = ctx.sessions.create('idem') - ctx.emit('session/created', session) // second create event for the same object - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - expect(await ctx.sessionPersistence.has(SessionId('idem'))).toBe(true) - await dispose() - }) - - it('a live session claims cursor-0 ownerless state created via the public API and persists its seed', async () => { - const { ctx, dispose } = await backend() - // create() registers ownerless state with cursor 0 (no events yet). - await ctx.sessionPersistence.create(meta('cursor0')) - // A live session reusing that id, seeded with a turn, claims the ownerless - // state (cursor 0 trivially matches any seed) and persists the whole seed. - const s = ctx.sessions.create('cursor0', { seed: [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'completed' } } }, - ] }) - await ctx.parallel('session/flush', s) - const loaded = await ctx.sessionPersistence.load(SessionId('cursor0')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1]) - await dispose() - }) - - it('HMR: reloading the backend adopts a still-live, already-materialized session', async () => { - const path = await freshDbPath() - const ctx = new Context() - await ctx.plugin(SessionStore) - // The session lives in its OWN fiber so it survives the backend reload. - let session!: Session - await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create('hmr-adopt') - }, { inject: ['sessions'] })) - - // Backend instance 1 materializes the session on disk. - const backend1 = await ctx.plugin(SessionPersistenceSqlite, { path }) - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - - // Hot-reload: dispose instance 1, plug in instance 2 over the SAME file - // while the session stays live. Instance 2 has an empty states map but the - // row is materialized on disk and is a prefix of the live events — it must - // ADOPT (not reject), and a second turn then persists. - await backend1.dispose() - await ctx.plugin(SessionPersistenceSqlite, { path }) - session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) - await expect(ctx.parallel('session/flush', session)).resolves.not.toThrow() - - const loaded = await ctx.sessionPersistence.load(SessionId('hmr-adopt')) - expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2) - await ctx.fiber.dispose() - }) - - it('HMR: adoption persists the live SUFFIX that was ahead of the on-disk prefix', async () => { - const path = await freshDbPath() - const ctx = new Context() - await ctx.plugin(SessionStore) - let session!: Session - await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create('hmr-suffix') - }, { inject: ['sessions'] })) - - const backend1 = await ctx.plugin(SessionPersistenceSqlite, { path }) - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - - // Append turn 2 to the LIVE session, then dispose instance 1 WITHOUT - // flushing turn 2: it is now ONLY in the live session's events. - await backend1.dispose() - session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) - - // Instance 2 adopts the on-disk prefix (turn 1) and MUST persist the live - // suffix (turn 2) carried in the session's events. - await ctx.plugin(SessionPersistenceSqlite, { path }) - await ctx.parallel('session/flush', session) - const loaded = await ctx.sessionPersistence.load(SessionId('hmr-suffix')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3]) - expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2) - await ctx.fiber.dispose() - }) - it('HMR: a DIFFERENT session colliding with a materialized on-disk id is rejected', async () => { const path = await freshDbPath() // Instance 1 materializes a session and disposes. diff --git a/packages/session-persistence/README.md b/packages/session-persistence/README.md index 521020e80d..a63ea6284a 100644 --- a/packages/session-persistence/README.md +++ b/packages/session-persistence/README.md @@ -21,11 +21,31 @@ The persisted unit IS the existing `SessionEvent` (event-sourced model — the l - **JSON-serializable data.** `append` rejects non-serializable `event.data`; backends snapshot each event when buffering (the live `session.events` object is mutable). - **Durability.** `append` returns only once the batch is durable. +## The write coordinator + +The two first-party backends were byte-identical (or same-algorithm) for ALL of their write-path orchestration — the in-memory bookkeeping (per-id state, write-behind buffers, per-id serialization chains, per-session init promises), the `session/event` → buffer → `session/flush` drain, lazy materialization, crash-tail repair on load, the four `session/created` adoption cases (new / HMR-adopt / collision / ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives differed (write bytes vs. INSERT rows). + +`PersistenceCoordinator` owns that orchestration once. A first-party backend composes one (`new PersistenceCoordinator(ctx, this)`), implements the small `PersistenceBackend` hook interface, and delegates its six public service methods to the coordinator. This keeps the duplicated, correctness-heavy orchestration in a single place (it used to receive the same fixes twice). + +The `PersistenceBackend` hooks (the only seam between the coordinator and storage): + +| Hook | Role | +|---|---| +| `name` | Backend label for the dispose-failure `AggregateError`. | +| `loadStored(id)` | Read a stored prefix by id, scanning ANY storage scope. Used by resume/load and, via `!== undefined`, the create-collision probe. Returns an opaque `tornMarker` iff a torn tail must be truncated. | +| `loadLive(id, cwd)` | Read a stored prefix SCOPED to `cwd` (HMR live-adoption must only adopt a log at the SAME cwd; a same-id log elsewhere is a collision, not a resume). A globally-unique-id backend ignores `cwd`. | +| `appendBatch(meta, events, isMaterialized)` | Durably append a contiguous batch, lazily materializing ATOMICALLY when not yet materialized. | +| `commitRepair(meta, tornMarker, closers)` | Make a crash repair durable: truncate the torn tail (iff `tornMarker`) and append `closers`. NOT required to be atomic. Used by load (truncate + closers) and live-adoption (truncate only). | +| `deleteStored(id)` / `list()` | Remove a stored artifact / list all stored metadata. | +| `close?()` | Optional lifecycle teardown (e.g. close a db handle), awaited after the dispose drain. | + +The `tornMarker` is fully OPAQUE: the coordinator only tests `!== undefined` and round-trips it to `commitRepair`, never inspecting its value (the JSONL backend uses the byte offset to truncate to, the SQLite backend the seq to delete from). The public `SessionPersistence` service shape is unchanged, so a third-party backend MAY still implement the abstract service directly without the coordinator. See [the write-coordinator RFC](../../docs/rfc/implemented/2026-06-18-shared-persistence-write-coordinator.md). + ## Testing backends -Import `runPersistenceContract` from `tests/contract.ts` and call it with a factory that yields a fresh, empty backend plus a teardown. Every backend is held to the same append-only / contiguous-seq / lazy-materialization / serializability semantics; a backend's own spec adds implementation-specific tests (crash repair, path sanitization) on top. +Import `runPersistenceContract` from `tests/contract.ts` (the public-API contract) and `runCoordinatorContract` from `tests/coordinator-contract.ts` (the shared write-path orchestration: adoption, HMR, collision, dispose-drain, crash-tail repair) and call each with a fixture for your backend. Every backend is held to the same append-only / contiguous-seq / lazy-materialization / serializability semantics AND the same orchestration, so a backend's own spec is left with only storage-mechanics tests (path sanitization, fsync rollback; schema version, transaction rollback) on top. -Two backends run this suite: `dsh-session-persistence-jsonl` (append-only file log) and `dsh-session-persistence-sqlite` (`node:sqlite`, each `SessionEvent` one row `(session_id, seq, type, time, data)`). Both passing the same contract is the proof that the seam is genuinely backend-agnostic — lazy materialization, crash-tail-on-load, and contiguous-seq hold identically over file bytes and over a transactional store. +Three backends run these suites: an in-memory reference (in `tests/`), `dsh-session-persistence-jsonl` (append-only file log) and `dsh-session-persistence-sqlite` (`node:sqlite`, each `SessionEvent` one row `(session_id, seq, type, time, data)`). All passing the same contract + coordinator suite is the proof that the seam is genuinely backend-agnostic — lazy materialization, crash-tail-on-load, and contiguous-seq hold identically over file bytes and over a transactional store. ## Metadata types diff --git a/packages/session-persistence/src/coordinator.ts b/packages/session-persistence/src/coordinator.ts new file mode 100644 index 0000000000..09bc6eb943 --- /dev/null +++ b/packages/session-persistence/src/coordinator.ts @@ -0,0 +1,556 @@ +/** + * The backend-agnostic write-path orchestration shared by every first-party + * {@link SessionPersistence} backend. + * + * The two durable backends (`dsh-session-persistence-jsonl` over file bytes, + * `dsh-session-persistence-sqlite` over `node:sqlite` rows) were byte-identical + * — or same-algorithm — for ALL of their orchestration: the in-memory + * bookkeeping (the per-id state, the write-behind buffers, the per-id + * serialization chains, the per-session init promises), the `session/event` → + * buffer → `session/flush` drain, lazy materialization, crash-tail repair on + * load, the four `session/created` adoption cases (new / HMR-adopt / collision / + * ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives + * differed (write bytes vs. INSERT rows). {@link PersistenceCoordinator} owns + * the orchestration once; a backend supplies the storage primitives as a small + * {@link PersistenceBackend} hook object. + * + * The abstract {@link SessionPersistence} service's public API is unchanged: a + * backend still IS a `SessionPersistence` (its six public methods delegate to a + * coordinator it composes), so a third-party backend MAY implement the service + * directly without using the coordinator at all. + * + * See the write-coordinator RFC (docs/rfc/implemented/2026-06-18-shared-persistence-write-coordinator.md) + * for the design rationale (composition over inheritance, the opaque torn marker). + * + * @module @deepseek-ai/dsh-session-persistence/coordinator + */ + +import { Context } from 'cordis' +import { interruptedTurnClosers } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' +import { assertSerializable, seedCoversPrefix } from './index.ts' + +/** + * A stored session's durable prefix as read back from a backend: its + * {@link SessionHeader}, the preserved (seq-contiguous, parseable) event prefix, + * and an OPAQUE `tornMarker` that is present iff a never-committed torn tail must + * be truncated before further writes. + * + * The coordinator NEVER inspects `tornMarker`'s value — it only tests + * `!== undefined` (is there a tail to repair?) and passes the value back to + * {@link PersistenceBackend.commitRepair}. Each backend chooses its own marker + * type: the JSONL backend uses the byte offset to truncate to, the SQLite + * backend uses the seq to delete from (both happen to be `number`). + */ +export interface StoredPrefix { + meta: SessionHeader + events: SessionEvent[] + tornMarker?: TornMarker +} + +/** + * The storage seam between {@link PersistenceCoordinator} and a concrete + * backend: the minimal set of durable primitives the orchestration calls. A + * backend implements these (over files, rows, an object store, …); the + * coordinator supplies everything else (buffering, serialization, cursors, + * adoption, crash repair sequencing, dispose quiescence). + * + * @typeParam TornMarker - the backend's opaque torn-tail repair token (see + * {@link StoredPrefix}). The coordinator treats it as fully opaque. + */ +export interface PersistenceBackend { + /** Human-readable backend name, used in the dispose-failure AggregateError. */ + readonly name: string + + /** + * Read a stored prefix by id, scanning ANY storage scope (for JSONL: every + * cwd bucket). Returns `undefined` if no stored artifact exists. Used by + * resume/load, and — via `!== undefined` — by the create-collision probe. + * The returned `tornMarker` is present iff there is a torn tail to truncate. + */ + loadStored(id: SessionId): Promise | undefined> + + /** + * Read a stored prefix SCOPED to `cwd`. Deliberately distinct from + * {@link loadStored}: HMR live-adoption must only adopt a persisted log at the + * SAME cwd as the live session (a same-id log at a different cwd is a + * collision, not a resume) — conflating the two reintroduces a cross-cwd + * adoption bug. For a globally-unique-id backend (SQLite) `cwd` is ignored. + */ + loadLive(id: SessionId, cwd: string | undefined): Promise | undefined> + + /** + * Durably append a CONTIGUOUS batch, lazily materializing the session first + * when `!isMaterialized`. The materialize-write and the first event batch MUST + * commit ATOMICALLY (a crash between them must not leave a materialized-but- + * empty session). Returns once the batch is durable. + */ + appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise + + /** + * Make a crash repair durable: truncate the torn tail (iff + * `tornMarker !== undefined`) and append `closers` (iff any). NOT required to + * be atomic — a file backend may truncate-then-append in two fsync'd steps. + * Used by load (truncate + synthetic closers) and by live-adoption (truncate + * only, `closers = []`). + */ + commitRepair(meta: SessionHeader, tornMarker: TornMarker | undefined, closers: readonly SessionEvent[]): Promise + + /** Remove the stored artifact for `id` (the coordinator clears in-memory state). */ + deleteStored(id: SessionId): Promise + + /** List all stored (materialized) sessions' metadata. */ + list(): Promise + + /** + * Optional lifecycle teardown (e.g. close a database handle). Awaited by the + * coordinator's dispose effect AFTER the quiescence drain. A stateless file + * backend omits it. + */ + close?(): Promise +} + +/** Per-session write state held by the coordinator's in-memory bookkeeping. */ +interface SessionState { + meta: SessionHeader + /** The next seq the backend expects to append (the stored log length). */ + cursor: number + /** Whether the session has been physically materialized. */ + materialized: boolean + /** + * The live Session this state was bound to via `onCreated`, if any. State + * created through the public `create()`/`load()` API has no owner; state bound + * to a live session lets `onCreated` reject a second, unrelated session on the + * same id (a collision) instead of silently no-opping. + */ + owner?: Session +} + +/** Collect the rejection reasons from a set of promises (none-throwing). */ +async function settledErrors(promises: Iterable>): Promise { + const settled = await Promise.allSettled([...promises]) + const errors: unknown[] = [] + for (const result of settled) { + if (result.status === 'rejected') errors.push(result.reason) + } + return errors +} + +/** + * Owns the backend-agnostic session write-path orchestration. A backend + * constructs one (`new PersistenceCoordinator(ctx, this)`), implements + * {@link PersistenceBackend}, and delegates its six public service methods to + * the matching coordinator methods. + * + * All per-id operations are serialized (a per-id promise chain) so concurrent + * flushes / a flush racing a load never interleave storage writes. The + * constructor installs the write-path listeners and the dispose effect. + * + * @typeParam TornMarker - the backend's opaque torn-tail repair token. + */ +export class PersistenceCoordinator { + /** Backend bookkeeping keyed by session id (NOT the live Session object). */ + private states = new Map() + /** Write-behind buffers keyed by the live Session (write path). */ + private buffers = new Map() + /** + * Per-session serialization: every operation chains onto the prior one for the + * same id, so writes for one session never interleave. Keyed by session id. + */ + private chains = new Map>() + /** + * Per-session init promise (onCreated). Keyed by the LIVE Session OBJECT, not + * its id: a disposed fiber's session can be replaced by a different live + * Session reusing the same id (HMR, an ACP reconnect), and an id-keyed cache + * would hand the new object the old object's init promise. + * + * Public (readonly) so a backend can expose it for white-box tests that await + * a specific session's init (there is no public API to await one init); the + * coordinator itself only ever mutates it internally. + */ + readonly inits = new Map>() + + constructor(private ctx: Context, private backend: PersistenceBackend) { + this.installWritePath() + } + + // --- public surface (the backend's service methods delegate here) --- + + /** + * Register a new session's metadata (lazy: no physical write until the first + * {@link append}). Rejects if the id is already tracked or already persisted. + */ + create(meta: SessionHeader): Promise { + // Snapshot the metadata at call time: the op runs later (behind the + // per-session chain) and the snapshot is stored as the lazy state, so keeping + // the caller's object by reference would let a later mutation of `id`/`cwd` + // register under one key but materialize under a different path/header. + const snapshot: SessionHeader = { ...meta } + return this.serialize(snapshot.id, () => this.createCore(snapshot)) + } + + private async createCore(meta: SessionHeader): Promise { + // Do NOT clobber an existing session: the SessionId IS the identity. + if (this.states.has(meta.id)) { + throw new Error(`session "${meta.id}" already exists in this backend`) + } + // A persisted artifact under this id (in ANY scope) blocks creation: load/ + // has/resume identify a session by id alone, so a second artifact would make + // resume nondeterministic. + if (await this.backend.loadStored(meta.id) !== undefined) { + throw new Error(`session "${meta.id}" already has a persisted log on disk; load/resume it instead of creating`) + } + // Pure lazy: record intent only. No artifact until the first append. + this.states.set(meta.id, { meta, cursor: 0, materialized: false }) + } + + // `async` so the synchronous validate/clone below reject (not throw) per the + // Promise contract — callers use `await expect(...).rejects`. + /** + * Durably persist a batch of events. Honors the append-only and contiguous-seq + * contracts; rejects non-JSON-serializable `event.data`. + */ + async append(id: SessionId, events: readonly SessionEvent[]): Promise { + // Validate serializability BEFORE cloning so a bad event surfaces the typed + // error rather than an opaque DataCloneError from structuredClone. + assertSerializable(events) + // Deep-snapshot the batch HERE, before the op waits behind the per-session + // chain: a caller that mutates a live array (e.g. session.events) — or an + // event inside it — before the op runs would otherwise have those changes + // persisted. The clone is taken synchronously (at call time). + const batch = events.map(e => structuredClone(e)) + return this.serialize(id, () => this.appendCore(id, batch)) + } + + private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise { + if (events.length === 0) return + let state = this.states.get(id) + if (state === undefined) state = await this.adopt(id) // calls loadCore, not load + + // Contiguity contract: each event's seq must continue the stored log. + for (const [i, event] of events.entries()) { + if (event.seq !== state.cursor + i) { + throw new Error(`append seq mismatch for "${id}": expected ${state.cursor + i} at index ${i}, got ${event.seq}`) + } + } + + await this.backend.appendBatch(state.meta, events, state.materialized) + // The durable write is the transaction: mark materialized + advance the + // cursor as soon as it commits (uniform across backends). + state.materialized = true + state.cursor += events.length + } + + /** + * Reload a session: its {@link SessionHeader} plus the event log up to the last + * durable checkpoint, with any interrupted final turn durably closed (synthetic + * boundary events) during load. + */ + load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { + return this.serialize(id, () => this.loadCore(id)) + } + + private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { + const stored = await this.backend.loadStored(id) + if (stored === undefined) throw new Error(`session "${id}" not found`) + const { meta, events, tornMarker } = stored + this.assertVersion(meta) + + // Crash-recovery: if the log ended mid-turn (real, preserved events but no + // closing turn/end), close it durably DURING load so disk, the returned log, + // and the cursor all agree. The interrupted turn's real events are preserved, + // never truncated (a turn can be huge — the session-persistence RFC); only a + // never-fully-written torn tail fragment is discarded. + const closers = interruptedTurnClosers(events) + const balanced = [...events, ...closers] + + // Make the repair durable (truncate the torn tail + append the synthetic + // closers) BEFORE recording state — commitRepair takes `meta` directly, so + // there is no state-path ordering dependency (uniform across backends). + if (tornMarker !== undefined || closers.length > 0) { + await this.backend.commitRepair(meta, tornMarker, closers) + } + // The state keeps its OWN copy of the meta; the returned value is separate so + // a consumer mutating loaded.meta cannot corrupt the backend's metadata. + this.states.set(id, { meta: { ...meta }, cursor: balanced.length, materialized: true }) + return { meta, events: balanced } + } + + // NOTE: there is deliberately no coordinator `list()`. Listing needs none of + // the coordinator's orchestration (no per-id serialization, no cursor, no + // in-memory state) — it is a pure read of stored metadata. A backend's public + // `list()` IS the {@link PersistenceBackend.list} hook (one method); routing it + // through the coordinator would only forward to that same hook, so the + // coordinator stays out of the listing path entirely. + + /** Whether a session is durably present (materialized). */ + async has(id: SessionId): Promise { + const state = this.states.get(id) + if (state?.materialized) return true + // Probe storage scoped to the tracked cwd if known, else any scope. A tracked + // lazy session has a known cwd, so loadLive(id, cwd) hits the exact artifact + // path — a storage fault there (e.g. a non-ENOENT lookup error) must surface, + // not be masked by an any-scope scan that filters a non-directory bucket out. + // For an untracked id `cwd` is undefined, where loadLive scans any scope (= + // loadStored), so this single call covers both. + return (await this.backend.loadLive(id, state?.meta.cwd)) !== undefined + } + + /** Remove a session and all its persisted artifacts. */ + delete(id: SessionId): Promise { + return this.serialize(id, () => this.deleteCore(id)) + } + + private async deleteCore(id: SessionId): Promise { + await this.backend.deleteStored(id) + this.states.delete(id) + } + + // --- per-id serialization + adoption helpers --- + + /** + * Run `op` after any in-flight operation for the same session id, so writes for + * one session never interleave. Errors do not poison the chain. NOTE: serialized + * public methods must NOT call each other (deadlock); they call the unserialized + * `*Core` helpers instead. + */ + private serialize(id: SessionId, op: () => Promise): Promise { + const prior = this.chains.get(id) ?? Promise.resolve() + const next = prior.then(op, op) + // Keep the chain alive but swallow this op's rejection for the NEXT waiter + // (the caller still sees the real rejection via `next`). + this.chains.set(id, next.then(() => undefined, () => undefined)) + return next + } + + /** Build a state for a session discovered in storage but not yet in memory. */ + private async adopt(id: SessionId): Promise { + // loadCore (NOT load) — adopt runs inside an already-serialized op, so + // re-entering the chain via the public load() would deadlock. + await this.loadCore(id) + const state = this.states.get(id) + /* v8 ignore next -- loadCore always sets the state for the id */ + if (!state) throw new Error(`failed to adopt session "${id}"`) + return state + } + + private assertVersion(meta: SessionHeader): void { + if (meta.version !== 1) { + throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`) + } + } + + // --- write path (session/event → flush drain) --- + + private installWritePath(): void { + const ctx = this.ctx + + // Capture the header on creation; persist a fork's seed once. Record the init + // promise so flush/dispose can await it (onCreated is async). + ctx.on('session/created', (session) => { void this.initFor(session) }) + + // Snapshot + buffer every event (the live object is mutable; clone so a later + // in-place mutation cannot rewrite a buffered event). Serializability is + // guaranteed at the source (Session.append), so structuredClone is safe. + ctx.on('session/event', (session, event) => { + let buffer = this.buffers.get(session) + if (!buffer) this.buffers.set(session, buffer = []) + buffer.push(structuredClone(event)) + }) + + // Drain to the backend at the durability checkpoint. + ctx.on('session/flush', session => this.flush(session)) + + // Dispose must reach quiescence: await every init + final drain BEFORE + // returning, then close the backend's own resources (AFTER the drain), so no + // write lands after teardown and a close failure never MASKS a drain error. + ctx.effect(() => async () => { + let disposeError: unknown + try { + const errors = [ + ...await settledErrors(this.inits.values()), + ...await settledErrors([...this.buffers.keys()].map(s => this.flush(s))), + ...await settledErrors(this.chains.values()), + ] + if (errors.length > 0) { + throw new AggregateError(errors, `${this.backend.name} dispose failed`) + } + } catch (error: unknown) { + disposeError = error + throw error + } finally { + try { + await this.backend.close?.() + } catch (closeError: unknown) { + // A close failure can only add teardown context; keep the already- + // captured drain AggregateError as the primary failure rather than + // masking it. Only surface the close error if the drain succeeded. + /* v8 ignore start -- close failure racing disposal is a defensive teardown edge */ + if (disposeError === undefined) throw closeError + /* v8 ignore stop */ + } + } + }, `${this.backend.name} write path`) + + // HMR: a hot reload does not replay session/created, so seed existing live + // sessions (mirrors dsh-invariants). + for (const session of ctx.sessions.list()) void this.initFor(session) + } + + /** Start (once) the async init for a session and remember its promise. */ + private initFor(session: Session): Promise { + const existing = this.inits.get(session) + if (existing) return existing + // Snapshot the seed SYNCHRONOUSLY — initFor runs inside the `session/created` + // emit, before any later `append` adds non-seed events. A clone freezes it + // against later mutation of the live event objects. + const seed = session.events.map(e => structuredClone(e)) + const p = this.onCreated(session, seed) + // Attach a no-op rejection handler so a failing init does not surface as an + // unhandled rejection if no flush observes `p` before it rejects. The REAL + // error is still delivered: flush/dispose await the same `p` from the map. + p.catch(() => { /* observed by flush/dispose via the stored promise */ }) + this.inits.set(session, p) + return p + } + + /** + * Whether a live session's `seed` reproduces the first `cursor` persisted + * events. A `cursor` of 0 (nothing persisted yet) trivially matches. Used when + * a live session claims ownerless state left by a prior `load()`/`create()`. + */ + private async seedMatchesPersisted(id: SessionId, seed: readonly SessionEvent[], cursor: number): Promise { + if (cursor === 0) return true + const stored = await this.backend.loadStored(id) + /* v8 ignore next -- a cursor > 0 means the session was materialized, so it exists */ + if (stored === undefined) return false + return seedCoversPrefix(seed, stored.events.slice(0, cursor)) + } + + /** + * On session/created: sync the backend's in-memory state to a live Session. + * + * Cases, by whether this backend tracks the id and whether an artifact exists: + * 1. Already tracked → no-op (or claim ownerless state if the seed matches, + * or reclaim a truly-abandoned id, else reject as a collision). + * 2. Not tracked, an artifact EXISTS at this cwd and is a seq-aligned PREFIX + * of the live events → ADOPT it (HMR/reload), persisting any live suffix. + * 3. Not tracked, an artifact EXISTS but is NOT a prefix → REJECT (collision). + * 4. Not tracked and NO artifact → a genuinely new session: register meta + * (lazy) and persist its seed once. + */ + private async onCreated(session: Session, seed: readonly SessionEvent[]): Promise { + const id = session.header.id + const tracked = this.states.get(id) + if (tracked !== undefined) { + // case 1: already tracked. + /* v8 ignore next -- initFor dedupes per session object; same-object re-entry can't occur */ + if (tracked.owner === session) return + if (tracked.owner === undefined) { + // Ownerless state from the public create()/load() API. The FIRST live + // session claims it — but ONLY if its seed reproduces the persisted + // prefix (else a fresh, unrelated session reusing the id would have its + // seq 0..cursor-1 events filtered as already-written and grafted on). + if (!await this.seedMatchesPersisted(id, seed, tracked.cursor)) { + throw new Error(`session "${id}" is already persisted with ${tracked.cursor} event(s) that do not match this live session (id collision)`) + } + tracked.owner = session + // Persist the seed SUFFIX beyond the persisted prefix. Constructor seed + // events never emit session/event, so the buffer never sees them. + const suffix = seed.slice(tracked.cursor) + if (suffix.length > 0) await this.append(id, suffix) + return + } + // Owned by a DIFFERENT live session. Reclaim ONLY a truly-abandoned id + // (never materialized, no pending buffer); else it is a real collision. + const ownerBuffer = this.buffers.get(tracked.owner) + if (!tracked.materialized && !ownerBuffer?.length) { + this.states.delete(id) + } else { + throw new Error(`session "${id}" is already bound to a different live session in this backend (id collision)`) + } + } + + // case 2/3: an artifact at THIS cwd is adopted as a live prefix (or rejected + // as a collision inside adoptLivePrefix). cwd-scoped (loadLive), never + // any-scope: a same-id artifact at a different cwd is a collision, not a + // resume. + const live = await this.backend.loadLive(id, session.header.cwd) + if (live !== undefined) { + // Do NOT route through loadCore(): that crash-repairs open turns as + // interrupted, which is wrong for HMR while the live Session is still the + // authority and may append the real step/turn end later. + await this.serialize(id, () => this.adoptLivePrefix(session, seed, live)) + return + } + + // case 4: a genuinely new session. Register its meta (lazy), then persist its + // seed (events present at creation time) once. + const meta: SessionHeader = { ...session.header } + await this.create(meta) + // Bind this state to the live session so a later DIFFERENT session reusing + // the id is detected as a collision (case 1) rather than silently no-opped. + const created = this.states.get(id) + /* v8 ignore next -- create() always sets the state for the id */ + if (created !== undefined) created.owner = session + if (seed.length > 0) await this.append(id, seed) + } + + /** + * Adopt a stored prefix as a live session's history (HMR/reload): verify the + * seed covers the stored prefix, truncate any torn tail (NOT the open turn — + * the live Session is still the authority), bind ownership, and persist the + * live suffix that was ahead of the stored prefix. + */ + private async adoptLivePrefix(session: Session, seed: readonly SessionEvent[], stored: StoredPrefix): Promise { + const { meta, events, tornMarker } = stored + this.assertVersion(meta) + if (!seedCoversPrefix(seed, events)) { + throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`) + } + // Truncate-only repair (no closers): the open turn is NOT closed here. + if (tornMarker !== undefined) await this.backend.commitRepair(meta, tornMarker, []) + this.states.set(session.header.id, { + meta: { ...meta }, + cursor: events.length, + materialized: true, + owner: session, + }) + const suffix = seed.slice(events.length) + if (suffix.length > 0) await this.appendCore(session.header.id, suffix) + } + + private async flush(session: Session): Promise { + // Wait for the session's init (onCreated) so the state/cursor and any + // fork-seed persistence are in place before draining. Awaiting the same + // promise initFor stored also surfaces an init failure (e.g. a collision) + // here, where the caller of session/flush observes it. + await this.inits.get(session) + // Serialize the WHOLE drain (read cursor → append → splice) on the per-session + // chain so two concurrent flushes cannot both read the same cursor and + // seq-mismatch on the second append. + await this.serialize(session.header.id, () => this.drain(session)) + } + + /** Drain a session's write buffer to the backend. Caller serializes this per id. */ + private async drain(session: Session): Promise { + const buffer = this.buffers.get(session) + if (!buffer?.length) return + // Copy WITHOUT removing: the buffer is the only durable-pending copy of these + // events. Drain it only AFTER the append commits; events pushed during the + // await sit past batch.length and survive the prefix splice, so a + // retry/dispose re-drains the rest. + const batch = buffer.slice() + const state = this.states.get(session.header.id) + // Only append events at or beyond the write cursor (a resumed session's seed + // is already stored). flush awaits the init above, which always sets state, + // so the `?? 0` fallback is a defensive guard that never fires in practice. + /* v8 ignore next -- state is always set by the awaited init before flush */ + const cursor = state?.cursor ?? 0 + const fresh = batch.filter(e => e.seq >= cursor) + // appendCore (NOT the serialized append) — drain already runs inside the + // per-session chain, so re-entering via append() would deadlock. + if (fresh.length > 0) await this.appendCore(session.header.id, fresh) + buffer.splice(0, batch.length) + } +} diff --git a/packages/session-persistence/src/index.ts b/packages/session-persistence/src/index.ts index 48402b54a7..8ff9aa8cb2 100644 --- a/packages/session-persistence/src/index.ts +++ b/packages/session-persistence/src/index.ts @@ -28,6 +28,10 @@ import type { SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-se // Re-export the metadata vocabulary so consumers import it from the seam. export type { SessionHeader } from '@deepseek-ai/dsh-session' +// The backend-agnostic write-path orchestration first-party backends compose. +export { PersistenceCoordinator } from './coordinator.ts' +export type { PersistenceBackend, StoredPrefix } from './coordinator.ts' + declare module 'cordis' { interface Context { sessionPersistence: SessionPersistence diff --git a/packages/session-persistence/tests/coordinator-contract.ts b/packages/session-persistence/tests/coordinator-contract.ts new file mode 100644 index 0000000000..24a8f4c0cf --- /dev/null +++ b/packages/session-persistence/tests/coordinator-contract.ts @@ -0,0 +1,734 @@ +/** + * Reusable ORCHESTRATION suite for any backend that composes a + * {@link PersistenceCoordinator}. Where {@link runPersistenceContract} (in + * contract.ts) pins the public read/write SEMANTICS, this suite pins the + * coordinator's WRITE-PATH ORCHESTRATION — the behavior that is identical across + * every first-party backend because it lives in the shared coordinator, not in + * the storage primitives: the `session/created` → `session/event` → + * `session/flush` → dispose drain, lazy materialization, fork-seed persistence, + * the four `onCreated` adoption cases (new / HMR-adopt / collision / + * ownerless-claim), crash-tail repair on load, and dispose-time quiescence. + * + * A backend imports {@link runCoordinatorContract} and calls it with a + * {@link CoordinatorFixture} factory that knows how to (a) mount the REAL + * backend plugin on a {@link Context} over a SHARED storage scope (so HMR/reload + * tests can dispose one instance and mount another over the same bytes/rows), + * and (b) inject a never-committed torn tail for one session + * ({@link CoordinatorFixture.corruptTail}) so the through-coordinator torn-tail + * repair branch is exercised against real storage. The suite drives everything + * through the PUBLIC {@link SessionPersistence} API + the cordis SessionStore + * write path — never the storage primitives directly — so it runs unchanged for + * every backend (memory / jsonl / sqlite). + * + * Each scenario here was previously DUPLICATED in `jsonl.spec.ts` and + * `sqlite.spec.ts`; it now lives once and runs once per backend through the + * fixture. The per-backend specs keep ONLY their storage-mechanics tests. + * + * @module @deepseek-ai/dsh-session-persistence/tests/coordinator-contract + */ + +import { describe, expect, it } from 'vitest' +import { Context, type Fiber } from 'cordis' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' +import type { SessionPersistence } from '../src/index.ts' +import { meta, oneTurnLog } from './contract.ts' + +/** + * The backend-specific capabilities the orchestration suite needs beyond the + * public service API. A fresh fixture is created per test (isolated storage); + * the suite mounts/disposes backend instances on it and cleans it up at the end. + */ +export interface CoordinatorFixture { + /** + * Mount the REAL backend plugin (via `ctx.plugin`, the Loader path) on `ctx`, + * over THIS fixture's shared storage scope. Returns the plugin fiber so the + * suite can dispose a single instance (HMR/reload) while the storage — and any + * still-live session in another fiber — survives. The caller has already + * mounted `SessionStore` on `ctx`. + */ + mount: (ctx: Context) => Promise + + /** + * Inject a NEVER-COMMITTED torn tail into the backend's storage for `id` at + * the given `cwd` (the cwd the session was created with): a half-written + * record past the committed region (JSONL: a partial line with no newline; + * SQLite: a row with invalid `data` JSON past the committed seq). This drives + * the coordinator's `loadCore` `tornMarker !== undefined` → `commitRepair` + * branch against real storage. + * + * OMITTED by a backend that structurally has no torn tails (memory): the + * torn-tail scenario then self-skips (asserted explicitly in the suite). + */ + corruptTail?: (id: SessionId, cwd: string | undefined) => Promise + + /** Tear down the storage scope (remove the temp dir / file). */ + cleanup: () => Promise +} + +/** A constant absolute cwd; jsonl keys directories off it, memory/sqlite ignore it. */ +const WORK = '/w' + +/** The per-session init map a backend exposes for white-box init awaits. */ +function inits(persistence: SessionPersistence): Map> { + return (persistence as unknown as { inits: Map> }).inits +} + +/** Append a whole event log to a live session, event by event (drives session/event). */ +function send(session: Session, events: readonly SessionEvent[]): void { + for (const e of events) session.append(e.type, e.data) +} + +/** A live session created inside its OWN fiber, so it survives a backend reload. */ +async function liveSessionInFiber( + ctx: Context, id: string, cwd: string | undefined, +): Promise { + let session!: Session + await ctx.plugin(Object.assign((inner: Context) => { + session = inner.sessions.create(id, cwd !== undefined ? { meta: { cwd } } : undefined) + }, { inject: ['sessions'] })) + return session +} + +/** + * Run the coordinator orchestration suite against a backend. `makeFixture()` + * MUST return a fresh fixture (isolated storage) each call. + */ +export function runCoordinatorContract(name: string, makeFixture: () => Promise): void { + describe(`PersistenceCoordinator orchestration: ${name}`, () => { + /** Mount SessionStore + a backend instance on a fresh context over the fixture's storage. */ + async function freshCtx(fix: CoordinatorFixture): Promise<{ ctx: Context; fiber: Fiber }> { + const ctx = new Context() + await ctx.plugin(SessionStore) + const fiber = await fix.mount(ctx) + return { ctx, fiber } + } + + // --- write path: live session → flush → reload --- + + it('persists a live session driven through the store, surviving reload', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const session = ctx.sessions.create('live', { meta: { cwd: WORK } }) + send(session, oneTurnLog()) + await ctx.parallel('session/flush', session) + + const loaded = await ctx.sessionPersistence.load(SessionId('live')) + expect(loaded.events).toHaveLength(6) + expect(loaded.meta.cwd).toBe(WORK) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('snapshot-on-buffer: mutating an event after session/event does not corrupt the persisted copy', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const session = ctx.sessions.create('mutate', { meta: { cwd: WORK } }) + const ev = session.append('user/message', { content: [{ type: 'text', text: 'original' }], source: { kind: 'user' } }) + // Mutate the live event object AFTER it was buffered by session/event. + ;(ev.data as { content: { type: 'text'; text: string }[] }).content[0]!.text = 'HACKED' + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', session) + + const loaded = await ctx.sessionPersistence.load(SessionId('mutate')) + const first = loaded.events[0] + expect(first?.type === 'user/message' && (first.data.content[0] as { text: string }).text).toBe('original') + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('append snapshots the batch: mutating the caller array/events after the call is ignored', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const m = meta('snapshot', WORK) + await ctx.sessionPersistence.create(m) + const events = oneTurnLog() // seqs 0..5 + const userMsg = events[1] // the user/message event + const p = ctx.sessionPersistence.append(m.id, events) + // Mutate the caller's array AND an event object after the call but before + // the queued op runs: the snapshot taken at call time must shield the copy. + events.push({ type: 'turn/start', seq: 6, time: 99, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }) + if (userMsg?.type === 'user/message') userMsg.data.content = [{ type: 'text', text: 'MUTATED' }] + await p + const loaded = await ctx.sessionPersistence.load(m.id) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) // not 0..6 + const persisted = JSON.stringify(loaded.events) + expect(persisted).toContain('hi') // original content + expect(persisted).not.toContain('MUTATED') + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + // --- fork / resume --- + + it('fork: a seeded new session persists its seed once (no double-write on a no-op flush)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const seed = oneTurnLog() + // A fork: a brand-new id whose seed came from elsewhere. + const forked = ctx.sessions.create('forked', { seed, meta: { cwd: WORK } }) + await inits(ctx.sessionPersistence).get(forked) // onCreated persisted the seed + const loaded = await ctx.sessionPersistence.load(SessionId('forked')) + expect(loaded.events).toEqual(seed) + // A flush with no NEW events must not double-write. + await ctx.parallel('session/flush', forked) + const reloaded = await ctx.sessionPersistence.load(SessionId('forked')) + expect(reloaded.events).toEqual(seed) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('resume: a re-created session seeded with the loaded log does not re-append its seed and continues the seq', async () => { + const fix = await makeFixture() + const first = await freshCtx(fix) + try { + // First lifecycle: persist a session through the store. + const s1 = first.ctx.sessions.create('resumed', { meta: { cwd: WORK } }) + send(s1, oneTurnLog()) + await first.ctx.parallel('session/flush', s1) + } finally { + await first.fiber.dispose() + } + + // Second lifecycle: a NEW backend instance + a session re-created with the + // same id SEEDED with the loaded events. onCreated adopts the stored log + // (does not re-persist the seed); a new turn appends at seq 6. + const second = await freshCtx(fix) + try { + const loaded = await second.ctx.sessionPersistence.load(SessionId('resumed')) + const s2 = second.ctx.sessions.create('resumed', { seed: loaded.events, meta: { cwd: WORK } }) + await inits(second.ctx.sessionPersistence).get(s2) // let onCreated adopt + s2.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) + s2.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) + await second.ctx.parallel('session/flush', s2) + + const reloaded = await second.ctx.sessionPersistence.load(SessionId('resumed')) + // 6 original + 2 new, contiguous, no duplicated seed. + expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + } finally { + await second.fiber.dispose() + await fix.cleanup() + } + }) + + // --- HMR --- + + it('HMR: applying the plugin seeds existing live sessions', async () => { + const fix = await makeFixture() + const ctx = new Context() + await ctx.plugin(SessionStore) + // A session exists BEFORE the persistence plugin is applied. + const session = ctx.sessions.create('pre-existing', { meta: { cwd: WORK } }) + session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + + const fiber = await fix.mount(ctx) + try { + // The plugin seeded it on apply; a subsequent flush persists its events. + await ctx.parallel('session/flush', session) + const loaded = await ctx.sessionPersistence.load(SessionId('pre-existing')) + expect(loaded.events.length).toBeGreaterThanOrEqual(2) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('HMR: dispose drains remaining buffers', async () => { + const fix = await makeFixture() + const ctx = new Context() + await ctx.plugin(SessionStore) + const fiber = await fix.mount(ctx) + const session = await liveSessionInFiber(ctx, 'drain', WORK) + session.append('user/message', { content: [{ type: 'text', text: 'buffered' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + // No explicit flush — dispose must drain. + await fiber.dispose() + + // A fresh backend instance reads what the disposed one drained. + const second = await freshCtx(fix) + try { + const loaded = await second.ctx.sessionPersistence.load(SessionId('drain')) + expect(loaded.events.length).toBeGreaterThanOrEqual(2) + } finally { + await second.fiber.dispose() + await fix.cleanup() + } + }) + + it('HMR: reloading the backend adopts a still-live, already-materialized session', async () => { + const fix = await makeFixture() + const ctx = new Context() + await ctx.plugin(SessionStore) + // The session lives in its OWN fiber so it survives the backend reload. + const session = await liveSessionInFiber(ctx, 'hmr-adopt', WORK) + try { + // Backend instance 1 materializes the session. + const backend1 = await fix.mount(ctx) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', session) + + // Hot-reload: dispose instance 1, mount instance 2 over the SAME storage + // while the session stays live. Instance 2 has an empty states map but the + // log is materialized and is a prefix of the live events — it must ADOPT + // (not reject). A second turn appended after reload then persists. + await backend1.dispose() + await fix.mount(ctx) + session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('user/message', { content: [{ type: 'text', text: 'again' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) + await expect(ctx.parallel('session/flush', session)).resolves.not.toThrow() + + const loaded = await ctx.sessionPersistence.load(SessionId('hmr-adopt')) + expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2) + } finally { + await ctx.fiber.dispose() + await fix.cleanup() + } + }) + + it('HMR: adoption persists the live SUFFIX that was ahead of the stored prefix', async () => { + const fix = await makeFixture() + const ctx = new Context() + await ctx.plugin(SessionStore) + const session = await liveSessionInFiber(ctx, 'hmr-suffix', WORK) + try { + // Instance 1 flushes turn 1. + const backend1 = await fix.mount(ctx) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', session) + + // Append turn 2 to the LIVE session, then dispose instance 1 WITHOUT + // flushing turn 2: it is now ONLY in the live session's events; the new + // backend never buffered it via session/event. + await backend1.dispose() + session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) + + // Instance 2 adopts the stored prefix (turn 1) and MUST also persist the + // live suffix (turn 2) carried in the session's events. + await fix.mount(ctx) + await ctx.parallel('session/flush', session) + const loaded = await ctx.sessionPersistence.load(SessionId('hmr-suffix')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3]) + expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2) + } finally { + await ctx.fiber.dispose() + await fix.cleanup() + } + }) + + it('HMR adoption does NOT crash-repair an active open turn as interrupted (truncate without closers)', async () => { + const fix = await makeFixture() + const ctx = new Context() + await ctx.plugin(SessionStore) + const session = await liveSessionInFiber(ctx, 'hmr-open', WORK) + try { + const first = await fix.mount(ctx) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('step/start', { turn: 1, step: 1 }) + await ctx.parallel('session/flush', session) + + // Crash-tail a torn fragment past the (open) committed turn, then reload. + await first.dispose() + if (fix.corruptTail) await fix.corruptTail(SessionId('hmr-open'), WORK) + const second = await fix.mount(ctx) + // The live session is still the authority: it appends the REAL step/turn + // end. Adoption must truncate the torn tail but NOT synthesize closers. + session.append('step/end', { turn: 1, step: 1 }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', session) + + const loaded = await ctx.sessionPersistence.load(SessionId('hmr-open')) + expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'step/start', 'step/end', 'turn/end']) + expect(loaded.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } }) + await second.dispose() + } finally { + await ctx.fiber.dispose() + await fix.cleanup() + } + }) + + // --- collision / id reuse --- + + it('a NEW live session colliding on a persisted id is rejected, not silently adopted', async () => { + const fix = await makeFixture() + const first = await freshCtx(fix) + try { + const s1 = first.ctx.sessions.create('collide', { meta: { cwd: WORK } }) + send(s1, oneTurnLog()) + await first.ctx.parallel('session/flush', s1) + } finally { + await first.fiber.dispose() + } + + // A FRESH backend + a NEW live session with the same id but NO explicit + // resume. onCreated treats it as new; create() rejects because a log already + // exists. The rejection surfaces via the init promise (flush awaits it). + const second = await freshCtx(fix) + try { + const s2 = second.ctx.sessions.create('collide', { meta: { cwd: WORK } }) + s2.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + await expect(inits(second.ctx.sessionPersistence).get(s2)) + .rejects.toThrow(/already has a persisted log|id collision/) + } finally { + await second.fiber.dispose() + await fix.cleanup() + } + }) + + it('an abandoned lazy session (never materialized) releases its id for reuse', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // A live session created then disposed BEFORE its first append: cursor 0, + // never materialized. A new live session reusing the id must reclaim it. + let firstSession!: Session + const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { + firstSession = inner.sessions.create('abandoned', { meta: { cwd: WORK } }) + }, { inject: ['sessions'] })) + await inits(ctx.sessionPersistence).get(firstSession) // register the lazy state + await firstFiber.dispose() // disposed before any append → never materialized + + let reuse!: Session + await ctx.plugin(Object.assign((inner: Context) => { + reuse = inner.sessions.create('abandoned', { meta: { cwd: WORK } }) + }, { inject: ['sessions'] })) + await expect(inits(ctx.sessionPersistence).get(reuse)).resolves.toBeUndefined() + reuse.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + reuse.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', reuse) + const loaded = await ctx.sessionPersistence.load(SessionId('abandoned')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1]) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('does NOT reclaim an id whose abandoned owner still has buffered (unflushed) events', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + let first!: Session + const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { + first = inner.sessions.create('buffered', { meta: { cwd: WORK } }) + }, { inject: ['sessions'] })) + await inits(ctx.sessionPersistence).get(first) + // Append a turn but do NOT flush — events sit in the write-behind buffer. + first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + first.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await firstFiber.dispose() // disposed before flush; not materialized, buffer pending + + let reuse!: Session + await ctx.plugin(Object.assign((inner: Context) => { + reuse = inner.sessions.create('buffered', { meta: { cwd: WORK } }) + }, { inject: ['sessions'] })) + await expect(inits(ctx.sessionPersistence).get(reuse)).rejects.toThrow(/already bound to a different live session/) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('initFor is idempotent: re-emitting session/created does not re-initialize', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const session = ctx.sessions.create('idem', { meta: { cwd: WORK } }) + session.append('user/message', { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', session) + // Re-emit session/created for the SAME live session (idempotent initFor). + ctx.emit('session/created', session) + await ctx.parallel('session/flush', session) + const loaded = await ctx.sessionPersistence.load(SessionId('idem')) + expect(loaded.events).toHaveLength(2) // not doubled + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + // --- ownerless-state claim (public create()/load() then a live session arrives) --- + + it('a live session claims cursor-0 ownerless state created via the public API and persists its seed', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // create() registers ownerless state with cursor 0 (lazy, nothing persisted). + await ctx.sessionPersistence.create(meta('lazy-claim', WORK)) + // A live session with that id arrives and claims it (cursor 0 matches + // trivially), persisting its seed. + const live = ctx.sessions.create('lazy-claim', { seed: oneTurnLog(), meta: { cwd: WORK } }) + await expect(inits(ctx.sessionPersistence).get(live)).resolves.toBeUndefined() + const loaded = await ctx.sessionPersistence.load(SessionId('lazy-claim')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('a fresh session reusing a previously-loaded id is rejected (ownerless guard)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // Materialize a log, then load() it WITHOUT a live session — ownerless + // state, cursor at the persisted length. + await ctx.sessionPersistence.create(meta('preview', WORK)) + await ctx.sessionPersistence.append(SessionId('preview'), oneTurnLog()) + await ctx.sessionPersistence.load(SessionId('preview')) + + // A FRESH (empty-seed) live session reusing that id must be rejected: its + // seq 0..cursor-1 events would otherwise be filtered as already-persisted. + let fresh!: Session + await ctx.plugin(Object.assign((inner: Context) => { + fresh = inner.sessions.create('preview', { meta: { cwd: WORK } }) + }, { inject: ['sessions'] })) + await expect(inits(ctx.sessionPersistence).get(fresh)) + .rejects.toThrow(/do not match this live session|already has a persisted log|id collision/) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('a live session whose seed matches the loaded prefix claims ownerless state and persists the suffix', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // Materialize and load (ownerless, cursor = 6). + await ctx.sessionPersistence.create(meta('claim', WORK)) + await ctx.sessionPersistence.append(SessionId('claim'), oneTurnLog()) + const { events } = await ctx.sessionPersistence.load(SessionId('claim')) + + // A live session SEEDED with the loaded log PLUS a new turn claims the + // ownerless state and persists only the suffix. + const cont = ctx.sessions.create('claim', { seed: [ + ...events, + { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, + ], meta: { cwd: WORK } }) + await inits(ctx.sessionPersistence).get(cont) + const loaded = await ctx.sessionPersistence.load(SessionId('claim')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + // --- append adopts a storage-only session (fresh instance, no prior create/load) --- + + it('append adopts a storage-only session (fresh instance) and continues the seq', async () => { + const fix = await makeFixture() + const first = await freshCtx(fix) + try { + const m = meta('adopt-append', WORK) + await first.ctx.sessionPersistence.create(m) + await first.ctx.sessionPersistence.append(m.id, oneTurnLog()) + } finally { + await first.fiber.dispose() + } + + // A fresh instance appends a second turn WITHOUT a prior create/load: append + // must adopt the stored session (cursor = stored length) and continue. + const second = await freshCtx(fix) + try { + await second.ctx.sessionPersistence.append(SessionId('adopt-append'), [ + { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, + ]) + const loaded = await second.ctx.sessionPersistence.load(SessionId('adopt-append')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + } finally { + await second.fiber.dispose() + await fix.cleanup() + } + }) + + // --- small public-API edges that the coordinator owns uniformly --- + + it('append of an empty batch is a no-op (stays lazy)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const m = meta('empty-batch', WORK) + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, []) + expect(await ctx.sessionPersistence.has(m.id)).toBe(false) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('load rejects a missing session', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + await expect(ctx.sessionPersistence.load(SessionId('nope'))).rejects.toThrow(/not found/) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('delete of a non-existent session is a no-op', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + await expect(ctx.sessionPersistence.delete(SessionId('ghost'))).resolves.toBeUndefined() + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('create rejects a duplicate id (in memory and on a persisted log)', async () => { + const fix = await makeFixture() + const first = await freshCtx(fix) + try { + const m = meta('dup', WORK) + await first.ctx.sessionPersistence.create(m) + // Same in-memory state. + await expect(first.ctx.sessionPersistence.create(m)).rejects.toThrow(/already exists in this backend/) + await first.ctx.sessionPersistence.append(m.id, oneTurnLog()) + } finally { + await first.fiber.dispose() + } + + // A fresh instance over the same storage sees the persisted log. + const second = await freshCtx(fix) + try { + await expect(second.ctx.sessionPersistence.create(meta('dup', WORK))) + .rejects.toThrow(/already has a persisted log on disk/) + } finally { + await second.fiber.dispose() + await fix.cleanup() + } + }) + + it('rejects an unknown format version on load (assertVersion)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const m = { version: 2, id: SessionId('v2'), createdAt: 1, cwd: WORK } + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + await expect(ctx.sessionPersistence.load(m.id)).rejects.toThrow(/version/) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('round-trips a header with parentSession (fork lineage)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const m = { version: 1, id: SessionId('forked-child'), createdAt: 1, cwd: WORK, parentSession: SessionId('the-parent') } + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + const loaded = await ctx.sessionPersistence.load(m.id) + expect(loaded.meta.parentSession).toBe('the-parent') + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('flush before init resolves uses cursor 0', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // Append directly to a live session and flush IMMEDIATELY, before the + // async onCreated init has necessarily set state (exercises the + // state-undefined cursor path). + const session = ctx.sessions.create('flush-nostate', { meta: { cwd: WORK } }) + session.append('user/message', { content: [{ type: 'text', text: 'q' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', session) + const loaded = await ctx.sessionPersistence.load(SessionId('flush-nostate')) + expect(loaded.events).toHaveLength(2) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + // --- crash-tail repair THROUGH the coordinator (real storage torn tail) --- + + it('torn-tail load: a never-committed tail is truncated and the open turn closed during load (commitRepair w/ tornMarker)', async () => { + const fix = await makeFixture() + if (!fix.corruptTail) { + // A memory-style store has no torn tails (every write is atomic in RAM), + // so there is no tornMarker path to exercise. Assert that explicitly + // instead of silently skipping, then bail. + expect(fix.corruptTail).toBeUndefined() + await fix.cleanup() + return + } + const first = await freshCtx(fix) + try { + const m = meta('torn', WORK) + await first.ctx.sessionPersistence.create(m) + await first.ctx.sessionPersistence.append(m.id, oneTurnLog()) // committed 0..5 (balanced) + // A second turn whose real events are durable but never closed (open turn). + await first.ctx.sessionPersistence.append(m.id, [ + { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } }, + ]) + } finally { + await first.fiber.dispose() + } + // Inject a torn fragment past the committed region (never-committed tail). + await fix.corruptTail(SessionId('torn'), WORK) + + // A FRESH instance loads: the torn tail is truncated (tornMarker !== + // undefined) AND the open turn 2 is closed with synthetic step/end + + // turn/end {interrupted} — commitRepair runs with BOTH a torn marker and + // closers. The preserved real events (0..7) are never truncated. + const second = await freshCtx(fix) + try { + const loaded = await second.ctx.sessionPersistence.load(SessionId('torn')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) + expect(loaded.events.map(e => e.type)).toEqual([ + 'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1 + 'turn/start', 'step/start', 'step/end', 'turn/end', // turn 2: real + synthetic closers + ]) + const last = loaded.events.at(-1)! + expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' }) + + // The repair is durable: the next append continues at the balanced length + // (seq 10) and a reload round-trips identically. + await second.ctx.sessionPersistence.append(SessionId('torn'), [ + { type: 'turn/start', seq: 10, time: 9, data: { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 11, time: 10, data: { turn: 3, reason: { kind: 'completed' } } }, + ]) + const reloaded = await second.ctx.sessionPersistence.load(SessionId('torn')) + expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]) + } finally { + await second.fiber.dispose() + await fix.cleanup() + } + }) + }) +} diff --git a/packages/session-persistence/tests/persistence.spec.ts b/packages/session-persistence/tests/persistence.spec.ts index 04ac108ed3..8b5a437735 100644 --- a/packages/session-persistence/tests/persistence.spec.ts +++ b/packages/session-persistence/tests/persistence.spec.ts @@ -1,76 +1,132 @@ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' -import { SessionId, isJsonValue, interruptedTurnClosers } from '@deepseek-ai/dsh-session' -import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' -import { SessionPersistence, assertSerializable, seedCoversPrefix } from '../src/index.ts' +import SessionStore, { SessionId, isJsonValue } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' +import { + SessionPersistence, PersistenceCoordinator, assertSerializable, seedCoversPrefix, + type PersistenceBackend, type StoredPrefix, +} from '../src/index.ts' import { runPersistenceContract, meta, oneTurnLog } from './contract.ts' +import { runCoordinatorContract, type CoordinatorFixture } from './coordinator-contract.ts' + +/** The durable store shape: materialized sessions only (no lazy entries). */ +type MemoryStore = Map + +/** Optional plugin config: an EXTERNAL store shared across backend instances. */ +interface MemoryConfig { store?: MemoryStore } /** - * A minimal in-memory {@link SessionPersistence} used to (a) cover the abstract - * base's constructor + service registration and (b) validate the reusable - * contract suite itself. The real durable backend is - * `@deepseek-ai/dsh-session-persistence-jsonl`. + * A trivial in-memory {@link SessionPersistence} that composes a + * {@link PersistenceCoordinator} over a dependency-free `Map`-backed + * {@link PersistenceBackend}. It is BOTH the coordinator's reference vehicle + * (the simplest possible storage — a `Map` with no torn + * tails, so `tornMarker` is always undefined) and the cover for the abstract + * base's constructor + service registration. The real durable backends are + * `@deepseek-ai/dsh-session-persistence-jsonl` / `-sqlite`. + * + * The store can be supplied via config so two backend instances share one Map — + * the in-RAM analogue of two backends over the same file/db, which the + * coordinator orchestration suite's HMR/reload tests need (a fresh instance with + * an empty in-memory states map adopting an already-materialized session). */ -class MemoryPersistence extends SessionPersistence { - private store = new Map() - private pending = new Map() +class MemoryPersistence extends SessionPersistence implements PersistenceBackend { + static inject = ['sessions'] - async create(m: SessionHeader): Promise { - // Lazy: record the intended meta, but stay absent from has/list until the - // first append materializes the session. - this.pending.set(m.id, m) + override readonly name = 'session-persistence-memory' + + /** The whole durable store: materialized sessions only (no lazy entries). */ + private store: MemoryStore + private coordinator: PersistenceCoordinator + + constructor(ctx: Context, config?: MemoryConfig) { + super(ctx) + // Assign the store BEFORE constructing the coordinator: the coordinator's + // constructor installs the write path and synchronously seeds existing live + // sessions (onCreated → loadLive → this.store), so store must exist first. + this.store = config?.store ?? new Map() + this.coordinator = new PersistenceCoordinator(this.ctx, this) } - async append(id: SessionId, events: readonly SessionEvent[]): Promise { - const existing = this.store.get(id) - const nextSeq = existing ? existing.events.length : 0 - if (events.length > 0 && events[0]!.seq !== nextSeq) { - throw new Error(`append seq mismatch for "${id}": expected ${nextSeq}, got ${events[0]!.seq}`) - } - for (let i = 0; i < events.length; i++) { - const e = events[i]! - if (e.seq !== nextSeq + i) throw new Error(`non-contiguous seq in batch for "${id}" at index ${i}`) - if (!isJsonValue(e.data)) { - throw new Error(`event "${e.type}" carries non-JSON-serializable data`) - } + // --- service surface (delegated to the coordinator) --- + + create(m: SessionHeader): Promise { + return this.coordinator.create(m) + } + + append(id: SessionId, events: readonly SessionEvent[]): Promise { + return this.coordinator.append(id, events) + } + + load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { + return this.coordinator.load(id) + } + + has(id: SessionId): Promise { + return this.coordinator.has(id) + } + + delete(id: SessionId): Promise { + return this.coordinator.delete(id) + } + + /** White-box accessor: await a specific session's onCreated init. */ + get inits(): Map> { + return this.coordinator.inits + } + + // --- PersistenceBackend hooks (the Map storage primitives) --- + + // A Map-backed store has no torn tails, so `tornMarker` is never set. Ids are + // globally unique, so loadStored and loadLive are identical (cwd is ignored). + async loadStored(id: SessionId): Promise | undefined> { + const entry = this.store.get(id) + if (!entry) return undefined + return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) } + } + + loadLive(id: SessionId, _cwd: string | undefined): Promise | undefined> { + return this.loadStored(id) + } + + async appendBatch(m: SessionHeader, events: readonly SessionEvent[], _isMaterialized: boolean): Promise { + // Defense-in-depth: the coordinator already validates serializability, but a + // durable store must reject non-JSON data at its own boundary too. + for (const e of events) { + if (!isJsonValue(e.data)) throw new Error(`event "${e.type}" carries non-JSON-serializable data`) } + const existing = this.store.get(m.id) if (!existing) { - const m = this.pending.get(id) - if (!m) throw new Error(`append before create for "${id}"`) - this.store.set(id, { meta: m, events: structuredClone(events) as SessionEvent[] }) + // First batch: `_isMaterialized` is false (the coordinator only omits + // materialization on the first batch); writing the entry IS the materialization. + this.store.set(m.id, { meta: structuredClone(m), events: structuredClone(events) as SessionEvent[] }) } else { existing.events.push(...structuredClone(events) as SessionEvent[]) } } - async load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { - const entry = this.store.get(id) - if (!entry) throw new Error(`session "${id}" not found`) - // Honor the crash-recovery contract: if the stored log ends mid-turn, close - // the orphaned turn durably with synthetic boundary events and continue from - // the balanced length. - const closers = interruptedTurnClosers(entry.events) - if (closers.length > 0) entry.events.push(...structuredClone(closers)) - return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) } + async commitRepair(m: SessionHeader, _tornMarker: undefined, closers: readonly SessionEvent[]): Promise { + // No torn tails in a Map store, so `_tornMarker` is always undefined; only the + // synthetic closers are appended (the same DELETE+INSERT a DB backend does, + // minus the truncate). + const entry = this.store.get(m.id) + /* v8 ignore next -- commitRepair only runs for a materialized (stored) session */ + if (!entry) return + if (closers.length > 0) entry.events.push(...structuredClone(closers) as SessionEvent[]) + } + + async deleteStored(id: SessionId): Promise { + this.store.delete(id) } async list(): Promise { return [...this.store.values()].map(e => structuredClone(e.meta)) } - - async has(id: SessionId): Promise { - return this.store.has(id) - } - - async delete(id: SessionId): Promise { - this.store.delete(id) - this.pending.delete(id) - } } // Run the shared contract against the in-memory backend. runPersistenceContract('memory', async () => { const ctx = new Context() + await ctx.plugin(SessionStore) const fiber = await ctx.plugin(MemoryPersistence) return { persistence: ctx.sessionPersistence, @@ -78,9 +134,24 @@ runPersistenceContract('memory', async () => { } }) +// Run the shared coordinator orchestration suite against the in-memory backend. +// A per-fixture Map is the shared "storage", so two mounted instances see the +// same materialized sessions (HMR/reload). `corruptTail` is OMITTED: a Map store +// writes atomically in RAM and has no torn tails, so the suite's torn-tail test +// self-skips (and asserts the omission). The real torn-tail repair branch is +// covered by the jsonl/sqlite fixtures, which CAN inject one. +runCoordinatorContract('memory', async (): Promise => { + const store: MemoryStore = new Map() + return { + mount: async ctx => ctx.plugin(MemoryPersistence, { store }), + cleanup: async () => { store.clear() }, + } +}) + describe('SessionPersistence service registration', () => { it('registers as ctx.sessionPersistence and is removed on fiber dispose (HMR safety)', async () => { const ctx = new Context() + await ctx.plugin(SessionStore) const fiber = await ctx.plugin(MemoryPersistence) expect(ctx.sessionPersistence).toBeInstanceOf(SessionPersistence) @@ -90,6 +161,7 @@ describe('SessionPersistence service registration', () => { it('round-trips through the registered service instance', async () => { const ctx = new Context() + await ctx.plugin(SessionStore) const fiber = await ctx.plugin(MemoryPersistence) const m = meta('reg') await ctx.sessionPersistence.create(m)