mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
refactor(session-persistence): extract a shared write coordinator
The JSONL and SQLite backends were byte-identical (or same-algorithm) for ALL of their write-path orchestration — the four maps (states/buffers/chains/inits), installWritePath, initFor, onCreated's four adoption cases, flush, drain, serialize, adopt/adoptLivePrefix, assertVersion, and the create/append/load/ has/delete skeletons. Only the storage primitives (write bytes vs INSERT rows) differed, so every fix landed twice. Extract that orchestration into a PersistenceCoordinator in the seam package. Each backend composes one (new PersistenceCoordinator(ctx, this)), implements a small PersistenceBackend hook interface (loadStored, loadLive, appendBatch, commitRepair, deleteStored, list, optional close), and delegates its six public service methods to it. Composition, not inheritance — a backend exposes only the hooks, can't reach the coordinator's private state, and the public SessionPersistence API is unchanged so a third-party backend may still implement it directly. The crash-repair torn-tail token is OPAQUE: the coordinator computes the synthetic closers (it owns interruptedTurnClosers) but only tests `tornMarker !== undefined` and round-trips it to commitRepair, never inspecting it (JSONL = byte offset, SQLite = seq). loadStored vs loadLive stay distinct so HMR adoption is cwd-scoped (a same-id log at a different cwd is a collision, not a resume). appendBatch carries meta so lazy-materialize + first-batch commit atomically (no separate materialize hook). Tests: the duplicated orchestration tests (adoption, HMR, collision, dispose-drain, crash-tail) move into one runCoordinatorContract suite run once per backend (memory + jsonl + sqlite) via hook fixtures; per-backend specs keep only storage mechanics. A through-coordinator torn-tail test per real backend keeps the commitRepair-with-marker branch covered under the 100% gate. Net -112 lines (the dedup outweighs the new coordinator + shared suite); 100% coverage; backends shrank ~1200 lines of duplicated churn. Migrates the write-coordinator RFC proposed -> implemented.
This commit is contained in:
Tianyi Cui
@@ -32,7 +32,6 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
|
||||
| [Optional Code Mode — model writes TypeScript against an SDK of all tools](proposed/2026-06-15-optional-code-mode.md) | 2026-06-15 |
|
||||
| [Runtime schemas for the event vocabulary (Zod vs the merge-extensible-map pattern)](proposed/2026-06-16-typed-event-schemas.md) | 2026-06-16 |
|
||||
| [Agent lifecycle and ownership seams](proposed/2026-06-18-agent-lifecycle-and-ownership-seams.md) | 2026-06-18 |
|
||||
| [Shared persistence write coordinator](proposed/2026-06-18-shared-persistence-write-coordinator.md) | 2026-06-18 |
|
||||
|
||||
## Implemented
|
||||
|
||||
@@ -61,6 +60,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
|
||||
| [ACP snapshot tests — record-once / replay-deterministic](implemented/2026-06-19-acp-snapshot-tests.md) | 2026-06-19 |
|
||||
| [Real-API e2e in CI against the external DeepSeek API](implemented/2026-06-19-real-api-e2e-ci.md) | 2026-06-19 |
|
||||
| [Drop the mutable session summary](implemented/2026-06-19-drop-mutable-session-summary.md) | 2026-06-19 |
|
||||
| [Shared persistence write coordinator](implemented/2026-06-18-shared-persistence-write-coordinator.md) | 2026-06-18 |
|
||||
|
||||
## Rejected
|
||||
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
# RFC: Shared persistence write coordinator
|
||||
|
||||
Status: implemented (proposed and accepted 2026-06-18, implemented 2026-06-20)
|
||||
|
||||
## Problem
|
||||
|
||||
`dsh-session-persistence-jsonl` and `dsh-session-persistence-sqlite` intentionally prove the same `SessionPersistence` contract over different storage media, but their write-path orchestration was duplicated: per-session state, `session/created` adoption, backend-specific prefix reads, write-behind buffers, serialized flush chains, HMR seeding, and dispose drains. The pure seed-prefix collision and serializability guards had already moved into the seam package; the remaining orchestration was still correctness-heavy and received the same fixes twice. A code-level diff showed the two backends were byte-identical — or same-algorithm — for ALL of it: the four maps (`states`/`buffers`/`chains`/`inits`), `installWritePath`, `initFor`, `onCreated`'s four cases, `flush`, `drain`, `serialize`, `adopt`, `adoptLivePrefix`, `assertVersion`, and the `create`/`append`/`load`/`has`/`delete` skeletons. Only the storage primitives (write bytes vs. INSERT rows) differed.
|
||||
|
||||
## Decision
|
||||
|
||||
Extract a backend-agnostic `PersistenceCoordinator` into `dsh-session-persistence`. The coordinator owns the orchestration once; each first-party backend composes one (`new PersistenceCoordinator(ctx, this)`), implements a small `PersistenceBackend` hook interface, and delegates its six public service methods (`create`/`append`/`load`/`list`/`has`/`delete`) to it.
|
||||
|
||||
Composition, not inheritance. The coordinator is a concrete class the backend holds, not a base class the backend extends. The RFC's risk — "a coordinator must not make unusual backends fight an inheritance hierarchy" — is avoided: a backend exposes only the hooks; it cannot reach the coordinator's private orchestration state, and the public `SessionPersistence` service shape is unchanged, so a third-party backend MAY still implement the abstract service directly without the coordinator at all.
|
||||
|
||||
### The hook interface (`PersistenceBackend<TornMarker>`)
|
||||
|
||||
Seven methods (six required + an optional lifecycle hook) — the only seam between the coordinator and storage:
|
||||
|
||||
- `name` — backend label for the dispose-failure `AggregateError`.
|
||||
- `loadStored(id)` — read a stored prefix by id, scanning ANY storage scope (every JSONL cwd bucket; SQLite's id is globally unique). Used by resume/load and, via `!== undefined`, the create-collision probe and `has`.
|
||||
- `loadLive(id, cwd)` — read a stored prefix SCOPED to `cwd`. **Deliberately distinct from `loadStored`**: HMR live-adoption must only adopt a persisted log at the SAME cwd as the live session; a same-id log at a different cwd is a collision, not a resume. Collapsing the two reintroduces a cross-cwd adoption bug. SQLite ignores `cwd`.
|
||||
- `appendBatch(meta, events, isMaterialized)` — durably append a contiguous batch, lazily materializing the session ATOMICALLY when not yet materialized (the materialize-write and the first event batch must commit together — a crash between them must not leave a materialized-but-empty session; this is why there is no separate `materialize` hook).
|
||||
- `commitRepair(meta, tornMarker, closers)` — make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined`) and append `closers`. **NOT required to be atomic** — JSONL legitimately truncates-then-appends in two fsync'd steps, SQLite does DELETE+INSERT in one transaction. Used by `load` (truncate + synthetic closers) and live-adoption (truncate only, `closers = []`).
|
||||
- `deleteStored(id)` / `list()` — remove a stored artifact / list all stored metadata.
|
||||
- `close?()` — optional lifecycle teardown (SQLite closes its db handle; JSONL omits it), awaited in the dispose effect AFTER the quiescence drain so a close failure never masks a drain error.
|
||||
|
||||
### The opaque torn marker
|
||||
|
||||
The single design choice that keeps the seam clean: the crash-repair "where is the torn tail" token is OPAQUE to the coordinator. The coordinator computes the synthetic closers (it owns `interruptedTurnClosers` from `dsh-session`), but it only ever tests `tornMarker !== undefined` and passes the value straight back to `commitRepair` — it never inspects it. Each backend picks its own marker type: JSONL uses the byte offset to truncate to, SQLite the seq to delete from (both happen to be `number`). The JSONL backend folds its `committedBytes < buffer.byteLength` comparison INSIDE the hook so the returned marker is already `number | undefined`; without that fold the coordinator would have to know about byte lengths.
|
||||
|
||||
## Testing
|
||||
|
||||
The shared `runPersistenceContract` (public-API contract) keeps running for every backend. A new `runCoordinatorContract` (`tests/coordinator-contract.ts`) holds the write-path orchestration — adoption, HMR, collision, dispose-drain, crash-tail repair — and runs once per backend through a `CoordinatorFixture` (an in-memory reference + jsonl + sqlite). The per-backend specs shrank to storage mechanics only (JSONL: path safety, fsync rollback, bucket listing; SQLite: schema version, `scanRows`, transaction rollback). A through-coordinator torn-tail→load→`commitRepair` test per real backend (via a `corruptTail` fixture hook) keeps the coordinator's torn-marker repair branch covered under the 100% per-file gate — the contract crash test only produces synthetic closers, never a torn marker, so it could not reach that branch.
|
||||
|
||||
## Risks and what we gave up
|
||||
|
||||
The pre-extraction duplication was verbose but explicit — each backend read top-to-bottom. The coordinator adds one indirection (the hook seam) and one new concept (the opaque torn marker). This clears the bar because the centralized logic is the correctness-heavy part that was already being fixed twice, and the hook set is narrow (seven methods, no inheritance). The hook surface was deliberately held to the minimum: `has` and the create-collision probe are NOT separate hooks — they fold into `loadStored(id) !== undefined`; there is no separate `materialize` hook (folded into `appendBatch` for atomicity); `list()` stays a backend method with no coordinator pass-through (listing needs none of the orchestration). The net effect is a reduction: one orchestration copy instead of two, the backends shrank by ~1200 lines of duplicated churn, and a future backend implements ~7 small primitives instead of copying the entire `session/event` → buffer → flush machinery.
|
||||
@@ -20,7 +20,7 @@ Delete the mutable session summary entirely. `SessionSummary` and the `SessionMe
|
||||
|
||||
Anything the summary was meant to provide is **derivable from the append-only log** when a consumer actually needs it (`firstPrompt` = first `user/message`; recency = the last event's `time` or the file mtime) or already lives in the immutable header (`createdAt`, `cwd`). The one thing *not* derivable — a user-*edited* title — had no implementation and is pure YAGNI; it can return as its own log event or header field if a real feature ever needs it.
|
||||
|
||||
This is recorded as a decision because it is **durable** (it narrows a public service contract and an on-disk format across two backends), **contested** (the summary was a deliberate forward-looking design, not an accident), and **surprising** (a future reader finding `SessionHeader` where the original RFC describes `SessionMeta` would otherwise ask why the summary vanished). It also unblocks the [shared persistence write coordinator](../proposed/2026-06-18-shared-persistence-write-coordinator.md): with no mutable summary, the coordinator's hook interface needs no `updateSummary` hook and the JSONL-sidecar-vs-SQLite-column durability divergence disappears, so the two backends' write paths converge.
|
||||
This is recorded as a decision because it is **durable** (it narrows a public service contract and an on-disk format across two backends), **contested** (the summary was a deliberate forward-looking design, not an accident), and **surprising** (a future reader finding `SessionHeader` where the original RFC describes `SessionMeta` would otherwise ask why the summary vanished). It also unblocks the [shared persistence write coordinator](2026-06-18-shared-persistence-write-coordinator.md): with no mutable summary, the coordinator's hook interface needs no `updateSummary` hook and the JSONL-sidecar-vs-SQLite-column durability divergence disappears, so the two backends' write paths converge.
|
||||
|
||||
## No migration
|
||||
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
# RFC: Shared persistence write coordinator
|
||||
|
||||
Status: proposed
|
||||
|
||||
## Problem
|
||||
|
||||
`dsh-session-persistence-jsonl` and `dsh-session-persistence-sqlite` intentionally prove the same `SessionPersistence` contract over different storage media, but their write-path orchestration is now duplicated: per-session state, `session/created` adoption, backend-specific prefix reads, write-behind buffers, serialized flush chains, HMR seeding, and dispose drains. The pure seed-prefix collision and serializability guards have already moved into the seam package; the remaining orchestration is still correctness-heavy and already receives the same fixes twice.
|
||||
|
||||
## Proposal
|
||||
|
||||
Extract a backend-agnostic coordinator into `dsh-session-persistence`. The coordinator owns live-session adoption, buffering, cursor filtering, per-id serialization, and disposal quiescence. Concrete backends provide small hooks for durable operations: create lazy state, find/load stored prefix, append a contiguous batch, delete, and list.
|
||||
|
||||
The public `SessionPersistence` service shape can stay the same. The coordinator can be an internal exported helper or protected base class used by first-party backends; third-party backends may still implement the abstract service directly if their write path is different.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- JSONL and SQLite keep passing the existing shared `runPersistenceContract`.
|
||||
- HMR/adoption/collision tests move to a shared coordinator test suite and run once for each backend through hook-driven fixtures.
|
||||
- Backend-specific tests focus on storage mechanics only: JSONL path safety/fsync behavior and SQLite schema/WAL/transaction behavior.
|
||||
- A future backend does not need to copy the current `session/event` → buffer → flush orchestration.
|
||||
|
||||
## Risks
|
||||
|
||||
The current duplication is verbose but explicit. A coordinator must not hide storage-specific durability semantics or make unusual backends fight an inheritance hierarchy. Prefer narrow hooks and contract tests over a large framework.
|
||||
@@ -1,20 +1,18 @@
|
||||
/**
|
||||
* JSONL durable session-persistence backend (`@deepseek-ai/dsh-session-persistence-jsonl`).
|
||||
*
|
||||
* Two concerns in one plugin:
|
||||
* One append-only `.jsonl` event log per session (a header line then one
|
||||
* `SessionEvent` per line, verbatim including `assistant/chunk` so `seq` stays
|
||||
* contiguous), with lazy materialization (no file until the first `append`),
|
||||
* atomic first write, and load-time repair of a never-committed crash tail.
|
||||
*
|
||||
* 1. **The backend** — a concrete {@link SessionPersistence}: one append-only
|
||||
* `.jsonl` event log per session (a header line then one `SessionEvent` per
|
||||
* line, verbatim including `assistant/chunk` so `seq` stays contiguous).
|
||||
* Lazy materialization (no file until the first `append`), atomic first
|
||||
* write, and load-time repair of a never-committed crash tail.
|
||||
*
|
||||
* 2. **The write path** — the `session/event` → buffer → `session/flush` drain
|
||||
* that generalizes the example `session-jsonl.ts`: snapshot each event when
|
||||
* it is buffered (the live `session.events` object is mutable), persist
|
||||
* forks once on `session/created`, maintain a per-session write cursor so a
|
||||
* resumed session never re-appends already-stored events, and seed existing
|
||||
* live sessions on plugin apply (HMR does not replay `session/created`).
|
||||
* The backend supplies ONLY the file-bytes storage primitives (the
|
||||
* {@link PersistenceBackend} hooks below); all the write-path orchestration
|
||||
* (the `session/event` → buffer → `session/flush` drain, per-session
|
||||
* serialization, write cursors, fork-seed persistence, HMR live-adoption,
|
||||
* crash-repair sequencing, dispose quiescence) lives in the backend-agnostic
|
||||
* {@link PersistenceCoordinator} this class composes. The six public
|
||||
* {@link SessionPersistence} methods delegate to the coordinator.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-session-persistence-jsonl
|
||||
*/
|
||||
@@ -25,9 +23,9 @@ import { open, mkdir, readFile, readdir, link, rm, truncate } from 'node:fs/prom
|
||||
import { dirname, resolve } from 'node:path'
|
||||
import { randomBytes } from 'node:crypto'
|
||||
import {
|
||||
SessionPersistence, assertSerializable, seedCoversPrefix,
|
||||
SessionPersistence, PersistenceCoordinator,
|
||||
type PersistenceBackend, type StoredPrefix,
|
||||
} from '@deepseek-ai/dsh-session-persistence'
|
||||
import { interruptedTurnClosers } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
encodeSegment, eventLine, logPath, parseHeaderMeta, scanLog, sessionDir, toHeaderLine,
|
||||
@@ -42,267 +40,152 @@ export interface Config {
|
||||
root: string
|
||||
}
|
||||
|
||||
/** Per-session write state held by the backend's in-memory bookkeeping. */
|
||||
interface SessionState {
|
||||
meta: SessionHeader
|
||||
/** The next seq the backend expects to append (the stored log length). */
|
||||
cursor: number
|
||||
/** Whether the `.jsonl` file has been physically materialized. */
|
||||
materialized: boolean
|
||||
/**
|
||||
* The live Session this state was bound to via `onCreated`, if any. Used to
|
||||
* detect a DIFFERENT live session reusing a tracked id (a collision): state
|
||||
* created through the public `create()`/`load()` API has no owner, but state
|
||||
* bound to a live session lets `onCreated` reject a second, unrelated session
|
||||
* object on the same id instead of silently no-opping (which would leave the
|
||||
* new session's events to be dropped against the old cursor).
|
||||
*/
|
||||
owner?: Session
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether `error` is a "no such file/directory" (`ENOENT`) failure — the ONLY
|
||||
* filesystem error that legitimately means "this session/root is absent" for a
|
||||
* durable backend. Any OTHER error (`EACCES`, `ENOTDIR`, transient I/O) must
|
||||
* surface rather than be silently reported as absence: masking it would let
|
||||
* `list()` report no sessions, `load()` report "not found", and collision
|
||||
* checks proceed under a false absence assumption — all unsafe for durable
|
||||
* persistence. (A NodeJS filesystem rejection carries a string `code`.)
|
||||
* surface rather than be silently reported as absence. (A NodeJS filesystem
|
||||
* rejection carries a string `code`.)
|
||||
*/
|
||||
function isENOENT(error: unknown): boolean {
|
||||
return (error as NodeJS.ErrnoException | null)?.code === 'ENOENT'
|
||||
}
|
||||
|
||||
async function settledErrors(promises: Iterable<Promise<unknown>>): Promise<unknown[]> {
|
||||
const settled = await Promise.allSettled([...promises])
|
||||
const errors: unknown[] = []
|
||||
for (const result of settled) {
|
||||
if (result.status === 'rejected') errors.push(result.reason)
|
||||
}
|
||||
return errors
|
||||
}
|
||||
|
||||
/**
|
||||
* The JSONL persistence backend. Load as a plugin; it registers as
|
||||
* `ctx.sessionPersistence` and installs the write-path listeners.
|
||||
* `ctx.sessionPersistence` and (via the coordinator) installs the write-path
|
||||
* listeners. Its torn-tail marker is the byte offset to truncate the log to.
|
||||
*/
|
||||
export class SessionPersistenceJsonl extends SessionPersistence {
|
||||
export class SessionPersistenceJsonl extends SessionPersistence implements PersistenceBackend<number> {
|
||||
static inject = ['sessions']
|
||||
|
||||
static Config: z<Config> = z.object({
|
||||
root: z.string().required(),
|
||||
})
|
||||
|
||||
/**
|
||||
* Backend label for the coordinator's dispose-failure AggregateError and
|
||||
* effect name. NOTE: this intentionally shadows cordis `Service.name` (which
|
||||
* the base sets to `'sessionPersistence'`). The service is registered under the
|
||||
* fixed key the Service constructor captured (`reflect.provide('sessionPersistence', …)`),
|
||||
* not via `this.name`, so overwriting the instance field with the backend label
|
||||
* does not affect `ctx.sessionPersistence` resolution — it only relabels the
|
||||
* dispose diagnostics, which is exactly what {@link PersistenceBackend.name} is for.
|
||||
*/
|
||||
override readonly name = 'session-persistence-jsonl'
|
||||
|
||||
private root: string
|
||||
/** Backend bookkeeping keyed by session id (NOT the live Session object). */
|
||||
private states = new Map<string, SessionState>()
|
||||
/** Write-behind buffers keyed by the live Session (write path). */
|
||||
private buffers = new Map<Session, SessionEvent[]>()
|
||||
/**
|
||||
* Per-session serialization: every backend operation chains onto the prior
|
||||
* one for the same id, so concurrent flushes / a flush racing onCreated never
|
||||
* interleave file writes or read a half-built state. Keyed by session id.
|
||||
*/
|
||||
private chains = new Map<string, Promise<unknown>>()
|
||||
/**
|
||||
* Per-session init promise (onCreated). Keyed by the LIVE Session OBJECT, not
|
||||
* its id: a disposed fiber's session can be replaced by a different live
|
||||
* Session reusing the same id (HMR, an ACP reconnect), and an id-keyed cache
|
||||
* would hand the new object the old object's init promise — skipping
|
||||
* onCreated for the new session, so its events start at seq 0 while flush
|
||||
* filters against the stale cursor and silently drops them. Keying by object
|
||||
* gives each live Session its own init. flush awaits it before appending.
|
||||
*/
|
||||
private inits = new Map<Session, Promise<void>>()
|
||||
private coordinator: PersistenceCoordinator<number>
|
||||
|
||||
constructor(ctx: Context, public config: Config) {
|
||||
super(ctx)
|
||||
// Resolve the configured root to an ABSOLUTE path ONCE, here. A relative
|
||||
// root (the examples use `./.sessions`) would otherwise re-resolve against
|
||||
// `process.cwd()` at every later readdir/open — so if any plugin or test
|
||||
// changed cwd between create, append, and load, one session's files could
|
||||
// split across directories. Pinning it at construction makes all paths
|
||||
// stable regardless of later cwd changes.
|
||||
// Resolve the configured root to an ABSOLUTE path ONCE, here. A relative root
|
||||
// would otherwise re-resolve against `process.cwd()` at every later
|
||||
// readdir/open — so if any plugin or test changed cwd between create, append,
|
||||
// and load, one session's files could split across directories.
|
||||
this.root = resolve(config.root)
|
||||
this.installWritePath()
|
||||
this.coordinator = new PersistenceCoordinator<number>(this.ctx, this)
|
||||
}
|
||||
|
||||
// --- SessionPersistence backend surface (all serialized per session id) ---
|
||||
// --- SessionPersistence service surface (delegated to the coordinator) ---
|
||||
|
||||
create(meta: SessionHeader): Promise<void> {
|
||||
// Snapshot the metadata at call time: the op runs later (behind the
|
||||
// per-session chain) and the snapshot is also stored as the lazy state, so
|
||||
// keeping the caller's object by reference would let a later mutation of
|
||||
// `id`/`cwd` register under one key but materialize under a different
|
||||
// path/header. A shallow copy is enough — SessionHeader is a flat record.
|
||||
const snapshot: SessionHeader = { ...meta }
|
||||
return this.serialize(snapshot.id, () => this.createCore(snapshot))
|
||||
return this.coordinator.create(meta)
|
||||
}
|
||||
|
||||
private async createCore(meta: SessionHeader): Promise<void> {
|
||||
// Do NOT clobber an existing session. If we already track it, or a log
|
||||
// exists on disk under this id, refuse — the SessionId IS the identity, and
|
||||
// silently resetting state (cursor 0, materialized false) over committed
|
||||
// data would let the next append rename over the existing log.
|
||||
if (this.states.has(meta.id)) {
|
||||
throw new Error(`session "${meta.id}" already exists in this backend`)
|
||||
}
|
||||
// Scan ALL cwd buckets (pass undefined), not just meta.cwd's: load/has/adopt
|
||||
// identify a session by id alone and search every bucket, so an id already
|
||||
// persisted under a DIFFERENT cwd must still block creation here. Probing
|
||||
// only meta.cwd's bucket would let two logs share one id and make resume
|
||||
// (which picks the first matching bucket) nondeterministic.
|
||||
if (await this.findLog(meta.id, undefined) !== undefined) {
|
||||
throw new Error(`session "${meta.id}" already has a persisted log on disk; load/resume it instead of creating`)
|
||||
}
|
||||
// Pure lazy: record intent only. No file until the first append, so an
|
||||
// abandoned (never-appended) session leaves nothing on disk and stays
|
||||
// absent from has()/list().
|
||||
this.states.set(meta.id, { meta, cursor: 0, materialized: false })
|
||||
}
|
||||
|
||||
/**
|
||||
* Run `op` after any in-flight operation for the same session id, so writes
|
||||
* for one session never interleave (two flushes, a flush racing a load, an
|
||||
* update racing an append). Errors do not poison the chain — the next op
|
||||
* still runs. NOTE: serialized public methods must NOT call each other (that
|
||||
* would deadlock on the same chain); they call the unserialized `*Core`
|
||||
* helpers instead.
|
||||
*/
|
||||
private serialize<T>(id: SessionId, op: () => Promise<T>): Promise<T> {
|
||||
const prior = this.chains.get(id) ?? Promise.resolve()
|
||||
const next = prior.then(op, op)
|
||||
// Keep the chain alive but swallow this op's rejection for the NEXT waiter
|
||||
// (the caller still sees the real rejection via `next`).
|
||||
this.chains.set(id, next.then(() => undefined, () => undefined))
|
||||
return next
|
||||
}
|
||||
|
||||
// `async` so the synchronous validate/clone below reject (not throw) per the
|
||||
// Promise<void> contract — callers use `await expect(...).rejects`.
|
||||
async append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
// Validate serializability BEFORE cloning, so a bad event surfaces the typed
|
||||
// "non-JSON-serializable" error rather than an opaque DataCloneError from
|
||||
// structuredClone below. (In an async method this throw becomes a rejection,
|
||||
// honoring the Promise<void> contract rather than throwing synchronously.)
|
||||
assertSerializable(events)
|
||||
// Deep-snapshot the batch here, BEFORE the op waits behind the per-session
|
||||
// chain: the op may await before serializing, so a caller that passes a live
|
||||
// array (e.g. session.events) and mutates it — OR mutates an event object
|
||||
// inside it — before the op runs would otherwise have those changes
|
||||
// persisted, or advance the cursor past what was actually written.
|
||||
// structuredClone covers both the array and the event objects (safe now that
|
||||
// serializability is checked above). The clone happens synchronously (before
|
||||
// the first await), so it is taken at call time.
|
||||
const batch = events.map(e => structuredClone(e))
|
||||
return this.serialize(id, () => this.appendCore(id, batch))
|
||||
}
|
||||
|
||||
private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
if (events.length === 0) return
|
||||
assertSerializable(events)
|
||||
let state = this.states.get(id)
|
||||
if (state === undefined) state = await this.adopt(id) // calls loadCore, not load
|
||||
|
||||
// Contiguity contract: each event's seq must continue the stored log.
|
||||
for (const [i, event] of events.entries()) {
|
||||
if (event.seq !== state.cursor + i) {
|
||||
throw new Error(`append seq mismatch for "${id}": expected ${state.cursor + i} at index ${i}, got ${event.seq}`)
|
||||
}
|
||||
}
|
||||
|
||||
if (!state.materialized) {
|
||||
await this.materialize(state, events)
|
||||
} else {
|
||||
await this.appendLines(state, events)
|
||||
}
|
||||
// The durable event log is the transaction: advance the cursor as soon as
|
||||
// the log write commits.
|
||||
state.cursor += events.length
|
||||
append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
return this.coordinator.append(id, events)
|
||||
}
|
||||
|
||||
load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
return this.serialize(id, () => this.loadCore(id))
|
||||
return this.coordinator.load(id)
|
||||
}
|
||||
|
||||
private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
const cwd = this.states.get(id)?.meta.cwd
|
||||
has(id: SessionId): Promise<boolean> {
|
||||
return this.coordinator.has(id)
|
||||
}
|
||||
|
||||
delete(id: SessionId): Promise<void> {
|
||||
return this.coordinator.delete(id)
|
||||
}
|
||||
|
||||
// `list` is BOTH the public service method and the PersistenceBackend hook —
|
||||
// one method, the bucket walk below. The coordinator adds no orchestration for
|
||||
// listing (no per-id serialization, no cursor), so it would just call back into
|
||||
// this same method; routing it through the coordinator would recurse. Defined
|
||||
// once, in the "PersistenceBackend hooks" section.
|
||||
|
||||
/**
|
||||
* The per-session init promises, exposed for white-box tests that await a
|
||||
* specific session's onCreated (there is no public API to await one init).
|
||||
*/
|
||||
get inits(): Map<Session, Promise<void>> {
|
||||
return this.coordinator.inits
|
||||
}
|
||||
|
||||
// --- PersistenceBackend hooks (the file-bytes storage primitives) ---
|
||||
|
||||
/** Read a stored prefix by id across ALL cwd buckets (cwd unknown). */
|
||||
async loadStored(id: SessionId): Promise<StoredPrefix<number> | undefined> {
|
||||
return this.readPrefix(id, undefined)
|
||||
}
|
||||
|
||||
/** Read a stored prefix SCOPED to `cwd` (HMR live-adoption must not cross cwd). */
|
||||
async loadLive(id: SessionId, cwd: string | undefined): Promise<StoredPrefix<number> | undefined> {
|
||||
return this.readPrefix(id, cwd)
|
||||
}
|
||||
|
||||
/**
|
||||
* Read and scan a session's log into a {@link StoredPrefix}. Folds the
|
||||
* torn-tail comparison HERE so the `tornMarker` is the byte offset to truncate
|
||||
* to (or `undefined` when nothing is torn) — the coordinator never sees the
|
||||
* raw byteLength.
|
||||
*/
|
||||
private async readPrefix(id: SessionId, cwd: string | undefined): Promise<StoredPrefix<number> | undefined> {
|
||||
const file = await this.findLog(id, cwd)
|
||||
if (file === undefined) throw new Error(`session "${id}" not found`)
|
||||
if (file === undefined) return undefined
|
||||
const buffer = await readFile(file.path)
|
||||
const { meta, events, committedBytes } = scanLog(buffer)
|
||||
this.assertVersion(meta)
|
||||
|
||||
// Crash-recovery: if the log ended mid-turn (an open turn with real,
|
||||
// preserved events but no closing turn/end), close it durably DURING load so
|
||||
// disk, the returned log, and the cursor all agree — both append routes then
|
||||
// continue with no special-casing. Synthesize the boundary events (a
|
||||
// step/end if a step was open, then a turn/end {kind:'interrupted'}); the
|
||||
// interrupted turn's real events are preserved, never truncated (a turn can
|
||||
// be huge — the session-persistence RFC).
|
||||
const closers = interruptedTurnClosers(events)
|
||||
const balanced = [...events, ...closers]
|
||||
|
||||
// Set state BEFORE the repair writes so they can resolve the log path.
|
||||
const needsTorn = committedBytes < buffer.byteLength
|
||||
const state: SessionState = {
|
||||
meta: { ...meta },
|
||||
cursor: events.length,
|
||||
materialized: true,
|
||||
return {
|
||||
meta,
|
||||
events,
|
||||
...committedBytes < buffer.byteLength ? { tornMarker: committedBytes } : {},
|
||||
}
|
||||
this.states.set(id, state)
|
||||
|
||||
if (needsTorn) {
|
||||
// Discard the torn trailing fragment (a final line never fully flushed)
|
||||
// before writing the closers, so the closers land at a clean EOF.
|
||||
await this.repair(state, committedBytes)
|
||||
}
|
||||
if (closers.length > 0) {
|
||||
// Durably append the synthetic closers, then advance the cursor to the
|
||||
// balanced length. After this, disk == balanced and the next append (live
|
||||
// or direct) continues cleanly.
|
||||
await this.appendLines(state, closers)
|
||||
state.cursor = balanced.length
|
||||
}
|
||||
|
||||
return { meta, events: balanced }
|
||||
}
|
||||
|
||||
private async adoptLiveDiskPrefix(
|
||||
session: Session,
|
||||
seed: readonly SessionEvent[],
|
||||
file: { path: string; cwd: string | undefined },
|
||||
): Promise<void> {
|
||||
const buffer = await readFile(file.path)
|
||||
const { meta, events, committedBytes } = scanLog(buffer)
|
||||
this.assertVersion(meta)
|
||||
if (!seedCoversPrefix(seed, events)) {
|
||||
throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`)
|
||||
/** Durably append a batch, lazily materializing the file when not yet present. */
|
||||
async appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise<void> {
|
||||
if (isMaterialized) {
|
||||
await this.appendLines(meta, events)
|
||||
} else {
|
||||
await this.materialize(meta, events)
|
||||
}
|
||||
|
||||
const state: SessionState = {
|
||||
meta: { ...meta },
|
||||
cursor: events.length,
|
||||
materialized: true,
|
||||
owner: session,
|
||||
}
|
||||
this.states.set(session.header.id, state)
|
||||
|
||||
if (committedBytes < buffer.byteLength) {
|
||||
await this.repair(state, committedBytes)
|
||||
}
|
||||
const suffix = seed.slice(events.length)
|
||||
if (suffix.length > 0) await this.appendCore(session.header.id, suffix)
|
||||
}
|
||||
|
||||
/**
|
||||
* Make a crash repair durable: truncate the torn tail to `tornMarker` bytes (if
|
||||
* any), then append the synthetic `closers` (if any). Two fsync'd steps — the
|
||||
* seam does not require this to be atomic.
|
||||
*/
|
||||
async commitRepair(meta: SessionHeader, tornMarker: number | undefined, closers: readonly SessionEvent[]): Promise<void> {
|
||||
if (tornMarker !== undefined) await this.repair(meta, tornMarker)
|
||||
if (closers.length > 0) await this.appendLines(meta, closers)
|
||||
}
|
||||
|
||||
/** Remove a session's log file (the coordinator clears its in-memory state). */
|
||||
async deleteStored(id: SessionId): Promise<void> {
|
||||
const file = await this.findLog(id, undefined)
|
||||
if (file) await rm(file.path, { force: true })
|
||||
}
|
||||
|
||||
/** List all stored sessions' metadata (header line only — no full-log parse). */
|
||||
async list(): Promise<SessionHeader[]> {
|
||||
const metas: SessionHeader[] = []
|
||||
for (const dir of await this.listCwdDirs()) {
|
||||
for (const name of await this.listJsonl(dir)) {
|
||||
// Read ONLY the header line, not the whole log: a session picker must
|
||||
// scale with the number of sessions, not the total size of every
|
||||
// conversation (the log persists every assistant/chunk verbatim, so a
|
||||
// full scanLog here would be O(total history)).
|
||||
// conversation (the log persists every assistant/chunk verbatim).
|
||||
const first = await this.readFirstLine(`${dir}/${name}`)
|
||||
if (first === undefined) continue // empty/half-written file
|
||||
const meta = parseHeaderMeta(first)
|
||||
@@ -313,11 +196,119 @@ export class SessionPersistenceJsonl extends SessionPersistence {
|
||||
return metas
|
||||
}
|
||||
|
||||
// --- materialization / append / repair (file mechanics) ---
|
||||
|
||||
/** Atomically write the header line + first batch (temp-write, fsync, rename). */
|
||||
private async materialize(meta: SessionHeader, events: readonly SessionEvent[]): Promise<void> {
|
||||
const dir = sessionDir(this.root, meta.cwd)
|
||||
await mkdir(this.root, { recursive: true, mode: 0o700 })
|
||||
await this.syncDir(dirname(this.root))
|
||||
await mkdir(dir, { recursive: true, mode: 0o700 })
|
||||
await this.syncDir(this.root)
|
||||
const finalPath = logPath(this.root, meta.cwd, meta.id)
|
||||
// Never rename over an existing committed log: materialize is the FIRST write
|
||||
// of a session the backend believes is new. A file here means a different
|
||||
// session shares this id on disk — reject loudly. (createCore already guards
|
||||
// the create path, so this is unreachable-in-practice TOCTOU defense.)
|
||||
/* v8 ignore next 3 -- createCore guards collisions before materialize; this is a TOCTOU backstop */
|
||||
if (await this.exists(finalPath)) {
|
||||
throw new Error(`refusing to materialize "${meta.id}": a log already exists on disk (load/resume it instead)`)
|
||||
}
|
||||
const header = JSON.stringify(toHeaderLine(meta))
|
||||
const body = events.map(eventLine).join('\n')
|
||||
const content = header + '\n' + body + '\n'
|
||||
|
||||
const tmp = `${finalPath}.${randomBytes(6).toString('hex')}.tmp`
|
||||
const handle = await open(tmp, 'wx', 0o600)
|
||||
try {
|
||||
await handle.writeFile(content)
|
||||
await handle.sync()
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
// Publish via link()+unlink(), NOT rename(): link fails with EEXIST if the
|
||||
// final path already exists, so two processes materializing the same id
|
||||
// concurrently cannot clobber each other. rename() would silently overwrite.
|
||||
let linked = false
|
||||
try {
|
||||
await link(tmp, finalPath)
|
||||
linked = true
|
||||
} finally {
|
||||
// If link FAILED, the temp is the only reference and must be removed before
|
||||
// the original error propagates. If it SUCCEEDED, defer temp cleanup to
|
||||
// AFTER the publish is durable (below) so a temp-rm failure can never reject
|
||||
// a session whose log already published.
|
||||
/* v8 ignore next -- link failure is the TOCTOU/IO race guarded above; not reachable in test */
|
||||
if (!linked) await rm(tmp, { force: true })
|
||||
}
|
||||
// link() succeeded — the log is published. fsync the directory so the new
|
||||
// entry survives a power loss: the new link is not crash-durable until the
|
||||
// parent directory's metadata is synced.
|
||||
await this.syncDir(dir)
|
||||
// Best-effort temp cleanup: the log is already published and durable, so a
|
||||
// failure to remove the (now-redundant) temp hard link must NOT reject the
|
||||
// append. Swallow only the rm failure; nothing else of consequence runs here.
|
||||
try {
|
||||
await rm(tmp, { force: true })
|
||||
} catch {
|
||||
/* v8 ignore next -- redundant temp link; publish already durable, rm failure is an unreachable IO edge */
|
||||
}
|
||||
}
|
||||
|
||||
/** fsync a directory so a just-created/renamed entry inside it is crash-durable. */
|
||||
private async syncDir(dir: string): Promise<void> {
|
||||
const handle = await open(dir, 'r')
|
||||
try {
|
||||
await handle.sync()
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Append event lines at EOF and fsync. On a write/sync failure AFTER the kernel
|
||||
* accepted some bytes (ENOSPC, an fsync error), truncate the file back to its
|
||||
* pre-append size before rethrowing: the cursor is unchanged, so the batch will
|
||||
* be retried, and without this rollback the retry would append AFTER the partial
|
||||
* bytes — producing duplicate seqs that make `scanLog` see a gap.
|
||||
*/
|
||||
private async appendLines(meta: SessionHeader, events: readonly SessionEvent[]): Promise<void> {
|
||||
const path = logPath(this.root, meta.cwd, meta.id)
|
||||
const handle = await open(path, 'a')
|
||||
try {
|
||||
const { size: before } = await handle.stat()
|
||||
try {
|
||||
await handle.writeFile(events.map(eventLine).join('\n') + '\n')
|
||||
await handle.sync()
|
||||
} catch (error) {
|
||||
// Roll back whatever bytes landed so a retry starts from a clean EOF.
|
||||
await handle.truncate(before)
|
||||
await handle.sync()
|
||||
throw error
|
||||
}
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
}
|
||||
|
||||
/** Truncate the log file to `offset` bytes and fsync (discard the crash tail). */
|
||||
private async repair(meta: SessionHeader, offset: number): Promise<void> {
|
||||
const path = logPath(this.root, meta.cwd, meta.id)
|
||||
await truncate(path, offset)
|
||||
const handle = await open(path, 'r+')
|
||||
try {
|
||||
await handle.sync()
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
}
|
||||
|
||||
// --- discovery helpers ---
|
||||
|
||||
/**
|
||||
* Read the first newline-terminated line of a file without loading the whole
|
||||
* file. Returns undefined if the file is empty or has no complete first line
|
||||
* (a half-written log). Reads in bounded chunks so a huge log costs only the
|
||||
* header read.
|
||||
* file. Returns undefined if the file is empty or has no complete first line.
|
||||
* Reads in bounded chunks so a huge log costs only the header read.
|
||||
*/
|
||||
private async readFirstLine(path: string): Promise<string | undefined> {
|
||||
const handle = await open(path, 'r')
|
||||
@@ -340,145 +331,6 @@ export class SessionPersistenceJsonl extends SessionPersistence {
|
||||
}
|
||||
}
|
||||
|
||||
async has(id: SessionId): Promise<boolean> {
|
||||
const state = this.states.get(id)
|
||||
if (state?.materialized) return true
|
||||
const cwd = state?.meta.cwd
|
||||
return (await this.findLog(id, cwd)) !== undefined
|
||||
}
|
||||
|
||||
delete(id: SessionId): Promise<void> {
|
||||
return this.serialize(id, () => this.deleteCore(id))
|
||||
}
|
||||
|
||||
private async deleteCore(id: SessionId): Promise<void> {
|
||||
const cwd = this.states.get(id)?.meta.cwd
|
||||
const file = await this.findLog(id, cwd)
|
||||
if (file) await rm(file.path, { force: true })
|
||||
this.states.delete(id)
|
||||
}
|
||||
|
||||
// --- materialization / append / repair ---
|
||||
|
||||
/** Atomically write the header line + first batch (temp-write, fsync, rename). */
|
||||
private async materialize(state: SessionState, events: readonly SessionEvent[]): Promise<void> {
|
||||
const dir = sessionDir(this.root, state.meta.cwd)
|
||||
await mkdir(this.root, { recursive: true, mode: 0o700 })
|
||||
await this.syncDir(dirname(this.root))
|
||||
await mkdir(dir, { recursive: true, mode: 0o700 })
|
||||
await this.syncDir(this.root)
|
||||
const finalPath = logPath(this.root, state.meta.cwd, state.meta.id)
|
||||
// Never rename over an existing committed log: materialize is the FIRST
|
||||
// write of a session the backend believes is new. A file here means a
|
||||
// different session shares this id on disk — reject loudly rather than
|
||||
// clobber committed data. (createCore already guards the create path before
|
||||
// this point, so this is unreachable-in-practice defense-in-depth against a
|
||||
// TOCTOU/fork race; ignored for coverage.)
|
||||
/* v8 ignore next 3 -- createCore guards collisions before materialize; this is a TOCTOU backstop */
|
||||
if (await this.exists(finalPath)) {
|
||||
throw new Error(`refusing to materialize "${state.meta.id}": a log already exists on disk (load/resume it instead)`)
|
||||
}
|
||||
const header = JSON.stringify(toHeaderLine(state.meta))
|
||||
const body = events.map(eventLine).join('\n')
|
||||
const content = header + '\n' + body + '\n'
|
||||
|
||||
const tmp = `${finalPath}.${randomBytes(6).toString('hex')}.tmp`
|
||||
const handle = await open(tmp, 'wx', 0o600)
|
||||
try {
|
||||
await handle.writeFile(content)
|
||||
await handle.sync()
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
// Publish via link()+unlink(), NOT rename(): link fails with EEXIST if the
|
||||
// final path already exists, so two processes materializing the same id
|
||||
// concurrently cannot clobber each other (both could pass the exists() check
|
||||
// above, but only one link() wins). rename() would silently overwrite the
|
||||
// log the other process just committed.
|
||||
let linked = false
|
||||
try {
|
||||
await link(tmp, finalPath)
|
||||
linked = true
|
||||
} finally {
|
||||
// If link FAILED (EEXIST on a race, or any I/O error), the temp is the
|
||||
// only reference and must be removed before the original error propagates.
|
||||
// If link SUCCEEDED, the temp cleanup is deferred to AFTER the publish is
|
||||
// durable (below) so a temp-rm failure can never reject a session whose
|
||||
// log already published — that would leave state.materialized false and
|
||||
// wedge every retry on the exists() backstop above.
|
||||
/* v8 ignore next -- link failure is the TOCTOU/IO race guarded above; not reachable in test */
|
||||
if (!linked) await rm(tmp, { force: true })
|
||||
}
|
||||
// link() succeeded — the log is published. fsync the directory so the new
|
||||
// entry survives a power loss: on POSIX filesystems the new link is not
|
||||
// crash-durable until the parent directory's metadata is synced. The seam
|
||||
// contract is "append returns once durable", and materialize is the first
|
||||
// append's write — so the directory entry must be durable before we return.
|
||||
await this.syncDir(dir)
|
||||
state.materialized = true
|
||||
// Best-effort temp cleanup: the log is already published and durable, so a
|
||||
// failure to remove the (now-redundant) temp hard link must NOT reject the
|
||||
// append. A leftover `*.tmp` is harmless — it is never read, and the next
|
||||
// materialize of this id is guarded by exists()/link(). Swallow only the
|
||||
// rm failure; nothing else of consequence runs in the try.
|
||||
try {
|
||||
await rm(tmp, { force: true })
|
||||
} catch {
|
||||
/* v8 ignore next -- redundant temp link; publish already durable, rm failure is an unreachable IO edge */
|
||||
}
|
||||
}
|
||||
|
||||
/** fsync a directory so a just-created/renamed entry inside it is crash-durable. */
|
||||
private async syncDir(dir: string): Promise<void> {
|
||||
const handle = await open(dir, 'r')
|
||||
try {
|
||||
await handle.sync()
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Append event lines at EOF and fsync. On a write/sync failure AFTER the
|
||||
* kernel accepted some bytes (ENOSPC, an fsync error), truncate the file back
|
||||
* to its pre-append size before rethrowing: `cursor` is unchanged, so the
|
||||
* batch will be retried, and without this rollback the retry would append
|
||||
* AFTER the partial bytes — producing duplicate seqs that make `scanLog` see a
|
||||
* gap and render the session unloadable.
|
||||
*/
|
||||
private async appendLines(state: SessionState, events: readonly SessionEvent[]): Promise<void> {
|
||||
const path = logPath(this.root, state.meta.cwd, state.meta.id)
|
||||
const handle = await open(path, 'a')
|
||||
try {
|
||||
const { size: before } = await handle.stat()
|
||||
try {
|
||||
await handle.writeFile(events.map(eventLine).join('\n') + '\n')
|
||||
await handle.sync()
|
||||
} catch (error) {
|
||||
// Roll back whatever bytes landed so a retry starts from a clean EOF.
|
||||
await handle.truncate(before)
|
||||
await handle.sync()
|
||||
throw error
|
||||
}
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
}
|
||||
|
||||
/** Truncate the log file to `offset` bytes and fsync (discard the crash tail). */
|
||||
private async repair(state: SessionState, offset: number): Promise<void> {
|
||||
const path = logPath(this.root, state.meta.cwd, state.meta.id)
|
||||
await truncate(path, offset)
|
||||
const handle = await open(path, 'r+')
|
||||
try {
|
||||
await handle.sync()
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
}
|
||||
|
||||
// --- discovery helpers ---
|
||||
|
||||
/** Find a session's log file across cwd buckets (when cwd is unknown). */
|
||||
private async findLog(id: SessionId, cwd: string | undefined): Promise<{ path: string; cwd: string | undefined } | undefined> {
|
||||
if (cwd !== undefined) {
|
||||
@@ -490,8 +342,7 @@ export class SessionPersistenceJsonl extends SessionPersistence {
|
||||
for (const dir of await this.listCwdDirs()) {
|
||||
const path = `${dir}/${target}`
|
||||
if (await this.exists(path)) {
|
||||
// Recover the cwd from the header so the caller has the session's
|
||||
// bucket location (which `findLog` was given an unknown cwd for).
|
||||
// Recover the cwd from the header so the caller has the session's bucket.
|
||||
const { meta } = scanLog(await readFile(path))
|
||||
return { path, cwd: meta.cwd }
|
||||
}
|
||||
@@ -505,10 +356,9 @@ export class SessionPersistenceJsonl extends SessionPersistence {
|
||||
const entries = await readdir(this.root, { withFileTypes: true })
|
||||
return entries.filter(e => e.isDirectory()).map(e => `${this.root}/${e.name}`)
|
||||
} catch (error) {
|
||||
// ENOENT = the root has not been created yet → genuinely no sessions.
|
||||
// Any other error (EACCES, ENOTDIR, transient I/O) must NOT be reported
|
||||
// as "no sessions" — a durable backend cannot silently pretend persisted
|
||||
// state is absent on a storage fault.
|
||||
// ENOENT = the root has not been created yet → genuinely no sessions. Any
|
||||
// other error (EACCES, ENOTDIR, transient I/O) must NOT be reported as "no
|
||||
// sessions" — a durable backend cannot silently pretend state is absent.
|
||||
if (isENOENT(error)) return []
|
||||
throw error
|
||||
}
|
||||
@@ -526,244 +376,12 @@ export class SessionPersistenceJsonl extends SessionPersistence {
|
||||
return true
|
||||
} catch (error) {
|
||||
// Only ENOENT means absent. A permission/I/O error must surface, not be
|
||||
// collapsed to `false` — otherwise load() reports "not found" and
|
||||
// collision checks proceed under a false absence assumption.
|
||||
// collapsed to `false` — otherwise load() reports "not found" and collision
|
||||
// checks proceed under a false absence assumption.
|
||||
if (isENOENT(error)) return false
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
/** Build a state for a session discovered on disk but not yet in memory. */
|
||||
private async adopt(id: SessionId): Promise<SessionState> {
|
||||
// loadCore (NOT load) — adopt runs inside an already-serialized op, so
|
||||
// re-entering the chain via the public load() would deadlock.
|
||||
await this.loadCore(id)
|
||||
const state = this.states.get(id)
|
||||
/* v8 ignore next -- loadCore always sets the state for the id */
|
||||
if (!state) throw new Error(`failed to adopt session "${id}"`)
|
||||
return state
|
||||
}
|
||||
|
||||
private assertVersion(meta: SessionHeader): void {
|
||||
if (meta.version !== 1) {
|
||||
throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`)
|
||||
}
|
||||
}
|
||||
|
||||
// --- write path (session/event → flush drain) ---
|
||||
|
||||
private installWritePath(): void {
|
||||
const ctx = this.ctx
|
||||
|
||||
// Capture the header on creation; persist a fork's seed once. Record the
|
||||
// init promise so flush/dispose can await it (onCreated is async).
|
||||
ctx.on('session/created', (session) => { void this.initFor(session) })
|
||||
|
||||
// Snapshot + buffer every event (the live object is mutable; clone so a
|
||||
// later in-place mutation of session.events cannot rewrite a buffered
|
||||
// event). Serializability is guaranteed at the source — `Session.append`
|
||||
// rejects non-JSON-serializable data before the event ever enters the log
|
||||
// or this emit — so structuredClone here can never hit a non-cloneable
|
||||
// value, and the durable log can never diverge from session.events.
|
||||
ctx.on('session/event', (session, event) => {
|
||||
let buffer = this.buffers.get(session)
|
||||
if (!buffer) this.buffers.set(session, buffer = [])
|
||||
buffer.push(structuredClone(event))
|
||||
})
|
||||
|
||||
// Drain to the backend at the durability checkpoint.
|
||||
ctx.on('session/flush', session => this.flush(session))
|
||||
|
||||
// Dispose must reach quiescence: await every session's init + final drain
|
||||
// BEFORE returning, so no write lands after teardown (orphan rename/ENOENT).
|
||||
ctx.effect(() => async () => {
|
||||
const errors = [
|
||||
...await settledErrors(this.inits.values()),
|
||||
...await settledErrors([...this.buffers.keys()].map(s => this.flush(s))),
|
||||
...await settledErrors(this.chains.values()),
|
||||
]
|
||||
if (errors.length > 0) {
|
||||
throw new AggregateError(errors, 'session-persistence-jsonl dispose failed')
|
||||
}
|
||||
}, 'session-persistence-jsonl write path')
|
||||
|
||||
// HMR: a hot reload does not replay session/created, so seed existing live
|
||||
// sessions (mirrors dsh-invariants).
|
||||
for (const session of ctx.sessions.list()) void this.initFor(session)
|
||||
}
|
||||
|
||||
/** Start (once) the async init for a session and remember its promise. */
|
||||
private initFor(session: Session): Promise<void> {
|
||||
const existing = this.inits.get(session)
|
||||
if (existing) return existing
|
||||
// Snapshot the seed SYNCHRONOUSLY here — initFor runs inside the
|
||||
// `session/created` emit, before any later `append` adds non-seed events.
|
||||
// A clone freezes it against later mutation of the live event objects.
|
||||
const seed = session.events.map(e => structuredClone(e))
|
||||
const p = this.onCreated(session, seed)
|
||||
// Attach a no-op rejection handler so a failing init (e.g. an id collision)
|
||||
// does not surface as an unhandled rejection if no flush observes `p` before
|
||||
// it rejects. The REAL error is still delivered: flush/dispose await the
|
||||
// same `p` from the map and see the rejection there.
|
||||
p.catch(() => { /* observed by flush/dispose via the stored promise */ })
|
||||
this.inits.set(session, p)
|
||||
return p
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a live `session`'s `seed` reproduces the first `cursor` persisted
|
||||
* events. Reads the on-disk committed prefix and compares. A `cursor` of 0
|
||||
* (nothing persisted yet) trivially matches. Used when a live session claims
|
||||
* ownerless state left by a prior `load()`/`create()` — to reject a fresh,
|
||||
* unrelated session that reuses the id and would otherwise have its seq
|
||||
* 0..cursor-1 events filtered as already-written.
|
||||
*/
|
||||
private async seedMatchesPersisted(session: Session, seed: readonly SessionEvent[], cursor: number): Promise<boolean> {
|
||||
if (cursor === 0) return true
|
||||
const onDisk = await this.findLog(session.header.id, session.header.cwd)
|
||||
/* v8 ignore next -- a cursor > 0 means the log was materialized, so it exists */
|
||||
if (onDisk === undefined) return false
|
||||
const { events: diskEvents } = scanLog(await readFile(onDisk.path))
|
||||
return seedCoversPrefix(seed, diskEvents.slice(0, cursor))
|
||||
}
|
||||
|
||||
/**
|
||||
* On session/created: sync the backend's in-memory state to a live Session.
|
||||
*
|
||||
* Cases, by whether this backend tracks the id and whether a log is on disk:
|
||||
* 1. Already in `states` (created here, or a prior load/resume) → no-op.
|
||||
* 2. Not tracked, a log EXISTS on disk, and it is a seq-aligned PREFIX of the
|
||||
* live session's current events → ADOPT it (HMR/reload): a fresh backend
|
||||
* instance (empty `states`) meets a live session whose log a previous
|
||||
* instance materialized; the live object already carries that history (it
|
||||
* is the source of truth this run), so we continue from the stored length
|
||||
* instead of re-creating. This keeps persistence alive across hot reload.
|
||||
* 3. Not tracked, a log EXISTS on disk, but it is NOT a prefix of the live
|
||||
* session's events → REJECT: a different session collides on the id. The
|
||||
* SessionId is the identity, so two unrelated sessions sharing one is a
|
||||
* bug, not a resume — fail loudly rather than clobber committed data.
|
||||
* 4. Not tracked and NO log on disk → a genuinely new session: register its
|
||||
* meta (lazy) and persist its `seed` once.
|
||||
*
|
||||
* The public `create(meta)` API is stricter still (rejects ANY on-disk id):
|
||||
* there the caller asserts "brand new", so even a prefix match is a bug.
|
||||
*
|
||||
* The seed events were copied into the Session by its constructor WITHOUT
|
||||
* emitting session/event, so the write-behind buffer never sees them — the
|
||||
* one explicit `append(seed)` below is the only persistence of the seed.
|
||||
* Events appended AFTER creation flow through the session/event buffer and
|
||||
* are persisted by flush (filtered by the write cursor), never here.
|
||||
*/
|
||||
private async onCreated(session: Session, seed: readonly SessionEvent[]): Promise<void> {
|
||||
const id = session.header.id
|
||||
const tracked = this.states.get(id)
|
||||
if (tracked !== undefined) {
|
||||
// case 1: already tracked.
|
||||
// (owner === session is a defensive same-object guard: initFor dedupes by
|
||||
// session object, so onCreated never actually runs twice for one session.)
|
||||
/* v8 ignore next -- initFor dedupes per session object; same-object re-entry can't occur */
|
||||
if (tracked.owner === session) return
|
||||
if (tracked.owner === undefined) {
|
||||
// Ownerless state was created via the public create()/load() API. The
|
||||
// FIRST live session to arrive claims it — but ONLY if its seed is the
|
||||
// already-persisted prefix. A load() for preview leaves cursor at the
|
||||
// persisted length; a fresh, unrelated session reusing that id has a
|
||||
// seed shorter than (or not matching) that prefix, so flush would filter
|
||||
// its seq 0..cursor-1 events as already-written and silently graft the
|
||||
// new conversation onto the old log. Verify the seed covers the cursor.
|
||||
if (!await this.seedMatchesPersisted(session, seed, tracked.cursor)) {
|
||||
throw new Error(`session "${id}" is already persisted with ${tracked.cursor} event(s) that do not match this live session (id collision)`)
|
||||
}
|
||||
tracked.owner = session
|
||||
// Persist the live seed SUFFIX beyond the persisted prefix. Constructor
|
||||
// seed events (from sessions.create(id, { seed })) never emit
|
||||
// session/event, so the write-behind buffer never sees them — without
|
||||
// this they would be lost and a later flush would seq-mismatch. (cursor
|
||||
// is 0 for a public create(), so this covers the whole seed there.)
|
||||
const suffix = seed.slice(tracked.cursor)
|
||||
if (suffix.length > 0) await this.append(id, suffix)
|
||||
return
|
||||
}
|
||||
// The state is owned by a DIFFERENT live session. We may reclaim the id
|
||||
// ONLY if that owner left nothing behind: never materialized a log (cursor
|
||||
// 0, not materialized) AND has no write-behind buffer still pending. A
|
||||
// session that appended events but was disposed before its first flush is
|
||||
// NOT materialized yet but DOES have buffered events — reclaiming then
|
||||
// would let that stale buffer drain against the new session's state
|
||||
// (persisting old events under the new id, or dropping the new session's
|
||||
// seq-0 events). Such an owner, and any materialized owner, is a real
|
||||
// collision and rejects; only a truly-abandoned (artifact-free) id is
|
||||
// freed, honoring lazy materialization's "leaves nothing behind" promise.
|
||||
const ownerBuffer = this.buffers.get(tracked.owner)
|
||||
if (!tracked.materialized && !ownerBuffer?.length) {
|
||||
this.states.delete(id)
|
||||
} else {
|
||||
throw new Error(`session "${id}" is already bound to a different live session in this backend (id collision)`)
|
||||
}
|
||||
}
|
||||
|
||||
const onDisk = await this.findLog(id, session.header.cwd)
|
||||
if (onDisk !== undefined) {
|
||||
// case 2: adopt a LIVE prefix. Do NOT route through loadCore(): loadCore
|
||||
// crash-repairs open turns as interrupted, which is right for a true load
|
||||
// after a crash but wrong for HMR while the live Session is still the
|
||||
// authority and may append the real step/turn end later.
|
||||
await this.serialize(id, () => this.adoptLiveDiskPrefix(session, seed, onDisk))
|
||||
return
|
||||
}
|
||||
|
||||
// case 4: a genuinely new session. Register its meta (lazy), then persist
|
||||
// its seed (events present at creation time) once.
|
||||
const meta: SessionHeader = { ...session.header }
|
||||
await this.create(meta)
|
||||
// Bind this state to the live session so a later DIFFERENT session reusing
|
||||
// the id is detected as a collision (case 1) rather than silently no-opped.
|
||||
const created = this.states.get(id)
|
||||
/* v8 ignore next -- create() always sets the state for the id */
|
||||
if (created !== undefined) created.owner = session
|
||||
if (seed.length > 0) {
|
||||
await this.append(id, seed)
|
||||
}
|
||||
}
|
||||
|
||||
private async flush(session: Session): Promise<void> {
|
||||
// Wait for the session's init (onCreated) to finish so the state/cursor and
|
||||
// any fork-seed persistence are in place before we drain. Awaiting the same
|
||||
// promise initFor stored also surfaces an init failure (e.g. an id
|
||||
// collision) here, where the caller of session/flush observes it.
|
||||
await this.inits.get(session)
|
||||
// Serialize the WHOLE drain (read cursor → append → splice) on the
|
||||
// per-session chain. Two concurrent flushes (e.g. an idle inject()'s
|
||||
// fire-and-forget flush racing an explicit checkpoint) would otherwise both
|
||||
// read the same cursor, both compute the same `fresh` slice, and the second
|
||||
// append would seq-mismatch against the cursor the first already advanced.
|
||||
await this.serialize(session.header.id, () => this.drain(session))
|
||||
}
|
||||
|
||||
/** Drain a session's write buffer to disk. Caller serializes this per id. */
|
||||
private async drain(session: Session): Promise<void> {
|
||||
const buffer = this.buffers.get(session)
|
||||
if (!buffer?.length) return
|
||||
// Copy WITHOUT removing: the buffer is the only durable-pending copy of
|
||||
// these events (session/event does not re-emit). Splicing before the append
|
||||
// means a failed append (disk error, or a seq mismatch after a dropped bad
|
||||
// event) permanently loses a completed turn. Drain the buffer only AFTER
|
||||
// the append commits; events pushed during the await sit past batch.length
|
||||
// and survive the prefix splice, so a retry/dispose re-drains the rest.
|
||||
const batch = buffer.slice()
|
||||
const state = this.states.get(session.header.id)
|
||||
// Only append events at or beyond the write cursor (a resumed session's
|
||||
// seed is already on disk; the cursor was set to the loaded length). flush
|
||||
// awaits the init above, which always sets state, so the `?? 0` fallback is
|
||||
// a defensive guard that never fires in practice.
|
||||
/* v8 ignore next -- state is always set by the awaited init before flush */
|
||||
const cursor = state?.cursor ?? 0
|
||||
const fresh = batch.filter(e => e.seq >= cursor)
|
||||
// appendCore (NOT the serialized append) — drain already runs inside the
|
||||
// per-session chain, so re-entering it via append() would deadlock.
|
||||
if (fresh.length > 0) await this.appendCore(session.header.id, fresh)
|
||||
buffer.splice(0, batch.length)
|
||||
}
|
||||
}
|
||||
|
||||
export default SessionPersistenceJsonl
|
||||
|
||||
@@ -4,10 +4,11 @@ import { appendFile, mkdtemp, mkdir, rm, readFile, writeFile, readdir, stat } fr
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
import { encodeSegment, logPath, scanLog, sessionDir } from '../src/format.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts'
|
||||
import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts'
|
||||
|
||||
let root: string
|
||||
const dirs: string[] = []
|
||||
@@ -37,6 +38,29 @@ runPersistenceContract('jsonl', async () => {
|
||||
}
|
||||
})
|
||||
|
||||
// Run the shared coordinator orchestration suite against the real JSONL backend.
|
||||
// One temp root is the shared storage scope (two mounted instances over the same
|
||||
// root = HMR/reload). `corruptTail` appends a partial, newline-less fragment to
|
||||
// the session's .jsonl past the committed region — a never-committed torn tail
|
||||
// that drives the coordinator's commitRepair-with-tornMarker branch over real
|
||||
// file bytes.
|
||||
runCoordinatorContract('jsonl', async (): Promise<CoordinatorFixture> => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-jsonl-coord-'))
|
||||
return {
|
||||
mount: async (ctx) => {
|
||||
const fiber = await ctx.plugin(SessionPersistenceJsonl, { root: dir })
|
||||
return fiber
|
||||
},
|
||||
corruptTail: async (id, cwd) => {
|
||||
// A half-written record with no trailing newline: scanLog treats it as an
|
||||
// uncommitted crash fragment and reports committedBytes < byteLength, so
|
||||
// the coordinator sees a tornMarker to truncate.
|
||||
await appendFile(logPath(dir, cwd, id), '{"type":"assistant/chunk","seq":8,"ti')
|
||||
},
|
||||
cleanup: async () => { await rm(dir, { recursive: true, force: true }) },
|
||||
}
|
||||
})
|
||||
|
||||
describe('SessionPersistenceJsonl: format helpers', () => {
|
||||
it('encodeSegment neutralizes traversal, separators, and absolute paths', () => {
|
||||
expect(encodeSegment('..')).toBe('~002E~002E')
|
||||
@@ -198,36 +222,6 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
})
|
||||
|
||||
it('append snapshots its batch: mutating the caller array after the call is ignored', async () => {
|
||||
const m = meta('snapshot')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
const events = oneTurnLog() // seqs 0..5
|
||||
const p = ctx.sessionPersistence.append(m.id, events)
|
||||
// Mutate the caller's array immediately after calling append (before the
|
||||
// queued op runs). The backend must persist the snapshot taken at call time,
|
||||
// not the mutated array.
|
||||
events.push({ type: 'turn/start', seq: 6, time: 99, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } })
|
||||
await p
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) // not 0..6
|
||||
})
|
||||
|
||||
it('append deep-snapshots event objects: mutating an event after the call is ignored', async () => {
|
||||
const m = meta('deep-snapshot')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
const events = oneTurnLog()
|
||||
const userMsg = events[1] // the user/message event
|
||||
const p = ctx.sessionPersistence.append(m.id, events)
|
||||
// Mutate an event OBJECT (not just the array) after calling append. The deep
|
||||
// snapshot taken at call time must shield the persisted data.
|
||||
if (userMsg?.type === 'user/message') userMsg.data.content = [{ type: 'text', text: 'MUTATED' }]
|
||||
await p
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
const persisted = JSON.stringify(loaded.events)
|
||||
expect(persisted).toContain('hi') // original content
|
||||
expect(persisted).not.toContain('MUTATED')
|
||||
})
|
||||
|
||||
it('load returns a meta copy: mutating it does not corrupt backend pathing', async () => {
|
||||
const m = meta('meta-copy', '/proj')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
@@ -246,25 +240,6 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
})
|
||||
|
||||
it('rejects an unknown format version on load', async () => {
|
||||
const m = meta('v2')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
// Corrupt the header version on disk.
|
||||
const path = logPath(root, undefined, m.id)
|
||||
const lines = (await readFile(path, 'utf8')).split('\n')
|
||||
const header = JSON.parse(lines[0]!) as { version: number }
|
||||
header.version = 2
|
||||
lines[0] = JSON.stringify(header)
|
||||
await writeFile(path, lines.join('\n'))
|
||||
// Fresh backend (no in-memory state) → must reject on load.
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root })
|
||||
await expect(ctx2.sessionPersistence.load(m.id)).rejects.toThrow(/version/)
|
||||
await ctx2.fiber.dispose()
|
||||
})
|
||||
|
||||
it('rejects a re-append of an already-stored seq', async () => {
|
||||
const m = meta('reappend')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
@@ -293,62 +268,6 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
})
|
||||
|
||||
describe('SessionPersistenceJsonl: write path (session/event → flush)', () => {
|
||||
it('persists a live session driven through the store, surviving reload', async () => {
|
||||
root = await freshRoot()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
|
||||
const session = ctx.sessions.create('live', { meta: { cwd: '/w' } })
|
||||
for (const e of oneTurnLog()) session.append(e.type, e.data)
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('live'))
|
||||
expect(loaded.events).toHaveLength(6)
|
||||
expect(loaded.meta.cwd).toBe('/w')
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('snapshot-on-buffer: mutating an event after session/event does not corrupt the persisted copy', async () => {
|
||||
root = await freshRoot()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
|
||||
const session = ctx.sessions.create('mutate')
|
||||
const ev = session.append('user/message', { content: [{ type: 'text', text: 'original' }], source: { kind: 'user' } })
|
||||
// Mutate the live event object AFTER it was buffered.
|
||||
;(ev.data as { content: { type: 'text'; text: string }[] }).content[0]!.text = 'HACKED'
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('mutate'))
|
||||
const first = loaded.events[0]
|
||||
expect(first?.type === 'user/message' && (first.data.content[0] as { text: string }).text).toBe('original')
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('fork: a seeded new session persists its seed once', async () => {
|
||||
root = await freshRoot()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
|
||||
const seed = oneTurnLog()
|
||||
// A fork: a brand-new id whose seed came from elsewhere.
|
||||
const forked = ctx.sessions.create('forked', { seed })
|
||||
// onCreated persisted the seed asynchronously; wait a tick.
|
||||
await new Promise(r => setTimeout(r, 10))
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('forked'))
|
||||
expect(loaded.events).toEqual(seed)
|
||||
// A flush with no NEW events must not double-write.
|
||||
await ctx.parallel('session/flush', forked)
|
||||
const reloaded = await ctx.sessionPersistence.load(SessionId('forked'))
|
||||
expect(reloaded.events).toEqual(seed)
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('concurrent sessions do not cross buffers', async () => {
|
||||
root = await freshRoot()
|
||||
const ctx = new Context()
|
||||
@@ -373,112 +292,6 @@ describe('SessionPersistenceJsonl: write path (session/event → flush)', () =>
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('HMR: applying the plugin seeds existing live sessions', async () => {
|
||||
root = await freshRoot()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
// A session exists BEFORE the persistence plugin is applied.
|
||||
const session = ctx.sessions.create('pre-existing')
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
// The plugin seeded it on apply; a subsequent flush persists its events.
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('pre-existing'))
|
||||
expect(loaded.events.length).toBeGreaterThanOrEqual(2)
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('HMR: dispose drains remaining buffers', async () => {
|
||||
root = await freshRoot()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
let session!: Session
|
||||
const fiber = await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
const sessFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create('drain')
|
||||
}, { inject: ['sessions'] }))
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'buffered' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
// No explicit flush — dispose must drain.
|
||||
await fiber.dispose()
|
||||
await sessFiber.dispose()
|
||||
|
||||
// A fresh backend reads what the disposed one drained.
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root })
|
||||
const loaded = await ctx2.sessionPersistence.load(SessionId('drain'))
|
||||
expect(loaded.events.length).toBeGreaterThanOrEqual(2)
|
||||
await ctx2.fiber.dispose()
|
||||
})
|
||||
|
||||
it('HMR: reloading the backend adopts a still-live, already-materialized session', async () => {
|
||||
root = await freshRoot()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
// The session lives in its OWN fiber so it survives the backend reload.
|
||||
let session!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create('hmr-adopt')
|
||||
}, { inject: ['sessions'] }))
|
||||
|
||||
// Backend instance 1 materializes the session on disk.
|
||||
const backend1 = await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Hot-reload the backend: dispose instance 1, plug in instance 2 over the
|
||||
// SAME root while the session stays live. Instance 2 has an empty states
|
||||
// map but the log is on disk — it must ADOPT (not reject) so flush keeps
|
||||
// working. A second turn appended after reload then persists.
|
||||
await backend1.dispose()
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'again' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
await expect(ctx.parallel('session/flush', session)).resolves.not.toThrow()
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-adopt'))
|
||||
expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2)
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('HMR: adoption persists the live SUFFIX that was ahead of the on-disk prefix', async () => {
|
||||
root = await freshRoot()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
let session!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create('hmr-suffix')
|
||||
}, { inject: ['sessions'] }))
|
||||
|
||||
// Instance 1 flushes turn 1 to disk.
|
||||
const backend1 = await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Append turn 2 to the LIVE session, then dispose instance 1 WITHOUT
|
||||
// flushing turn 2. Turn 2 is now ONLY in the live session's events; the new
|
||||
// backend never buffered it via session/event.
|
||||
await backend1.dispose()
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
|
||||
// Instance 2 adopts the on-disk prefix (turn 1) and MUST also persist the
|
||||
// live suffix (turn 2) carried in the session's events — otherwise turn 2 is
|
||||
// lost and a later flush would mismatch.
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-suffix'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3])
|
||||
expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2)
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
})
|
||||
|
||||
|
||||
@@ -570,17 +383,6 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
})
|
||||
afterEach(async () => { await ctx.fiber.dispose() })
|
||||
|
||||
it('load rejects a missing session', async () => {
|
||||
await expect(ctx.sessionPersistence.load(SessionId('nope'))).rejects.toThrow(/not found/)
|
||||
})
|
||||
|
||||
it('append of an empty batch is a no-op', async () => {
|
||||
const m = meta('empty-batch')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, [])
|
||||
expect(await ctx.sessionPersistence.has(m.id)).toBe(false)
|
||||
})
|
||||
|
||||
it('append rejects non-JSON-serializable undefined-producing data', async () => {
|
||||
const m = meta('undef')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
@@ -589,60 +391,6 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
await expect(ctx.sessionPersistence.append(m.id, bad)).rejects.toThrow(/non-JSON-serializable/)
|
||||
})
|
||||
|
||||
it('delete of a non-existent session is a no-op', async () => {
|
||||
await expect(ctx.sessionPersistence.delete(SessionId('ghost'))).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('an abandoned lazy session (never materialized) releases its id for reuse', async () => {
|
||||
// A live session is created then disposed BEFORE its first append: cursor 0,
|
||||
// never materialized, nothing on disk. A new live session reusing the id
|
||||
// must reclaim it (lazy materialization promises no lingering artifact),
|
||||
// not wedge on an "already bound" collision until restart.
|
||||
const backend = ctx.sessionPersistence as unknown as { inits: Map<Session, Promise<void>> }
|
||||
let firstSession!: Session
|
||||
const firstFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
firstSession = inner.sessions.create('abandoned', { meta: { cwd: '/a' } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await backend.inits.get(firstSession) // let the lazy create register the state
|
||||
await firstFiber.dispose() // disposed before any append → never materialized
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create('abandoned', { meta: { cwd: '/a' } })
|
||||
}, { inject: ['sessions'] }))
|
||||
// The new session claims the id without error and can persist a turn.
|
||||
await expect(backend.inits.get(reuse)).resolves.toBeUndefined()
|
||||
reuse.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
reuse.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', reuse)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('abandoned'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1])
|
||||
})
|
||||
|
||||
it('does NOT reclaim an id whose abandoned owner still has buffered (unflushed) events', async () => {
|
||||
// A session that appended events but was disposed BEFORE its first flush is
|
||||
// not materialized yet but still holds a write-behind buffer. Reusing the id
|
||||
// must be rejected (not reclaimed), or the stale buffer would drain against
|
||||
// the new session — persisting old events under the new id or dropping the
|
||||
// new session's seq-0 events.
|
||||
const backend = ctx.sessionPersistence as unknown as { inits: Map<Session, Promise<void>> }
|
||||
let first!: Session
|
||||
const firstFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
first = inner.sessions.create('buffered', { meta: { cwd: '/a' } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await backend.inits.get(first)
|
||||
// Append a turn but do NOT flush — events sit in the write-behind buffer.
|
||||
first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
first.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await firstFiber.dispose() // disposed before flush; not materialized, buffer pending
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create('buffered', { meta: { cwd: '/a' } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(backend.inits.get(reuse)).rejects.toThrow(/already bound to a different live session/)
|
||||
})
|
||||
|
||||
it('create snapshots its meta: mutating the caller object after the call is ignored', async () => {
|
||||
const m = meta('create-snap', '/orig')
|
||||
const p = ctx.sessionPersistence.create(m)
|
||||
@@ -713,81 +461,6 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
await ctx2.fiber.dispose()
|
||||
})
|
||||
|
||||
it('resume/adopt: a live session whose id is already on disk continues from the stored length', async () => {
|
||||
// First lifecycle: persist a session through the store.
|
||||
const s1 = ctx.sessions.create('resumed', { meta: { cwd: '/r' } })
|
||||
for (const e of oneTurnLog()) s1.append(e.type, e.data)
|
||||
await ctx.parallel('session/flush', s1)
|
||||
|
||||
// Second lifecycle: a NEW backend + a session re-created with the same id
|
||||
// and SEEDED with the loaded events (the resume path). onCreated must adopt
|
||||
// the on-disk log (not re-persist the seed), and a new turn appends at seq 6.
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root })
|
||||
const loaded = await ctx2.sessionPersistence.load(SessionId('resumed'))
|
||||
const s2 = ctx2.sessions.create('resumed', { seed: loaded.events, meta: { cwd: '/r' } })
|
||||
await new Promise(r => setTimeout(r, 10)) // let onCreated adopt
|
||||
// Append a fresh turn through the live session.
|
||||
s2.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
s2.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
await ctx2.parallel('session/flush', s2)
|
||||
|
||||
const reloaded = await ctx2.sessionPersistence.load(SessionId('resumed'))
|
||||
// 6 original + 2 new, contiguous, no duplicated seed.
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
await ctx2.fiber.dispose()
|
||||
})
|
||||
|
||||
it('HMR adoption does not crash-repair an active open turn as interrupted', async () => {
|
||||
const dir = await freshRoot()
|
||||
const hmr = new Context()
|
||||
await hmr.plugin(SessionStore)
|
||||
const first = await hmr.plugin(SessionPersistenceJsonl, { root: dir })
|
||||
const session = hmr.sessions.create('hmr-open', { meta: { cwd: '/hmr' } })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
await hmr.parallel('session/flush', session)
|
||||
|
||||
await first.dispose()
|
||||
await appendFile(logPath(dir, '/hmr', SessionId('hmr-open')), '{"torn":')
|
||||
const second = await hmr.plugin(SessionPersistenceJsonl, { root: dir })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await hmr.parallel('session/flush', session)
|
||||
|
||||
const loaded = await hmr.sessionPersistence.load(SessionId('hmr-open'))
|
||||
expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'step/start', 'step/end', 'turn/end'])
|
||||
expect(loaded.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } })
|
||||
await second.dispose()
|
||||
await hmr.fiber.dispose()
|
||||
})
|
||||
|
||||
it('a NEW live session whose id collides with an on-disk log is rejected, not silently adopted', async () => {
|
||||
// Persist a session on disk.
|
||||
const s1 = ctx.sessions.create('collide', { meta: { cwd: '/a' } })
|
||||
for (const e of oneTurnLog()) s1.append(e.type, e.data)
|
||||
await ctx.parallel('session/flush', s1)
|
||||
const before = await readFile(logPath(root, '/a', SessionId('collide')), 'utf8')
|
||||
|
||||
// A FRESH backend + a NEW live session with the same id but NO explicit
|
||||
// load/resume. onCreated must NOT adopt-from-disk (resume is explicit); it
|
||||
// treats this as a new session and create() rejects because a log already
|
||||
// exists on disk. The rejection surfaces via the init promise (flush awaits
|
||||
// it); the on-disk committed log is left byte-for-byte intact.
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root })
|
||||
const backend = ctx2.sessionPersistence as unknown as { inits: Map<Session, Promise<void>> }
|
||||
const s2 = ctx2.sessions.create('collide', { meta: { cwd: '/a' } })
|
||||
// The init for the new live session rejects (observed via the per-session
|
||||
// init map and, in production, via flush which awaits the same promise).
|
||||
await expect(backend.inits.get(s2)).rejects.toThrow(/already has a persisted log on disk/)
|
||||
// The committed log is untouched (no clobber).
|
||||
expect(await readFile(logPath(root, '/a', SessionId('collide')), 'utf8')).toBe(before)
|
||||
await ctx2.fiber.dispose()
|
||||
})
|
||||
|
||||
it('a DIFFERENT live session object reusing a disposed id gets its own init (no stale cache)', async () => {
|
||||
// Session A materializes a log under id "reuse".
|
||||
const sessFiberA = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
@@ -811,98 +484,6 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
await expect(backend.inits.get(b)).rejects.toThrow(/already bound to a different live session|already has a persisted log on disk/)
|
||||
})
|
||||
|
||||
it('a live session claims cursor-0 ownerless state created via the public API', async () => {
|
||||
// create() registers ownerless state with cursor 0 (lazy, nothing persisted
|
||||
// yet). A live session with that id then arrives and claims it without a
|
||||
// prefix check (cursor 0 matches trivially), persisting its seed.
|
||||
await ctx.sessionPersistence.create(meta('lazy-claim', '/a'))
|
||||
const backend = ctx.sessionPersistence as unknown as { inits: Map<Session, Promise<void>> }
|
||||
let live!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
live = inner.sessions.create('lazy-claim', { meta: { cwd: '/a' } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(backend.inits.get(live)).resolves.toBeUndefined()
|
||||
live.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
live.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', live)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('lazy-claim'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1])
|
||||
})
|
||||
|
||||
it('a fresh session reusing a previously-loaded id is rejected (ownerless guard)', async () => {
|
||||
// Materialize a log, then load() it into the backend's state WITHOUT a live
|
||||
// session — leaving state.owner undefined and cursor at the persisted length
|
||||
// (the public preview path).
|
||||
await ctx.sessionPersistence.create(meta('preview', '/a'))
|
||||
await ctx.sessionPersistence.append(SessionId('preview'), oneTurnLog())
|
||||
await ctx.sessionPersistence.load(SessionId('preview'))
|
||||
|
||||
const backend = ctx.sessionPersistence as unknown as { inits: Map<Session, Promise<void>> }
|
||||
// A FRESH (empty-seed) live session reusing that id must be rejected: its
|
||||
// seq 0..cursor-1 events would otherwise be filtered as already-persisted
|
||||
// and its conversation grafted onto the old log.
|
||||
let fresh!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
fresh = inner.sessions.create('preview', { meta: { cwd: '/a' } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(backend.inits.get(fresh)).rejects.toThrow(/do not match this live session|already has a persisted log/)
|
||||
})
|
||||
|
||||
it('a session whose seed matches the loaded prefix claims ownerless state', async () => {
|
||||
// Materialize a log and load it (ownerless state, cursor = 6).
|
||||
await ctx.sessionPersistence.create(meta('match', '/a'))
|
||||
await ctx.sessionPersistence.append(SessionId('match'), oneTurnLog())
|
||||
await ctx.sessionPersistence.load(SessionId('match'))
|
||||
|
||||
const backend = ctx.sessionPersistence as unknown as { inits: Map<Session, Promise<void>> }
|
||||
// A live session SEEDED with the persisted log legitimately continues it —
|
||||
// its seed reproduces the loaded prefix, so it claims the ownerless state.
|
||||
let cont!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
cont = inner.sessions.create('match', { seed: oneTurnLog(), meta: { cwd: '/a' } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(backend.inits.get(cont)).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('claiming ownerless state persists the seed suffix beyond the prefix', async () => {
|
||||
// Materialize a one-turn log and load it (ownerless state, cursor = 6).
|
||||
await ctx.sessionPersistence.create(meta('suffix-claim', '/a'))
|
||||
await ctx.sessionPersistence.append(SessionId('suffix-claim'), oneTurnLog())
|
||||
await ctx.sessionPersistence.load(SessionId('suffix-claim'))
|
||||
|
||||
const backend = ctx.sessionPersistence as unknown as { inits: Map<Session, Promise<void>> }
|
||||
// A live session seeded with the prefix PLUS a second turn (seqs 6,7). The
|
||||
// suffix (constructor seed, never emits session/event) must be persisted on
|
||||
// claim, not lost.
|
||||
const seed = [
|
||||
...oneTurnLog(),
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
] as SessionEvent[]
|
||||
let cont!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
cont = inner.sessions.create('suffix-claim', { seed, meta: { cwd: '/a' } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await backend.inits.get(cont)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('suffix-claim'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
})
|
||||
|
||||
it('claiming cursor-0 ownerless state persists the whole constructor seed', async () => {
|
||||
// create() registers ownerless state with cursor 0 (lazy, nothing on disk).
|
||||
await ctx.sessionPersistence.create(meta('lazy-seed', '/a'))
|
||||
const backend = ctx.sessionPersistence as unknown as { inits: Map<Session, Promise<void>> }
|
||||
// A live session seeded with a full turn claims it; the whole seed (cursor
|
||||
// is 0) must be persisted.
|
||||
let cont!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
cont = inner.sessions.create('lazy-seed', { seed: oneTurnLog(), meta: { cwd: '/a' } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await backend.inits.get(cont)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('lazy-seed'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5])
|
||||
})
|
||||
|
||||
it('a seed with matching seq/type/time but DIFFERENT data is rejected (deep prefix compare)', async () => {
|
||||
// Materialize and load (ownerless, cursor = 6).
|
||||
await ctx.sessionPersistence.create(meta('divergent', '/a'))
|
||||
@@ -942,14 +523,6 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
.rejects.toThrow(/already bound to a different live session|already has a persisted log|do not match/)
|
||||
})
|
||||
|
||||
it('round-trips a header with parentSession (fork lineage)', async () => {
|
||||
const m: SessionHeader = { version: 1, id: SessionId('forked-child'), createdAt: 1, parentSession: SessionId('the-parent') }
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.meta.parentSession).toBe('the-parent')
|
||||
})
|
||||
|
||||
it('list returns nothing when the root directory does not exist', async () => {
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
@@ -1025,48 +598,6 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
expect(end.type === 'turn/end' && end.data.reason).toEqual({ kind: 'interrupted' })
|
||||
})
|
||||
|
||||
it('initFor is idempotent: a re-seeded existing session is not re-initialized', async () => {
|
||||
const session = ctx.sessions.create('idem', { meta: { cwd: '/i' } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
// Re-emit session/created for the SAME live session (idempotent initFor).
|
||||
ctx.emit('session/created', session)
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('idem'))
|
||||
expect(loaded.events).toHaveLength(2) // not doubled
|
||||
})
|
||||
|
||||
|
||||
it('flush before init resolves with no state uses cursor 0', async () => {
|
||||
// Drive a fork (seed) flush where the buffer holds the seed; the fresh
|
||||
// events filter against cursor. Exercises the state-undefined cursor path.
|
||||
const session = ctx.sessions.create('flush-nostate')
|
||||
// Append directly to the live session and flush IMMEDIATELY, before the
|
||||
// async onCreated init has necessarily set state.
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'q' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('flush-nostate'))
|
||||
expect(loaded.events).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('createCore rejects creating an id this backend already tracks', async () => {
|
||||
await ctx.sessionPersistence.create(meta('dup'))
|
||||
await expect(ctx.sessionPersistence.create(meta('dup'))).rejects.toThrow(/already exists in this backend/)
|
||||
})
|
||||
|
||||
it('createCore rejects creating an id whose log already exists on disk', async () => {
|
||||
const m = meta('on-disk', '/od')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
// A fresh backend (no in-memory state) must refuse to create over the log.
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root })
|
||||
await expect(ctx2.sessionPersistence.create(meta('on-disk', '/od'))).rejects.toThrow(/already has a persisted log on disk/)
|
||||
await ctx2.fiber.dispose()
|
||||
})
|
||||
|
||||
it('createCore rejects an id already on disk under a DIFFERENT cwd bucket', async () => {
|
||||
// Persist the id under cwd A.
|
||||
|
||||
@@ -1,19 +1,18 @@
|
||||
/**
|
||||
* SQLite durable session-persistence backend (`@deepseek-ai/dsh-session-persistence-sqlite`).
|
||||
*
|
||||
* A SECOND {@link SessionPersistence} implementation, built to validate that
|
||||
* the abstract seam + the shared `runPersistenceContract` suite are genuinely
|
||||
* A SECOND {@link SessionPersistence} implementation, built to validate that the
|
||||
* abstract seam + the shared `runPersistenceContract` suite are genuinely
|
||||
* backend-agnostic: the same append-only / contiguous-seq / lazy-materialization
|
||||
* / interrupted-turn-close-on-load semantics the JSONL backend expresses over
|
||||
* file bytes, expressed here over `node:sqlite` rows. Each `SessionEvent` maps
|
||||
* 1:1 onto a row `(session_id, seq, type, time, data)`; `append` is an INSERT
|
||||
* inside a transaction that asserts the contiguous-seq contract.
|
||||
* 1:1 onto a row `(session_id, seq, type, time, data)`.
|
||||
*
|
||||
* Like the JSONL backend it is also the write-path plugin: it installs the
|
||||
* `session/event` → buffer → `session/flush` drain, persists a fork's seed once
|
||||
* on `session/created`, keeps a per-session write cursor so a resumed session
|
||||
* never re-appends stored events, and seeds existing live sessions on apply
|
||||
* (HMR does not replay `session/created`).
|
||||
* Like the JSONL backend it supplies ONLY the storage primitives (the
|
||||
* {@link PersistenceBackend} hooks below — INSERT/DELETE/SELECT inside
|
||||
* transactions); all the write-path orchestration lives in the backend-agnostic
|
||||
* {@link PersistenceCoordinator} this class composes. The six public
|
||||
* {@link SessionPersistence} methods delegate to the coordinator.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-session-persistence-sqlite
|
||||
*/
|
||||
@@ -24,9 +23,9 @@ import { DatabaseSync } from 'node:sqlite'
|
||||
import { mkdir } from 'node:fs/promises'
|
||||
import { dirname, resolve } from 'node:path'
|
||||
import {
|
||||
SessionPersistence, assertSerializable, seedCoversPrefix,
|
||||
SessionPersistence, PersistenceCoordinator,
|
||||
type PersistenceBackend, type StoredPrefix,
|
||||
} from '@deepseek-ai/dsh-session-persistence'
|
||||
import { interruptedTurnClosers } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
openDatabase, rowToMeta, scanRows, type EventRow, type SessionRow,
|
||||
@@ -44,55 +43,36 @@ export interface Config {
|
||||
path: string
|
||||
}
|
||||
|
||||
/** Backend bookkeeping for a session id (NOT the live Session object). */
|
||||
interface SessionState {
|
||||
meta: SessionHeader
|
||||
/** Next seq to write — equals the number of committed events. */
|
||||
cursor: number
|
||||
/** Whether the session has at least one persisted event (materialized). */
|
||||
materialized: boolean
|
||||
/** The live Session that owns this state (collision detection); see onCreated. */
|
||||
owner?: Session
|
||||
}
|
||||
|
||||
async function settledErrors(promises: Iterable<Promise<unknown>>): Promise<unknown[]> {
|
||||
const settled = await Promise.allSettled([...promises])
|
||||
const errors: unknown[] = []
|
||||
for (const result of settled) {
|
||||
if (result.status === 'rejected') errors.push(result.reason)
|
||||
}
|
||||
return errors
|
||||
}
|
||||
|
||||
/**
|
||||
* The SQLite persistence backend. Load as a plugin; it registers as
|
||||
* `ctx.sessionPersistence` and installs the write-path listeners.
|
||||
* `ctx.sessionPersistence` and (via the coordinator) installs the write-path
|
||||
* listeners. Its torn-tail marker is the seq to delete from.
|
||||
*/
|
||||
export class SessionPersistenceSqlite extends SessionPersistence {
|
||||
export class SessionPersistenceSqlite extends SessionPersistence implements PersistenceBackend<number> {
|
||||
static inject = ['sessions']
|
||||
|
||||
static Config: z<Config> = z.object({
|
||||
path: z.string().required(),
|
||||
})
|
||||
|
||||
/**
|
||||
* Backend label for the coordinator's dispose diagnostics. Intentionally
|
||||
* shadows cordis `Service.name` (set to `'sessionPersistence'` by the base);
|
||||
* see the JSONL backend for why this does not affect service resolution.
|
||||
*/
|
||||
override readonly name = 'session-persistence-sqlite'
|
||||
|
||||
private db!: DatabaseSync
|
||||
private ready: Promise<void>
|
||||
/** Backend bookkeeping keyed by session id (NOT the live Session object). */
|
||||
private states = new Map<string, SessionState>()
|
||||
/** Write-behind buffers keyed by the live Session (write path). */
|
||||
private buffers = new Map<Session, SessionEvent[]>()
|
||||
/** Per-session serialization chain (keyed by session id). */
|
||||
private chains = new Map<string, Promise<unknown>>()
|
||||
/** Per-session init promise (onCreated), keyed by the LIVE Session object. */
|
||||
private inits = new Map<Session, Promise<void>>()
|
||||
private coordinator: PersistenceCoordinator<number>
|
||||
|
||||
constructor(ctx: Context, public config: Config) {
|
||||
super(ctx)
|
||||
// Open the database asynchronously (the parent directory may need creating);
|
||||
// every backend op awaits `ready` first. Opening synchronously in the ctor
|
||||
// would force a sync mkdir and block plugin apply.
|
||||
// every hook awaits `ready` first. Opening synchronously would force a sync
|
||||
// mkdir and block plugin apply.
|
||||
this.ready = this.openDb(config.path)
|
||||
this.installWritePath()
|
||||
this.coordinator = new PersistenceCoordinator<number>(this.ctx, this)
|
||||
}
|
||||
|
||||
private async openDb(path: string): Promise<void> {
|
||||
@@ -105,229 +85,156 @@ export class SessionPersistenceSqlite extends SessionPersistence {
|
||||
}
|
||||
}
|
||||
|
||||
// --- SessionPersistence backend surface (all serialized per session id) ---
|
||||
// --- SessionPersistence service surface (delegated to the coordinator) ---
|
||||
|
||||
create(meta: SessionHeader): Promise<void> {
|
||||
const snapshot: SessionHeader = { ...meta }
|
||||
return this.serialize(snapshot.id, () => this.createCore(snapshot))
|
||||
return this.coordinator.create(meta)
|
||||
}
|
||||
|
||||
private async createCore(meta: SessionHeader): Promise<void> {
|
||||
append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
return this.coordinator.append(id, events)
|
||||
}
|
||||
|
||||
load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
return this.coordinator.load(id)
|
||||
}
|
||||
|
||||
has(id: SessionId): Promise<boolean> {
|
||||
return this.coordinator.has(id)
|
||||
}
|
||||
|
||||
delete(id: SessionId): Promise<void> {
|
||||
return this.coordinator.delete(id)
|
||||
}
|
||||
|
||||
// `list` is BOTH the public service method and the PersistenceBackend hook —
|
||||
// one method (the SELECT below). The coordinator adds no orchestration for
|
||||
// listing, so routing it through the coordinator would just recurse. Defined
|
||||
// once, in the "PersistenceBackend hooks" section.
|
||||
|
||||
/**
|
||||
* The per-session init promises, exposed for white-box tests that await a
|
||||
* specific session's onCreated (there is no public API to await one init).
|
||||
*/
|
||||
get inits(): Map<Session, Promise<void>> {
|
||||
return this.coordinator.inits
|
||||
}
|
||||
|
||||
// --- PersistenceBackend hooks (the SQLite storage primitives) ---
|
||||
|
||||
/** Read a stored prefix by id (ids are globally unique — no scope to scan). */
|
||||
loadStored(id: SessionId): Promise<StoredPrefix<number> | undefined> {
|
||||
return this.readPrefix(id)
|
||||
}
|
||||
|
||||
/** Read a stored prefix; `cwd` is ignored (the id is globally unique in SQLite). */
|
||||
loadLive(id: SessionId, _cwd: string | undefined): Promise<StoredPrefix<number> | undefined> {
|
||||
return this.readPrefix(id)
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a session's row + ordered events into a {@link StoredPrefix}. The
|
||||
* torn-tail marker is the seq from which a never-committed tail must be deleted
|
||||
* (`scanRows` already returns it as `number | undefined`).
|
||||
*/
|
||||
private async readPrefix(id: SessionId): Promise<StoredPrefix<number> | undefined> {
|
||||
await this.ready
|
||||
if (this.states.has(meta.id)) {
|
||||
throw new Error(`session "${meta.id}" already exists in this backend`)
|
||||
}
|
||||
if (this.rowFor(meta.id) !== undefined) {
|
||||
throw new Error(`session "${meta.id}" already has a persisted row; load/resume it instead of creating`)
|
||||
}
|
||||
// Lazy: record intent in memory only. No row until the first append, so an
|
||||
// abandoned (never-appended) session leaves nothing behind and stays absent
|
||||
// from has()/list().
|
||||
this.states.set(meta.id, { meta, cursor: 0, materialized: false })
|
||||
const row = this.rowFor(id)
|
||||
if (row === undefined) return undefined
|
||||
const meta = rowToMeta(row)
|
||||
const eventRows = this.db
|
||||
.prepare('SELECT seq, type, time, data FROM events WHERE session_id = ? ORDER BY seq')
|
||||
.all(id) as unknown as EventRow[]
|
||||
const { preserved, tornFrom } = scanRows(eventRows)
|
||||
return { meta, events: preserved, ...tornFrom !== undefined ? { tornMarker: tornFrom } : {} }
|
||||
}
|
||||
|
||||
// `async` so the synchronous validate/clone below reject (not throw) per the
|
||||
// Promise<void> contract — callers use `await expect(...).rejects`.
|
||||
async append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
// Validate serializability BEFORE cloning so a bad event surfaces the typed
|
||||
// "non-JSON-serializable" error rather than an opaque DataCloneError from
|
||||
// structuredClone. Then deep-snapshot the batch HERE, before the op waits
|
||||
// behind the per-session chain: a caller that passes a live array (e.g.
|
||||
// session.events) and mutates it — OR mutates an event inside it — before
|
||||
// the op runs would otherwise have those changes persisted, or advance the
|
||||
// cursor past what was written. The clone is taken at call time (before the
|
||||
// first await), matching the JSONL backend.
|
||||
assertSerializable(events)
|
||||
const batch = events.map(e => structuredClone(e))
|
||||
return this.serialize(id, () => this.appendCore(id, batch))
|
||||
}
|
||||
|
||||
private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
/**
|
||||
* Durably append a batch in ONE transaction: materialize the sessions row (if
|
||||
* lazy) and INSERT every event, or roll back entirely. The transaction is the
|
||||
* atomicity + durability boundary, so a mid-batch failure (a UNIQUE violation
|
||||
* on a duplicated seq) leaves the stored log untouched.
|
||||
*/
|
||||
async appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise<void> {
|
||||
await this.ready
|
||||
if (events.length === 0) return
|
||||
let state = this.states.get(id)
|
||||
if (state === undefined) state = await this.adopt(id)
|
||||
|
||||
// Contiguity contract: each event's seq must continue the stored log.
|
||||
for (const [i, event] of events.entries()) {
|
||||
if (event.seq !== state.cursor + i) {
|
||||
throw new Error(`append seq mismatch for "${id}": expected ${state.cursor + i} at index ${i}, got ${event.seq}`)
|
||||
}
|
||||
}
|
||||
|
||||
// The transaction is the durability + atomicity boundary: materialize the
|
||||
// sessions row (if lazy) and INSERT every event, or roll back entirely. A
|
||||
// BEGIN/COMMIT around the batch means a mid-batch failure (a UNIQUE
|
||||
// violation on a duplicated seq from a concurrent writer) leaves the stored
|
||||
// log untouched, so the cursor stays truthful and a retry is clean. (A crash
|
||||
// tail is already gone: load() physically deletes the torn fragment and
|
||||
// durably closes the interrupted turn before returning, so by the time any
|
||||
// append runs the stored log is balanced and contiguous.)
|
||||
const insertEvent = this.db.prepare(
|
||||
'INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)',
|
||||
)
|
||||
this.db.exec('BEGIN')
|
||||
try {
|
||||
if (!state.materialized) this.writeRow(state.meta)
|
||||
if (!isMaterialized) this.writeRow(meta)
|
||||
for (const event of events) {
|
||||
insertEvent.run(id, event.seq, event.type, event.time, JSON.stringify(event.data))
|
||||
insertEvent.run(meta.id, event.seq, event.type, event.time, JSON.stringify(event.data))
|
||||
}
|
||||
this.db.exec('COMMIT')
|
||||
} catch (error) {
|
||||
this.db.exec('ROLLBACK')
|
||||
throw error
|
||||
}
|
||||
state.materialized = true
|
||||
state.cursor += events.length
|
||||
}
|
||||
|
||||
load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
return this.serialize(id, () => this.loadCore(id))
|
||||
}
|
||||
|
||||
private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
/**
|
||||
* Make a crash repair durable in ONE transaction: DELETE the torn tail (from
|
||||
* `tornMarker`) and INSERT the synthetic `closers`. After COMMIT the stored rows
|
||||
* == the balanced log.
|
||||
*/
|
||||
async commitRepair(meta: SessionHeader, tornMarker: number | undefined, closers: readonly SessionEvent[]): Promise<void> {
|
||||
await this.ready
|
||||
const row = this.rowFor(id)
|
||||
if (row === undefined) throw new Error(`session "${id}" not found`)
|
||||
const meta = rowToMeta(row)
|
||||
this.assertVersion(meta)
|
||||
|
||||
// Read every stored row ordered by seq, then scan for the preserved prefix:
|
||||
// the longest seq-contiguous, parseable run, INCLUDING the real events of an
|
||||
// interrupted final turn after the last turn/end (a turn can be huge — they
|
||||
// are never truncated). scanRows works off the seq+type COLUMNS for the
|
||||
// last-turn/end boundary, so a malformed `data` in a torn tail row is
|
||||
// discarded (not unloadable); only a parse error / seq gap in the COMMITTED
|
||||
// region (at or before the last turn/end) throws (genuine corruption).
|
||||
const eventRows = this.db
|
||||
.prepare('SELECT seq, type, time, data FROM events WHERE session_id = ? ORDER BY seq')
|
||||
.all(id) as unknown as EventRow[]
|
||||
const { preserved, tornFrom } = scanRows(eventRows)
|
||||
|
||||
// Crash-recovery (mutating load, same as the JSONL backend): if the log ended
|
||||
// mid-turn, close it DURING load so disk, the returned log, and the cursor all
|
||||
// agree — both append routes then continue with no special-casing. Synthesize
|
||||
// the boundary events (a step/end if a step was open, then a
|
||||
// turn/end {kind:'interrupted'}); the interrupted turn's real events are
|
||||
// preserved, never truncated (the session-persistence RFC).
|
||||
const closers = interruptedTurnClosers(preserved)
|
||||
const balanced = [...preserved, ...closers]
|
||||
|
||||
// Physically repair the stored log inside one transaction: DELETE the torn
|
||||
// tail fragment (if any), then INSERT the synthetic closers. After COMMIT the
|
||||
// stored rows == balanced, so the cursor is truthful and the next append
|
||||
// continues cleanly with no deferred repair. The metadata row stays as-is
|
||||
// even when preserved.length === 0 (an all-tail crash): the session WAS
|
||||
// materialized by the partial append, so has()/list() still report it — the
|
||||
// same as the JSONL backend, whose file likewise survives a first append that
|
||||
// never reached turn/end.
|
||||
if (tornFrom !== undefined || closers.length > 0) {
|
||||
this.db.exec('BEGIN')
|
||||
try {
|
||||
if (tornFrom !== undefined) {
|
||||
this.db.prepare('DELETE FROM events WHERE session_id = ? AND seq >= ?').run(id, tornFrom)
|
||||
}
|
||||
if (closers.length > 0) {
|
||||
const insertEvent = this.db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)')
|
||||
for (const event of closers) {
|
||||
insertEvent.run(id, event.seq, event.type, event.time, JSON.stringify(event.data))
|
||||
}
|
||||
}
|
||||
this.db.exec('COMMIT')
|
||||
} catch (error) {
|
||||
// The DELETE+INSERT cannot collide (a row at a closer's seq is preserved
|
||||
// or deleted as torn first); this rolls back a DB-level failure (disk
|
||||
// full, etc.), unreachable in test.
|
||||
/* v8 ignore start */
|
||||
this.db.exec('ROLLBACK')
|
||||
throw error
|
||||
/* v8 ignore stop */
|
||||
this.db.exec('BEGIN')
|
||||
try {
|
||||
if (tornMarker !== undefined) {
|
||||
this.db.prepare('DELETE FROM events WHERE session_id = ? AND seq >= ?').run(meta.id, tornMarker)
|
||||
}
|
||||
if (closers.length > 0) {
|
||||
const insertEvent = this.db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)')
|
||||
for (const event of closers) {
|
||||
insertEvent.run(meta.id, event.seq, event.type, event.time, JSON.stringify(event.data))
|
||||
}
|
||||
}
|
||||
this.db.exec('COMMIT')
|
||||
} catch (error) {
|
||||
// The DELETE+INSERT cannot collide (a row at a closer's seq is preserved or
|
||||
// deleted as torn first); this rolls back a DB-level failure (disk full,
|
||||
// etc.), unreachable in test.
|
||||
/* v8 ignore start */
|
||||
this.db.exec('ROLLBACK')
|
||||
throw error
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
|
||||
// Record state at the balanced length. The state keeps its OWN copy of the
|
||||
// meta; the returned value is separate so a consumer mutating loaded.meta
|
||||
// cannot corrupt the backend's row metadata.
|
||||
this.states.set(id, {
|
||||
meta: { ...meta },
|
||||
cursor: balanced.length,
|
||||
materialized: true,
|
||||
})
|
||||
return { meta, events: balanced }
|
||||
}
|
||||
|
||||
private async adoptLiveStoredPrefix(session: Session, seed: readonly SessionEvent[]): Promise<void> {
|
||||
/** Remove a session's row (ON DELETE CASCADE drops its events). */
|
||||
async deleteStored(id: SessionId): Promise<void> {
|
||||
await this.ready
|
||||
const row = this.rowFor(session.header.id)
|
||||
/* v8 ignore next -- caller checked row existence */
|
||||
if (row === undefined) throw new Error(`session "${session.header.id}" not found`)
|
||||
const meta = rowToMeta(row)
|
||||
this.assertVersion(meta)
|
||||
|
||||
const rows = this.db
|
||||
.prepare('SELECT seq, type, time, data FROM events WHERE session_id = ? ORDER BY seq')
|
||||
.all(session.header.id) as unknown as EventRow[]
|
||||
const { preserved, tornFrom } = scanRows(rows)
|
||||
if (!seedCoversPrefix(seed, preserved)) {
|
||||
throw new Error(`session "${session.header.id}" already has a persisted log that does not match this live session (id collision)`)
|
||||
}
|
||||
|
||||
if (tornFrom !== undefined) {
|
||||
this.db.prepare('DELETE FROM events WHERE session_id = ? AND seq >= ?').run(session.header.id, tornFrom)
|
||||
}
|
||||
this.states.set(session.header.id, {
|
||||
meta: { ...meta },
|
||||
cursor: preserved.length,
|
||||
materialized: true,
|
||||
owner: session,
|
||||
})
|
||||
const suffix = seed.slice(preserved.length)
|
||||
if (suffix.length > 0) await this.appendCore(session.header.id, suffix)
|
||||
this.db.prepare('DELETE FROM sessions WHERE id = ?').run(id)
|
||||
}
|
||||
|
||||
/** List all materialized sessions' metadata (every row is a materialized session). */
|
||||
async list(): Promise<SessionHeader[]> {
|
||||
await this.ready
|
||||
// Every metadata row is a materialized session: the row is written only by
|
||||
// the first append (a created-but-never-appended session has no row), so
|
||||
// listing all rows is exactly the materialized set.
|
||||
const rows = this.db
|
||||
.prepare('SELECT * FROM sessions')
|
||||
.all() as unknown as SessionRow[]
|
||||
return rows.map(rowToMeta)
|
||||
}
|
||||
|
||||
async has(id: SessionId): Promise<boolean> {
|
||||
/** Close the database handle (awaited by the coordinator's dispose, post-drain). */
|
||||
async close(): Promise<void> {
|
||||
await this.ready
|
||||
const state = this.states.get(id)
|
||||
if (state?.materialized) return true
|
||||
// A metadata row exists iff the session was materialized by a first append.
|
||||
return this.rowFor(id) !== undefined
|
||||
}
|
||||
|
||||
delete(id: SessionId): Promise<void> {
|
||||
return this.serialize(id, () => this.deleteCore(id))
|
||||
}
|
||||
|
||||
private async deleteCore(id: SessionId): Promise<void> {
|
||||
await this.ready
|
||||
// ON DELETE CASCADE drops the session's events with its row.
|
||||
this.db.prepare('DELETE FROM sessions WHERE id = ?').run(id)
|
||||
this.states.delete(id)
|
||||
this.db.close()
|
||||
}
|
||||
|
||||
// --- row helpers ---
|
||||
|
||||
/** Fetch a session's row, or undefined if absent. */
|
||||
private rowFor(id: SessionId): SessionRow | undefined {
|
||||
const row = this.db.prepare('SELECT * FROM sessions WHERE id = ?').get(id) as unknown as SessionRow | undefined
|
||||
return row
|
||||
return this.db.prepare('SELECT * FROM sessions WHERE id = ?').get(id) as unknown as SessionRow | undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* Insert-or-replace a session's metadata row. The only caller is the first
|
||||
* materializing `append`, so writing the row IS the materialization (its
|
||||
* existence is the signal `has`/`list` read); a never-appended session has no
|
||||
* row at all.
|
||||
* materializing `appendBatch`, so writing the row IS the materialization (its
|
||||
* existence is the signal `has`/`list` read).
|
||||
*/
|
||||
private writeRow(meta: SessionHeader): void {
|
||||
this.db.prepare(`
|
||||
@@ -346,190 +253,6 @@ export class SessionPersistenceSqlite extends SessionPersistence {
|
||||
meta.parentSession ?? null,
|
||||
)
|
||||
}
|
||||
|
||||
/** Build a state for a session present in the DB but not yet in memory. */
|
||||
private async adopt(id: SessionId): Promise<SessionState> {
|
||||
await this.loadCore(id) // sets the state; load (serialized) would deadlock
|
||||
const state = this.states.get(id)
|
||||
/* v8 ignore next -- loadCore always sets the state for the id */
|
||||
if (!state) throw new Error(`failed to adopt session "${id}"`)
|
||||
return state
|
||||
}
|
||||
|
||||
private assertVersion(meta: SessionHeader): void {
|
||||
if (meta.version !== 1) {
|
||||
throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run `op` after any in-flight operation for the same session id, so writes
|
||||
* for one session never interleave. Errors do not poison the chain. NOTE:
|
||||
* serialized public methods must NOT call each other (deadlock); they call
|
||||
* the unserialized `*Core` helpers instead.
|
||||
*/
|
||||
private serialize<T>(id: SessionId, op: () => Promise<T>): Promise<T> {
|
||||
const prior = this.chains.get(id) ?? Promise.resolve()
|
||||
const next = prior.then(op, op)
|
||||
this.chains.set(id, next.then(() => undefined, () => undefined))
|
||||
return next
|
||||
}
|
||||
|
||||
// --- write path (session/event → flush drain) ---
|
||||
|
||||
private installWritePath(): void {
|
||||
const ctx = this.ctx
|
||||
|
||||
ctx.on('session/created', (session) => { void this.initFor(session) })
|
||||
|
||||
// Snapshot + buffer every event (the live object is mutable; clone so a
|
||||
// later in-place mutation cannot rewrite a buffered event). Serializability
|
||||
// is guaranteed at the source (Session.append), so structuredClone is safe.
|
||||
ctx.on('session/event', (session, event) => {
|
||||
let buffer = this.buffers.get(session)
|
||||
if (!buffer) this.buffers.set(session, buffer = [])
|
||||
buffer.push(structuredClone(event))
|
||||
})
|
||||
|
||||
ctx.on('session/flush', session => this.flush(session))
|
||||
|
||||
// Dispose must reach quiescence: await every init + final drain, then close
|
||||
// the database, BEFORE returning, so no write lands after teardown.
|
||||
ctx.effect(() => async () => {
|
||||
let disposeError: unknown
|
||||
try {
|
||||
const errors = [
|
||||
...await settledErrors(this.inits.values()),
|
||||
...await settledErrors([...this.buffers.keys()].map(s => this.flush(s))),
|
||||
...await settledErrors(this.chains.values()),
|
||||
]
|
||||
if (errors.length > 0) {
|
||||
throw new AggregateError(errors, 'session-persistence-sqlite dispose failed')
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
disposeError = error
|
||||
throw error
|
||||
} finally {
|
||||
try {
|
||||
await this.ready
|
||||
this.db.close()
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next -- open/close failure racing disposal is a defensive teardown edge */
|
||||
if (disposeError === undefined) throw error
|
||||
// Opening/closing the database can only add teardown context here; keep
|
||||
// the already-captured init/flush/chain AggregateError as the primary
|
||||
// disposal failure instead of masking it from callers.
|
||||
}
|
||||
}
|
||||
}, 'session-persistence-sqlite write path')
|
||||
|
||||
// HMR: a hot reload does not replay session/created, so seed existing live
|
||||
// sessions (mirrors dsh-invariants and the JSONL backend).
|
||||
for (const session of ctx.sessions.list()) void this.initFor(session)
|
||||
}
|
||||
|
||||
/** Start (once) the async init for a session and remember its promise. */
|
||||
private initFor(session: Session): Promise<void> {
|
||||
const existing = this.inits.get(session)
|
||||
if (existing) return existing
|
||||
const seed = session.events.map(e => structuredClone(e))
|
||||
const p = this.onCreated(session, seed)
|
||||
p.catch(() => { /* observed by flush/dispose via the stored promise */ })
|
||||
this.inits.set(session, p)
|
||||
return p
|
||||
}
|
||||
|
||||
/**
|
||||
* On session/created: sync the backend's state to a live Session. Cases
|
||||
* mirror the JSONL backend:
|
||||
* 1. Already tracked → no-op (or claim ownerless state if the seed matches).
|
||||
* 2. A row EXISTS and is a seq-aligned PREFIX of the live events → adopt
|
||||
* (HMR/resume), persisting any live suffix beyond the stored prefix.
|
||||
* 3. A row EXISTS but is NOT a prefix → reject (id collision).
|
||||
* 4. No row → a genuinely new session: register meta (lazy) + persist seed.
|
||||
*/
|
||||
private async onCreated(session: Session, seed: readonly SessionEvent[]): Promise<void> {
|
||||
await this.ready
|
||||
const id = session.header.id
|
||||
const tracked = this.states.get(id)
|
||||
if (tracked !== undefined) {
|
||||
/* v8 ignore next -- initFor dedupes per session object; same-object re-entry can't occur */
|
||||
if (tracked.owner === session) return
|
||||
if (tracked.owner === undefined) {
|
||||
// Ownerless state from a public create()/load(). The first live session
|
||||
// claims it ONLY if its seed reproduces the persisted prefix.
|
||||
if (!await this.seedMatchesPersisted(id, seed, tracked.cursor)) {
|
||||
throw new Error(`session "${id}" is already persisted with ${tracked.cursor} event(s) that do not match this live session (id collision)`)
|
||||
}
|
||||
tracked.owner = session
|
||||
const suffix = seed.slice(tracked.cursor)
|
||||
if (suffix.length > 0) await this.append(id, suffix)
|
||||
return
|
||||
}
|
||||
// Owned by a DIFFERENT live session. Reclaim ONLY a truly-abandoned id
|
||||
// (never materialized, no pending buffer); else it is a real collision.
|
||||
const ownerBuffer = this.buffers.get(tracked.owner)
|
||||
if (!tracked.materialized && !ownerBuffer?.length) {
|
||||
this.states.delete(id)
|
||||
} else {
|
||||
throw new Error(`session "${id}" is already bound to a different live session in this backend (id collision)`)
|
||||
}
|
||||
}
|
||||
|
||||
const row = this.rowFor(id)
|
||||
if (row !== undefined) {
|
||||
// Adopt a LIVE prefix without crash-repairing an open turn as interrupted;
|
||||
// HMR may still append the real completion from the live Session.
|
||||
await this.serialize(id, () => this.adoptLiveStoredPrefix(session, seed))
|
||||
return
|
||||
}
|
||||
|
||||
// case 4: a genuinely new session.
|
||||
const meta: SessionHeader = { ...session.header }
|
||||
await this.create(meta)
|
||||
const created = this.states.get(id)
|
||||
/* v8 ignore next -- create() always sets the state for the id */
|
||||
if (created !== undefined) created.owner = session
|
||||
if (seed.length > 0) await this.append(id, seed)
|
||||
}
|
||||
|
||||
/** The preserved events for a session id (torn tail excluded, turn NOT yet closed). */
|
||||
private eventsFor(id: SessionId): SessionEvent[] {
|
||||
const rows = this.db
|
||||
.prepare('SELECT seq, type, time, data FROM events WHERE session_id = ? ORDER BY seq')
|
||||
.all(id) as unknown as EventRow[]
|
||||
// Scan on seq+type columns, parsing `data` only for the preserved prefix (a
|
||||
// malformed torn tail must not throw here — same as loadCore). Returns the
|
||||
// preserved events WITHOUT the synthetic closers, so a collision check
|
||||
// compares a live seed against the real on-disk events, mirroring the JSONL
|
||||
// backend's scanLog use in onCreated.
|
||||
return scanRows(rows).preserved
|
||||
}
|
||||
|
||||
/** Whether a live session's seed reproduces the first `cursor` stored events. */
|
||||
private async seedMatchesPersisted(id: SessionId, seed: readonly SessionEvent[], cursor: number): Promise<boolean> {
|
||||
await this.ready
|
||||
if (cursor === 0) return true
|
||||
return seedCoversPrefix(seed, this.eventsFor(id).slice(0, cursor))
|
||||
}
|
||||
|
||||
private async flush(session: Session): Promise<void> {
|
||||
await this.inits.get(session)
|
||||
await this.serialize(session.header.id, () => this.drain(session))
|
||||
}
|
||||
|
||||
/** Drain a session's write buffer to the database. Caller serializes per id. */
|
||||
private async drain(session: Session): Promise<void> {
|
||||
const buffer = this.buffers.get(session)
|
||||
if (!buffer?.length) return
|
||||
const batch = buffer.slice()
|
||||
const state = this.states.get(session.header.id)
|
||||
/* v8 ignore next -- state is always set by the awaited init before flush */
|
||||
const cursor = state?.cursor ?? 0
|
||||
const fresh = batch.filter(e => e.seq >= cursor)
|
||||
if (fresh.length > 0) await this.appendCore(session.header.id, fresh)
|
||||
buffer.splice(0, batch.length)
|
||||
}
|
||||
}
|
||||
|
||||
export default SessionPersistenceSqlite
|
||||
|
||||
@@ -3,11 +3,12 @@ import { Context } from 'cordis'
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import SessionStore from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistenceSqlite, { SCHEMA_VERSION } from '@deepseek-ai/dsh-session-persistence-sqlite'
|
||||
import { openDatabase, scanRows, type EventRow } from '../src/schema.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts'
|
||||
import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts'
|
||||
|
||||
const dirs: string[] = []
|
||||
afterEach(async () => { for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) })
|
||||
@@ -38,6 +39,31 @@ runPersistenceContract('sqlite', async () => {
|
||||
}
|
||||
})
|
||||
|
||||
// Run the shared coordinator orchestration suite against the real SQLite backend.
|
||||
// A FILE-backed db (not :memory:) is the shared storage scope so two mounted
|
||||
// instances see the same rows (HMR/reload). `corruptTail` INSERTs a row past the
|
||||
// committed seq whose `data` is invalid JSON — a never-committed torn tail that
|
||||
// drives the coordinator's commitRepair-with-tornMarker branch over real db rows.
|
||||
runCoordinatorContract('sqlite', async (): Promise<CoordinatorFixture> => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-sqlite-coord-'))
|
||||
const path = join(dir, 'sessions.db')
|
||||
return {
|
||||
mount: async ctx => ctx.plugin(SessionPersistenceSqlite, { path }),
|
||||
corruptTail: async (id) => {
|
||||
// A row past the committed region whose `data` does not parse: scanRows
|
||||
// bounds the preserved prefix at it and returns its seq as tornFrom, which
|
||||
// the backend surfaces to the coordinator as the tornMarker to delete from.
|
||||
const db = openDatabase(path)
|
||||
const next = (db.prepare('SELECT COALESCE(MAX(seq), -1) + 1 AS n FROM events WHERE session_id = ?')
|
||||
.get(id) as { n: number }).n
|
||||
db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)')
|
||||
.run(id, next, 'assistant/chunk', 99, '{not valid json')
|
||||
db.close()
|
||||
},
|
||||
cleanup: async () => { await rm(dir, { recursive: true, force: true }) },
|
||||
}
|
||||
})
|
||||
|
||||
describe('scanRows', () => {
|
||||
// scanRows works off EventRows (data is a JSON string column); build them from
|
||||
// SessionEvents so the unit tests read in terms of the event vocabulary.
|
||||
@@ -108,35 +134,6 @@ describe('scanRows', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistenceSqlite: HMR adoption', () => {
|
||||
it('does not crash-repair an active open turn as interrupted', async () => {
|
||||
const path = await freshDbPath()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const first = await ctx.plugin(SessionPersistenceSqlite, { path })
|
||||
const session = ctx.sessions.create('hmr-open', { meta: { cwd: '/hmr' } })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
await first.dispose()
|
||||
const db = openDatabase(path)
|
||||
db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)')
|
||||
.run('hmr-open', 2, 'step/end', 2, '{"torn":')
|
||||
db.close()
|
||||
const second = await ctx.plugin(SessionPersistenceSqlite, { path })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-open'))
|
||||
expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'step/start', 'step/end', 'turn/end'])
|
||||
expect(loaded.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } })
|
||||
await second.dispose()
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistenceSqlite: durability and crash semantics', () => {
|
||||
it('an interrupted turn (rows after the last turn/end) is PRESERVED and closed during load', async () => {
|
||||
const path = await freshDbPath()
|
||||
@@ -251,31 +248,6 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
|
||||
expect(() => openDatabase(olderPath)).toThrow(/incompatible with this build/)
|
||||
})
|
||||
|
||||
it('append snapshots the batch: mutating an event after the call does not corrupt the persisted copy', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' })
|
||||
const m = meta('snapshot')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
const batch: SessionEvent[] = [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'user/message', seq: 1, time: 2, data: { content: [{ type: 'text', text: 'original' }], source: { kind: 'user' } } },
|
||||
{ type: 'turn/end', seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' } } },
|
||||
]
|
||||
const p = ctx.sessionPersistence.append(m.id, batch)
|
||||
// Mutate the live array AND an event's data AFTER the call but before it
|
||||
// drains behind the per-session chain. The snapshot taken at call time must
|
||||
// shield the persisted copy.
|
||||
;(batch[1]!.data as { content: { type: 'text'; text: string }[] }).content[0]!.text = 'HACKED'
|
||||
batch.push({ type: 'user/message', seq: 3, time: 4, data: { content: [{ type: 'text', text: 'injected' }], source: { kind: 'user' } } })
|
||||
await p
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events).toHaveLength(3) // the pushed event was not persisted
|
||||
const um = loaded.events[1]
|
||||
expect(um?.type === 'user/message' && (um.data.content[0] as { text: string }).text).toBe('original')
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('a corrupt-JSON row in the uncommitted tail is discarded on load, not unloadable', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('corrupt-tail')
|
||||
@@ -344,183 +316,12 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
|
||||
await fiber2.dispose()
|
||||
})
|
||||
|
||||
it('rejects an unknown format version on load', async () => {
|
||||
const path = await freshDbPath()
|
||||
// Materialize a row with version 2 directly via the real schema.
|
||||
const db = openDatabase(path)
|
||||
db.prepare('INSERT INTO sessions (id, version, created_at) VALUES (?, ?, ?)')
|
||||
.run('v2', 2, 1)
|
||||
db.close()
|
||||
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path })
|
||||
await expect(ctx.sessionPersistence.load(SessionId('v2'))).rejects.toThrow(/version 2/)
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('create rejects a duplicate id (in memory and on a persisted row)', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('dup')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path })
|
||||
await ctx.sessionPersistence.create(m)
|
||||
// Same in-memory state.
|
||||
await expect(ctx.sessionPersistence.create(m)).rejects.toThrow(/already exists/)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
await fiber.dispose()
|
||||
|
||||
// A fresh instance over the same file sees the persisted row.
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path })
|
||||
await expect(ctx2.sessionPersistence.create(m)).rejects.toThrow(/already has a persisted row/)
|
||||
await fiber2.dispose()
|
||||
})
|
||||
|
||||
it('exposes the schema version constant', () => {
|
||||
expect(SCHEMA_VERSION).toBe(2)
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistenceSqlite: write path (session/event → flush)', () => {
|
||||
function send(session: Session, events: SessionEvent[]): void {
|
||||
for (const e of events) session.append(e.type, e.data)
|
||||
}
|
||||
|
||||
it('persists a turn appended through the live session on flush', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' })
|
||||
const session = ctx.sessions.create('w1')
|
||||
send(session, oneTurnLog())
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('w1'))
|
||||
expect(loaded.events.map(e => e.type)).toEqual(oneTurnLog().map(e => e.type))
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('a resumed session does not re-append its seed', async () => {
|
||||
const path = await freshDbPath()
|
||||
// Run 1: persist a full turn through the live session.
|
||||
const ctx1 = new Context()
|
||||
await ctx1.plugin(SessionStore)
|
||||
const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path })
|
||||
const s1 = ctx1.sessions.create('resume')
|
||||
for (const e of oneTurnLog()) s1.append(e.type, e.data)
|
||||
await ctx1.parallel('session/flush', s1)
|
||||
await fiber1.dispose()
|
||||
|
||||
// Run 2: reconstruct the live session from the loaded log (seed), then add a
|
||||
// second turn. The seed must NOT be re-appended (no UNIQUE collision), and
|
||||
// the second turn continues the seq.
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path })
|
||||
const { events } = await ctx2.sessionPersistence.load(SessionId('resume'))
|
||||
const s2 = ctx2.sessions.create('resume', { seed: events })
|
||||
s2.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
s2.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
await ctx2.parallel('session/flush', s2)
|
||||
const reloaded = await ctx2.sessionPersistence.load(SessionId('resume'))
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
await fiber2.dispose()
|
||||
})
|
||||
|
||||
it('HMR: applying the plugin seeds existing live sessions', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const session = ctx.sessions.create('hmr')
|
||||
for (const e of oneTurnLog()) session.append(e.type, e.data)
|
||||
// Plugin applied AFTER the session already has events.
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' })
|
||||
await ctx.parallel('session/flush', session)
|
||||
expect(await ctx.sessionPersistence.has(SessionId('hmr'))).toBe(true)
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('dispose drains a pending buffer before closing the database', async () => {
|
||||
const path = await freshDbPath()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path })
|
||||
const session = ctx.sessions.create('drain')
|
||||
for (const e of oneTurnLog()) session.append(e.type, e.data)
|
||||
// No explicit flush — dispose must drain the buffer.
|
||||
await fiber.dispose()
|
||||
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path })
|
||||
expect(await ctx2.sessionPersistence.has(SessionId('drain'))).toBe(true)
|
||||
await fiber2.dispose()
|
||||
})
|
||||
|
||||
it('rejects a different live session colliding on a persisted id', async () => {
|
||||
const path = await freshDbPath()
|
||||
const ctx1 = new Context()
|
||||
await ctx1.plugin(SessionStore)
|
||||
const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path })
|
||||
const s1 = ctx1.sessions.create('collide')
|
||||
for (const e of oneTurnLog()) s1.append(e.type, e.data)
|
||||
await ctx1.parallel('session/flush', s1)
|
||||
await fiber1.dispose()
|
||||
|
||||
// A fresh, unrelated session reusing the id (no seed) must be rejected.
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path })
|
||||
const s2 = ctx2.sessions.create('collide')
|
||||
s2.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
await expect(ctx2.parallel('session/flush', s2)).rejects.toThrow(/id collision/)
|
||||
await fiber2.dispose()
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistenceSqlite: edge cases', () => {
|
||||
it('append of an empty batch is a no-op', async () => {
|
||||
const { ctx, dispose } = await backend()
|
||||
const m = meta('empty-batch')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, [])
|
||||
expect(await ctx.sessionPersistence.has(m.id)).toBe(false) // still lazy
|
||||
await dispose()
|
||||
})
|
||||
|
||||
it('load rejects a missing session', async () => {
|
||||
const { ctx, dispose } = await backend()
|
||||
await expect(ctx.sessionPersistence.load(SessionId('nope'))).rejects.toThrow(/not found/)
|
||||
await dispose()
|
||||
})
|
||||
|
||||
it('delete of a non-existent session is a no-op', async () => {
|
||||
const { ctx, dispose } = await backend()
|
||||
await ctx.sessionPersistence.delete(SessionId('ghost'))
|
||||
expect(await ctx.sessionPersistence.has(SessionId('ghost'))).toBe(false)
|
||||
await dispose()
|
||||
})
|
||||
|
||||
it('append adopts a session that exists only in the DB (fresh instance)', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('adopt-append')
|
||||
const b1 = await backend(path)
|
||||
await b1.ctx.sessionPersistence.create(m)
|
||||
await b1.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
await b1.dispose()
|
||||
|
||||
// A fresh instance appends a second turn WITHOUT a prior create/load: append
|
||||
// must adopt the on-disk row (cursor = stored length) and continue the seq.
|
||||
const b2 = await backend(path)
|
||||
await b2.ctx.sessionPersistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
])
|
||||
const loaded = await b2.ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
await b2.dispose()
|
||||
})
|
||||
|
||||
it('append rolls back and rethrows when an event INSERT fails inside the transaction', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('rollback-insert')
|
||||
@@ -549,190 +350,6 @@ describe('SessionPersistenceSqlite: edge cases', () => {
|
||||
await b2.dispose()
|
||||
})
|
||||
|
||||
it('round-trips a header with parentSession (fork lineage)', async () => {
|
||||
const { ctx, dispose } = await backend()
|
||||
const m: SessionHeader = { ...meta('child'), parentSession: SessionId('parent') }
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.meta.parentSession).toBe(SessionId('parent'))
|
||||
await dispose()
|
||||
})
|
||||
|
||||
it('a fresh live session reusing a previously-loaded id is rejected (ownerless guard)', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('ownerless')
|
||||
const b1 = await backend(path)
|
||||
await b1.ctx.sessionPersistence.create(m)
|
||||
await b1.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
await b1.dispose()
|
||||
|
||||
const b2 = await backend(path)
|
||||
// load() leaves ownerless state with cursor 6.
|
||||
await b2.ctx.sessionPersistence.load(m.id)
|
||||
// A fresh, unrelated live session reusing the id has a shorter/non-matching
|
||||
// seed → its onCreated must reject rather than graft onto the loaded prefix.
|
||||
const s = b2.ctx.sessions.create('ownerless')
|
||||
s.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
await expect(b2.ctx.parallel('session/flush', s)).rejects.toThrow(/id collision/)
|
||||
await b2.dispose()
|
||||
})
|
||||
|
||||
it('a live session whose seed matches the loaded prefix claims ownerless state and persists the suffix', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('claim')
|
||||
const b1 = await backend(path)
|
||||
await b1.ctx.sessionPersistence.create(m)
|
||||
await b1.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
await b1.dispose()
|
||||
|
||||
const b2 = await backend(path)
|
||||
const { events } = await b2.ctx.sessionPersistence.load(m.id) // ownerless, cursor 6
|
||||
// A live session seeded with the loaded log PLUS a new turn claims the state
|
||||
// and persists only the suffix.
|
||||
const s = b2.ctx.sessions.create('claim', { seed: [
|
||||
...events,
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
] })
|
||||
await b2.ctx.parallel('session/flush', s)
|
||||
const loaded = await b2.ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
await b2.dispose()
|
||||
})
|
||||
|
||||
it('an abandoned lazy session (never materialized) releases its id for reuse', async () => {
|
||||
const { ctx, dispose } = await backend()
|
||||
const inits = (ctx.sessionPersistence as unknown as { inits: Map<Session, Promise<void>> }).inits
|
||||
let first!: Session
|
||||
const firstFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
first = inner.sessions.create('reuse')
|
||||
}, { inject: ['sessions'] }))
|
||||
await inits.get(first) // let the lazy create register the state
|
||||
await firstFiber.dispose() // disposed before any append → never materialized
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create('reuse')
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(inits.get(reuse)).resolves.toBeUndefined()
|
||||
reuse.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
reuse.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', reuse)
|
||||
expect(await ctx.sessionPersistence.has(SessionId('reuse'))).toBe(true)
|
||||
await dispose()
|
||||
})
|
||||
|
||||
it('does NOT reclaim an id whose abandoned owner still has buffered (unflushed) events', async () => {
|
||||
const { ctx, dispose } = await backend()
|
||||
const inits = (ctx.sessionPersistence as unknown as { inits: Map<Session, Promise<void>> }).inits
|
||||
let first!: Session
|
||||
const firstFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
first = inner.sessions.create('buffered')
|
||||
}, { inject: ['sessions'] }))
|
||||
await inits.get(first)
|
||||
// Append a turn but do NOT flush — events sit in the write-behind buffer.
|
||||
first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
first.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await firstFiber.dispose() // disposed before flush; not materialized, buffer pending
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create('buffered')
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(inits.get(reuse)).rejects.toThrow(/already bound to a different live session/)
|
||||
await dispose()
|
||||
})
|
||||
|
||||
it('initFor is idempotent: re-emitting session/created does not re-initialize', async () => {
|
||||
const { ctx, dispose } = await backend()
|
||||
const session = ctx.sessions.create('idem')
|
||||
ctx.emit('session/created', session) // second create event for the same object
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
expect(await ctx.sessionPersistence.has(SessionId('idem'))).toBe(true)
|
||||
await dispose()
|
||||
})
|
||||
|
||||
it('a live session claims cursor-0 ownerless state created via the public API and persists its seed', async () => {
|
||||
const { ctx, dispose } = await backend()
|
||||
// create() registers ownerless state with cursor 0 (no events yet).
|
||||
await ctx.sessionPersistence.create(meta('cursor0'))
|
||||
// A live session reusing that id, seeded with a turn, claims the ownerless
|
||||
// state (cursor 0 trivially matches any seed) and persists the whole seed.
|
||||
const s = ctx.sessions.create('cursor0', { seed: [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'completed' } } },
|
||||
] })
|
||||
await ctx.parallel('session/flush', s)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('cursor0'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1])
|
||||
await dispose()
|
||||
})
|
||||
|
||||
it('HMR: reloading the backend adopts a still-live, already-materialized session', async () => {
|
||||
const path = await freshDbPath()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
// The session lives in its OWN fiber so it survives the backend reload.
|
||||
let session!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create('hmr-adopt')
|
||||
}, { inject: ['sessions'] }))
|
||||
|
||||
// Backend instance 1 materializes the session on disk.
|
||||
const backend1 = await ctx.plugin(SessionPersistenceSqlite, { path })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Hot-reload: dispose instance 1, plug in instance 2 over the SAME file
|
||||
// while the session stays live. Instance 2 has an empty states map but the
|
||||
// row is materialized on disk and is a prefix of the live events — it must
|
||||
// ADOPT (not reject), and a second turn then persists.
|
||||
await backend1.dispose()
|
||||
await ctx.plugin(SessionPersistenceSqlite, { path })
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
await expect(ctx.parallel('session/flush', session)).resolves.not.toThrow()
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-adopt'))
|
||||
expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2)
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('HMR: adoption persists the live SUFFIX that was ahead of the on-disk prefix', async () => {
|
||||
const path = await freshDbPath()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
let session!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create('hmr-suffix')
|
||||
}, { inject: ['sessions'] }))
|
||||
|
||||
const backend1 = await ctx.plugin(SessionPersistenceSqlite, { path })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Append turn 2 to the LIVE session, then dispose instance 1 WITHOUT
|
||||
// flushing turn 2: it is now ONLY in the live session's events.
|
||||
await backend1.dispose()
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
|
||||
// Instance 2 adopts the on-disk prefix (turn 1) and MUST persist the live
|
||||
// suffix (turn 2) carried in the session's events.
|
||||
await ctx.plugin(SessionPersistenceSqlite, { path })
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-suffix'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3])
|
||||
expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2)
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('HMR: a DIFFERENT session colliding with a materialized on-disk id is rejected', async () => {
|
||||
const path = await freshDbPath()
|
||||
// Instance 1 materializes a session and disposes.
|
||||
|
||||
@@ -21,11 +21,31 @@ The persisted unit IS the existing `SessionEvent` (event-sourced model — the l
|
||||
- **JSON-serializable data.** `append` rejects non-serializable `event.data`; backends snapshot each event when buffering (the live `session.events` object is mutable).
|
||||
- **Durability.** `append` returns only once the batch is durable.
|
||||
|
||||
## The write coordinator
|
||||
|
||||
The two first-party backends were byte-identical (or same-algorithm) for ALL of their write-path orchestration — the in-memory bookkeeping (per-id state, write-behind buffers, per-id serialization chains, per-session init promises), the `session/event` → buffer → `session/flush` drain, lazy materialization, crash-tail repair on load, the four `session/created` adoption cases (new / HMR-adopt / collision / ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives differed (write bytes vs. INSERT rows).
|
||||
|
||||
`PersistenceCoordinator` owns that orchestration once. A first-party backend composes one (`new PersistenceCoordinator(ctx, this)`), implements the small `PersistenceBackend` hook interface, and delegates its six public service methods to the coordinator. This keeps the duplicated, correctness-heavy orchestration in a single place (it used to receive the same fixes twice).
|
||||
|
||||
The `PersistenceBackend<TornMarker>` hooks (the only seam between the coordinator and storage):
|
||||
|
||||
| Hook | Role |
|
||||
|---|---|
|
||||
| `name` | Backend label for the dispose-failure `AggregateError`. |
|
||||
| `loadStored(id)` | Read a stored prefix by id, scanning ANY storage scope. Used by resume/load and, via `!== undefined`, the create-collision probe. Returns an opaque `tornMarker` iff a torn tail must be truncated. |
|
||||
| `loadLive(id, cwd)` | Read a stored prefix SCOPED to `cwd` (HMR live-adoption must only adopt a log at the SAME cwd; a same-id log elsewhere is a collision, not a resume). A globally-unique-id backend ignores `cwd`. |
|
||||
| `appendBatch(meta, events, isMaterialized)` | Durably append a contiguous batch, lazily materializing ATOMICALLY when not yet materialized. |
|
||||
| `commitRepair(meta, tornMarker, closers)` | Make a crash repair durable: truncate the torn tail (iff `tornMarker`) and append `closers`. NOT required to be atomic. Used by load (truncate + closers) and live-adoption (truncate only). |
|
||||
| `deleteStored(id)` / `list()` | Remove a stored artifact / list all stored metadata. |
|
||||
| `close?()` | Optional lifecycle teardown (e.g. close a db handle), awaited after the dispose drain. |
|
||||
|
||||
The `tornMarker` is fully OPAQUE: the coordinator only tests `!== undefined` and round-trips it to `commitRepair`, never inspecting its value (the JSONL backend uses the byte offset to truncate to, the SQLite backend the seq to delete from). The public `SessionPersistence` service shape is unchanged, so a third-party backend MAY still implement the abstract service directly without the coordinator. See [the write-coordinator RFC](../../docs/rfc/implemented/2026-06-18-shared-persistence-write-coordinator.md).
|
||||
|
||||
## Testing backends
|
||||
|
||||
Import `runPersistenceContract` from `tests/contract.ts` and call it with a factory that yields a fresh, empty backend plus a teardown. Every backend is held to the same append-only / contiguous-seq / lazy-materialization / serializability semantics; a backend's own spec adds implementation-specific tests (crash repair, path sanitization) on top.
|
||||
Import `runPersistenceContract` from `tests/contract.ts` (the public-API contract) and `runCoordinatorContract` from `tests/coordinator-contract.ts` (the shared write-path orchestration: adoption, HMR, collision, dispose-drain, crash-tail repair) and call each with a fixture for your backend. Every backend is held to the same append-only / contiguous-seq / lazy-materialization / serializability semantics AND the same orchestration, so a backend's own spec is left with only storage-mechanics tests (path sanitization, fsync rollback; schema version, transaction rollback) on top.
|
||||
|
||||
Two backends run this suite: `dsh-session-persistence-jsonl` (append-only file log) and `dsh-session-persistence-sqlite` (`node:sqlite`, each `SessionEvent` one row `(session_id, seq, type, time, data)`). Both passing the same contract is the proof that the seam is genuinely backend-agnostic — lazy materialization, crash-tail-on-load, and contiguous-seq hold identically over file bytes and over a transactional store.
|
||||
Three backends run these suites: an in-memory reference (in `tests/`), `dsh-session-persistence-jsonl` (append-only file log) and `dsh-session-persistence-sqlite` (`node:sqlite`, each `SessionEvent` one row `(session_id, seq, type, time, data)`). All passing the same contract + coordinator suite is the proof that the seam is genuinely backend-agnostic — lazy materialization, crash-tail-on-load, and contiguous-seq hold identically over file bytes and over a transactional store.
|
||||
|
||||
## Metadata types
|
||||
|
||||
|
||||
556
packages/session-persistence/src/coordinator.ts
Normal file
556
packages/session-persistence/src/coordinator.ts
Normal file
@@ -0,0 +1,556 @@
|
||||
/**
|
||||
* The backend-agnostic write-path orchestration shared by every first-party
|
||||
* {@link SessionPersistence} backend.
|
||||
*
|
||||
* The two durable backends (`dsh-session-persistence-jsonl` over file bytes,
|
||||
* `dsh-session-persistence-sqlite` over `node:sqlite` rows) were byte-identical
|
||||
* — or same-algorithm — for ALL of their orchestration: the in-memory
|
||||
* bookkeeping (the per-id state, the write-behind buffers, the per-id
|
||||
* serialization chains, the per-session init promises), the `session/event` →
|
||||
* buffer → `session/flush` drain, lazy materialization, crash-tail repair on
|
||||
* load, the four `session/created` adoption cases (new / HMR-adopt / collision /
|
||||
* ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives
|
||||
* differed (write bytes vs. INSERT rows). {@link PersistenceCoordinator} owns
|
||||
* the orchestration once; a backend supplies the storage primitives as a small
|
||||
* {@link PersistenceBackend} hook object.
|
||||
*
|
||||
* The abstract {@link SessionPersistence} service's public API is unchanged: a
|
||||
* backend still IS a `SessionPersistence` (its six public methods delegate to a
|
||||
* coordinator it composes), so a third-party backend MAY implement the service
|
||||
* directly without using the coordinator at all.
|
||||
*
|
||||
* See the write-coordinator RFC (docs/rfc/implemented/2026-06-18-shared-persistence-write-coordinator.md)
|
||||
* for the design rationale (composition over inheritance, the opaque torn marker).
|
||||
*
|
||||
* @module @deepseek-ai/dsh-session-persistence/coordinator
|
||||
*/
|
||||
|
||||
import { Context } from 'cordis'
|
||||
import { interruptedTurnClosers } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import { assertSerializable, seedCoversPrefix } from './index.ts'
|
||||
|
||||
/**
|
||||
* A stored session's durable prefix as read back from a backend: its
|
||||
* {@link SessionHeader}, the preserved (seq-contiguous, parseable) event prefix,
|
||||
* and an OPAQUE `tornMarker` that is present iff a never-committed torn tail must
|
||||
* be truncated before further writes.
|
||||
*
|
||||
* The coordinator NEVER inspects `tornMarker`'s value — it only tests
|
||||
* `!== undefined` (is there a tail to repair?) and passes the value back to
|
||||
* {@link PersistenceBackend.commitRepair}. Each backend chooses its own marker
|
||||
* type: the JSONL backend uses the byte offset to truncate to, the SQLite
|
||||
* backend uses the seq to delete from (both happen to be `number`).
|
||||
*/
|
||||
export interface StoredPrefix<TornMarker = unknown> {
|
||||
meta: SessionHeader
|
||||
events: SessionEvent[]
|
||||
tornMarker?: TornMarker
|
||||
}
|
||||
|
||||
/**
|
||||
* The storage seam between {@link PersistenceCoordinator} and a concrete
|
||||
* backend: the minimal set of durable primitives the orchestration calls. A
|
||||
* backend implements these (over files, rows, an object store, …); the
|
||||
* coordinator supplies everything else (buffering, serialization, cursors,
|
||||
* adoption, crash repair sequencing, dispose quiescence).
|
||||
*
|
||||
* @typeParam TornMarker - the backend's opaque torn-tail repair token (see
|
||||
* {@link StoredPrefix}). The coordinator treats it as fully opaque.
|
||||
*/
|
||||
export interface PersistenceBackend<TornMarker = unknown> {
|
||||
/** Human-readable backend name, used in the dispose-failure AggregateError. */
|
||||
readonly name: string
|
||||
|
||||
/**
|
||||
* Read a stored prefix by id, scanning ANY storage scope (for JSONL: every
|
||||
* cwd bucket). Returns `undefined` if no stored artifact exists. Used by
|
||||
* resume/load, and — via `!== undefined` — by the create-collision probe.
|
||||
* The returned `tornMarker` is present iff there is a torn tail to truncate.
|
||||
*/
|
||||
loadStored(id: SessionId): Promise<StoredPrefix<TornMarker> | undefined>
|
||||
|
||||
/**
|
||||
* Read a stored prefix SCOPED to `cwd`. Deliberately distinct from
|
||||
* {@link loadStored}: HMR live-adoption must only adopt a persisted log at the
|
||||
* SAME cwd as the live session (a same-id log at a different cwd is a
|
||||
* collision, not a resume) — conflating the two reintroduces a cross-cwd
|
||||
* adoption bug. For a globally-unique-id backend (SQLite) `cwd` is ignored.
|
||||
*/
|
||||
loadLive(id: SessionId, cwd: string | undefined): Promise<StoredPrefix<TornMarker> | undefined>
|
||||
|
||||
/**
|
||||
* Durably append a CONTIGUOUS batch, lazily materializing the session first
|
||||
* when `!isMaterialized`. The materialize-write and the first event batch MUST
|
||||
* commit ATOMICALLY (a crash between them must not leave a materialized-but-
|
||||
* empty session). Returns once the batch is durable.
|
||||
*/
|
||||
appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise<void>
|
||||
|
||||
/**
|
||||
* Make a crash repair durable: truncate the torn tail (iff
|
||||
* `tornMarker !== undefined`) and append `closers` (iff any). NOT required to
|
||||
* be atomic — a file backend may truncate-then-append in two fsync'd steps.
|
||||
* Used by load (truncate + synthetic closers) and by live-adoption (truncate
|
||||
* only, `closers = []`).
|
||||
*/
|
||||
commitRepair(meta: SessionHeader, tornMarker: TornMarker | undefined, closers: readonly SessionEvent[]): Promise<void>
|
||||
|
||||
/** Remove the stored artifact for `id` (the coordinator clears in-memory state). */
|
||||
deleteStored(id: SessionId): Promise<void>
|
||||
|
||||
/** List all stored (materialized) sessions' metadata. */
|
||||
list(): Promise<SessionHeader[]>
|
||||
|
||||
/**
|
||||
* Optional lifecycle teardown (e.g. close a database handle). Awaited by the
|
||||
* coordinator's dispose effect AFTER the quiescence drain. A stateless file
|
||||
* backend omits it.
|
||||
*/
|
||||
close?(): Promise<void>
|
||||
}
|
||||
|
||||
/** Per-session write state held by the coordinator's in-memory bookkeeping. */
|
||||
interface SessionState {
|
||||
meta: SessionHeader
|
||||
/** The next seq the backend expects to append (the stored log length). */
|
||||
cursor: number
|
||||
/** Whether the session has been physically materialized. */
|
||||
materialized: boolean
|
||||
/**
|
||||
* The live Session this state was bound to via `onCreated`, if any. State
|
||||
* created through the public `create()`/`load()` API has no owner; state bound
|
||||
* to a live session lets `onCreated` reject a second, unrelated session on the
|
||||
* same id (a collision) instead of silently no-opping.
|
||||
*/
|
||||
owner?: Session
|
||||
}
|
||||
|
||||
/** Collect the rejection reasons from a set of promises (none-throwing). */
|
||||
async function settledErrors(promises: Iterable<Promise<unknown>>): Promise<unknown[]> {
|
||||
const settled = await Promise.allSettled([...promises])
|
||||
const errors: unknown[] = []
|
||||
for (const result of settled) {
|
||||
if (result.status === 'rejected') errors.push(result.reason)
|
||||
}
|
||||
return errors
|
||||
}
|
||||
|
||||
/**
|
||||
* Owns the backend-agnostic session write-path orchestration. A backend
|
||||
* constructs one (`new PersistenceCoordinator(ctx, this)`), implements
|
||||
* {@link PersistenceBackend}, and delegates its six public service methods to
|
||||
* the matching coordinator methods.
|
||||
*
|
||||
* All per-id operations are serialized (a per-id promise chain) so concurrent
|
||||
* flushes / a flush racing a load never interleave storage writes. The
|
||||
* constructor installs the write-path listeners and the dispose effect.
|
||||
*
|
||||
* @typeParam TornMarker - the backend's opaque torn-tail repair token.
|
||||
*/
|
||||
export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
/** Backend bookkeeping keyed by session id (NOT the live Session object). */
|
||||
private states = new Map<string, SessionState>()
|
||||
/** Write-behind buffers keyed by the live Session (write path). */
|
||||
private buffers = new Map<Session, SessionEvent[]>()
|
||||
/**
|
||||
* Per-session serialization: every operation chains onto the prior one for the
|
||||
* same id, so writes for one session never interleave. Keyed by session id.
|
||||
*/
|
||||
private chains = new Map<string, Promise<unknown>>()
|
||||
/**
|
||||
* Per-session init promise (onCreated). Keyed by the LIVE Session OBJECT, not
|
||||
* its id: a disposed fiber's session can be replaced by a different live
|
||||
* Session reusing the same id (HMR, an ACP reconnect), and an id-keyed cache
|
||||
* would hand the new object the old object's init promise.
|
||||
*
|
||||
* Public (readonly) so a backend can expose it for white-box tests that await
|
||||
* a specific session's init (there is no public API to await one init); the
|
||||
* coordinator itself only ever mutates it internally.
|
||||
*/
|
||||
readonly inits = new Map<Session, Promise<void>>()
|
||||
|
||||
constructor(private ctx: Context, private backend: PersistenceBackend<TornMarker>) {
|
||||
this.installWritePath()
|
||||
}
|
||||
|
||||
// --- public surface (the backend's service methods delegate here) ---
|
||||
|
||||
/**
|
||||
* Register a new session's metadata (lazy: no physical write until the first
|
||||
* {@link append}). Rejects if the id is already tracked or already persisted.
|
||||
*/
|
||||
create(meta: SessionHeader): Promise<void> {
|
||||
// Snapshot the metadata at call time: the op runs later (behind the
|
||||
// per-session chain) and the snapshot is stored as the lazy state, so keeping
|
||||
// the caller's object by reference would let a later mutation of `id`/`cwd`
|
||||
// register under one key but materialize under a different path/header.
|
||||
const snapshot: SessionHeader = { ...meta }
|
||||
return this.serialize(snapshot.id, () => this.createCore(snapshot))
|
||||
}
|
||||
|
||||
private async createCore(meta: SessionHeader): Promise<void> {
|
||||
// Do NOT clobber an existing session: the SessionId IS the identity.
|
||||
if (this.states.has(meta.id)) {
|
||||
throw new Error(`session "${meta.id}" already exists in this backend`)
|
||||
}
|
||||
// A persisted artifact under this id (in ANY scope) blocks creation: load/
|
||||
// has/resume identify a session by id alone, so a second artifact would make
|
||||
// resume nondeterministic.
|
||||
if (await this.backend.loadStored(meta.id) !== undefined) {
|
||||
throw new Error(`session "${meta.id}" already has a persisted log on disk; load/resume it instead of creating`)
|
||||
}
|
||||
// Pure lazy: record intent only. No artifact until the first append.
|
||||
this.states.set(meta.id, { meta, cursor: 0, materialized: false })
|
||||
}
|
||||
|
||||
// `async` so the synchronous validate/clone below reject (not throw) per the
|
||||
// Promise<void> contract — callers use `await expect(...).rejects`.
|
||||
/**
|
||||
* Durably persist a batch of events. Honors the append-only and contiguous-seq
|
||||
* contracts; rejects non-JSON-serializable `event.data`.
|
||||
*/
|
||||
async append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
// Validate serializability BEFORE cloning so a bad event surfaces the typed
|
||||
// error rather than an opaque DataCloneError from structuredClone.
|
||||
assertSerializable(events)
|
||||
// Deep-snapshot the batch HERE, before the op waits behind the per-session
|
||||
// chain: a caller that mutates a live array (e.g. session.events) — or an
|
||||
// event inside it — before the op runs would otherwise have those changes
|
||||
// persisted. The clone is taken synchronously (at call time).
|
||||
const batch = events.map(e => structuredClone(e))
|
||||
return this.serialize(id, () => this.appendCore(id, batch))
|
||||
}
|
||||
|
||||
private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
if (events.length === 0) return
|
||||
let state = this.states.get(id)
|
||||
if (state === undefined) state = await this.adopt(id) // calls loadCore, not load
|
||||
|
||||
// Contiguity contract: each event's seq must continue the stored log.
|
||||
for (const [i, event] of events.entries()) {
|
||||
if (event.seq !== state.cursor + i) {
|
||||
throw new Error(`append seq mismatch for "${id}": expected ${state.cursor + i} at index ${i}, got ${event.seq}`)
|
||||
}
|
||||
}
|
||||
|
||||
await this.backend.appendBatch(state.meta, events, state.materialized)
|
||||
// The durable write is the transaction: mark materialized + advance the
|
||||
// cursor as soon as it commits (uniform across backends).
|
||||
state.materialized = true
|
||||
state.cursor += events.length
|
||||
}
|
||||
|
||||
/**
|
||||
* Reload a session: its {@link SessionHeader} plus the event log up to the last
|
||||
* durable checkpoint, with any interrupted final turn durably closed (synthetic
|
||||
* boundary events) during load.
|
||||
*/
|
||||
load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
return this.serialize(id, () => this.loadCore(id))
|
||||
}
|
||||
|
||||
private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
const stored = await this.backend.loadStored(id)
|
||||
if (stored === undefined) throw new Error(`session "${id}" not found`)
|
||||
const { meta, events, tornMarker } = stored
|
||||
this.assertVersion(meta)
|
||||
|
||||
// Crash-recovery: if the log ended mid-turn (real, preserved events but no
|
||||
// closing turn/end), close it durably DURING load so disk, the returned log,
|
||||
// and the cursor all agree. The interrupted turn's real events are preserved,
|
||||
// never truncated (a turn can be huge — the session-persistence RFC); only a
|
||||
// never-fully-written torn tail fragment is discarded.
|
||||
const closers = interruptedTurnClosers(events)
|
||||
const balanced = [...events, ...closers]
|
||||
|
||||
// Make the repair durable (truncate the torn tail + append the synthetic
|
||||
// closers) BEFORE recording state — commitRepair takes `meta` directly, so
|
||||
// there is no state-path ordering dependency (uniform across backends).
|
||||
if (tornMarker !== undefined || closers.length > 0) {
|
||||
await this.backend.commitRepair(meta, tornMarker, closers)
|
||||
}
|
||||
// The state keeps its OWN copy of the meta; the returned value is separate so
|
||||
// a consumer mutating loaded.meta cannot corrupt the backend's metadata.
|
||||
this.states.set(id, { meta: { ...meta }, cursor: balanced.length, materialized: true })
|
||||
return { meta, events: balanced }
|
||||
}
|
||||
|
||||
// NOTE: there is deliberately no coordinator `list()`. Listing needs none of
|
||||
// the coordinator's orchestration (no per-id serialization, no cursor, no
|
||||
// in-memory state) — it is a pure read of stored metadata. A backend's public
|
||||
// `list()` IS the {@link PersistenceBackend.list} hook (one method); routing it
|
||||
// through the coordinator would only forward to that same hook, so the
|
||||
// coordinator stays out of the listing path entirely.
|
||||
|
||||
/** Whether a session is durably present (materialized). */
|
||||
async has(id: SessionId): Promise<boolean> {
|
||||
const state = this.states.get(id)
|
||||
if (state?.materialized) return true
|
||||
// Probe storage scoped to the tracked cwd if known, else any scope. A tracked
|
||||
// lazy session has a known cwd, so loadLive(id, cwd) hits the exact artifact
|
||||
// path — a storage fault there (e.g. a non-ENOENT lookup error) must surface,
|
||||
// not be masked by an any-scope scan that filters a non-directory bucket out.
|
||||
// For an untracked id `cwd` is undefined, where loadLive scans any scope (=
|
||||
// loadStored), so this single call covers both.
|
||||
return (await this.backend.loadLive(id, state?.meta.cwd)) !== undefined
|
||||
}
|
||||
|
||||
/** Remove a session and all its persisted artifacts. */
|
||||
delete(id: SessionId): Promise<void> {
|
||||
return this.serialize(id, () => this.deleteCore(id))
|
||||
}
|
||||
|
||||
private async deleteCore(id: SessionId): Promise<void> {
|
||||
await this.backend.deleteStored(id)
|
||||
this.states.delete(id)
|
||||
}
|
||||
|
||||
// --- per-id serialization + adoption helpers ---
|
||||
|
||||
/**
|
||||
* Run `op` after any in-flight operation for the same session id, so writes for
|
||||
* one session never interleave. Errors do not poison the chain. NOTE: serialized
|
||||
* public methods must NOT call each other (deadlock); they call the unserialized
|
||||
* `*Core` helpers instead.
|
||||
*/
|
||||
private serialize<T>(id: SessionId, op: () => Promise<T>): Promise<T> {
|
||||
const prior = this.chains.get(id) ?? Promise.resolve()
|
||||
const next = prior.then(op, op)
|
||||
// Keep the chain alive but swallow this op's rejection for the NEXT waiter
|
||||
// (the caller still sees the real rejection via `next`).
|
||||
this.chains.set(id, next.then(() => undefined, () => undefined))
|
||||
return next
|
||||
}
|
||||
|
||||
/** Build a state for a session discovered in storage but not yet in memory. */
|
||||
private async adopt(id: SessionId): Promise<SessionState> {
|
||||
// loadCore (NOT load) — adopt runs inside an already-serialized op, so
|
||||
// re-entering the chain via the public load() would deadlock.
|
||||
await this.loadCore(id)
|
||||
const state = this.states.get(id)
|
||||
/* v8 ignore next -- loadCore always sets the state for the id */
|
||||
if (!state) throw new Error(`failed to adopt session "${id}"`)
|
||||
return state
|
||||
}
|
||||
|
||||
private assertVersion(meta: SessionHeader): void {
|
||||
if (meta.version !== 1) {
|
||||
throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`)
|
||||
}
|
||||
}
|
||||
|
||||
// --- write path (session/event → flush drain) ---
|
||||
|
||||
private installWritePath(): void {
|
||||
const ctx = this.ctx
|
||||
|
||||
// Capture the header on creation; persist a fork's seed once. Record the init
|
||||
// promise so flush/dispose can await it (onCreated is async).
|
||||
ctx.on('session/created', (session) => { void this.initFor(session) })
|
||||
|
||||
// Snapshot + buffer every event (the live object is mutable; clone so a later
|
||||
// in-place mutation cannot rewrite a buffered event). Serializability is
|
||||
// guaranteed at the source (Session.append), so structuredClone is safe.
|
||||
ctx.on('session/event', (session, event) => {
|
||||
let buffer = this.buffers.get(session)
|
||||
if (!buffer) this.buffers.set(session, buffer = [])
|
||||
buffer.push(structuredClone(event))
|
||||
})
|
||||
|
||||
// Drain to the backend at the durability checkpoint.
|
||||
ctx.on('session/flush', session => this.flush(session))
|
||||
|
||||
// Dispose must reach quiescence: await every init + final drain BEFORE
|
||||
// returning, then close the backend's own resources (AFTER the drain), so no
|
||||
// write lands after teardown and a close failure never MASKS a drain error.
|
||||
ctx.effect(() => async () => {
|
||||
let disposeError: unknown
|
||||
try {
|
||||
const errors = [
|
||||
...await settledErrors(this.inits.values()),
|
||||
...await settledErrors([...this.buffers.keys()].map(s => this.flush(s))),
|
||||
...await settledErrors(this.chains.values()),
|
||||
]
|
||||
if (errors.length > 0) {
|
||||
throw new AggregateError(errors, `${this.backend.name} dispose failed`)
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
disposeError = error
|
||||
throw error
|
||||
} finally {
|
||||
try {
|
||||
await this.backend.close?.()
|
||||
} catch (closeError: unknown) {
|
||||
// A close failure can only add teardown context; keep the already-
|
||||
// captured drain AggregateError as the primary failure rather than
|
||||
// masking it. Only surface the close error if the drain succeeded.
|
||||
/* v8 ignore start -- close failure racing disposal is a defensive teardown edge */
|
||||
if (disposeError === undefined) throw closeError
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
}
|
||||
}, `${this.backend.name} write path`)
|
||||
|
||||
// HMR: a hot reload does not replay session/created, so seed existing live
|
||||
// sessions (mirrors dsh-invariants).
|
||||
for (const session of ctx.sessions.list()) void this.initFor(session)
|
||||
}
|
||||
|
||||
/** Start (once) the async init for a session and remember its promise. */
|
||||
private initFor(session: Session): Promise<void> {
|
||||
const existing = this.inits.get(session)
|
||||
if (existing) return existing
|
||||
// Snapshot the seed SYNCHRONOUSLY — initFor runs inside the `session/created`
|
||||
// emit, before any later `append` adds non-seed events. A clone freezes it
|
||||
// against later mutation of the live event objects.
|
||||
const seed = session.events.map(e => structuredClone(e))
|
||||
const p = this.onCreated(session, seed)
|
||||
// Attach a no-op rejection handler so a failing init does not surface as an
|
||||
// unhandled rejection if no flush observes `p` before it rejects. The REAL
|
||||
// error is still delivered: flush/dispose await the same `p` from the map.
|
||||
p.catch(() => { /* observed by flush/dispose via the stored promise */ })
|
||||
this.inits.set(session, p)
|
||||
return p
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a live session's `seed` reproduces the first `cursor` persisted
|
||||
* events. A `cursor` of 0 (nothing persisted yet) trivially matches. Used when
|
||||
* a live session claims ownerless state left by a prior `load()`/`create()`.
|
||||
*/
|
||||
private async seedMatchesPersisted(id: SessionId, seed: readonly SessionEvent[], cursor: number): Promise<boolean> {
|
||||
if (cursor === 0) return true
|
||||
const stored = await this.backend.loadStored(id)
|
||||
/* v8 ignore next -- a cursor > 0 means the session was materialized, so it exists */
|
||||
if (stored === undefined) return false
|
||||
return seedCoversPrefix(seed, stored.events.slice(0, cursor))
|
||||
}
|
||||
|
||||
/**
|
||||
* On session/created: sync the backend's in-memory state to a live Session.
|
||||
*
|
||||
* Cases, by whether this backend tracks the id and whether an artifact exists:
|
||||
* 1. Already tracked → no-op (or claim ownerless state if the seed matches,
|
||||
* or reclaim a truly-abandoned id, else reject as a collision).
|
||||
* 2. Not tracked, an artifact EXISTS at this cwd and is a seq-aligned PREFIX
|
||||
* of the live events → ADOPT it (HMR/reload), persisting any live suffix.
|
||||
* 3. Not tracked, an artifact EXISTS but is NOT a prefix → REJECT (collision).
|
||||
* 4. Not tracked and NO artifact → a genuinely new session: register meta
|
||||
* (lazy) and persist its seed once.
|
||||
*/
|
||||
private async onCreated(session: Session, seed: readonly SessionEvent[]): Promise<void> {
|
||||
const id = session.header.id
|
||||
const tracked = this.states.get(id)
|
||||
if (tracked !== undefined) {
|
||||
// case 1: already tracked.
|
||||
/* v8 ignore next -- initFor dedupes per session object; same-object re-entry can't occur */
|
||||
if (tracked.owner === session) return
|
||||
if (tracked.owner === undefined) {
|
||||
// Ownerless state from the public create()/load() API. The FIRST live
|
||||
// session claims it — but ONLY if its seed reproduces the persisted
|
||||
// prefix (else a fresh, unrelated session reusing the id would have its
|
||||
// seq 0..cursor-1 events filtered as already-written and grafted on).
|
||||
if (!await this.seedMatchesPersisted(id, seed, tracked.cursor)) {
|
||||
throw new Error(`session "${id}" is already persisted with ${tracked.cursor} event(s) that do not match this live session (id collision)`)
|
||||
}
|
||||
tracked.owner = session
|
||||
// Persist the seed SUFFIX beyond the persisted prefix. Constructor seed
|
||||
// events never emit session/event, so the buffer never sees them.
|
||||
const suffix = seed.slice(tracked.cursor)
|
||||
if (suffix.length > 0) await this.append(id, suffix)
|
||||
return
|
||||
}
|
||||
// Owned by a DIFFERENT live session. Reclaim ONLY a truly-abandoned id
|
||||
// (never materialized, no pending buffer); else it is a real collision.
|
||||
const ownerBuffer = this.buffers.get(tracked.owner)
|
||||
if (!tracked.materialized && !ownerBuffer?.length) {
|
||||
this.states.delete(id)
|
||||
} else {
|
||||
throw new Error(`session "${id}" is already bound to a different live session in this backend (id collision)`)
|
||||
}
|
||||
}
|
||||
|
||||
// case 2/3: an artifact at THIS cwd is adopted as a live prefix (or rejected
|
||||
// as a collision inside adoptLivePrefix). cwd-scoped (loadLive), never
|
||||
// any-scope: a same-id artifact at a different cwd is a collision, not a
|
||||
// resume.
|
||||
const live = await this.backend.loadLive(id, session.header.cwd)
|
||||
if (live !== undefined) {
|
||||
// Do NOT route through loadCore(): that crash-repairs open turns as
|
||||
// interrupted, which is wrong for HMR while the live Session is still the
|
||||
// authority and may append the real step/turn end later.
|
||||
await this.serialize(id, () => this.adoptLivePrefix(session, seed, live))
|
||||
return
|
||||
}
|
||||
|
||||
// case 4: a genuinely new session. Register its meta (lazy), then persist its
|
||||
// seed (events present at creation time) once.
|
||||
const meta: SessionHeader = { ...session.header }
|
||||
await this.create(meta)
|
||||
// Bind this state to the live session so a later DIFFERENT session reusing
|
||||
// the id is detected as a collision (case 1) rather than silently no-opped.
|
||||
const created = this.states.get(id)
|
||||
/* v8 ignore next -- create() always sets the state for the id */
|
||||
if (created !== undefined) created.owner = session
|
||||
if (seed.length > 0) await this.append(id, seed)
|
||||
}
|
||||
|
||||
/**
|
||||
* Adopt a stored prefix as a live session's history (HMR/reload): verify the
|
||||
* seed covers the stored prefix, truncate any torn tail (NOT the open turn —
|
||||
* the live Session is still the authority), bind ownership, and persist the
|
||||
* live suffix that was ahead of the stored prefix.
|
||||
*/
|
||||
private async adoptLivePrefix(session: Session, seed: readonly SessionEvent[], stored: StoredPrefix<TornMarker>): Promise<void> {
|
||||
const { meta, events, tornMarker } = stored
|
||||
this.assertVersion(meta)
|
||||
if (!seedCoversPrefix(seed, events)) {
|
||||
throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`)
|
||||
}
|
||||
// Truncate-only repair (no closers): the open turn is NOT closed here.
|
||||
if (tornMarker !== undefined) await this.backend.commitRepair(meta, tornMarker, [])
|
||||
this.states.set(session.header.id, {
|
||||
meta: { ...meta },
|
||||
cursor: events.length,
|
||||
materialized: true,
|
||||
owner: session,
|
||||
})
|
||||
const suffix = seed.slice(events.length)
|
||||
if (suffix.length > 0) await this.appendCore(session.header.id, suffix)
|
||||
}
|
||||
|
||||
private async flush(session: Session): Promise<void> {
|
||||
// Wait for the session's init (onCreated) so the state/cursor and any
|
||||
// fork-seed persistence are in place before draining. Awaiting the same
|
||||
// promise initFor stored also surfaces an init failure (e.g. a collision)
|
||||
// here, where the caller of session/flush observes it.
|
||||
await this.inits.get(session)
|
||||
// Serialize the WHOLE drain (read cursor → append → splice) on the per-session
|
||||
// chain so two concurrent flushes cannot both read the same cursor and
|
||||
// seq-mismatch on the second append.
|
||||
await this.serialize(session.header.id, () => this.drain(session))
|
||||
}
|
||||
|
||||
/** Drain a session's write buffer to the backend. Caller serializes this per id. */
|
||||
private async drain(session: Session): Promise<void> {
|
||||
const buffer = this.buffers.get(session)
|
||||
if (!buffer?.length) return
|
||||
// Copy WITHOUT removing: the buffer is the only durable-pending copy of these
|
||||
// events. Drain it only AFTER the append commits; events pushed during the
|
||||
// await sit past batch.length and survive the prefix splice, so a
|
||||
// retry/dispose re-drains the rest.
|
||||
const batch = buffer.slice()
|
||||
const state = this.states.get(session.header.id)
|
||||
// Only append events at or beyond the write cursor (a resumed session's seed
|
||||
// is already stored). flush awaits the init above, which always sets state,
|
||||
// so the `?? 0` fallback is a defensive guard that never fires in practice.
|
||||
/* v8 ignore next -- state is always set by the awaited init before flush */
|
||||
const cursor = state?.cursor ?? 0
|
||||
const fresh = batch.filter(e => e.seq >= cursor)
|
||||
// appendCore (NOT the serialized append) — drain already runs inside the
|
||||
// per-session chain, so re-entering via append() would deadlock.
|
||||
if (fresh.length > 0) await this.appendCore(session.header.id, fresh)
|
||||
buffer.splice(0, batch.length)
|
||||
}
|
||||
}
|
||||
@@ -28,6 +28,10 @@ import type { SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-se
|
||||
// Re-export the metadata vocabulary so consumers import it from the seam.
|
||||
export type { SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
|
||||
// The backend-agnostic write-path orchestration first-party backends compose.
|
||||
export { PersistenceCoordinator } from './coordinator.ts'
|
||||
export type { PersistenceBackend, StoredPrefix } from './coordinator.ts'
|
||||
|
||||
declare module 'cordis' {
|
||||
interface Context {
|
||||
sessionPersistence: SessionPersistence
|
||||
|
||||
734
packages/session-persistence/tests/coordinator-contract.ts
Normal file
734
packages/session-persistence/tests/coordinator-contract.ts
Normal file
@@ -0,0 +1,734 @@
|
||||
/**
|
||||
* Reusable ORCHESTRATION suite for any backend that composes a
|
||||
* {@link PersistenceCoordinator}. Where {@link runPersistenceContract} (in
|
||||
* contract.ts) pins the public read/write SEMANTICS, this suite pins the
|
||||
* coordinator's WRITE-PATH ORCHESTRATION — the behavior that is identical across
|
||||
* every first-party backend because it lives in the shared coordinator, not in
|
||||
* the storage primitives: the `session/created` → `session/event` →
|
||||
* `session/flush` → dispose drain, lazy materialization, fork-seed persistence,
|
||||
* the four `onCreated` adoption cases (new / HMR-adopt / collision /
|
||||
* ownerless-claim), crash-tail repair on load, and dispose-time quiescence.
|
||||
*
|
||||
* A backend imports {@link runCoordinatorContract} and calls it with a
|
||||
* {@link CoordinatorFixture} factory that knows how to (a) mount the REAL
|
||||
* backend plugin on a {@link Context} over a SHARED storage scope (so HMR/reload
|
||||
* tests can dispose one instance and mount another over the same bytes/rows),
|
||||
* and (b) inject a never-committed torn tail for one session
|
||||
* ({@link CoordinatorFixture.corruptTail}) so the through-coordinator torn-tail
|
||||
* repair branch is exercised against real storage. The suite drives everything
|
||||
* through the PUBLIC {@link SessionPersistence} API + the cordis SessionStore
|
||||
* write path — never the storage primitives directly — so it runs unchanged for
|
||||
* every backend (memory / jsonl / sqlite).
|
||||
*
|
||||
* Each scenario here was previously DUPLICATED in `jsonl.spec.ts` and
|
||||
* `sqlite.spec.ts`; it now lives once and runs once per backend through the
|
||||
* fixture. The per-backend specs keep ONLY their storage-mechanics tests.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-session-persistence/tests/coordinator-contract
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context, type Fiber } from 'cordis'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionPersistence } from '../src/index.ts'
|
||||
import { meta, oneTurnLog } from './contract.ts'
|
||||
|
||||
/**
|
||||
* The backend-specific capabilities the orchestration suite needs beyond the
|
||||
* public service API. A fresh fixture is created per test (isolated storage);
|
||||
* the suite mounts/disposes backend instances on it and cleans it up at the end.
|
||||
*/
|
||||
export interface CoordinatorFixture {
|
||||
/**
|
||||
* Mount the REAL backend plugin (via `ctx.plugin`, the Loader path) on `ctx`,
|
||||
* over THIS fixture's shared storage scope. Returns the plugin fiber so the
|
||||
* suite can dispose a single instance (HMR/reload) while the storage — and any
|
||||
* still-live session in another fiber — survives. The caller has already
|
||||
* mounted `SessionStore` on `ctx`.
|
||||
*/
|
||||
mount: (ctx: Context) => Promise<Fiber>
|
||||
|
||||
/**
|
||||
* Inject a NEVER-COMMITTED torn tail into the backend's storage for `id` at
|
||||
* the given `cwd` (the cwd the session was created with): a half-written
|
||||
* record past the committed region (JSONL: a partial line with no newline;
|
||||
* SQLite: a row with invalid `data` JSON past the committed seq). This drives
|
||||
* the coordinator's `loadCore` `tornMarker !== undefined` → `commitRepair`
|
||||
* branch against real storage.
|
||||
*
|
||||
* OMITTED by a backend that structurally has no torn tails (memory): the
|
||||
* torn-tail scenario then self-skips (asserted explicitly in the suite).
|
||||
*/
|
||||
corruptTail?: (id: SessionId, cwd: string | undefined) => Promise<void>
|
||||
|
||||
/** Tear down the storage scope (remove the temp dir / file). */
|
||||
cleanup: () => Promise<void>
|
||||
}
|
||||
|
||||
/** A constant absolute cwd; jsonl keys directories off it, memory/sqlite ignore it. */
|
||||
const WORK = '/w'
|
||||
|
||||
/** The per-session init map a backend exposes for white-box init awaits. */
|
||||
function inits(persistence: SessionPersistence): Map<Session, Promise<void>> {
|
||||
return (persistence as unknown as { inits: Map<Session, Promise<void>> }).inits
|
||||
}
|
||||
|
||||
/** Append a whole event log to a live session, event by event (drives session/event). */
|
||||
function send(session: Session, events: readonly SessionEvent[]): void {
|
||||
for (const e of events) session.append(e.type, e.data)
|
||||
}
|
||||
|
||||
/** A live session created inside its OWN fiber, so it survives a backend reload. */
|
||||
async function liveSessionInFiber(
|
||||
ctx: Context, id: string, cwd: string | undefined,
|
||||
): Promise<Session> {
|
||||
let session!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create(id, cwd !== undefined ? { meta: { cwd } } : undefined)
|
||||
}, { inject: ['sessions'] }))
|
||||
return session
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the coordinator orchestration suite against a backend. `makeFixture()`
|
||||
* MUST return a fresh fixture (isolated storage) each call.
|
||||
*/
|
||||
export function runCoordinatorContract(name: string, makeFixture: () => Promise<CoordinatorFixture>): void {
|
||||
describe(`PersistenceCoordinator orchestration: ${name}`, () => {
|
||||
/** Mount SessionStore + a backend instance on a fresh context over the fixture's storage. */
|
||||
async function freshCtx(fix: CoordinatorFixture): Promise<{ ctx: Context; fiber: Fiber }> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await fix.mount(ctx)
|
||||
return { ctx, fiber }
|
||||
}
|
||||
|
||||
// --- write path: live session → flush → reload ---
|
||||
|
||||
it('persists a live session driven through the store, surviving reload', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const session = ctx.sessions.create('live', { meta: { cwd: WORK } })
|
||||
send(session, oneTurnLog())
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('live'))
|
||||
expect(loaded.events).toHaveLength(6)
|
||||
expect(loaded.meta.cwd).toBe(WORK)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('snapshot-on-buffer: mutating an event after session/event does not corrupt the persisted copy', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const session = ctx.sessions.create('mutate', { meta: { cwd: WORK } })
|
||||
const ev = session.append('user/message', { content: [{ type: 'text', text: 'original' }], source: { kind: 'user' } })
|
||||
// Mutate the live event object AFTER it was buffered by session/event.
|
||||
;(ev.data as { content: { type: 'text'; text: string }[] }).content[0]!.text = 'HACKED'
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('mutate'))
|
||||
const first = loaded.events[0]
|
||||
expect(first?.type === 'user/message' && (first.data.content[0] as { text: string }).text).toBe('original')
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('append snapshots the batch: mutating the caller array/events after the call is ignored', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('snapshot', WORK)
|
||||
await ctx.sessionPersistence.create(m)
|
||||
const events = oneTurnLog() // seqs 0..5
|
||||
const userMsg = events[1] // the user/message event
|
||||
const p = ctx.sessionPersistence.append(m.id, events)
|
||||
// Mutate the caller's array AND an event object after the call but before
|
||||
// the queued op runs: the snapshot taken at call time must shield the copy.
|
||||
events.push({ type: 'turn/start', seq: 6, time: 99, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } })
|
||||
if (userMsg?.type === 'user/message') userMsg.data.content = [{ type: 'text', text: 'MUTATED' }]
|
||||
await p
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) // not 0..6
|
||||
const persisted = JSON.stringify(loaded.events)
|
||||
expect(persisted).toContain('hi') // original content
|
||||
expect(persisted).not.toContain('MUTATED')
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- fork / resume ---
|
||||
|
||||
it('fork: a seeded new session persists its seed once (no double-write on a no-op flush)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const seed = oneTurnLog()
|
||||
// A fork: a brand-new id whose seed came from elsewhere.
|
||||
const forked = ctx.sessions.create('forked', { seed, meta: { cwd: WORK } })
|
||||
await inits(ctx.sessionPersistence).get(forked) // onCreated persisted the seed
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('forked'))
|
||||
expect(loaded.events).toEqual(seed)
|
||||
// A flush with no NEW events must not double-write.
|
||||
await ctx.parallel('session/flush', forked)
|
||||
const reloaded = await ctx.sessionPersistence.load(SessionId('forked'))
|
||||
expect(reloaded.events).toEqual(seed)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('resume: a re-created session seeded with the loaded log does not re-append its seed and continues the seq', async () => {
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
// First lifecycle: persist a session through the store.
|
||||
const s1 = first.ctx.sessions.create('resumed', { meta: { cwd: WORK } })
|
||||
send(s1, oneTurnLog())
|
||||
await first.ctx.parallel('session/flush', s1)
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// Second lifecycle: a NEW backend instance + a session re-created with the
|
||||
// same id SEEDED with the loaded events. onCreated adopts the stored log
|
||||
// (does not re-persist the seed); a new turn appends at seq 6.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('resumed'))
|
||||
const s2 = second.ctx.sessions.create('resumed', { seed: loaded.events, meta: { cwd: WORK } })
|
||||
await inits(second.ctx.sessionPersistence).get(s2) // let onCreated adopt
|
||||
s2.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
s2.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
await second.ctx.parallel('session/flush', s2)
|
||||
|
||||
const reloaded = await second.ctx.sessionPersistence.load(SessionId('resumed'))
|
||||
// 6 original + 2 new, contiguous, no duplicated seed.
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- HMR ---
|
||||
|
||||
it('HMR: applying the plugin seeds existing live sessions', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
// A session exists BEFORE the persistence plugin is applied.
|
||||
const session = ctx.sessions.create('pre-existing', { meta: { cwd: WORK } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
|
||||
const fiber = await fix.mount(ctx)
|
||||
try {
|
||||
// The plugin seeded it on apply; a subsequent flush persists its events.
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('pre-existing'))
|
||||
expect(loaded.events.length).toBeGreaterThanOrEqual(2)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('HMR: dispose drains remaining buffers', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await fix.mount(ctx)
|
||||
const session = await liveSessionInFiber(ctx, 'drain', WORK)
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'buffered' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
// No explicit flush — dispose must drain.
|
||||
await fiber.dispose()
|
||||
|
||||
// A fresh backend instance reads what the disposed one drained.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('drain'))
|
||||
expect(loaded.events.length).toBeGreaterThanOrEqual(2)
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('HMR: reloading the backend adopts a still-live, already-materialized session', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
// The session lives in its OWN fiber so it survives the backend reload.
|
||||
const session = await liveSessionInFiber(ctx, 'hmr-adopt', WORK)
|
||||
try {
|
||||
// Backend instance 1 materializes the session.
|
||||
const backend1 = await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Hot-reload: dispose instance 1, mount instance 2 over the SAME storage
|
||||
// while the session stays live. Instance 2 has an empty states map but the
|
||||
// log is materialized and is a prefix of the live events — it must ADOPT
|
||||
// (not reject). A second turn appended after reload then persists.
|
||||
await backend1.dispose()
|
||||
await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'again' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
await expect(ctx.parallel('session/flush', session)).resolves.not.toThrow()
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-adopt'))
|
||||
expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2)
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('HMR: adoption persists the live SUFFIX that was ahead of the stored prefix', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const session = await liveSessionInFiber(ctx, 'hmr-suffix', WORK)
|
||||
try {
|
||||
// Instance 1 flushes turn 1.
|
||||
const backend1 = await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Append turn 2 to the LIVE session, then dispose instance 1 WITHOUT
|
||||
// flushing turn 2: it is now ONLY in the live session's events; the new
|
||||
// backend never buffered it via session/event.
|
||||
await backend1.dispose()
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
|
||||
// Instance 2 adopts the stored prefix (turn 1) and MUST also persist the
|
||||
// live suffix (turn 2) carried in the session's events.
|
||||
await fix.mount(ctx)
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-suffix'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3])
|
||||
expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2)
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('HMR adoption does NOT crash-repair an active open turn as interrupted (truncate without closers)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const session = await liveSessionInFiber(ctx, 'hmr-open', WORK)
|
||||
try {
|
||||
const first = await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Crash-tail a torn fragment past the (open) committed turn, then reload.
|
||||
await first.dispose()
|
||||
if (fix.corruptTail) await fix.corruptTail(SessionId('hmr-open'), WORK)
|
||||
const second = await fix.mount(ctx)
|
||||
// The live session is still the authority: it appends the REAL step/turn
|
||||
// end. Adoption must truncate the torn tail but NOT synthesize closers.
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-open'))
|
||||
expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'step/start', 'step/end', 'turn/end'])
|
||||
expect(loaded.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } })
|
||||
await second.dispose()
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- collision / id reuse ---
|
||||
|
||||
it('a NEW live session colliding on a persisted id is rejected, not silently adopted', async () => {
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
const s1 = first.ctx.sessions.create('collide', { meta: { cwd: WORK } })
|
||||
send(s1, oneTurnLog())
|
||||
await first.ctx.parallel('session/flush', s1)
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// A FRESH backend + a NEW live session with the same id but NO explicit
|
||||
// resume. onCreated treats it as new; create() rejects because a log already
|
||||
// exists. The rejection surfaces via the init promise (flush awaits it).
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const s2 = second.ctx.sessions.create('collide', { meta: { cwd: WORK } })
|
||||
s2.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
await expect(inits(second.ctx.sessionPersistence).get(s2))
|
||||
.rejects.toThrow(/already has a persisted log|id collision/)
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('an abandoned lazy session (never materialized) releases its id for reuse', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// A live session created then disposed BEFORE its first append: cursor 0,
|
||||
// never materialized. A new live session reusing the id must reclaim it.
|
||||
let firstSession!: Session
|
||||
const firstFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
firstSession = inner.sessions.create('abandoned', { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await inits(ctx.sessionPersistence).get(firstSession) // register the lazy state
|
||||
await firstFiber.dispose() // disposed before any append → never materialized
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create('abandoned', { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(inits(ctx.sessionPersistence).get(reuse)).resolves.toBeUndefined()
|
||||
reuse.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
reuse.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', reuse)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('abandoned'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1])
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('does NOT reclaim an id whose abandoned owner still has buffered (unflushed) events', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
let first!: Session
|
||||
const firstFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
first = inner.sessions.create('buffered', { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await inits(ctx.sessionPersistence).get(first)
|
||||
// Append a turn but do NOT flush — events sit in the write-behind buffer.
|
||||
first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
first.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await firstFiber.dispose() // disposed before flush; not materialized, buffer pending
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create('buffered', { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(inits(ctx.sessionPersistence).get(reuse)).rejects.toThrow(/already bound to a different live session/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('initFor is idempotent: re-emitting session/created does not re-initialize', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const session = ctx.sessions.create('idem', { meta: { cwd: WORK } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
// Re-emit session/created for the SAME live session (idempotent initFor).
|
||||
ctx.emit('session/created', session)
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('idem'))
|
||||
expect(loaded.events).toHaveLength(2) // not doubled
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- ownerless-state claim (public create()/load() then a live session arrives) ---
|
||||
|
||||
it('a live session claims cursor-0 ownerless state created via the public API and persists its seed', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// create() registers ownerless state with cursor 0 (lazy, nothing persisted).
|
||||
await ctx.sessionPersistence.create(meta('lazy-claim', WORK))
|
||||
// A live session with that id arrives and claims it (cursor 0 matches
|
||||
// trivially), persisting its seed.
|
||||
const live = ctx.sessions.create('lazy-claim', { seed: oneTurnLog(), meta: { cwd: WORK } })
|
||||
await expect(inits(ctx.sessionPersistence).get(live)).resolves.toBeUndefined()
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('lazy-claim'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5])
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('a fresh session reusing a previously-loaded id is rejected (ownerless guard)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// Materialize a log, then load() it WITHOUT a live session — ownerless
|
||||
// state, cursor at the persisted length.
|
||||
await ctx.sessionPersistence.create(meta('preview', WORK))
|
||||
await ctx.sessionPersistence.append(SessionId('preview'), oneTurnLog())
|
||||
await ctx.sessionPersistence.load(SessionId('preview'))
|
||||
|
||||
// A FRESH (empty-seed) live session reusing that id must be rejected: its
|
||||
// seq 0..cursor-1 events would otherwise be filtered as already-persisted.
|
||||
let fresh!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
fresh = inner.sessions.create('preview', { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(inits(ctx.sessionPersistence).get(fresh))
|
||||
.rejects.toThrow(/do not match this live session|already has a persisted log|id collision/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('a live session whose seed matches the loaded prefix claims ownerless state and persists the suffix', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// Materialize and load (ownerless, cursor = 6).
|
||||
await ctx.sessionPersistence.create(meta('claim', WORK))
|
||||
await ctx.sessionPersistence.append(SessionId('claim'), oneTurnLog())
|
||||
const { events } = await ctx.sessionPersistence.load(SessionId('claim'))
|
||||
|
||||
// A live session SEEDED with the loaded log PLUS a new turn claims the
|
||||
// ownerless state and persists only the suffix.
|
||||
const cont = ctx.sessions.create('claim', { seed: [
|
||||
...events,
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
], meta: { cwd: WORK } })
|
||||
await inits(ctx.sessionPersistence).get(cont)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('claim'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- append adopts a storage-only session (fresh instance, no prior create/load) ---
|
||||
|
||||
it('append adopts a storage-only session (fresh instance) and continues the seq', async () => {
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('adopt-append', WORK)
|
||||
await first.ctx.sessionPersistence.create(m)
|
||||
await first.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// A fresh instance appends a second turn WITHOUT a prior create/load: append
|
||||
// must adopt the stored session (cursor = stored length) and continue.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
await second.ctx.sessionPersistence.append(SessionId('adopt-append'), [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
])
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('adopt-append'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- small public-API edges that the coordinator owns uniformly ---
|
||||
|
||||
it('append of an empty batch is a no-op (stays lazy)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('empty-batch', WORK)
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, [])
|
||||
expect(await ctx.sessionPersistence.has(m.id)).toBe(false)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('load rejects a missing session', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
await expect(ctx.sessionPersistence.load(SessionId('nope'))).rejects.toThrow(/not found/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('delete of a non-existent session is a no-op', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
await expect(ctx.sessionPersistence.delete(SessionId('ghost'))).resolves.toBeUndefined()
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('create rejects a duplicate id (in memory and on a persisted log)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('dup', WORK)
|
||||
await first.ctx.sessionPersistence.create(m)
|
||||
// Same in-memory state.
|
||||
await expect(first.ctx.sessionPersistence.create(m)).rejects.toThrow(/already exists in this backend/)
|
||||
await first.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// A fresh instance over the same storage sees the persisted log.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
await expect(second.ctx.sessionPersistence.create(meta('dup', WORK)))
|
||||
.rejects.toThrow(/already has a persisted log on disk/)
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects an unknown format version on load (assertVersion)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const m = { version: 2, id: SessionId('v2'), createdAt: 1, cwd: WORK }
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
await expect(ctx.sessionPersistence.load(m.id)).rejects.toThrow(/version/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('round-trips a header with parentSession (fork lineage)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const m = { version: 1, id: SessionId('forked-child'), createdAt: 1, cwd: WORK, parentSession: SessionId('the-parent') }
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.meta.parentSession).toBe('the-parent')
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('flush before init resolves uses cursor 0', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// Append directly to a live session and flush IMMEDIATELY, before the
|
||||
// async onCreated init has necessarily set state (exercises the
|
||||
// state-undefined cursor path).
|
||||
const session = ctx.sessions.create('flush-nostate', { meta: { cwd: WORK } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'q' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('flush-nostate'))
|
||||
expect(loaded.events).toHaveLength(2)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- crash-tail repair THROUGH the coordinator (real storage torn tail) ---
|
||||
|
||||
it('torn-tail load: a never-committed tail is truncated and the open turn closed during load (commitRepair w/ tornMarker)', async () => {
|
||||
const fix = await makeFixture()
|
||||
if (!fix.corruptTail) {
|
||||
// A memory-style store has no torn tails (every write is atomic in RAM),
|
||||
// so there is no tornMarker path to exercise. Assert that explicitly
|
||||
// instead of silently skipping, then bail.
|
||||
expect(fix.corruptTail).toBeUndefined()
|
||||
await fix.cleanup()
|
||||
return
|
||||
}
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('torn', WORK)
|
||||
await first.ctx.sessionPersistence.create(m)
|
||||
await first.ctx.sessionPersistence.append(m.id, oneTurnLog()) // committed 0..5 (balanced)
|
||||
// A second turn whose real events are durable but never closed (open turn).
|
||||
await first.ctx.sessionPersistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } },
|
||||
])
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
// Inject a torn fragment past the committed region (never-committed tail).
|
||||
await fix.corruptTail(SessionId('torn'), WORK)
|
||||
|
||||
// A FRESH instance loads: the torn tail is truncated (tornMarker !==
|
||||
// undefined) AND the open turn 2 is closed with synthetic step/end +
|
||||
// turn/end {interrupted} — commitRepair runs with BOTH a torn marker and
|
||||
// closers. The preserved real events (0..7) are never truncated.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('torn'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
|
||||
expect(loaded.events.map(e => e.type)).toEqual([
|
||||
'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1
|
||||
'turn/start', 'step/start', 'step/end', 'turn/end', // turn 2: real + synthetic closers
|
||||
])
|
||||
const last = loaded.events.at(-1)!
|
||||
expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' })
|
||||
|
||||
// The repair is durable: the next append continues at the balanced length
|
||||
// (seq 10) and a reload round-trips identically.
|
||||
await second.ctx.sessionPersistence.append(SessionId('torn'), [
|
||||
{ type: 'turn/start', seq: 10, time: 9, data: { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 11, time: 10, data: { turn: 3, reason: { kind: 'completed' } } },
|
||||
])
|
||||
const reloaded = await second.ctx.sessionPersistence.load(SessionId('torn'))
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
@@ -1,76 +1,132 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { SessionId, isJsonValue, interruptedTurnClosers } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import { SessionPersistence, assertSerializable, seedCoversPrefix } from '../src/index.ts'
|
||||
import SessionStore, { SessionId, isJsonValue } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
SessionPersistence, PersistenceCoordinator, assertSerializable, seedCoversPrefix,
|
||||
type PersistenceBackend, type StoredPrefix,
|
||||
} from '../src/index.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog } from './contract.ts'
|
||||
import { runCoordinatorContract, type CoordinatorFixture } from './coordinator-contract.ts'
|
||||
|
||||
/** The durable store shape: materialized sessions only (no lazy entries). */
|
||||
type MemoryStore = Map<string, { meta: SessionHeader; events: SessionEvent[] }>
|
||||
|
||||
/** Optional plugin config: an EXTERNAL store shared across backend instances. */
|
||||
interface MemoryConfig { store?: MemoryStore }
|
||||
|
||||
/**
|
||||
* A minimal in-memory {@link SessionPersistence} used to (a) cover the abstract
|
||||
* base's constructor + service registration and (b) validate the reusable
|
||||
* contract suite itself. The real durable backend is
|
||||
* `@deepseek-ai/dsh-session-persistence-jsonl`.
|
||||
* A trivial in-memory {@link SessionPersistence} that composes a
|
||||
* {@link PersistenceCoordinator} over a dependency-free `Map`-backed
|
||||
* {@link PersistenceBackend}. It is BOTH the coordinator's reference vehicle
|
||||
* (the simplest possible storage — a `Map<id, {meta, events}>` with no torn
|
||||
* tails, so `tornMarker` is always undefined) and the cover for the abstract
|
||||
* base's constructor + service registration. The real durable backends are
|
||||
* `@deepseek-ai/dsh-session-persistence-jsonl` / `-sqlite`.
|
||||
*
|
||||
* The store can be supplied via config so two backend instances share one Map —
|
||||
* the in-RAM analogue of two backends over the same file/db, which the
|
||||
* coordinator orchestration suite's HMR/reload tests need (a fresh instance with
|
||||
* an empty in-memory states map adopting an already-materialized session).
|
||||
*/
|
||||
class MemoryPersistence extends SessionPersistence {
|
||||
private store = new Map<string, { meta: SessionHeader; events: SessionEvent[] }>()
|
||||
private pending = new Map<string, SessionHeader>()
|
||||
class MemoryPersistence extends SessionPersistence implements PersistenceBackend<never> {
|
||||
static inject = ['sessions']
|
||||
|
||||
async create(m: SessionHeader): Promise<void> {
|
||||
// Lazy: record the intended meta, but stay absent from has/list until the
|
||||
// first append materializes the session.
|
||||
this.pending.set(m.id, m)
|
||||
override readonly name = 'session-persistence-memory'
|
||||
|
||||
/** The whole durable store: materialized sessions only (no lazy entries). */
|
||||
private store: MemoryStore
|
||||
private coordinator: PersistenceCoordinator<never>
|
||||
|
||||
constructor(ctx: Context, config?: MemoryConfig) {
|
||||
super(ctx)
|
||||
// Assign the store BEFORE constructing the coordinator: the coordinator's
|
||||
// constructor installs the write path and synchronously seeds existing live
|
||||
// sessions (onCreated → loadLive → this.store), so store must exist first.
|
||||
this.store = config?.store ?? new Map<string, { meta: SessionHeader; events: SessionEvent[] }>()
|
||||
this.coordinator = new PersistenceCoordinator<never>(this.ctx, this)
|
||||
}
|
||||
|
||||
async append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
const existing = this.store.get(id)
|
||||
const nextSeq = existing ? existing.events.length : 0
|
||||
if (events.length > 0 && events[0]!.seq !== nextSeq) {
|
||||
throw new Error(`append seq mismatch for "${id}": expected ${nextSeq}, got ${events[0]!.seq}`)
|
||||
}
|
||||
for (let i = 0; i < events.length; i++) {
|
||||
const e = events[i]!
|
||||
if (e.seq !== nextSeq + i) throw new Error(`non-contiguous seq in batch for "${id}" at index ${i}`)
|
||||
if (!isJsonValue(e.data)) {
|
||||
throw new Error(`event "${e.type}" carries non-JSON-serializable data`)
|
||||
}
|
||||
// --- service surface (delegated to the coordinator) ---
|
||||
|
||||
create(m: SessionHeader): Promise<void> {
|
||||
return this.coordinator.create(m)
|
||||
}
|
||||
|
||||
append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
return this.coordinator.append(id, events)
|
||||
}
|
||||
|
||||
load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
return this.coordinator.load(id)
|
||||
}
|
||||
|
||||
has(id: SessionId): Promise<boolean> {
|
||||
return this.coordinator.has(id)
|
||||
}
|
||||
|
||||
delete(id: SessionId): Promise<void> {
|
||||
return this.coordinator.delete(id)
|
||||
}
|
||||
|
||||
/** White-box accessor: await a specific session's onCreated init. */
|
||||
get inits(): Map<Session, Promise<void>> {
|
||||
return this.coordinator.inits
|
||||
}
|
||||
|
||||
// --- PersistenceBackend hooks (the Map storage primitives) ---
|
||||
|
||||
// A Map-backed store has no torn tails, so `tornMarker` is never set. Ids are
|
||||
// globally unique, so loadStored and loadLive are identical (cwd is ignored).
|
||||
async loadStored(id: SessionId): Promise<StoredPrefix<never> | undefined> {
|
||||
const entry = this.store.get(id)
|
||||
if (!entry) return undefined
|
||||
return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) }
|
||||
}
|
||||
|
||||
loadLive(id: SessionId, _cwd: string | undefined): Promise<StoredPrefix<never> | undefined> {
|
||||
return this.loadStored(id)
|
||||
}
|
||||
|
||||
async appendBatch(m: SessionHeader, events: readonly SessionEvent[], _isMaterialized: boolean): Promise<void> {
|
||||
// Defense-in-depth: the coordinator already validates serializability, but a
|
||||
// durable store must reject non-JSON data at its own boundary too.
|
||||
for (const e of events) {
|
||||
if (!isJsonValue(e.data)) throw new Error(`event "${e.type}" carries non-JSON-serializable data`)
|
||||
}
|
||||
const existing = this.store.get(m.id)
|
||||
if (!existing) {
|
||||
const m = this.pending.get(id)
|
||||
if (!m) throw new Error(`append before create for "${id}"`)
|
||||
this.store.set(id, { meta: m, events: structuredClone(events) as SessionEvent[] })
|
||||
// First batch: `_isMaterialized` is false (the coordinator only omits
|
||||
// materialization on the first batch); writing the entry IS the materialization.
|
||||
this.store.set(m.id, { meta: structuredClone(m), events: structuredClone(events) as SessionEvent[] })
|
||||
} else {
|
||||
existing.events.push(...structuredClone(events) as SessionEvent[])
|
||||
}
|
||||
}
|
||||
|
||||
async load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
const entry = this.store.get(id)
|
||||
if (!entry) throw new Error(`session "${id}" not found`)
|
||||
// Honor the crash-recovery contract: if the stored log ends mid-turn, close
|
||||
// the orphaned turn durably with synthetic boundary events and continue from
|
||||
// the balanced length.
|
||||
const closers = interruptedTurnClosers(entry.events)
|
||||
if (closers.length > 0) entry.events.push(...structuredClone(closers))
|
||||
return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) }
|
||||
async commitRepair(m: SessionHeader, _tornMarker: undefined, closers: readonly SessionEvent[]): Promise<void> {
|
||||
// No torn tails in a Map store, so `_tornMarker` is always undefined; only the
|
||||
// synthetic closers are appended (the same DELETE+INSERT a DB backend does,
|
||||
// minus the truncate).
|
||||
const entry = this.store.get(m.id)
|
||||
/* v8 ignore next -- commitRepair only runs for a materialized (stored) session */
|
||||
if (!entry) return
|
||||
if (closers.length > 0) entry.events.push(...structuredClone(closers) as SessionEvent[])
|
||||
}
|
||||
|
||||
async deleteStored(id: SessionId): Promise<void> {
|
||||
this.store.delete(id)
|
||||
}
|
||||
|
||||
async list(): Promise<SessionHeader[]> {
|
||||
return [...this.store.values()].map(e => structuredClone(e.meta))
|
||||
}
|
||||
|
||||
async has(id: SessionId): Promise<boolean> {
|
||||
return this.store.has(id)
|
||||
}
|
||||
|
||||
async delete(id: SessionId): Promise<void> {
|
||||
this.store.delete(id)
|
||||
this.pending.delete(id)
|
||||
}
|
||||
}
|
||||
|
||||
// Run the shared contract against the in-memory backend.
|
||||
runPersistenceContract('memory', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence)
|
||||
return {
|
||||
persistence: ctx.sessionPersistence,
|
||||
@@ -78,9 +134,24 @@ runPersistenceContract('memory', async () => {
|
||||
}
|
||||
})
|
||||
|
||||
// Run the shared coordinator orchestration suite against the in-memory backend.
|
||||
// A per-fixture Map is the shared "storage", so two mounted instances see the
|
||||
// same materialized sessions (HMR/reload). `corruptTail` is OMITTED: a Map store
|
||||
// writes atomically in RAM and has no torn tails, so the suite's torn-tail test
|
||||
// self-skips (and asserts the omission). The real torn-tail repair branch is
|
||||
// covered by the jsonl/sqlite fixtures, which CAN inject one.
|
||||
runCoordinatorContract('memory', async (): Promise<CoordinatorFixture> => {
|
||||
const store: MemoryStore = new Map()
|
||||
return {
|
||||
mount: async ctx => ctx.plugin(MemoryPersistence, { store }),
|
||||
cleanup: async () => { store.clear() },
|
||||
}
|
||||
})
|
||||
|
||||
describe('SessionPersistence service registration', () => {
|
||||
it('registers as ctx.sessionPersistence and is removed on fiber dispose (HMR safety)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence)
|
||||
expect(ctx.sessionPersistence).toBeInstanceOf(SessionPersistence)
|
||||
|
||||
@@ -90,6 +161,7 @@ describe('SessionPersistence service registration', () => {
|
||||
|
||||
it('round-trips through the registered service instance', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence)
|
||||
const m = meta('reg')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
|
||||
Reference in New Issue
Block a user