mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
fix(session-query): qualify persistence revisions by store
This commit is contained in:
@@ -574,7 +574,7 @@ export interface Config {
|
||||
export type JournalMode = 'wal' | 'delete' | 'truncate' | 'persist'
|
||||
```
|
||||
|
||||
Source: [`packages/session-persistence/session-persistence-sqlite/src/index.ts:50`](../packages/session-persistence/session-persistence-sqlite/src/index.ts)
|
||||
Source: [`packages/session-persistence/session-persistence-sqlite/src/index.ts:51`](../packages/session-persistence/session-persistence-sqlite/src/index.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-session-query`
|
||||
|
||||
|
||||
@@ -32,7 +32,7 @@ Both persistent and live FTS5 tables use `unicode61`. The implementation experim
|
||||
|
||||
The shared extractor includes message text, reasoning, nested tool-call/result content, tool names and arguments, blocked-prompt reasons, todo status/content, and error or terminal status detail. Structural boundaries, stream chunks, request headers, successful completion markers, and unknown declaration-merged event/content variants produce no document. Surface classification reuses `foldSurface()` so search agrees with model-history derivation.
|
||||
|
||||
One serialized operation reads the provider-neutral `SessionPersistence` snapshot listing, compares each opaque revision with the revision stored beside the indexed session, loads only new or changed logs, reconciles rows in one transaction, and executes the query. Observation repeats when listing changes during a load; this incorporates a mutating load repair's refreshed revision before commit. Repeated queries and unchanged reopen load no full persisted logs. New, changed, and deleted sessions update on the next stable search. A source or extraction failure cannot mark a row current, and a transaction failure rolls back so a later search retries.
|
||||
One serialized operation reads the provider-neutral `SessionPersistence` snapshot listing, compares each source-qualified opaque revision with the revision stored beside the indexed session, loads only new or changed logs, reconciles rows in one transaction, and executes the query. A revision identifies its backing persistence store as well as the backend-local log revision, so reopening against the same store reuses indexed rows while switching to an independent store cannot collide on a session id and local counter. Observation repeats when listing changes during a load; this incorporates a mutating load repair's refreshed revision before commit. Repeated queries and unchanged reopen load no full persisted logs. New, changed, and deleted sessions update on the next stable search. A source or extraction failure cannot mark a row current, and a transaction failure rolls back so a later search retries.
|
||||
|
||||
Persisted documents survive restarts. Live sessions use connection-local TEMP tables, shadow the persisted base for the same id, and reveal that base on detach. Closing the database drops live rows. Unmounting persistence hides durable rows without treating absence as authoritative deletion; remounting observes and reconciles the backend again. Conflicting immutable live and durable headers fail rather than combining sources.
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@ The JSONL durable session-persistence backend — a concrete `SessionPersistence
|
||||
- **Append-only.** Committed events (at or below a flushed `turn/end`) are never rewritten. Subsequent appends are line appends at EOF + `fsync`.
|
||||
- **Crash recovery — close, don't truncate.** A crash can leave a log whose final turn never closed (real events after the last `turn/end`). `load` PRESERVES those events (a turn can be huge — they are real work) and closes the orphaned turn by durably appending synthetic boundary events: an error `tool/result` for every `tool-call` the crash left unanswered (the loop logs the assistant message before running the tools, so a mid-tool crash leaves dangling calls — and `deriveMessages()` would replay an assistant tool-call with no result, which providers reject), then a `step/end` if a step was open, then `turn/end {kind:'interrupted'}`, returning a balanced log. Only a never-fully-written **torn tail fragment** (a final line with no newline / unparseable) is `ftruncate`d away before the closers are written. See [session persistence](../../../docs/rfc/implemented/architecture/2026-06-14-session-persistence.md).
|
||||
- **Contiguous-seq.** `load` rejects a mid-log parse error or `seq` gap (unloadable); `append` rejects a batch whose first `seq` does not continue the stored log, and rejects non-JSON-serializable `event.data` naming the offending event type.
|
||||
- **Lightweight revisions.** `listSnapshots()` reads each header and returns an opaque identity from the file device/inode, size, and nanosecond mtime/ctime. Under the append-only single-writer contract, unchanged files retain revisions while append, repair, or replacement changes them without parsing event lines.
|
||||
- **Lightweight revisions.** `listSnapshots()` reads each header and returns an opaque identity from the file device/inode, size, and nanosecond mtime/ctime. Under the append-only single-writer contract, unchanged files retain revisions while append, repair, replacement, or switching to an independent root changes them without parsing event lines.
|
||||
- **Format version.** Only the current `SESSION_FORMAT_VERSION` (v0) is supported; `load` rejects any other version. While the harness is unreleased the on-disk format is pre-release/unstable: a breaking format change is absorbed at v0 (no bump until the first tagged release) and non-current logs are rejected — there is no migration (no persisted user data to preserve).
|
||||
|
||||
## Write path
|
||||
|
||||
@@ -147,6 +147,29 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
expect(loaded.events).toEqual(log) // chunks preserved, contiguous seqs
|
||||
})
|
||||
|
||||
it('source-qualifies revisions across roots while preserving same-log reopen identity', async () => {
|
||||
const m = meta('revision-source')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const revision = (await ctx.sessionPersistence.listSnapshots())[0]?.revision
|
||||
|
||||
const reopenedCtx = new Context()
|
||||
await reopenedCtx.plugin(SessionStore)
|
||||
await reopenedCtx.plugin(SessionPersistenceJsonl, { root })
|
||||
expect((await reopenedCtx.sessionPersistence.listSnapshots())[0]?.revision).toBe(revision)
|
||||
|
||||
const otherRoot = await freshRoot()
|
||||
const otherCtx = new Context()
|
||||
await otherCtx.plugin(SessionStore)
|
||||
await otherCtx.plugin(SessionPersistenceJsonl, { root: otherRoot })
|
||||
await otherCtx.sessionPersistence.create(m)
|
||||
await otherCtx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
expect((await otherCtx.sessionPersistence.listSnapshots())[0]?.revision).not.toBe(revision)
|
||||
|
||||
await reopenedCtx.fiber.dispose()
|
||||
await otherCtx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('persists a forked child seed through the existing session write path', async () => {
|
||||
const source = ctx.sessions.create(SessionId('persist-parent'), { meta: { cwd: '/workspace' } })
|
||||
appendClosedTurn(source)
|
||||
|
||||
@@ -6,7 +6,7 @@ A SQLite durable session-persistence backend — a second `SessionPersistence` i
|
||||
|
||||
## Storage model
|
||||
|
||||
Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data, source_event_seqs, surface_op)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). The two `TEXT` columns `source_event_seqs` and `surface_op` are nullable; they store the event's optional surface-metadata fields (see [session surface](../../../docs/rfc/implemented/architecture/2026-06-18-session-surface.md)). Out-of-log metadata (`SessionHeader`) and a monotonic per-log revision live in a `sessions` row. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`list` reports exactly the sessions that have a row).
|
||||
Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data, source_event_seqs, surface_op)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). The two `TEXT` columns `source_event_seqs` and `surface_op` are nullable; they store the event's optional surface-metadata fields (see [session surface](../../../docs/rfc/implemented/architecture/2026-06-18-session-surface.md)). Out-of-log metadata (`SessionHeader`) and a monotonic per-log revision live in a `sessions` row; a singleton state row carries the immutable store id. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`list` reports exactly the sessions that have a row).
|
||||
|
||||
The repo's `engines.node` is `^22.19.0 || >=24.0.0` (Node 22.19+ or 24+), matching the LTS floor required by the installed Pi adapter dependency; `node:sqlite` itself ships without the `--experimental-sqlite` flag from Node 22.13 (LTS) and 23.4 / 24 (Current) on. The range deliberately excludes Node 23 because that line is non-LTS/EOL and still has flagged runtime features before 23.6. The database opens with `foreign_keys = ON` (so `ON DELETE CASCADE` drops a session's events with its row) and the configured `journal_mode` (default `wal`; pick a rollback-journal mode like `delete` on filesystems where WAL's shared-memory files do not work, e.g. network mounts). The table-layout version is stored in `PRAGMA user_version` and checked on open: a fresh database is stamped with the current `SCHEMA_VERSION`; a database written by any other, incompatible build (a non-current `user_version`, older or newer) is rejected rather than opened against an unknown layout — there is no migration (unreleased software).
|
||||
|
||||
@@ -14,7 +14,7 @@ The repo's `engines.node` is `^22.19.0 || >=24.0.0` (Node 22.19+ or 24+), matchi
|
||||
|
||||
- **Append = a transaction.** `append` runs `BEGIN`/`COMMIT` around the batch: it materializes the `sessions` row (if still lazy) and INSERTs every event, asserting the contiguous-seq contract first (the first event's `seq` must equal the stored next-seq). A mid-batch failure (a UNIQUE violation on a duplicated seq) rolls back entirely, so the stored log and the in-memory cursor stay consistent. (`load()` already balanced the stored log, so `append` never has to repair a crash tail.)
|
||||
- **Lazy materialization.** `create()` records intent in memory only — no row is written until the first `append`. A created-but-never-appended session has no `sessions` row, so it is absent from `list()` (which reports exactly the sessions that have a row).
|
||||
- **Lightweight revisions.** `listSnapshots()` reads the monotonic revision stored beside each session header. Append and mutating load repair increment it in the same transaction as their event changes, so unchanged observations are stable and no full-log count or parse is required.
|
||||
- **Lightweight revisions.** `listSnapshots()` combines the database's immutable random store id and physical file identity with the monotonic revision stored beside each session header; an in-memory database uses the store id alone. Append and mutating load repair increment the local counter in the same transaction as their event changes, so unchanged same-file observations are stable, independent stores and file replacements cannot collide on a local counter, and no full-log count or parse is required.
|
||||
- **Interrupted-turn close on load.** `load()` reads every stored event ordered by `seq` and finds the longest seq-contiguous, parseable prefix — INCLUDING the real events of an interrupted final turn after the last `turn/end` (the loop only flushes at `turn/end`, so a process killed mid-turn leaves real, fully-written rows past it). A single turn can be huge in a long-horizon task, so those events are **preserved, never truncated**: `load()` CLOSES the orphaned turn by durably appending the minimal synthetic boundary events (an error `tool/result` for every assistant tool call left unanswered, a `step/end` if a step was open, then a `turn/end` carrying `{ kind: 'interrupted' }`), inside one transaction that also DELETEs any never-fully-written torn tail row. `load()` is therefore mutating — after it the stored rows are balanced and the cursor is truthful, so the next `append` continues cleanly. The boundary (last `turn/end`, torn-tail detection) is computed from the `seq`/`type` columns so a malformed `data` in a torn tail row is never parsed (discarded, not unloadable). A parse error or `seq` gap inside the committed region (at or before the last real `turn/end`) makes the session unloadable. A session whose only turn never closed keeps its metadata row and stays present in `list()` — the same as the JSONL backend, whose file likewise survives a first append that never reached `turn/end`.
|
||||
|
||||
## Configuration (schemastery)
|
||||
|
||||
@@ -19,6 +19,7 @@
|
||||
|
||||
import { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import { statSync } from 'node:fs'
|
||||
import { DatabaseSync } from 'node:sqlite'
|
||||
import { mkdir } from 'node:fs/promises'
|
||||
import { dirname, resolve } from 'node:path'
|
||||
@@ -84,6 +85,7 @@ export class SessionPersistenceSqlite extends SessionPersistence implements Pers
|
||||
override readonly name = 'session-persistence-sqlite'
|
||||
|
||||
private db!: DatabaseSync
|
||||
private storeIdentity!: string
|
||||
private ready: Promise<void>
|
||||
private coordinator: PersistenceCoordinator<number>
|
||||
|
||||
@@ -98,12 +100,29 @@ export class SessionPersistenceSqlite extends SessionPersistence implements Pers
|
||||
}
|
||||
|
||||
private async openDb(path: string, journalMode: JournalMode): Promise<void> {
|
||||
if (path !== ':memory:') {
|
||||
const abs = resolve(path)
|
||||
await mkdir(dirname(abs), { recursive: true, mode: 0o700 })
|
||||
this.db = openDatabase(abs, journalMode)
|
||||
} else {
|
||||
this.db = openDatabase(path, journalMode)
|
||||
const actual = path === ':memory:' ? path : resolve(path)
|
||||
if (actual !== ':memory:') await mkdir(dirname(actual), { recursive: true, mode: 0o700 })
|
||||
this.db = openDatabase(actual, journalMode)
|
||||
try {
|
||||
const row = this.db.prepare(
|
||||
'SELECT store_id FROM persistence_state WHERE singleton = 1',
|
||||
).get() as { store_id: string } | undefined
|
||||
/* v8 ignore next -- openDatabase inserts the singleton before returning. */
|
||||
if (row === undefined) {
|
||||
throw new Error(`session database at "${actual}" has no store identity`)
|
||||
}
|
||||
if (row.store_id.length === 0) {
|
||||
throw new Error(`session database at "${actual}" has no valid store identity`)
|
||||
}
|
||||
if (actual !== ':memory:') {
|
||||
const identity = statSync(actual, { bigint: true })
|
||||
this.storeIdentity = `file:${identity.dev}:${identity.ino}:${identity.birthtimeNs}:store:${row.store_id}`
|
||||
} else {
|
||||
this.storeIdentity = `memory:store:${row.store_id}`
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
this.db.close()
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
@@ -234,13 +253,13 @@ export class SessionPersistenceSqlite extends SessionPersistence implements Pers
|
||||
return rows.map(rowToMeta)
|
||||
}
|
||||
|
||||
/** List metadata with an append-only event-count revision per session. */
|
||||
/** List metadata with a source-qualified monotonic revision per session. */
|
||||
async listSnapshots(): Promise<SessionPersistenceSnapshot[]> {
|
||||
await this.ready
|
||||
const rows = this.db.prepare('SELECT * FROM sessions').all() as unknown as SessionRow[]
|
||||
return rows.map(row => ({
|
||||
header: rowToMeta(row),
|
||||
revision: SessionPersistenceRevision(`revision:${row.revision}`),
|
||||
revision: SessionPersistenceRevision(`${this.storeIdentity}:revision:${row.revision}`),
|
||||
}))
|
||||
}
|
||||
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
/**
|
||||
* Schema + load-time helpers for the SQLite session-persistence backend: the
|
||||
* DDL (a `sessions` metadata table and a 1:1 `events` row per `SessionEvent`),
|
||||
* the database open/configure step, and the last-`turn/end` cut that gives the
|
||||
* SQLite backend the SAME crash-tail-on-load semantics as the JSONL backend.
|
||||
* DDL (a store-identity row, `sessions` metadata, and a 1:1 `events` row per
|
||||
* `SessionEvent`), the database open/configure step, and the last-`turn/end`
|
||||
* cut that gives the SQLite backend the SAME crash-tail-on-load semantics as
|
||||
* the JSONL backend.
|
||||
*
|
||||
* @module dsh-session-persistence-sqlite/schema
|
||||
*/
|
||||
|
||||
import { randomUUID } from 'node:crypto'
|
||||
import { DatabaseSync } from 'node:sqlite'
|
||||
import type { SessionEvent, SessionId, SessionHeader, SurfaceOp } from '@deepseek-ai/dsh-session'
|
||||
|
||||
@@ -15,7 +17,7 @@ import type { SessionEvent, SessionId, SessionHeader, SurfaceOp } from '@deepsee
|
||||
* layout; orthogonal to a session's own `version` (which versions the EVENT
|
||||
* vocabulary, stored per session in the `sessions` row).
|
||||
*/
|
||||
export const SCHEMA_VERSION = 5
|
||||
export const SCHEMA_VERSION = 6
|
||||
|
||||
/**
|
||||
* A row of the `sessions` table — the out-of-log metadata ({@link SessionHeader}).
|
||||
@@ -70,14 +72,25 @@ export type JournalMode = 'wal' | 'delete' | 'truncate' | 'persist'
|
||||
* current one (written by a different, incompatible build — older or newer) is
|
||||
* REJECTED rather than opened against a layout this build does not understand.
|
||||
* There are no migrations: an incompatible layout is rejected. The current
|
||||
* sessions row carries every header field plus its monotonic snapshot revision;
|
||||
* the events row carries the complete surface metadata.
|
||||
* persistence-state row carries an immutable random store id, the sessions row
|
||||
* carries every header field plus its monotonic snapshot revision, and the
|
||||
* events row carries the complete surface metadata.
|
||||
* @param path - the SQLite database file to open (created when absent).
|
||||
* @param journalMode - the journal pragma to apply — a closed in-code union, validated by the plugin Config.
|
||||
* @returns the open handle with pragmas applied and both tables ensured.
|
||||
* @returns the open handle with pragmas applied and all three tables ensured.
|
||||
*/
|
||||
export function openDatabase(path: string, journalMode: JournalMode): DatabaseSync {
|
||||
const db = new DatabaseSync(path)
|
||||
try {
|
||||
configureDatabase(db, path, journalMode)
|
||||
return db
|
||||
} catch (error: unknown) {
|
||||
db.close()
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
function configureDatabase(db: DatabaseSync, path: string, journalMode: JournalMode): void {
|
||||
db.exec('PRAGMA foreign_keys = ON')
|
||||
// journalMode is a closed in-code union (validated by the plugin Config), not
|
||||
// user-controlled SQL — safe to interpolate (PRAGMA takes no bound params).
|
||||
@@ -85,7 +98,6 @@ export function openDatabase(path: string, journalMode: JournalMode): DatabaseSy
|
||||
// `PRAGMA user_version` always returns exactly one row { user_version }.
|
||||
const { user_version: onDisk } = db.prepare('PRAGMA user_version').get() as { user_version: number }
|
||||
if (onDisk !== 0 && onDisk !== SCHEMA_VERSION) {
|
||||
db.close()
|
||||
throw new Error(`session database at "${path}" has schema version ${onDisk}, incompatible with this build (${SCHEMA_VERSION})`)
|
||||
}
|
||||
if (onDisk === 0) {
|
||||
@@ -94,6 +106,15 @@ export function openDatabase(path: string, journalMode: JournalMode): DatabaseSy
|
||||
// constant (SCHEMA_VERSION is a trusted in-code number, not user input).
|
||||
db.exec(`PRAGMA user_version = ${SCHEMA_VERSION}`)
|
||||
}
|
||||
db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS persistence_state (
|
||||
singleton INTEGER PRIMARY KEY CHECK (singleton = 1),
|
||||
store_id TEXT NOT NULL
|
||||
) STRICT
|
||||
`)
|
||||
db.prepare(
|
||||
'INSERT OR IGNORE INTO persistence_state (singleton, store_id) VALUES (1, ?)',
|
||||
).run(randomUUID())
|
||||
db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS sessions (
|
||||
id TEXT PRIMARY KEY,
|
||||
@@ -117,7 +138,6 @@ export function openDatabase(path: string, journalMode: JournalMode): DatabaseSy
|
||||
PRIMARY KEY (session_id, seq)
|
||||
) STRICT
|
||||
`)
|
||||
return db
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { existsSync } from 'node:fs'
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { mkdtemp, rm, symlink } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
@@ -245,12 +245,12 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
|
||||
dbNewer.close()
|
||||
expect(() => openDatabase(path, 'wal')).toThrow(/incompatible with this build/)
|
||||
|
||||
// A stale OLDER version (e.g. a pre-summary-drop v1 DB) is also rejected —
|
||||
// we do not migrate (unreleased software, no backward-compat).
|
||||
// The immediately preceding layout lacks the required store identity and is
|
||||
// rejected rather than migrated (unreleased software, no backward-compat).
|
||||
const olderPath = await freshDbPath()
|
||||
openDatabase(olderPath, 'wal').close()
|
||||
const dbOlder = openDatabase(olderPath, 'wal')
|
||||
dbOlder.exec('PRAGMA user_version = 1')
|
||||
dbOlder.exec(`PRAGMA user_version = ${SCHEMA_VERSION - 1}`)
|
||||
dbOlder.close()
|
||||
expect(() => openDatabase(olderPath, 'wal')).toThrow(/incompatible with this build/)
|
||||
})
|
||||
@@ -337,8 +337,46 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
|
||||
await fiber2.dispose()
|
||||
})
|
||||
|
||||
it('source-qualifies revisions across stores while preserving same-file reopen identity', async () => {
|
||||
const pathA = await freshDbPath()
|
||||
const pathB = await freshDbPath()
|
||||
const m = meta('revision-source')
|
||||
const a = await backend(pathA)
|
||||
await a.ctx.sessionPersistence.create(m)
|
||||
await a.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const revisionA = (await a.ctx.sessionPersistence.listSnapshots())[0]?.revision
|
||||
await a.dispose()
|
||||
|
||||
const probeA = openDatabase(pathA, 'wal')
|
||||
const storeIdA = (probeA.prepare(
|
||||
'SELECT store_id FROM persistence_state WHERE singleton = 1',
|
||||
).get() as { store_id: string }).store_id
|
||||
probeA.close()
|
||||
|
||||
const aliasA = `${pathA}.alias`
|
||||
await symlink(pathA, aliasA)
|
||||
const reopenedA = await backend(aliasA)
|
||||
expect((await reopenedA.ctx.sessionPersistence.listSnapshots())[0]?.revision).toBe(revisionA)
|
||||
await reopenedA.dispose()
|
||||
|
||||
const b = await backend(pathB)
|
||||
await b.ctx.sessionPersistence.create(m)
|
||||
await b.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const revisionB = (await b.ctx.sessionPersistence.listSnapshots())[0]?.revision
|
||||
const probeB = openDatabase(pathB, 'wal')
|
||||
const storeIdB = (probeB.prepare(
|
||||
'SELECT store_id FROM persistence_state WHERE singleton = 1',
|
||||
).get() as { store_id: string }).store_id
|
||||
probeB.close()
|
||||
expect(storeIdB).not.toBe(storeIdA)
|
||||
expect(revisionB).not.toBe(revisionA)
|
||||
expect(String(revisionA)).toMatch(/:revision:1$/)
|
||||
expect(String(revisionB)).toMatch(/:revision:1$/)
|
||||
await b.dispose()
|
||||
})
|
||||
|
||||
it('exposes the schema version constant', () => {
|
||||
expect(SCHEMA_VERSION).toBe(5)
|
||||
expect(SCHEMA_VERSION).toBe(6)
|
||||
})
|
||||
|
||||
it('keeps the revision stable for an empty repair hook', async () => {
|
||||
@@ -354,6 +392,17 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
|
||||
})
|
||||
|
||||
describe('SessionPersistenceSqlite: edge cases', () => {
|
||||
it('rejects and closes a current-schema database with an invalid store identity', async () => {
|
||||
const path = await freshDbPath()
|
||||
const db = openDatabase(path, 'wal')
|
||||
db.exec("UPDATE persistence_state SET store_id = '' WHERE singleton = 1")
|
||||
db.close()
|
||||
|
||||
const b = await backend(path)
|
||||
await expect(b.ctx.sessionPersistence.listSnapshots()).rejects.toThrow(/no valid store identity/)
|
||||
await expect(b.dispose()).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('append rolls back and rethrows when an event INSERT fails inside the transaction', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('rollback-insert')
|
||||
|
||||
@@ -12,7 +12,7 @@ The persisted unit IS the existing `SessionEvent` (event-sourced model — the l
|
||||
| `append(id, events): Promise<void>` | Durably persist a batch (from the `session/flush` drain). Append-only; first event `seq` == stored next-seq after any repair; rejects non-JSON-serializable data naming the offending type. |
|
||||
| `load(id): Promise<{ meta; events }>` | Reload meta + log. Preserves an interrupted (unclosed) final turn and closes it with synthetic closers — an error `tool/result` per unanswered `tool-call`, then `step/end?`+`turn/end {interrupted}` (a turn can be huge — never truncated); only a torn tail fragment is dropped. Events contiguous (`events[i].seq === i`); rejects a committed-region gap/parse error or unknown `version`. |
|
||||
| `list(): Promise<SessionHeader[]>` | Lightweight listing from metadata, no full-log parse. A zero-event lazily-materialized session is absent from `list`. |
|
||||
| `listSnapshots(): Promise<SessionPersistenceSnapshot[]>` | Lightweight metadata plus an opaque branded per-log revision, without loading event logs. A revision stays equal while that log is unchanged and changes after append or mutating load repair. |
|
||||
| `listSnapshots(): Promise<SessionPersistenceSnapshot[]>` | Lightweight metadata plus an opaque branded per-log revision, without loading event logs. A revision stays equal while that log and its backing store are unchanged, changes after append or mutating load repair, and cannot collide solely because two stores use the same local counter. |
|
||||
|
||||
## Invariants every backend must honor
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ export { SessionPersistenceRevision } from './revision.ts'
|
||||
export interface SessionPersistenceSnapshot {
|
||||
/** Detached metadata for one materialized session. */
|
||||
header: SessionHeader
|
||||
/** Opaque token that changes whenever this stored log changes. */
|
||||
/** Opaque source-qualified token that changes whenever this stored log changes. */
|
||||
revision: SessionPersistenceRevision
|
||||
}
|
||||
|
||||
@@ -172,6 +172,8 @@ export abstract class SessionPersistence extends Service {
|
||||
*
|
||||
* Repeated observations of an unchanged log return the same revision. A
|
||||
* successful mutating {@link load} repair changes the next listed revision.
|
||||
* Revisions also distinguish independently backed stores so backend-local
|
||||
* counters cannot compare equal across different persistence sources.
|
||||
* @returns one header and opaque revision per materialized session without loading full logs.
|
||||
*/
|
||||
abstract listSnapshots(): Promise<SessionPersistenceSnapshot[]>
|
||||
|
||||
@@ -2,7 +2,10 @@
|
||||
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
|
||||
/** Backend-owned token that changes whenever one persisted session log changes. */
|
||||
/**
|
||||
* Backend-owned token that identifies both one storage source and one revision
|
||||
* of a persisted session log.
|
||||
*/
|
||||
export type SessionPersistenceRevision = Branded<'SessionPersistenceRevision'>
|
||||
|
||||
/**
|
||||
|
||||
@@ -12,7 +12,7 @@ All three surfaces (`current`, `shadowed`, and `log-only`) are searchable by def
|
||||
|
||||
## Source and index lifecycle
|
||||
|
||||
The service requires `ctx.sessions` and observes optional `ctx.sessionPersistence` dynamically. One serialized state machine compares lightweight durable snapshot revisions, loads only new or changed logs, extracts shared semantic documents, reconciles changes transactionally, and runs the query. Repeated queries and unchanged reopen load no full durable logs; new, changed, deleted, or load-repaired sources reconcile on the next stable observation. Source or transaction failure commits nothing, and the next search retries.
|
||||
The service requires `ctx.sessions` and observes optional `ctx.sessionPersistence` dynamically. One serialized state machine compares source-qualified lightweight durable snapshot revisions, loads only new or changed logs, extracts shared semantic documents, reconciles changes transactionally, and runs the query. Repeated queries and unchanged same-store reopen load no full durable logs; switching stores, or observing new, changed, deleted, or load-repaired sources, reconciles on the next stable observation. Source or transaction failure commits nothing, and the next search retries.
|
||||
|
||||
Persisted FTS rows live in a dedicated derived database. Connection-local TEMP tables hold live rows, which shadow the durable base for the same session and reveal it when the live owner disappears. Unmounting persistence hides durable rows without discarding the cache; remounting reconciles it. Closing or reopening the database drops every live overlay while retaining persisted rows.
|
||||
|
||||
|
||||
@@ -926,4 +926,49 @@ describe('SQLite schema, cancellation, and real persistence integration', () =>
|
||||
await expect(ctx.sessionPersistence.load(meta.id)).resolves.toMatchObject({ meta, events: [{ seq: 0 }] })
|
||||
await persistence.dispose()
|
||||
})
|
||||
|
||||
it('reconciles colliding local revisions when a derived index reopens against another SQLite store', async () => {
|
||||
const persistencePathA = await temporaryPath('canonical-a.db')
|
||||
const persistencePathB = await temporaryPath('canonical-b.db')
|
||||
const searchPath = await temporaryPath('derived-collision.db')
|
||||
const shared = header('same-id', 10)
|
||||
|
||||
const first = new Context()
|
||||
await first.plugin(SessionStore)
|
||||
const persistenceA = await first.plugin(SessionPersistenceSqlite, { path: persistencePathA })
|
||||
await first.sessionPersistence.create(shared)
|
||||
await first.sessionPersistence.append(shared.id, messageEvents('alpha source'))
|
||||
const loadA = vi.spyOn(first.sessionPersistence, 'load')
|
||||
const searchA = await first.plugin(SessionSearchSqlite, { path: searchPath })
|
||||
await expect(first.sessionSearch.searchSessions({ query: 'alpha' }))
|
||||
.resolves.toMatchObject({ items: [{ header: shared }] })
|
||||
expect(loadA).toHaveBeenCalledTimes(1)
|
||||
await searchA.dispose()
|
||||
await persistenceA.dispose()
|
||||
|
||||
const reopened = new Context()
|
||||
await reopened.plugin(SessionStore)
|
||||
const persistenceAAgain = await reopened.plugin(SessionPersistenceSqlite, { path: persistencePathA })
|
||||
const reopenedLoad = vi.spyOn(reopened.sessionPersistence, 'load')
|
||||
const searchAAgain = await reopened.plugin(SessionSearchSqlite, { path: searchPath })
|
||||
await expect(reopened.sessionSearch.searchSessions({ query: 'alpha' }))
|
||||
.resolves.toMatchObject({ items: [{ header: shared }] })
|
||||
expect(reopenedLoad).not.toHaveBeenCalled()
|
||||
await searchAAgain.dispose()
|
||||
await persistenceAAgain.dispose()
|
||||
|
||||
const second = new Context()
|
||||
await second.plugin(SessionStore)
|
||||
const persistenceB = await second.plugin(SessionPersistenceSqlite, { path: persistencePathB })
|
||||
await second.sessionPersistence.create(shared)
|
||||
await second.sessionPersistence.append(shared.id, messageEvents('bravo source'))
|
||||
const loadB = vi.spyOn(second.sessionPersistence, 'load')
|
||||
const searchB = await second.plugin(SessionSearchSqlite, { path: searchPath })
|
||||
await expect(second.sessionSearch.searchSessions({ query: 'bravo' }))
|
||||
.resolves.toMatchObject({ items: [{ header: shared }] })
|
||||
await expect(second.sessionSearch.searchSessions({ query: 'alpha' })).resolves.toEqual({ items: [] })
|
||||
expect(loadB).toHaveBeenCalledTimes(1)
|
||||
await searchB.dispose()
|
||||
await persistenceB.dispose()
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user