deepseek-harness/docs/adr/0016-session-persistence.md

ADR 0016: Session persistence as an abstract service over the existing SessionEvent

Status: accepted (2026-06-15)

Context

Sessions lived only in memory. The example session-jsonl.ts plugin (duplicated byte-for-byte in both examples) was write-only telemetry: it buffered session/event and appended JSON lines, with no read/replay path, no crash-safety (no fsync, no atomic write, a fire-and-forget dispose drain), no listing, and no format versioning. Nothing could rehydrate a past session from disk into a live agent, so durable resume ("continue yesterday's task"), durable forking, and the ACP session/load method (RFC 010) were all impossible.

The event-sourced model makes the append-only log the single source of truth and derives LLM history from it. Persistence had to stay faithful to that: persist the existing SessionEvent directly, with no parallel "persisted message" type that the log is converted to and from. The backend also had to be swappable — a file store now, a database store later — behind one interface.

Decision

Persistence is an abstract capability seam (ADR 0009, the dsh-bash template), not loop or core logic:

1. Interface (dsh-session-persistence, ctx.sessionPersistence) — an abstract SessionPersistence service: create/append/load/list/has/delete/update. Its persisted unit IS the existing SessionEvent ({ type, seq, time, data }), reused verbatim — no conversion type.
2. Implementation (dsh-session-persistence-jsonl) — an append-only JSONL log per session (a SessionHeader line then one SessionEvent per line, verbatim including assistant/chunk) plus an atomic .summary.json sidecar for the mutable SessionSummary.

Key choices recorded here because they are durable, contested, and surprising:

• The canonical durable log persists every SessionEvent verbatim, including assistant/chunk. deriveMessages() skips chunks, and a chunk-filtered rollout (Codex's policy.rs) is tempting — but seq = log.length and the load-validation events[i].seq === i require a contiguous log; filtering chunks out would leave holes and break both the contract and resume. A chunk-filtered projection is possible later as a derived view with its own renumbering, but it is NOT the canonical log.
• Append-only with a single exception. Committed events — those at or below a flushed turn/end — are never rewritten. The loop only flushes at turn/end, so a crash can leave a half-written final turn below the last checkpoint; load returns events only up to the last complete turn/end, and the first post-load append runs a one-time truncation-repair (ftruncate + fsync) that physically discards only that never-committed crash tail before writing.
• File backend canonical, DB backend a drop-in. SessionEvent maps 1:1 onto a row (session_id, seq, type, time, data) — append is INSERT (in a transaction asserting the contiguous-seq contract), load is SELECT … ORDER BY seq. A future dsh-session-persistence-sqlite is a SessionPersistence subclass with no interface change (opencode runs this exact shape on SQLite/WAL).
• Metadata is out-of-log. Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a SessionMeta (SessionHeader & SessionSummary) owned by dsh-session and attached to a Session via a new readonly session.header — never in SessionEventMap, never reaching deriveMessages(). The alternative (a merge-extensible session/meta event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header seam is the cleaner cost.
• Resume is an async factory, not a change to synchronous create. ctx.agents.resume({ resumeSessionId }) awaits ctx.sessionPersistence.load, recreates the live session with the loaded events (so lastTurnNumber/deriveMessages continue), and starts a fresh agent on the resumed id (NOT ${agentId}-session). The agent-loop does NOT hard-inject sessionPersistence (that would pend non-persistent demos forever); resume rejects with a clear error when it is absent.

Format versioning: the header carries a version; load rejects an unknown version (no v1 migration). Stated honestly: append-only + flush is robust to partial trailing writes (tolerated on load) but not to fsync-less power loss mid-line; a DB/WAL backend is the stronger option later.

Consequences

Two new packages and the metadata seam in dsh-session (session.header, the create(id?, options?) signature). Bought: durable resume/fork, a read/replay path, crash tolerance, and the foundation RFC 010's session/load needs — all over the existing event-sourced log, with the backend swappable behind one interface. The reusable runPersistenceContract suite holds every backend to the same append-only / contiguous-seq / lazy-materialization / serializability semantics. This completes ADR 0003's deferred "real persistence backend" and resolves its TODO(review) on the event vocabulary: persisting the log freezes its shape, and the assistant/chunk fidelity question is answered above (persist verbatim).