mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
Merge branch 'master' into worktree/web-carrier-chain
This commit is contained in:
@@ -16,11 +16,10 @@ The coordinator retires each live session from its `session/disposed` notificati
|
||||
|
||||
### The hook interface (`PersistenceBackend<TornMarker>`)
|
||||
|
||||
Six methods (five required + an optional lifecycle hook) — the only seam between the coordinator and storage:
|
||||
Five required members plus an optional lifecycle hook form the only boundary between the coordinator and storage:
|
||||
|
||||
- `name` — backend label for the dispose-failure `AggregateError`.
|
||||
- `loadStored(id)` — read a stored prefix by id, scanning ANY storage scope (every JSONL cwd bucket; SQLite's id is globally unique). Used by resume/load and, via `!== undefined`, the create-collision probe.
|
||||
- `loadLive(id, cwd)` — read a stored prefix SCOPED to `cwd`. **Deliberately distinct from `loadStored`**: HMR live-adoption must only adopt a persisted log at the SAME cwd as the live session; a same-id log at a different cwd is a collision, not a resume. Collapsing the two reintroduces a cross-cwd adoption bug. SQLite ignores `cwd`.
|
||||
- `loadStored(id)` — read one stored prefix by id across every storage scope (every JSONL cwd bucket; SQLite's id is globally unique). Resume/load, live adoption, and the create-collision probe share this lookup. The coordinator asserts the returned id and rejects a stored/live cwd mismatch before repair or state publication.
|
||||
- `appendBatch(meta, events, isMaterialized)` — durably append a contiguous batch, lazily materializing the session ATOMICALLY when not yet materialized (the materialize-write and the first event batch must commit together — a crash between them must not leave a materialized-but-empty session; this is why there is no separate `materialize` hook).
|
||||
- `commitRepair(meta, tornMarker, closers)` — make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined`) and append `closers`. **NOT required to be atomic** — JSONL legitimately truncates-then-appends in two fsync'd steps, SQLite does DELETE+INSERT in one transaction. Used by `load` (truncate + synthetic closers) and live-adoption (truncate only, `closers = []`).
|
||||
- `list()` — list all stored metadata.
|
||||
@@ -37,8 +36,8 @@ The shared `runPersistenceContract` (public-API contract) keeps running for ever
|
||||
## Alternatives considered
|
||||
|
||||
- **A base class the backends extend** — rejected for composition: a backend exposes only the hooks, cannot reach the coordinator's private orchestration state, and a third-party backend may still implement the abstract service directly without the coordinator at all.
|
||||
- **A wider hook surface** — each candidate hook folded away: there is no separate `materialize` hook (the materialize-write must commit atomically with the first event batch inside `appendBatch`), no separate create-collision probe (it is `loadStored(id) !== undefined`), and no coordinator pass-through for `list()` (listing needs none of the orchestration).
|
||||
- **A wider hook surface** — each candidate hook folds away: there is no scope-specific live lookup because `loadStored` plus the coordinator's cwd check preserves the collision boundary, no storage-locator generic because validated JSONL metadata reproduces its path while SQLite is already id-bound, no separate `materialize` hook because the first batch must commit atomically with materialization, no separate create-collision probe because it is `loadStored(id) !== undefined`, and no coordinator pass-through for `list()` because listing needs none of the orchestration.
|
||||
|
||||
## Consequences
|
||||
|
||||
The coordinator adds one indirection, an opaque torn marker, and detached session-retirement tasks, but centralizes correctness-heavy orchestration previously duplicated by every backend. Session disposal remains an observe-only event, so the session owner does not await persistence retirement; the coordinator contains failures, preserves uncommitted buffers, and makes backend teardown the quiescence boundary. Its hook surface stays narrow: collision checks reuse `loadStored`, materialization stays atomic inside `appendBatch`, and listing bypasses the coordinator. New backends implement storage primitives rather than copy the event-buffer-flush lifecycle.
|
||||
The coordinator adds one indirection, an opaque torn marker, and detached session-retirement tasks, but centralizes correctness-heavy orchestration previously duplicated by every backend. Session disposal remains an observe-only event, so the session owner does not await persistence retirement; the coordinator contains failures, preserves uncommitted buffers, and makes backend teardown the quiescence boundary. Its hook surface stays narrow: identity, adoption, and collision checks reuse `loadStored`; materialization stays atomic inside `appendBatch`; and listing bypasses the coordinator. New backends implement storage primitives rather than copy the event-buffer-flush lifecycle.
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-20-jsonl-storage-identity.md: 1ada16791f411a54fbcf9271c7d7963223bbe683
|
||||
2026-07-20-jsonl-storage-identity.zh.md: 8027c51dbf6c7d01463b7851d859a40890bf03e1
|
||||
@@ -0,0 +1,29 @@
|
||||
# Agent Note: Bind JSONL session identity before mutation
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-20-jsonl-storage-identity.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
JSONL lookup selects a physical log from the requested session id across cwd buckets, while the parsed `SessionHeader` supplies the metadata used by later repair and append operations. Without binding those two facts, a log selected for session A can declare session B's id or cwd and redirect a repair or later append to B's path. The bucket scan also needs a defined result when the same encoded id exists in more than one bucket. SQLite does not share this ambiguity because its primary-key query binds metadata and events to the requested id.
|
||||
|
||||
## Decision
|
||||
|
||||
`loadStored(id)` is the coordinator's single stored-prefix lookup. The JSONL backend scans every cwd bucket, requires at most one matching encoded filename, parses that file, then validates both `header.id === id` and `selectedPath === logPath(root, header.cwd, header.id)` before returning metadata. `list()` applies the same path validation and rejects duplicate ids across buckets.
|
||||
|
||||
The coordinator independently asserts the returned id and compares the stored cwd with a live session's cwd before repair, state publication, or suffix persistence. It keeps a detached copy of validated metadata; JSONL append and repair derive their path from that copy. The `PersistenceBackend<TornMarker>` interface therefore needs neither a scope-specific live lookup nor a storage-locator type.
|
||||
|
||||
An existing configured JSONL root must be a readable directory when the plugin loads. An absent root remains valid and is created on first materialization. The backend supports one live writer per session; another backend instance or process must not mutate that session until the owner finishes disposal and all writes stop.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Flatten storage by session id.** A flat namespace makes duplicate publication collide on one path, but path validation and duplicate rejection close the identity defect without changing the project-grouped cwd layout or its consumers.
|
||||
|
||||
**Carry an opaque storage locator through the coordinator.** A locator binds JSONL mutations directly to a selected path, but JSONL can reproduce that path from metadata it has already validated. Adding another generic and argument to SQLite, test backends, append, and repair makes every implementation carry a concept only the file backend needs.
|
||||
|
||||
**Coordinate multiple live writers.** A dedicated coordination service, process-global registry, or cross-process lock would define a new deployment topology rather than repair identity validation. The supported topology has one live writer; no-overwrite hard-link publication still arbitrates an initial same-id creation race.
|
||||
|
||||
## Consequences
|
||||
|
||||
Mismatched, misplaced, and duplicate JSONL logs fail before repair or coordinator state mutation. The cwd-bucket format stays unchanged and needs no migration. Lookup remains proportional to the number of buckets, and one-live-writer ownership remains an explicit limitation. Coordinator and JSONL tests pin rejection before repair, unchanged bytes for both affected logs, path validation during listing, duplicate-id rejection, cwd collision handling, and load-time root validation.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Agent Note: 在变更前绑定 JSONL 会话身份
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-20-jsonl-storage-identity.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
JSONL 查找会根据请求的会话 id 在各个 cwd 分桶目录中选出物理日志,而解析得到的 `SessionHeader` 会提供后续修复和追加操作使用的元数据。如果这两个事实没有绑定,为会话 A 选中的日志就能声明会话 B 的 id 或 cwd,并将修复或后续追加重定向到 B 的路径。当同一个编码后 id 出现在多个分桶目录中时,分桶扫描也必须给出确定的结果。SQLite 不存在这种歧义,因为主键查询会将元数据和事件绑定到请求的 id。
|
||||
|
||||
## 决策
|
||||
|
||||
`loadStored(id)` 是协调器唯一的已存前缀查找操作。JSONL 后端扫描所有 cwd 分桶目录,要求匹配编码文件名的日志至多有一个,解析该文件,然后在返回元数据前同时验证 `header.id === id` 和 `selectedPath === logPath(root, header.cwd, header.id)`。`list()` 执行相同的路径验证,并拒绝跨分桶目录重复的 id。
|
||||
|
||||
协调器会独立断言返回的 id,并在修复、发布状态或持久化后缀之前比较已存 cwd 和活动会话的 cwd。协调器保留一份已验证元数据的独立副本;JSONL 的追加和修复操作根据该副本派生路径。因此,`PersistenceBackend<TornMarker>` 接口既不需要限定范围的活动会话查找,也不需要存储定位器类型。
|
||||
|
||||
如果配置的 JSONL 根目录已存在,插件加载时该路径必须是可读目录。根目录不存在仍然是有效配置,首次物化时会创建该目录。后端对每个会话只支持一个活动写入方;在所有者完成资源释放且所有写入停止之前,另一个后端实例或进程不得变更该会话。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**按会话 id 扁平化存储。** 扁平命名空间会让重复发布在同一路径上冲突,但路径验证和重复项拒绝无需改变按项目分组的 cwd 布局及其消费方,也能消除身份缺陷。
|
||||
|
||||
**通过协调器传递不透明存储定位器。** 定位器可以将 JSONL 变更直接绑定到选定路径,但 JSONL 可以根据已经验证的元数据重新得到该路径。为 SQLite、测试后端、追加和修复操作增加一个泛型和参数,会让每个实现都承担只有文件后端需要的概念。
|
||||
|
||||
**协调多个活动写入方。** 专用协调服务、进程级全局注册表或跨进程锁会定义新的部署拓扑,而不是修复身份验证。受支持的拓扑只有一个活动写入方;禁止覆盖的硬链接发布仍会裁决初始的同 id 创建竞争。
|
||||
|
||||
## 后果
|
||||
|
||||
JSONL 日志的身份不匹配、位置错误和重复会在修复或协调器状态变更前失败。cwd 分桶格式保持不变,无需迁移。查找开销仍与分桶目录数量成正比,单一活动写入方的所有权仍是明确限制。协调器和 JSONL 测试固定了修复前拒绝、两个受影响日志的字节均保持不变、列出时的路径验证、重复 id 拒绝、cwd 冲突处理以及加载时的根目录验证。
|
||||
@@ -12,7 +12,7 @@ A capability seam ([interface / implementation / consumer](../architecture/2026-
|
||||
|
||||
The abstract service declared its operations beyond create/append: `load`, `list`, `has`, `delete`. Production consumers of `ctx.sessionPersistence` use only two: the agent-loop resume path calls `load()` ([packages/core/agent-loop/src/index.ts:176](../../../../packages/core/agent-loop/src/index.ts)), and the ACP bridge calls `list()` for `session/list` ([packages/ui/acp/src/index.ts:494](../../../../packages/ui/acp/src/index.ts)). Grepping every `sessionPersistence.*` / `persistence.*` use across `packages/*/src` and `examples/` finds no `has(` and no `delete(` on the service. The `.has(`/`.delete(` calls in `packages/ui/acp/src/index.ts` are on the in-memory `SessionStore` and a local `Set` of loading ids, not persistence. The only callers of `has`/`delete` were the contract suites and per-backend specs.
|
||||
|
||||
`has()` was not just unused — it was the most intricate branch in the shared coordinator: a tracked-vs-untracked dual-probe (`loadLive(id, cwd)` for a live-tracked session vs `loadStored(id)` for an untracked one) with a multi-line rationale. `delete()` dragged the `deleteStored` backend hook that every backend had to implement. This is the [drop-mutable-session-summary](2026-06-19-drop-mutable-session-summary.md) pattern: a contract test exercised both, but no shipping code asks "is this session persisted?" or removes one.
|
||||
`has()` was not just unused: it added a tracked-vs-untracked coordinator probe and a contract branch even though `loadStored(id)` already owns durable existence checks. `delete()` dragged the `deleteStored` backend hook that every backend had to implement. This is the [drop-mutable-session-summary](2026-06-19-drop-mutable-session-summary.md) pattern: a contract test exercised both, but no shipping code asks "is this session persisted?" or removes one.
|
||||
|
||||
## Decision
|
||||
|
||||
|
||||
Reference in New Issue
Block a user