feat(compact): prune tool results before summarization (round 1)

This commit is contained in:
Hypatia May
2026-07-16 18:02:15 +08:00
parent 231aeabe55
commit ce96104a77
42 changed files with 1093 additions and 52 deletions

View File

@@ -55,7 +55,7 @@ sequenceDiagram
The `assistant/message` edge records every successful provider call, including content-less and `max-tokens` finishes. Empty content stays out of derived history while the durable anchor retains usage and exact chunk provenance, including an explicit empty source set.
`dsh-compact-basic` uses `agent/post-step` for pressure after those durable facts and `agent/request-error` only for canonical context overflow. Recovery compacts between the closed failed step and a fresh retry step, and returns retry only when the surface replacement generation advances; otherwise the original request error remains authoritative.
`dsh-compact-basic` uses `agent/post-step` for pressure after those durable facts and `agent/request-error` only for canonical context overflow. Once either trigger qualifies, optional tool-result pruning runs before summary selection. Recovery works between the closed failed step and a fresh retry step, and returns retry only when pruning or summarization advances the surface replacement generation; otherwise the original request error remains authoritative.
SDK users that need replayable transcript data should consume `session/event`; `agent/*` is the live coordination surface for queue/status, prompt interception, request shaping, steering, continuation, and errors.

View File

@@ -32,6 +32,7 @@ A harness is one [Cordis](cordis-primer.md) context. Packages add services (`ctx
| `ctx.skills` | [`skill/`](../packages/skill/README.md) | skill provider registry and progressive disclosure |
| `ctx.web` | [`web/`](../packages/web/README.md) | search/fetch provider registries |
| `ctx.compact` | [`compact/`](../packages/compact/README.md) | session-log compaction |
| `ctx.toolResultPrune` | [`compact/tool-result-prune`](../packages/compact/tool-result-prune/README.md) | optional model-free tool-result pruning |
| `ctx.subagents` | [`subagent/`](../packages/subagent/README.md) | named delegation providers |
| `ctx.tasks` | [`tasks/`](../packages/tasks/README.md) | background task registry + generic `task_*` control tools |
| `ctx.workflows` | [`workflow/`](../packages/workflow/README.md) | script-driven multi-agent orchestration |
@@ -106,7 +107,7 @@ Each step renders one prompt assembly. Plugins contribute ordered sections, tool
Post-tool context follows all results, preserving call/result adjacency. Steering drains before `agent/post-step`, which observes durable output, results, context, and steering while the step signal remains open. Leftover steering becomes next-turn input. `agent/turn-stop` is terminal through close and flush: later steering is discarded, while ordinary queued prompts survive.
When loaded, `dsh-compact-basic` consumes that post-step checkpoint for `ctx.tokenMeter` pressure under the actual routed header. It also consumes canonical context overflow at `agent/request-error`, but authorizes retry only after a tool-balanced compaction advances `surface.replaceGeneration`. The same turn signal owns both summarization paths.
When loaded, `dsh-compact-basic` consumes that post-step checkpoint for `ctx.tokenMeter` pressure under the actual routed header. Once pressure or canonical context overflow qualifies, it runs optional `ctx.toolResultPrune` rewriting before summary selection and remeasures the replayed surface. Overflow recovery authorizes retry after either pruning or tool-balanced summary compaction advances `surface.replaceGeneration`. The same turn signal owns both paths.
### Failure Boundaries

View File

@@ -16,6 +16,8 @@ flowchart LR
pkg_compact_basic["compact-basic"]
pkg_token_meter["token-meter"]
svc_tokenMeter["ctx.tokenMeter<br/>Replay token measurement"]
pkg_tool_result_prune["tool-result-prune"]
svc_toolResultPrune["ctx.toolResultPrune<br/>Model-free tool-result pruning"]
pkg_session["session"]
svc_sessions["ctx.sessions<br/>In-memory session store"]
pkg_agent["agent"]
@@ -127,6 +129,7 @@ flowchart LR
pkg_system_prompt --> svc_systemPrompt
pkg_tasks --> svc_tasks
pkg_token_meter --> svc_tokenMeter
pkg_tool_result_prune --> svc_toolResultPrune
pkg_tools --> svc_tools
pkg_user_interaction --> svc_userInteraction
pkg_web --> svc_web
@@ -173,6 +176,7 @@ flowchart LR
svc_tasks --> pkg_tool_subagent
svc_tasks --> pkg_tool_tasks
svc_tokenMeter --> pkg_compact_basic
svc_toolResultPrune --> pkg_compact_basic
svc_tools --> pkg_acp
svc_tools --> pkg_agent_loop
svc_tools --> pkg_tool_ask_user
@@ -195,6 +199,7 @@ flowchart LR
| --- | --- | --- | --- | --- | --- | --- |
| `ctx.llm` | `seam` | [`llm`](../packages/llm/llm) | [`llm-deepseek`](../packages/llm/llm-deepseek), [`llm-pi-ai`](../packages/llm/llm-pi-ai), [`llm-replay`](../packages/support/llm-replay) | [`agent-loop`](../packages/core/agent-loop), [`compact-basic`](../packages/compact/compact-basic) | - | Adapters register provider implementations; the loop and compaction call the provider-neutral stream service. |
| `ctx.tokenMeter` | `core` | [`token-meter`](../packages/llm/token-meter) | - | [`compact-basic`](../packages/compact/compact-basic) | - | Owns isolated per-session replay folds; pressure consumers share immutable revisioned measurements. |
| `ctx.toolResultPrune` | `core` | [`tool-result-prune`](../packages/compact/tool-result-prune) | - | [`compact-basic`](../packages/compact/compact-basic) | - | Rewrites oversized current tool results through replayable single-node surface replacements before summary compaction. |
| `ctx.sessions` | `core` | [`session`](../packages/core/session) | - | [`agent-loop`](../packages/core/agent-loop), [`agent`](../packages/core/agent), [`session-persistence`](../packages/session-persistence/session-persistence), [`session-query`](../packages/session-query/session-query), [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`invariants`](../packages/support/invariants) | - | Owns append-only Session instances and emits the durable session event feed. |
| `ctx.sessionPersistence` | `seam` | [`session-persistence`](../packages/session-persistence/session-persistence) | [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl), [`session-persistence-sqlite`](../packages/session-persistence/session-persistence-sqlite) | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/ui/acp), [`session-query`](../packages/session-query/session-query) | - | Backends persist the same SessionEvent vocabulary; apps choose a backend at composition time. |
| `ctx.sessionQuery` | `seam` | [`session-query`](../packages/session-query/session-query) | - | - | - | Resolves live and optional persisted logs into one logical corpus for exact reads. |

View File

@@ -930,6 +930,22 @@ export interface Config {
Source: [`packages/fs/tool-fs/src/index.ts:22`](../packages/fs/tool-fs/src/index.ts)
## `@deepseek-ai/dsh-tool-result-prune`
```ts config-catalog
/** Character-budget policy for deterministic tool-result pruning. */
export interface ToolResultPruneConfig {
/** Prune when total text exceeds this many Unicode code points. Defaults to `8192`. */
thresholdChars?: number
/** Maximum leading Unicode code points retained. Defaults to `4096`. */
headChars?: number
/** Maximum trailing Unicode code points retained. Defaults to `1024`. */
tailChars?: number
}
```
Source: [`packages/compact/tool-result-prune/src/types.ts:4`](../packages/compact/tool-result-prune/src/types.ts)
## `@deepseek-ai/dsh-tool-skill`
Requires: `tools` · `skills`

View File

@@ -272,6 +272,20 @@ Types: [Message](../core-data-structures/core.md)
Source: [`packages/llm/token-meter/src/index.ts:106`](../../packages/llm/token-meter/src/index.ts)
## `ctx.toolResultPrune` — `ToolResultPruneService`
Deterministic head/middle/tail pruning for current tool-result surface nodes.
```ts cordis-catalog
measureContent(blocks: readonly ContentBlock[]): number
pruneContent(blocks: readonly ContentBlock[]): ContentBlock[] | null
pruneSession(session: Session): PruneResult
```
Types: [ContentBlock](../core-data-structures/core.md)
Source: [`packages/compact/tool-result-prune/src/index.ts:39`](../../packages/compact/tool-result-prune/src/index.ts)
## `ctx.tools` — `ToolRegistry`
Tool registry and execution pipeline. Scoped registrations shadow globals; one visibility resolver feeds presentation, lookup, and dispatch.

View File

@@ -58,6 +58,6 @@ export type CompactionTrigger = 'pressure' | 'context-overflow'
`CompactService` exposes `compactIfNeeded(agent, trigger, signal)` for automatic `pressure` or `context-overflow` policy, returning `null` when no safe work exists, and `compactRegion(...)` for an explicit inclusive surface range. Implementations must forward the supplied signal to summarization. The seam owns no pricing API: the singleton [`ctx.tokenMeter`](token-meter.md) directly owns estimation and replay, while `dsh-compact-basic` owns retention, event sequencing, routed summarization calls, and their configuration.
Pressure compaction runs at serial `agent/post-step`, after successful assistant output, tool results, buffered context, and steering are durable but before `step/end`. Failed-request recovery runs through `agent/request-error` after the failed step closes, and authorizes a fresh numbered-step retry only when the surface replacement generation advances. Region boundaries preserve tool-call/result pairing but do not preserve whole turns, allowing early closed steps of one oversized turn to compact. `dsh-compact-basic` owns thresholds, retained-tail policy, overflow caps, and failure handling.
Pressure compaction runs at serial `agent/post-step`, after successful assistant output, tool results, buffered context, and steering are durable but before `step/end`. Once pressure or canonical overflow qualifies, compact-basic invokes optional [`ctx.toolResultPrune`](../../packages/compact/tool-result-prune/README.md) before range selection, remeasures through `ctx.tokenMeter`, and can advance the surface without a summary. Failed-request recovery runs through `agent/request-error` after the failed step closes and authorizes a fresh numbered-step retry only when the surface replacement generation advances. Region boundaries preserve tool-call/result pairing but not whole turns, allowing early closed steps of one oversized turn to compact. `dsh-compact-basic` owns thresholds, retained-tail policy, overflow caps, and failure handling.
The seam exports `toolPairingBalancedBefore(session, node)` and `toolPairingBalancedAfter(session, node)` for those edge checks. Both validate current surface membership, reject stale or missing seqs and orphan results, and ignore a caller-retained `node.next`; the [package contract](../../packages/compact/compact/README.md#tool-pairing-boundaries) owns their cache semantics.

View File

@@ -45,6 +45,7 @@ flowchart TD
subgraph group_compact["packages/compact"]
pkg_compact["compact"]
pkg_compact_basic["compact-basic"]
pkg_tool_result_prune["tool-result-prune"]
end
subgraph group_subagent["packages/subagent"]
pkg_subagent["subagent"]
@@ -168,6 +169,8 @@ flowchart TD
pkg_skill_local --> pkg_skill
pkg_compact --> pkg_llm
pkg_compact --> pkg_session
pkg_tool_result_prune --> pkg_llm
pkg_tool_result_prune --> pkg_session
pkg_web_fetch_local --> pkg_timeout
pkg_web_fetch_local --> pkg_web
pkg_web_search_deepseek --> pkg_web
@@ -185,6 +188,7 @@ flowchart TD
pkg_compact_basic --> pkg_llm
pkg_compact_basic --> pkg_session
pkg_compact_basic --> pkg_token_meter
pkg_compact_basic --> pkg_tool_result_prune
pkg_hook_protocol --> pkg_bash
pkg_hook_protocol --> pkg_session
pkg_session_persistence_jsonl --> pkg_session
@@ -407,6 +411,7 @@ flowchart TD
| [`fs-policy`](../packages/fs/fs-policy) | `fs` | [`fs`](../packages/fs/fs) |
| [`skill-local`](../packages/skill/skill-local) | `skill` | [`fs`](../packages/fs/fs), [`skill`](../packages/skill/skill) |
| [`compact`](../packages/compact/compact) | `compact` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
| [`tool-result-prune`](../packages/compact/tool-result-prune) | `compact` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
| [`web-fetch-local`](../packages/web/web-fetch-local) | `web` | [`timeout`](../packages/util/timeout), [`web`](../packages/web/web) |
| [`web-search-deepseek`](../packages/web/web-search-deepseek) | `web` | [`web`](../packages/web/web) |
| [`web-search-exa`](../packages/web/web-search-exa) | `web` | [`web`](../packages/web/web) |
@@ -415,7 +420,7 @@ flowchart TD
| [`llm-replay`](../packages/support/llm-replay) | `support` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
| [`sandbox-local`](../packages/sandbox/sandbox-local) | `sandbox` | [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox) |
| [`bash-local`](../packages/bash/bash-local) | `bash` | [`bash`](../packages/bash/bash), [`timeout`](../packages/util/timeout) |
| [`compact-basic`](../packages/compact/compact-basic) | `compact` | [`agent`](../packages/core/agent), [`compact`](../packages/compact/compact), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`token-meter`](../packages/llm/token-meter) |
| [`compact-basic`](../packages/compact/compact-basic) | `compact` | [`agent`](../packages/core/agent), [`compact`](../packages/compact/compact), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`token-meter`](../packages/llm/token-meter), [`tool-result-prune`](../packages/compact/tool-result-prune) |
| [`hook-protocol`](../packages/hooks/hook-protocol) | `hooks` | [`bash`](../packages/bash/bash), [`session`](../packages/core/session) |
| [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl) | `session-persistence` | [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence) |
| [`session-persistence-sqlite`](../packages/session-persistence/session-persistence-sqlite) | `session-persistence` | [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence) |

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md: 7d68bc32d3860bf5edd94c4eda76922c91ae6af2
2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: 2315bd4d9ca9b93eb9a8d4850f917e6aa1bc6476
2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md: 99dc7625b8e185464d7a8a1ea8eda5baf0674df7
2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: 4f84d4435341005c058e32582e6d26b2a9f29bc1

View File

@@ -32,9 +32,9 @@ If cancellation lands after assistant tool calls are durable but before all call
`CompactService.compactIfNeeded(agent, trigger, signal)` accepts `trigger: 'pressure' | 'context-overflow'`. The interface gains no estimation methods or token types; `ctx.tokenMeter` remains the reusable accounting owner.
For `pressure`, compact-basic applies the service-wide threshold and retained-tail policy to one unified `ctx.tokenMeter.measure()` result. The same singleton meter owns range pricing, provenance, shadowed token counts, and non-shrinking-summary rejection. The common defaults remain threshold ratio `0.8`, retained history `floor(contextWindow × 0.16)`, summarization model `''`, `maxTokens: 8192`, `compactionRetries: 1`, and `auto: true`.
For `pressure`, compact-basic applies the service-wide threshold and retained-tail policy to one unified `ctx.tokenMeter.measure()` result. Below pressure it returns without pruning. Once pressure qualifies, optional `ctx.toolResultPrune` rewrites oversized current results and compact-basic remeasures through the same meter; safe pressure skips the model call, while remaining pressure selects and summarizes from the pruned surface. The same singleton meter owns range pricing, provenance, shadowed token counts, and non-shrinking-summary rejection. The common defaults remain threshold ratio `0.8`, retained history `floor(contextWindow × 0.16)`, summarization model `''`, `maxTokens: 8192`, `compactionRetries: 1`, and `auto: true`.
For canonical overflow, compact-basic bypasses scalar pressure and the normal retained-token budget. It chooses the maximal tool-balanced head range while leaving the newest indivisible unit, then attempts exactly one shrinking compaction under the same signal. The automatic listener snapshots `session.surface.replaceGeneration` and returns `{ action: 'retry' }` only when compaction succeeds and the generation increases. A backend returning a result without replacement cannot authorize retry.
For canonical overflow, compact-basic bypasses scalar pressure and the normal retained-token budget. It prunes first, then chooses the maximal tool-balanced head range while leaving the newest indivisible unit and attempts one shrinking summary compaction under the same signal when a range exists. The automatic listener snapshots `session.surface.replaceGeneration` and returns `{ action: 'retry' }` whenever pruning or summarization increases it. A backend returning a result without replacement cannot authorize retry, while pruning-only progress can authorize a retry without a `CompactionResult`.
`maxOverflowRetries` is optional and defaults to `1`; `0` disables overflow recovery without disabling pressure. `auto: false` registers neither automatic listener. Noncanonical errors, exhausted attempts, an already-aborted signal, a missing routed model, no safe range, no generation change, and recovery throws all delegate to the next listener. With no later recovery, the loop reports the original provider error object and code. Cancellation or disposal remains authoritative even if recovery work completes concurrently.
@@ -44,7 +44,7 @@ The default summarizer still resolves explicit configuration, then the latest lo
Lifecycle tests pin post-step ordering after durable tool/context/steering work, content-less and max-token successes, final-adapter dispatch/iterator/in-band boundaries, retry numbering, attempt reset, cancellation, disposal, synthetic tool results, and original error identity.
Compact tests pin low-friction service-wide defaults, actual routed-model selection, unlisted-model measurement, unified pressure-and-retention decisions, below-threshold forced overflow, newest tool-pair retention, non-shrinking rejection, generation proof, caps, disabled listeners, single downstream delegation, and auxiliary summary routing provenance. Real-loop composition covers both thrown and in-band overflow: the failed step closes, compaction lands between attempts, and the next numbered request is reconstructed from the replacement surface.
Compact tests pin low-friction service-wide defaults, actual routed-model selection, unlisted-model measurement, unified pressure-and-retention decisions, pressure-gated pruning, pruning-only relief, summarization from pruned input, optional-plugin fallback, pruning-only and summarized overflow recovery, newest tool-pair retention, non-shrinking rejection, generation proof, caps, disabled listeners, single downstream delegation, and auxiliary summary routing provenance. Real-loop composition covers both thrown and in-band overflow: the failed step closes, compaction lands between attempts, and the next numbered request is reconstructed from the replacement surface.
## Alternatives considered
@@ -56,7 +56,7 @@ Compact tests pin low-friction service-wide defaults, actual routed-model select
## Consequences
Pressure now describes the actual completed routed request, including durable tool results and request-only prefix fields, rather than a provisional next-call guess. Canonical overflow supplies the backstop when no successful usage anchor exists. Recovery is bounded, cancellation-owned, and monotonic: it retries only after a visible surface generation change.
Pressure describes the actual completed routed request, including durable tool results and request-only prefix fields, rather than a provisional next-call guess. Optional model-free pruning removes predictable tool-output bulk before summary selection and can independently create retry-worthy progress. Canonical overflow supplies the backstop when no successful usage anchor exists. Recovery is bounded, cancellation-owned, and monotonic: it retries only after a visible surface generation change.
The cost is one additional serial checkpoint on successful steps and adapter-maintained overflow classification. Provider wording and heuristic character density remain maintenance risks. Surface compaction still cannot repair an envelope that alone exceeds the window or split one indivisible oversized message/tool unit.

View File

@@ -32,9 +32,9 @@ Status: implemented
`CompactService.compactIfNeeded(agent, trigger, signal)` 接收 `trigger: 'pressure' | 'context-overflow'`。接口不增加估算方法或 token 类型;`ctx.tokenMeter` 继续作为可复用的核算所有者。
对于 `pressure`compact-basic 把服务级阈值与保留尾部策略应用到一次统一的 `ctx.tokenMeter.measure()` 结果。范围定价、来源、被遮蔽 token 数与非缩小摘要拒绝也由同一个单例 meter 完成。通用默认值保持为阈值比例 `0.8`、保留历史 `floor(contextWindow × 0.16)`、摘要模型 `''``maxTokens: 8192``compactionRetries: 1``auto: true`
对于 `pressure`compact-basic 把服务级阈值与保留尾部策略应用到一次统一的 `ctx.tokenMeter.measure()` 结果。低于压力时直接返回,不执行剪枝。压力达到条件后,可选的 `ctx.toolResultPrune` 会改写当前表层中过大的工具结果compact-basic 再通过同一个 meter 重新计量;若压力恢复安全则跳过模型调用,否则从已剪枝表层选择范围并生成摘要。范围定价、来源、被遮蔽 token 数与非缩小摘要拒绝也由同一个单例 meter 完成。通用默认值保持为阈值比例 `0.8`、保留历史 `floor(contextWindow × 0.16)`、摘要模型 `''``maxTokens: 8192``compactionRetries: 1``auto: true`
对于规范化溢出compact-basic 绕过标量压力与普通保留 token 预算。它在保留最新不可分割单元的同时选择最大的工具配对平衡头部范围,并在同一 signal 下尝试一次缩小压缩。自动监听器先记录 `session.surface.replaceGeneration`只有压缩成功且 generation 增加时返回 `{ action: 'retry' }`。后端若只返回结果但没有替换表层,不能授权重试。
对于规范化溢出compact-basic 绕过标量压力与普通保留 token 预算。它先执行剪枝,再在保留最新不可分割单元的同时选择最大的工具配对平衡头部范围;存在范围时,才在同一 signal 下尝试一次缩小摘要压缩。自动监听器先记录 `session.surface.replaceGeneration`剪枝或摘要让 generation 增加时返回 `{ action: 'retry' }`。后端若只返回结果但没有替换表层,不能授权重试;只有剪枝取得进展时,即使没有 `CompactionResult` 也可以授权重试
`maxOverflowRetries` 可选且默认为 `1``0` 只禁用溢出恢复,不会禁用压力检查。`auto: false` 不注册任何自动监听器。非规范化错误、尝试耗尽、已经中止的 signal、缺失路由模型、没有安全范围、generation 未变化,以及恢复抛错都会委托给下一个监听器。若没有后续恢复,循环报告原始提供方错误对象与代码。即使恢复工作并发完成,取消或销毁仍具有最终优先级。
@@ -44,7 +44,7 @@ Status: implemented
生命周期测试固定 post-step 位于持久工具、上下文与 steering 工作之后,覆盖无内容与达到 token 上限的成功、最终适配器分发/迭代器/带内边界、重试编号、尝试重置、取消、销毁、合成工具结果与原始错误身份。
压缩测试固定低摩擦服务级默认值、实际路由模型选择、未列出模型计量、统一压力与保留决策、低于阈值的强制溢出、最新工具配对保留、非缩小拒绝、generation 证明、上限、禁用监听器、单次下游委托与辅助摘要路由来源。真实循环组合同时覆盖抛出式和带内溢出:失败 step 关闭,压缩落在两次尝试之间,下一个编号请求从替换表层重建。
压缩测试固定低摩擦服务级默认值、实际路由模型选择、未列出模型计量、统一压力与保留决策、压力门控剪枝、剪枝独立解除压力、从已剪枝输入生成摘要、可选插件回退、仅剪枝与剪枝后摘要两类溢出恢复、最新工具配对保留、非缩小拒绝、generation 证明、上限、禁用监听器、单次下游委托与辅助摘要路由来源。真实循环组合同时覆盖抛出式和带内溢出:失败 step 关闭,压缩落在两次尝试之间,下一个编号请求从替换表层重建。
## 考虑过的替代方案
@@ -56,7 +56,7 @@ Status: implemented
## 后果
压力现在描述实际完成的路由请求,包括持久工具结果与仅请求前缀字段,而不是对下一次调用的临时猜测。当成功 usage 锚点不存在时,规范化溢出提供兜底路径。恢复有上限、受取消所有,并保持单调:只有模型可见的表层 generation 变化后才重试。
压力描述实际完成的路由请求,包括持久工具结果与仅请求前缀字段,而不是对下一次调用的临时猜测。可选的无模型剪枝会在选择摘要前移除可预测的工具输出体积,也能独立产生足以重试的进展。当成功 usage 锚点不存在时,规范化溢出提供兜底路径。恢复有上限、受取消所有,并保持单调:只有模型可见的表层 generation 变化后才重试。
代价是成功 step 增加一个串行检查点,并需要适配器持续维护溢出分类。提供方措辞与启发式字符密度仍是维护风险。表层压缩依然无法修复仅信封本身就超出窗口的情况,也不能拆分单个不可分割的超大消息或工具单元。

View File

@@ -18,7 +18,8 @@ Per the [capability-seams RFC](../../implemented/architecture/2026-06-13-capabil
1. **Interface**`@deepseek-ai/dsh-compact`: an abstract `CompactService` owning the `ctx.compact` key, the `CompactionResult` vocabulary, and the `compact/*` session events. It declares `compactIfNeeded()` and `compactRegion()` as **abstract** — the contract states *what* compaction does, not *how*.
2. **Implementation**`@deepseek-ai/dsh-compact-basic`: a concrete `BasicCompactService` that consumes `ctx.tokenMeter` and owns the tail→head retention walk, summarization via `ctx.llm.stream()`, the surface replacement, the lock, post-step pressure, and canonical context-overflow recovery. `summarize()` is its sole subclass hook; pricing and replay stay with the meter.
3. **Consumer** — deferred. A `/compact` tool and slash command will `inject: ['compact']` and call the contract; they are intentionally out of scope here so the seam settles first.
3. **Model-free companion**`@deepseek-ai/dsh-tool-result-prune`: a concrete optional service that rewrites oversized current `tool/result` nodes before the backend selects a summary range. It is not a second compaction implementation and does not implement `CompactService`.
4. **Consumer** — deferred. A `/compact` tool and slash command will `inject: ['compact']` and call the contract; they are intentionally out of scope here so the seam settles first.
### The contract depends on `dsh-session` and `dsh-llm` — a deliberate deviation
@@ -34,9 +35,9 @@ An earlier draft put the full algorithm (the retention walk, token-summing, text
### Automatic pressure runs after successful durable step work
The original pre-step placement used a provisional envelope and could not see final `agent/request` routing, tools, provider output, tool results, buffered context, or steering. The corrected lifecycle fires serial `agent/post-step(agent, turn, step, signal)` after those successful facts are durable and before `step/end`. `dsh-compact-basic` measures the canonical logged request through `ctx.tokenMeter`, so the next request sees any replacement without a speculative envelope override.
The original pre-step placement used a provisional envelope and could not see final `agent/request` routing, tools, provider output, tool results, buffered context, or steering. The corrected lifecycle fires serial `agent/post-step(agent, turn, step, signal)` after those successful facts are durable and before `step/end`. `dsh-compact-basic` measures the canonical logged request through `ctx.tokenMeter`, so the next request sees any replacement without a speculative envelope override. Once pressure qualifies, it invokes optional `ctx.toolResultPrune`, remeasures the durable surface, and summarizes only if pruning did not restore safe pressure.
Canonical provider context overflow takes a separate path. The failed step closes, `agent/request-error` receives the original request error and consecutive retry count, and compact-basic forces one useful balanced reduction. It returns retry only if `session.surface.replaceGeneration` increases; the loop then opens a new numbered step and reconstructs its request from the durable log. No range, no replacement, recovery failure, cancellation, an exhausted cap, or an unrelated error preserves the original provider failure. The complete lifecycle decision is in the [after-call recovery RFC](../../implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md).
Canonical provider context overflow takes a separate path. The failed step closes, `agent/request-error` receives the original request error and consecutive retry count, and compact-basic prunes before forcing one useful balanced reduction. It returns retry only if `session.surface.replaceGeneration` increases, including pruning-only progress when no summary range exists; the loop then opens a new numbered step and reconstructs its request from the durable log. No replacement, recovery failure, cancellation, an exhausted cap, or an unrelated error preserves the original provider failure. The complete lifecycle decision is in the [after-call recovery RFC](../../implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md).
```
assistant/message → tool/result/context/steering
@@ -110,16 +111,16 @@ Two failure paths, both documented:
## Consequences
- **Packages**: `packages/compact/compact` supplies the interface and `compact-basic` supplies the backend. `packages/llm/token-meter` owns replay-aware measurement independently. The consumer tier is deferred.
- **Packages**: `packages/compact/compact` supplies the interface, `compact-basic` supplies the backend, and `tool-result-prune` supplies optional deterministic rewriting. `packages/llm/token-meter` owns replay-aware measurement independently. The consumer tier is deferred.
- **Automatic seams**: `agent/post-step` (`@mode serial`) handles successful-call pressure and `agent/request-error` (`@mode waterfall`) handles final request failures after the failed step closes. Generic `agent/pre-step` remains a four-argument checkpoint with no compaction-only prompt/prefix payload.
- **`SessionEventMap`** gains `compact/start` / `compact/summary` / `compact/end` by declaration merging (merge-extensible); `SurfaceEventType` is **not** touched. These are session events, not cordis `Events`, so the event-taxonomy gate needs no entry.
- **`dsh-compact`** owns `toolPairingBalancedBefore(session, node)` and `toolPairingBalancedAfter(session, node)`, the cached surface-edge checks that `compactRegion` and `compactIfNeeded` use to avoid splitting a tool-call/result pair. The cache validates current membership by seq and answers both edges from one per-cut balance sequence instead of trusting a caller-retained `node.next`; stale or missing seqs and orphan results reject. `dsh-session` continues to own the surface `replace` operation, positional nodes, and rewrite generation.
- **`dsh-invariants`** drops its `surface replace: start must be <= end` assertion: a head-anchored compaction lands a high-seq replacement node at an older range's *position*, so `start > end` numerically is normal and valid (the range is positional, validated by the surface's `indexOf` checks that remain). The turn-enclosure invariant is reused unchanged.
- **Wiring**: `examples/coding-agent/cordis.yml` loads zero-config `dsh-token-meter` before `dsh-compact-basic`; the service-wide window and compact defaults make the pair usable without repeated numeric policy.
- **`dsh-invariants`** treats fresh appended tool results as executions that require an open step and pending call, while provenance-backed replacements are turn-enclosed surface rewrites. Positional replacement and complete-source checks validate the rewritten node.
- **Wiring**: `examples/coding-agent/cordis.yml` loads zero-config `dsh-token-meter`, `dsh-tool-result-prune`, then `dsh-compact-basic`; service-wide defaults make the composition usable without repeated numeric policy.
## Testing
- **Unit:** Real Loader and invariant plugins cover whole-unit retention, convergence failure, both `compact/end` outcomes, head anchoring, open-tail refusal, inert crash orphans, forced below-threshold overflow, generation proof, caps, and original-error preservation.
- **Unit:** Real Loader and invariant plugins cover whole-unit retention, pruning configuration and replay, rich-block ordering, metadata preservation, convergence, both `compact/end` outcomes, open-tail refusal, pruning-only and summarized overflow recovery, generation proof, caps, and original-error preservation.
- **Loop:** Tests pin post-step after durable tool results and before `step/end`, actual `agent/request` routing, closed failed steps, fresh retry numbering, and complete thrown/in-band overflow → compaction → reconstructed retry composition.
- **With-key e2e:** A real model and bash session with lowered limits triggers compaction, records a complete `compact/start…end` pair, shrinks the surface, and finishes the task.
- **Snapshot gap:** Runaway-turn compaction cannot yet replay because the summarization call records no `assistant/chunk` events or `sessionId`; interleaved summarization-call replay remains follow-up work.

View File

@@ -5,9 +5,9 @@
Every model-facing tool a shipped plugin contributes to `ctx.tools`: the `name`, `description`, and JSON-Schema `parameters` the model receives via the system-prompt assembly. It complements the cordis [events](cordis-catalog/events.md) & [services](cordis-catalog/services.md) catalogs (the wiring a plugin listens to and calls) and [core-data-structures/](core-data-structures/core.md) (the types those signatures move) — this page is the *tools* the agent is offered.
This file is GENERATED and verified fresh by `pnpm run verify-tool-catalog` (part of `doc-sync`) — do not edit it by hand. Unlike the cordis catalog (a pure source-AST pass), this generator BOOTS each tool plugin on a real context and reads `ctx.tools.schemas()`, because a tool schema is not statically knowable (runtime-spread enums, concatenated descriptions, config-driven names, raw-JSON-Schema MCP tools). A completeness guard globs `packages/*/tool-*` and fails if any package is missing from the generator's boot manifest, so a new tool cannot be silently undocumented. See [the tool-schema-catalog RFC](rfc/implemented/process/2026-07-02-tool-schema-catalog.md).
This file is GENERATED and verified fresh by `pnpm run verify-tool-catalog` (part of `doc-sync`) — do not edit it by hand. Unlike the cordis catalog (a pure source-AST pass), this generator BOOTS each tool plugin on a real context and reads `ctx.tools.schemas()`, because a tool schema is not statically knowable (runtime-spread enums, concatenated descriptions, config-driven names, raw-JSON-Schema MCP tools). A completeness guard globs `packages/*/tool-*` and fails if any model-facing package is missing from the generator's boot manifest; service-only packages that share the prefix are explicitly excluded. See [the tool-schema-catalog RFC](rfc/implemented/process/2026-07-02-tool-schema-catalog.md).
Scope: shipped product tools under `packages/*/tool-*`, each booted with its DEFAULT config. The registered tool NAME can be a load-time config (e.g. `tool-subagent`'s `toolName`), so a deployment may surface a package under a different or additional name — a per-package note records those shipped aliases where they exist. The `examples/` demo tools (e.g. `echo`) are excluded, matching the cordis catalog's packages-only scope.
Scope: shipped model-facing product tools under `packages/*/tool-*`, each booted with its DEFAULT config. Runtime service packages such as `tool-result-prune` do not register `ctx.tools` schemas and are explicitly excluded. The registered tool NAME can be a load-time config (e.g. `tool-subagent`'s `toolName`), so a deployment may surface a package under a different or additional name — a per-package note records those shipped aliases where they exist. The `examples/` demo tools (e.g. `echo`) are excluded, matching the cordis catalog's packages-only scope.
## Tool Package Map

View File

@@ -56,6 +56,7 @@ This example is a thin leaf `cordis.yml`: it picks the swappable backends, loads
| `llm-deepseek` | real `LlmAdapter` via config (`!!js process.env.…` secrets); swap one line to `@deepseek-ai/dsh-llm-pi-ai` for the library-backed twin |
| `bash` (`dsh-bash-local`) | the executor implementation — the swappable half of the bash seam. The model-facing `bash` schema (`tool-bash`) and generic `task_*` controls (`tool-tasks`) come from `dsh-agent-spine-demo`, so only the executor is a leaf choice |
| `stdio-agent` (`@deepseek-ai/dsh-stdio-demo`) | the app bundle: the agent-spine demo + console logger + JSONL persistence + readline UI + a pre-created `main` agent. Its config carries the model, system prompt, `persistenceRoot` (`./.sessions`), and `resumeSessionId` — so persistence and the agent are configured here, not wired as separate leaf plugins |
| `token-meter`, `tool-result-prune`, `compact-basic` | replay-aware pressure, model-free oversized tool-result pruning, and LLM summary compaction. Pruning runs only after a compaction trigger qualifies and can avoid the summarization call |
| `subagent`, `subagent-spawn`, `subagent-fork` | the subagent provider registry plus the two in-process backends: a fresh child and a child seeded with the parent's completed-turn prefix |
| `tool-subagent`, `tool-subagent-fork` | two model-facing `dsh-tool-subagent` loads, each bound to a different provider and exposed under a distinct tool name (`subagent`, `subagent_fork`) |
| `tool-todo` | the model-facing `todo_write` tool; writes the whole task list to the session log and renders as a checklist in stdio |
@@ -66,7 +67,7 @@ This example is a thin leaf `cordis.yml`: it picks the swappable backends, loads
- `tests/full-loop.e2e.ts` — the canary: real model runs `echo e2e-ok` through the real bash tool; asserts `tool/call`/`tool/result` session events and the final answer.
- `tests/coding-task.e2e.ts` — the swebench-style smoke: a temp dir holds `add.js` (with `a - b` where `a + b` belongs) and a failing `add.test.js`; the agent must fix the bug and verify. The test re-runs `node add.test.js` ITSELF and inspects the files — agent claims are not trusted.
- `tests/resume.e2e.ts` — durable continuity across processes: run 1 tells the real model a secret code and persists the turn to a temp JSONL root, then the whole context is disposed; run 2 is a fresh context over the same root that RESUMES the session id and asks the model to recall the code. The recall can only come from the rehydrated log.
- `tests/compaction.e2e.ts` — the compaction smoke: a real multi-step bash task runs with a deliberately tiny context window so the auto-compaction listener fires MID-SESSION. Verifies the WORLD — a `compact/start…end` pair landed in the real log, the surface shrank (a replace node shadowed older nodes), and the agent still produced a correct final answer after compaction.
- `tests/compaction.e2e.ts` — the compaction smoke: a real multi-step bash task runs with a deliberately tiny context window so automatic pruning or summary compaction fires mid-session. It verifies the world: a replayable surface replacement lands, summary brackets are complete when summarization is needed, the surface shrinks, and the agent still produces a correct final answer.
- `tests/todo-write.e2e.ts` — a real model drives the real `todo_write` tool and the test verifies the resulting `todo/write` session event.
These self-skip without `DEEPSEEK_API_KEY`. `tests/code-mode.e2e.ts` is the with-key Code Mode proof — a real model, a two-tool task, asserting the wire tool list was exactly `[run_code]`, the `tool/code-dispatch` events landed under the parent call, and the curated answer came back. The keyless boot smokes run in the default e2e gate: `tests/keyless-smoke.e2e.ts` (the full real tree, dummy key, no prompt → no model call) and `tests/code-mode-keyless-smoke.e2e.ts` (the same guard for the Code Mode overlay).

View File

@@ -3,7 +3,7 @@
# Coding Agent App Composition
The coding REPL demo adds the real DeepSeek adapter, filesystem tools, todo_write, compaction, and both subagent transports on top of the stdio app package.
The coding REPL demo adds the real DeepSeek adapter, filesystem tools, todo_write, tool-result pruning, compaction, and both subagent transports on top of the stdio app package.
```mermaid
flowchart LR
@@ -25,6 +25,8 @@ flowchart LR
bundle_agent_core --> spine_loop["ctx.agents + ctx.agentLoop"]
plugin_coding_token_meter["token-meter<br/>@deepseek-ai/dsh-token-meter"]
cfg --> plugin_coding_token_meter
plugin_coding_tool_result_prune["tool-result-prune<br/>@deepseek-ai/dsh-tool-result-prune"]
cfg --> plugin_coding_tool_result_prune
plugin_coding_compact_basic["compact-basic<br/>@deepseek-ai/dsh-compact-basic"]
cfg --> plugin_coding_compact_basic
plugin_coding_subagent["subagent<br/>@deepseek-ai/dsh-subagent"]
@@ -58,6 +60,7 @@ flowchart LR
| `bash` | `@deepseek-ai/dsh-bash-local` |
| `stdio-agent` | `@deepseek-ai/dsh-stdio-demo` |
| `token-meter` | `@deepseek-ai/dsh-token-meter` |
| `tool-result-prune` | `@deepseek-ai/dsh-tool-result-prune` |
| `compact-basic` | `@deepseek-ai/dsh-compact-basic` |
| `subagent` | `@deepseek-ai/dsh-subagent` |
| `subagent-spawn` | `@deepseek-ai/dsh-subagent-spawn` |

View File

@@ -50,6 +50,10 @@
- id: token-meter
name: '@deepseek-ai/dsh-token-meter'
# Prune oversized tool output without a model call before summary compaction.
- id: tool-result-prune
name: '@deepseek-ai/dsh-tool-result-prune'
# Summarize an older range after measured pressure or a canonical provider overflow.
# Service-wide policy provides pressure, retention, and one overflow-retry default.
- id: compact-basic

View File

@@ -12,6 +12,7 @@ import * as ToolTodo from '@deepseek-ai/dsh-tool-todo'
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
import type { TokenMeterConfig } from '@deepseek-ai/dsh-token-meter'
import ToolResultPruneService from '@deepseek-ai/dsh-tool-result-prune'
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
import { BasicCompactService } from '@deepseek-ai/dsh-compact-basic'
import type { BasicCompactConfig } from '@deepseek-ai/dsh-compact-basic'
@@ -68,6 +69,7 @@ export async function codingHarness(workdir: string, options: CodingHarnessOptio
// backend, with a lower context window so a short real session crosses the threshold.
if (options.compact !== undefined) {
await ctx.plugin(TokenMeterService, options.tokenMeter)
await ctx.plugin(ToolResultPruneService)
await ctx.plugin(BasicCompactService, options.compact)
}
// Durable JSONL persistence is opt-in: only the resume e2e needs it, and the

View File

@@ -1,11 +1,12 @@
# compact/ — compaction capability family
A three-package capability seam (see [capability seams](../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): an abstract compaction interface, a backend that summarizes, and the model-facing tool that consumes it. The interface and a first backend (`compact-basic/`) exist; the consumer tool is deferred. All **product** packages.
A compaction capability family (see [capability seams](../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): an abstract interface, a summarizing backend, a model-free tool-result pruning companion, and a deferred model-facing consumer. All **product** packages.
| Package | Role | ctx key |
|---|---|---|
| `compact/` | Abstract compaction seam (interface + `compact/*` events + `CompactionResult`) | `ctx.compact` |
| `compact-basic/` | A backend: `ctx.tokenMeter` pressure + token-budget retention + `llm.stream()` summarization | (registers `ctx.compact`) |
| `tool-result-prune/` | Optional model-free head/middle/tail rewriting before summary compaction | `ctx.toolResultPrune` |
| `tool-compact/` (deferred) | Model-facing `/compact` tool over `ctx.compact` | (registers on `ctx.tools`) |
The interface lives at `compact/compact/`, the backend at `compact/compact-basic/`. Unlike the bash seam, it depends on `dsh-session` and `dsh-llm` its verbs are defined over a `Session` and its output is the `ContentBlock` vocabulary, so the contract cannot be expressed without naming them. That deviation from the "interface depends only on cordis" guidance is intentional and recorded in the [compaction capability-seam RFC](../../docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md). Token measurement is a reusable LLM-family service rather than a `CompactService` method; a template- or model-backed compactor can replace `compact-basic` without changing the meter or callers.
The interface lives at `compact/compact/`, the backend at `compact/compact-basic/`, and deterministic pruning at `compact/tool-result-prune/`. Unlike the bash seam, the interface depends on `dsh-session` and `dsh-llm` because its verbs are defined over a `Session` and its output uses `ContentBlock`. That deviation is recorded in the [compaction capability-seam RFC](../../docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md). Token measurement remains a reusable LLM-family service; a template- or model-backed compactor can replace `compact-basic` without changing the meter, pruner, or callers.

View File

@@ -9,12 +9,13 @@ This is the implementation tier of the compaction capability — see the [interf
This backend owns the compaction policy:
- **Measurement** — the singleton `ctx.tokenMeter` prices the latest canonical logged envelope and current surface at one consumed-log revision. Post-step pressure therefore includes the actual system prompt, tools, prefix, routing, assistant completion, tool results, buffered context, and steering.
- **Model-free pruning** — after pressure or canonical overflow qualifies, the optional [`ctx.toolResultPrune`](../tool-result-prune/README.md) service rewrites oversized tool results before range selection. Compact-basic remeasures through `ctx.tokenMeter`, skips summarization when pressure becomes safe, and otherwise summarizes the pruned surface. Below-pressure post-step checks never prune.
- **Retention** — compact the oldest whole surface units while preserving a recent tail and balanced tool-call/result cuts through the [`dsh-compact` boundary helpers](../compact/README.md#tool-pairing-boundaries). Turn boundaries do not protect old steps inside a runaway turn. An open indivisible tail declines until it closes; a single unit larger than the budget remains out of scope.
- **Convergence** — retry head-checkpoint compaction up to `compactionRetries`; reject a summary that does not shrink its source, and throw if retries cannot return below threshold.
- **Summarization** — a direct `llm/stream` call uses the configured model and cap without running the loop-only `agent/request` seam. The input transcript preserves non-text blocks as tagged placeholders; only returned text enters the checkpoint, excluding reasoning and tool calls that would leak private reasoning or create an orphaned call.
- **Framing** — the replacement user message marks established checkpoint context with `<compacted-summary>` tags. The raw summary remains on the provenance event, and later automatic cycles merge the prior checkpoint.
- **Lifecycle** — `compactRegion()` requires its agent to own the exact target session and rejects mismatch before resolution or mutation; a valid call records its start, summary, replacement, and end. The serial `agent/post-step` listener checks pressure after successful output and tool work are durable but before `step/end`. Canonical provider overflow is handled through `agent/request-error` after the failed step closes.
- **Overflow recovery** — below-threshold overflow bypasses normal retention and attempts one maximal balanced head reduction while leaving the newest indivisible unit. Retry is authorized only when `surface.replaceGeneration` advances; no range, no replacement, recovery failure, an exhausted cap, cancellation, or an unknown/noncanonical error preserves the original provider failure.
- **Overflow recovery** — below-threshold overflow bypasses normal retention and first prunes, then attempts one maximal balanced head reduction while leaving the newest indivisible unit. Retry is authorized whenever `surface.replaceGeneration` advances, including pruning-only progress on an otherwise indivisible surface; no replacement, recovery failure, an exhausted cap, cancellation, or an unknown/noncanonical error preserves the original provider failure.
- **Failure handling** — an unmatched `compact/start` is an inert crash marker because no replacement landed. Operational post-step failures warn and continue; overflow-recovery failure preserves the original provider error.
`summarize()` is the sole subclass hook. A template- or remote-summarizer subclass can override it while pressure, retention, provenance, shrink validation, and shadowed-token accounting stay on `ctx.tokenMeter`. The hook returns the summary blocks together with the call envelope it used (`{ summary, model, maxTokens? }`), which is logged on `compact/summary`.
@@ -49,15 +50,15 @@ export function apply(ctx: Context): void {
}
```
Loading the plugin registers `ctx.compact`. With `auto: true` (the default) it compacts automatically under token pressure; a consumer (a future `/compact` tool) can also call `ctx.compact.compactIfNeeded(...)` or `ctx.compact.compactRegion(...)` directly.
Loading the plugin registers `ctx.compact`. Add [`dsh-tool-result-prune`](../tool-result-prune/README.md) as a sibling before this plugin to enable the optional model-free pass. With `auto: true` (the default) it compacts automatically under token pressure; a consumer (a future `/compact` tool) can also call `ctx.compact.compactIfNeeded(...)` or `ctx.compact.compactRegion(...)` directly.
## Model Experience
### Conversation history
**What the model sees**: After a successful step crosses the threshold, the next request receives the checkpoint preamble below, a blank line, `<compacted-summary>`, the data-dependent summary, and `</compacted-summary>`. Overflow recovery rebuilds the immediate retry from that replacement. This one checkpoint replaces the selected older range and is followed by the retained recent units.
**What the model sees**: After a successful step crosses the threshold, oversized tool results are first rewritten when the optional pruner is loaded. If summarization remains necessary, the next request receives the checkpoint preamble below, a blank line, `<compacted-summary>`, the data-dependent summary, and `</compacted-summary>`. Overflow recovery rebuilds the immediate retry from whatever replacement advanced the surface.
**Token effect**: The replacement reduces future input history rather than appending a second copy. The summary remains until a later compaction replaces it; one oversized indivisible unit can still exceed the budget.
**Token effect**: Model-free pruning can avoid the auxiliary call entirely; otherwise it reduces that call's transcript before the summary replaces an older range. A summary remains until a later compaction replaces it, while an indivisible non-tool unit can still exceed the budget.
#### Conversation checkpoint preamble

View File

@@ -27,8 +27,14 @@
"@deepseek-ai/dsh-llm": "^0.0.1",
"@deepseek-ai/dsh-session": "^0.0.1",
"@deepseek-ai/dsh-token-meter": "^0.0.1",
"@deepseek-ai/dsh-tool-result-prune": "^0.0.1",
"cordis": "^4.0.0-rc.7"
},
"peerDependenciesMeta": {
"@deepseek-ai/dsh-tool-result-prune": {
"optional": true
}
},
"dependencies": {
"schemastery": "^3.18.0"
},
@@ -43,6 +49,7 @@
"@deepseek-ai/dsh-session": "workspace:^",
"@deepseek-ai/dsh-system-prompt": "workspace:^",
"@deepseek-ai/dsh-token-meter": "workspace:^",
"@deepseek-ai/dsh-tool-result-prune": "workspace:^",
"@deepseek-ai/dsh-tools": "workspace:^",
"cordis": "^4.0.0-rc.7"
}

View File

@@ -12,6 +12,8 @@ import type { Session } from '@deepseek-ai/dsh-session'
import { CONTEXT_WINDOW_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import type { Agent } from '@deepseek-ai/dsh-agent'
// Type-only: makes the optional sibling service available to `ctx.get()`.
import type {} from '@deepseek-ai/dsh-tool-result-prune'
import { resolveConfig } from './config.ts'
import { compactSurfaceRegion, selectCompactableRange } from './region.ts'
import { summarizeWithLlm } from './summarizer.ts'
@@ -111,9 +113,9 @@ export class BasicCompactService extends CompactService {
return next()
}
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- signal can abort while compaction is awaited.
if (signal.aborted || result === null
if (signal.aborted
|| agent.session.surface.replaceGeneration <= generation) return next()
logResult(result, 'context overflow recovery')
if (result !== null) logResult(result, 'context overflow recovery')
return { action: 'retry' }
})
}
@@ -142,7 +144,7 @@ export class BasicCompactService extends CompactService {
* @param agent - agent whose latest durable routed request is measured.
* @param trigger - normal post-step pressure or context-overflow recovery.
* @param signal - live turn cancellation signal forwarded to summarization.
* @returns the latest compaction result, or `null` when no check/work applies.
* @returns the latest summary compaction result, or `null` when no summary ran.
*/
override async compactIfNeeded(
agent: Agent,
@@ -152,15 +154,25 @@ export class BasicCompactService extends CompactService {
const model = routedModel(agent.session)
if (model === undefined) return null
const meter = this.ctx.tokenMeter
const threshold = Math.floor(meter.contextWindow * this.config.thresholdRatio)
let measurement = meter.measure(agent.session)
if (trigger === 'pressure' && measurement.totalTokens < threshold) return null
// Pruning is optional so compact-basic remains independently composable.
// Once either trigger qualifies, land the model-free pass before choosing
// a summary range, then remeasure through the singleton replay fold.
const prune = this.ctx.get('toolResultPrune')
if (prune !== undefined) {
prune.pruneSession(agent.session)
measurement = meter.measure(agent.session)
}
if (trigger === 'context-overflow') {
const measurement = meter.measure(agent.session)
const range = selectCompactableRange(agent.session, measurement, 0)
if (range === null) return null
return this.compactRegion(agent.session, range.start, range.end, agent, signal)
}
const threshold = Math.floor(meter.contextWindow * this.config.thresholdRatio)
let measurement = meter.measure(agent.session)
if (measurement.totalTokens < threshold) return null
let result: CompactionResult | null = null

View File

@@ -9,6 +9,7 @@ import LlmService, { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, LlmAdapter } from '@d
import type { ContentBlock, GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
import { Session, SessionId } from '@deepseek-ai/dsh-session'
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
import ToolResultPruneService from '@deepseek-ai/dsh-tool-result-prune'
import type { Agent } from '@deepseek-ai/dsh-agent'
const SIGNAL = new AbortController().signal
@@ -94,6 +95,42 @@ function toolConversation(): Session {
return session
}
/** One closed routed tool step followed by an open turn for rewrite events. */
function oversizedToolResult(chars = 3_000, withCompactablePrompt = false): Session {
const session = new Session(SessionId(`oversized-tool-${chars}`))
const callId = CallId('oversized')
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
if (withCompactablePrompt) {
session.append('user/message', {
content: [{ type: 'text', text: 'older history '.repeat(200) }],
source: { kind: 'user' },
}, { surfaceOp: 'append' })
}
session.append('step/start', { turn: 1, step: 1 })
session.append('request/header', {
header: { config: { model: MODEL } },
reason: 'initial',
})
session.append('assistant/message', {
turn: 1,
step: 1,
content: [{ type: 'tool-call', id: callId, name: 'bash', arguments: '{}' }],
}, { surfaceOp: 'append' })
session.append('tool/call', { turn: 1, step: 1, callId, name: 'bash', arguments: '{}' })
session.append('tool/result', {
turn: 1,
step: 1,
callId,
content: [{ type: 'text', text: 'X'.repeat(chars) }],
isError: false,
meta: { presentation: 'preserved' },
}, { surfaceOp: 'append' })
session.append('step/end', { turn: 1, step: 1 })
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
return session
}
class TestCompactService extends BasicCompactService {
summary: ContentBlock[] = [{ type: 'text', text: 'small checkpoint' }]
summaryModel = 'summary-model'
@@ -401,6 +438,78 @@ describe('pressure measurement and retention', () => {
})
})
describe('optional model-free tool-result pruning', () => {
const pruneConfig = { thresholdChars: 100, headChars: 20, tailChars: 10 }
it('does not prune a below-pressure session opportunistically', async () => {
const ctx = createContext(10_000)
const prune = new ToolResultPruneService(ctx, pruneConfig)
const compact = new TestCompactService(ctx, {
auto: false,
thresholdRatio: 0.8,
retainTokens: 100,
})
const session = oversizedToolResult()
const pruneSession = vi.spyOn(prune, 'pruneSession')
expect(await compactIfNeeded(compact, session)).toBeNull()
expect(pruneSession).not.toHaveBeenCalled()
expect(compact.calls).toHaveLength(0)
expect(session.surface.replaceGeneration).toBe(0)
})
it('skips LLM summarization when pruning alone clears pressure', async () => {
const ctx = createContext(1_000)
void new ToolResultPruneService(ctx, pruneConfig)
const compact = new TestCompactService(ctx, {
auto: false,
thresholdRatio: 0.5,
retainTokens: 50,
})
const session = oversizedToolResult()
expect(ctx.tokenMeter.measure(session).totalTokens).toBeGreaterThanOrEqual(500)
expect(await compactIfNeeded(compact, session)).toBeNull()
expect(ctx.tokenMeter.measure(session).totalTokens).toBeLessThan(500)
expect(compact.calls).toHaveLength(0)
expect(session.surface.replaceGeneration).toBe(1)
})
it('summarizes the pruned surface when pruning is insufficient', async () => {
const ctx = createContext(2_000)
void new ToolResultPruneService(ctx, pruneConfig)
const compact = new TestCompactService(ctx, {
auto: false,
thresholdRatio: 0.5,
retainTokens: 50,
})
const session = toolConversation()
expect(await compactIfNeeded(compact, session)).not.toBeNull()
expect(compact.calls).toHaveLength(1)
expect(compact.calls[0]!.text).toContain('tool result middle pruned')
expect(compact.calls[0]!.text).not.toContain('result 1 '.repeat(300))
})
it('retains the original compact-basic behavior without the optional plugin', async () => {
const ctx = createContext(2_000)
const compact = new TestCompactService(ctx, {
auto: false,
thresholdRatio: 0.5,
retainTokens: 50,
})
const session = oversizedToolResult(3_000, true)
expect(await compactIfNeeded(compact, session)).not.toBeNull()
expect(compact.calls).toHaveLength(1)
const original = session.events.find(event => event.type === 'tool/result')
expect(original?.type === 'tool/result' && original.data.content[0])
.toEqual({ type: 'text', text: 'X'.repeat(3_000) })
expect(session.events.filter(event =>
event.type === 'tool/result' && event.surfaceOp !== 'append')).toHaveLength(0)
})
})
describe('compaction region transaction', () => {
it('rejects an agent that does not own the exact target session before mutation', async () => {
const compact = service()
@@ -875,6 +984,44 @@ describe('automatic listener and loader composition', () => {
expect(session.surface.nodes.some(node => node.seq === retainedSeq)).toBe(true)
})
it('authorizes overflow retry when pruning alone advances an indivisible surface', async () => {
const ctx = createContext(10_000)
void new ToolResultPruneService(ctx, {
thresholdChars: 100,
headChars: 20,
tailChars: 10,
})
const compact = new TestCompactService(ctx, {
thresholdRatio: 1,
retainTokens: 900,
})
const session = oversizedToolResult()
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'retry' })
expect(session.surface.replaceGeneration).toBe(1)
expect(session.events.some(event => event.type === 'compact/summary')).toBe(false)
expect(compact.calls).toHaveLength(0)
})
it('continues overflow recovery with summarization on the pruned surface', async () => {
const ctx = createContext(10_000)
void new ToolResultPruneService(ctx, {
thresholdChars: 100,
headChars: 20,
tailChars: 10,
})
const compact = new TestCompactService(ctx, {
thresholdRatio: 1,
retainTokens: 900,
})
const session = toolConversation()
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'retry' })
expect(session.events.some(event => event.type === 'compact/summary')).toBe(true)
expect(compact.calls).toHaveLength(1)
expect(compact.calls[0]!.text).toContain('tool result middle pruned')
})
it('preserves the newest whole tool-call/result pair during forced overflow compaction', async () => {
const ctx = createContext()
void new TestCompactService(ctx, {

View File

@@ -9,6 +9,7 @@ import Include from '@cordisjs/plugin-include'
import LlmService from '@deepseek-ai/dsh-llm'
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
import BasicCompactService from '@deepseek-ai/dsh-compact-basic'
import ToolResultPruneService from '@deepseek-ai/dsh-tool-result-prune'
let root: string | undefined
let context: Context | undefined
@@ -32,6 +33,7 @@ async function loadYaml(lines: readonly string[]): Promise<Context> {
const modules = new Map<string, unknown>([
['@deepseek-ai/dsh-llm', LlmService],
['@deepseek-ai/dsh-token-meter', TokenMeterService],
['@deepseek-ai/dsh-tool-result-prune', ToolResultPruneService],
['@deepseek-ai/dsh-compact-basic', BasicCompactService],
])
context.loader.internal = {
@@ -50,12 +52,17 @@ async function loadYaml(lines: readonly string[]): Promise<Context> {
}
describe('real Loader composition', () => {
it('loads the flat token-meter and compact-basic YAML shape', async () => {
it('loads the shipped token-meter, pruning, and compact-basic YAML order', async () => {
const loaded = await loadYaml([
"- name: '@deepseek-ai/dsh-llm'",
"- name: '@deepseek-ai/dsh-token-meter'",
' config:',
' contextWindow: 4096',
"- name: '@deepseek-ai/dsh-tool-result-prune'",
' config:',
' thresholdChars: 100',
' headChars: 20',
' tailChars: 10',
"- name: '@deepseek-ai/dsh-compact-basic'",
' config:',
' thresholdRatio: 0.5',
@@ -68,6 +75,7 @@ describe('real Loader composition', () => {
.map(entry => entry.options.name)
expect(unloaded).toEqual([])
expect(loaded.tokenMeter.contextWindow).toBe(4096)
expect(loaded.get('toolResultPrune')).toBeInstanceOf(ToolResultPruneService)
expect(loaded.get('compact')).toBeInstanceOf(BasicCompactService)
expect((loaded.compact as BasicCompactService).config).toMatchObject({
thresholdRatio: 0.5,

View File

@@ -13,6 +13,7 @@
{ "path": "../../llm/token-meter" },
{ "path": "../../core/session" },
{ "path": "../../core/agent" },
{ "path": "../compact" }
{ "path": "../compact" },
{ "path": "../tool-result-prune" }
]
}

View File

@@ -0,0 +1,50 @@
# @deepseek-ai/dsh-tool-result-prune
The replay-safe model-free pruning service (`ctx.toolResultPrune`). It rewrites over-budget `tool/result` surface nodes to a bounded head, a fixed omission marker, and a bounded tail while retaining the full original event in the append-only session log.
This is a concrete companion to [`dsh-compact-basic`](../compact-basic/README.md), not a compaction backend or model-facing tool. Compact-basic reads it through optional `ctx.get('toolResultPrune')`, so either package remains independently composable.
## Service API
`pruneSession(session)` scans one stable snapshot of the current surface. Every over-budget tool result is replaced by one newly appended `tool/result` carrying `{ surfaceOp: { op: 'replace', start: originalSeq, end: originalSeq }, sourceEventSeqs: [originalSeq] }`. The replacement spreads the complete original data and changes only `content`, preserving `turn`, `step`, `callId`, error fields, `meta`, and later data additions. The original event remains available for persistence, replay, and exact-log inspection.
`measureContent(blocks)` counts Unicode code points in `text` blocks. `pruneContent(blocks)` returns the bounded replacement or `null` when content is already within the threshold. Non-text blocks are retained at their original relative positions; text slicing never splits a UTF-16 surrogate pair, though it can split a multi-code-point grapheme cluster.
Every emitted result has exactly the configured head budget, fixed marker, and tail budget in text code points, is no larger than `thresholdChars`, and is strictly smaller than the triggering input. A second pass therefore emits no replacement.
## Config
Unrecognized keys fail at plugin construction. Resolved config is detached and deeply immutable.
| Key | Required | Meaning |
|---|---|---|
| `thresholdChars` | no (default `8192`) | Prune when combined text exceeds this many Unicode code points. |
| `headChars` | no (default `4096`) | Leading Unicode code points retained. |
| `tailChars` | no (default `1024`) | Trailing Unicode code points retained. |
All values are integers; the threshold is positive and head/tail are non-negative. `headChars + marker + tailChars` must fit within `thresholdChars`, so a valid configuration can prune every over-budget result without growth or repeated rewriting.
## Usage
```ts
import type { Context } from 'cordis'
import ToolResultPruneService from '@deepseek-ai/dsh-tool-result-prune'
export function apply(ctx: Context): void {
ctx.plugin(ToolResultPruneService)
}
```
## Model Experience
### Pruned tool result
**What the model sees**: Once a compaction trigger qualifies, future requests see the retained head, `\n\n[... tool result middle pruned ...]\n\n`, and retained tail in place of the removed text. Rich blocks keep their order. The model does not see a second copy of the original.
**Token effect**: Each rewritten tool result has at most `thresholdChars` text code points. Pruning itself makes no model call; compact-basic skips summarization when the remeasured request falls below pressure, otherwise the summarizer reads the pruned surface.
## Known Limitations and Deferred Work
- **Character budgets are not token budgets** — provider token density varies, so `ctx.tokenMeter` remains the authority for deciding whether pruning relieved request pressure.
- **Pruning is syntactic** — it retains the beginning and end without interpreting which middle lines are semantically important.
- **Grapheme clusters can split** — code-point slicing protects surrogate pairs but does not perform locale-aware grapheme segmentation.

View File

@@ -0,0 +1,40 @@
{
"name": "@deepseek-ai/dsh-tool-result-prune",
"description": "Replay-safe model-free head/middle/tail pruning for tool-result surface nodes",
"version": "0.0.1",
"private": true,
"type": "module",
"main": "lib/index.js",
"types": "lib/types/index.d.ts",
"exports": {
".": {
"types": "./lib/types/index.d.ts",
"default": "./lib/index.js"
},
"./src/*": "./src/*",
"./package.json": "./package.json"
},
"files": [
"lib/index.js",
"lib/types/**/*.d.ts",
"lib/types/**/*.d.ts.map",
"src"
],
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-llm": "^0.0.1",
"@deepseek-ai/dsh-session": "^0.0.1",
"cordis": "^4.0.0-rc.7"
},
"dependencies": {
"schemastery": "^3.18.0"
},
"devDependencies": {
"@cordisjs/plugin-include": "workspace:^",
"@cordisjs/plugin-loader": "workspace:^",
"@deepseek-ai/dsh-invariants": "workspace:^",
"@deepseek-ai/dsh-llm": "workspace:^",
"@deepseek-ai/dsh-session": "workspace:^",
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -0,0 +1,77 @@
/** Configuration resolution for deterministic tool-result pruning. */
import { deepFreeze } from '@deepseek-ai/dsh-llm'
import type { ResolvedConfig, ToolResultPruneConfig } from './types.ts'
/** Fixed marker substituted for every removed middle span. */
export const PRUNE_MARKER = '\n\n[... tool result middle pruned ...]\n\n'
/** Low-friction defaults for coding-agent tool output. */
export const DEFAULTS: ResolvedConfig = deepFreeze({
thresholdChars: 8192,
headChars: 4096,
tailChars: 1024,
})
const CONFIG_KEYS: ReadonlySet<string> = new Set([
'thresholdChars',
'headChars',
'tailChars',
])
/**
* Count Unicode code points without splitting surrogate pairs.
* @param text - text to measure.
* @returns the Unicode code-point count.
*/
export function codePointLength(text: string): number {
return Array.from(text).length
}
/**
* Resolve and validate pruning budgets.
* @param config - raw plugin configuration.
* @returns a detached deeply immutable configuration.
*/
export function resolveConfig(config: ToolResultPruneConfig = {}): ResolvedConfig {
for (const key of Object.keys(config)) {
if (!CONFIG_KEYS.has(key)) {
throw new Error(
`ToolResultPruneConfig: unknown key "${key}" `
+ '(allowed: thresholdChars, headChars, tailChars)',
)
}
}
const resolved: ResolvedConfig = {
thresholdChars: config.thresholdChars ?? DEFAULTS.thresholdChars,
headChars: config.headChars ?? DEFAULTS.headChars,
tailChars: config.tailChars ?? DEFAULTS.tailChars,
}
assertPositiveInteger('thresholdChars', resolved.thresholdChars)
assertNonNegativeInteger('headChars', resolved.headChars)
assertNonNegativeInteger('tailChars', resolved.tailChars)
const emittedChars = resolved.headChars
+ codePointLength(PRUNE_MARKER)
+ resolved.tailChars
if (emittedChars > resolved.thresholdChars) {
throw new Error(
`ToolResultPruneConfig: headChars + marker + tailChars (${emittedChars}) `
+ `must be at most thresholdChars (${resolved.thresholdChars})`,
)
}
return deepFreeze(structuredClone(resolved))
}
function assertPositiveInteger(name: string, value: number): void {
if (!Number.isInteger(value) || value <= 0) {
throw new Error(`ToolResultPruneConfig: ${name} (${value}) must be a positive integer`)
}
}
function assertNonNegativeInteger(name: string, value: number): void {
if (!Number.isInteger(value) || value < 0) {
throw new Error(`ToolResultPruneConfig: ${name} (${value}) must be a non-negative integer`)
}
}

View File

@@ -0,0 +1,157 @@
/**
* Replay-safe, model-free tool-result pruning service.
*
* @module @deepseek-ai/dsh-tool-result-prune
*/
import { Context, Service } from 'cordis'
import z from 'schemastery'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
import { codePointLength, DEFAULTS, PRUNE_MARKER, resolveConfig } from './config.ts'
import type {
PrunedEntry,
PruneResult,
ResolvedConfig,
ToolResultPruneConfig,
} from './types.ts'
export { codePointLength, DEFAULTS, PRUNE_MARKER, resolveConfig } from './config.ts'
export type {
PrunedEntry,
PruneResult,
ResolvedConfig,
ToolResultPruneConfig,
} from './types.ts'
declare module 'cordis' {
interface Context {
toolResultPrune: ToolResultPruneService
}
}
interface SnapshotCandidate {
readonly seq: number
readonly event: SessionEvent<'tool/result'>
}
/** Deterministic head/middle/tail pruning for current tool-result surface nodes. */
export class ToolResultPruneService extends Service {
static Config: z<ToolResultPruneConfig> = z.object({
thresholdChars: z.number().step(1).min(1).default(DEFAULTS.thresholdChars),
headChars: z.number().step(1).min(0).default(DEFAULTS.headChars),
tailChars: z.number().step(1).min(0).default(DEFAULTS.tailChars),
})
/** Resolved and immutable character budgets. */
readonly config: ResolvedConfig
constructor(ctx: Context, config: ToolResultPruneConfig = {}) {
super(ctx, 'toolResultPrune')
this.config = resolveConfig(config)
}
/**
* Measure text content in Unicode code points; non-text blocks cost zero.
* @param blocks - tool-result content to measure.
* @returns total Unicode code points across text blocks.
*/
measureContent(blocks: readonly ContentBlock[]): number {
let chars = 0
for (const block of blocks) {
if (block.type === 'text') chars += codePointLength(block.text)
}
return chars
}
/**
* Replace an over-budget text middle while retaining rich-block order.
* Text slicing is by Unicode code point, not UTF-16 code unit, so a retained
* boundary cannot split a surrogate pair. Grapheme clusters may still split.
* @param blocks - original tool-result content.
* @returns pruned content, or `null` when the text is within budget.
*/
pruneContent(blocks: readonly ContentBlock[]): ContentBlock[] | null {
const totalChars = this.measureContent(blocks)
if (totalChars <= this.config.thresholdChars) return null
const removedStart = this.config.headChars
const removedEnd = totalChars - this.config.tailChars
const pruned: ContentBlock[] = []
let consumed = 0
let markerInserted = false
for (const block of blocks) {
if (block.type !== 'text') {
pruned.push(block)
continue
}
const points = Array.from(block.text)
const blockStart = consumed
const blockEnd = blockStart + points.length
const headEnd = Math.min(points.length, Math.max(0, removedStart - blockStart))
const tailStart = Math.min(points.length, Math.max(0, removedEnd - blockStart))
const intersectsRemoved = blockStart < removedEnd && blockEnd > removedStart
const marker = intersectsRemoved && !markerInserted ? PRUNE_MARKER : ''
if (marker.length > 0) markerInserted = true
const text = points.slice(0, headEnd).join('')
+ marker
+ points.slice(tailStart).join('')
if (text.length > 0) pruned.push({ ...block, text })
consumed = blockEnd
}
/* v8 ignore next -- totalChars > threshold and valid budgets guarantee a removed text span. */
if (!markerInserted) throw new Error('tool-result prune: failed to locate the removed text span')
const charsAfter = this.measureContent(pruned)
/* v8 ignore next -- config validation fixes the emitted head + marker + tail budget. */
if (charsAfter > this.config.thresholdChars || charsAfter >= totalChars) {
throw new Error('tool-result prune: replacement must be smaller and within threshold')
}
return pruned
}
/**
* Prune every over-budget tool result from one stable current-surface snapshot.
* Each replacement preserves the complete event data except for `content`,
* and points at the shadowed node for durable provenance and replay.
* @param session - session whose current surface is rewritten.
* @returns landed replacements and aggregate Unicode-code-point savings.
*/
pruneSession(session: Session): PruneResult {
const candidates: SnapshotCandidate[] = []
for (const node of [...session.surface.nodes]) {
const event = session.events[node.seq]
/* v8 ignore next -- surface seqs are validated contiguous log references. */
if (event?.type === 'tool/result') candidates.push({ seq: node.seq, event })
}
const pruned: PrunedEntry[] = []
let charsRemoved = 0
for (const { seq, event } of candidates) {
const content = this.pruneContent(event.data.content)
if (content === null) continue
const charsBefore = this.measureContent(event.data.content)
const charsAfter = this.measureContent(content)
const replacement = session.append('tool/result', {
...event.data,
content,
}, {
surfaceOp: { op: 'replace', start: seq, end: seq },
sourceEventSeqs: [seq],
})
pruned.push({
originalSeq: seq,
replacementSeq: replacement.seq,
callId: event.data.callId,
charsBefore,
charsAfter,
})
charsRemoved += charsBefore - charsAfter
}
return { pruned, charsRemoved }
}
}
export default ToolResultPruneService

View File

@@ -0,0 +1,40 @@
import type { CallId } from '@deepseek-ai/dsh-llm'
/** Character-budget policy for deterministic tool-result pruning. */
export interface ToolResultPruneConfig {
/** Prune when total text exceeds this many Unicode code points. Defaults to `8192`. */
thresholdChars?: number
/** Maximum leading Unicode code points retained. Defaults to `4096`. */
headChars?: number
/** Maximum trailing Unicode code points retained. Defaults to `1024`. */
tailChars?: number
}
/** Validated, detached, deeply immutable pruning configuration. */
export interface ResolvedConfig {
readonly thresholdChars: number
readonly headChars: number
readonly tailChars: number
}
/** Provenance and size accounting for one landed surface replacement. */
export interface PrunedEntry {
/** Full-fidelity tool-result event shadowed by the replacement. */
readonly originalSeq: number
/** Newly appended pruned tool-result event. */
readonly replacementSeq: number
/** Tool call shared by the original and replacement. */
readonly callId: CallId
/** Original text size in Unicode code points. */
readonly charsBefore: number
/** Replacement text size in Unicode code points. */
readonly charsAfter: number
}
/** Aggregate outcome of one stable-surface pruning pass. */
export interface PruneResult {
/** Replacements in the snapshotted surface order. */
readonly pruned: readonly PrunedEntry[]
/** Total Unicode code points removed across replacements. */
readonly charsRemoved: number
}

View File

@@ -0,0 +1,67 @@
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { pathToFileURL } from 'node:url'
import { afterEach, describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import Loader from '@cordisjs/plugin-loader'
import Include from '@cordisjs/plugin-include'
import ToolResultPruneService from '@deepseek-ai/dsh-tool-result-prune'
let root: string | undefined
let context: Context | undefined
afterEach(async () => {
await context?.fiber.dispose()
context = undefined
if (root !== undefined) await rm(root, { recursive: true, force: true })
root = undefined
})
describe('tool-result-prune real Loader composition', () => {
it('loads and resolves the flat YAML plugin shape', async () => {
root = await mkdtemp(join(tmpdir(), 'dsh-tool-result-prune-loader-'))
const configPath = join(root, 'cordis.yml')
await writeFile(configPath, [
"- name: '@deepseek-ai/dsh-tool-result-prune'",
' config:',
' thresholdChars: 100',
' headChars: 20',
' tailChars: 10',
'',
].join('\n'))
context = new Context()
context.baseUrl = pathToFileURL(root).href + '/'
await context.plugin(Loader)
context.loader.builtins.include = Include
context.loader.internal = {
version: 'v2',
async import(specifier: string) {
if (specifier !== '@deepseek-ai/dsh-tool-result-prune') {
throw new Error(`unexpected Loader import: ${specifier}`)
}
return ToolResultPruneService
},
} as unknown as NonNullable<typeof context.loader.internal>
await context.loader.create({
name: 'cordis:include',
config: { path: pathToFileURL(configPath).href },
})
await context.loader.await()
expect(context.get('toolResultPrune')).toBeInstanceOf(ToolResultPruneService)
expect(context.toolResultPrune.config).toEqual({
thresholdChars: 100,
headChars: 20,
tailChars: 10,
})
})
it('rejects stale config after plugin schema normalization', async () => {
context = new Context()
await expect(context.plugin(ToolResultPruneService, {
maxChars: 100,
} as never)).rejects.toThrow(/unknown key "maxChars"/)
})
})

View File

@@ -0,0 +1,237 @@
import { describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import { CallId } from '@deepseek-ai/dsh-llm'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import SessionStore, { Session, SessionId } from '@deepseek-ai/dsh-session'
import type { SurfaceEvent } from '@deepseek-ai/dsh-session'
import * as Invariants from '@deepseek-ai/dsh-invariants'
import ToolResultPruneService, {
codePointLength,
DEFAULTS,
PRUNE_MARKER,
resolveConfig,
} from '@deepseek-ai/dsh-tool-result-prune'
import type { ToolResultPruneConfig } from '@deepseek-ai/dsh-tool-result-prune'
const SMALL: ToolResultPruneConfig = {
thresholdChars: 50,
headChars: 4,
tailChars: 3,
}
function service(config: ToolResultPruneConfig = SMALL): ToolResultPruneService {
return new ToolResultPruneService(new Context(), config)
}
function appendToolStep(
session: Session,
turn: number,
call: string,
content: ContentBlock[],
extra: Record<string, unknown> = {},
): number {
const callId = CallId(call)
session.append('turn/start', {
turn,
trigger: { kind: 'message', source: { kind: 'user' } },
})
session.append('step/start', { turn, step: 1 })
session.append('assistant/message', {
turn,
step: 1,
content: [{ type: 'tool-call', id: callId, name: 'bash', arguments: '{}' }],
}, { surfaceOp: 'append' })
session.append('tool/call', { turn, step: 1, callId, name: 'bash', arguments: '{}' })
const result = session.append('tool/result', {
turn,
step: 1,
callId,
content,
isError: false,
...extra,
}, { surfaceOp: 'append' })
session.append('step/end', { turn, step: 1 })
session.append('turn/end', { turn, reason: { kind: 'completed' } })
return result.seq
}
describe('tool-result pruning configuration', () => {
it('resolves detached immutable defaults and partial overrides', () => {
const raw = { thresholdChars: 100, headChars: 20, tailChars: 10 }
const resolved = resolveConfig(raw)
raw.headChars = 1
expect(resolved).toEqual({ thresholdChars: 100, headChars: 20, tailChars: 10 })
expect(Object.isFrozen(resolved)).toBe(true)
expect(DEFAULTS).toEqual({ thresholdChars: 8192, headChars: 4096, tailChars: 1024 })
expect(Object.isFrozen(DEFAULTS)).toBe(true)
})
it('rejects stale keys, invalid scalars, and an output budget above threshold', () => {
const bad = [
[{ thresholdChars: 0 }, /thresholdChars .* positive integer/],
[{ headChars: -1 }, /headChars .* non-negative integer/],
[{ tailChars: 1.5 }, /tailChars .* non-negative integer/],
[{ thresholdChars: 50, headChars: 20, tailChars: 20 }, /headChars \+ marker \+ tailChars/],
[{ threshold: 10 }, /unknown key "threshold"/],
] as Array<[unknown, RegExp]>
for (const [config, pattern] of bad) {
expect(() => resolveConfig(config as ToolResultPruneConfig)).toThrow(pattern)
}
})
})
describe('ToolResultPruneService content transform', () => {
it('measures text code points only and skips content within threshold', () => {
const prune = service()
const blocks = [
{ type: 'text', text: 'a😀b' },
{ type: 'reasoning', text: 'not measured' },
] satisfies ContentBlock[]
expect(prune.measureContent(blocks)).toBe(3)
expect(prune.pruneContent(blocks)).toBeNull()
expect(codePointLength('a😀b')).toBe(3)
})
it('keeps configured head and tail without splitting surrogate pairs', () => {
const prune = service()
const result = prune.pruneContent([{ type: 'text', text: '😀'.repeat(60) }])
expect(result).toEqual([{
type: 'text',
text: `${'😀'.repeat(4)}${PRUNE_MARKER}${'😀'.repeat(3)}`,
}])
expect(prune.measureContent(result!)).toBeLessThanOrEqual(50)
expect(result![0]).toMatchObject({ type: 'text' })
expect((result![0] as { text: string }).text).not.toContain('\uFFFD')
})
it('preserves non-text blocks and their relative ordering across removed text', () => {
const prune = service()
const reasoning: ContentBlock = { type: 'reasoning', text: 'private-rich-block' }
const call: ContentBlock = {
type: 'tool-call',
id: CallId('nested'),
name: 'nested',
arguments: '{}',
}
const result = prune.pruneContent([
{ type: 'text', text: 'A'.repeat(40) },
reasoning,
{ type: 'text', text: 'B'.repeat(30) },
call,
{ type: 'text', text: 'C'.repeat(30) },
])
expect(result).toEqual([
{ type: 'text', text: `AAAA${PRUNE_MARKER}` },
reasoning,
call,
{ type: 'text', text: 'CCC' },
])
expect(prune.measureContent(result!)).toBeLessThanOrEqual(50)
})
it('supports zero-sized head and tail while still shrinking', () => {
const prune = service({
thresholdChars: codePointLength(PRUNE_MARKER),
headChars: 0,
tailChars: 0,
})
const result = prune.pruneContent([{ type: 'text', text: 'x'.repeat(100) }])
expect(result).toEqual([{ type: 'text', text: PRUNE_MARKER }])
expect(prune.measureContent(result!)).toBe(prune.config.thresholdChars)
})
})
describe('ToolResultPruneService session transaction', () => {
it('prunes a stable snapshot, preserves all data, and records provenance', () => {
const session = new Session(SessionId('preserve'))
const originalSeq = appendToolStep(session, 1, 'one', [{
type: 'text',
text: 'x'.repeat(100),
}], {
isError: true,
error: { name: 'ExitError', code: 'EXIT_1' },
meta: { diff: ['a', 'b'] },
futureField: { nested: true },
})
session.append('turn/start', {
turn: 2,
trigger: { kind: 'message', source: { kind: 'user' } },
})
const result = service().pruneSession(session)
expect(result.pruned).toHaveLength(1)
expect(result.charsRemoved).toBeGreaterThan(0)
const entry = result.pruned[0]!
expect(entry).toMatchObject({ originalSeq, callId: CallId('one'), charsBefore: 100 })
expect(entry.charsAfter).toBeLessThanOrEqual(50)
const original = session.events[originalSeq]!
const replacement = session.events[entry.replacementSeq]! as SurfaceEvent
expect(original).toMatchObject({
type: 'tool/result',
data: { content: [{ type: 'text', text: 'x'.repeat(100) }] },
})
expect(replacement).toMatchObject({
type: 'tool/result',
data: {
turn: 1,
step: 1,
callId: CallId('one'),
isError: true,
error: { name: 'ExitError', code: 'EXIT_1' },
meta: { diff: ['a', 'b'] },
futureField: { nested: true },
},
surfaceOp: { op: 'replace', start: originalSeq, end: originalSeq },
sourceEventSeqs: [originalSeq],
})
expect(session.surface.nodes.some(node => node.seq === originalSeq)).toBe(false)
})
it('prunes multiple results, skips short ones, and converges in one pass', () => {
const session = new Session(SessionId('multiple'))
appendToolStep(session, 1, 'a', [{ type: 'text', text: 'A'.repeat(100) }])
appendToolStep(session, 2, 'b', [{ type: 'text', text: 'short' }])
appendToolStep(session, 3, 'c', [{ type: 'text', text: 'C'.repeat(80) }])
session.append('turn/start', {
turn: 4,
trigger: { kind: 'message', source: { kind: 'user' } },
})
const prune = service()
const first = prune.pruneSession(session)
const second = prune.pruneSession(session)
expect(first.pruned.map(entry => entry.callId)).toEqual([CallId('a'), CallId('c')])
expect(first.charsRemoved).toBe(
first.pruned.reduce((sum, entry) => sum + entry.charsBefore - entry.charsAfter, 0),
)
expect(second).toEqual({ pruned: [], charsRemoved: 0 })
})
it('replays to the identical pruned model messages', () => {
const session = new Session(SessionId('replay'))
appendToolStep(session, 1, 'a', [{ type: 'text', text: 'A'.repeat(100) }])
session.append('turn/start', {
turn: 2,
trigger: { kind: 'message', source: { kind: 'user' } },
})
service().pruneSession(session)
const replay = new Session(session.id, [...session.events])
expect(replay.deriveMessages()).toEqual(session.deriveMessages())
expect(replay.surface.replaceGeneration).toBe(session.surface.replaceGeneration)
})
it('runs under real invariants between closed steps but not outside a turn', async () => {
const ctx = new Context()
await ctx.plugin(SessionStore)
await ctx.plugin(Invariants)
const prune = new ToolResultPruneService(ctx, SMALL)
const session = ctx.sessions.create(SessionId('invariants'))
appendToolStep(session, 1, 'a', [{ type: 'text', text: 'A'.repeat(100) }])
expect(() => prune.pruneSession(session)).toThrow(/outside any open turn/)
session.append('turn/start', {
turn: 2,
trigger: { kind: 'message', source: { kind: 'user' } },
})
expect(() => prune.pruneSession(session)).not.toThrow()
})
})

View File

@@ -0,0 +1,15 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib/types"
},
"include": ["src"],
"references": [
{ "path": "../../../vendor/cosmokit" },
{ "path": "../../../vendor/cordis" },
{ "path": "../../../vendor/schemastery" },
{ "path": "../../llm/llm" },
{ "path": "../../core/session" }
]
}

View File

@@ -230,6 +230,15 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
'estimateMessage(message: Message): number',
],
},
{
key: 'toolResultPrune',
summary: 'Deterministic head/middle/tail pruning for current tool-result surface nodes.',
methods: [
'measureContent(blocks: readonly ContentBlock[]): number',
'pruneContent(blocks: readonly ContentBlock[]): ContentBlock[] | null',
'pruneSession(session: Session): PruneResult',
],
},
{
key: 'tools',
summary: 'Tool registry and execution pipeline.',
@@ -793,6 +802,14 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'PromptSection',
declaration: 'export interface PromptSection {\n readonly name: string;\n readonly order: number;\n readonly text: string | ((context: AssembleContext) => string);\n}',
},
{
name: 'PrunedEntry',
declaration: 'export interface PrunedEntry {\n readonly originalSeq: number;\n readonly replacementSeq: number;\n readonly callId: CallId;\n readonly charsBefore: number;\n readonly charsAfter: number;\n}',
},
{
name: 'PruneResult',
declaration: 'export interface PruneResult {\n readonly pruned: readonly PrunedEntry[];\n readonly charsRemoved: number;\n}',
},
{
name: 'ReasoningBlock',
declaration: 'export interface ReasoningBlock {\n type: \'reasoning\';\n text: string;\n}',

View File

@@ -69,6 +69,11 @@ describe('gen-tool-catalog assertManifestComplete', () => {
// is unlisted, so the guard must fire and name them.
expect(() => { assertManifestComplete([]) }).toThrow(/not in the boot manifest/)
expect(() => { assertManifestComplete([]) }).toThrow(/tool-bash/)
try {
assertManifestComplete([])
} catch (error) {
expect(String(error)).not.toContain('tool-result-prune')
}
})
})

View File

@@ -31,7 +31,7 @@ Session log (per session):
- **turns pair and nest** — `turn/start` opens a turn, `turn/end` closes the matching one; no overlapping turns.
- **steps nest in turns** — `step/start` opens a step in the open turn; `step/end` closes the matching step.
- **chunks belong to an open step** — `step/start` precedes its `assistant/chunk`s.
- **a `tool/result` needs a prior `tool/call`** — but NOT the converse: a `tool/call` may have no result (a thrown tool-execution pipeline step ends the turn with no `tool/result`, which is legal).
- **an appended `tool/result` needs a prior `tool/call`** — fresh `surfaceOp: 'append'` results name the open step and consume its pending call, while a provenance-backed single-node `replace` is a turn-enclosed surface rewrite of an already-executed result. A `tool/call` may still have no result when the execution pipeline throws.
- **provenance sources are valid and unambiguous** — `sourceEventSeqs` contains unique earlier known seqs; only `assistant/message` may carry an explicit empty list, which denotes a known empty provider stream rather than absent legacy provenance.
Agent status (per agent):

View File

@@ -232,6 +232,17 @@ function validateEvent(trace: SessionTrace, event: SessionEvent): SessionTraceTr
break
}
case 'tool/result': {
// A replacement rewrites an already-executed result whose recorded
// turn/step can be closed. Surface provenance above validates the rewrite;
// only fresh appends consume an open step's pending call.
if (se.surfaceOp !== undefined && se.surfaceOp !== 'append') {
if (trace.openTurn === null) {
throw new InvariantError(
'tool/result surface replacement appended outside any open turn',
)
}
break
}
requireOpenStep(trace, 'tool/result', event.data.turn, event.data.step)
// A result needs a prior matching call in the same step. (The converse
// does NOT hold: a call may have no result — a throwing tool-execution

View File

@@ -189,6 +189,19 @@ describe('session-log invariants', () => {
.toThrow(/no prior tool\/call/)
})
it('keeps fresh tool-result appends open-step and pending-call checked', async () => {
const { ctx } = await setup()
const session = ctx.sessions.create()
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
expect(() => session.append('tool/result', {
turn: 1,
step: 1,
callId: CallId('closed'),
content: [],
isError: false,
}, { surfaceOp: 'append' })).toThrow(/open is turn 1\/step null/)
})
it('allows a synthetic interrupted tool/result from crash repair without a prior tool/call event', async () => {
const { ctx } = await setup()
const session = ctx.sessions.create()
@@ -487,6 +500,38 @@ describe('surface invariants', () => {
// no throw — well-formed replace op
})
it('treats a provenance-backed tool-result replacement as a turn-enclosed rewrite', async () => {
const { ctx } = await setup()
const session = ctx.sessions.create()
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
session.append('step/start', { turn: 1, step: 1 })
session.append('tool/call', {
turn: 1,
step: 1,
callId: CallId('rewrite'),
name: 'echo',
arguments: '{}',
})
const original = session.append('tool/result', {
turn: 1,
step: 1,
callId: CallId('rewrite'),
content: [{ type: 'text', text: 'original' }],
isError: false,
}, { surfaceOp: 'append' })
session.append('step/end', { turn: 1, step: 1 })
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
expect(() => session.append('tool/result', {
...original.data,
content: [{ type: 'text', text: 'pruned' }],
}, {
surfaceOp: { op: 'replace', start: original.seq, end: original.seq },
sourceEventSeqs: [original.seq],
})).not.toThrow()
})
it('accepts known-empty assistant provenance and rejects empty provenance elsewhere', async () => {
const { ctx } = await setup()
const session = ctx.sessions.create()

31
pnpm-lock.yaml generated
View File

@@ -256,6 +256,9 @@ importers:
'@deepseek-ai/dsh-token-meter':
specifier: workspace:^
version: link:../../llm/token-meter
'@deepseek-ai/dsh-tool-result-prune':
specifier: workspace:^
version: link:../tool-result-prune
'@deepseek-ai/dsh-tools':
specifier: workspace:^
version: link:../../core/tools
@@ -263,6 +266,31 @@ importers:
specifier: ^4.0.0-rc.7
version: 4.0.0-rc.7(@cordisjs/plugin-include@vendor+include)(@cordisjs/plugin-loader@vendor+loader)
packages/compact/tool-result-prune:
dependencies:
schemastery:
specifier: ^3.18.0
version: 3.18.0
devDependencies:
'@cordisjs/plugin-include':
specifier: workspace:^
version: link:../../../vendor/include
'@cordisjs/plugin-loader':
specifier: workspace:^
version: link:../../../vendor/loader
'@deepseek-ai/dsh-invariants':
specifier: workspace:^
version: link:../../support/invariants
'@deepseek-ai/dsh-llm':
specifier: workspace:^
version: link:../../llm/llm
'@deepseek-ai/dsh-session':
specifier: workspace:^
version: link:../../core/session
cordis:
specifier: ^4.0.0-rc.7
version: 4.0.0-rc.7(@cordisjs/plugin-include@vendor+include)(@cordisjs/plugin-loader@vendor+loader)
packages/context/time-context:
dependencies:
schemastery:
@@ -2112,6 +2140,9 @@ importers:
'@deepseek-ai/dsh-tool-fs':
specifier: workspace:^
version: link:../../packages/fs/tool-fs
'@deepseek-ai/dsh-tool-result-prune':
specifier: workspace:^
version: link:../../packages/compact/tool-result-prune
'@deepseek-ai/dsh-tool-skill':
specifier: workspace:^
version: link:../../packages/skill/tool-skill

View File

@@ -31,6 +31,7 @@
"@deepseek-ai/dsh-jsonrpc-demo": "workspace:^",
"@deepseek-ai/dsh-llm": "workspace:^",
"@deepseek-ai/dsh-token-meter": "workspace:^",
"@deepseek-ai/dsh-tool-result-prune": "workspace:^",
"@deepseek-ai/dsh-llm-deepseek": "workspace:^",
"@deepseek-ai/dsh-llm-pi-ai": "workspace:^",
"@deepseek-ai/dsh-permission": "workspace:^",

View File

@@ -94,6 +94,14 @@ const SERVICE_ROLES: ServiceRole[] = [
consumers: ['compact-basic'],
note: 'Owns isolated per-session replay folds; pressure consumers share immutable revisioned measurements.',
},
{
key: 'toolResultPrune',
pkg: 'tool-result-prune',
title: 'Model-free tool-result pruning',
mode: 'core',
consumers: ['compact-basic'],
note: 'Rewrites oversized current tool results through replayable single-node surface replacements before summary compaction.',
},
{
key: 'sessions',
pkg: 'session',
@@ -415,7 +423,7 @@ const APP_EXAMPLES = [
title: 'Coding Agent App Composition',
label: 'examples/coding-agent',
config: 'examples/coding-agent/cordis.yml',
summary: 'The coding REPL demo adds the real DeepSeek adapter, filesystem tools, todo_write, compaction, and both subagent transports on top of the stdio app package.',
summary: 'The coding REPL demo adds the real DeepSeek adapter, filesystem tools, todo_write, tool-result pruning, compaction, and both subagent transports on top of the stdio app package.',
},
{
id: 'cordis',
@@ -851,7 +859,7 @@ function renderLifecycle(): string {
'',
'The `assistant/message` edge records every successful provider call, including content-less and `max-tokens` finishes. Empty content stays out of derived history while the durable anchor retains usage and exact chunk provenance, including an explicit empty source set.',
'',
'`dsh-compact-basic` uses `agent/post-step` for pressure after those durable facts and `agent/request-error` only for canonical context overflow. Recovery compacts between the closed failed step and a fresh retry step, and returns retry only when the surface replacement generation advances; otherwise the original request error remains authoritative.',
'`dsh-compact-basic` uses `agent/post-step` for pressure after those durable facts and `agent/request-error` only for canonical context overflow. Once either trigger qualifies, optional tool-result pruning runs before summary selection. Recovery works between the closed failed step and a fresh retry step, and returns retry only when pruning or summarization advances the surface replacement generation; otherwise the original request error remains authoritative.',
'',
'SDK users that need replayable transcript data should consume `session/event`; `agent/*` is the live coordination surface for queue/status, prompt interception, request shaping, steering, continuation, and errors.',
'',

View File

@@ -1,8 +1,9 @@
/**
* Generate `docs/tool-catalog.md` from schemas collected by booting each tool
* plugin. Runtime registration is the source of truth for computed schemas;
* the manifest is checked against every on-disk `tool-*` package. `--check`
* verifies the committed artifact. Rationale and ownership live in
* the manifest is checked against every on-disk model-facing `tool-*` package;
* non-model service packages with that prefix are explicitly excluded.
* `--check` verifies the committed artifact. Rationale and ownership live in
* `docs/rfc/implemented/process/2026-07-02-tool-schema-catalog.md`.
*/
@@ -38,6 +39,9 @@ import * as ToolWorkflow from '@deepseek-ai/dsh-tool-workflow'
const root = resolve(import.meta.dirname, '..')
const OUT = 'docs/tool-catalog.md'
/** `tool-*` leaves that are runtime services, not contributors to `ctx.tools`. */
const NON_MODEL_TOOL_PACKAGES = new Set(['tool-result-prune'])
/**
* Tool package plus its hand-maintained boot recipe. The caller mounts the
* prompt and registry; each recipe supplies only package-specific seams and
@@ -77,9 +81,10 @@ interface ToolPackage {
}
/**
* The boot manifest: every shipped tool package (a `tool-*` leaf under
* `packages/`). Ordered by package name (the render order); the completeness
* guard proves it is exhaustive against the on-disk glob.
* The boot manifest: every shipped model-facing tool package (a `tool-*` leaf
* under `packages/`, excluding {@link NON_MODEL_TOOL_PACKAGES}). Ordered by
* package name (the render order); the completeness guard proves it is
* exhaustive against the filtered on-disk glob.
*/
const TOOL_PACKAGES: ToolPackage[] = [
{
@@ -256,8 +261,9 @@ interface CatalogPackage {
export type ToolCatalog = CatalogPackage[]
/**
* Assert the boot manifest covers every shipped tool package on disk (a
* `tool-*` leaf under `packages/`).
* Assert the boot manifest covers every shipped model-facing tool package on
* disk (a `tool-*` leaf under `packages/`, excluding explicit service-only
* entries in {@link NON_MODEL_TOOL_PACKAGES}).
* Booting has no source declaration to enumerate, so this glob restores the
* "a new tool cannot be silently undocumented" guarantee: an unlisted package
* fails the generator (and the freshness gate) until it is added to
@@ -266,7 +272,10 @@ export type ToolCatalog = CatalogPackage[]
* `scanRoot` defaults to the repo root; a test may point it at a fixture tree.
*/
export function assertManifestComplete(packages: ToolPackage[] = TOOL_PACKAGES, scanRoot: string = root): void {
const onDisk = globSync('packages/*/tool-*', { cwd: scanRoot }).map(p => basename(p)).sort()
const onDisk = globSync('packages/*/tool-*', { cwd: scanRoot })
.map(p => basename(p))
.filter(dir => !NON_MODEL_TOOL_PACKAGES.has(dir))
.sort()
const listed = new Set(packages.map(p => p.dir))
const missing = onDisk.filter(dir => !listed.has(dir))
if (missing.length > 0) {
@@ -339,9 +348,9 @@ export function render(catalog: ToolCatalog): string {
'',
'Every model-facing tool a shipped plugin contributes to `ctx.tools`: the `name`, `description`, and JSON-Schema `parameters` the model receives via the system-prompt assembly. It complements the cordis [events](cordis-catalog/events.md) & [services](cordis-catalog/services.md) catalogs (the wiring a plugin listens to and calls) and [core-data-structures/](core-data-structures/core.md) (the types those signatures move) — this page is the *tools* the agent is offered.',
'',
'This file is GENERATED and verified fresh by `pnpm run verify-tool-catalog` (part of `doc-sync`) — do not edit it by hand. Unlike the cordis catalog (a pure source-AST pass), this generator BOOTS each tool plugin on a real context and reads `ctx.tools.schemas()`, because a tool schema is not statically knowable (runtime-spread enums, concatenated descriptions, config-driven names, raw-JSON-Schema MCP tools). A completeness guard globs `packages/*/tool-*` and fails if any package is missing from the generator\'s boot manifest, so a new tool cannot be silently undocumented. See [the tool-schema-catalog RFC](rfc/implemented/process/2026-07-02-tool-schema-catalog.md).',
'This file is GENERATED and verified fresh by `pnpm run verify-tool-catalog` (part of `doc-sync`) — do not edit it by hand. Unlike the cordis catalog (a pure source-AST pass), this generator BOOTS each tool plugin on a real context and reads `ctx.tools.schemas()`, because a tool schema is not statically knowable (runtime-spread enums, concatenated descriptions, config-driven names, raw-JSON-Schema MCP tools). A completeness guard globs `packages/*/tool-*` and fails if any model-facing package is missing from the generator\'s boot manifest; service-only packages that share the prefix are explicitly excluded. See [the tool-schema-catalog RFC](rfc/implemented/process/2026-07-02-tool-schema-catalog.md).',
'',
'Scope: shipped product tools under `packages/*/tool-*`, each booted with its DEFAULT config. The registered tool NAME can be a load-time config (e.g. `tool-subagent`\'s `toolName`), so a deployment may surface a package under a different or additional name — a per-package note records those shipped aliases where they exist. The `examples/` demo tools (e.g. `echo`) are excluded, matching the cordis catalog\'s packages-only scope.',
'Scope: shipped model-facing product tools under `packages/*/tool-*`, each booted with its DEFAULT config. Runtime service packages such as `tool-result-prune` do not register `ctx.tools` schemas and are explicitly excluded. The registered tool NAME can be a load-time config (e.g. `tool-subagent`\'s `toolName`), so a deployment may surface a package under a different or additional name — a per-package note records those shipped aliases where they exist. The `examples/` demo tools (e.g. `echo`) are excluded, matching the cordis catalog\'s packages-only scope.',
'',
'## Tool Package Map',
'',

View File

@@ -38,6 +38,7 @@
{ "path": "./packages/code-runtime/code-runtime-worker" },
{ "path": "./packages/compact/compact" },
{ "path": "./packages/compact/compact-basic" },
{ "path": "./packages/compact/tool-result-prune" },
{ "path": "./packages/llm/llm-deepseek" },
{ "path": "./packages/llm/llm-pi-ai" },
{ "path": "./packages/bash/bash-local" },

View File

@@ -60,6 +60,7 @@
{ "path": "./packages/fs/tool-fs" },
{ "path": "./packages/compact/compact" },
{ "path": "./packages/compact/compact-basic" },
{ "path": "./packages/compact/tool-result-prune" },
{ "path": "./packages/web/web" },
{ "path": "./packages/web/web-search-exa" },
{ "path": "./packages/web/web-search-perplexity" },