The `if (error instanceof WebError) throw error` guard is dead code in the
exa and perplexity providers: their mappers (mapExaResponse /
mapPerplexityResponse) never throw a WebError — a wrong-shape body throws a
TypeError, which the catch correctly translates to WEB_PROVIDER_ERROR. The
guard was added for symmetry with the deepseek provider, whose mapper DOES
throw a WebError in strict mode (no web_search_tool_result block), so it
keeps the rethrow. The unreachable lines tripped the per-file 100% coverage
gate.
- search providers (exa/perplexity/deepseek): map the parsed response
INSIDE the parse try, so a well-formed body of the wrong shape surfaces
as WEB_PROVIDER_ERROR instead of escaping as a raw TypeError; a WebError
the mapper throws on purpose is re-thrown untouched
- web-fetch-local: validate numeric limits at plugin construction (positive
finite caps; non-negative integer maxRedirects) rather than constructing a
provider with nonsensical values
- web-fetch-local: enforce the redirect budget BEFORE resolving each hop, so
maxRedirects:N follows exactly N redirects and an over-limit hop reports
"exceeded the maximum" rather than misdiagnosing a cross-origin block
- drop the stale dsh-tool-web/search and /fetch path aliases (the package no
longer declares those subpath exports)
- strip trailing EOF blank lines flagged by git diff --check
Each fix carries a regression test.
Use maxTokens as the provider generation cap and remove the confusing stored-summary max config.
Strip reasoning blocks before storing compaction summaries, reject non-shrinking summaries, and retry bounded re-compaction when the surface remains over threshold.
Add config validation for numeric and type-shaped knobs plus unit and real-API e2e coverage for reasoning-capable summarization.
Honor cancellation and disposal around async pre-step setup before the loop can open a step or call the model.
Route compaction summarization through agent/request so router agents can select the model, and remove the stale model argument from agent/pre-step.
Document serial events and the approximate convergence bound, regenerate the Cordis catalog, and add regression coverage for router compaction, HMR cleanup, and assembly/pre-step interruption.
The registry's validateArgs rejects a bad `status` enum before execute runs, so
the in-body re-check (`status !== 'pending' && …` → throw) was unreachable dead
code — line 70 was uncovered, failing the per-file 100% coverage gate. Narrow
the registry-guaranteed value with `status as TodoItem['status']` instead of
re-validating it, mirroring tool-bash (which only checks what the DSL can't
express). The malformed-status test still passes — it exercises the registry's
rejection, the actual path. Coverage back to 100%.
Add @deepseek-ai/dsh-web-search-deepseek: a WebSearchProvider that calls
DeepSeek's Anthropic-compatible Messages API with the native
web_search_20250305 server tool and parses the structured
web_search_tool_result blocks into the ctx.web seam's WebSearchResult.
- Namespace plugin (inject: ['web']), no default export — registers into
ctx.web like dsh-llm-deepseek registers into ctx.llm.
- Strict mode: a response with no web_search_tool_result block throws
WEB_PROVIDER_ERROR rather than scraping URLs from model prose.
- Reuses $DEEPSEEK_API_KEY; baseURL defaults to the Anthropic-compatible
base (api.deepseek.com/anthropic/v1) and does NOT reuse
$DEEPSEEK_BASE_URL, which belongs to the chat-completions LLM adapter.
- snippet joined from text-block citations; sources deduped by url.
- Two-stage build layout (outDir lib/types) matching the other web
packages; registered in tsconfig.json, tsconfig.build.json, knip.json,
and docs/module-graph.md.
Codex confirmation review: the trim-the-stored-content fix had no test that
would fail if it regressed (existing assertions use already-trimmed todos).
Add a focused test asserting " plan the work " appends content "plan the
work". Verified it fails red against the pre-fix code.
toTodoList dedupes and length-checks on the trimmed content but stored the raw
item.content, so a todo with leading/trailing whitespace was deduped by its
trimmed form yet persisted untrimmed — the stored value and the uniqueness key
could differ. Store the trimmed content so the persisted list matches what was
validated.
Two cohesion cleanups on the filesystem tool package:
- Fold window.ts + types.ts + formatReadOutput into one cordis-free
read-render.ts. Line windowing, the FileReadOutcome shape, and output
formatting are one concern (the read tool's rendering); splitting them across
three files added no value. read.ts is now just the tool (schema + I/O).
- Drop observe.ts and emit fs/observed with a plain ctx.emit in read/write/edit.
The event is contractually a synchronous, side-effect-only recorder
(file-context's listener is a WeakMap.set), so the per-call try/catch guarded
against a contract violation that cannot happen under the shipped listener —
defensive code for an impossible case. The event contract (dsh-fs JSDoc,
README, RFC) is updated to state the fire-and-forget semantics plainly.
Add @deepseek-ai/dsh-tool-todo (a new packages/todo/ group): a model-facing
todo_write(todos: [{content, status}]) tool with whole-list-replace semantics.
Each call appends the full list as a todo/write event to the calling agent's
session log; the current list is the most recent such event (last-write-wins).
Single-owner — a non-agent caller is rejected. Beyond the schema's
type/required/enum checks, execute rejects empty/duplicate content and more than
one in_progress task, narrowing the loosely-typed args into a real TodoItem[].
Both UIs render off the existing session/event: the stdio UI prints a glyphed
checklist; the ACP bridge maps the list to a `plan` sessionUpdate (todosToPlan
synthesizes the priority ACP requires; status maps 1:1). Wired into the
coding-agent, acp-agent, and snapshot example configs with a system-prompt nudge.
Tests: unit (schema, validation, append/replace, no-agent rejection, presentCall,
HMR-safety, Loader export-shape guard), full-loop integration through the agent
loop, the ACP todosToPlan mapping + stream-update arm, the stdio render arm, and
a session/load replay that re-emits the plan. New-group TS wiring added to
tsconfig.base/json/build. RFC + a doc-inventory sweep (architecture, packages
README, AGENTS layout, cookbook group list, example READMEs) ship with it.
The todo-plan ACP snapshot scenario is recorded separately (needs an API key).
Codex Phase 1 review: the event JSDoc described Phase 2 consumers (the
todo_write tool, stdio printing, ACP plan mapping) as current state, and put an
@mode tag on a SessionEventMap member. @mode is for first-class Cordis
`interface Events` entries the catalog generator reads — this event rides the
existing session/event emit and has no catalog row, so the tag was wrong.
Trim the JSDoc to the event's own contract (snapshot data shape,
last-write-wins, not-a-surface-event) and drop @mode; phrase TodoItem in terms
of its own purpose rather than a not-yet-present tool.
Add the TodoItem type and a todo/write SessionEventMap variant carrying the
whole todo list as a snapshot (last-write-wins on replay). It is NOT a
SurfaceEventType: it produces no LLM message and never reaches
deriveMessages(), so it carries no surfaceOp and stays off the surface — it is
durable, replayable UI state that rides the existing session/event emit.
Tests cover the snapshot-clone-on-append contract, last-write-wins, the
not-on-surface guarantee, and a seeded replay round-trip. Docs: session.md
gains the TodoItem type-equiv block + the event member; core.md's variant count
goes to twelve; the type-equiv manifest gains TodoItem.
Adapt the four fs packages to master's single-tsconfig build convention
(lib/types outDir + types path + files allowlist), brought in by the merge.
While doing so, drop dsh-tool-fs's /read//write//edit subpath plugins. They
were the only subpath-export package in the tree and forced bespoke tsdown,
tsconfig path, package.json files, and workspace-constraint handling that no
sibling tool package (e.g. dsh-tool-bash) carries, for a focused-deployment
use case no consumer needed. dsh-tool-fs is now a single root plugin that
registers read/write/edit, mirroring dsh-tool-bash; the per-tool registration
helpers stay internal modules the root composes. The file-context event-gate
RFC is amended to record the narrowed scope.
The web tool package exposed ./search and ./fetch as standalone subpath
plugins, but nothing consumed them, the RFC never called for them, and the
sibling dsh-tool-bash (also a multi-tool consumer) ships a single entry and
selects tools via config. The extra entries also tripped the workspace
constraints gate, whose expected `files` list covers single-entry and bin
packages but not a non-bin multi-entry one.
Collapse to a single `.` entry: drop the ./search|./fetch exports and their
lib/*.js from package.json files, delete the per-package tsdown override (the
root config's lib/types/index.js entry now suffices), and remove the
plugin-shaped name/inject exports from search.ts/fetch.ts (renaming each
apply to its applyWeb{Search,Fetch}Tool helper, still composed by the root
plugin and re-exported from the index). Selective enablement stays via the
existing { search?, fetch? } config. Docs updated to match.
- edit tool: add the read-before-edit requirement to the model-facing prompt
(with the just-created/edited-this-session exception), matching write's
guidance so the model doesn't only learn it via a failed FS_NOT_OBSERVED call.
- fsspec-style-fs-seam RFC: correct the acceptance criteria that still claimed
a ctx.fileContext service and a fileContext inject — the landed design is the
fs/* event gate with the tool injecting fs.
- filesystem-tool-schemas RFC: replace the stale "prior full file state"
edit requirement with version-freshness wording (any windowed read authorizes
a fresh edit; no partial-view flag).
Correct the filesystem-wiring claims flagged by codex: no default/example
config wires the fs tools yet (the demo agents do file ops through bash), so
the docs and RFC no longer assert that "the default product config loads
dsh-file-context". They now state the intended stance — a deployment that
loads the fs tools is expected to also load dsh-file-context for
read-before-write/edit.
Invert the tool↔policy control flow per the file-context event-gate RFC.
dsh-tool-fs becomes the executor — it reads/writes/edits through ctx.fs
directly, owns read windowing, and dispatches fs/write-expectation /
fs/edit-expectation (single-slot waterfalls) plus a contained fs/observed
emit. dsh-file-context drops its ctx.fileContext service and becomes a pure
event-gate plugin (observed-state + read-before-edit + version-guarded
write/edit, decided on those events). The provider's version guard becomes
optional so ctx.fs alone is a complete unconstrained text-storage seam:
removing the policy plugin gracefully loses the policy instead of breaking
the tool at a service-injection boundary.
The per-file 100% coverage gate flagged surface.ts line 46 — the
branch where a surface-eligible event type carries no surfaceOp marker
(isSurfaceEvent returns false). Exercise both guards directly: the
type-only eligibility check, the positive narrowing path, a
non-eligible type, and the markerless-but-eligible branch.
The web packages were authored against the old single-stage layout where
tsc emitted directly to lib/. Master compiles declarations to lib/types/
via tsc -b, then bundles JS into lib/ via tsdown. Point every web package's
tsc outDir at lib/types, update package.json types/exports/files to the
lib/types declaration + lib/ bundle shape (matching dsh-bash/dsh-tool-bash),
and bundle tool-web's subpath entries from lib/types/*.js rather than src.
The function carried two consecutive JSDoc blocks; the first was an
outdated short version missing the timeout-recovery contract. Keep only
the accurate detailed block.
Resource-lifecycle and error-classification fixes in the local fetch provider:
- Classify a timeout that fires DURING the body read as WEB_FETCH_TIMEOUT, not
WEB_ABORTED: thread the controller signal into the body-read translate path
and recover the timeout WebError from signal.reason, honoring the public
WEB_FETCH_TIMEOUT contract for a stalled response body.
- Cancel the response body before every blocked-redirect throw path
(cross-origin, invalid target, missing Location), so a rejected redirect with
a large or streaming body does not leak the socket after the tool returns
WEB_REDIRECT_BLOCKED.
- Cancel the body when charset validation fails, matching the
unsupported-content-type and over-size paths (the round-1 charset check threw
before readCapped owned the stream).
- Preserve abort errors while parsing search responses: when the caller's
AbortSignal fires after headers but during response.json() (both the success
and HTTP-error body parses), surface WEB_ABORTED instead of wrapping it as
WEB_PROVIDER_ERROR, so agent cancel/dispose is not misreported as a provider
failure. Applied to both the Exa and Perplexity providers.
- Report a malformed baseURL as misconfigured in status() (URL.canParse), so
selection diagnostics and execution agree (configured-unavailable up front
rather than a late WEB_PROVIDER_ERROR). WebProviderStatus already had the
reason.
- Re-validate redirect targets through validateFetchUrl before following, so a
same-origin Location carrying credentials (or a non-http(s)/over-long URL)
cannot bypass the transport hygiene a direct request enforces.
- Treat only DROPPED bytes as truncation: a body exactly at maxResponseBytes is
no longer falsely flagged truncated (which emitted a spurious footer).
- Honor the declared response charset: parse the Content-Type charset and decode
with it (rejecting unsupported labels as WEB_UNSUPPORTED_CONTENT_TYPE) instead
of always assuming UTF-8 and returning replacement characters.
- Catalog the web seam vocabulary in docs/core-data-structures/web.md with
type-equiv blocks + manifest entries, per the core-data-structures rule.
Introduce web access as a first-class capability seam so the model-facing
web tools stay stable while backends change. dsh-web owns ctx.web as a
provider registry with registration-order-independent selection and the
WebError taxonomy; dsh-web-search-exa, dsh-web-search-perplexity, and
dsh-web-fetch-local register capabilities into it; dsh-tool-web is the sole
owner of the model-facing web_search/web_fetch schemas, prompt sections, and
HTML-to-markdown presentation. Search and fetch are deliberately one seam.
Providers ship as namespace plugins that register into ctx.web (like an
LlmAdapter into ctx.llm), not key-owning services, since multiple search
providers cannot each own the key. Tool registration follows product
enablement, not backend availability, so load order/credentials never enter
the model contract; the seam resolves the provider at execution time and
surfaces a structured WebError otherwise.
Moves the RFC to implemented/ amended to match what shipped. Example/app
configs are intentionally not wired yet (RFC migration step 6).
Record the freshness token observed AFTER the read (re-stat post-read, falling
back to the routing stat if the file vanished) so the version returned/recorded
matches the bytes returned — a writer racing between the routing stat and the
read can no longer make a follow-up edit spuriously stale. Stream reads when the
backend reports no size, so a size-less backend never buffers a large file
whole. Update the cordis-catalog link map to the current filesystem API symbols
(FileContextExec/FileReadRequest/FileReadOutcome/FsInfo/FsWriteExpectation).
Resolve targetKey by realpathing the nearest EXISTING ancestor and re-appending
the missing suffix, so a not-yet-created file under a symlinked ancestor with
missing intermediate dirs gets the same key before and after creation — keeping
observed-state intact across a write→edit cycle. Make the socket-type probe test
skip (not fail) when a sandbox forbids unix-domain sockets.
Translate a mid-read AbortError from readFile into the seam's structured
FsError('FS_ABORTED') in readWholeText and readForEdit (the streaming/write
paths already did), and make the socket-type probe test reject on a listen
error instead of hanging where unix-domain sockets are unavailable.
Implements the split-the-filesystem-seam RFC. ctx.fs shrinks to a text-storage
provider seam (resolve/stat/readText/streamText/writeText/editText with branded
FsTargetKey/FsVersion and an explicit FsWriteExpectation); the new
dsh-file-context package owns the model-facing policy (read windowing,
observed-state, write/edit freshness) as the concrete ctx.fileContext service.
Authorization is now freshness-based rather than full/partial view: a windowed
read records the file version and authorizes a later edit when the file is
unchanged, removing the dead-end where reading lines 100-150 of a large file
could not edit line 120. editText stays a provider primitive so version guard +
literal match + atomic rewrite remain one critical section, and the stale check
runs before matching so a stale edit reports FS_STALE_VERSION. tool-fs injects
fileContext, never reaching around to ctx.fs (the no-bypass contract).
The compaction e2e never exercised compaction: its window/fixture combo
(contextWindow 8000, thresholdRatio 0.5 → threshold 4000; four small files)
peaked at ~1389 estimated tokens, so compactIfNeeded declined every pre-step
and compact/start never landed. Shrink the window (contextWindow 2400 →
threshold 1200; retainTokens 500 + summarizationMaxTokens 300 = 800 < 1200,
convergence holds) and grow the fixture to six files so a couple of bash steps
reliably cross the threshold. Verified compaction fires and the suite passes
across repeated real-API runs.
Sync docs left stale by the landed compaction work: list compaction.e2e.ts and
keyless-smoke.e2e.ts in the coding-agent README (and fix the wrong "Both
self-skip" count), add compaction to the examples with-key inventory, and
replace the hypothetical compaction/marker / "future plugin" naming in the
session README, session types JSDoc, and the core-data-structures catalog with
the real compact/start, compact/summary, compact/end events.
Manual review round, two non-blocking findings:
- CBR-005: _extractText's JSDoc claimed it "walks events in log order",
but it walks the seqs in surface order (the inline comment already said
so) — the exact distinction CBR-001 paid for, since after a replace a
high-seq checkpoint heads the surface before lower-seq retained nodes.
Corrected the JSDoc to match.
- CBR-006: the "HMR safety" suite only asserted registration; the actual
dispose-and-confirm-cleanup test lived under "llm inject", so a reader
searching by name could miss it. Added a disposal test to the HMR-safety
suite (mount via the real plugin fiber with LlmService present so inject
resolves, dispose, assert ctx.get('compact') is undefined) and reframed
the llm-inject test's trailing teardown to point at it.
Codex round 1 CBR-003: several docs still described compaction as an
`agent/request` waterfall concern, and the implemented compaction RFC
claimed "No changes to dsh-session or dsh-invariants" while the diff
changed both.
- Package READMEs / JSDoc (agent, agent-loop, system-prompt, compact,
compact-basic): compaction now lives on the serial `agent/pre-step`
seam (fired after turn/start, before step/start); the structural guard
is tool-pairing balance (`isToolPairingBalanced`), not step-alignment;
the convergence bound is strict (`>=` rejects).
- architecture.md / core-data-structures/compaction.md: same seam +
predicate + dispatch-mode updates; regenerated cordis catalog.
- Implemented compaction RFC, updated in place to describe shipped
reality: the seam is `agent/pre-step` (@mode serial) fired before
step/start; alignment is surface tool-pairing balance; the convergence
invariant rejects `>=`; and the "no dsh-session/dsh-invariants changes"
claim is corrected — dsh-session gains the tool-pairing predicate and
dsh-invariants drops its `start <= end` replace assertion (a positional
replace makes start > end normal).
Codex round 1 CBR-002: `resolveConfig` rejected only
`summarizationMaxTokens + retainTokens > threshold` (allowing equality),
but `compactIfNeeded` declines only when the estimate is `< threshold`.
At exact equality the post-compaction history sits at the threshold and
re-triggers on the very next check.
Make the bound strict (`>=` rejects), so post-compaction history is
guaranteed strictly below the threshold. Updated the boundary test (the
sum-equals-threshold case is now rejected, not accepted) and added an
"accepts just below the threshold" case; nudged one unrelated config that
incidentally sat at the equality boundary.
Codex round 1 CBR-001: a head-anchored compaction checkpoint was
mis-classified by the log-position step-alignment scan, so a second
auto-compaction over a checkpoint-headed surface silently failed.
Root cause: `isStepAlignedStart/End` scanned the LOG by seq, but a
`replace` op lands a checkpoint at a high log seq whose SURFACE position
is the head — its log neighbours (the open step's assistant/message) are
not its surface neighbours, so the forward scan wrongly reported mid-step.
Fix, per the agreed direction:
- Replace the two log-position predicates with one surface-anchored
helper `isToolPairingBalanced(nodes, events, beforeSeq)` in
`dsh-session` (renamed step-boundary.ts → tool-pairing.ts). A cut is
balanced when no unanswered tool-call precedes it on the surface; a
region is collapsible iff both edges are balanced cuts. The open-tail
and free-node cases fall out of the same counter. It also throws on a
corrupt surface (a tool/result with no matching call).
- Move compaction off the in-step seam to a new "pre-step" seam fired
after turn/start and before step/start, so a compaction's log-only
compact/* records and its replacement node land cleanly OUTSIDE any
step (the honest structure crash-safety relies on). Renamed the event
agent/pre-request → agent/pre-step and switched its dispatch from
parallel → serial (listeners mutate the surface as a side effect;
serial isolates them so concurrent appends can't interleave). Extended
the catalog generator to accept @mode serial.
Regression coverage: a real-loop test driving an auto-compaction asserts
the landed checkpoint is a balanced cut on both sides; unit tests pin the
checkpoint case, the mid-step injection case, multi-call steps, and the
corrupt-surface guard. Proven red on the old log-position logic.
Reform the compaction blueprint so a runaway turn survives and the design
stops drifting across review rounds:
- Drop in-flight-turn protection ("layer 2"). Retention is a uniform tail→head
whole-unit walk; the only structural guard is step-alignment. A single turn
that alone exceeds the window now compacts its own early closed steps instead
of being retained verbatim (the failure mode that motivated this).
- Move auto-compaction off the agent/request waterfall onto a new awaited
agent/pre-request loop seam, fired before history derivation. Compaction
mutates the surface; the loop derives once from the result — no double-derive,
and a listener structurally cannot act on not-yet-derived messages.
- Tighten compactIfNeeded to required (session, system, model, signal).
- Enforce a single-pass convergence invariant in resolveConfig: reject configs
where summarizationMaxTokens + retainTokens exceeds the threshold, so a
compaction can never immediately re-trigger.
- Document the crash vs recoverable failure taxonomy; core session repair stays
compaction-agnostic (a log-only orphaned compact/start is inert).
- Wire dsh-compact-basic into examples/coding-agent and add a with-key
compaction e2e (compaction's first real-world exercise + runaway net).
- Rewrite the RFC to encode the blueprint and move it to implemented/.
The runaway-turn snapshot is a named deferred follow-up: dsh-llm-replay cannot
yet serve the interleaved summarization model call.
Collapses the per-round review churn of the prior compact-basic branch into a
single clean baseline on top of compact-interface, so the upcoming retention
refactor lands as fresh, well-scoped commits rather than stacking on a history
of fixes that are being superseded.
