Commit Graph

18 Commits

Author SHA1 Message Date
Tianyi Cui
3f83a4ee96 review: the persona becomes the system-prompt plugin's deployment config
Review round 2 (tianyicui inline comments):

- dsh-system-prompt itself registers the harness:identity (-100) and
  deployment:persona (0) sections — they must survive a swapped loop
  plugin, so they leave dsh-agent-loop; the persona text is the plugin's
  own validated 'persona' config. The model/cwd variables STAY on the
  loop: runtime facts of the agents it drives.
- AgentOptions.systemPrompt is deleted with all its forwarding plumbing:
  the app configs' systemPrompt keys become 'persona' routed through
  dsh-agent-core (schema = z.intersect of the owners'), the ACP bridge
  and tool-subagent stop carrying persona configuration, and subagent
  children now render the deployment persona like every other agent.
- Example personas drop transport/interface trivia (ACP, CLI) — facts
  irrelevant to the model.
- Root CONTEXT.md removed (not idiomatic); its persona definition was
  wrong under the new ownership anyway.
- Docs, READMEs, the prompt-variables RFC, and generated catalogs
  updated; new loop test pins the assemble-waterfall escape valve
  (an emptied assembly sends NO system field).
2026-07-05 23:23:46 +08:00
Tianyi Cui
f256f3961d feat(system-prompt): prompt variables, persona-as-section, tool-guidance ownership
One principle: every fact in the assembled prompt has exactly one owner.

- dsh-system-prompt: merge-extensible AssembleContext on assemble();
  a variable(name, provider) registry; {{name}} interpolation in
  renderPrompt, strict (unknown/valueless/malformed references throw);
  duplicate section and variable names rejected; assembly carries
  resolved section text + variables through the assemble waterfall.
- dsh-agent declares AssembleContext.agent; dsh-agent-loop registers
  the agent:persona section (order 0 - identity renders before tool
  guidance) and the model/cwd variables, and drops its string join:
  renderPrompt(assembly) IS the full prompt.
- Tool guidance moves to its owners: descriptions carry per-tool
  semantics; sections only cross-call habits (tool:bash exit-code
  habit at order 105; read's not-shell nudge). todo/subagent need no
  section - their descriptions already carry the contract.
- SubagentProvider.inheritsParentContext (spawn/acp false, fork true);
  dsh-tool-subagent derives truthful per-provider wording and resolves
  the provider at load (backend must be listed first).
- Example personas shrink to identity + behavior with {{model}} (and
  {{cwd}} in the ACP tree); the welcome banner stops enumerating tools.

RFC: docs/rfc/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md
2026-07-05 01:54:46 +08:00
Tianyi Cui
456cb3b532 Merge remote-tracking branch 'origin/master' into simpl-a1-drop-image
# Conflicts:
#	docs/architecture.md
#	packages/compact/compact-basic/README.md
2026-07-04 21:12:33 +08:00
Tianyi Cui
774d460889 Expose audited hardcoded tunables as plugin config
The audit swept every packages/*/* plugin for the new AGENTS.md
convention (no hardcoded tunables in plugins) and exposes each finding
as a defaulted, validated Config field. Defaults are the previously
hardcoded values throughout, so no deployment or golden changes.

- tool-fs (had NO Config): readLimit, readMaxLineLength, readMaxBytes,
  readStreamMinSize. The caps thread through ReadToolCaps/ReadWindow —
  read-render already documented that the consumer applies the caps, so
  they become explicit per-request fields.
- tool-web: searchMaxResults (WEB_SEARCH_MAX_RESULTS stays as the
  schemastery default). Also fixes the stale GREP_LIMIT references in
  search.ts and the web-capability-seam RFC (no such constant exists).
- bash-local: graceMs (SIGTERM->SIGKILL escalation grace). The
  RunInternals.graceMs test seam is gone: graceMs is now a required
  SpawnSpec field filled from config, so tests exercise the real
  config path and the defaults live in exactly one place.
- subagent-acp: disposeEofGraceMs / disposeGraceMs. The AcpRunSpec
  fields become required for the same one-defaulting-layer reason.
- session-persistence-sqlite: journalMode ('wal' default; the
  rollback-journal modes serve filesystems where WAL's shared-memory
  files do not work, e.g. network mounts).
- hooks-claude + hooks-codex: stderrSummaryMaxChars for the persisted
  hook/result stderr summary. The duplicated summarize() helpers merge
  into hook-protocol's summarizeStderr(stderr, maxChars), beside the
  HookResultRecord field it feeds, with the bound parameterized the
  same way runHook's defaultTimeoutMs already is.
- compact-basic: charsPerToken for the token estimator (default 4, the
  English-text heuristic; CJK-heavy deployments need ~1-2 or compaction
  fires far too late). Also corrects the BasicCompactService class doc,
  which claimed defaults the required-field config never had.
- fs-local: deletes the dead STREAM_MIN_SIZE constant and the dead
  FsIoInternals.streamMinSize seam — the read-routing bound lives in
  the consumer (tool-fs), where it is now config. This is item 1 of
  the proposed prune-write-only-fs-surface RFC, annotated accordingly.

Every new field gets range validation (following the existing
assertPositiveFinite pattern), a README row, and tests covering the
configured behavior, the schema default, and load-time rejection.
2026-07-04 17:37:23 +08:00
Tianyi Cui
2745879132 refactor(llm): drop the image content block until a path can honor it
ImageBlock had no production producer and every consumer dropped it:
the deepseek serializer skipped it, the pi-ai converter skipped it as
unrepresentable, the ACP bridge neither advertises image prompt
capability nor forwards image blocks, and compact-basic charged a flat
85-token estimate and rendered an [image] placeholder. A block
constructed today would silently vanish from the wire — the vocabulary
advertised a capability no path honors, the silent-data-loss shape the
defensive patterns warn against. The only constructors were tests
pinning the skip/estimate branches.

Remove ImageBlock and its ContentBlockMap entry (its cache?: CacheHint
field leaves with it; CacheHint itself and the other two cache? fields
are out of scope). compact-basic loses its explicit image estimate and
placeholder arms (the merge-extensible default arms absorb the case);
the deepseek serializer, pi-ai converter, and ACP codec already handled
image in their default arms, so only their image-naming comments
change. The codec's inbound rejection of ACP-protocol image prompt
content stays — that guards wire content a client can send regardless
of our vocabulary.

Tests that constructed harness image blocks to pin the removed branches
are dropped (the 85-token estimate pin) or retargeted onto plugin-added
block types / other non-text blocks, which the surviving default arms
own. Docs, the type-equiv pastes, and the content-block vocabulary
RFC's block list and multimodal-home consequence are updated in the
same change; the RFC moves to implemented/ and the index is
regenerated. A real multimodal feature reintroduces image via
declaration merging together with the adapter mapping, ACP
advertisement, and compaction pricing that honor it.
2026-07-04 17:21:13 +08:00
Tianyi Cui
89f3c484c1 Merge branch 'worktree-hooks-a-taxonomy' into worktree-hooks-b-bash-seam
# Conflicts:
#	docs/rfc/README.md
2026-07-04 02:26:19 +08:00
Tianyi Cui
1a57d67058 refactor(tools): tagged render-intent union for tool-call presentation
Replace the "bag of optional fields" tool-presentation types
(ToolCallPresentation / ToolResultPresentation / ToolTerminal) with a
card-tagged discriminated union — the standing FIXME(tool-presentation).
A tool declares one render intent per call/result and the ACP bridge
switches on `card`:

  ToolCallView   = generic | terminal | diff
  ToolResultView = generic | terminal

The `diff` card is new: fs write/edit now emit an ACP {type:'diff'}
content block (an editor's inline diff), which the old shapes could not
express. The bridge also relativizes a file card's title against the
session cwd (mirroring claude-agent-acp's toDisplayPath) while keeping
locations/diff paths raw, and derives the no-capability fenced console
fallback from a terminal result's output. read gains the window-in-title
(`Read foo.txt (5 - 8)`) and an always-set location line, matching the
reference adapter field-for-field.

Migrates all three producer families (tool-fs, tool-bash, tool-todo) and
the sole consumer (the ACP bridge) together — the source does not compile
piecewise. Adds snapshot coverage for the terminal _meta path (a new
capability-advertising scenario) and re-records the fs goldens to show the
diff cards. Applied-hunk (result-time, context-line) diffs need a new
result/event shape and are a follow-up.

RFC: docs/rfc/implemented/architecture/2026-07-02-tool-render-intent-union.md
2026-07-03 02:04:03 +08:00
Tianyi Cui
5533bb783a docs(bash): purge remaining trusted-plugin wording caught in review
Codex review of the reframe found stale "trusted-plugin surface/boundary"
wording still in review-relevant spots the first pass missed:

- docs/rfc/README.md index title for the RFC.
- packages/bash/bash-local/src/run.ts (childEnv JSDoc + SpawnSpec stdin/env
  JSDoc + the spawn stdin comment) and src/index.ts (resolve carry-through
  comment); run.ts also pointed at a tool-bash README section name that no
  longer exists.
- the two bash-local test descriptors (run.spec.ts / executor.spec.ts).
- the tool-bash guard test's `boundary-*` call ids and one "boundary
  assertion" comment (renamed to `no-forward-*`).

All reworded to the scrub-is-the-control framing (or neutral wording). The RFC
FILENAME keeps `-trusted-plugin-surface` as a stable id (many links point at it;
the index title and content are corrected). No code or behavior change.
2026-07-02 04:30:40 +08:00
Tianyi Cui
bb9ae2ba99 docs(bash): reframe stdin/env — the scrub is the security control, not a trust boundary
Address review: the "trusted-plugin surface" framing overstated the security
story. A model driving the `bash` tool already has equivalent power to set env
vars and feed stdin through ordinary shell syntax (`FOO=bar cmd`, heredocs), so
the `env`/`stdin` seam fields grant it no new capability — and they cannot
exfiltrate the harness's ambient credentials, because the credential SCRUB in
dsh-bash-local (which strips *KEY*/*SECRET*/*TOKEN* from process.env before the
child sees it) is the actual control, and it works regardless of these fields
(tool-call args are static JSON, never shell-evaluated).

So drop the "dangerous / trusted-plugin boundary" language across the RFC, the
three bash-package READMEs, the bash/src/types.ts JSDoc, and docs/bash.md (both
the type-equiv blocks — kept 1:1 with source — and the prose). The reality that
remains: the `bash` tool doesn't EXPOSE env/stdin as parameters because they'd
be redundant with shell syntax; the fields exist for in-process plugins (the
hooks bridges) to pass a JSON payload + CLAUDE_* vars cleanly. The guard test is
kept but reframed: it catches a future `...args` spread that would silently
forward model input into the post-scrub env merge, NOT a trust wall. No code or
behavior change.
2026-07-02 04:08:24 +08:00
Tianyi Cui
13c6e847a2 feat(bash): add stdin + extra env to the executor seam as a trusted-plugin surface
The hooks subsystem runs external hook commands the Claude Code / Codex way:
JSON payload on stdin, context in CLAUDE_PROJECT_DIR / CLAUDE_PLUGIN_ROOT env.
Reusing the ctx.bash seam for that needs two new inputs — but stdin and arbitrary
env are exactly what dsh-bash-local's credential scrub exists to keep away from
model-driven commands. So this adds them as a TRUSTED-PLUGIN surface:

- BashExecRequest + BashExecSpec gain optional `stdin` and `env`. They are plain
  optionals on the resolved spec (not required-but-nullable like `owner`): a
  missing one means "none", the safe default, not a security footgun.
- dsh-bash-local threads them through resolve/run/start. `env` merges AFTER the
  credential scrub, so a trusted caller's explicit entry wins even on a
  credential-shaped name — the scrub guards the harness's OWN ambient creds from
  model-driven commands, not a trusted plugin. stdin is always a pipe, closed
  immediately (with bytes when supplied, empty otherwise — EOF as before); an
  EPIPE from a child that exits without reading is swallowed.
- The model-facing dsh-tool-bash NEVER forwards model input into stdin/env (its
  request is command/workdir/timeoutMs/signal/owner only). A regression guard
  drives the real tool with adversarial args and asserts the request carries
  neither field — proven to go red if the consumer ever forwards them.

Configurable scrub (in an earlier sketch) is dropped as speculative: the explicit
`env` field already gives a trusted caller full control, and no caller needs to
broaden the ambient scrub. Documented in a new architecture RFC, the bash.md
type-equiv blocks, and the three bash READMEs.
2026-06-30 13:52:25 +08:00
Yichen Jiang
01dea07922 Merge origin/master into codex/fix-stdio-readline-terminal 2026-06-23 16:27:49 +08:00
Tianyi Cui
b0422f2a50 fix review findings: bump session format version + restore late turn-end warn
Codex review of the trace-event fold found two merge-blockers.

Blocker #1 — format version. Folding usage onto assistant/message and removing
the standalone usage/error events changed the persisted SessionEventMap shape,
which per the AGENTS.md "bump the version and reject — don't migrate" policy
requires a backend to reject any non-current log. Centralize the version in an
exported SESSION_FORMAT_VERSION constant (dsh-session), read by both write sites
(Session constructor default, SessionStore.prepare header) and the coordinator's
load-time assertVersion check. The constant is pinned at 0: while unreleased the
on-disk format is unstable/pre-release, so breaking shape churn is absorbed at v0
(no monotonic bump until the first tagged release) and any non-0 log is rejected
on load — no migration. Update every test/fixture/doc that stamps a
currently-written header to the constant, bump the ACP snapshot fixture + golden
headers to v0, and keep the version-rejection test meaningful by switching its
bad value to a clearly non-current 99. AGENTS.md documents both the monotonic
(SQLite SCHEMA_VERSION) and pinned-0 (session log) pre-release stances.

Blocker #2 — restore the late turn-end warn. failTurn now sets the error reason
only while the turn is still open; once turn/end is appended (a throwing
agent/turn-end listener after closeTurn) the reason can no longer reach the
durable log, so the late throw is logged via ctx.logger.warn instead of
vanishing into a futile post-close assignment. A regression test asserts the
warn fires.

Also guard the normal-step assistant/message append with the same
content-or-usage condition as the max-tokens branch (a content-less, usage-less
step records no trace-only row), with a covering test.
2026-06-21 11:08:10 +08:00
Tianyi Cui
d6a2ab30c8 feat(types): brand bash ids + stop brand erosion; extract Branded to dsh-brand
Type-only change (brands are zero-cost casts; no runtime/wire impact). Closes
the two gaps in the "brand ids that cross package boundaries" policy and fixes
the dependency direction so a capability package never pulls in an unrelated one.

- Extract the `Branded<B>` primitive into a new standalone type-only package
  `@deepseek-ai/dsh-brand` (packages/util/brand) with no harness-package deps.
  dsh-llm keeps its owned CallId but imports Branded from dsh-brand; dsh-session,
  dsh-agent, and dsh-bash all import Branded from there. dsh-bash depends on
  dsh-brand ALONE — never on dsh-llm or dsh-session (the architectural fix: a
  generic execution backend must not couple to the LLM or session vocabulary).
- Mint BashTaskId + OwnerToken in dsh-bash and thread them through BashTask.id,
  the get/ownerOf/list/readOutput/kill seam, the bash-local generation site, and
  the dsh-tool-bash validate/access surface. OwnerToken is a DISTINCT brand from
  SessionId so the seam stays decoupled; dsh-tool-bash is the single boundary
  that casts SessionId -> OwnerToken.
- Brand at the SOURCE, not via mid-pipeline casts: agent-loop's Config types
  agents[].id as AgentId and resumeSessionId as SessionId, so the brand enters
  at the config boundary and the inner create()/resume casts disappear (only the
  genuinely-new per-run session-id string is cast).
- Stop brand erosion: propagate CallId/SessionId/AgentId to the registry/store
  Map keys and public params/exports (SessionStore, AgentRegistry + factory
  options, the ACP session-id surface + ToolPresenter CallId map, the
  persistence coordinator, invariants pendingCalls, the pi-ai tool-call maps).
- Docs: document BashTaskId/OwnerToken in bash.md (type-equiv re-pasted), point
  the Branded type-equiv at dsh-brand, fix stale param types in the session/
  agent/bash READMEs, regenerate the cordis catalog + module graph.

Implements docs/rfc/proposed/architecture/2026-06-20-branded-ids.md
2026-06-21 07:19:59 +08:00
Tianyi Cui
24168aee70 revert bash get()/list() removal — keep persistence-only prune
The original prune removed BashExecutor.get()/.list() too, but each is a
one-line accessor over the executor's already-tracked tasks map, and removing
them forced dsh-tool-bash's tests onto a ~35-line onTaskDone completion-tracking
harness just to replace the one-line ctx.bash.get(id) lookup. Per the AGENTS.md
"RFCs are proposals, not golden truth" principle, that disproportionate
migration cost is evidence the methods earn their keep — a test harness IS a
consumer programming against the seam.

Restore get()/list() (seam + LocalBashExecutor impl + the bash tests that used
them, dropping the doneFor/trackCompletions scaffolding). The persistence
has()/delete()/deleteStored removal stands — it had only contract-test callers
and no test-ergonomics cost. The RFC is retitled persistence-only with an
implementation note recording the bash revert.
2026-06-21 06:17:11 +08:00
Tianyi Cui
6ca8c3b99a fix review findings: stale get/list in proposed RFCs + doneFor double-await
Second Codex pass caught two proposed RFCs that describe the bash seam as it
WAS (with get/list) and would read stale once this prune lands, plus a latent
test-helper edge:
- docs/rfc/proposed/architecture/2026-06-20-branded-ids.md and
  2026-06-20-generic-long-running-tool-runtime.md: drop get/list from the
  BashExecutor seam description (surviving: resolve/run/start/ownerOf/
  readOutput/kill/onTaskDone). branded-ids will be further updated when it is
  implemented; this keeps it accurate in the meantime.
- trackCompletions now records every completion to `done` unconditionally (and
  also wakes a parked waiter), so a second doneFor(id) after completion resolves
  instead of hanging.
2026-06-21 03:00:07 +08:00
Tianyi Cui
5f9d10c587 fix review findings: stale seam docs + race-free doneFor test helper
Codex review of the PR2 diff caught doc/comment sites doc-sync does not gate
(core-data-structures prose) and a latent test-helper race:
- docs/core-data-structures/persistence.md + bash.md, sqlite README, and two
  source comments (coordinator.ts, jsonl.spec.ts) still listed the removed
  has/delete/get/list methods — updated to the surviving four-method
  persistence surface and the get/list-free bash seam.
- doneFor(ctx, id) attached its onTaskDone listener lazily, after the task
  could already have closed (e.g. `true`), so it could miss the completion and
  hang. Replaced with trackCompletions(ctx): one eagerly-installed listener
  (mounted in setup() before any task starts) records every completion, and
  doneFor resolves immediately for an already-finished task or on completion
  otherwise. Race-free, and there is no get-by-id seam left to poll instead.
2026-06-21 02:45:21 +08:00
Tianyi Cui
7792347c4f simplify(seams): prune dead methods from the persistence and bash seams
Two capability seams carried abstract methods no production consumer calls.
A method no consumer programs against is not a seam — it is speculative
surface every implementation must still provide and test.

- SessionPersistence: remove has() and delete(), the coordinator's
  has/delete/deleteCore, and the PersistenceBackend.deleteStored hook (with its
  jsonl + sqlite + in-spec memory-stub impls). Surviving service surface:
  create/append/load/list. Production uses only load() (resume) and list()
  (ACP session/list).
- BashExecutor: remove get(id) and list(), the abstract decls and the
  LocalBashExecutor impls. The internal tasks map survives (it backs
  ownerOf/readOutput/kill); get/list were pure public accessors over it with no
  shipping caller and no bash_list tool.
- Migrate tests that reached through ctx.bash.get(id) to the public completion
  seam: a doneFor(id) helper over onTaskDone awaits a task by id, and the
  HMR-reload ownership test now proves task survival through A's own bash_output
  ([status: running]) plus ownerOf + B-rejection — a stronger through-the-tool
  assertion than the removed lookup peek.
- Update seam READMEs (six -> four service methods, drop the deleteStored hook
  and the get/list row) and the two implemented persistence RFCs in place.

Implements docs/rfc/implemented/simplification/2026-06-20-prune-dead-seam-methods.md
2026-06-21 02:17:27 +08:00
Tianyi Cui
d02e9f1bd6 Reorganize packages into a modular hierarchy
Move the 18 flat packages/<name> packages into role-grouped dirs:
core/, llm/, bash/, session-persistence/, ui/, support/. Group dirs are
pure containers; each package keeps its @deepseek-ai/dsh-* name.

Collapse the per-package tsconfig paths maps (base + typecheck) into one
@deepseek-ai/dsh-* wildcard with a candidate per group, and derive the
publint list from the hierarchy. Update all depth-coupled globs/configs
(workspace, tsdown, vitest, eslint, knip, tsconfig includes/refs,
per-package tsconfigs, generators, doc-script scopes, type-equiv manifest)
and the cross-package/script relative imports in tests.

Fix doc-typecheck's workspacePaths() to parse tsconfig JSONC via the
TypeScript API instead of a regex comment-strip, which corrupted the
new wildcard `/*/` path candidates.

WIP: doc cross-links and package/RFC docs still to update.
2026-06-20 22:55:20 +08:00