Commit Graph

324 Commits

Author SHA1 Message Date
Tianyi Cui
c2f8af30da Merge branch 'feat/acp-1-max-tokens-turn-end' into feat/acp-2-bridge
# Conflicts:
#	AGENTS.md
#	docs/cookbook/extension-cookbook.md
#	yarn.lock
2026-06-16 23:40:23 +08:00
Tianyi Cui
840dfbab84 Merge remote-tracking branch 'origin/master' into worktree-md-wrap-guard
# Conflicts:
#	AGENTS.md
2026-06-16 23:38:58 +08:00
Tianyi Cui
ddf121320b Merge remote-tracking branch 'origin/split/session-persistence-sqlite' into feat/acp-1-max-tokens-turn-end
# Conflicts:
#	packages/session/src/types.ts
2026-06-16 23:31:14 +08:00
Tianyi Cui
53b37b39c5 docs: point AGENTS.md to development.md for TODO marker semantics 2026-06-16 23:02:48 +08:00
Tianyi Cui
67447fcdc3 feat: enforce merge-commit policy and markdown wrap convention
- AGENTS.md: require merging PRs with a merge commit (gh pr merge
  --merge), never squash/rebase — the per-PR commit history (review-fix
  and regression-test commits) is intentional record.
- Add scripts/verify-md-wrap.ts: a doc-sync gate that fails on
  hard-wrapped prose paragraphs (one physical line per paragraph), with
  smart exemptions for fenced code, tables, lists, blockquotes, headings,
  HTML comments, hrs, and reference defs. Scope covers README.md,
  docs/**/*.md, packages/*/README.md, plus AGENTS.md / packages/AGENTS.md
  (the files doc-sync did not previously cover). Folded into doc-sync so
  it rides the existing pre-push and CI gates.
- Sync AGENTS.md and docs/development.md doc-sync descriptions and
  command lists to include verify-md-wrap.
2026-06-16 22:14:25 +08:00
Tianyi Cui
4c8c1da8b3 feat: generate module dependency graph with freshness gate
Add scripts/gen-module-graph.ts, which derives the inter-package
dependency graph from each package's @deepseek-ai/dsh-* peerDependencies
and renders docs/module-graph.md (a GitHub-native Mermaid graph plus a
dependency table). Output is deterministic so a regenerate-and-diff
check is stable.

Wire a freshness gate the same way doc-sync is wired (ADR 0007: hooks
and CI run the same package.json scripts): verify-module-graph runs in
pre-push (lefthook) and as a CI step. It fails if the committed file
drifts from what the generator would produce.
2026-06-16 21:00:24 +08:00
Tianyi Cui
fb9636db44 feat(acp): ACP bridge — drive the coding agent from an editor over JSON-RPC stdio
Implements the RFC 010 MVP: a new `@deepseek-ai/dsh-acp` package bridges the
harness agent to the Agent Client Protocol (JSON-RPC 2.0 over newline-delimited
stdio), so Zed and other ACP editors can drive the coding agent — streaming
render, tool-call display, and resumable sessions via `session/load`.

- packages/acp: AgentSideConnection wiring; initialize/newSession/loadSession/
  prompt/cancel; a total TurnEndReason→StopReason codec; settle-once with a
  fallback chain (agent/turn-end → logged turn/end → idle); single-session
  guard; cwd-must-equal-launch-dir validation; load replays from the persisted
  event log (assistant/chunk→agent_message_chunk, tool/call/result→tool_call*).
- agent: add Agent.whenIdle() quiescence signal to the interface; LoopAgent
  implements it (resolves on the first running→idle/disposed transition). The
  bridge awaits it on disposal so teardown reaches quiescence, not just abort.
- examples: extract the shared provider/tool core into examples/base.yml;
  coding-agent nest-includes it; new examples/acp-agent serves the agent over
  ACP with JSONL persistence and no stdout logger (stdout is the protocol).
- Permission gate deferred (TODO(rfc010-permission-gate)): tools run with the
  executor's full authority; only the Agent→sessionId ownership seam is laid
  down. Cancel is best-effort for a not-yet-started queued turn
  (TODO(rfc010-cancel-prestep)). RFC 010 stays `proposed`.
- Docs: package README + Zed snippet; client-driver cookbook section; root and
  packages layout/commands; RFC 010 implementation-status note.

48 bridge tests + whenIdle coverage; 100% per-file coverage; e2e boots the
example as a subprocess and verifies a written file on disk (key-gated, with a
no-key stdout-purity check).
2026-06-16 18:44:31 +08:00
07akioni
dabc2ff411 feat: migrate to pnpm 2026-06-16 14:55:37 +08:00
Tianyi Cui
add59a3336 feat(agent-loop): surface max-tokens as a distinct turn-end reason
Add a `max-tokens` variant to `TurnEndReasonMap` and carry the model
finish reason up from `runStep` to `runTurn`, applying the rule "any
max-tokens step in the turn surfaces as max-tokens" (disposed/aborted/
error still take precedence). This lets consumers distinguish a clean
stop from a truncated one — the contract RFC 010's ACP bridge maps to
the `max_tokens` stop reason.

Also add an AGENTS.md rule: write an ADR when (and only when) a PR makes
a durable, contested, surprising decision.
2026-06-16 11:27:19 +08:00
Tianyi Cui
b8d4790e98 docs: sync high-authority docs with the doc-sync gate and dsh-invariants (P1-16)
The `yarn doc-sync` gate (doc-typecheck + verify-event-taxonomy) and the
@deepseek-ai/dsh-invariants package both exist now, but the instruction docs
never caught up and the gate's markdown scope (README.md, docs/**/*.md,
packages/*/README.md) does not cover AGENTS.md / packages/AGENTS.md, so they
drifted silently.

- AGENTS.md: add invariants/ to the Repository Layout; add doc-typecheck /
  verify-event-taxonomy / doc-sync to Commands; rewrite the false "CI has no
  doc-sync gate" sentence to describe the gate's actual coverage and what
  remains outside it (AGENTS.md, packages/README.md, prose drift).
- packages/AGENTS.md: fix the same stale "no doc-sync gate" line.
- packages/README.md: add dsh-invariants to the dependency graph and the
  package table.

Verification: `yarn doc-sync` green; `grep -rn "no doc-sync gate"` returns
nothing; the three command names + dsh-invariants are present.
2026-06-14 23:07:52 +08:00
Tianyi Cui
39b3db4b9c docs: accuracy sweep, architecture restructure, two ADRs, review skill
- AGENTS.md Commands: fix typecheck/build descriptions; add lint, lint:fix,
  test:coverage, knip, publint, hygiene (were undocumented).
- Drop the bare `yarn demo` for explicit `demo:echo` + `demo:coding`; update
  README, examples READMEs (and document coding-agent in examples/README).
- New cookbook guide: adding-a-vendored-package.md (the missing "add" half of
  vendor/README's update-only procedure).
- architecture.md: add a table-of-contents and extract the Extension cookbook
  to docs/cookbook/extension-cookbook.md (link-preserving); drop the completed
  "restructure this document" TODO.
- ADR 0009 (capability seams) + 0010 (twin LLM adapters), and a "when to write
  an ADR" standard in adr/README.
- Add a committed dsh-code-review skill under .agents/skills, exposed to Claude
  Code via a tracked .claude/skills symlink (gitignore carve-out).
2026-06-13 22:05:34 +08:00
Tianyi Cui
066f94c7e0 docs: unwrap hard-wrapped Markdown to one line per paragraph
Hard line breaks mid-paragraph make docs harder to edit and diff — a
one-word change reflows and re-diffs the whole paragraph. Reflow all
tracked non-vendor Markdown (plus vendor/AGENTS.md) so each prose
paragraph is a single line; soft-wrapping is the editor's job. Fenced
code, tables, and list structure are preserved (wrapped list items fold
to one line per bullet). Documents the convention in AGENTS.md.
2026-06-13 20:27:04 +08:00
Tianyi Cui
e98c1c5d42 Add examples/coding-agent and the docs cookbook
The first real agent wiring: DeepSeek V4 + the bash tool suite + stdio
chat + JSONL persistence, runnable via yarn demo:coding (reads the
gitignored repo-root .env through process.loadEnvFile).

- examples/coding-agent: cordis.yml wiring both real plugin families
  (llm-deepseek with !!js env secrets; bash-local + tool-bash), a
  bash-only coding system prompt, a max-steps-guard plugin (bounds
  runaway turns via the agent/turn-continuation waterfall — abort()
  from step-end is a no-op by then), and a stdio UI with dimmed
  reasoning and exit-on-idle for piped stdin.
- e2e (yarn test:e2e, key-gated): full-loop.e2e.ts runs a real model
  against the real bash tool; coding-task.e2e.ts is the swebench-style
  smoke — the model fixes a buggy add.js in a temp dir and the test
  re-runs node add.test.js itself rather than trusting the agent.
- docs/cookbook: adding-a-package (the verified checklist),
  adding-a-tool (execute() contract, background pattern, seams),
  adding-an-llm-adapter (protocol obligations, mock-server testing,
  e2e policy). AGENTS.md layout/commands/secrets sections updated;
  architecture.md points at both examples and the cookbook.
- vitest.e2e.config.ts: serialize test files + retry twice — parallel
  e2e files trip the shared internal key's concurrency quota.
- fix: the !js YAML tag spelling in docs/JSDoc is actually !!js
  (js-yaml resolves custom tags under tag:yaml.org,2002:js).
2026-06-13 18:30:50 +08:00
Tianyi Cui
ab19fed77c Add two DeepSeek LLM adapters: dsh-llm-deepseek and dsh-llm-pi-ai
The first real LlmAdapter implementations, shipped as a deliberate pair:
same models and wire protocol, completely different internals, so the
StreamChunk protocol is verified across independent implementations.

- dsh-llm-deepseek: hand-rolled fetch + SSE parser + chunk-translation
  state machine against the official chat-completions format (thinking
  mode via top-level thinking/reasoning_effort; the empty-string
  reasoning_content first chunk; usage attached to the finish chunk or
  trailing; reasoning_content passback on tool-call turns; disjoint
  cache-token accounting).
- dsh-llm-pi-ai: the same endpoint through @earendil-works/pi-ai,
  mapping its event vocabulary (parsed tool arguments, in-stream error
  events, folded reasoning tokens) onto the same chunks.

The agent loop now honors the in-band error path: an adapter that ends
its stream with finish {kind:error|aborted} (the only option for
adapters that can't throw mid-stream, like pi-ai) is translated into a
step error, so the turn ends error/aborted with a logged error event
instead of a normal completed assistant message. This makes the
StreamChunk error contract real for both adapters; docs/architecture.md
and the StreamChunk doc are updated accordingly.

New yarn test:e2e (vitest.e2e.config.ts, *.e2e.ts) runs key-gated
real-API matrices for both adapters across V4 Flash/Pro and all
thinking/effort levels; it self-skips without DEEPSEEK_API_KEY. Unit
suites run against local node:http mock SSE servers at 100% per-file
coverage.
2026-06-13 18:30:03 +08:00
Tianyi Cui
8b5a3ef730 Add bash execution: dsh-bash seam, dsh-bash-local impl, dsh-tool-bash tools
Three packages following the new capability-seam pattern (interface /
implementation / consumer, now documented in docs/architecture.md):

- dsh-bash: abstract BashExecutor service (ctx.bash) + vocabulary types.
- dsh-bash-local: local subprocesses — bash -c per call in a detached
  process group, SIGTERM→SIGKILL group kills, tail-keep truncation with
  full-stream spill files, model-friendly env, background task registry.
- dsh-tool-bash: the bash / bash_output / bash_kill tool schemas with
  runtime arg validation and background completion notices via
  agent.inject(). Non-zero exits are reported, not errored.

Design surveyed against the bash tools of Claude Code, OpenCode, Codex,
and pi (notes in the package READMEs). Permissions/sandbox stay TODO on
the tools/execute waterfall seam; stateful-shell alternatives recorded
in run.ts.
2026-06-13 18:28:10 +08:00
Tianyi Cui
fa91bbac54 Fix CI: order steps by artifact dependency, resolve fresh-clone builds
CI failed at Lint with 1519 no-unsafe-* errors on every cross-package
import. Three fresh-checkout issues, invisible locally because lib/
persists between runs:

- Lint ran before Typecheck, but the type-aware ESLint config resolves
  vendor packages via their built declarations (tsconfig.typecheck.json
  -> vendor/*/lib), which Typecheck emits. Reordered.
- The first-ever tsc -b resolved sibling vendor plugins through their
  package.json types (lib/index.d.ts, not yet emitted) — TS2307 until a
  second run. The source-level paths map moves from the root
  tsconfig.json (dev-only, not inherited by package builds) into
  tsconfig.base.json so the whole build graph resolves source-first;
  tsconfig.typecheck.json still overrides wholesale to lib resolution.
- Hygiene ran publint (validates packed lib/index.js bundles) before
  Build emitted them. Reordered.

Also: checkout/setup-node bumped v4 -> v6 (node20 runners are
force-switched to node24 on 2026-06-16), the Build step name catches up
with tsdown, and AGENTS.md documents the one case where a fresh clone
needs `yarn typecheck` before `yarn lint`.
2026-06-11 23:10:43 +08:00
Tianyi Cui
630bbddf9a Replace dumble with tsdown for JS bundling
dumble (0.2.x, ~530 dl/wk, single-maintainer) was a bus-factor risk as
the load-bearing bundler. tsdown (rolldown-based, ~2.5M dl/wk, actively
maintained) replaces it while output stays list-identical, verified by
snapshot diff: 17 JS bundles, externals preserved, schemastery dual
.mjs/.cjs and logger-console node+browser entries intact.

Root tsdown.config.ts uses workspace globs ['vendor/*', 'packages/*']
(explicit, so examples/* stays excluded); two per-package overrides in
vendor/ cover the special shapes and are logged in vendor/README.md as
ours (not upstream sync surface). scripts/build.ts (dumble
orchestration) is deleted; yarn build = tsc -b && tsdown. tsc -b keeps
owning declarations (dts: false, clean: false).

Rationale recorded in ADR 0008 (also covers the direct-esbuild and
pkgroll alternatives).

Gates: lint, typecheck, 134 tests, hygiene (knip/publint/constraints),
demo smoke all green.
2026-06-11 22:34:43 +08:00
Tianyi Cui
4dafad4db6 Add RFCs for the remaining quality-proposal ideas
Eight proposals grouped by category, each with problem statement,
concrete plan, and risks: property-based testing over the
protocol-shaped core (chunk streams, event logs, schema DSL);
mutation testing as the counterweight to the 100%-coverage gate;
deterministic tests + a universal replay-invariant fixture + nightly
race stress; architectural conformance (dependency-cruiser rules and
the LlmAdapter conformance kit); runtime arg validation at the model
boundary with a structured error taxonomy and dev-mode invariants;
doc-sync enforcement (typechecked doc snippets, API reports);
supply-chain checks and nightly vendor-drift verification against the
manifest; and deep-readonly public surfaces (logged-vs-in-flight
mutability boundary). AGENTS.md points at docs/adr and docs/rfc.
2026-06-11 15:27:07 +08:00
Tianyi Cui
370b5d3aab Add assertNever with closed-vs-extensible exhaustiveness guidance
assertNever (dsh-llm) marks unreachable defaults on CLOSED unions:
adding a StreamChunk variant now breaks compilation at
BlockAssembler.push, and a value escaping its type at runtime throws
with diagnostics. The module doc and a new AGENTS.md convention spell
out the dividing line: merge-extensible unions (SessionEventMap,
ContentBlockMap, …) must NOT use assertNever — plugin-added variants
are valid unknown values there; handle known cases and fall through
with a comment.
2026-06-11 15:21:25 +08:00
Tianyi Cui
7f024a1a9d Document the codebase thoroughly and tighten type safety
Docs: per-folder README.md for packages/ (family overview + one per
package: service, events, API, extension points, TODOs), examples/,
and examples/echo-agent/; folder-level AGENTS.md (+ CLAUDE.md
symlinks) for packages/ and vendor/; module-level doc comments in
every packages/*/src file; richer JSDoc on all exported API
(event side effects, disposal contracts, error behavior). Root
AGENTS.md gains a "Type Safety and Documentation" policy section:
the codebase aims to be very type-safe and well documented; type
gymnastics are acceptable in core packages when they improve
plugin-author DX; verbose docs are fine as long as they stay strictly
in sync with the code.

Type safety: removed the upstream-inherited "noImplicitAny": false
from tsconfig.base.json — packages/* now compile under full strict
mode; vendor/loader and vendor/include set it locally (vendor/cordis
already did). Eliminated every `: any` / `as any` from packages and
examples (catch clauses use unknown + a CodedError narrowing type;
event data access uses discriminated-union narrowing).

Typed tool schemas: new @deepseek-ai/dsh-tools schema DSL —
SchemaSpec with per-property `required: true` booleans, type-level
InferArgs<S>, a runtime SchemaSpec → JSON Schema converter, and
defineTool() so first-party tools get typed execute(args) with zero
casts (raw JSON Schema still accepted for MCP interop; chosen over
schemastery because it targets JSON Schema generation directly).
echo-tool and all test tools migrated; +7 tests.
2026-06-11 13:01:00 +08:00
Tianyi Cui
217b8ec0e2 Fix architecture-review findings in the loop and service packages
High (loop pipeline): agent/step-result now runs before the
assistant/message append so the session log records what tool dispatch
actually uses; abort is honored between tool calls, not just
mid-stream; steering drains at step start, pending steering overrides
a negative turn-continuation decision (/goal pattern), and leftover
steering is re-enqueued as queued messages so it is never stranded;
exceptions from turn-continuation listeners and session/flush are
contained to the turn (error event + agent/error) instead of killing
the driver loop.

Medium: disposal emits agent/status('disposed') and mid-turn disposal
records reason 'disposed'; duplicate LLM adapter registration throws
(all-or-nothing); SessionEvent is a real discriminated union (casts
removed); model-less agents fail with a clear actionable error unless
agent/request supplies a model.

Low: agent/queued and agent/steering carry the resolved MessageSource;
streamBlocks() yields strictly in stream order and flushes delta-only
blocks (matches generate()); BlockAssembler freezes blocks on
block-end and ignores stragglers from malformed streams; turn
numbering is a counter seeded from the log (fork-safe); LoopAgent's
stop disposer is infallible (a throwing status listener cannot skip
registry cleanup); AgentLoop.create uses a generator effect so stop
and unregister are independent disposables; SessionStore wires
onAppend inside its effect.

21 regression tests added (review-fixes.spec.ts), organized by
finding. Docs updated: loop pseudocode (status emissions, ordering,
error containment, steering guarantees) and waterfall composition
caveat in docs/architecture.md; AGENTS.md notes that excessive tests
are welcome.
2026-06-11 12:19:16 +08:00
Tianyi Cui
cacfae3cae Document the architecture and rewrite AGENTS.md
docs/architecture.md: layering, service map, event taxonomy, the
session/turn/step lifecycle, Cordis waterfall semantics, an extension
cookbook, the plugin sanity checklist mapping every MVP feature to its
extension mechanism, and the deferred-work TODO list (sub-agents,
persistence backends, compaction, DeepSeek V4 adapter, parallel tool
execution, streaming-protocol review).

AGENTS.md: repo layout, commands, conventions (dsh-* naming, ESM,
effect-based registrations, declaration merging, waterfall semantics),
and the vendoring policy pointer.
2026-06-11 10:55:05 +08:00
Tianyi Cui
804eede9eb Link MVP requirement analysis and microkernel architecture docs in AGENTS.md 2026-06-10 23:12:38 +08:00
Tianyi Cui
b67e81ac97 Initialize repo with README, AGENTS.md, and CLAUDE.md symlink 2026-06-10 22:58:56 +08:00