mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
Merge pull request #63 from deepseek-ai/worktree-extract-examples-logic
refactor(examples): extract reusable logic into tested packages (ui-stdio, llm-replay)
This commit is contained in:
Tianyi Cui
GitHub
1
.gitignore
vendored
1
.gitignore
vendored
@@ -10,5 +10,6 @@ examples/*/*.jsonl
|
||||
examples/*/.sessions/
|
||||
coverage/
|
||||
.doc-typecheck-*/
|
||||
.humanize/
|
||||
.vscode/
|
||||
.DS_Store
|
||||
|
||||
14
AGENTS.md
14
AGENTS.md
@@ -2,6 +2,10 @@
|
||||
|
||||
This is the monorepo for the DeepSeek Harness group. It currently hosts the code for **DeepSeek Code**, DeepSeek's coding agent product.
|
||||
|
||||
## Pre-release stance: foundation over blast radius
|
||||
|
||||
**This applies only while the harness is unreleased — remove this section at the first tagged/published release.** There are no external consumers yet, so optimize for the *correct foundation*, not for a small diff. When the right structure means moving a file across package boundaries, renaming a public symbol, or repackaging a plugin, do it — and update every reference in the same change. Do **not** add backward-compat shims, deprecation aliases, re-export stubs, or "keep it where it is to avoid churn" hedges; those are debts you take on to protect callers you do not have. Churn now is cheap; a wrong foundation set in stone is not. (Once released, this inverts — backward compatibility becomes a real constraint and this section comes out.)
|
||||
|
||||
## Architecture
|
||||
|
||||
This codebase is based on the **Cordis** framework, built microkernel-style: **everything is a plugin**. All necessary Cordis dependencies are copied into this monorepo as vendored source (under `vendor/`) instead of being depended on via npm.
|
||||
@@ -35,9 +39,13 @@ packages/ Harness packages, all named @deepseek-ai/dsh-<name>:
|
||||
tool-bash/ model-facing bash/bash_output/bash_kill tool schemas
|
||||
acp/ Agent Client Protocol bridge: drive the agent from an ACP
|
||||
editor (Zed) over JSON-RPC stdio
|
||||
examples/ Runnable demos (not workspaces). echo-agent = mock model + echo
|
||||
tool + stdio UI + JSONL persistence, wired via cordis.yml.
|
||||
coding-agent = the real thing: DeepSeek V4 + bash tools
|
||||
ui-stdio/ minimal stdio (readline) UI plugin: renders agent/* events,
|
||||
feeds stdin lines to the agent (shared by the demos)
|
||||
llm-replay/ record/replay adapter: short-circuits llm/stream from a
|
||||
recorded session JSONL (keyless snapshot tests)
|
||||
examples/ Runnable demos (not workspaces; see examples/AGENTS.md). echo-agent
|
||||
= mock model + echo tool + stdio UI + JSONL persistence, wired via
|
||||
cordis.yml. coding-agent = the real thing: DeepSeek V4 + bash tools
|
||||
(pnpm run demo:coding, needs DEEPSEEK_API_KEY).
|
||||
acp-agent = the coding agent exposed as an ACP server over
|
||||
JSON-RPC stdio (pnpm run demo:acp, needs DEEPSEEK_API_KEY).
|
||||
|
||||
@@ -14,6 +14,8 @@ graph TD
|
||||
system-prompt --> llm
|
||||
agent --> llm
|
||||
agent --> session
|
||||
llm-replay --> llm
|
||||
llm-replay --> session
|
||||
session-persistence --> session
|
||||
invariants --> agent
|
||||
invariants --> llm
|
||||
@@ -25,6 +27,9 @@ graph TD
|
||||
tools --> agent
|
||||
tools --> llm
|
||||
tools --> system-prompt
|
||||
ui-stdio --> agent
|
||||
ui-stdio --> llm
|
||||
ui-stdio --> session
|
||||
acp --> agent
|
||||
acp --> llm
|
||||
acp --> session
|
||||
@@ -52,11 +57,13 @@ graph TD
|
||||
| `session` | `llm` |
|
||||
| `system-prompt` | `llm` |
|
||||
| `agent` | `llm`, `session` |
|
||||
| `llm-replay` | `llm`, `session` |
|
||||
| `session-persistence` | `session` |
|
||||
| `invariants` | `agent`, `llm`, `session` |
|
||||
| `session-persistence-jsonl` | `session`, `session-persistence` |
|
||||
| `session-persistence-sqlite` | `session`, `session-persistence` |
|
||||
| `tools` | `agent`, `llm`, `system-prompt` |
|
||||
| `ui-stdio` | `agent`, `llm`, `session` |
|
||||
| `acp` | `agent`, `llm`, `session`, `session-persistence`, `tools` |
|
||||
| `agent-loop` | `agent`, `llm`, `session`, `session-persistence`, `system-prompt`, `tools` |
|
||||
| `tool-bash` | `agent`, `bash`, `llm`, `tools` |
|
||||
|
||||
@@ -7,7 +7,7 @@ One kind of design doc lives here. An **RFC** records a decision or proposal tha
|
||||
Files are grouped by lifecycle into three folders, and an RFC moves between them as its status changes:
|
||||
|
||||
- **`proposed/`** — proposals reviewed before implementation; not yet built (or only partly).
|
||||
- **`implemented/`** — the decision shipped. The file records what was decided and what was rejected.
|
||||
- **`implemented/`** — the decision shipped. The file records what was decided and what was rejected, and is **kept current with what actually shipped**: when the code later moves a file, renames a package, or changes a key/default, the RFC is updated in the same change to match (facts only — paths, names, structure — not the decision itself). See [implemented/AGENTS.md](implemented/AGENTS.md).
|
||||
- **`rejected/`** — the proposal was considered and declined. Kept for the record so the rejection isn't re-litigated.
|
||||
|
||||
Each file is named `yyyy-mm-dd-topic-title.md`, where the date is when the topic was **first proposed** (per git history). Cross-references between RFCs use relative markdown links (`[topic](../implemented/2026-…-….md)`) — never bare prose or numbers — so they are mechanically checkable and survive moves between folders.
|
||||
@@ -16,7 +16,7 @@ Each file is named `yyyy-mm-dd-topic-title.md`, where the date is when the topic
|
||||
|
||||
Write an RFC when a decision is **durable** (it shapes the codebase beyond a single function or package), **contested** (there was a real alternative a reasonable engineer might have chosen), and **surprising** (a future reader would otherwise ask "why on earth is it done this way?"). A proposal for substantial future work starts in `proposed/`; a decision already made starts in `implemented/`.
|
||||
|
||||
Do NOT write one for a mechanical or local choice (a variable name, a one-file refactor), for anything already enforced and explained by a gate or a convention in AGENTS.md, or for a still-provisional decision tagged `TODO(...)` in the code — record those as TODOs and promote to an RFC only once they settle. An RFC is never edited into a different decision: supersede it with a new one and cross-link.
|
||||
Do NOT write one for a mechanical or local choice (a variable name, a one-file refactor), for anything already enforced and explained by a gate or a convention in AGENTS.md, or for a still-provisional decision tagged `TODO(...)` in the code — record those as TODOs and promote to an RFC only once they settle. An RFC is never edited into a *different decision*: supersede it with a new one and cross-link. (Editing an `implemented/` RFC to track where its already-made decision now *lives* — a moved file, a renamed package — is not a different decision and is required, not forbidden; see [implemented/AGENTS.md](implemented/AGENTS.md).)
|
||||
|
||||
## Proposed
|
||||
|
||||
|
||||
@@ -64,9 +64,9 @@ Both surfaces contain non-deterministic values that a pure normalization functio
|
||||
|
||||
Determinism of the tool environment comes from a per-test `mkdtemp` cwd, the executor's existing secret-scrubbing env (`/KEY|SECRET|TOKEN/i`), the fresh non-login `bash -c` per call, and the normalization pass — **not** from an OS sandbox. A real rootless sandbox (bwrap on Linux, sandbox-exec/Seatbelt on macOS) is the established cross-platform pattern (Claude Code, Codex), but it is per-OS, fragile on newer kernels (Ubuntu 24.04+ AppArmor blocks unprivileged user namespaces), and unnecessary for transcript determinism. It is reserved as a future tier via the documented `BashExecutor` capability seam ([a sandboxing executor replaces dsh-bash-local without touching a tool schema](2026-06-13-capability-seams.md)) — a new `bash-*` package, not a change here. Scenarios keep bash commands tightly constrained (no `date`/`env`/background/large-output) so the temp-dir tier suffices.
|
||||
|
||||
### Example-local plugin, not a new package
|
||||
### The replay plugin is its own package
|
||||
|
||||
The replay plugin lives at `examples/acp-agent/src/llm-replay.ts`, referenced from the snapshot config by relative path — exactly how echo-agent wires its [mock-llm.ts](../../../examples/echo-agent/src/mock-llm.ts). It is test/example infrastructure with one consumer; the capability-seams rule says not to split into a published `packages/` trio preemptively. It is promoted to a package only when a second example needs it.
|
||||
The replay plugin lives in its own package, `@deepseek-ai/dsh-llm-replay` (`packages/llm-replay/`), and the snapshot config references it by package name. It is the keyless replacement for the real LLM adapter: it installs an `llm/stream` waterfall listener and short-circuits it, serving model streams reconstructed from a recorded session JSONL. Its sole consumer is the ACP snapshot harness here, but it is a package (not example-local glue like echo-agent's [mock-llm.ts](../../../examples/echo-agent/src/mock-llm.ts)) so that its derive/parse/replay branches fall under the per-file 100% coverage gate on package `src` trees — logic under `examples/` is not measured by that gate, which would leave those branches unguarded.
|
||||
|
||||
### Two subcommands, replay in the default gate
|
||||
|
||||
|
||||
15
docs/rfc/implemented/AGENTS.md
Normal file
15
docs/rfc/implemented/AGENTS.md
Normal file
@@ -0,0 +1,15 @@
|
||||
# AGENTS.md — Implemented RFCs
|
||||
|
||||
These are RFCs whose decision has **shipped**. The repo-wide and docs-wide rules still apply ([root AGENTS.md](../../../AGENTS.md) § "Type Safety and Documentation", [docs/AGENTS.md](../../AGENTS.md)); this file adds one rule specific to this folder.
|
||||
|
||||
## Keep an implemented RFC current with what actually shipped
|
||||
|
||||
An RFC in `implemented/` describes a decision that is now **live code**. Keep its description of the shipped reality accurate: when the implementation later moves a file, renames a package or symbol, changes a config key/default/error code, or relocates a plugin, update the RFC in the **same change** that touches the code — exactly as you would a package README. A stale implemented RFC (pointing at a path that no longer exists, naming a package that was renamed, describing a structure that was refactored) is worse than no RFC: a future reader trusts it and is misled.
|
||||
|
||||
Update it **in place** to state the current truth. Do **not** leave the outdated text in and bolt on a "superseded / now actually…" note — that makes the document a changelog of its own drift and forces the reader to reconstruct the present from a pile of corrections. Write what is true now.
|
||||
|
||||
### This is not a license to rewrite the *decision*
|
||||
|
||||
Keeping the shipped-state description current is about **facts** (paths, names, structure, defaults) — not about silently flipping the **decision and its rationale** into a different one. If the underlying choice itself is reversed or materially changed (not just relocated), that is a new decision: write a new RFC and cross-link, per [rfc/README.md](../README.md) ("An RFC is never edited into a different decision"). The line: a refactor that moves where the decision is *realized* → edit this RFC to match; a reversal of *what was decided* → a new RFC.
|
||||
|
||||
When in doubt, ask whether a reader following this RFC to the code would land on something real. If not, it needs updating.
|
||||
1
docs/rfc/implemented/CLAUDE.md
Symbolic link
1
docs/rfc/implemented/CLAUDE.md
Symbolic link
@@ -0,0 +1 @@
|
||||
AGENTS.md
|
||||
26
examples/AGENTS.md
Normal file
26
examples/AGENTS.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# AGENTS.md — Examples
|
||||
|
||||
Runnable demos that show how the harness is wired. **Examples are NOT workspaces** — each `examples/*/package.json` is a private, dependency-free stub with no build. They are booted as unbuilt `tsx` subprocesses via the cordis Loader reading a `cordis.yml`; the `@deepseek-ai/dsh-*` plugin names in those YAML files resolve through the root `tsconfig.json` `paths` map, not through `node_modules`.
|
||||
|
||||
Because examples are not under the `packages/*/src` coverage gate, an example that grows real, reusable *logic* should extract it into a `packages/` package (where it gets the per-file 100% gate and a README). Keep only example-specific glue here: `start.ts`, the `cordis.yml` wiring, demo-only mocks/teaching artifacts, and the e2e/snapshot scenarios.
|
||||
|
||||
## Every example ships e2e smokes (keyless + with-key)
|
||||
|
||||
Each example must have **both** kinds of end-to-end smoke, because they catch different failures:
|
||||
|
||||
- **Keyless smoke** — boot the example through its real `cordis.yml` via the Loader (no API key), drive it, and assert the rendered output and a clean exit. This is the guard a hand-mounted unit test structurally cannot be: it exercises the REAL load path (`unwrapExports`, `inject`, the whole plugin tree), so a broken plugin export shape — e.g. a stray `export default` that collapses a namespace plugin and drops `inject` — fails here even when unit tests stay green (see [docs/postmortem/0001](../docs/postmortem/0001-acp-default-export-drops-inject.md)). It runs in the default e2e gate (CI has no secrets).
|
||||
- **With-key smoke** — send a real prompt against the live model and verify the WORLD (a file on disk, a non-empty assistant turn), not the agent's self-report. This proves the actual product works, which a mock/keyless run structurally cannot. Key-gated: it self-skips without `DEEPSEEK_API_KEY` (see [the with-key policy](../AGENTS.md#secrets--env) — inference is cheap here, so write many).
|
||||
|
||||
**Exception — keyless-by-nature examples.** An example whose model is itself a mock/deterministic stand-in (no real provider) has no meaningful with-key smoke; the keyless smoke is the complete requirement. State the exception inline in the test.
|
||||
|
||||
A keyless smoke that spawns the example from a temp cwd must set `TSX_TSCONFIG_PATH` to the repo-root tsconfig — the unbuilt `paths` map is found by searching UP from cwd, so a temp cwd outside the repo would otherwise fall back to stale built `lib/`. Pass `--expose-internals` when the example's `cordis.yml` loads the HMR plugin (mirror the `demo:*` script).
|
||||
|
||||
## Current state
|
||||
|
||||
| Example | Keyless smoke | With-key smoke |
|
||||
|---|---|---|
|
||||
| `echo-agent` | `tests/echo.e2e.ts` — boots the real `cordis.yml`, drives the echo tool round-trip and the direct canned reply | **N/A — keyless by nature** (the `mock-echo` model has no real provider) |
|
||||
| `coding-agent` | `tests/keyless-smoke.e2e.ts` — boots the full real tree (dummy key, no prompt → no model call), asserts banner + clean exit | `tests/{full-loop,coding-task,resume}.e2e.ts` — real model + real bash, world-verified |
|
||||
| `acp-agent` | `pnpm run test:snapshot` — boots the real ACP subprocess and replays a recorded session keyless; `tests/acp.e2e.ts` also asserts stdout purity without a key | `tests/acp.e2e.ts` — real ACP prompt, verifies a file the agent wrote |
|
||||
|
||||
See [the root AGENTS.md](../AGENTS.md) for repo-wide conventions and [docs/architecture.md](../docs/architecture.md) for the design.
|
||||
1
examples/CLAUDE.md
Symbolic link
1
examples/CLAUDE.md
Symbolic link
@@ -0,0 +1 @@
|
||||
AGENTS.md
|
||||
@@ -32,7 +32,7 @@ The editor sets each session's `cwd` to the project it opens; the agent's bash t
|
||||
|
||||
## Snapshot tests (record-once / replay-deterministic)
|
||||
|
||||
This example is the home of the harness's **snapshot tests** — they boot this server as a real subprocess, drive it with a deterministic input script, and diff its normalized output against committed golden files. The model is made deterministic by `src/llm-replay.ts`, a function/namespace plugin that installs an `llm/stream` waterfall listener and short-circuits it, serving model streams reconstructed from a recorded **session JSONL** fixture (`<scenario>/session.jsonl`) — so replay needs no API key. The fixture IS the persisted session log: its `assistant/chunk` events carry every `StreamChunk`, so grouping them by `(turn, step)` reconstructs each `stream()` call (one model call per loop step). Recording is therefore "run the real agent once and harvest the `.jsonl`". The two failure modes not expressible as logged chunks — a pure throw before any chunk, and cancel/hang — use an optional `<scenario>/replay.override.json` sidecar (a `ReplayEntry[]` that replaces the derived script). A scenario that needs the agent to operate on existing files ships an optional `<scenario>/workspace/` directory — the harness copies its contents into the temp cwd before the run (see `workspace-edit`). See [docs/rfc/implemented/2026-06-19-acp-snapshot-tests.md](../../docs/rfc/implemented/2026-06-19-acp-snapshot-tests.md) for the full design.
|
||||
This example is the home of the harness's **snapshot tests** — they boot this server as a real subprocess, drive it with a deterministic input script, and diff its normalized output against committed golden files. The model is made deterministic by `@deepseek-ai/dsh-llm-replay`, a function/namespace plugin that installs an `llm/stream` waterfall listener and short-circuits it, serving model streams reconstructed from a recorded **session JSONL** fixture (`<scenario>/session.jsonl`) — so replay needs no API key. The fixture IS the persisted session log: its `assistant/chunk` events carry every `StreamChunk`, so grouping them by `(turn, step)` reconstructs each `stream()` call (one model call per loop step). Recording is therefore "run the real agent once and harvest the `.jsonl`". The two failure modes not expressible as logged chunks — a pure throw before any chunk, and cancel/hang — use an optional `<scenario>/replay.override.json` sidecar (a `ReplayEntry[]` that replaces the derived script). A scenario that needs the agent to operate on existing files ships an optional `<scenario>/workspace/` directory — the harness copies its contents into the temp cwd before the run (see `workspace-edit`). See [docs/rfc/implemented/2026-06-19-acp-snapshot-tests.md](../../docs/rfc/implemented/2026-06-19-acp-snapshot-tests.md) for the full design.
|
||||
|
||||
## MVP limitations
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@
|
||||
# The replay adapter: short-circuits llm/stream with the recorded log's chunks,
|
||||
# in place of llm-deepseek.
|
||||
- id: llm-replay
|
||||
name: './src/llm-replay.ts'
|
||||
name: '@deepseek-ai/dsh-llm-replay'
|
||||
|
||||
# agent-loop + persistence + the ACP bridge — shared with cordis.yml.
|
||||
- id: acp-tail
|
||||
|
||||
@@ -54,4 +54,6 @@
|
||||
root: './.sessions'
|
||||
|
||||
- id: stdio-chat
|
||||
name: './src/stdio-chat.ts'
|
||||
name: '@deepseek-ai/dsh-ui-stdio'
|
||||
config:
|
||||
welcome: 'coding-agent ready. Give it a coding task (bash is its only tool).'
|
||||
|
||||
@@ -1,119 +0,0 @@
|
||||
import { createInterface } from 'node:readline'
|
||||
import type { Context } from 'cordis'
|
||||
import type {} from '@deepseek-ai/dsh-agent'
|
||||
|
||||
export const name = 'stdio-chat'
|
||||
export const inject = ['agents']
|
||||
|
||||
// Copied from examples/echo-agent (welcome text + reasoning rendering
|
||||
// adjusted). Deliberately example-local rather than a shared package — two
|
||||
// examples don't justify the abstraction yet; revisit at the third.
|
||||
|
||||
/**
|
||||
* Minimal UI plugin: reads lines from stdin → agent.send(); renders the
|
||||
* agent's stream chunks and tool activity to stdout. Demonstrates that a UI
|
||||
* is "just a plugin" — it only consumes the agent/* event taxonomy.
|
||||
*/
|
||||
export function apply(ctx: Context) {
|
||||
let inReasoning = false
|
||||
ctx.on('agent/stream-chunk', (_agent, _turn, _step, chunk) => {
|
||||
if (chunk.type === 'reasoning-delta') {
|
||||
// Dim the chain-of-thought so the answer stands out.
|
||||
if (!inReasoning) process.stdout.write('\x1B[2m')
|
||||
inReasoning = true
|
||||
process.stdout.write(chunk.text)
|
||||
} else if (chunk.type === 'text-delta') {
|
||||
if (inReasoning) process.stdout.write('\x1B[0m\n')
|
||||
inReasoning = false
|
||||
process.stdout.write(chunk.text)
|
||||
}
|
||||
})
|
||||
|
||||
ctx.on('agent/turn-start', (agent, turn) => {
|
||||
process.stdout.write(`\n[${agent.id} turn ${turn}] `)
|
||||
})
|
||||
|
||||
ctx.on('agent/turn-end', () => {
|
||||
if (inReasoning) process.stdout.write('\x1B[0m')
|
||||
inReasoning = false
|
||||
process.stdout.write('\n> ')
|
||||
})
|
||||
|
||||
ctx.on('session/event', (_session, event) => {
|
||||
if (event.type === 'tool/call') {
|
||||
const { name: toolName, arguments: args } = event.data
|
||||
if (inReasoning) process.stdout.write('\x1B[0m')
|
||||
inReasoning = false
|
||||
process.stdout.write(`\n [tool call] ${toolName}(${args})`)
|
||||
} else if (event.type === 'tool/result') {
|
||||
const { content } = event.data
|
||||
const text = content.filter(b => b.type === 'text').map(b => b.text).join('')
|
||||
process.stdout.write(`\n [tool result] ${text}\n `)
|
||||
}
|
||||
})
|
||||
|
||||
ctx.effect(() => {
|
||||
const reader = createInterface({ input: process.stdin })
|
||||
// Piped-input exit, once stdin reaches EOF:
|
||||
// - If no line ever submitted work (empty stdin, blank-only lines), exit
|
||||
// immediately — no turn will ever start, so there is nothing to wait
|
||||
// for. (Gating on an observed 'running' here would hang forever.)
|
||||
// - If work WAS submitted, exit the next time the agent settles to idle
|
||||
// AFTER having run. Two subtleties this handles: the loop batches
|
||||
// several queued messages into ONE turn (one idle), so we don't count
|
||||
// sends; and agent.send() does NOT synchronously flip status to
|
||||
// 'running', so requiring an observed 'running' first (`sawRunning`)
|
||||
// avoids exiting in the gap before the turn starts and dropping work.
|
||||
let stdinClosed = false
|
||||
let disposed = false
|
||||
let submittedWork = false
|
||||
let sawRunning = false
|
||||
|
||||
const maybeExit = (): void => {
|
||||
if (disposed || !stdinClosed) return
|
||||
// No work submitted: nothing will ever run, exit straight away.
|
||||
// Work submitted: wait until a turn has run and the agent is idle.
|
||||
if (submittedWork) {
|
||||
if (!sawRunning) return
|
||||
const agent = ctx.agents.get('main')
|
||||
if (agent && agent.status !== 'idle') return // a turn is still running
|
||||
}
|
||||
// Let any final output flush, then exit.
|
||||
setTimeout(() => process.exit(0), 200)
|
||||
}
|
||||
|
||||
const disposeStatusListener = ctx.on('agent/status', (subject, status) => {
|
||||
if (subject.id !== 'main') return
|
||||
if (status === 'running') sawRunning = true
|
||||
if (status === 'idle') maybeExit()
|
||||
})
|
||||
|
||||
reader.on('line', (line) => {
|
||||
const text = line.trim()
|
||||
if (!text) return
|
||||
const agent = ctx.agents.get('main')
|
||||
if (!agent) {
|
||||
console.error('agent "main" is not running')
|
||||
return
|
||||
}
|
||||
submittedWork = true
|
||||
if (agent.status === 'running') {
|
||||
agent.steer([{ type: 'text', text }])
|
||||
} else {
|
||||
agent.send([{ type: 'text', text }])
|
||||
}
|
||||
})
|
||||
reader.on('close', () => {
|
||||
// Fires for BOTH stdin EOF and plugin disposal (reader.close() below);
|
||||
// `disposed` guards teardown so HMR/dispose never exits the process.
|
||||
stdinClosed = true
|
||||
maybeExit()
|
||||
})
|
||||
process.stdout.write('coding-agent ready. Give it a coding task (bash is its only tool).\n> ')
|
||||
return () => {
|
||||
disposed = true
|
||||
disposeStatusListener()
|
||||
reader.close()
|
||||
}
|
||||
}, 'stdio-chat')
|
||||
}
|
||||
92
examples/coding-agent/tests/keyless-smoke.e2e.ts
Normal file
92
examples/coding-agent/tests/keyless-smoke.e2e.ts
Normal file
@@ -0,0 +1,92 @@
|
||||
import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process'
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
|
||||
/**
|
||||
* Keyless Loader-path smoke for examples/coding-agent: boot the REAL example
|
||||
* through its `cordis.yml` (the cordis Loader, `unwrapExports`, the full plugin
|
||||
* tree incl. the extracted `@deepseek-ai/dsh-ui-stdio`), then close stdin with
|
||||
* no prompt and assert the ready banner + a clean exit.
|
||||
*
|
||||
* No prompt is ever sent, so the model is NEVER called — this is why it runs
|
||||
* without a real key. coding-agent's `cordis.yml` loads `llm-deepseek`, whose
|
||||
* `apply()` only requires a key to be PRESENT (it does not validate it and only
|
||||
* uses it when a stream actually starts), so a dummy key lets the tree boot
|
||||
* while the absence of any prompt guarantees no network call. The value is the
|
||||
* real-Loader-path guard for the shared UI plugin's export shape (a broken
|
||||
* `export default` that drops `inject` would crash here — see postmortem 0001),
|
||||
* complementing coding-agent's with-key e2e suites which prove the real product.
|
||||
*/
|
||||
|
||||
const startScript = fileURLToPath(new URL('../start.ts', import.meta.url))
|
||||
const tsxLoader = fileURLToPath(import.meta.resolve('tsx'))
|
||||
// Dev/test run UNBUILT: resolve `@deepseek-ai/dsh-*` through the root tsconfig
|
||||
// `paths` map; tsx searches UP from cwd, and we spawn from a temp dir outside
|
||||
// the repo, so point it at the repo tsconfig (root is four levels up).
|
||||
const repoTsconfig = fileURLToPath(new URL('../../../tsconfig.json', import.meta.url))
|
||||
|
||||
let child: ChildProcessWithoutNullStreams | undefined
|
||||
let workdir: string | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
if (child !== undefined && child.exitCode === null) child.kill('SIGKILL')
|
||||
child = undefined
|
||||
if (workdir !== undefined) await rm(workdir, { recursive: true, force: true })
|
||||
workdir = undefined
|
||||
})
|
||||
|
||||
async function bootAndEof(): Promise<{ stdout: string; code: number }> {
|
||||
workdir = await mkdtemp(join(tmpdir(), 'coding-smoke-'))
|
||||
const cwd = workdir
|
||||
return new Promise((resolve, reject) => {
|
||||
const proc = spawn(
|
||||
process.execPath,
|
||||
// --expose-internals: cordis.yml loads the HMR plugin (mirrors demo:coding).
|
||||
['--expose-internals', '--import', tsxLoader, startScript],
|
||||
{
|
||||
cwd,
|
||||
env: {
|
||||
...process.env,
|
||||
TSX_TSCONFIG_PATH: repoTsconfig,
|
||||
// A dummy key so llm-deepseek's apply() (key-PRESENT check only) boots.
|
||||
// No prompt is sent, so the adapter never streams — no network call.
|
||||
DEEPSEEK_API_KEY: 'keyless-smoke-no-call',
|
||||
},
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
},
|
||||
)
|
||||
child = proc
|
||||
let stdout = ''
|
||||
let stderr = ''
|
||||
proc.stdout.setEncoding('utf8')
|
||||
proc.stdout.on('data', (chunk: string) => { stdout += chunk })
|
||||
proc.stderr.setEncoding('utf8')
|
||||
proc.stderr.on('data', (chunk: string) => { stderr += chunk })
|
||||
|
||||
const timer = setTimeout(() => {
|
||||
proc.kill('SIGKILL')
|
||||
reject(new Error(`coding-agent did not exit within 10s. stdout:\n${stdout}\nstderr:\n${stderr}`))
|
||||
}, 10_000)
|
||||
|
||||
proc.on('exit', (code) => {
|
||||
clearTimeout(timer)
|
||||
if (code === 0) resolve({ stdout, code })
|
||||
else reject(new Error(`coding-agent exited ${code}. stderr:\n${stderr}`))
|
||||
})
|
||||
proc.on('error', (err) => { clearTimeout(timer); reject(err) })
|
||||
|
||||
// No prompt — just EOF, so the stdio UI exits without ever running a turn.
|
||||
proc.stdin.end()
|
||||
})
|
||||
}
|
||||
|
||||
describe('coding-agent keyless smoke (real cordis.yml via the Loader)', () => {
|
||||
it('boots the full plugin tree, prints its banner, and exits cleanly on EOF', async () => {
|
||||
const { stdout, code } = await bootAndEof()
|
||||
expect(code).toBe(0)
|
||||
expect(stdout).toContain('coding-agent ready.')
|
||||
}, 15_000)
|
||||
})
|
||||
@@ -52,4 +52,6 @@
|
||||
root: './.sessions'
|
||||
|
||||
- id: stdio-chat
|
||||
name: './src/stdio-chat.ts'
|
||||
name: '@deepseek-ai/dsh-ui-stdio'
|
||||
config:
|
||||
welcome: 'echo-agent ready. Type a message ("echo <text>" triggers the tool).'
|
||||
|
||||
@@ -1,60 +0,0 @@
|
||||
import { createInterface } from 'node:readline'
|
||||
import type { Context } from 'cordis'
|
||||
import type {} from '@deepseek-ai/dsh-agent'
|
||||
|
||||
export const name = 'stdio-chat'
|
||||
export const inject = ['agents']
|
||||
|
||||
/**
|
||||
* Minimal UI plugin: reads lines from stdin → agent.send(); renders the
|
||||
* agent's stream chunks and tool activity to stdout. Demonstrates that a UI
|
||||
* is "just a plugin" — it only consumes the agent/* event taxonomy.
|
||||
*/
|
||||
export function apply(ctx: Context) {
|
||||
ctx.on('agent/stream-chunk', (_agent, _turn, _step, chunk) => {
|
||||
if (chunk.type === 'text-delta') process.stdout.write(chunk.text)
|
||||
})
|
||||
|
||||
ctx.on('agent/turn-start', (agent, turn) => {
|
||||
process.stdout.write(`\n[${agent.id} turn ${turn}] `)
|
||||
})
|
||||
|
||||
ctx.on('agent/turn-end', () => {
|
||||
process.stdout.write('\n> ')
|
||||
})
|
||||
|
||||
ctx.on('session/event', (_session, event) => {
|
||||
if (event.type === 'tool/call') {
|
||||
const { name: toolName, arguments: args } = event.data
|
||||
process.stdout.write(`\n [tool call] ${toolName}(${args})`)
|
||||
} else if (event.type === 'tool/result') {
|
||||
const { content } = event.data
|
||||
const text = content.filter(b => b.type === 'text').map(b => b.text).join('')
|
||||
process.stdout.write(`\n [tool result] ${text}\n `)
|
||||
}
|
||||
})
|
||||
|
||||
ctx.effect(() => {
|
||||
const reader = createInterface({ input: process.stdin })
|
||||
reader.on('line', (line) => {
|
||||
const text = line.trim()
|
||||
if (!text) return
|
||||
const agent = ctx.agents.get('main')
|
||||
if (!agent) {
|
||||
console.error('agent "main" is not running')
|
||||
return
|
||||
}
|
||||
if (agent.status === 'running') {
|
||||
agent.steer([{ type: 'text', text }])
|
||||
} else {
|
||||
agent.send([{ type: 'text', text }])
|
||||
}
|
||||
})
|
||||
reader.on('close', () => {
|
||||
// allow the process to exit when stdin ends (piped input)
|
||||
setTimeout(() => process.exit(0), 200)
|
||||
})
|
||||
process.stdout.write('echo-agent ready. Type a message ("echo <text>" triggers the tool).\n> ')
|
||||
return () => { reader.close() }
|
||||
}, 'stdio-chat')
|
||||
}
|
||||
106
examples/echo-agent/tests/echo.e2e.ts
Normal file
106
examples/echo-agent/tests/echo.e2e.ts
Normal file
@@ -0,0 +1,106 @@
|
||||
import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process'
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
|
||||
/**
|
||||
* Keyless Loader-path smoke for examples/echo-agent: boot the REAL example
|
||||
* through its `cordis.yml` (the cordis Loader, `unwrapExports`, the whole
|
||||
* plugin tree), pipe a script of stdin lines, and assert the rendered stdout.
|
||||
*
|
||||
* This is the guard the per-file unit suite structurally cannot be: it drives
|
||||
* the extracted `@deepseek-ai/dsh-ui-stdio` plugin AND the example-local
|
||||
* `mock-llm.ts` / `echo-tool.ts` through their REAL load path, so a broken
|
||||
* plugin export shape (a stray `export default` that `unwrapExports` would
|
||||
* collapse, dropping `inject`) fails here even though hand-mounted unit tests
|
||||
* stay green (see docs/postmortem/0001). It needs no API key — the `mock-echo`
|
||||
* adapter never touches the network — so it runs in the default e2e gate.
|
||||
*
|
||||
* Both branches of mock-llm.ts are exercised: an `echo …` line (the tool
|
||||
* round-trip → `ECHO: …`) and a plain line (the direct canned reply).
|
||||
*/
|
||||
|
||||
const startScript = fileURLToPath(new URL('../start.ts', import.meta.url))
|
||||
const tsxLoader = fileURLToPath(import.meta.resolve('tsx'))
|
||||
// Dev/test run UNBUILT: `@deepseek-ai/dsh-*` imports resolve through the root
|
||||
// tsconfig `paths` map, which tsx finds by searching UP from cwd. We spawn from
|
||||
// a temp cwd OUTSIDE the repo, so point tsx at the repo tsconfig explicitly
|
||||
// (repo root is four levels up from examples/echo-agent/tests).
|
||||
const repoTsconfig = fileURLToPath(new URL('../../../tsconfig.json', import.meta.url))
|
||||
|
||||
let child: ChildProcessWithoutNullStreams | undefined
|
||||
let workdir: string | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
if (child !== undefined && child.exitCode === null) child.kill('SIGKILL')
|
||||
child = undefined
|
||||
if (workdir !== undefined) await rm(workdir, { recursive: true, force: true })
|
||||
workdir = undefined
|
||||
})
|
||||
|
||||
/**
|
||||
* Boot echo-agent, write `lines` to its stdin, close stdin, and resolve with
|
||||
* the full stdout once the process exits (the stdio UI exits on EOF after the
|
||||
* agent settles). Rejects on a non-zero exit or a 10s timeout.
|
||||
*/
|
||||
async function runEcho(lines: string[]): Promise<{ stdout: string; code: number }> {
|
||||
workdir = await mkdtemp(join(tmpdir(), 'echo-smoke-'))
|
||||
const cwd = workdir
|
||||
return new Promise((resolve, reject) => {
|
||||
const proc = spawn(
|
||||
process.execPath,
|
||||
// --expose-internals: the example's cordis.yml loads the HMR plugin, which
|
||||
// requires it (mirrors the `demo:echo` script). The whole point is to boot
|
||||
// the example EXACTLY as it really runs, through the Loader.
|
||||
['--expose-internals', '--import', tsxLoader, startScript],
|
||||
{ cwd, env: { ...process.env, TSX_TSCONFIG_PATH: repoTsconfig }, stdio: ['pipe', 'pipe', 'pipe'] },
|
||||
)
|
||||
child = proc
|
||||
let stdout = ''
|
||||
let stderr = ''
|
||||
proc.stdout.setEncoding('utf8')
|
||||
proc.stdout.on('data', (chunk: string) => { stdout += chunk })
|
||||
proc.stderr.setEncoding('utf8')
|
||||
proc.stderr.on('data', (chunk: string) => { stderr += chunk })
|
||||
|
||||
const timer = setTimeout(() => {
|
||||
proc.kill('SIGKILL')
|
||||
reject(new Error(`echo-agent did not exit within 10s. stdout:\n${stdout}\nstderr:\n${stderr}`))
|
||||
}, 10_000)
|
||||
|
||||
proc.on('exit', (code) => {
|
||||
clearTimeout(timer)
|
||||
if (code === 0) resolve({ stdout, code })
|
||||
else reject(new Error(`echo-agent exited ${code}. stderr:\n${stderr}`))
|
||||
})
|
||||
proc.on('error', (err) => { clearTimeout(timer); reject(err) })
|
||||
|
||||
// Feed the script, then EOF so the stdio UI exits after the agent settles.
|
||||
for (const line of lines) proc.stdin.write(`${line}\n`)
|
||||
proc.stdin.end()
|
||||
})
|
||||
}
|
||||
|
||||
describe('echo-agent keyless smoke (real cordis.yml via the Loader)', () => {
|
||||
it('boots, prints its welcome banner, and exits cleanly on stdin EOF', async () => {
|
||||
const { stdout, code } = await runEcho([])
|
||||
expect(code).toBe(0)
|
||||
expect(stdout).toContain('echo-agent ready.')
|
||||
}, 15_000)
|
||||
|
||||
it('runs the echo tool round-trip for an "echo …" line', async () => {
|
||||
const { stdout } = await runEcho(['echo hello world'])
|
||||
// mock-llm.ts emits a tool-call for the echo tool; echo-tool.ts uppercases.
|
||||
expect(stdout).toContain('[tool call] echo')
|
||||
expect(stdout).toContain('[tool result] ECHO: HELLO WORLD')
|
||||
}, 15_000)
|
||||
|
||||
it('streams a direct canned reply for a non-echo line', async () => {
|
||||
const { stdout } = await runEcho(['just chatting'])
|
||||
// The direct-response branch of mock-llm.ts quotes the input back.
|
||||
expect(stdout).toContain('just chatting')
|
||||
expect(stdout).not.toContain('[tool call]')
|
||||
}, 15_000)
|
||||
})
|
||||
@@ -6,7 +6,8 @@
|
||||
".": {
|
||||
"entry": [
|
||||
"examples/echo-agent/src/*.ts",
|
||||
"examples/coding-agent/src/*.ts",
|
||||
"examples/echo-agent/tests/**/*.e2e.ts",
|
||||
"examples/coding-agent/tests/**/*.e2e.ts",
|
||||
"examples/acp-agent/src/*.ts",
|
||||
"examples/acp-agent/tests/**/*.e2e.ts",
|
||||
"examples/acp-agent/tests/**/*.snapshot.ts"
|
||||
|
||||
@@ -2,6 +2,14 @@
|
||||
|
||||
Harness packages, all under the `@deepseek-ai/dsh-*` scope. Each package is a Cordis plugin (microkernel-style): it exports either a default `Service` subclass or a functional plugin that gets registered via `ctx.plugin()`, declares its ctx key/events where applicable through declaration merging, and exposes extension points through `ctx.effect()`, `ctx.on()`, and `ctx.waterfall()`.
|
||||
|
||||
<!-- FIXME(package-hierarchy): packages/ is currently FLAT, mixing product
|
||||
packages (llm, session, agent, agent-loop, …) with example-coupled support
|
||||
packages (ui-stdio, llm-replay — extracted from examples/ for the coverage
|
||||
gate). ALL packages should eventually be regrouped into a deliberate
|
||||
hierarchy, e.g. packages/{core,examples,…}/, so the workspace-glob and
|
||||
tsconfig-paths churn happens ONCE rather than per extraction. Deferred to a
|
||||
dedicated restructure PR; do not add new top-level subgroups piecemeal. -->
|
||||
|
||||
## Dependency graph
|
||||
|
||||
```
|
||||
@@ -18,6 +26,8 @@ dsh-llm-pi-ai ← dsh-llm (pi-ai-backed adapter)
|
||||
dsh-agent-loop ← dsh-llm, dsh-session, dsh-system-prompt, dsh-tools, dsh-agent
|
||||
dsh-invariants ← dsh-llm, dsh-session, dsh-agent (dev-mode contract checks)
|
||||
dsh-acp ← dsh-agent, dsh-llm, dsh-session, dsh-session-persistence (ACP JSON-RPC bridge)
|
||||
dsh-ui-stdio ← dsh-agent, dsh-llm, dsh-session (stdio readline UI plugin)
|
||||
dsh-llm-replay ← dsh-llm, dsh-session (record/replay adapter for keyless snapshot tests)
|
||||
```
|
||||
|
||||
The rule: plugins depend on interfaces, never on the concrete loop. `dsh-agent-loop` is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. A swappable capability splits into interface / implementation / consumer packages (the bash trio is the template — see [capability seams](../docs/rfc/implemented/2026-06-13-capability-seams.md)).
|
||||
@@ -39,6 +49,8 @@ The rule: plugins depend on interfaces, never on the concrete loop. `dsh-agent-l
|
||||
| `llm-pi-ai/` | DeepSeek adapter via `@earendil-works/pi-ai` (design twin) | (registers on `ctx.llm`) |
|
||||
| `invariants/` | Dev-mode event-contract invariants + session-log freeze | (listens on `session/*`, `agent/*`) |
|
||||
| `acp/` | Agent Client Protocol bridge: serves the agent to an ACP editor over JSON-RPC stdio | (drives `ctx.agents`/`ctx.sessions`) |
|
||||
| `ui-stdio/` | Minimal stdio (readline) UI plugin: renders `agent/*` events, feeds stdin lines to the agent | (drives `ctx.agents`) |
|
||||
| `llm-replay/` | Record/replay adapter: short-circuits `llm/stream` with chunks from a recorded session JSONL (keyless snapshot tests) | (listens on `llm/stream`) |
|
||||
|
||||
Each package has its own `README.md` with purpose, service API, events, extension points, and deliberate non-goals (TODOs).
|
||||
|
||||
|
||||
36
packages/llm-replay/README.md
Normal file
36
packages/llm-replay/README.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# @deepseek-ai/dsh-llm-replay
|
||||
|
||||
A replay LLM plugin for keyless snapshot tests. It installs a single `llm/stream` waterfall listener that short-circuits the waterfall (never calls `next()`) and yields model streams reconstructed from a recorded **session JSONL** fixture — so a test can boot the real agent against a fixed model transcript with no API key.
|
||||
|
||||
Its consumer is the ACP snapshot harness in `examples/acp-agent`, which loads this plugin (via `cordis.snapshot.yml`) in place of a real LLM adapter. The package exists so its derive/parse/replay logic falls under the per-file 100% coverage gate on `packages/*/src` (the same logic, while it lived under `examples/`, was outside the gate).
|
||||
|
||||
## How the fixture works
|
||||
|
||||
The fixture IS the persisted session log (`<scenario>/session.jsonl`). Its `assistant/chunk` events carry every `StreamChunk`, so grouping them by `(turn, step)` reconstructs each `stream()` call's chunk sequence (one model call per loop step). Recording is therefore "run the real agent once and harvest the `.jsonl`", done by the snapshot harness — this plugin does not record.
|
||||
|
||||
Two failure modes are not reconstructable from `assistant/chunk` alone — a pure throw before any chunk (e.g. an HTTP 401, where the log holds only a `turn/end {error}` and no chunks) and a cancel/hang (timing, not chunk content). A scenario that needs those supplies an optional sidecar (`<scenario>/replay.override.json`: a `ReplayEntry[]`) that REPLACES the derived script.
|
||||
|
||||
## Config
|
||||
|
||||
| Key | Type | Default | Notes |
|
||||
|---|---|---|---|
|
||||
| `file` | string | `$DSH_SNAPSHOT_FILE` | Path to the per-scenario `session.jsonl` fixture. Required (config or env). |
|
||||
| `overrideFile` | string | `$DSH_SNAPSHOT_OVERRIDE` | Optional path to a `ReplayEntry[]` sidecar that replaces the derived script. |
|
||||
|
||||
```yaml
|
||||
- id: llm-replay
|
||||
name: '@deepseek-ai/dsh-llm-replay'
|
||||
# file/overrideFile default to $DSH_SNAPSHOT_FILE / $DSH_SNAPSHOT_OVERRIDE,
|
||||
# set by the snapshot harness per scenario.
|
||||
```
|
||||
|
||||
## Exports
|
||||
|
||||
- `installLlmReplay(ctx, config)` — install the `llm/stream` listener; returns the disposer (HMR safety). Use this in tests to drive replay without the Loader or env vars.
|
||||
- `loadReplayScript(config)` — resolve the `ReplayEntry[]` for a scenario (sidecar override if present, else derived from the JSONL; fail-loud if the fixture is missing).
|
||||
- `deriveReplayScript(events)` / `parseSessionLog(text)` — the pure helpers that turn a recorded session log into a script. A derived group must end in a `finish` chunk; a group without one is the fingerprint of a thrown `stream()` and must instead be expressed via an override sidecar.
|
||||
- Types `ReplayEntry` / `ReplayConfig` / `Config`.
|
||||
|
||||
## Plugin export shape
|
||||
|
||||
Named `name` / `inject` / `Config` / `apply`, with **no default export**: the cordis Loader's `unwrapExports` does `exports.default ?? exports`, so a stray default would collapse the module to the bare function and drop the `inject` namespace (see [docs/postmortem/0001](../../docs/postmortem/0001-acp-default-export-drops-inject.md)).
|
||||
32
packages/llm-replay/package.json
Normal file
32
packages/llm-replay/package.json
Normal file
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-llm-replay",
|
||||
"description": "Replay LLM plugin: short-circuits llm/stream with model chunks reconstructed from a recorded session JSONL (keyless snapshot tests)",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
}
|
||||
@@ -21,14 +21,18 @@
|
||||
* (`<scenario>/replay.override.json`: a `ReplayEntry[]`) that REPLACES the
|
||||
* derived script.
|
||||
*
|
||||
* It lives in the example (not packages/) because it is example/test
|
||||
* infrastructure with one consumer, exactly like echo-agent's `mock-llm.ts`;
|
||||
* the capability-seams rule says not to split into a published package
|
||||
* preemptively.
|
||||
* It lives in its own package (not under `examples/`) so its derive/parse/
|
||||
* replay logic falls under the per-file 100% coverage gate on package `src`
|
||||
* trees — its tests previously lived under `examples/`, which the gate does
|
||||
* not measure, leaving these branches (clean chunks / mid-stream throw / hang)
|
||||
* unguarded. Its consumer is the ACP snapshot harness in `examples/acp-agent`,
|
||||
* which loads it (via `cordis.snapshot.yml`) in place of a real LLM adapter.
|
||||
*
|
||||
* Plugin export shape: named `name`/`inject`/`Config`/`apply`, NO default
|
||||
* export (the cordis Loader's `unwrapExports` does `exports.default ?? exports`,
|
||||
* so a stray default would drop the namespace — see docs/postmortem/0001).
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm-replay
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync } from 'node:fs'
|
||||
@@ -188,6 +192,7 @@ async function* replayEntry(entry: ReplayEntry, signal: AbortSignal | undefined)
|
||||
if (signal?.aborted) { reject(new Error('aborted')); return }
|
||||
signal?.addEventListener('abort', () => { reject(new Error('aborted')) }, { once: true })
|
||||
})
|
||||
/* v8 ignore next -- unreachable: the hang promise only ever rejects (on abort), never resolves; control never reaches here */
|
||||
return
|
||||
default:
|
||||
// Closed local union: an unknown kind means malformed (hand-edited or
|
||||
@@ -7,11 +7,14 @@ import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import LlmService, { GenerateOptions, LlmAdapter, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import {
|
||||
type ReplayEntry,
|
||||
apply,
|
||||
deriveReplayScript,
|
||||
inject,
|
||||
installLlmReplay,
|
||||
loadReplayScript,
|
||||
name,
|
||||
parseSessionLog,
|
||||
} from '../src/llm-replay.ts'
|
||||
} from '../src/index.ts'
|
||||
|
||||
/**
|
||||
* Unit tests for the replay llm/stream plugin. These drive the listener through
|
||||
@@ -290,4 +293,129 @@ describe('installLlmReplay (through the real waterfall)', () => {
|
||||
expect(await drain(ctx.llm.stream({ model: 'm', messages: [] })))
|
||||
.toEqual([{ type: 'finish', reason: { kind: 'stop' } }])
|
||||
})
|
||||
|
||||
it('throws on a malformed sidecar entry kind (the assertNever guard)', async () => {
|
||||
writeFileSync(file, sessionJsonl([]), 'utf8')
|
||||
const overrideFile = join(dir, 'replay.override.json')
|
||||
// A kind the union does not know — hand-edited/drifted sidecar data.
|
||||
writeFileSync(overrideFile, JSON.stringify([{ kind: 'bogus' }]), 'utf8')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
installLlmReplay(ctx, { file, overrideFile })
|
||||
await expect(drain(ctx.llm.stream({ model: 'm', messages: [] })))
|
||||
.rejects.toThrow(/llm-replay replay entry/)
|
||||
})
|
||||
|
||||
it('rejects a hang entry when the signal fires DURING the wait (abort listener path)', async () => {
|
||||
writeFileSync(file, sessionJsonl([]), 'utf8')
|
||||
const overrideFile = join(dir, 'replay.override.json')
|
||||
writeFileSync(overrideFile, JSON.stringify([{ kind: 'hang' }]), 'utf8')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
installLlmReplay(ctx, { file, overrideFile })
|
||||
const controller = new AbortController()
|
||||
const iterator = ctx.llm.stream({ model: 'm', messages: [], signal: controller.signal })[Symbol.asyncIterator]()
|
||||
// Consume the two pre-hang chunks, then start the third pull so the generator
|
||||
// is parked inside the await (signal NOT yet aborted — exercises the
|
||||
// addEventListener('abort') registration), and only THEN abort.
|
||||
expect((await iterator.next()).value).toMatchObject({ type: 'block-start' })
|
||||
expect((await iterator.next()).value).toMatchObject({ type: 'text-delta' })
|
||||
const pending = iterator.next()
|
||||
await new Promise(r => setImmediate(r))
|
||||
controller.abort()
|
||||
await expect(pending).rejects.toThrow('aborted')
|
||||
})
|
||||
|
||||
it('aborts mid-replay of a throw-entry prefix when the signal is set', async () => {
|
||||
writeFileSync(file, sessionJsonl([]), 'utf8')
|
||||
const overrideFile = join(dir, 'replay.override.json')
|
||||
const partial: StreamChunk[] = [{ type: 'block-start', index: 0, blockType: 'text' }]
|
||||
writeFileSync(overrideFile, JSON.stringify([
|
||||
{ kind: 'throw', chunks: partial, message: 'unauthorized', code: 'AUTH', status: 401 },
|
||||
]), 'utf8')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
installLlmReplay(ctx, { file, overrideFile })
|
||||
const controller = new AbortController()
|
||||
controller.abort()
|
||||
// Already aborted: the throw-entry's prefix loop surfaces 'aborted' before
|
||||
// it can reach the recorded LlmError.
|
||||
await expect(drain(ctx.llm.stream({ model: 'm', messages: [], signal: controller.signal })))
|
||||
.rejects.toThrow('aborted')
|
||||
})
|
||||
|
||||
it('surfaces an already-aborted signal on a hang entry before waiting', async () => {
|
||||
writeFileSync(file, sessionJsonl([]), 'utf8')
|
||||
const overrideFile = join(dir, 'replay.override.json')
|
||||
writeFileSync(overrideFile, JSON.stringify([{ kind: 'hang' }]), 'utf8')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
installLlmReplay(ctx, { file, overrideFile })
|
||||
const controller = new AbortController()
|
||||
controller.abort()
|
||||
// The two pre-hang chunks still flow; the abort surfaces at the await.
|
||||
const iterator = ctx.llm.stream({ model: 'm', messages: [], signal: controller.signal })[Symbol.asyncIterator]()
|
||||
await iterator.next()
|
||||
await iterator.next()
|
||||
await expect(iterator.next()).rejects.toThrow('aborted')
|
||||
})
|
||||
})
|
||||
|
||||
describe('apply (the plugin entry)', () => {
|
||||
const ORIG = { file: process.env.DSH_SNAPSHOT_FILE, override: process.env.DSH_SNAPSHOT_OVERRIDE }
|
||||
afterEach(() => {
|
||||
if (ORIG.file === undefined) delete process.env.DSH_SNAPSHOT_FILE
|
||||
else process.env.DSH_SNAPSHOT_FILE = ORIG.file
|
||||
if (ORIG.override === undefined) delete process.env.DSH_SNAPSHOT_OVERRIDE
|
||||
else process.env.DSH_SNAPSHOT_OVERRIDE = ORIG.override
|
||||
})
|
||||
|
||||
it('exposes the namespace plugin shape (name/inject, no default export)', () => {
|
||||
expect(name).toBe('llm-replay')
|
||||
expect(inject).toEqual(['llm'])
|
||||
})
|
||||
|
||||
it('installs replay from an explicit config.file', async () => {
|
||||
writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c))), 'utf8')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
apply(ctx, { file })
|
||||
expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS)
|
||||
})
|
||||
|
||||
it('falls back to $DSH_SNAPSHOT_FILE / $DSH_SNAPSHOT_OVERRIDE when config is empty', async () => {
|
||||
writeFileSync(file, sessionJsonl([]), 'utf8')
|
||||
const overrideFile = join(dir, 'replay.override.json')
|
||||
writeFileSync(overrideFile, JSON.stringify([{ kind: 'chunks', chunks: TEXT_CHUNKS }]), 'utf8')
|
||||
process.env.DSH_SNAPSHOT_FILE = file
|
||||
process.env.DSH_SNAPSHOT_OVERRIDE = overrideFile
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
apply(ctx)
|
||||
expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS)
|
||||
})
|
||||
|
||||
it('uses only the file when no override path is configured or in the env', async () => {
|
||||
writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c))), 'utf8')
|
||||
process.env.DSH_SNAPSHOT_FILE = file
|
||||
delete process.env.DSH_SNAPSHOT_OVERRIDE
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
apply(ctx)
|
||||
expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS)
|
||||
})
|
||||
|
||||
it('throws when no fixture path is given by config or env', async () => {
|
||||
delete process.env.DSH_SNAPSHOT_FILE
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
expect(() => { apply(ctx, {}) }).toThrow(/a fixture path is required/)
|
||||
})
|
||||
|
||||
it('treats an empty-string fixture path as missing', async () => {
|
||||
delete process.env.DSH_SNAPSHOT_FILE
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
expect(() => { apply(ctx, { file: '' }) }).toThrow(/a fixture path is required/)
|
||||
})
|
||||
})
|
||||
14
packages/llm-replay/tsconfig.json
Normal file
14
packages/llm-replay/tsconfig.json
Normal file
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"extends": "../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib"
|
||||
},
|
||||
"include": ["src"],
|
||||
"references": [
|
||||
{ "path": "../../vendor/cosmokit" },
|
||||
{ "path": "../../vendor/cordis" },
|
||||
{ "path": "../llm" },
|
||||
{ "path": "../session" }
|
||||
]
|
||||
}
|
||||
44
packages/ui-stdio/README.md
Normal file
44
packages/ui-stdio/README.md
Normal file
@@ -0,0 +1,44 @@
|
||||
# @deepseek-ai/dsh-ui-stdio
|
||||
|
||||
A minimal stdio (readline) UI, as a plugin. It reads lines from stdin and feeds them to an agent (`send` when idle, `steer` while a turn is running), and renders that agent's streamed output and tool activity to stdout. A UI is "just a plugin" here — it only consumes the `agent/*` event taxonomy plus the `agents` service (`inject: ['agents']`), so the same plugin drives any example or product surface.
|
||||
|
||||
This package consolidates what were two near-identical copies under `examples/echo-agent` and `examples/coding-agent`. The coding copy was a superset; this package IS that superset — dimmed chain-of-thought rendering plus robust piped-stdin EOF handling — with the per-consumer differences moved into `Config`.
|
||||
|
||||
## Config
|
||||
|
||||
| Key | Type | Default | Notes |
|
||||
|---|---|---|---|
|
||||
| `welcome` | string | `'ready.'` | Banner printed once on start, before the first `> ` prompt. |
|
||||
| `agent` | string | `'main'` | Id of the agent that stdin **drives** (`send`/`steer`) and whose `agent/status` gates the EOF exit. Rendering is **not** scoped by it — see below. |
|
||||
|
||||
```yaml
|
||||
- id: ui-stdio
|
||||
name: '@deepseek-ai/dsh-ui-stdio'
|
||||
config:
|
||||
welcome: 'coding-agent ready. Give it a coding task.'
|
||||
```
|
||||
|
||||
## Rendering
|
||||
|
||||
Rendering is **global** — every agent's events are written to stdout, not just `config.agent`'s. `config.agent` scopes only *input* (which agent stdin drives) and the EOF-exit gate; the single-agent demos this serves have just one agent, so the distinction is moot for them. (A multi-agent UI that needs per-agent panes would filter these handlers by the agent argument — deliberately out of scope here.)
|
||||
|
||||
- `agent/stream-chunk` — `text-delta` is written verbatim; `reasoning-delta` is wrapped in the dim SGR (`\x1B[2m … \x1B[0m`) so the chain-of-thought is visually subordinate to the answer. Reasoning rendering is inert when no `reasoning-delta` chunks arrive (e.g. a mock model), so it is always on.
|
||||
- `agent/turn-start` / `agent/turn-end` — a `[<agent> turn N]` header and a trailing `> ` prompt.
|
||||
- `session/event` — `tool/call` renders `[tool call] name(args)`; `tool/result` renders the joined text blocks as `[tool result] …`.
|
||||
|
||||
## The I/O seam
|
||||
|
||||
The production entry point `apply(ctx, config)` binds the real `process` streams. The testable core is `createStdioChat(ctx, config, runtime)`, where `runtime: StdioRuntime` supplies `input` / `output` / `exit`. This seam is deliberately **not** part of the serializable `Config` (streams and functions do not belong in YAML config); it exists so the render, EOF, and disposal branches can be exercised with fakes instead of hijacking globals.
|
||||
|
||||
## Piped-stdin exit
|
||||
|
||||
On stdin EOF the plugin exits the process, but carefully:
|
||||
|
||||
- **No work submitted** (empty stdin, blank-only lines): exit immediately — no turn will ever start, so there is nothing to wait for. Gating on an observed `running` here would hang forever.
|
||||
- **Work submitted**: exit the next time the agent settles to `idle` *after* having been observed `running`. `agent.send()` does not synchronously flip status to `running`, so requiring an observed `running` first (`sawRunning`) avoids exiting in the gap before the turn starts and dropping work; and the loop batches several queued messages into one turn, so the exit keys off the idle transition rather than counting sends.
|
||||
|
||||
Disposal (HMR or fiber teardown) closes the readline interface, which also fires `close` — a `disposed` guard ensures teardown never calls `process.exit`.
|
||||
|
||||
## Plugin export shape
|
||||
|
||||
Named `name` / `inject` / `Config` / `apply`, with **no default export**: the cordis Loader's `unwrapExports` does `exports.default ?? exports`, so a stray default would collapse the module to the bare function and drop the `inject` namespace (see [docs/postmortem/0001](../../docs/postmortem/0001-acp-default-export-drops-inject.md)). The keyless Loader-path e2e smokes in `examples/{echo,coding}-agent` guard this end-to-end.
|
||||
37
packages/ui-stdio/package.json
Normal file
37
packages/ui-stdio/package.json
Normal file
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-ui-stdio",
|
||||
"description": "Minimal stdio (readline) UI plugin: renders agent/* events to stdout and feeds stdin lines to the agent",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
}
|
||||
201
packages/ui-stdio/src/index.ts
Normal file
201
packages/ui-stdio/src/index.ts
Normal file
@@ -0,0 +1,201 @@
|
||||
/**
|
||||
* Minimal stdio UI plugin: reads lines from stdin → `agent.send()`/`steer()`,
|
||||
* and renders the agent's stream chunks and tool activity to stdout. A UI is
|
||||
* "just a plugin" — it only consumes the `agent/*` event taxonomy and the
|
||||
* `agents` service, so the same plugin drives any example or product surface.
|
||||
*
|
||||
* Consolidates what were two near-identical copies under `examples/echo-agent`
|
||||
* and `examples/coding-agent` (the latter a superset). This package IS that
|
||||
* superset: dimmed chain-of-thought rendering plus the robust piped-stdin
|
||||
* EOF→idle exit handling, configured per consumer via {@link Config}.
|
||||
*
|
||||
* Plugin export shape: named `name`/`inject`/`Config`/`apply`, NO default
|
||||
* export — the cordis Loader's `unwrapExports` does `exports.default ?? exports`,
|
||||
* so a stray default would collapse the module to the bare function and drop
|
||||
* the `inject` namespace (see docs/postmortem/0001). The keyless Loader-path
|
||||
* e2e smokes in `examples/{echo,coding}-agent` guard this end-to-end.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-ui-stdio
|
||||
*/
|
||||
|
||||
import { createInterface } from 'node:readline'
|
||||
import type { Readable, Writable } from 'node:stream'
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type {} from '@deepseek-ai/dsh-agent'
|
||||
|
||||
export const name = 'ui-stdio'
|
||||
export const inject = ['agents']
|
||||
|
||||
/** Serializable plugin configuration (cordis-native, schemastery). */
|
||||
export interface Config {
|
||||
/** Banner printed once on start, before the first `> ` prompt. */
|
||||
welcome?: string
|
||||
/** Id of the agent stdin drives (`send`/`steer`) and whose status gates the EOF exit; rendering is global. Defaults to `'main'`. */
|
||||
agent?: string
|
||||
}
|
||||
|
||||
export const Config: z<Config> = z.object({
|
||||
welcome: z.string().default('ready.'),
|
||||
agent: z.string().default('main'),
|
||||
})
|
||||
|
||||
/**
|
||||
* Process-I/O seam — the side-effecting handles the plugin would otherwise
|
||||
* reach for as globals. Defaulted to the real `process` streams in
|
||||
* {@link apply}; injected by tests so the EOF, render, and disposal branches
|
||||
* are exercised without hijacking globals. Deliberately NOT part of the
|
||||
* serializable {@link Config} (streams/functions don't belong in YAML config).
|
||||
*/
|
||||
export interface StdioRuntime {
|
||||
/** Line source (default `process.stdin`). */
|
||||
input: Readable
|
||||
/** Render sink (default `process.stdout`). */
|
||||
output: Writable
|
||||
/** Process-exit hook (default `process.exit`); called once on stdin EOF. */
|
||||
exit: (code: number) => void
|
||||
}
|
||||
|
||||
/**
|
||||
* The plugin body, parameterized over its I/O runtime. `apply` is the thin
|
||||
* production wrapper that binds the real `process` streams; tests call this
|
||||
* directly with fakes. Returns nothing — all registration is via `ctx.on`/
|
||||
* `ctx.effect`, so fiber disposal tears every listener and the readline
|
||||
* interface down.
|
||||
*/
|
||||
export function createStdioChat(ctx: Context, config: Config, runtime: StdioRuntime): void {
|
||||
// Default here too (not just via schemastery's `.default()`): this helper is
|
||||
// exported and called directly by tests / programmatic consumers that bypass
|
||||
// Loader validation, so it must be self-contained rather than trusting the
|
||||
// cast — `config.welcome as string` would otherwise be `undefined` on `{}`.
|
||||
const welcome = config.welcome ?? 'ready.'
|
||||
const agentId = config.agent ?? 'main'
|
||||
const { input, output, exit } = runtime
|
||||
|
||||
let inReasoning = false
|
||||
ctx.on('agent/stream-chunk', (_agent, _turn, _step, chunk) => {
|
||||
if (chunk.type === 'reasoning-delta') {
|
||||
// Dim the chain-of-thought so the final answer stands out.
|
||||
if (!inReasoning) output.write('\x1B[2m')
|
||||
inReasoning = true
|
||||
output.write(chunk.text)
|
||||
} else if (chunk.type === 'text-delta') {
|
||||
if (inReasoning) output.write('\x1B[0m\n')
|
||||
inReasoning = false
|
||||
output.write(chunk.text)
|
||||
}
|
||||
})
|
||||
|
||||
ctx.on('agent/turn-start', (agent, turn) => {
|
||||
output.write(`\n[${agent.id} turn ${turn}] `)
|
||||
})
|
||||
|
||||
ctx.on('agent/turn-end', () => {
|
||||
if (inReasoning) output.write('\x1B[0m')
|
||||
inReasoning = false
|
||||
output.write('\n> ')
|
||||
})
|
||||
|
||||
ctx.on('session/event', (_session, event) => {
|
||||
if (event.type === 'tool/call') {
|
||||
const { name: toolName, arguments: args } = event.data
|
||||
if (inReasoning) output.write('\x1B[0m')
|
||||
inReasoning = false
|
||||
output.write(`\n [tool call] ${toolName}(${args})`)
|
||||
} else if (event.type === 'tool/result') {
|
||||
const { content } = event.data
|
||||
const text = content.filter(block => block.type === 'text').map(block => block.text).join('')
|
||||
output.write(`\n [tool result] ${text}\n `)
|
||||
}
|
||||
})
|
||||
|
||||
ctx.effect(() => {
|
||||
const reader = createInterface({ input })
|
||||
// Piped-input exit, once stdin reaches EOF:
|
||||
// - If no line ever submitted work (empty stdin, blank-only lines), exit
|
||||
// immediately — no turn will ever start, so there is nothing to wait
|
||||
// for. (Gating on an observed 'running' here would hang forever.)
|
||||
// - If work WAS submitted, exit the next time the agent settles to idle
|
||||
// AFTER having run. Two subtleties this handles: the loop batches
|
||||
// several queued messages into ONE turn (one idle), so we don't count
|
||||
// sends; and agent.send() does NOT synchronously flip status to
|
||||
// 'running', so requiring an observed 'running' first (`sawRunning`)
|
||||
// avoids exiting in the gap before the turn starts and dropping work.
|
||||
let stdinClosed = false
|
||||
let disposed = false
|
||||
let submittedWork = false
|
||||
let sawRunning = false
|
||||
let exitTimer: ReturnType<typeof setTimeout> | undefined
|
||||
|
||||
const maybeExit = (): void => {
|
||||
if (disposed || !stdinClosed) return
|
||||
// No work submitted: nothing will ever run, exit straight away.
|
||||
// Work submitted: wait until a turn has run and the agent is idle.
|
||||
if (submittedWork) {
|
||||
if (!sawRunning) return
|
||||
const agent = ctx.agents.get(agentId)
|
||||
if (agent && agent.status !== 'idle') return // a turn is still running
|
||||
}
|
||||
// Let any final output flush, then exit. The handle is tracked so the
|
||||
// disposer can cancel it — a dispose within the flush window must not let
|
||||
// the process exit out from under HMR. Re-entrant `maybeExit` calls (e.g.
|
||||
// repeated idle signals) coalesce onto the one pending timer.
|
||||
if (exitTimer !== undefined) {
|
||||
return // exit already scheduled — coalesce re-entrant calls
|
||||
}
|
||||
exitTimer = setTimeout(() => { exit(0) }, 200)
|
||||
}
|
||||
|
||||
const disposeStatusListener = ctx.on('agent/status', (subject, status) => {
|
||||
if (subject.id !== agentId) return
|
||||
if (status === 'running') sawRunning = true
|
||||
if (status === 'idle') maybeExit()
|
||||
})
|
||||
|
||||
reader.on('line', (line) => {
|
||||
const text = line.trim()
|
||||
if (!text) return
|
||||
const agent = ctx.agents.get(agentId)
|
||||
if (!agent) {
|
||||
ctx.logger.error('ui-stdio: agent "%s" is not running', agentId)
|
||||
return
|
||||
}
|
||||
submittedWork = true
|
||||
if (agent.status === 'running') {
|
||||
agent.steer([{ type: 'text', text }])
|
||||
} else {
|
||||
agent.send([{ type: 'text', text }])
|
||||
}
|
||||
})
|
||||
reader.on('close', () => {
|
||||
// Fires for BOTH stdin EOF and plugin disposal (reader.close() below);
|
||||
// `disposed` guards teardown so HMR/dispose never exits the process.
|
||||
stdinClosed = true
|
||||
maybeExit()
|
||||
})
|
||||
output.write(`${welcome}\n> `)
|
||||
return () => {
|
||||
disposed = true
|
||||
if (exitTimer !== undefined) clearTimeout(exitTimer)
|
||||
disposeStatusListener()
|
||||
reader.close()
|
||||
}
|
||||
}, 'ui-stdio')
|
||||
}
|
||||
|
||||
/**
|
||||
* Cordis entry point. Binds the real `process` streams and delegates to
|
||||
* {@link createStdioChat}; the indirection keeps the side-effecting handles out
|
||||
* of the testable core, which is why the unit suite drives `createStdioChat`
|
||||
* directly. This thin wrapper is exercised end-to-end by the keyless
|
||||
* Loader-path e2e smoke in `examples/echo-agent` (the real product entry).
|
||||
*/
|
||||
/* v8 ignore start -- production stdio wiring; testable core is createStdioChat() (covered), exercised e2e by echo-agent keyless smoke */
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
createStdioChat(ctx, config, {
|
||||
input: process.stdin,
|
||||
output: process.stdout,
|
||||
exit: code => process.exit(code),
|
||||
})
|
||||
}
|
||||
/* v8 ignore stop */
|
||||
347
packages/ui-stdio/tests/ui-stdio.spec.ts
Normal file
347
packages/ui-stdio/tests/ui-stdio.spec.ts
Normal file
@@ -0,0 +1,347 @@
|
||||
import { Readable } from 'node:stream'
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import type { Agent, AgentStatus } from '@deepseek-ai/dsh-agent'
|
||||
import AgentRegistry from '@deepseek-ai/dsh-agent'
|
||||
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import { createStdioChat, type Config, type StdioRuntime } from '../src/index.ts'
|
||||
|
||||
/**
|
||||
* Unit tests for the stdio UI plugin. They drive the REAL plugin body
|
||||
* (`createStdioChat`) with an injected {@link StdioRuntime} so every render,
|
||||
* input, EOF, and disposal branch runs without touching the real `process`
|
||||
* streams — the I/O seam is what makes the per-file gate reachable. The
|
||||
* `agents` service is real (`@deepseek-ai/dsh-agent`); a minimal fake `Agent`
|
||||
* stands in for the loop, since the loop is the genuinely expensive collaborator
|
||||
* and we only need its `status` + `send`/`steer` surface here.
|
||||
*/
|
||||
|
||||
/** A controllable stdin: a Readable we push lines into and can end on demand. */
|
||||
function makeInput(): Readable & { feed(line: string): void; finish(): void } {
|
||||
const stream = new Readable({ read() {} }) as Readable & { feed(line: string): void; finish(): void }
|
||||
stream.feed = (line: string) => stream.push(`${line}\n`)
|
||||
stream.finish = () => stream.push(null)
|
||||
return stream
|
||||
}
|
||||
|
||||
/** A stdout sink that accumulates everything written, for assertions. */
|
||||
function makeOutput(): { write: (s: string) => boolean; text: () => string } {
|
||||
let buf = ''
|
||||
return { write: (s: string) => { buf += s; return true }, text: () => buf }
|
||||
}
|
||||
|
||||
function makeRuntime(over: Partial<StdioRuntime> = {}): {
|
||||
runtime: StdioRuntime
|
||||
input: ReturnType<typeof makeInput>
|
||||
out: ReturnType<typeof makeOutput>
|
||||
exit: ReturnType<typeof vi.fn>
|
||||
} {
|
||||
const input = makeInput()
|
||||
const out = makeOutput()
|
||||
const exit = vi.fn()
|
||||
return { runtime: { input, output: { write: out.write } as never, exit, ...over }, input, out, exit }
|
||||
}
|
||||
|
||||
/** A minimal Agent fake exposing the surface the UI touches. */
|
||||
function makeAgent(id: string, status: AgentStatus = 'idle'): Agent & {
|
||||
status: AgentStatus
|
||||
sent: ContentBlock[][]
|
||||
steered: ContentBlock[][]
|
||||
} {
|
||||
const sent: ContentBlock[][] = []
|
||||
const steered: ContentBlock[][] = []
|
||||
return {
|
||||
id: id as Agent['id'],
|
||||
status,
|
||||
sent,
|
||||
steered,
|
||||
send: (content: ContentBlock[]) => void sent.push(content),
|
||||
steer: (content: ContentBlock[]) => void steered.push(content),
|
||||
} as never
|
||||
}
|
||||
|
||||
const CONFIG: Config = { welcome: 'hi there', agent: 'main' }
|
||||
|
||||
async function setup(config: Config = CONFIG, runtimeOver: Partial<StdioRuntime> = {}) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(AgentRegistry)
|
||||
const { runtime, input, out, exit } = makeRuntime(runtimeOver)
|
||||
const fiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
createStdioChat(inner, config, runtime)
|
||||
}, { inject: ['agents'] }))
|
||||
return { ctx, fiber, input, out, exit }
|
||||
}
|
||||
|
||||
/** Drive a fake idle timer past the 200ms flush delay. */
|
||||
function flushExit(): Promise<void> {
|
||||
return new Promise(resolve => setTimeout(resolve, 250))
|
||||
}
|
||||
|
||||
describe('createStdioChat rendering', () => {
|
||||
it('writes the welcome banner and prompt on start', async () => {
|
||||
const { out } = await setup()
|
||||
expect(out.text()).toBe('hi there\n> ')
|
||||
})
|
||||
|
||||
it('falls back to default welcome/agent when called with empty config', async () => {
|
||||
// createStdioChat is exported and may be driven directly (bypassing the
|
||||
// Loader's schemastery validation), so it must default welcome/agent itself.
|
||||
const { out } = await setup({})
|
||||
expect(out.text()).toBe('ready.\n> ')
|
||||
// And it drives the default agent id 'main'.
|
||||
})
|
||||
|
||||
it('renders text-delta chunks verbatim', async () => {
|
||||
const { ctx, out } = await setup()
|
||||
const agent = makeAgent('main')
|
||||
ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'text-delta', index: 0, text: 'hello' })
|
||||
expect(out.text()).toContain('hello')
|
||||
})
|
||||
|
||||
it('wraps reasoning-delta in the dim SGR and resets on the following text-delta', async () => {
|
||||
const { ctx, out } = await setup()
|
||||
const agent = makeAgent('main')
|
||||
ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'reasoning-delta', index: 0, text: 'think' })
|
||||
ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'reasoning-delta', index: 0, text: 'more' })
|
||||
ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'text-delta', index: 0, text: 'answer' })
|
||||
expect(out.text()).toContain('\x1B[2mthinkmore\x1B[0m\nanswer')
|
||||
})
|
||||
|
||||
it('ignores stream-chunk types it does not render', async () => {
|
||||
const { ctx, out } = await setup()
|
||||
const before = out.text()
|
||||
const agent = makeAgent('main')
|
||||
ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'block-start', index: 0, blockType: 'text' })
|
||||
expect(out.text()).toBe(before)
|
||||
})
|
||||
|
||||
it('renders turn-start and turn-end markers', async () => {
|
||||
const { ctx, out } = await setup()
|
||||
const agent = makeAgent('main')
|
||||
ctx.emit('agent/turn-start', agent, 3)
|
||||
expect(out.text()).toContain('[main turn 3] ')
|
||||
ctx.emit('agent/turn-end', agent, 3, { kind: 'completed' })
|
||||
expect(out.text()).toContain('\n> ')
|
||||
})
|
||||
|
||||
it('resets dim styling at turn-end if a turn ends mid-reasoning', async () => {
|
||||
const { ctx, out } = await setup()
|
||||
const agent = makeAgent('main')
|
||||
ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'reasoning-delta', index: 0, text: 'mid' })
|
||||
ctx.emit('agent/turn-end', agent, 1, { kind: 'completed' })
|
||||
expect(out.text()).toContain('\x1B[2mmid\x1B[0m')
|
||||
})
|
||||
|
||||
it('renders tool/call and tool/result session events', async () => {
|
||||
const { ctx, out } = await setup()
|
||||
const session = {} as Session
|
||||
const callEvent = {
|
||||
type: 'tool/call', seq: 1, time: 0,
|
||||
data: { turn: 1, step: 0, callId: 'c1', name: 'bash', arguments: '{"command":"ls"}' },
|
||||
} as SessionEvent
|
||||
ctx.emit('session/event', session, callEvent)
|
||||
expect(out.text()).toContain('[tool call] bash({"command":"ls"})')
|
||||
|
||||
const resultEvent = {
|
||||
type: 'tool/result', seq: 2, time: 0,
|
||||
data: { turn: 1, step: 0, callId: 'c1', content: [{ type: 'text', text: 'file.txt' }], isError: false },
|
||||
} as SessionEvent
|
||||
ctx.emit('session/event', session, resultEvent)
|
||||
expect(out.text()).toContain('[tool result] file.txt')
|
||||
})
|
||||
|
||||
it('resets dim styling when a tool/call interrupts reasoning', async () => {
|
||||
const { ctx, out } = await setup()
|
||||
const agent = makeAgent('main')
|
||||
ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'reasoning-delta', index: 0, text: 'r' })
|
||||
const session = {} as Session
|
||||
ctx.emit('session/event', session, {
|
||||
type: 'tool/call', seq: 1, time: 0,
|
||||
data: { turn: 1, step: 0, callId: 'c1', name: 'bash', arguments: '{}' },
|
||||
} as SessionEvent)
|
||||
expect(out.text()).toContain('\x1B[2mr\x1B[0m')
|
||||
})
|
||||
|
||||
it('ignores session events it does not render', async () => {
|
||||
const { ctx, out } = await setup()
|
||||
const before = out.text()
|
||||
ctx.emit('session/event', {} as Session, {
|
||||
type: 'turn/start', seq: 1, time: 0, data: { turn: 1, trigger: { kind: 'continuation' } },
|
||||
} as SessionEvent)
|
||||
expect(out.text()).toBe(before)
|
||||
})
|
||||
})
|
||||
|
||||
describe('createStdioChat input', () => {
|
||||
it('sends a typed line to an idle agent', async () => {
|
||||
const { ctx, input } = await setup()
|
||||
const agent = makeAgent('main', 'idle')
|
||||
ctx.agents.register(agent)
|
||||
input.feed('do a thing')
|
||||
await new Promise(r => setImmediate(r))
|
||||
expect(agent.sent).toEqual([[{ type: 'text', text: 'do a thing' }]])
|
||||
expect(agent.steered).toEqual([])
|
||||
})
|
||||
|
||||
it('steers a typed line into a running agent', async () => {
|
||||
const { ctx, input } = await setup()
|
||||
const agent = makeAgent('main', 'running')
|
||||
ctx.agents.register(agent)
|
||||
input.feed('steer me')
|
||||
await new Promise(r => setImmediate(r))
|
||||
expect(agent.steered).toEqual([[{ type: 'text', text: 'steer me' }]])
|
||||
expect(agent.sent).toEqual([])
|
||||
})
|
||||
|
||||
it('ignores blank lines', async () => {
|
||||
const { ctx, input } = await setup()
|
||||
const agent = makeAgent('main')
|
||||
ctx.agents.register(agent)
|
||||
input.feed(' ')
|
||||
await new Promise(r => setImmediate(r))
|
||||
expect(agent.sent).toEqual([])
|
||||
})
|
||||
|
||||
it('logs and drops a line when the target agent is not running', async () => {
|
||||
const { ctx, input } = await setup()
|
||||
const spy = vi.spyOn(ctx.logger, 'error').mockImplementation(() => {})
|
||||
input.feed('nobody home')
|
||||
await new Promise(r => setImmediate(r))
|
||||
expect(spy).toHaveBeenCalledWith('ui-stdio: agent "%s" is not running', 'main')
|
||||
})
|
||||
|
||||
it('drives the agent named in config, not a hardcoded id', async () => {
|
||||
const { ctx, input } = await setup({ welcome: 'w', agent: 'worker' })
|
||||
const agent = makeAgent('worker')
|
||||
ctx.agents.register(agent)
|
||||
input.feed('hi')
|
||||
await new Promise(r => setImmediate(r))
|
||||
expect(agent.sent).toHaveLength(1)
|
||||
})
|
||||
})
|
||||
|
||||
describe('createStdioChat EOF exit', () => {
|
||||
it('exits immediately on EOF when no work was submitted', async () => {
|
||||
const { input, exit } = await setup()
|
||||
input.finish()
|
||||
await flushExit()
|
||||
expect(exit).toHaveBeenCalledWith(0)
|
||||
})
|
||||
|
||||
it('waits for the agent to settle idle after running before exiting', async () => {
|
||||
const { ctx, input, exit } = await setup()
|
||||
const agent = makeAgent('main', 'idle')
|
||||
ctx.agents.register(agent)
|
||||
input.feed('work')
|
||||
await new Promise(r => setImmediate(r))
|
||||
input.finish()
|
||||
await new Promise(r => setImmediate(r))
|
||||
// Work submitted but no 'running' observed yet — must NOT exit.
|
||||
expect(exit).not.toHaveBeenCalled()
|
||||
// The turn starts, then settles.
|
||||
ctx.emit('agent/status', agent, 'running')
|
||||
;(agent as { status: AgentStatus }).status = 'idle'
|
||||
ctx.emit('agent/status', agent, 'idle')
|
||||
await flushExit()
|
||||
expect(exit).toHaveBeenCalledWith(0)
|
||||
})
|
||||
|
||||
it('schedules the exit only once when idle fires repeatedly', async () => {
|
||||
const { ctx, input, exit } = await setup()
|
||||
const agent = makeAgent('main', 'running')
|
||||
ctx.agents.register(agent)
|
||||
input.feed('work')
|
||||
await new Promise(r => setImmediate(r))
|
||||
ctx.emit('agent/status', agent, 'running') // sawRunning = true
|
||||
input.finish()
|
||||
await new Promise(r => setImmediate(r)) // let readline 'close' set stdinClosed
|
||||
;(agent as { status: AgentStatus }).status = 'idle'
|
||||
// Two idle signals while stdin is already closed: the first arms the timer,
|
||||
// the second must hit the already-scheduled guard, not arm a second.
|
||||
ctx.emit('agent/status', agent, 'idle')
|
||||
ctx.emit('agent/status', agent, 'idle')
|
||||
await flushExit()
|
||||
expect(exit).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('does not exit on an idle transition for a different agent', async () => {
|
||||
const { ctx, input, exit } = await setup()
|
||||
const agent = makeAgent('main', 'idle')
|
||||
ctx.agents.register(agent)
|
||||
input.feed('work')
|
||||
await new Promise(r => setImmediate(r))
|
||||
input.finish()
|
||||
const other = makeAgent('other')
|
||||
ctx.emit('agent/status', other, 'running')
|
||||
ctx.emit('agent/status', other, 'idle')
|
||||
await flushExit()
|
||||
expect(exit).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('does not exit while a turn is still running at EOF', async () => {
|
||||
const { ctx, input, exit } = await setup()
|
||||
const agent = makeAgent('main', 'idle')
|
||||
ctx.agents.register(agent)
|
||||
input.feed('work')
|
||||
await new Promise(r => setImmediate(r))
|
||||
ctx.emit('agent/status', agent, 'running')
|
||||
;(agent as { status: AgentStatus }).status = 'running'
|
||||
input.finish()
|
||||
// sawRunning is true, but the agent is still running — the idle gate holds.
|
||||
ctx.emit('agent/status', agent, 'idle') // a stale/duplicate signal while status stays 'running'
|
||||
await flushExit()
|
||||
expect(exit).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
describe('createStdioChat disposal (HMR safety)', () => {
|
||||
it('never exits the process when EOF arrives after fiber dispose', async () => {
|
||||
const { fiber, input, exit } = await setup()
|
||||
await fiber.dispose()
|
||||
// A late EOF after disposal (reader.close() also fires 'close') must not exit.
|
||||
input.finish()
|
||||
await flushExit()
|
||||
expect(exit).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('cancels a scheduled exit if disposed within the flush window', async () => {
|
||||
const { fiber, input, exit } = await setup()
|
||||
// EOF with no work submitted schedules the 200ms flush-then-exit timer.
|
||||
input.finish()
|
||||
await new Promise(r => setImmediate(r))
|
||||
expect(exit).not.toHaveBeenCalled() // not yet — still inside the window
|
||||
// Dispose BEFORE the timer fires: the tracked handle must be cleared.
|
||||
await fiber.dispose()
|
||||
await flushExit()
|
||||
expect(exit).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('stops handling input after dispose', async () => {
|
||||
const { ctx, fiber, input } = await setup()
|
||||
const agent = makeAgent('main')
|
||||
ctx.agents.register(agent)
|
||||
await fiber.dispose()
|
||||
// The readline interface is closed on dispose; a late line reaches no handler.
|
||||
input.feed('too late')
|
||||
await new Promise(r => setImmediate(r))
|
||||
expect(agent.sent).toEqual([])
|
||||
})
|
||||
|
||||
it('removes the agent/status listener on dispose', async () => {
|
||||
const { ctx, fiber, input, exit } = await setup()
|
||||
const agent = makeAgent('main', 'idle')
|
||||
ctx.agents.register(agent)
|
||||
input.feed('work')
|
||||
await new Promise(r => setImmediate(r))
|
||||
await fiber.dispose()
|
||||
// After dispose, status transitions must neither throw nor schedule an exit
|
||||
// (the listener and the EOF-exit path are both torn down).
|
||||
expect(() => {
|
||||
ctx.emit('agent/status', agent, 'running')
|
||||
ctx.emit('agent/status', agent, 'idle')
|
||||
}).not.toThrow()
|
||||
await flushExit()
|
||||
expect(exit).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
16
packages/ui-stdio/tsconfig.json
Normal file
16
packages/ui-stdio/tsconfig.json
Normal file
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"extends": "../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib"
|
||||
},
|
||||
"include": ["src"],
|
||||
"references": [
|
||||
{ "path": "../../vendor/cosmokit" },
|
||||
{ "path": "../../vendor/cordis" },
|
||||
{ "path": "../../vendor/schemastery" },
|
||||
{ "path": "../agent" },
|
||||
{ "path": "../llm" },
|
||||
{ "path": "../session" }
|
||||
]
|
||||
}
|
||||
31
pnpm-lock.yaml
generated
31
pnpm-lock.yaml
generated
@@ -230,6 +230,18 @@ importers:
|
||||
specifier: ^4.0.0-rc.6
|
||||
version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4)
|
||||
|
||||
packages/llm-replay:
|
||||
devDependencies:
|
||||
'@deepseek-ai/dsh-llm':
|
||||
specifier: workspace:^
|
||||
version: link:../llm
|
||||
'@deepseek-ai/dsh-session':
|
||||
specifier: workspace:^
|
||||
version: link:../session
|
||||
cordis:
|
||||
specifier: ^4.0.0-rc.6
|
||||
version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4)
|
||||
|
||||
packages/session:
|
||||
devDependencies:
|
||||
'@deepseek-ai/dsh-llm':
|
||||
@@ -334,6 +346,25 @@ importers:
|
||||
specifier: ^4.0.0-rc.6
|
||||
version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4)
|
||||
|
||||
packages/ui-stdio:
|
||||
dependencies:
|
||||
schemastery:
|
||||
specifier: ^3.18.0
|
||||
version: 3.18.0
|
||||
devDependencies:
|
||||
'@deepseek-ai/dsh-agent':
|
||||
specifier: workspace:^
|
||||
version: link:../agent
|
||||
'@deepseek-ai/dsh-llm':
|
||||
specifier: workspace:^
|
||||
version: link:../llm
|
||||
'@deepseek-ai/dsh-session':
|
||||
specifier: workspace:^
|
||||
version: link:../session
|
||||
cordis:
|
||||
specifier: ^4.0.0-rc.6
|
||||
version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4)
|
||||
|
||||
vendor/cordis:
|
||||
dependencies:
|
||||
'@cordisjs/plugin-include':
|
||||
|
||||
@@ -20,6 +20,8 @@ const packages = [
|
||||
'packages/tool-bash',
|
||||
'packages/invariants',
|
||||
'packages/acp',
|
||||
'packages/ui-stdio',
|
||||
'packages/llm-replay',
|
||||
]
|
||||
|
||||
const root = resolve(import.meta.dirname, '..')
|
||||
|
||||
@@ -49,7 +49,9 @@
|
||||
"@deepseek-ai/dsh-bash-local": ["./packages/bash-local/src"],
|
||||
"@deepseek-ai/dsh-tool-bash": ["./packages/tool-bash/src"],
|
||||
"@deepseek-ai/dsh-invariants": ["./packages/invariants/src"],
|
||||
"@deepseek-ai/dsh-acp": ["./packages/acp/src"]
|
||||
"@deepseek-ai/dsh-acp": ["./packages/acp/src"],
|
||||
"@deepseek-ai/dsh-ui-stdio": ["./packages/ui-stdio/src"],
|
||||
"@deepseek-ai/dsh-llm-replay": ["./packages/llm-replay/src"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -25,6 +25,8 @@
|
||||
{ "path": "./packages/bash-local" },
|
||||
{ "path": "./packages/tool-bash" },
|
||||
{ "path": "./packages/invariants" },
|
||||
{ "path": "./packages/acp" }
|
||||
{ "path": "./packages/acp" },
|
||||
{ "path": "./packages/ui-stdio" },
|
||||
{ "path": "./packages/llm-replay" }
|
||||
]
|
||||
}
|
||||
|
||||
@@ -31,7 +31,9 @@
|
||||
"@deepseek-ai/dsh-bash-local": ["./packages/bash-local/src"],
|
||||
"@deepseek-ai/dsh-tool-bash": ["./packages/tool-bash/src"],
|
||||
"@deepseek-ai/dsh-invariants": ["./packages/invariants/src"],
|
||||
"@deepseek-ai/dsh-acp": ["./packages/acp/src"]
|
||||
"@deepseek-ai/dsh-acp": ["./packages/acp/src"],
|
||||
"@deepseek-ai/dsh-ui-stdio": ["./packages/ui-stdio/src"],
|
||||
"@deepseek-ai/dsh-llm-replay": ["./packages/llm-replay/src"]
|
||||
}
|
||||
},
|
||||
"include": ["packages/*/src", "packages/*/tests", "examples", "scripts"]
|
||||
|
||||
Reference in New Issue
Block a user