mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
Merge remote-tracking branch 'origin/master' into codex/pr332-merge-master-20260719
# Conflicts: # docs/cookbook/extension-cookbook.i18n.yaml
This commit is contained in:
Tianyi Cui
66
examples/repl-agent/README.md
Normal file
66
examples/repl-agent/README.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# repl-agent
|
||||
|
||||
The repl-agent wiring: DeepSeek V4 + the `read`/`write`/`edit` filesystem tools + the bash tool suite + subagent delegation + `todo_write` + readline chat + JSONL persistence, loaded from `cordis.yml`. The sibling [`tui-agent`](../tui-agent/README.md) fixes the same agent composition to the full-screen terminal front door.
|
||||
|
||||
## Run it
|
||||
|
||||
```sh
|
||||
# repo root .env (gitignored) or exported env:
|
||||
# DEEPSEEK_API_KEY=sk-…
|
||||
# DEEPSEEK_BASE_URL=https://… # optional; defaults to the public API
|
||||
pnpm run demo:repl
|
||||
```
|
||||
|
||||
Type a coding task. The agent works through the `read`/`write`/`edit` filesystem tools for ordinary file operations and `bash` (+ the generic `task_output` / `task_list` / `task_kill` for background tasks) for shell commands, searches, and test runs, each in a fresh `bash -c` (the system prompt tells the model to pass `workdir` instead of `cd`). Both the fs tools and bash resolve relative paths against the session workspace. It can also delegate with `subagent`/`subagent_fork` and track multi-step work with `todo_write`.
|
||||
|
||||
The REPL renders reasoning, tool calls/results, and the latest todo list as line-oriented output suitable for terminals and pipes. Use `pnpm run demo:tui` for the interactive Markdown/card interface.
|
||||
|
||||
### Resuming a prior session
|
||||
|
||||
Each run starts a fresh session by default (its event log lands under `./.sessions/`). To **continue** a previous conversation, set `RESUME_SESSION_ID` to that session's id — the `main` agent then rehydrates the persisted log instead of starting fresh, so the model sees the earlier turns as history:
|
||||
|
||||
```sh
|
||||
RESUME_SESSION_ID=<prior-session-id> pnpm run demo:repl
|
||||
```
|
||||
|
||||
The id is wired through `cordis.yml` (`resumeSessionId: !!js process.env.RESUME_SESSION_ID`); unset, the agent starts a new session. A missing or unreadable id starts no agent and emits `agent-loop/config-start-failed`: the TUI prints the failure and exits nonzero, while readline reports any dropped queued input and allows piped EOF to finish. Unset it or choose an existing session id.
|
||||
|
||||
## Code Mode
|
||||
|
||||
[`code-mode.cordis.yml`](code-mode.cordis.yml) overlays the same tree with the worker-thread runtime and `tools: { mode: code }`. The model receives one `run_code` transport plus a generated TypeScript SDK for the visible tools; only program output returns to model context. Use `mode: both` to expose native calls alongside `run_code`. See the [Code Mode RFC](../../docs/rfc/implemented/feature/2026-06-15-code-mode.md) for the execution contract.
|
||||
|
||||
```sh
|
||||
pnpm run demo:code-mode # this overlay under the REPL (default UI)
|
||||
pnpm run demo:code-mode acp # the acp-agent example's same-shaped overlay
|
||||
```
|
||||
|
||||
Try a task that spans several tool calls, e.g.:
|
||||
|
||||
> Count the lines of every `*.md` file under docs/ and write the three largest to summary.txt.
|
||||
|
||||
and watch the transcript: one `run_code` call, a program looping over tools, and a result the model curated instead of five round-trips of raw tool output.
|
||||
|
||||
## What each leaf entry demonstrates
|
||||
|
||||
This example is a thin leaf `cordis.yml`: it picks the swappable backends, loads one app package, and adds product tools that are intentionally outside the shared spine. The spine (sessions, system-prompt, tools, agents, invariants, `agent-loop`) and the front-door cluster (JSONL persistence, the selected terminal channel, the pre-created `main` agent) live inside the [`@deepseek-ai/dsh-stdio-demo`](../../packages/examples/stdio-demo) app and the [`@deepseek-ai/dsh-agent-spine-demo`](../../packages/examples/agent-spine-demo) bundle it loads; the leaf wires the backends and model-facing optional tools:
|
||||
|
||||
| Entry | Demonstrates |
|
||||
|---|---|
|
||||
| `hmr` (`@cordisjs/plugin-hmr`) | the dev/demo edit-reload loop — a **leaf** entry (not baked into the app) because it is Loader-only and needs `node --expose-internals`, which `demo:repl` passes |
|
||||
| `llm-deepseek` | real `LlmAdapter` via config (`!!js process.env.…` secrets); swap one line to `@deepseek-ai/dsh-llm-pi-ai` for the library-backed twin |
|
||||
| `bash` (`dsh-bash-local`) | the executor implementation — the swappable half of the bash seam. The model-facing `bash` schema (`tool-bash`) and generic `task_*` controls (`tool-tasks`) come from `dsh-agent-spine-demo`, so only the executor is a leaf choice |
|
||||
| `stdio-agent` (`@deepseek-ai/dsh-stdio-demo`) | the app bundle: the agent-spine demo + JSONL persistence + the configured terminal channel + a pre-created `main` agent. This leaf fixes `ui.mode` to `readline`; `tui-agent` owns the corresponding TUI leaf |
|
||||
| `subagent`, `subagent-spawn`, `subagent-fork` | the subagent provider registry plus the two in-process backends: a fresh child and a child seeded with the parent's completed-turn prefix |
|
||||
| `tool-subagent`, `tool-subagent-fork` | two model-facing `dsh-tool-subagent` loads, each bound to a different provider and exposed under a distinct tool name (`subagent`, `subagent_fork`) |
|
||||
| `tool-todo` | the model-facing `todo_write` tool; writes the whole task list to the session log and renders as a persistent TUI plan or readline checklist |
|
||||
| `fs-local`, `fs-policy`, `tool-fs` | the filesystem stack: the local `ctx.fs` provider, the read-before-write/edit policy gate (on the `fs/*` event gate), and the model-facing `read`/`write`/`edit` tools. Relative paths resolve against the session workspace |
|
||||
|
||||
## End-to-end tests (`pnpm run test:e2e`, key-gated)
|
||||
|
||||
- `tests/full-loop.e2e.ts` — the canary: real model runs `echo e2e-ok` through the real bash tool; asserts `tool/call`/`tool/result` session events and the final answer.
|
||||
- `tests/coding-task.e2e.ts` — the swebench-style smoke: a temp dir holds `add.js` (with `a - b` where `a + b` belongs) and a failing `add.test.js`; the agent must fix the bug and verify. The test re-runs `node add.test.js` ITSELF and inspects the files — agent claims are not trusted.
|
||||
- `tests/resume.e2e.ts` — durable continuity across processes: run 1 tells the real model a secret code and persists the turn to a temp JSONL root, then the whole context is disposed; run 2 is a fresh context over the same root that RESUMES the session id and asks the model to recall the code. The recall can only come from the rehydrated log.
|
||||
- `tests/compaction.e2e.ts` — the compaction smoke: a real multi-step bash task runs with a deliberately tiny context window so the auto-compaction listener fires MID-SESSION. Verifies the WORLD — a `compact/start…end` pair landed in the real log, the surface shrank (a replace node shadowed older nodes), and the agent still produced a correct final answer after compaction.
|
||||
- `tests/todo-write.e2e.ts` — a real model drives the real `todo_write` tool and the test verifies the resulting `todo/write` session event.
|
||||
|
||||
These self-skip without `DEEPSEEK_API_KEY`. `tests/code-mode.e2e.ts` is the with-key Code Mode proof — a real model, a two-tool task, asserting the wire tool list was exactly `[run_code]`, the `tool/code-dispatch` events landed under the parent call, and the curated answer came back. The keyless boot smokes run in the default e2e gate: `tests/keyless-smoke.e2e.ts` (the full real tree, dummy key, no prompt → no model call) and `tests/code-mode-keyless-smoke.e2e.ts` (the same guard for the Code Mode overlay).
|
||||
33
examples/repl-agent/code-mode.cordis.yml
Normal file
33
examples/repl-agent/code-mode.cordis.yml
Normal file
@@ -0,0 +1,33 @@
|
||||
# Code Mode adds `ctx.codeRuntime` and changes the registry to one wire tool,
|
||||
# `run_code`, plus a generated SDK for bash/read/write/edit/subagent/todo_write.
|
||||
# `demo:code-mode` selects this overlay; the ACP example has the same UI-specific
|
||||
# shape. A config patch replaces the whole app config, so unchanged base fields
|
||||
# are restated; only `tools`, `welcome`, and the persona's second paragraph differ.
|
||||
- id: base
|
||||
name: '@cordisjs/plugin-include'
|
||||
config:
|
||||
path: ./cordis.yml
|
||||
patches:
|
||||
- id: stdio-agent
|
||||
name: '@deepseek-ai/dsh-stdio-demo'
|
||||
config:
|
||||
provider: deepseek
|
||||
model: deepseek-v4-flash
|
||||
resumeSessionId: !!js process.env.RESUME_SESSION_ID
|
||||
persistenceRoot: './.sessions'
|
||||
workspaceContext:
|
||||
maxBytes: 65536
|
||||
tools:
|
||||
mode: code
|
||||
welcome: 'code-mode agent ready. Give it a multi-tool task.'
|
||||
ui:
|
||||
mode: readline
|
||||
persona: |
|
||||
You are a coding agent powered by the {{model}} model.
|
||||
|
||||
You work by writing TypeScript programs for run_code: batch related
|
||||
tool work into one program, loop and branch where it helps, and print
|
||||
or return ONLY the findings that matter.
|
||||
- insert:
|
||||
- id: code-runtime
|
||||
name: '@deepseek-ai/dsh-code-runtime-worker'
|
||||
88
examples/repl-agent/composition.md
Normal file
88
examples/repl-agent/composition.md
Normal file
@@ -0,0 +1,88 @@
|
||||
<!-- Generated by scripts/gen-doc-graphs.ts - do not edit by hand.
|
||||
Run `pnpm run gen-doc-graphs` to regenerate. -->
|
||||
|
||||
# REPL Agent App Composition
|
||||
|
||||
The REPL agent demo adds the real DeepSeek adapter, filesystem tools, todo_write, compaction, and both subagent transports on top of the stdio app package.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
cfg["examples/repl-agent<br/>cordis.yml"]
|
||||
plugin_repl_hmr["hmr<br/>@cordisjs/plugin-hmr"]
|
||||
cfg --> plugin_repl_hmr
|
||||
plugin_repl_llm_deepseek["llm-deepseek<br/>@deepseek-ai/dsh-llm-deepseek"]
|
||||
cfg --> plugin_repl_llm_deepseek
|
||||
plugin_repl_bash["bash<br/>@deepseek-ai/dsh-bash-local"]
|
||||
cfg --> plugin_repl_bash
|
||||
plugin_repl_stdio_agent["stdio-agent<br/>@deepseek-ai/dsh-stdio-demo"]
|
||||
cfg --> plugin_repl_stdio_agent
|
||||
plugin_repl_stdio_agent --> bundle_agent_core["@deepseek-ai/dsh-agent-spine-demo"]
|
||||
plugin_repl_stdio_agent --> bundle_jsonl["@deepseek-ai/dsh-session-persistence-jsonl"]
|
||||
plugin_repl_stdio_agent --> frontdoor_stdio["@deepseek-ai/dsh-stdio<br/>pre-created main agent"]
|
||||
bundle_agent_core --> spine_llm["ctx.llm"]
|
||||
bundle_agent_core --> spine_sessions["ctx.sessions"]
|
||||
bundle_agent_core --> spine_tools["ctx.tools + tool-bash"]
|
||||
bundle_agent_core --> spine_loop["ctx.agents + ctx.agentLoop"]
|
||||
plugin_repl_token_meter["token-meter<br/>@deepseek-ai/dsh-token-meter"]
|
||||
cfg --> plugin_repl_token_meter
|
||||
plugin_repl_compact_basic["compact-basic<br/>@deepseek-ai/dsh-compact-basic"]
|
||||
cfg --> plugin_repl_compact_basic
|
||||
plugin_repl_subagent["subagent<br/>@deepseek-ai/dsh-subagent"]
|
||||
cfg --> plugin_repl_subagent
|
||||
plugin_repl_subagent_spawn["subagent-spawn<br/>@deepseek-ai/dsh-subagent-spawn"]
|
||||
cfg --> plugin_repl_subagent_spawn
|
||||
plugin_repl_subagent_fork["subagent-fork<br/>@deepseek-ai/dsh-subagent-fork"]
|
||||
cfg --> plugin_repl_subagent_fork
|
||||
plugin_repl_tool_subagent["tool-subagent<br/>@deepseek-ai/dsh-tool-subagent"]
|
||||
cfg --> plugin_repl_tool_subagent
|
||||
plugin_repl_tool_subagent_fork["tool-subagent-fork<br/>@deepseek-ai/dsh-tool-subagent"]
|
||||
cfg --> plugin_repl_tool_subagent_fork
|
||||
plugin_repl_workflow_workerthread["workflow-workerthread<br/>@deepseek-ai/dsh-workflow-workerthread"]
|
||||
cfg --> plugin_repl_workflow_workerthread
|
||||
plugin_repl_tool_workflow["tool-workflow<br/>@deepseek-ai/dsh-tool-workflow"]
|
||||
cfg --> plugin_repl_tool_workflow
|
||||
plugin_repl_tool_todo["tool-todo<br/>@deepseek-ai/dsh-tool-todo"]
|
||||
cfg --> plugin_repl_tool_todo
|
||||
plugin_repl_fs_local["fs-local<br/>@deepseek-ai/dsh-fs-local"]
|
||||
cfg --> plugin_repl_fs_local
|
||||
plugin_repl_fs_policy["fs-policy<br/>@deepseek-ai/dsh-fs-policy"]
|
||||
cfg --> plugin_repl_fs_policy
|
||||
plugin_repl_tool_fs["tool-fs<br/>@deepseek-ai/dsh-tool-fs"]
|
||||
cfg --> plugin_repl_tool_fs
|
||||
plugin_repl_tool_fs_search["tool-fs-search<br/>@deepseek-ai/dsh-tool-fs-search"]
|
||||
cfg --> plugin_repl_tool_fs_search
|
||||
plugin_repl_timeout_policy["timeout-policy<br/>@deepseek-ai/dsh-timeout-policy"]
|
||||
cfg --> plugin_repl_timeout_policy
|
||||
plugin_repl_spill_local["spill-local<br/>@deepseek-ai/dsh-spill-local"]
|
||||
cfg --> plugin_repl_spill_local
|
||||
plugin_repl_spill_policy["spill-policy<br/>@deepseek-ai/dsh-spill-policy"]
|
||||
cfg --> plugin_repl_spill_policy
|
||||
```
|
||||
|
||||
| Plugin id | Package / module |
|
||||
| --- | --- |
|
||||
| `hmr` | `@cordisjs/plugin-hmr` |
|
||||
| `llm-deepseek` | `@deepseek-ai/dsh-llm-deepseek` |
|
||||
| `bash` | `@deepseek-ai/dsh-bash-local` |
|
||||
| `stdio-agent` | `@deepseek-ai/dsh-stdio-demo` |
|
||||
| `token-meter` | `@deepseek-ai/dsh-token-meter` |
|
||||
| `compact-basic` | `@deepseek-ai/dsh-compact-basic` |
|
||||
| `subagent` | `@deepseek-ai/dsh-subagent` |
|
||||
| `subagent-spawn` | `@deepseek-ai/dsh-subagent-spawn` |
|
||||
| `subagent-fork` | `@deepseek-ai/dsh-subagent-fork` |
|
||||
| `tool-subagent` | `@deepseek-ai/dsh-tool-subagent` |
|
||||
| `tool-subagent-fork` | `@deepseek-ai/dsh-tool-subagent` |
|
||||
| `workflow-workerthread` | `@deepseek-ai/dsh-workflow-workerthread` |
|
||||
| `tool-workflow` | `@deepseek-ai/dsh-tool-workflow` |
|
||||
| `tool-todo` | `@deepseek-ai/dsh-tool-todo` |
|
||||
| `fs-local` | `@deepseek-ai/dsh-fs-local` |
|
||||
| `fs-policy` | `@deepseek-ai/dsh-fs-policy` |
|
||||
| `tool-fs` | `@deepseek-ai/dsh-tool-fs` |
|
||||
| `tool-fs-search` | `@deepseek-ai/dsh-tool-fs-search` |
|
||||
| `timeout-policy` | `@deepseek-ai/dsh-timeout-policy` |
|
||||
| `spill-local` | `@deepseek-ai/dsh-spill-local` |
|
||||
| `spill-policy` | `@deepseek-ai/dsh-spill-policy` |
|
||||
|
||||
Source config: [`examples/repl-agent/cordis.yml`](cordis.yml).
|
||||
|
||||
Maintenance mode: hybrid: the leaf plugin list is parsed from its `cordis.yml`; app package expansion is curated from package source.
|
||||
138
examples/repl-agent/cordis.yml
Normal file
138
examples/repl-agent/cordis.yml
Normal file
@@ -0,0 +1,138 @@
|
||||
# Readline coding REPL with swappable DeepSeek and local-bash backends.
|
||||
# `dsh-stdio-demo` supplies the agent spine, workspace instructions, generic
|
||||
# task controls, JSONL persistence, the line-oriented front door, and `main`.
|
||||
# HMR remains a leaf because it requires Loader internals; `demo:repl` passes
|
||||
# `--expose-internals`. The app bin loads the gitignored root `.env`; this file
|
||||
# reads `DEEPSEEK_API_KEY` and optional `DEEPSEEK_BASE_URL` through `!!js`.
|
||||
|
||||
# Hot-module reload for the dev/demo loop (needs `node --expose-internals`).
|
||||
- id: hmr
|
||||
name: '@cordisjs/plugin-hmr'
|
||||
config:
|
||||
root: ['.']
|
||||
|
||||
# The native DeepSeek adapter.
|
||||
- id: llm-deepseek
|
||||
name: '@deepseek-ai/dsh-llm-deepseek'
|
||||
config:
|
||||
apiKey: !!js process.env.DEEPSEEK_API_KEY
|
||||
baseURL: !!js process.env.DEEPSEEK_BASE_URL
|
||||
|
||||
# Local executor for the app bundle's bash tool.
|
||||
- id: bash
|
||||
name: '@deepseek-ai/dsh-bash-local'
|
||||
config:
|
||||
timeoutMs: 60000
|
||||
|
||||
# The app bundle pre-creates the REPL's `main` agent.
|
||||
- id: stdio-agent
|
||||
name: '@deepseek-ai/dsh-stdio-demo'
|
||||
config:
|
||||
provider: deepseek
|
||||
model: deepseek-v4-flash
|
||||
# Set RESUME_SESSION_ID to continue a prior persisted session (the ids live
|
||||
# under ./.sessions); unset starts a fresh session each run.
|
||||
resumeSessionId: !!js process.env.RESUME_SESSION_ID
|
||||
persistenceRoot: './.sessions'
|
||||
workspaceContext:
|
||||
maxBytes: 65536
|
||||
welcome: 'agent REPL ready. Give it a coding task.'
|
||||
ui:
|
||||
mode: readline
|
||||
# Keep the persona to identity and behavior; tool plugins own tool guidance.
|
||||
# The loop resolves {{model}} from this agent's configuration.
|
||||
persona: |
|
||||
You are a coding agent powered by the {{model}} model.
|
||||
|
||||
Verify your work by running the code or tests. Keep answers brief and
|
||||
factual.
|
||||
|
||||
# Replay-aware request pressure with one service-wide context window.
|
||||
- id: token-meter
|
||||
name: '@deepseek-ai/dsh-token-meter'
|
||||
|
||||
# Summarize an older range after measured pressure or a canonical provider overflow.
|
||||
# Service-wide policy provides pressure, retention, and one overflow-retry default.
|
||||
- id: compact-basic
|
||||
name: '@deepseek-ai/dsh-compact-basic'
|
||||
|
||||
# Expose fresh-child `spawn` and completed-prefix `fork` through independent
|
||||
# in-process backends. Each tool instance needs a distinct `toolName`; the registry
|
||||
# rejects duplicates. These leaves follow the app because it provides `ctx.agents` and `ctx.tools`.
|
||||
- id: subagent
|
||||
name: '@deepseek-ai/dsh-subagent'
|
||||
|
||||
- id: subagent-spawn
|
||||
name: '@deepseek-ai/dsh-subagent-spawn'
|
||||
config:
|
||||
providerName: spawn
|
||||
|
||||
- id: subagent-fork
|
||||
name: '@deepseek-ai/dsh-subagent-fork'
|
||||
config:
|
||||
providerName: fork
|
||||
|
||||
- id: tool-subagent
|
||||
name: '@deepseek-ai/dsh-tool-subagent'
|
||||
config:
|
||||
provider: spawn
|
||||
toolName: subagent
|
||||
|
||||
- id: tool-subagent-fork
|
||||
name: '@deepseek-ai/dsh-tool-subagent'
|
||||
config:
|
||||
provider: fork
|
||||
toolName: subagent_fork
|
||||
|
||||
|
||||
# The worker-thread workflow engine fans a model-written JavaScript script's
|
||||
# `agent()` calls out through the spawn backend; the adjacent tool exposes it to the model.
|
||||
- id: workflow-workerthread
|
||||
name: '@deepseek-ai/dsh-workflow-workerthread'
|
||||
config:
|
||||
provider: spawn
|
||||
|
||||
- id: tool-workflow
|
||||
name: '@deepseek-ai/dsh-tool-workflow'
|
||||
# `todo_write` replaces the logged whole list and renders as a stdio checklist or ACP plan.
|
||||
- id: tool-todo
|
||||
name: '@deepseek-ai/dsh-tool-todo'
|
||||
|
||||
# Policy loads before the model-facing filesystem tools so writes and edits require
|
||||
# an observed file. This single-session app resolves relative paths from the process cwd.
|
||||
- id: fs-local
|
||||
name: '@deepseek-ai/dsh-fs-local'
|
||||
config:
|
||||
cwd: !!js process.cwd()
|
||||
|
||||
- id: fs-policy
|
||||
name: '@deepseek-ai/dsh-fs-policy'
|
||||
|
||||
- id: tool-fs
|
||||
name: '@deepseek-ai/dsh-tool-fs'
|
||||
|
||||
# Bash-backed discovery tools (glob/grep): fixed ripgrep commands through the
|
||||
# local bash executor above — not ctx.fs. Capped results save the complete
|
||||
# formatted list through the spill backend below (ctx.spillStore, optional).
|
||||
- id: tool-fs-search
|
||||
name: '@deepseek-ai/dsh-tool-fs-search'
|
||||
|
||||
# The tool-call timeout enforcer: arms each declared ToolDefinition.timeoutMs
|
||||
# (the search tools above declare 30s) as a deadline on exec.signal. Without
|
||||
# it a declared budget is advisory and only the bash executor's own timeout
|
||||
# backstop applies.
|
||||
- id: timeout-policy
|
||||
name: '@deepseek-ai/dsh-timeout-policy'
|
||||
|
||||
# Tool-output spill stack: a local backend that saves oversized tool text under
|
||||
# a private session-scoped dir, and the tools/post-execute policy that replaces
|
||||
# an over-budget plain-text result with a preview + the spill locator/retrieval
|
||||
# hint. A leaf pair after the app (needs ctx.tools). The policy is a no-op until
|
||||
# a tool returns more than maxInlineBytes of plain text.
|
||||
- id: spill-local
|
||||
name: '@deepseek-ai/dsh-spill-local'
|
||||
|
||||
- id: spill-policy
|
||||
name: '@deepseek-ai/dsh-spill-policy'
|
||||
config:
|
||||
maxInlineBytes: 50000
|
||||
7
examples/repl-agent/package.json
Normal file
7
examples/repl-agent/package.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"name": "repl-agent-example",
|
||||
"private": true,
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"description": "Runnable demo: an agent REPL UI with DeepSeek V4 and coding tools"
|
||||
}
|
||||
27
examples/repl-agent/tests/code-mode-keyless-smoke.e2e.ts
Normal file
27
examples/repl-agent/tests/code-mode-keyless-smoke.e2e.ts
Normal file
@@ -0,0 +1,27 @@
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { LOADER_SMOKE_TEST_TIMEOUT_MS, runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke'
|
||||
|
||||
/**
|
||||
* Keyless Loader-path smoke for the Code Mode overlay: boot the real include
|
||||
* tree through stdio-agent and `code-mode.cordis.yml`, then close stdin without
|
||||
* a prompt and assert the banner. No model or `run_code` turn runs.
|
||||
*/
|
||||
|
||||
const binScript = fileURLToPath(new URL('../../../packages/examples/stdio-demo/src/bin.ts', import.meta.url))
|
||||
const configPath = fileURLToPath(new URL('../code-mode.cordis.yml', import.meta.url))
|
||||
const tsconfigPath = fileURLToPath(new URL('../../../tsconfig.json', import.meta.url))
|
||||
|
||||
describe('code-mode overlay keyless smoke (real code-mode.cordis.yml via the Loader)', () => {
|
||||
it('boots the Code Mode plugin tree, prints its banner, and exits cleanly on EOF', async () => {
|
||||
const { stdout } = await runLoaderSmoke({
|
||||
label: 'code-mode overlay',
|
||||
tempDirPrefix: 'code-mode-smoke-',
|
||||
binScript,
|
||||
configPath,
|
||||
tsconfigPath,
|
||||
env: { DEEPSEEK_API_KEY: 'keyless-smoke-no-call' },
|
||||
})
|
||||
expect(stdout).toContain('code-mode agent ready.')
|
||||
}, LOADER_SMOKE_TEST_TIMEOUT_MS)
|
||||
})
|
||||
169
examples/repl-agent/tests/code-mode.e2e.ts
Normal file
169
examples/repl-agent/tests/code-mode.e2e.ts
Normal file
@@ -0,0 +1,169 @@
|
||||
import { mkdtemp, mkdir, readFile, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry, { RUN_CODE_NAME } from '@deepseek-ai/dsh-tools'
|
||||
import AgentRegistry, { type Agent } from '@deepseek-ai/dsh-agent'
|
||||
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local'
|
||||
import * as ToolBash from '@deepseek-ai/dsh-tool-bash'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import { WorkerCodeRuntime } from '@deepseek-ai/dsh-code-runtime-worker'
|
||||
import LocalFileSystem from '@deepseek-ai/dsh-fs-local'
|
||||
import * as ToolFs from '@deepseek-ai/dsh-tool-fs'
|
||||
import * as WorkspaceContext from '@deepseek-ai/dsh-workspace-context'
|
||||
|
||||
/**
|
||||
* With-key Code Mode proof: a real model receives only `run_code`, composes two
|
||||
* sub-calls, writes a file, and returns curated output while the log records
|
||||
* each `tool/code-dispatch`. The keyless Loader smoke is in the sibling test.
|
||||
*/
|
||||
|
||||
const PERSONA = 'You are a coding agent. You work by writing TypeScript programs for run_code: '
|
||||
+ 'batch related tool work into one program and print or return ONLY the findings that matter.'
|
||||
const WORKSPACE_PROBE = 'dragonfruit-8675309'
|
||||
|
||||
let ctx: Context | undefined
|
||||
let workdir: string | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
// Always dispose, even on failure/retry/timeout: agent-loop teardown stops
|
||||
// the loop, the executor kills stray processes, and the code runtime's
|
||||
// dispose awaits worker exits.
|
||||
await ctx?.fiber.dispose()
|
||||
ctx = undefined
|
||||
if (workdir !== undefined) await rm(workdir, { recursive: true, force: true })
|
||||
workdir = undefined
|
||||
})
|
||||
|
||||
async function codeModeHarness(cwd: string): Promise<Context> {
|
||||
const harness = new Context()
|
||||
await harness.plugin(LlmService)
|
||||
await harness.plugin(SessionStore)
|
||||
await harness.plugin(SystemPrompt, { persona: PERSONA })
|
||||
await harness.plugin(ToolRegistry, { mode: 'code' })
|
||||
await harness.plugin(AgentRegistry)
|
||||
await harness.plugin(AgentLoop, { agents: [] })
|
||||
await harness.plugin(LlmDeepSeek)
|
||||
await harness.plugin(LocalBashExecutor, { cwd, timeoutMs: 30_000 })
|
||||
await harness.plugin(ToolBash)
|
||||
await harness.plugin(WorkerCodeRuntime, {})
|
||||
return harness
|
||||
}
|
||||
|
||||
async function workspaceCodeModeHarness(): Promise<Context> {
|
||||
const harness = new Context()
|
||||
await harness.plugin(LlmService)
|
||||
await harness.plugin(SessionStore)
|
||||
await harness.plugin(SystemPrompt, { persona: PERSONA })
|
||||
await harness.plugin(ToolRegistry, { mode: 'code' })
|
||||
await harness.plugin(AgentRegistry)
|
||||
await harness.plugin(LocalFileSystem, { cwd: '/' })
|
||||
await harness.plugin(ToolFs)
|
||||
await harness.plugin(WorkspaceContext, { maxBytes: 65536 })
|
||||
await harness.plugin(AgentLoop, { agents: [] })
|
||||
await harness.plugin(LlmDeepSeek, { models: [{ id: 'deepseek-v4-flash' }] })
|
||||
await harness.plugin(WorkerCodeRuntime, {})
|
||||
return harness
|
||||
}
|
||||
|
||||
function waitForIdle(harness: Context, agent: Agent): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
const dispose = harness.on('agent/status', (subject, status) => {
|
||||
if (subject === agent && status === 'idle') {
|
||||
dispose()
|
||||
resolve()
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('Code Mode: real model writes a program over real tools', () => {
|
||||
it('collapses the wire tool list to [run_code], bridges sub-calls, and returns curated output', async () => {
|
||||
workdir = await mkdtemp(join(tmpdir(), 'dsh-code-mode-e2e-'))
|
||||
ctx = await codeModeHarness(workdir)
|
||||
const agent = ctx.agentLoop.create(SessionId('e2e-code-mode'), { provider: 'deepseek', model: 'deepseek-v4-flash' })
|
||||
|
||||
agent.send([{
|
||||
type: 'text',
|
||||
text: 'Using one run_code program: run `echo alpha-7` with the bash tool, run `echo beta-9` with the bash tool, '
|
||||
+ 'then write both outputs joined by a plus sign into combined.txt (bash heredoc or redirect), '
|
||||
+ 'and return only the joined string.',
|
||||
}])
|
||||
await waitForIdle(ctx, agent)
|
||||
const events: SessionEvent[] = [...agent.session.events]
|
||||
|
||||
// The wire contract: every request this session made offered EXACTLY ONE
|
||||
// tool — run_code (the logged header snapshots the assembled list).
|
||||
const headers = events.filter(event => event.type === 'request/header')
|
||||
expect(headers.length).toBeGreaterThan(0)
|
||||
for (const header of headers) {
|
||||
expect(header.data.header.tools?.map(tool => tool.name)).toEqual([RUN_CODE_NAME])
|
||||
}
|
||||
// The model actually went through run_code…
|
||||
const calls = events.filter(event => event.type === 'tool/call')
|
||||
expect(calls.length).toBeGreaterThan(0)
|
||||
expect(calls.every(event => event.data.name === RUN_CODE_NAME)).toBe(true)
|
||||
// …and the program's tool calls landed as dispatch events under it.
|
||||
const dispatches = events.filter(event => event.type === 'tool/code-dispatch')
|
||||
expect(dispatches.length).toBeGreaterThanOrEqual(2)
|
||||
expect(dispatches.every(event => event.data.name === 'bash')).toBe(true)
|
||||
const parents = new Set(calls.map(event => event.data.callId))
|
||||
expect(dispatches.every(event => parents.has(event.data.parentCallId))).toBe(true)
|
||||
|
||||
// World verification: the file the program wrote, and the curated answer.
|
||||
const combined = await readFile(join(workdir, 'combined.txt'), 'utf8')
|
||||
expect(combined).toContain('alpha-7')
|
||||
expect(combined).toContain('beta-9')
|
||||
const finalMessage = events.findLast(event => event.type === 'assistant/message')
|
||||
const finalText = finalMessage !== undefined
|
||||
? finalMessage.data.content.filter(block => block.type === 'text').map(block => block.text).join('')
|
||||
: ''
|
||||
expect(finalText).toContain('alpha-7')
|
||||
expect(finalText).toContain('beta-9')
|
||||
}, 180_000)
|
||||
|
||||
it('delivers nested workspace instructions discovered by an fs sub-call after the outer result', async () => {
|
||||
workdir = await mkdtemp(join(tmpdir(), 'dsh-code-mode-workspace-e2e-'))
|
||||
await mkdir(join(workdir, '.git'), { recursive: true })
|
||||
await mkdir(join(workdir, 'pkg/deep'), { recursive: true })
|
||||
await writeFile(join(workdir, 'pkg/AGENTS.md'), `If asked for the Code Mode workspace handshake, reply with exactly ${WORKSPACE_PROBE} and nothing else.\n`)
|
||||
await writeFile(join(workdir, 'pkg/deep/task.txt'), 'Touch this file to discover the nested instructions.\n')
|
||||
ctx = await workspaceCodeModeHarness()
|
||||
const handle = await ctx.agents.create({
|
||||
sessionId: SessionId('e2e-code-mode-workspace-session'),
|
||||
meta: { cwd: workdir },
|
||||
agentOptions: { provider: 'deepseek', model: 'deepseek-v4-flash' },
|
||||
})
|
||||
|
||||
handle.agent.send([{
|
||||
type: 'text',
|
||||
text: 'Use one run_code program to call tools.read on pkg/deep/task.txt. After it finishes, answer: Code Mode workspace handshake?',
|
||||
}])
|
||||
await waitForIdle(ctx, handle.agent)
|
||||
|
||||
const events: SessionEvent[] = [...handle.agent.session.events]
|
||||
const dispatch = events.find(event => event.type === 'tool/code-dispatch' && event.data.name === 'read')
|
||||
const outerResult = events.find(event => event.type === 'tool/result')
|
||||
const workspaceContext = events.find(event => event.type === 'context/message'
|
||||
&& typeof event.data.meta === 'object'
|
||||
&& event.data.meta !== null
|
||||
&& !Array.isArray(event.data.meta)
|
||||
&& event.data.meta.kind === 'workspace-instructions')
|
||||
expect(dispatch).toBeDefined()
|
||||
expect(outerResult).toBeDefined()
|
||||
expect(workspaceContext).toBeDefined()
|
||||
expect(workspaceContext!.seq).toBeGreaterThan(outerResult!.seq)
|
||||
const finalMessage = events.findLast(event => event.type === 'assistant/message')
|
||||
const answer = finalMessage?.type === 'assistant/message'
|
||||
? finalMessage.data.content.filter(block => block.type === 'text').map(block => block.text).join('')
|
||||
: ''
|
||||
expect(answer).toContain(WORKSPACE_PROBE)
|
||||
}, 180_000)
|
||||
})
|
||||
84
examples/repl-agent/tests/coding-task.e2e.ts
Normal file
84
examples/repl-agent/tests/coding-task.e2e.ts
Normal file
@@ -0,0 +1,84 @@
|
||||
import { spawnSync } from 'node:child_process'
|
||||
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import type { Context } from 'cordis'
|
||||
import { codingHarness, finalText, SYSTEM_PROMPT, waitForIdle } from './harness.ts'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/**
|
||||
* The swebench-style smoke test: a real model fixes a real bug in a temp
|
||||
* directory using only the bash tool, and the fix is verified OUTSIDE the
|
||||
* agent by re-running the test script. Key-gated.
|
||||
*/
|
||||
|
||||
const TEST_FILE = [
|
||||
"const assert = require('node:assert');",
|
||||
"const { add } = require('./add.js');",
|
||||
'assert.strictEqual(add(2, 3), 5);',
|
||||
'assert.strictEqual(add(-1, 1), 0);',
|
||||
"console.log('PASS');",
|
||||
'',
|
||||
].join('\n')
|
||||
|
||||
const BUGGY_ADD = [
|
||||
'// A tiny module with an obvious bug.',
|
||||
'function add(a, b) {',
|
||||
' return a - b;',
|
||||
'}',
|
||||
'module.exports = { add };',
|
||||
'',
|
||||
].join('\n')
|
||||
|
||||
let workdir: string | undefined
|
||||
let ctx: Context | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
// Dispose the harness even on failure/retry: agent-loop teardown stops the
|
||||
// loop and LocalBashExecutor teardown kills anything the model left running.
|
||||
await ctx?.fiber.dispose()
|
||||
ctx = undefined
|
||||
if (workdir !== undefined) await rm(workdir, { recursive: true, force: true })
|
||||
workdir = undefined
|
||||
})
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('coding task: fix a failing test via bash', () => {
|
||||
it('repairs add.js so node add.test.js passes', async () => {
|
||||
workdir = await mkdtemp(join(tmpdir(), 'dsh-coding-task-'))
|
||||
await writeFile(join(workdir, 'add.js'), BUGGY_ADD)
|
||||
await writeFile(join(workdir, 'add.test.js'), TEST_FILE)
|
||||
|
||||
// Confirm the fixture actually fails before the agent touches it.
|
||||
const before = spawnSync('node', ['add.test.js'], { cwd: workdir })
|
||||
expect(before.status).not.toBe(0)
|
||||
|
||||
ctx = await codingHarness(workdir, { persona: SYSTEM_PROMPT })
|
||||
const agent = ctx.agentLoop.create(SessionId('e2e-task'), { provider: 'deepseek', model: 'deepseek-v4-flash' })
|
||||
|
||||
agent.send([{
|
||||
type: 'text',
|
||||
text: 'In the current directory, `node add.test.js` fails because add.js has a bug. '
|
||||
+ 'Fix add.js so the test passes, run `node add.test.js` to verify, and report the result. '
|
||||
+ 'Do not modify add.test.js.',
|
||||
}])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
// The agent claims success…
|
||||
const summary = finalText([...agent.session.events]).toLowerCase()
|
||||
expect(summary.length).toBeGreaterThan(0)
|
||||
|
||||
// …and the world agrees: the test passes when WE run it, and the test
|
||||
// file is byte-identical (an agent that neutered the test instead of
|
||||
// fixing the bug fails here, not just on a keyword probe).
|
||||
const untouchedTest = await readFile(join(workdir, 'add.test.js'), 'utf8')
|
||||
expect(untouchedTest).toBe(TEST_FILE)
|
||||
|
||||
const after = spawnSync('node', ['add.test.js'], { cwd: workdir, encoding: 'utf8' })
|
||||
expect(after.stdout).toContain('PASS')
|
||||
expect(after.status).toBe(0)
|
||||
|
||||
const fixed = await readFile(join(workdir, 'add.js'), 'utf8')
|
||||
expect(fixed).not.toMatch(/a\s*-\s*b/)
|
||||
}, 180_000)
|
||||
})
|
||||
88
examples/repl-agent/tests/compaction.e2e.ts
Normal file
88
examples/repl-agent/tests/compaction.e2e.ts
Normal file
@@ -0,0 +1,88 @@
|
||||
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import type { Context } from 'cordis'
|
||||
import { codingHarness, finalText, SYSTEM_PROMPT, waitForIdle } from './harness.ts'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/**
|
||||
* Key-gated smoke for mid-session compaction. It verifies the compact event
|
||||
* pair, replacement of older surface nodes, and a final answer after compaction.
|
||||
*/
|
||||
// FIXME(compaction-snapshot): this is the only full compaction coverage because
|
||||
// replay cannot serve the summarizer's unlogged model call.
|
||||
|
||||
let workdir: string | undefined
|
||||
let ctx: Context | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
await ctx?.fiber.dispose()
|
||||
ctx = undefined
|
||||
if (workdir !== undefined) await rm(workdir, { recursive: true, force: true })
|
||||
workdir = undefined
|
||||
})
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('compaction: a long session compacts mid-flight and keeps running', () => {
|
||||
it('summarizes older history into a checkpoint without breaking the task', async () => {
|
||||
workdir = await mkdtemp(join(tmpdir(), 'dsh-compaction-'))
|
||||
for (let i = 1; i <= 4; i++) {
|
||||
await writeFile(join(workdir, `file${i}.txt`), `This is file number ${i}. `.repeat(50))
|
||||
}
|
||||
|
||||
// Reasoning tokens require a larger generation cap than the retained checkpoint.
|
||||
ctx = await codingHarness(workdir, {
|
||||
persona: SYSTEM_PROMPT,
|
||||
tokenMeter: {
|
||||
contextWindow: 2000,
|
||||
},
|
||||
compact: {
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 400,
|
||||
summarizationProvider: '',
|
||||
summarizationModel: '',
|
||||
maxTokens: 1024,
|
||||
compactionRetries: 1,
|
||||
},
|
||||
persistenceRoot: join(workdir, '.sessions'),
|
||||
})
|
||||
const agent = ctx.agentLoop.create(SessionId('e2e-compaction'), { provider: 'deepseek', model: 'deepseek-v4-flash' })
|
||||
|
||||
agent.send([{
|
||||
type: 'text',
|
||||
text: 'Read file1.txt, file2.txt, file3.txt, and file4.txt one at a '
|
||||
+ 'time using cat (a separate bash command for each). After reading all four, tell me how '
|
||||
+ 'many files you read and the number mentioned in file1.txt.',
|
||||
}])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const events = [...agent.session.events]
|
||||
|
||||
// A compaction ran: the start…end bracket landed in the real log.
|
||||
const starts = events.filter(e => e.type === 'compact/start')
|
||||
const ends = events.filter(e => e.type === 'compact/end')
|
||||
expect(starts.length).toBeGreaterThan(0)
|
||||
expect(ends.length).toBe(starts.length) // every start was released
|
||||
|
||||
// It succeeded at least once: a compact/summary provenance event and a
|
||||
// replace-op user/message (the surface mutation) both landed.
|
||||
const summaries = events.filter(e => e.type === 'compact/summary')
|
||||
expect(summaries.length).toBeGreaterThan(0)
|
||||
const replaceNode = events.find((e) => {
|
||||
const se = e as unknown as { type: string; surfaceOp?: unknown }
|
||||
return se.type === 'user/message' && typeof se.surfaceOp === 'object' && se.surfaceOp !== null
|
||||
})
|
||||
expect(replaceNode).toBeDefined()
|
||||
|
||||
// The summary shadowed real older nodes (the surface shrank vs. the raw
|
||||
// message-producing event count).
|
||||
const summaryData = summaries[0]!.data as { shadowedSeqs: number[] }
|
||||
expect(summaryData.shadowedSeqs.length).toBeGreaterThan(0)
|
||||
|
||||
// The conversation survived compaction: the agent produced a final answer
|
||||
// that reflects the work (it read four files).
|
||||
const answer = finalText(events).toLowerCase()
|
||||
expect(answer.length).toBeGreaterThan(0)
|
||||
expect(answer).toMatch(/\b(4|four)\b/)
|
||||
}, 240_000)
|
||||
})
|
||||
48
examples/repl-agent/tests/full-loop.e2e.ts
Normal file
48
examples/repl-agent/tests/full-loop.e2e.ts
Normal file
@@ -0,0 +1,48 @@
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import type { Context } from 'cordis'
|
||||
import { codingHarness, finalText, SYSTEM_PROMPT, waitForIdle } from './harness.ts'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/**
|
||||
* The first place a REAL model meets the REAL bash tool: the cheap canary
|
||||
* before the coding-task e2e. Key-gated (see vitest.e2e.config.ts).
|
||||
*/
|
||||
|
||||
let ctx: Context | undefined
|
||||
let workdir: string | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
// Always dispose the harness, even on failure/retry/timeout: agent-loop
|
||||
// teardown stops the loop and LocalBashExecutor teardown kills any
|
||||
// process the model left behind.
|
||||
await ctx?.fiber.dispose()
|
||||
ctx = undefined
|
||||
if (workdir !== undefined) await rm(workdir, { recursive: true, force: true })
|
||||
workdir = undefined
|
||||
})
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('full loop: real model + real bash tool', () => {
|
||||
it('runs a bash command on request and reports its output', async () => {
|
||||
workdir = await mkdtemp(join(tmpdir(), 'dsh-full-loop-e2e-'))
|
||||
ctx = await codingHarness(workdir, { persona: SYSTEM_PROMPT })
|
||||
const agent = ctx.agentLoop.create(SessionId('e2e-loop'), { provider: 'deepseek', model: 'deepseek-v4-flash' })
|
||||
|
||||
agent.send([{ type: 'text', text: 'Run `echo e2e-ok` with the bash tool and tell me its exact output.' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const events = [...agent.session.events]
|
||||
const calls = events.filter(event => event.type === 'tool/call')
|
||||
expect(calls.length).toBeGreaterThan(0)
|
||||
expect(calls.some(event => event.data.name === 'bash')).toBe(true)
|
||||
|
||||
const results = events.filter(event => event.type === 'tool/result')
|
||||
const resultTexts = results.flatMap(event =>
|
||||
event.data.content.filter(block => block.type === 'text').map(block => block.text))
|
||||
expect(resultTexts.some(text => text.includes('e2e-ok'))).toBe(true)
|
||||
|
||||
expect(finalText(events)).toContain('e2e-ok')
|
||||
}, 120_000)
|
||||
})
|
||||
93
examples/repl-agent/tests/harness.ts
Normal file
93
examples/repl-agent/tests/harness.ts
Normal file
@@ -0,0 +1,93 @@
|
||||
import { Context } from 'cordis'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import { mountAgentLoopTestDependencies } from '@deepseek-ai/dsh-agent-loop-testkit'
|
||||
import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local'
|
||||
import * as ToolBash from '@deepseek-ai/dsh-tool-bash'
|
||||
import * as ToolTodo from '@deepseek-ai/dsh-tool-todo'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
|
||||
import type { TokenMeterConfig } from '@deepseek-ai/dsh-token-meter'
|
||||
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
import { BasicCompactService } from '@deepseek-ai/dsh-compact-basic'
|
||||
import type { BasicCompactConfig } from '@deepseek-ai/dsh-compact-basic'
|
||||
|
||||
/**
|
||||
* Shared harness for the repl-agent e2e suites: the full plugin stack
|
||||
* with the real DeepSeek adapter and the real bash + todo_write tools. Lives
|
||||
* outside the *.e2e.ts pattern so importing it never re-registers another
|
||||
* file's tests.
|
||||
*/
|
||||
|
||||
export const SYSTEM_PROMPT = 'You are a coding agent. Use bash for file operations '
|
||||
+ 'with cat/grep/heredocs; check [exit code: N] markers, '
|
||||
+ 'and report results briefly.'
|
||||
|
||||
/** System prompt for the todo_write e2e: nudges the model to plan with the tool. */
|
||||
export const TODO_SYSTEM_PROMPT = 'You are a coding agent. For multi-step work, '
|
||||
+ 'use the todo_write tool to track a task list: send the WHOLE list each call, '
|
||||
+ 'keep at most one task in_progress (exactly one while work remains), and mark '
|
||||
+ 'a task completed as soon as it is done.'
|
||||
|
||||
/** Options for {@link codingHarness}. */
|
||||
export interface CodingHarnessOptions {
|
||||
/**
|
||||
* Deployment persona for the tree (the system-prompt plugin's `persona`
|
||||
* config — per-context, not per-agent). Omitted ⇒ no persona section.
|
||||
*/
|
||||
persona?: string
|
||||
/** Durable JSONL persistence root (the resume suite needs it; others stay file-free). */
|
||||
persistenceRoot?: string
|
||||
/**
|
||||
* Load {@link BasicCompactService} with this config so the compaction e2e can
|
||||
* trigger compaction at a small, controlled history size. Omitted ⇒ no
|
||||
* compaction plugin (the default suites run without it).
|
||||
*/
|
||||
compact?: BasicCompactConfig
|
||||
/** Optional token-meter capacity loaded before compact-basic. */
|
||||
tokenMeter?: TokenMeterConfig
|
||||
}
|
||||
|
||||
export async function codingHarness(workdir: string, options: CodingHarnessOptions = {}): Promise<Context> {
|
||||
const ctx = new Context()
|
||||
await mountAgentLoopTestDependencies(ctx, {
|
||||
systemPrompt: { persona: options.persona ?? '' },
|
||||
})
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
await ctx.plugin(LlmDeepSeek)
|
||||
await ctx.plugin(LocalBashExecutor, { cwd: workdir, timeoutMs: 30_000 })
|
||||
await ctx.plugin(ToolBash)
|
||||
await ctx.plugin(ToolTodo)
|
||||
// Compaction is opt-in: only the compaction e2e loads the reusable meter and
|
||||
// backend, with a lower context window so a short real session crosses the threshold.
|
||||
if (options.compact !== undefined) {
|
||||
await ctx.plugin(TokenMeterService, options.tokenMeter)
|
||||
await ctx.plugin(BasicCompactService, options.compact)
|
||||
}
|
||||
// Durable JSONL persistence is opt-in: only the resume e2e needs it, and the
|
||||
// other suites stay file-free. Loaded last so a resume's deferred
|
||||
// `ctx.inject(['sessionPersistence'])` resolves once this is present.
|
||||
if (options.persistenceRoot !== undefined) await ctx.plugin(SessionPersistenceJsonl, { root: options.persistenceRoot })
|
||||
return ctx
|
||||
}
|
||||
|
||||
export function waitForIdle(ctx: Context, agent: Agent): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
const dispose = ctx.on('agent/status', (subject, status) => {
|
||||
if (subject === agent && status === 'idle') {
|
||||
dispose()
|
||||
resolve()
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
export function finalText(events: SessionEvent[]): string {
|
||||
const message = events.findLast(event => event.type === 'assistant/message')
|
||||
if (message?.type !== 'assistant/message') return ''
|
||||
return message.data.content
|
||||
.filter(block => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('')
|
||||
}
|
||||
28
examples/repl-agent/tests/keyless-smoke.e2e.ts
Normal file
28
examples/repl-agent/tests/keyless-smoke.e2e.ts
Normal file
@@ -0,0 +1,28 @@
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { LOADER_SMOKE_TEST_TIMEOUT_MS, runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke'
|
||||
|
||||
/**
|
||||
* Keyless Loader-path smoke for examples/repl-agent: boot the real example
|
||||
* through the stdio-agent bin and its `cordis.yml`, then close stdin without a
|
||||
* prompt and assert the banner. The dummy key satisfies adapter construction;
|
||||
* immediate EOF guarantees there is no model call.
|
||||
*/
|
||||
|
||||
const binScript = fileURLToPath(new URL('../../../packages/examples/stdio-demo/src/bin.ts', import.meta.url))
|
||||
const configPath = fileURLToPath(new URL('../cordis.yml', import.meta.url))
|
||||
const tsconfigPath = fileURLToPath(new URL('../../../tsconfig.json', import.meta.url))
|
||||
|
||||
describe('repl-agent keyless smoke (real cordis.yml via the Loader)', () => {
|
||||
it('boots the full plugin tree, prints its banner, and exits cleanly on EOF', async () => {
|
||||
const { stdout } = await runLoaderSmoke({
|
||||
label: 'repl-agent',
|
||||
tempDirPrefix: 'repl-smoke-',
|
||||
binScript,
|
||||
configPath,
|
||||
tsconfigPath,
|
||||
env: { DEEPSEEK_API_KEY: 'keyless-smoke-no-call' },
|
||||
})
|
||||
expect(stdout).toContain('agent REPL ready.')
|
||||
}, LOADER_SMOKE_TEST_TIMEOUT_MS)
|
||||
})
|
||||
67
examples/repl-agent/tests/resume.e2e.ts
Normal file
67
examples/repl-agent/tests/resume.e2e.ts
Normal file
@@ -0,0 +1,67 @@
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import type { Context } from 'cordis'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import { codingHarness, finalText, SYSTEM_PROMPT, waitForIdle } from './harness.ts'
|
||||
|
||||
/**
|
||||
* Proves durable conversation continuity end-to-end: run 1 tells the REAL model
|
||||
* a fact and persists the turn to JSONL; run 2 is a fresh harness (new Context,
|
||||
* same `.sessions` root) that RESUMES the persisted session id and asks the
|
||||
* model to recall the fact. The recall can only come from the rehydrated event
|
||||
* log — a fresh session would have no idea. Key-gated like the other e2es.
|
||||
*/
|
||||
|
||||
const SECRET = 'plum-galaxy-1791'
|
||||
const SESSION_ID = SessionId('resume-e2e-session')
|
||||
|
||||
let ctx: Context | undefined
|
||||
let root: string | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
// Dispose even on failure/retry: agent-loop teardown stops the loop and the
|
||||
// JSONL backend flushes; then drop the on-disk session log.
|
||||
await ctx?.fiber.dispose()
|
||||
ctx = undefined
|
||||
if (root !== undefined) await rm(root, { recursive: true, force: true })
|
||||
root = undefined
|
||||
})
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('resume: continue a persisted session across processes', () => {
|
||||
it('recalls a fact stored in a prior, separately-disposed session', async () => {
|
||||
root = await mkdtemp(join(tmpdir(), 'dsh-resume-e2e-'))
|
||||
|
||||
// Run 1: a fresh agent on a KNOWN session id learns a secret, then we
|
||||
// dispose the whole context (simulating process exit) so only the JSONL
|
||||
// log on disk survives.
|
||||
ctx = await codingHarness(process.cwd(), { persona: SYSTEM_PROMPT, persistenceRoot: root })
|
||||
const first = (await ctx.agents.create({
|
||||
sessionId: SESSION_ID,
|
||||
agentOptions: { provider: 'deepseek', model: 'deepseek-v4-flash' },
|
||||
})).agent
|
||||
first.send([{ type: 'text', text: `Remember this code for later: ${SECRET}. Just acknowledge it.` }])
|
||||
await waitForIdle(ctx, first)
|
||||
await ctx.fiber.dispose()
|
||||
ctx = undefined
|
||||
|
||||
// Run 2: a brand-new context over the SAME root resumes the persisted
|
||||
// session. The loaded event log seeds the live session, so the model sees
|
||||
// run 1's exchange as conversation history.
|
||||
ctx = await codingHarness(process.cwd(), { persona: SYSTEM_PROMPT, persistenceRoot: root })
|
||||
const resumed = (await ctx.agents.resume({
|
||||
resumeSessionId: SESSION_ID,
|
||||
agentOptions: { provider: 'deepseek', model: 'deepseek-v4-flash' },
|
||||
})).agent
|
||||
expect(resumed.session.id).toBe(SESSION_ID)
|
||||
// The prior user turn is in the rehydrated log before the model is asked.
|
||||
expect(JSON.stringify(resumed.session.deriveMessages())).toContain(SECRET)
|
||||
|
||||
resumed.send([{ type: 'text', text: 'What was the code I asked you to remember? Reply with just the code.' }])
|
||||
await waitForIdle(ctx, resumed)
|
||||
|
||||
// The model recalls it — only possible from the resumed history.
|
||||
expect(finalText([...resumed.session.events])).toContain(SECRET)
|
||||
}, 180_000)
|
||||
})
|
||||
53
examples/repl-agent/tests/todo-write.e2e.ts
Normal file
53
examples/repl-agent/tests/todo-write.e2e.ts
Normal file
@@ -0,0 +1,53 @@
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import type { Context } from 'cordis'
|
||||
import { codingHarness, TODO_SYSTEM_PROMPT, waitForIdle } from './harness.ts'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/**
|
||||
* A REAL model drives the REAL todo_write tool: verify the WORLD (the session
|
||||
* log gains a todo/write event whose snapshot the model actually produced), not
|
||||
* the agent's self-report. Key-gated (see vitest.e2e.config.ts).
|
||||
*/
|
||||
|
||||
let ctx: Context | undefined
|
||||
let workdir: string | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
await ctx?.fiber.dispose()
|
||||
ctx = undefined
|
||||
if (workdir !== undefined) await rm(workdir, { recursive: true, force: true })
|
||||
workdir = undefined
|
||||
})
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('todo_write: real model records a plan', () => {
|
||||
it('appends a todo/write event with the model-produced task list', async () => {
|
||||
workdir = await mkdtemp(join(tmpdir(), 'dsh-todo-write-e2e-'))
|
||||
ctx = await codingHarness(workdir, { persona: TODO_SYSTEM_PROMPT })
|
||||
const agent = ctx.agentLoop.create(SessionId('e2e-todo'), { provider: 'deepseek', model: 'deepseek-v4-flash' })
|
||||
|
||||
agent.send([{ type: 'text', text:
|
||||
'Use the todo_write tool to record a plan of exactly two steps: first '
|
||||
+ '"inspect the failing test" (in_progress), then "apply the fix" (pending). '
|
||||
+ 'Send both in one todo_write call, then reply with the single word DONE.' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const events = [...agent.session.events]
|
||||
|
||||
// The model actually called the tool.
|
||||
const calls = events.filter(event => event.type === 'tool/call')
|
||||
expect(calls.some(event => event.data.name === 'todo_write')).toBe(true)
|
||||
|
||||
// And the tool wrote a todo/write event to the log — verify the WORLD.
|
||||
const todoEvents = events.filter(event => event.type === 'todo/write')
|
||||
expect(todoEvents.length).toBeGreaterThan(0)
|
||||
|
||||
const todos = (todoEvents.at(-1)!).data.todos
|
||||
expect(todos).toEqual([
|
||||
{ content: 'inspect the failing test', status: 'in_progress' },
|
||||
{ content: 'apply the fix', status: 'pending' },
|
||||
])
|
||||
}, 120_000)
|
||||
})
|
||||
Reference in New Issue
Block a user