mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
fix(headless): terminal model failures reach stderr
This commit is contained in:
Tianyi Cui
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-09-headless-direct-core-front-door.md
|
||||
2026-08-09-headless-direct-core-front-door.md: cae17970633d6eda9d4f823118827ee2bfe32abf
|
||||
2026-08-09-headless-direct-core-front-door.zh.md: 8ff5e870b7f0fbc82065c6c55b4635cdc586036c
|
||||
2026-08-09-headless-direct-core-front-door.md: 875939b424059907949e350d50d88ded331f3dc9
|
||||
2026-08-09-headless-direct-core-front-door.zh.md: 7de5efbe172c0c43c03d55724d27e32d08c11e9f
|
||||
|
||||
@@ -14,7 +14,7 @@ The direct front door still needs the same deployment model state as Web-created
|
||||
|
||||
The shipped `headless` profile contains `dsh-base` and `dsh-headless`. The headless bundle supplies its persona and tool mode, disables HMR, mounts the Code Mode worker explicitly, and inserts `headless-runner`. Its tree contains no `@deepseek-ai/dsh-host-*` package, ApiProxy, HTTP server, Web runtime, or browser client. Code Mode and Session persistence are one-shot Agent capabilities independent of Web presentation.
|
||||
|
||||
`headless-runner` is a direct core front door. After Loader settlement, it reads `ctx.agentDefaultModel.currentSelection()`, creates a fresh persisted Agent through `ctx.agents.create`, installs that `ModelSelection` in the Agent scope, waits for startup quiescence, anchors the Session sequence, submits one ordinary user message, and waits for quiescence again. It awaits `ctx.sessions.flush`, folds its durable event interval for the last non-empty assistant text and final `turn/end` reason, writes the text plus one newline to stdout, and requests bounded launcher shutdown with exit 0 exactly when the reason is `completed`. Unexpected driver failures use stderr and exit 1.
|
||||
`headless-runner` is a direct core front door. After Loader settlement, it reads `ctx.agentDefaultModel.currentSelection()`, creates a fresh persisted Agent through `ctx.agents.create`, installs that `ModelSelection` in the Agent scope, waits for startup quiescence, anchors the Session sequence, submits one ordinary user message, and waits for quiescence again. It awaits `ctx.sessions.flush`, folds its durable event interval for the last non-empty assistant text and final `turn/end` reason, writes the text plus one newline to stdout, and requests bounded launcher shutdown with exit 0 exactly when the reason is `completed`. A terminal `error` reason writes its durable code and message to stderr; unexpected driver failures also use stderr and exit 1.
|
||||
|
||||
`@deepseek-ai/dsh-agent-default-model` owns the transport-independent default used for an Agent without a session-local selection. `AgentDefaultModelService` provides `ctx.agentDefaultModel` and registers the `agent-default-model` Settings section. Composition config supplies `{provider, model}`; user settings may also supply `reasoningEffort`. `currentSelection()` returns the live complete selection and `saveSelection()` writes it as a complete section, so a selection without an effort clears any stored effort. `dsh-base` supplies the composition entry. Direct and ApiProxy front doors consume this service; ApiProxy alone owns session-local precedence, model validation, and persistence of accepted Web selections.
|
||||
|
||||
@@ -24,7 +24,7 @@ This note owns the headless transport and completion contracts. [`dsh run` owns
|
||||
|
||||
## Verification
|
||||
|
||||
Package tests use the real Session store and Agent registry around a scripted Agent factory to pin idle-to-idle aggregation, late asynchronous completion, non-completed exits, direct failures, Loader-time disposal, and flush-before-exit ordering. The keyless assembled snapshot drives `dsh run` through a replayed tool round trip and records direct user-message provenance. Built-bin acceptance reaches a mock provider through the published entry and requires final text on stdout, exit 0, and empty stderr. Config-dump acceptance excludes every Host, Web, and Client package from the shipped headless tree; PTY shutdown coverage requires no observation line and bounded disposal.
|
||||
Package tests use the real Session store and Agent registry around a scripted Agent factory to pin idle-to-idle aggregation, late asynchronous completion, terminal model diagnostics, other non-completed exits, direct failures, Loader-time disposal, and flush-before-exit ordering. The keyless assembled snapshots drive `dsh run` through a replayed tool round trip, record direct user-message provenance, and expose a terminal model failure on stderr. Built-bin acceptance reaches a mock provider through the published entry and requires final text on stdout, exit 0, and empty stderr. Config-dump acceptance excludes every Host, Web, and Client package from the shipped headless tree; PTY shutdown coverage requires no observation line and bounded disposal.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ Status: implemented
|
||||
|
||||
随附的 `headless` profile 包含 `dsh-base` 与 `dsh-headless`。headless 组合包提供自身的 persona 与工具模式、禁用 HMR（热模块替换）、显式挂载 Code Mode worker，并插入 `headless-runner`。其插件树不包含任何 `@deepseek-ai/dsh-host-*` 包、ApiProxy、HTTP server、Web 运行时或浏览器客户端。Code Mode 与会话持久化均为独立于 Web 呈现的一次性 Agent 能力。
|
||||
|
||||
`headless-runner` 是直接使用核心服务的入口。Loader 完全加载后，它读取 `ctx.agentDefaultModel.currentSelection()`，通过 `ctx.agents.create` 创建一个新的持久化 Agent，在 Agent 作用域中安装该 `ModelSelection`，等待启动工作完全停稳，锚定会话事件序号，提交一条普通用户消息，再次等待完全停稳。随后，它等待 `ctx.sessions.flush`，折叠自身持有的持久事件区间，以取得最后一条非空 assistant 文本和最终 `turn/end` 结束原因，将文本连同一个换行写入 stdout，并且仅在结束原因为 `completed` 时请求启动器以退出状态 0 有界关闭。驱动器的意外失败写入 stderr 并以 1 退出。
|
||||
`headless-runner` 是直接使用核心服务的入口。Loader 完全加载后，它读取 `ctx.agentDefaultModel.currentSelection()`，通过 `ctx.agents.create` 创建一个新的持久化 Agent，在 Agent 作用域中安装该 `ModelSelection`，等待启动工作完全停稳，锚定会话事件序号，提交一条普通用户消息，再次等待完全停稳。随后，它等待 `ctx.sessions.flush`，折叠自身持有的持久事件区间，以取得最后一条非空 assistant 文本和最终 `turn/end` 结束原因，将文本连同一个换行写入 stdout，并且仅在结束原因为 `completed` 时请求启动器以退出状态 0 有界关闭。结束原因为 `error` 时，其持久化错误码与消息写入 stderr；驱动器的意外失败也写入 stderr 并以 1 退出。
|
||||
|
||||
`@deepseek-ai/dsh-agent-default-model` 拥有与传输无关的默认值，供没有会话级选择的 Agent 使用。`AgentDefaultModelService` 提供 `ctx.agentDefaultModel` 并注册 `agent-default-model` Settings 分节。组合配置提供 `{provider, model}`，用户设置还可以提供 `reasoningEffort`。`currentSelection()` 返回当前的完整选择，`saveSelection()` 则写入完整分节，因此不含强度的选择会清除已存强度。`dsh-base` 提供组合条目。直接前门与 ApiProxy 前门均消费该服务；只有 ApiProxy 负责会话级优先级、模型校验与已接受 Web 选择的持久化。
|
||||
|
||||
@@ -24,7 +24,7 @@ Status: implemented
|
||||
|
||||
## 验证
|
||||
|
||||
包测试围绕脚本化 Agent 工厂使用真实的会话存储与 Agent 注册表，固定空闲态到空闲态的聚合、延迟异步完成、未完成退出、直接失败、Loader 加载期间的 dispose（资源释放），以及退出前 flush 的顺序。组装后的无密钥快照通过回放的工具往返驱动 `dsh run`，并记录直接用户消息的来源。构建后二进制验收通过已发布入口访问 mock 提供方，并要求最终文本出现在 stdout、退出状态为 0 且 stderr 为空。配置转储验收排除随附 headless 树中的所有 Host、Web 与 Client 包；PTY 关闭覆盖要求不出现观察行，并在有界时间内完成 dispose。
|
||||
包测试围绕脚本化 Agent 工厂使用真实的会话存储与 Agent 注册表，固定空闲态到空闲态的聚合、延迟异步完成、终止态模型诊断、其他未完成退出、直接失败、Loader 加载期间的 dispose（资源释放），以及退出前 flush 的顺序。组装后的无密钥快照通过回放的工具往返驱动 `dsh run`，记录直接用户消息的来源，并在 stderr 暴露终止态模型失败。构建后二进制验收通过已发布入口访问 mock 提供方，并要求最终文本出现在 stdout、退出状态为 0 且 stderr 为空。配置转储验收排除随附 headless 树中的所有 Host、Web 与 Client 包；PTY 关闭覆盖要求不出现观察行，并在有界时间内完成 dispose。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
|
||||
@@ -29,6 +29,10 @@ class CliMockAdapter extends LlmAdapter {
|
||||
}
|
||||
|
||||
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
if (process.env.DSH_CLI_MOCK_FAILURE === '1') {
|
||||
yield { type: 'finish', reason: { kind: 'error', failure: { code: 'SERVER', message: 'CLI mock provider failed' } } }
|
||||
return
|
||||
}
|
||||
const toolResult = options.messages.at(-1)?.content.find(block => block.type === 'tool-result')
|
||||
if (toolResult === undefined) {
|
||||
const args = JSON.stringify({ command: 'printf CLI_TOOL_ROUND_TRIP', description: 'Prove the CLI tool round trip.' })
|
||||
|
||||
@@ -54,6 +54,7 @@ const reasoningConfigPath = fileURLToPath(new URL('./fixtures/cli.cordis.yml', i
|
||||
const deepseekDefaultsConfigPath = fileURLToPath(new URL('./fixtures/deepseek-defaults.cordis.yml', import.meta.url))
|
||||
const dshRunOverlayPath = fileURLToPath(new URL('./fixtures/dsh-run.cordis.yml', import.meta.url))
|
||||
const dshRunSessionExpected = join(snapshotsDir, 'dsh-run', 'session.expected.jsonl')
|
||||
const dshRunFailureExpected = join(snapshotsDir, 'dsh-run', 'stderr.expected.txt')
|
||||
const cliMockLlmPluginPath = fileURLToPath(new URL('./fixtures/cli-mock-llm.ts', import.meta.url))
|
||||
const refreshing = process.env.DSH_SNAPSHOT === 'refresh'
|
||||
|
||||
@@ -196,6 +197,16 @@ async function persistedLogs(cwd: string, root: string = join(cwd, '.sessions'))
|
||||
}))
|
||||
}
|
||||
|
||||
/** Install the keyless product-CLI adapter into the temporary headless profile. */
|
||||
async function prepareCliMockFixture(cwd: string): Promise<void> {
|
||||
const fixtureDir = join(cwd, '.dsh', 'profiles', 'headless', 'snapshot-fixtures')
|
||||
await mkdir(fixtureDir, { recursive: true })
|
||||
await Promise.all([
|
||||
copyFile(cliMockLlmPluginPath, join(fixtureDir, 'cli-mock-llm.ts')),
|
||||
writeFile(join(fixtureDir, 'package.json'), '{"type":"module"}\n'),
|
||||
])
|
||||
}
|
||||
|
||||
describe('headless stream-json snapshots', () => {
|
||||
it('runs one task through the product dsh run command', async () => {
|
||||
const task = 'Prove the product dsh run path with one real tool round trip.'
|
||||
@@ -211,14 +222,7 @@ describe('headless stream-json snapshots', () => {
|
||||
DSH_TELEMETRY_DISABLED: '1',
|
||||
NODE_OPTIONS: [process.env.NODE_OPTIONS, '--disable-warning=ExperimentalWarning'].filter(Boolean).join(' '),
|
||||
},
|
||||
prepare: async (cwd) => {
|
||||
const fixtureDir = join(cwd, '.dsh', 'profiles', 'headless', 'snapshot-fixtures')
|
||||
await mkdir(fixtureDir, { recursive: true })
|
||||
await Promise.all([
|
||||
copyFile(cliMockLlmPluginPath, join(fixtureDir, 'cli-mock-llm.ts')),
|
||||
writeFile(join(fixtureDir, 'package.json'), '{"type":"module"}\n'),
|
||||
])
|
||||
},
|
||||
prepare: prepareCliMockFixture,
|
||||
inspect: async (cwd) => {
|
||||
const logs = await persistedLogs(cwd, join(cwd, '.dsh', 'sessions'))
|
||||
expect(logs).toHaveLength(1)
|
||||
@@ -237,6 +241,27 @@ describe('headless stream-json snapshots', () => {
|
||||
expect(result.stderr).toBe('')
|
||||
}, LOADER_SMOKE_TEST_TIMEOUT_MS)
|
||||
|
||||
it('prints a terminal model failure through the product dsh run command', async () => {
|
||||
const result = await runLoaderSmoke({
|
||||
label: 'product dsh run model failure snapshot',
|
||||
tempDirPrefix: 'headless-snapshot-dsh-run-failure-',
|
||||
binScript: dshBinScript,
|
||||
configPath: dshRunOverlayPath,
|
||||
binArgs: ['run', '--patch', dshRunOverlayPath, 'Trigger the keyless model failure.'],
|
||||
tsconfigPath,
|
||||
expectedExitCode: 1,
|
||||
env: {
|
||||
DSH_CLI_MOCK_FAILURE: '1',
|
||||
DSH_TELEMETRY_DISABLED: '1',
|
||||
NODE_OPTIONS: [process.env.NODE_OPTIONS, '--disable-warning=ExperimentalWarning'].filter(Boolean).join(' '),
|
||||
},
|
||||
prepare: prepareCliMockFixture,
|
||||
})
|
||||
|
||||
expect(result.stdout).toBe('\n')
|
||||
await expect(result.stderr).toMatchFileSnapshot(dshRunFailureExpected)
|
||||
}, LOADER_SMOKE_TEST_TIMEOUT_MS)
|
||||
|
||||
it('prints the original Loader activation error through the assembled one-shot app', async () => {
|
||||
const result = await runLoaderSmoke({
|
||||
label: 'headless startup activation error snapshot',
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
dsh: SERVER: CLI mock provider failed
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/bundle/headless/README.md
|
||||
README.md: 6faf1dc732345a86ec1d7b60c4f9a606e0491512
|
||||
README.zh.md: a189f77f57cf0f641abe7c11288dbc1cdf05355a
|
||||
README.md: fbdf80be3d587b0ef526c1a36952a761035f7312
|
||||
README.zh.md: 833737e08643e0ad6b585f93fdc6ae018242b937
|
||||
|
||||
@@ -4,7 +4,7 @@ English | [中文](README.zh.md)
|
||||
|
||||
The dsh one-shot bundle. [`cordis.patch.yml`](cordis.patch.yml) rides directly over [`dsh-base`](../base/README.md): it supplies the coding persona and tool mode, disables HMR, mounts Code Mode's worker as a core execution capability, and inserts this package's `headless-runner` plugin (config `{task}`). It mounts no Host, HTTP server, Web runtime, or browser plugin.
|
||||
|
||||
After the Loader settles, the runner reads the shared [`ctx.agentDefaultModel`](../../core/agent-default-model/README.md), creates one fresh persisted Agent through `ctx.agents`, submits the task as an ordinary user message, and waits for quiescence. It flushes the Session before folding the owned durable event interval, writes the last non-empty assistant text to stdout, and requests exit through the launcher-provided `ctx.headlessIo` seam (final `turn/end` completed → 0, otherwise 1). A successful run writes nothing to stderr and opens no listening port. The launcher patches the task text in (`dsh run "task"`) and fails loud when the selected profile lacks this row.
|
||||
After the Loader settles, the runner reads the shared [`ctx.agentDefaultModel`](../../core/agent-default-model/README.md), creates one fresh persisted Agent through `ctx.agents`, submits the task as an ordinary user message, and waits for quiescence. It flushes the Session before folding the owned durable event interval, writes the last non-empty assistant text to stdout, and requests exit through the launcher-provided `ctx.headlessIo` seam (final `turn/end` completed → 0, otherwise 1). A terminal `error` reason also writes its code and message to stderr; successful runs keep stderr empty. The process opens no listening port. The launcher patches the task text in (`dsh run "task"`) and fails loud when the selected profile lacks this row.
|
||||
|
||||
## Model Experience
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
dsh 一次性任务组合包。[`cordis.patch.yml`](cordis.patch.yml) 直接叠加在 [`dsh-base`](../base/README.md) 之上：提供编码 persona 和工具模式、禁用 HMR（热模块替换）、将 Code Mode 的 worker 作为核心执行能力挂载，并插入本包的 `headless-runner` 插件（配置为 `{task}`）。它不挂载任何 Host、HTTP server、Web runtime 或浏览器插件。
|
||||
|
||||
Loader 结算后，runner 读取共享的 [`ctx.agentDefaultModel`](../../core/agent-default-model/README.md)，通过 `ctx.agents` 创建一个全新的持久化 Agent（智能体），将任务作为普通用户消息提交，并等待完全停稳。它对 Session 执行 flush 后再汇总自身持有的持久化事件区间，将最后一条非空 assistant 文本写入 stdout，再经启动器提供的 `ctx.headlessIo` seam 请求退出（最终 `turn/end` 完成 → 0，否则为 1）。成功运行时不会向 stderr 写入任何内容，也不会打开监听端口。启动器把任务文本 patch 进来（`dsh run "task"`）；若所选 profile 缺少该行，则显式报错。
|
||||
Loader 结算后，runner 读取共享的 [`ctx.agentDefaultModel`](../../core/agent-default-model/README.md)，通过 `ctx.agents` 创建一个全新的持久化 Agent（智能体），将任务作为普通用户消息提交，并等待完全停稳。它对 Session 执行 flush 后再汇总自身持有的持久化事件区间，将最后一条非空 assistant 文本写入 stdout，再经启动器提供的 `ctx.headlessIo` seam 请求退出（最终 `turn/end` 完成 → 0，否则为 1）。最终 reason 为 `error` 时，还会将持久化的 code 与 message 写入 stderr；成功运行时 stderr 保持为空。进程不会打开监听端口。启动器把任务文本 patch 进来（`dsh run "task"`）；若所选 profile 缺少该行，则显式报错。
|
||||
|
||||
## 模型体验
|
||||
|
||||
|
||||
@@ -38,7 +38,7 @@ export const Config: z<Config> = z.object({
|
||||
/** Outcome of one owned run interval. */
|
||||
interface RunOutcome {
|
||||
text: string
|
||||
reason: string
|
||||
reason: SessionEvent<'turn/end'>['data']['reason'] | undefined
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -63,7 +63,7 @@ declare module 'cordis' {
|
||||
function summarize(events: readonly SessionEvent[], firstSeq: number): RunOutcome {
|
||||
let started = false
|
||||
let text = ''
|
||||
let reason = 'error'
|
||||
let reason: SessionEvent<'turn/end'>['data']['reason'] | undefined
|
||||
for (const event of events) {
|
||||
if (event.seq < firstSeq) continue
|
||||
if (event.type === 'turn/start') {
|
||||
@@ -78,7 +78,7 @@ function summarize(events: readonly SessionEvent[], firstSeq: number): RunOutcom
|
||||
.join('')
|
||||
if (joined !== '') text = joined
|
||||
}
|
||||
if (event.type === 'turn/end') reason = event.data.reason.kind
|
||||
if (event.type === 'turn/end') reason = event.data.reason
|
||||
}
|
||||
return { text, reason }
|
||||
}
|
||||
@@ -125,7 +125,10 @@ async function run(ctx: Context, task: string, io: HeadlessIo): Promise<void> {
|
||||
await sessions.flush(agent.session)
|
||||
const outcome = summarize(agent.session.events, firstSeq)
|
||||
io.stdout.write(outcome.text + '\n')
|
||||
io.exit(outcome.reason === 'completed' ? 0 : 1)
|
||||
if (outcome.reason?.kind === 'error') {
|
||||
io.stderr.write(`dsh: ${outcome.reason.error.code}: ${outcome.reason.error.message}\n`)
|
||||
}
|
||||
io.exit(outcome.reason?.kind === 'completed' ? 0 : 1)
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -151,6 +151,27 @@ describe('headless runner', () => {
|
||||
await test.ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('prints the durable model failure when the final turn ends in error', async () => {
|
||||
const test = await bench({
|
||||
afterPrompt(session, message) {
|
||||
session.append('turn/start', { turn: 1 })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
session.append('user/message', message, { surfaceOp: 'append' })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('turn/end', {
|
||||
turn: 1,
|
||||
reason: { kind: 'error', error: { code: 'SERVER', message: 'provider unavailable' } },
|
||||
})
|
||||
},
|
||||
})
|
||||
expect(await test.run()).toMatchObject({
|
||||
code: 1,
|
||||
out: '\n',
|
||||
err: 'dsh: SERVER: provider unavailable\n',
|
||||
})
|
||||
await test.ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('exits 1 when the owned interval contains no turn', async () => {
|
||||
const test = await bench({ afterPrompt: () => {} })
|
||||
expect(await test.run()).toMatchObject({ code: 1, out: '\n', err: '' })
|
||||
|
||||
Reference in New Issue
Block a user