mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
fix(llm): classify empty model completions as retryable EMPTY_RESPONSE
A well-formed provider stream that ends with finish_reason stop and zero
content blocks previously became a successful empty assistant message: the
turn completed silently, and drivers like goal-session counted the no-op
round. Both adapters now map that degenerate completion to a finish
{kind:'error'} with the new canonical EMPTY_RESPONSE code from dsh-llm, and
dsh-llm-retry adds the code to its default retryable set, so the existing
closed-step recovery path retries it and fails loud once the budget is
exhausted.
Covered by adapter unit tests, an llm-retry default-policy test, and a new
authored keyless ACP snapshot (empty-response-retry) with a deterministic
1 ms zero-jitter retry overlay.
This commit is contained in:
Turtle
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-24-empty-model-response-is-retryable.md: 1c9f6092efe7cc6702117c53ea3f1b7f14445100
|
||||
2026-07-24-empty-model-response-is-retryable.zh.md: 8a124d6ac80c751fc2dbc46f1ed4d50ec5e7348f
|
||||
@@ -0,0 +1,36 @@
|
||||
# Agent Note: Empty model completions are retryable EMPTY_RESPONSE failures
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-24-empty-model-response-is-retryable.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Providers occasionally return a degenerate completion: a well-formed stream that carries a terminal `stop` finish and zero content blocks — no text, no reasoning, no tool calls. Before this change both adapters mapped it to a successful `{kind: 'stop'}` finish, so the loop logged an empty `assistant/message` and ended the turn as `completed`. Nothing retried, nothing failed loud, and a driver like goal-session counted the silent no-op as a consumed round. A live incident showed an openrouter-served model burning three of six goal rounds on empty completions before the goal blocked on its round limit.
|
||||
|
||||
## Decision
|
||||
|
||||
An adapter classifies a completed empty response as a provider-boundary failure, and retry policy treats it as transient:
|
||||
|
||||
- `dsh-llm` exports the canonical code `EMPTY_RESPONSE_CODE` (`'EMPTY_RESPONSE'`) beside `CONTEXT_WINDOW_EXCEEDED_CODE`/`QUOTA_EXCEEDED_CODE`.
|
||||
- `dsh-llm-pi-ai` (`mapStopReason`): a terminal `stop` whose assistant message has no content blocks becomes a `finish {kind: 'error'}` with that code. Context-overflow detection still wins where it applies (it is checked first and is the more actionable classification).
|
||||
- `dsh-llm-deepseek` (`translate`): at `[DONE]`, a `stop` (or absent) finish with no opened blocks becomes the same error finish. Reasoning-only streams count as content and stay successful.
|
||||
- `dsh-llm-retry` adds `EMPTY_RESPONSE` to `DEFAULT_RETRYABLE_CODES`: the attempt produced nothing durable, so repeating it is safe; deployments can still remove it via `retryableCodes`.
|
||||
|
||||
Detection is scoped to `stop` finishes only. `max-tokens` with empty content keeps its existing meaning (pi-ai already normalizes the zero-output overflow case), `tool-calls` cannot be block-empty in practice, and error/aborted finishes already fail.
|
||||
|
||||
The classification rides the existing loop machinery — `finishError` → `agent/request-error` → `dsh-llm-retry` — so no `agent-loop` change was needed, and after the retry budget exhausts, the turn fails loud with `EMPTY_RESPONSE` instead of silently completing empty.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Detect in the loop or `BlockAssembler`.** One shared implementation, but it moves provider-response judgment into the loop, against "plugins, not loop changes", and the assembler is a pure assembly algorithm. The adapter is where wire facts become harness classification, with the overflow reclassification as exact precedent.
|
||||
|
||||
**A stream-transform plugin on the `llm/stream` waterfall.** Provider-neutral and one implementation, but it adds a package plus wiring for what is a boundary fact each adapter can state in a few lines, and default-on behavior would still require touching every bundle.
|
||||
|
||||
**Treat whitespace-only or reasoning-only responses as empty too.** Rejected as overreach: those carry model-produced content, and misclassifying a legitimate (if useless) response as a transport-class failure risks retry loops on models that intentionally stop after reasoning. The scope is exactly "zero content blocks".
|
||||
|
||||
## Consequences
|
||||
|
||||
- A transiently misbehaving provider now costs a bounded retry instead of a silently wasted turn; a persistently empty model surfaces as a loud `EMPTY_RESPONSE` turn failure users can act on.
|
||||
- A model that genuinely intends to say nothing (rare, but possible after a tool result) is now retried and, if consistently empty, fails the turn. This trade was accepted deliberately: an empty assistant message is indistinguishable from the provider defect and has no value to the user.
|
||||
- The `empty-response-retry` ACP snapshot (an authored keyless scenario with a deterministic 1 ms zero-jitter retry overlay, `examples/acp-agent/retry.cordis.yml`) pins the product-visible arc: durable `llm/retry` event, the discarded-attempt marker, and a clean completed turn.
|
||||
@@ -0,0 +1,36 @@
|
||||
# Agent Note: Empty model completions are retryable EMPTY_RESPONSE failures
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-24-empty-model-response-is-retryable.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
提供方偶尔会返回一种退化的 completion：流本身格式完好，携带一个终止性的 `stop` 结束，却没有任何内容块——没有文本、没有 reasoning（推理）、没有工具调用。本次改动前，两个适配器都会把它映射为成功的 `{kind: 'stop'}` 结束，于是主循环记录了一条空的 `assistant/message`，并把该轮次以 `completed` 结束。没有任何重试，也没有任何显式失败，而像 goal-session 这样的驱动方会把这次静默的空操作计为一次已消耗的 goal 轮数。一次线上事故显示，某个由 openrouter 提供的模型在触及 goal 的轮数上限而被阻塞前，把六轮 goal 中的三轮消耗在了空 completion 上。
|
||||
|
||||
## Decision
|
||||
|
||||
由适配器把「已完成但为空」的响应归类为一次提供方边界失败，重试策略则将其视为瞬时性问题：
|
||||
|
||||
- `dsh-llm` 在 `CONTEXT_WINDOW_EXCEEDED_CODE`/`QUOTA_EXCEEDED_CODE` 之外，导出规范代码 `EMPTY_RESPONSE_CODE`（`'EMPTY_RESPONSE'`）。
|
||||
- `dsh-llm-pi-ai`（`mapStopReason`）：当终止性 `stop` 所对应的 assistant 消息没有内容块时，它会变成一个携带该代码的 `finish {kind: 'error'}`。上下文溢出检测在其适用场景中仍然优先（它先被检查，也是更具可操作性的归类）。
|
||||
- `dsh-llm-deepseek`（`translate`）：在 `[DONE]` 处，若 `stop`（或缺失）结束且没有打开过任何块，则同样变成该错误结束。仅含 reasoning 的流算作有内容，仍视为成功。
|
||||
- `dsh-llm-retry` 把 `EMPTY_RESPONSE` 加入 `DEFAULT_RETRYABLE_CODES`：这次尝试没有产生任何持久内容，因此重复它是安全的；部署方仍可通过 `retryableCodes` 将其移除。
|
||||
|
||||
检测仅限于 `stop` 结束。内容为空的 `max-tokens` 保持其既有含义（pi-ai 已经把零输出的溢出场景归一化处理），`tool-calls` 在实践中不可能是空块，而 error／aborted 结束本身已经算失败。
|
||||
|
||||
这套归类沿用既有的主循环机制——`finishError` → `agent/request-error` → `dsh-llm-retry`——因此无需改动 `agent-loop`；在重试预算耗尽后，该轮次会以 `EMPTY_RESPONSE` 显式失败，而不再静默地以空内容完成。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**在主循环或 `BlockAssembler` 中检测。** 只需一份共享实现，但这会把对提供方响应的判断挪进主循环，违背「插件优先，而非改动主循环」，且 assembler 是纯粹的组装算法。适配器才是把协议层面的事实转化为 harness 归类的地方，而溢出重归类正是精确的先例。
|
||||
|
||||
**在 `llm/stream` waterfall（瀑布式事件）上做一个流转换插件。** 这种做法提供方无关且只需一份实现，但它为「每个适配器几行就能声明的边界事实」额外增加了一个包和相应接线，而且默认开启的行为仍需改动每一个 bundle。
|
||||
|
||||
**把仅含空白或仅含 reasoning 的响应也当作空响应。** 作为过度设计予以否决：这类响应携带了模型产生的内容，把一个合法（哪怕无用）的响应误判为传输类失败，会在那些故意在 reasoning 之后停止的模型上引发重试循环。其范围严格限定为「零内容块」。
|
||||
|
||||
## Consequences
|
||||
|
||||
- 一个偶发异常的提供方现在只会花费一次有界的重试，而不再是一个被静默浪费的轮次；一个持续返回空内容的模型则会显式暴露为一次用户可据以行动的 `EMPTY_RESPONSE` 轮次失败。
|
||||
- 一个确实打算什么都不说的模型（罕见，但在一次工具结果之后有可能出现）现在会被重试，若始终为空，则该轮次失败。这个取舍是经过审慎权衡后接受的：一条空的 assistant 消息与提供方缺陷无法区分，且对用户毫无价值。
|
||||
- `empty-response-retry` ACP 快照（一个人工编写的无密钥场景，配有确定性的 1 ms 零抖动重试 overlay，`examples/acp-agent/retry.cordis.yml`）钉住了产品可见的整个过程：持久的 `llm/retry` 事件、被丢弃尝试的标记，以及一次干净的已完成轮次。
|
||||
41
examples/acp-agent/retry.cordis.snapshot.yml
Normal file
41
examples/acp-agent/retry.cordis.snapshot.yml
Normal file
@@ -0,0 +1,41 @@
|
||||
# Keyless replay for the retry overlay: disable the key-requiring DeepSeek
|
||||
# adapter, insert `llm-replay`, and restate the app config with the same
|
||||
# deterministic 1 ms zero-jitter retry policy as the live sibling. A config
|
||||
# patch replaces the whole app config, so the base fields are restated
|
||||
# verbatim (raw JSONL persistence so the harness can harvest the log).
|
||||
- id: base
|
||||
name: '@cordisjs/plugin-include'
|
||||
config:
|
||||
path: ./cordis.yml
|
||||
patches:
|
||||
- id: llm-deepseek
|
||||
name: '@deepseek-ai/dsh-llm-deepseek'
|
||||
disabled: true
|
||||
- id: acp-agent
|
||||
name: '@deepseek-ai/dsh-acp-demo'
|
||||
config:
|
||||
provider: deepseek
|
||||
model: deepseek-v4-flash
|
||||
persistenceRoot: !!js process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? './.sessions'
|
||||
persistenceCompression: none
|
||||
workspaceContext:
|
||||
maxBytes: 65536
|
||||
llmRetry:
|
||||
maxTransientRetries: 2
|
||||
initialDelayMs: 1
|
||||
maxDelayMs: 1
|
||||
jitterRatio: 0
|
||||
persona: |
|
||||
You are a coding assistant powered by the {{model}} model. Your working directory is {{cwd}}. Your bash tool runs under a file sandbox — a `[sandbox: file access denied …]` result is policy, not a command bug.
|
||||
|
||||
Verify your work by running the code or tests. Keep answers brief and factual.
|
||||
- insert:
|
||||
- id: llm-replay
|
||||
name: '@deepseek-ai/dsh-llm-replay'
|
||||
config:
|
||||
providers:
|
||||
- id: deepseek
|
||||
name: DeepSeek
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
- id: deepseek-v4-pro
|
||||
30
examples/acp-agent/retry.cordis.yml
Normal file
30
examples/acp-agent/retry.cordis.yml
Normal file
@@ -0,0 +1,30 @@
|
||||
# Retry-scenario overlay: pin the bounded transient retry policy to a
|
||||
# deterministic 1 ms zero-jitter delay so the durable `llm/retry` event
|
||||
# (`delayMs`) and replay wall time stay reproducible. The overlay changes no
|
||||
# tool or prompt composition, so its scenarios share the default header class.
|
||||
# A config patch replaces the whole app config, so the base fields are restated
|
||||
# verbatim; the model is re-pinned to `deepseek-v4-flash` like the other
|
||||
# snapshot overlays because the recorded corpus was captured on flash.
|
||||
- id: base
|
||||
name: '@cordisjs/plugin-include'
|
||||
config:
|
||||
path: ./cordis.yml
|
||||
patches:
|
||||
- id: acp-agent
|
||||
name: '@deepseek-ai/dsh-acp-demo'
|
||||
config:
|
||||
provider: deepseek
|
||||
model: deepseek-v4-flash
|
||||
persistenceRoot: !!js process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? './.sessions'
|
||||
persistenceCompression: !!js "process.env.DSH_SNAPSHOT === undefined ? 'zstd' : 'none'"
|
||||
workspaceContext:
|
||||
maxBytes: 65536
|
||||
llmRetry:
|
||||
maxTransientRetries: 2
|
||||
initialDelayMs: 1
|
||||
maxDelayMs: 1
|
||||
jitterRatio: 0
|
||||
persona: |
|
||||
You are a coding assistant powered by the {{model}} model. Your working directory is {{cwd}}. Your bash tool runs under a file sandbox — a `[sandbox: file access denied …]` result is policy, not a command bug.
|
||||
|
||||
Verify your work by running the code or tests. Keep answers brief and factual.
|
||||
@@ -38,6 +38,7 @@ const PTY_CONFIG = fileURLToPath(new URL('../pty.cordis.yml', import.meta.url))
|
||||
const DEPTH_TWO_CONFIG = fileURLToPath(new URL('../depth-two.cordis.yml', import.meta.url))
|
||||
const PACKED_CHUNKS_CONFIG = fileURLToPath(new URL('../packed-chunks.cordis.yml', import.meta.url))
|
||||
const SESSION_SANDBOX_ROOT_CONFIG = fileURLToPath(new URL('../session-sandbox-root.cordis.yml', import.meta.url))
|
||||
const RETRY_CONFIG = fileURLToPath(new URL('../retry.cordis.yml', import.meta.url))
|
||||
const LSP_CONFIG = fileURLToPath(new URL('./lsp.cordis.yml', import.meta.url))
|
||||
const SNAPSHOTS_DIR = join(dirname(fileURLToPath(import.meta.url)), 'snapshots')
|
||||
const PACKED_CHUNKS_SOURCE = 'hook-cc-pretool-deny'
|
||||
@@ -137,6 +138,14 @@ const SCENARIOS: Scenario[] = [
|
||||
headerClass: 'model-switching',
|
||||
},
|
||||
{ name: 'error-finish', hasModelTurn: true, recorded: false, overridden: true },
|
||||
// Keyless, authored (like error-finish): a live provider cannot be coaxed
|
||||
// into a degenerate empty completion, so the fixture scripts the adapters'
|
||||
// EMPTY_RESPONSE error finish (step 1) followed by the recovered reply
|
||||
// (step 2), proving the default retry policy end to end: the durable
|
||||
// llm/retry event, the ACP discarded-attempt marker, and a clean completed
|
||||
// turn. Its overlay only pins a deterministic 1 ms zero-jitter delay, so it
|
||||
// shares the default header class.
|
||||
{ name: 'empty-response-retry', hasModelTurn: true, recorded: false, configPath: RETRY_CONFIG },
|
||||
// Keyless, authored (like error-finish/cancel): deterministically forcing a
|
||||
// LIVE model to repeat one call three times is not a stable recording, so
|
||||
// the fixture scripts five identical todo_write calls and pins BOTH reminder
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"steps": [
|
||||
{ "op": "initialize" },
|
||||
{ "op": "newSession" },
|
||||
{ "op": "prompt", "text": "This prompt first receives an empty completion, then a retried reply." }
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0}
|
||||
{"type":"turn/start","seq":0,"time":0,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}}
|
||||
{"type":"user/message","seq":1,"time":0,"data":{"content":[{"type":"text","text":"This prompt first receives an empty completion, then a retried reply."}],"source":{"kind":"user"}},"surfaceOp":"append"}
|
||||
{"type":"session/title","seq":2,"time":0,"data":{"title":"This prompt first receives an","messageSeqs":[1],"source":{"kind":"fallback"}}}
|
||||
{"type":"step/start","seq":3,"time":0,"data":{"turn":1,"step":1}}
|
||||
{"type":"request/header","seq":4,"time":0,"data":{"header":{"config":{"provider":"deepseek","model":"deepseek-v4-flash"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}
|
||||
{"type":"assistant/chunk","seq":5,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":0,"outputTokens":0}}}}
|
||||
{"type":"assistant/chunk","seq":6,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"error","failure":{"message":"model returned a completed response with no content","code":"EMPTY_RESPONSE"}}}}}
|
||||
{"type":"step/end","seq":7,"time":0,"data":{"turn":1,"step":1}}
|
||||
{"type":"llm/retry","seq":8,"time":0,"data":{"turn":1,"step":1,"retry":1,"maxRetries":2,"delayMs":1,"failure":{"message":"model returned a completed response with no content","code":"EMPTY_RESPONSE"}}}
|
||||
{"type":"step/start","seq":9,"time":0,"data":{"turn":1,"step":2}}
|
||||
{"type":"assistant/chunk","seq":10,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"text"}}}
|
||||
{"type":"assistant/chunk","seq":11,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"text-delta","index":0,"text":"Recovered."}}}
|
||||
{"type":"assistant/chunk","seq":12,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"Recovered."}}}}
|
||||
{"type":"assistant/chunk","seq":13,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":12,"outputTokens":3}}}}
|
||||
{"type":"assistant/chunk","seq":14,"time":0,"data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"stop"}}}}
|
||||
{"type":"assistant/message","seq":15,"time":0,"data":{"turn":1,"step":2,"content":[{"type":"text","text":"Recovered."}],"provenance":{"provider":"deepseek","model":"deepseek-v4-flash"},"usage":{"inputTokens":12,"outputTokens":3}},"sourceEventSeqs":[10,11,12,13,14],"surfaceOp":"append"}
|
||||
{"type":"step/end","seq":16,"time":0,"data":{"turn":1,"step":2}}
|
||||
{"type":"turn/end","seq":17,"time":0,"data":{"turn":1,"reason":{"kind":"completed"}}}
|
||||
@@ -0,0 +1,7 @@
|
||||
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"sessionCapabilities":{"list":{}},"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}}
|
||||
{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}","modes":{"availableModes":[{"id":"default","name":"default"},{"id":"plan","name":"plan"}],"currentModeId":"default"},"configOptions":[{"id":"model","name":"Model","description":"Sets this session's provider and model.","category":"model","type":"select","currentValue":"[\"deepseek\",\"deepseek-v4-flash\"]","options":[{"value":"[\"deepseek\",\"deepseek-v4-flash\"]","name":"deepseek-v4-flash"},{"value":"[\"deepseek\",\"deepseek-v4-pro\"]","name":"deepseek-v4-pro"}]},{"id":"permission","name":"Permissions","description":"The session permission preset: each choice bundles a sandbox mode and an approval policy.","category":"mode","type":"select","currentValue":"danger-full-access","options":[{"value":"workspace-write","name":"workspace-write","description":"Write inside the workspace and permitted temporary directories; wider retries require approval."},{"value":"danger-full-access","name":"danger-full-access","description":"Full file access without approval prompts."}]}]}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"available_commands_update","availableCommands":[{"name":"goal","description":"set or view the goal for a long-running task","input":{"hint":"[<objective>|clear|edit <objective>|pause|resume]"}},{"name":"plan","description":"Enter or leave plan mode","input":{"hint":"[off|message]"}}]}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"session_info_update","title":"This prompt first receives an","updatedAt":"{{updatedAt}}"}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"\n\n[Previous model attempt discarded; retrying 1/2 in 1ms: model returned a completed response with no content]\n\n"}}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"Recovered."}}}}
|
||||
{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}}
|
||||
@@ -49,7 +49,7 @@ Every request carries the shared attribution header from dsh-llm's `attributionH
|
||||
|
||||
## Errors
|
||||
|
||||
Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA` (a response whose provider details identify exhausted quota, balance, or credits), `RATE_LIMIT` (other 429s), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s), `SERVER` (5xx), `HTTP_<status>` otherwise. Its serializable `failure` retains the HTTP status plus a valid positive `Retry-After` seconds/date delay and `x-request-id` / `x-deepseek-request-id` when present. A pre-response transport failure (DNS, refused connection, TLS, proxy) throws `TRANSPORT` naming the configured endpoint and chaining the original rejection as `cause`; caller aborts throw `ABORTED`, and the loop's cancellation signal remains authoritative. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', failure}` chunks.
|
||||
Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA` (a response whose provider details identify exhausted quota, balance, or credits), `RATE_LIMIT` (other 429s), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s), `SERVER` (5xx), `HTTP_<status>` otherwise. Its serializable `failure` retains the HTTP status plus a valid positive `Retry-After` seconds/date delay and `x-request-id` / `x-deepseek-request-id` when present. A pre-response transport failure (DNS, refused connection, TLS, proxy) throws `TRANSPORT` naming the configured endpoint and chaining the original rejection as `cause`; caller aborts throw `ABORTED`, and the loop's cancellation signal remains authoritative. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', failure}` chunks, and a completed stream whose `stop` (or absent) finish opened no content blocks becomes a `finish {kind: 'error'}` with code `EMPTY_RESPONSE` (retried by default policy).
|
||||
|
||||
## Testing
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
* @module dsh-llm-deepseek/translate
|
||||
*/
|
||||
|
||||
import { CallId, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId, EMPTY_RESPONSE_CODE, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, FinishReason, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
import { DONE } from './sse.ts'
|
||||
import type { WireChunk, WireUsage } from './types.ts'
|
||||
@@ -80,6 +80,8 @@ function closeBlock(block: OpenBlock): ContentBlock {
|
||||
* Malformed JSON payloads abort the stream with `MALFORMED_RESPONSE`.
|
||||
* @param payloads - SSE data payloads from {@link parseSse}, `[DONE]`-terminated.
|
||||
* @returns deltas as they arrive; `block-end`s, `usage`, and `finish` are all deferred to the `[DONE]` sentinel.
|
||||
* A `stop` (or absent) finish with no opened blocks is a degenerate provider completion and maps to an
|
||||
* `EMPTY_RESPONSE` error finish instead of a successful empty message.
|
||||
*/
|
||||
export async function* translate(payloads: AsyncIterable<string>): AsyncGenerator<StreamChunk> {
|
||||
let nextIndex = 0
|
||||
@@ -102,7 +104,16 @@ export async function* translate(payloads: AsyncIterable<string>): AsyncGenerato
|
||||
yield { type: 'block-end', index: block.index, block: closeBlock(block) }
|
||||
}
|
||||
if (pendingUsage) yield { type: 'usage', usage: pendingUsage }
|
||||
yield { type: 'finish', reason: pendingFinish ?? { kind: 'stop' } }
|
||||
const reason = pendingFinish ?? { kind: 'stop' as const }
|
||||
yield {
|
||||
type: 'finish',
|
||||
reason: reason.kind === 'stop' && order.length === 0
|
||||
? {
|
||||
kind: 'error',
|
||||
failure: { message: 'model returned a completed response with no content', code: EMPTY_RESPONSE_CODE },
|
||||
}
|
||||
: reason,
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { BlockAssembler, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { BlockAssembler, EMPTY_RESPONSE_CODE, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { DONE } from '../src/sse.ts'
|
||||
import { mapFinishReason, mapUsage, translate } from '../src/translate.ts'
|
||||
@@ -203,7 +203,50 @@ describe('translate: finish and usage handling', () => {
|
||||
|
||||
it('handles chunks with no choices at all', async () => {
|
||||
const chunks = await collect(translate(feed({}, DONE)))
|
||||
expect(chunks).toEqual([{ type: 'finish', reason: { kind: 'stop' } }])
|
||||
expect(chunks).toEqual([{
|
||||
type: 'finish',
|
||||
reason: {
|
||||
kind: 'error',
|
||||
failure: { message: 'model returned a completed response with no content', code: EMPTY_RESPONSE_CODE },
|
||||
},
|
||||
}])
|
||||
})
|
||||
|
||||
it('classifies an explicit stop with no opened blocks as EMPTY_RESPONSE, after usage', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: {}, finish_reason: 'stop' }], usage: { prompt_tokens: 7, completion_tokens: 0 } },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks).toEqual([
|
||||
{ type: 'usage', usage: { inputTokens: 7, outputTokens: 0 } },
|
||||
{
|
||||
type: 'finish',
|
||||
reason: {
|
||||
kind: 'error',
|
||||
failure: { message: 'model returned a completed response with no content', code: EMPTY_RESPONSE_CODE },
|
||||
},
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
it('keeps a reasoning-only stream a successful stop (any opened block counts)', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { content: null, reasoning_content: 'mull' } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'stop' }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks.at(-1)).toEqual({ type: 'finish', reason: { kind: 'stop' } })
|
||||
})
|
||||
|
||||
it('leaves non-stop finishes unclassified even with no opened blocks', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: {}, finish_reason: 'length' }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks.at(-1)).toEqual({ type: 'finish', reason: { kind: 'max-tokens' } })
|
||||
})
|
||||
})
|
||||
|
||||
|
||||
@@ -45,7 +45,7 @@ If a listener rewrites assembled assistant content, the loop drops replay state
|
||||
## Vocabulary differences
|
||||
|
||||
- pi-ai tool-call arguments are parsed objects; the harness stores raw JSON strings. The adapter parses input and re-stringifies output.
|
||||
- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted', failure}` chunks. Provider-specific error text distinguishes terminal `QUOTA` from transient `RATE_LIMIT`, while text and usage signals evaluated against the resolved model's context window normalize overflow to `CONTEXT_WINDOW_EXCEEDED`.
|
||||
- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted', failure}` chunks. Provider-specific error text distinguishes terminal `QUOTA` from transient `RATE_LIMIT`, while text and usage signals evaluated against the resolved model's context window normalize overflow to `CONTEXT_WINDOW_EXCEEDED`. A terminal `stop` whose message carries no content blocks maps to a `finish {kind:'error'}` with code `EMPTY_RESPONSE` (retried by default policy) instead of a successful empty message.
|
||||
- pi-ai folds reasoning tokens into output usage; there is no separate reasoning count to map.
|
||||
- `GenerateOptions.stop` is rejected with `UNSUPPORTED_OPTION` because pi-ai's common streaming surface cannot guarantee it across providers.
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
* @module dsh-llm-pi-ai/stream
|
||||
*/
|
||||
|
||||
import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmError, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, EMPTY_RESPONSE_CODE, isContextWindowExceededError, isQuotaExceededError, LlmError, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
|
||||
import type { FinishReason, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
import { isContextOverflow } from '@earendil-works/pi-ai'
|
||||
import type { AssistantMessage, AssistantMessageEvent, Usage as PiUsage } from '@earendil-works/pi-ai'
|
||||
@@ -48,7 +48,8 @@ function classifyPiAiError(message: string): string {
|
||||
* @param contextWindow - resolved catalog capacity for usage-based overflow detection.
|
||||
* @returns the mapped harness reason. Recognized error text, `stop` usage above
|
||||
* `contextWindow`, and zero-output `length` usage that fills the window map
|
||||
* to `CONTEXT_WINDOW_EXCEEDED`.
|
||||
* to `CONTEXT_WINDOW_EXCEEDED`; a `stop` with no content blocks maps to an
|
||||
* `EMPTY_RESPONSE` error.
|
||||
*/
|
||||
export function mapStopReason(message: AssistantMessage, contextWindow?: number): FinishReason {
|
||||
const piAiOverflow = isContextOverflow(message, contextWindow)
|
||||
@@ -66,7 +67,19 @@ export function mapStopReason(message: AssistantMessage, contextWindow?: number)
|
||||
}
|
||||
|
||||
switch (message.stopReason) {
|
||||
case 'stop': return { kind: 'stop' }
|
||||
case 'stop':
|
||||
// A terminal stop that produced no content blocks is a degenerate
|
||||
// provider completion, not a successful (empty) assistant message.
|
||||
if (message.content.length === 0) {
|
||||
return {
|
||||
kind: 'error',
|
||||
failure: {
|
||||
message: `model "${message.model}" returned a completed response with no content`,
|
||||
code: EMPTY_RESPONSE_CODE,
|
||||
},
|
||||
}
|
||||
}
|
||||
return { kind: 'stop' }
|
||||
case 'length': return { kind: 'max-tokens' }
|
||||
case 'toolUse': return { kind: 'tool-calls' }
|
||||
case 'aborted': return {
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, EMPTY_RESPONSE_CODE, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import type { AssistantMessage, AssistantMessageEvent, Usage } from '@earendil-works/pi-ai'
|
||||
import { toPiContext } from '../src/context.ts'
|
||||
@@ -520,7 +520,22 @@ describe('mapStopReason / mapUsage', () => {
|
||||
['toolUse', { kind: 'tool-calls' }],
|
||||
['aborted', { kind: 'aborted', failure: { message: 'pi-ai stream aborted', code: 'ABORTED' } }],
|
||||
] as const)('maps %s', (stopReason, expected) => {
|
||||
expect(mapStopReason(assistant({ stopReason }))).toEqual(expected)
|
||||
expect(mapStopReason(assistant({ stopReason, content: [{ type: 'text', text: 'ok' }] }))).toEqual(expected)
|
||||
})
|
||||
|
||||
it('classifies a completed stop with no content as an EMPTY_RESPONSE error', () => {
|
||||
expect(mapStopReason(assistant({ stopReason: 'stop' }))).toEqual({
|
||||
kind: 'error',
|
||||
failure: {
|
||||
message: 'model "deepseek-v4-flash" returned a completed response with no content',
|
||||
code: EMPTY_RESPONSE_CODE,
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps a thinking-only stop successful (any block counts as content)', () => {
|
||||
expect(mapStopReason(assistant({ stopReason: 'stop', content: [{ type: 'thinking', thinking: 'mull' }] })))
|
||||
.toEqual({ kind: 'stop' })
|
||||
})
|
||||
|
||||
it('defaults the error message when pi-ai omits it', () => {
|
||||
@@ -580,7 +595,9 @@ describe('mapStopReason / mapUsage', () => {
|
||||
})
|
||||
|
||||
it('uses the resolved context window for silent and length-stop overflows', () => {
|
||||
const silent = assistant({ stopReason: 'stop', usage: usage(101, 0) })
|
||||
// Non-empty content keeps the no-window branch on the successful stop path
|
||||
// (an empty stop is EMPTY_RESPONSE, covered above); overflow wins over both.
|
||||
const silent = assistant({ stopReason: 'stop', usage: usage(101, 0), content: [{ type: 'text', text: 'x' }] })
|
||||
expect(mapStopReason(silent)).toEqual({ kind: 'stop' })
|
||||
expect(mapStopReason(silent, 100)).toEqual({
|
||||
kind: 'error',
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Function plugin that retries selected transient model-request failures on the agent loop's closed-step recovery seam. It does not wrap `ctx.llm.stream()`: every adapter call remains one provider attempt, and every retry opens a fresh numbered step.
|
||||
|
||||
The default policy permits two retries for `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, using bounded exponential backoff from 500 ms to 10 seconds with 10 percent jitter. Delay bounds must fit Node's supported timer range. A valid `providerRetryAfterMs` replaces local backoff when it is within the configured cap; an over-cap instruction delegates to the next recovery policy instead.
|
||||
The default policy permits two retries for `EMPTY_RESPONSE`, `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, using bounded exponential backoff from 500 ms to 10 seconds with 10 percent jitter. `EMPTY_RESPONSE` is the adapters' classification of a degenerate provider completion (a terminal stop with zero content blocks); the attempt produced nothing durable, so repeating it is safe. Delay bounds must fit Node's supported timer range. A valid `providerRetryAfterMs` replaces local backoff when it is within the configured cap; an over-cap instruction delegates to the next recovery policy instead.
|
||||
|
||||
Before waiting, the plugin appends a non-surface `llm/retry` event with the failure and scheduled delay. Cancellation and plugin disposal abort the wait; disposal drains the plugin's active backoffs, and a callback captured before disposal fails closed if invoked afterward.
|
||||
|
||||
@@ -15,7 +15,7 @@ The separately published `./invariant` companion checks that every retry record
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
retryableCodes: [RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT]
|
||||
retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT]
|
||||
```
|
||||
|
||||
## Model Experience
|
||||
|
||||
@@ -33,7 +33,7 @@ const DEFAULT_MAX_TRANSIENT_RETRIES = 2
|
||||
const DEFAULT_INITIAL_DELAY_MS = 500
|
||||
const DEFAULT_MAX_DELAY_MS = 10_000
|
||||
const DEFAULT_JITTER_RATIO = 0.1
|
||||
const DEFAULT_RETRYABLE_CODES = Object.freeze(['RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT'])
|
||||
const DEFAULT_RETRYABLE_CODES = Object.freeze(['EMPTY_RESPONSE', 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT'])
|
||||
|
||||
/** Deployment-owned limits and classification for transient request recovery. */
|
||||
export interface Config {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import type { Fiber } from 'cordis'
|
||||
import LlmService, { CallId, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, { CallId, EMPTY_RESPONSE_CODE, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
@@ -51,6 +51,25 @@ function textResponse(text: string): StreamChunk[] {
|
||||
]
|
||||
}
|
||||
|
||||
/**
|
||||
* A degenerate empty provider completion as an error finish chunk. Both
|
||||
* adapters emit this shape and the EMPTY_RESPONSE code (the field the policy
|
||||
* routes on); the message text here is the deepseek adapter's phrasing (pi-ai
|
||||
* qualifies it with the model name).
|
||||
*/
|
||||
function emptyCompletion(): StreamChunk[] {
|
||||
return [
|
||||
{ type: 'usage', usage: { inputTokens: 0, outputTokens: 0 } },
|
||||
{
|
||||
type: 'finish',
|
||||
reason: {
|
||||
kind: 'error',
|
||||
failure: { message: 'model returned a completed response with no content', code: EMPTY_RESPONSE_CODE },
|
||||
},
|
||||
},
|
||||
]
|
||||
}
|
||||
|
||||
async function harness(
|
||||
adapter: LlmAdapter,
|
||||
config: retry.Config = {},
|
||||
@@ -158,6 +177,39 @@ describe('bounded transient retry policy', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('retries an EMPTY_RESPONSE error finish under the default retryable codes', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
emptyCompletion(),
|
||||
textResponse('recovered'),
|
||||
])
|
||||
// No retryableCodes override: this proves the default policy covers the
|
||||
// adapters' empty-completion classification end to end (finish-chunk error
|
||||
// delivery, not a thrown stream error).
|
||||
;({ ctx: context } = await harness(adapter))
|
||||
const agent = context.agentLoop.create(SessionId('retry-empty-response'), { provider: 'mock', model: 'mock' })
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
const event = await scheduled
|
||||
expect(event.data.failure).toEqual({
|
||||
message: 'model returned a completed response with no content',
|
||||
code: EMPTY_RESPONSE_CODE,
|
||||
})
|
||||
|
||||
const idle = waitForIdle(context, agent)
|
||||
await vi.advanceTimersByTimeAsync(500)
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(2)
|
||||
expect(agent.session.events.filter(event => event.type === 'assistant/message').map(event => event.data.step))
|
||||
.toEqual([2])
|
||||
expect(agent.session.deriveMessages().at(-1)).toMatchObject({
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'recovered' }],
|
||||
})
|
||||
})
|
||||
|
||||
it('leaves partial failed chunks on their step without committing a message or tool side effect', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
|
||||
@@ -54,6 +54,7 @@ Every product adapter sends application identity on provider HTTP requests. `att
|
||||
- `errorChain(value)` — renders a thrown value with its full `cause` chain and AggregateError members for diagnostic surfaces (UI notices, logger lines, durable `turn/end` messages), so transport wrappers like undici's `TypeError: fetch failed` surface the underlying `ECONNREFUSED`/DNS/TLS detail instead of masking it. Rendering only — route on `code`, never by parsing the result.
|
||||
- `CONTEXT_WINDOW_EXCEEDED_CODE` — the provider-neutral code both DeepSeek adapters use when a request exceeds the model context window, regardless of thrown-HTTP versus in-band finish delivery. `isContextWindowExceededError(detail)` is their shared conservative classifier for OpenAI-compatible provider detail.
|
||||
- `QUOTA_EXCEEDED_CODE` — the non-transient provider-neutral code for exhausted account quota, balance, credits, budget, or usage limits. `isQuotaExceededError(detail)` keeps those failures distinct from request-rate limits.
|
||||
- `EMPTY_RESPONSE_CODE` — the provider-neutral code both adapters use for a degenerate provider completion: a terminal `stop` that carried no content blocks at all. Classified as an error finish (not a successful empty message) because the attempt produced nothing durable; `dsh-llm-retry` retries it by default.
|
||||
|
||||
### Real adapters
|
||||
|
||||
|
||||
@@ -27,6 +27,17 @@ export const CONTEXT_WINDOW_EXCEEDED_CODE = 'CONTEXT_WINDOW_EXCEEDED'
|
||||
/** Canonical provider-neutral code for an exhausted account quota or balance. */
|
||||
export const QUOTA_EXCEEDED_CODE = 'QUOTA'
|
||||
|
||||
/**
|
||||
* Canonical provider-neutral code for a response that completed normally but
|
||||
* carried no content blocks at all. Providers occasionally emit a degenerate
|
||||
* completion (a terminal stop with zero output); adapters classify it as this
|
||||
* failure instead of yielding an empty assistant message, because an empty
|
||||
* message silently ends the turn with nothing for the user or the loop to act
|
||||
* on. The attempt produced nothing durable, so retry policy treats it as safe
|
||||
* to repeat.
|
||||
*/
|
||||
export const EMPTY_RESPONSE_CODE = 'EMPTY_RESPONSE'
|
||||
|
||||
/** Structured codes and plain phrases that explicitly name a context bound being exceeded. */
|
||||
const STRUCTURED_CONTEXT_OVERFLOW = new RegExp(
|
||||
String.raw`(?:^|[^a-z0-9])context[\s_-](?:length|window)[\s_-]`
|
||||
|
||||
Reference in New Issue
Block a user