mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
fix(llm): bind reasoning resolution to adapter lifecycle
This commit is contained in:
Yichen Jiang
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-24-adapter-owned-reasoning-effort-capabilities.md: 806e808ab18d617003f757200f0cdee5853476f5
|
||||
2026-07-24-adapter-owned-reasoning-effort-capabilities.zh.md: 9be8b6f4f7bcbe063ff90492ac44b9590860f67f
|
||||
2026-07-24-adapter-owned-reasoning-effort-capabilities.md: 04aebf5fa61d896ac16b29578a39c9b0e6e38cf4
|
||||
2026-07-24-adapter-owned-reasoning-effort-capabilities.zh.md: a1c89417562276fd70ac5e5c1be17c9e639eb3ed
|
||||
|
||||
@@ -12,7 +12,7 @@ Reasoning strength was adapter configuration only, so a conversation could not d
|
||||
|
||||
`dsh-llm` represents a reasoning effort as the opaque branded `ReasoningEffortId`. An adapter's `resolveModelReasoning(provider, model)` returns a non-empty ordered list of ids with display metadata and may name one configured default. The core validates metadata, requires an explicit or configured effort to appear exactly in that list, and never clamps or aliases a value.
|
||||
|
||||
`LlmCallConfig` and `GenerateOptions` carry the optional effort. The agent loop resolves the post-`agent/request` config before writing `request/header`, so defaults and dynamic changes are model-visible only after becoming durable facts. A route with no registered adapter retains its proposed config so an `llm/stream` middleware can own and short-circuit it; terminal dispatch still rejects an unhandled route. A resumed loop retains the logged effort only when its initial provider/model route is unchanged; a route change discards the previous model's opaque id. The terminal `LlmService` adapter boundary repeats resolution for direct calls that do not pass through the loop.
|
||||
`LlmCallConfig` and `GenerateOptions` carry the optional effort. The agent loop prepares the post-`agent/request` config under the active turn signal before writing `request/header`, so defaults and dynamic changes are model-visible only after becoming durable facts. The prepared call retains the exact adapter registration across asynchronous capability resolution, durable header logging, and dispatch; direct `LlmService.stream()` calls likewise capture their final registration before awaiting resolution. A route with no registered adapter retains its proposed config so an `llm/stream` middleware can own and short-circuit it; terminal dispatch still rejects an unhandled route. A resumed loop retains the logged effort only when its initial provider/model route is unchanged; a route change discards the previous model's opaque id.
|
||||
|
||||
The native DeepSeek adapter advertises `high` and `max`, defaults to configured effort or `high`, and exposes no effort capability while thinking is disabled. The pi-ai adapter derives each exact model's list from `getSupportedThinkingLevels()`, excludes `off`, preserves an absent profile default as a provider default, and leaves provider wire-value mapping inside pi-ai.
|
||||
|
||||
@@ -30,4 +30,4 @@ The native DeepSeek adapter advertises `high` and `max`, defaults to configured
|
||||
|
||||
Clients can query one exact route and render the adapter's order and names without knowing a global enum. Adapter configuration remains the deployment-default owner, while `agent/request` can replace the effective effort on each step. Invalid metadata fails with `INVALID_MODEL_REASONING`, and unsupported explicit or configured values fail with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
|
||||
|
||||
The capability query is asynchronous and exact-model resolution may fail for adapters backed by authoritative catalogs. Keyless service, adapter, loop, session, and request-header tests pin validation, defaulting, dynamic changes, logging, and resume behavior; runnable snapshots pin the resolved effort in real assembled request headers, while key-gated adapter tests exercise provider serialization.
|
||||
The capability query is asynchronous and exact-model resolution may fail for adapters backed by authoritative catalogs. Its optional signal is the caller's cancellation boundary; an asynchronous adapter must settle promptly after abort so loop disposal can reach quiescence. Keyless service, adapter, loop, session, and request-header tests pin validation, defaulting, dynamic changes, logging, resume behavior, HMR registration ownership, and cancellation; runnable snapshots pin the resolved effort in real assembled request headers, while key-gated adapter tests exercise provider serialization.
|
||||
|
||||
@@ -12,7 +12,7 @@ Status: implemented
|
||||
|
||||
`dsh-llm` 使用不透明的品牌类型 `ReasoningEffortId` 表示推理强度。适配器的 `resolveModelReasoning(provider, model)` 返回非空的有序 ID 列表及其展示元数据，并可指定一个由配置确定的默认值。核心会校验元数据，要求显式指定或配置指定的推理强度与列表中的某个 ID 完全一致，且绝不自动调整或为值提供别名。
|
||||
|
||||
`LlmCallConfig` 和 `GenerateOptions` 携带可选的推理强度。agent loop（智能体循环）在 `agent/request` 处理完成后、写入 `request/header` 前解析配置，因此默认值和动态变更只有成为持久化事实后才对模型可见。没有已注册适配器的路由会保留原定配置，使 `llm/stream` 中间件可以接管并短路该请求；若仍未得到处理，最终分发会拒绝该路由。恢复后的主循环仅在初始提供方/模型路由未变时保留日志中记录的推理强度；如果路由发生变化，则丢弃上一模型的不透明 ID。最终的 `LlmService` 适配器边界会再次执行解析，以覆盖未经过主循环的直接调用。
|
||||
`LlmCallConfig` 和 `GenerateOptions` 携带可选的推理强度。agent loop（智能体循环）在活跃轮次信号的控制下准备 `agent/request` 处理完成后的配置，再写入 `request/header`，因此默认值和动态变更只有成为持久化事实后才对模型可见。准备完成的调用在异步能力解析、请求头持久记录和分派全程保留同一项确切的适配器注册；直接调用 `LlmService.stream()` 时，也会在等待解析前捕获最终的适配器注册。没有已注册适配器的路由会保留原定配置，使 `llm/stream` 中间件可以接管并短路该请求；若仍未得到处理，最终分发会拒绝该路由。恢复后的主循环仅在初始提供方/模型路由未变时保留日志中记录的推理强度；如果路由发生变化，则丢弃上一模型的不透明 ID。
|
||||
|
||||
原生 DeepSeek 适配器声明 `high` 和 `max`，默认使用配置指定的推理强度，若未配置则使用 `high`；禁用思考时不暴露推理强度能力。pi-ai 适配器通过 `getSupportedThinkingLevels()` 按具体模型推导等级列表，排除 `off`，在 profile 未指定默认值时保留提供方默认行为，并将提供方协议值的映射留在 pi-ai 内部。
|
||||
|
||||
@@ -30,4 +30,4 @@ Status: implemented
|
||||
|
||||
客户端可以查询一条确切路由，并按适配器给出的顺序和名称渲染等级，而无需了解全局枚举。适配器配置仍负责提供部署默认值，`agent/request` 则可以在每个步骤替换实际生效的推理强度。元数据无效时抛出 `INVALID_MODEL_REASONING`；显式指定或配置指定的值不受支持时，会在提供方 I/O 前抛出 `UNSUPPORTED_REASONING_EFFORT`。
|
||||
|
||||
能力查询采用异步方式；对于由权威目录支持的适配器，确切模型解析可能失败。无密钥的服务、适配器、主循环、会话和请求头测试为校验、默认值解析、动态变更、日志记录和恢复行为提供回归保障；可运行快照锁定实际组装请求头中的已解析推理强度，仅在有密钥时运行的适配器测试则覆盖提供方序列化。
|
||||
能力查询采用异步方式；对于由权威目录支持的适配器，确切模型解析可能失败。可选信号构成调用方的取消边界；异步适配器必须在信号中止后迅速完成结算，使主循环的资源释放达到完全停稳。无密钥的服务、适配器、主循环、会话和请求头测试为校验、默认值解析、动态变更、日志记录、恢复行为、HMR（热模块替换）期间的注册所有权和取消提供回归保障；可运行快照锁定实际组装请求头中的已解析推理强度，仅在有密钥时运行的适配器测试则覆盖提供方序列化。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
architecture.md: 124f1caadb3f8557698965ccb3d17587b7b9ed8a
|
||||
architecture.zh.md: af6fdee935f665809dfd35218a0398118ed5dba9
|
||||
architecture.md: 88b3a1d133c6e4b28ae39b379678a3202124979f
|
||||
architecture.zh.md: ccc29ddfb1c7e80f550fda6fb3f759be1bef1946
|
||||
|
||||
@@ -91,7 +91,7 @@ forever:
|
||||
agent/pre-step
|
||||
snapshot the derived messages (the reconstruction boundary)
|
||||
'step/start'
|
||||
agent/request -> resolve reasoning/default -> log request/header -> checkpoint -> llm/stream (frozen)
|
||||
agent/request -> prepare reasoning/default under turn signal -> log request/header -> checkpoint -> llm/stream (frozen, registration-bound)
|
||||
on final adapter-path or terminal in-band failure:
|
||||
'step/end'
|
||||
agent/request-error(original error, failure facts, immutable prior failures, signal)
|
||||
@@ -125,7 +125,7 @@ Pruning precedes summaries; overflow retries require durable progress. Bounded r
|
||||
|
||||
Adapter failures close the step before `agent/request-error` with exact `Error`, `LlmFailure`, and history. Retries open steps; success clears history; exhaustion stores failure on `turn/end`. Failed chunks commit nothing.
|
||||
|
||||
Other failures use `agent/error`. Cancellation and disposal beat recovery; undispatched tools get synthetic `tool/call`/`ABORTED_BEFORE_DISPATCH` pairs. The signal retires before `turn/end`. Effective `cancel()` emits its cause, clears queues, and aborts; observers cannot veto, idle calls emit nothing, and durability records `aborted`. Disposal awaits quiescence ([decision](../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md)).
|
||||
Other failures use `agent/error`. Cancellation and disposal beat recovery; the turn signal also cancels asynchronous model-capability preparation before any request header is committed, and undispatched tools get synthetic `tool/call`/`ABORTED_BEFORE_DISPATCH` pairs. The signal retires before `turn/end`. Effective `cancel()` emits its cause, clears queues, and aborts; observers cannot veto, idle calls emit nothing, and durability records `aborted`. Disposal awaits quiescence ([decision](../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md)).
|
||||
|
||||
Session events are turn-enclosed; reload closes an interrupted tail with a synthetic `interrupted` turn end. Post-close failures use `agent/error`. Each turn has one [TurnEndReason](core-data-structures/session.md#why-a-turn-ended-turnendreasonmap).
|
||||
|
||||
|
||||
@@ -91,7 +91,7 @@ forever:
|
||||
agent/pre-step
|
||||
snapshot the derived messages (the reconstruction boundary)
|
||||
'step/start'
|
||||
agent/request -> resolve reasoning/default -> log request/header -> checkpoint -> llm/stream (frozen)
|
||||
agent/request -> prepare reasoning/default under turn signal -> log request/header -> checkpoint -> llm/stream (frozen, registration-bound)
|
||||
on final adapter-path or terminal in-band failure:
|
||||
'step/end'
|
||||
agent/request-error(original error, failure facts, immutable prior failures, signal)
|
||||
@@ -125,7 +125,7 @@ forever:
|
||||
|
||||
适配器故障会先关闭步骤，再进入 `agent/request-error`；该事件会收到准确的 `Error`、`LlmFailure` 和历史记录。重试会开启步骤；成功会清除历史记录；重试耗尽后，故障存入 `turn/end`。失败分片不会提交任何内容。
|
||||
|
||||
其他故障使用 `agent/error`。取消和资源释放均优先于恢复；尚未分派的工具会得到合成的 `tool/call`/`ABORTED_BEFORE_DISPATCH` 对。信号会在 `turn/end` 前失效。实际生效的 `cancel()` 会发出原因、清空队列并中止；观察方不能否决该操作，空闲状态下的调用不发出任何事件，持久化会记录 `aborted`。dispose（资源释放）会等待系统停稳（[决策](../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md)）。
|
||||
其他故障使用 `agent/error`。取消和资源释放均优先于恢复；轮次信号还会在提交任何请求头之前取消异步模型能力准备，尚未分派的工具会得到合成的 `tool/call`/`ABORTED_BEFORE_DISPATCH` 对。信号会在 `turn/end` 前失效。实际生效的 `cancel()` 会发出原因、清空队列并中止；观察方不能否决该操作，空闲状态下的调用不发出任何事件，持久化会记录 `aborted`。dispose（资源释放）会等待系统停稳（[决策](../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md)）。
|
||||
|
||||
会话事件均位于轮次边界内；重新加载会用合成的 `interrupted` 轮次结束事件闭合中断的日志尾部。关闭后的故障使用 `agent/error`。每个轮次有一个 [TurnEndReason](core-data-structures/session.md#why-a-turn-ended-turnendreasonmap)。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
adding-an-llm-adapter.md: 3adae89f360cd81e264912d74bd86f5a06cd42f8
|
||||
adding-an-llm-adapter.zh.md: 80329da5e7c89bfb5ee2e2aa5b9931a3ef0e83f6
|
||||
adding-an-llm-adapter.md: 76bcfdd6638ef8b40977a5e1fe4f678648c043cd
|
||||
adding-an-llm-adapter.zh.md: 13ae388bb0c6f29ab6269f364b849cb1694e2ef2
|
||||
|
||||
@@ -32,7 +32,7 @@ Registration is effect-based (HMR-safe); one adapter per provider route — dupl
|
||||
- A `GenerateOptions` field your provider cannot honor (e.g. a `stop` list on a provider without stop sequences): throw `LlmError(..., 'UNSUPPORTED')` rather than silently dropping it.
|
||||
- If the provider requires response ids, signatures, or other native metadata on follow-up calls, emit the minimal lossless-JSON projection as `finish.replayState`. Validate it when rebuilding history. `LlmService` passes it only when the historical provider route and target provider route are currently owned by the exact same adapter instance; your adapter decides whether same-model, cross-model, or cross-provider restoration is legal. Never infer native replay from provider/model names alone when state is absent.
|
||||
|
||||
Provider-specific thinking-mode toggles remain in the adapter's Config. Selectable reasoning strength uses the provider-neutral capability seam: return ordered opaque ids from `resolveModelReasoning()`, declare a configured `defaultEffort` only when one exists, and map `GenerateOptions.reasoningEffort` to the provider wire value. Do not expose provider wire spellings, clamp unsupported values, or include an `off` mode as an effort.
|
||||
Provider-specific thinking-mode toggles remain in the adapter's Config. Selectable reasoning strength uses the provider-neutral capability seam: return ordered opaque ids from `resolveModelReasoning()`, declare a configured `defaultEffort` only when one exists, honor the resolver's optional `AbortSignal`, and map `GenerateOptions.reasoningEffort` to the provider wire value. Do not expose provider wire spellings, clamp unsupported values, or include an `off` mode as an effort.
|
||||
|
||||
## Structure that worked
|
||||
|
||||
|
||||
@@ -32,7 +32,7 @@ export function apply(ctx: Context, config: Config) {
|
||||
- 如果 `GenerateOptions` 中某个字段你的提供方无法支持（例如提供方不支持 stop sequences 时收到 `stop` 列表）：抛出 `LlmError(..., 'UNSUPPORTED')`，而非静默丢弃。
|
||||
- 如果提供方在后续调用中需要响应 ID、签名或其他原生元数据，请将其最小无损 JSON 投影作为 `finish.replayState` 发出。重建历史时验证该状态。只有历史提供方路由和目标提供方路由当前由完全相同的适配器实例拥有时，`LlmService` 才会传递该状态；由适配器决定同模型、跨模型或跨提供方恢复是否合法。状态缺失时，切勿仅根据提供方/模型名称推断原生回放。
|
||||
|
||||
提供方特有的 thinking 模式开关仍放在适配器的 Config 中。可选的推理强度使用提供方无关的能力 seam：`resolveModelReasoning()` 返回有序的不透明 ID；仅当存在配置指定的默认值时才声明 `defaultEffort`；并将 `GenerateOptions.reasoningEffort` 映射为提供方协议值。不得暴露提供方协议值的具体拼写、自动调整不支持的值，也不得把 `off` 模式列为推理强度。
|
||||
提供方特有的 thinking 模式开关仍放在适配器的 Config 中。可选的推理强度使用提供方无关的能力 seam：`resolveModelReasoning()` 返回有序的不透明 ID；仅当存在配置指定的默认值时才声明 `defaultEffort`；响应传给解析器的可选 `AbortSignal`；并将 `GenerateOptions.reasoningEffort` 映射为提供方协议值。不得暴露提供方协议值的具体拼写、自动调整不支持的值，也不得把 `off` 模式列为推理强度。
|
||||
|
||||
## 经验证有效的结构
|
||||
|
||||
|
||||
@@ -745,37 +745,52 @@ async resolveModelContext( provider: string, model: string, ): Promise<LlmModelC
|
||||
* effort selector is unsupported for that model.
|
||||
* @param provider - registered provider route to inspect.
|
||||
* @param model - exact model id passed to the adapter.
|
||||
* @param signal - optional cancellation for adapter-owned asynchronous lookup.
|
||||
* @returns detached reasoning metadata, or `undefined` when unsupported.
|
||||
*/
|
||||
async resolveModelReasoning( provider: string, model: string, ): Promise<LlmModelReasoningInfo | undefined>
|
||||
async resolveModelReasoning( provider: string, model: string, signal?: AbortSignal, ): Promise<LlmModelReasoningInfo | undefined>
|
||||
|
||||
/**
|
||||
* Validate a conversation call config against its exact model capability and
|
||||
* materialize an adapter-configured default. Unsupported explicit efforts
|
||||
* reject before provider I/O; no clamping or aliasing is performed.
|
||||
* reject before provider I/O; no clamping or aliasing is performed. This
|
||||
* standalone query does not bind a later dispatch; use {@link prepareCall}
|
||||
* when logging and streaming must share one adapter registration.
|
||||
* @param config - provider/model route and optional request controls.
|
||||
* @param signal - optional cancellation for adapter-owned capability lookup.
|
||||
* @returns a detached config only when a default must be materialized.
|
||||
*/
|
||||
async resolveCallConfig(config: LlmCallConfig): Promise<LlmCallConfig>
|
||||
async resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>
|
||||
|
||||
/**
|
||||
* Resolve one call under its current adapter registration. The returned
|
||||
* one-shot handle keeps that registration across header logging and dispatch,
|
||||
* so HMR cannot combine one adapter's capability result with another adapter.
|
||||
* @param config - provider/model route and optional request controls.
|
||||
* @param signal - optional cancellation for adapter-owned capability lookup.
|
||||
* @returns a prepared config and its registration-bound stream entry point.
|
||||
*/
|
||||
async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>
|
||||
|
||||
/**
|
||||
* Stream one model call as raw chunks (token-level deltas). Throws
|
||||
* `LlmError` with code `NO_ADAPTER` if no adapter is registered for
|
||||
* `options.provider`. Replay state is retained only when the same adapter
|
||||
* instance owns its historical provider and the target provider. Final
|
||||
* adapter selection, dispatch, and iteration failures retain their original
|
||||
* Error identity and are tagged in a call-local scope for narrow agent-loop
|
||||
* request recovery; middleware and nested-call failures remain untagged for
|
||||
* the outer call.
|
||||
* adapter selection remains fixed through asynchronous reasoning resolution
|
||||
* and dispatch. Selection, dispatch, and iteration failures retain their
|
||||
* original Error identity and are tagged in a call-local scope for narrow
|
||||
* agent-loop request recovery; middleware and nested-call failures remain
|
||||
* untagged for the outer call.
|
||||
* @param options - the full request; `options.provider` selects the adapter.
|
||||
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
```
|
||||
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmCallConfig](../core-data-structures/core.md) · [LlmModelContext](../core-data-structures/core.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmModelReasoningInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmCallConfig](../core-data-structures/core.md) · [LlmModelContext](../core-data-structures/core.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmModelReasoningInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [PreparedLlmCall](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/llm/llm/src/index.ts:175`](../../packages/llm/llm/src/index.ts)
|
||||
Source: [`packages/llm/llm/src/index.ts:192`](../../packages/llm/llm/src/index.ts)
|
||||
|
||||
## `ctx.permission` — `PermissionService`
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
core.md: 7938bd2766735303153b741b7442b698067d9089
|
||||
core.zh.md: 7c5298659cd5e9b004969ee4e626be4fad62dc18
|
||||
core.md: d60268a163580558b09ae0af7b15b78232436a4b
|
||||
core.zh.md: b81c6a90ab7f74580b0f6599bd78868771f3e339
|
||||
|
||||
@@ -325,7 +325,7 @@ The model-facing `ToolSchema` is the wire shape; the registered `ToolDefinition`
|
||||
|
||||
The loop builds each request from logged state. `EpochHeader` records call config, rendered prompt, authoritative returned tool order (configured by `toolOrder`, or lexicographic when unset), and session prefix through full `request/header` snapshots. Together with derived history, this makes the request reconstructable from the session log. See [session.md](session.md#the-request-header-event-requestheader) and the [reconstructability Agent Note](../../.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md).
|
||||
|
||||
`agent/request` receives a frozen call-config seed and may return a replacement to switch provider, model, reasoning effort, or sampling. The loop resolves the exact model capability after the waterfall, rejects unsupported explicit effort ids without clamping, materializes an adapter-configured default, and logs the effective value. `agent/session-prefix` composes request-only prefix messages once per loop instance, and the header records the exact result used. Requests reaching `llm/stream` are deep-frozen, so mutation throws, and carry a process-local loop identity so observers do not confuse separately logged frozen auxiliary calls with conversation requests.
|
||||
`agent/request` receives a frozen call-config seed and may return a replacement to switch provider, model, reasoning effort, or sampling. After the waterfall, the loop prepares the exact model capability under the turn signal, rejects unsupported explicit effort ids without clamping, materializes an adapter-configured default, and logs the effective value. The prepared call keeps one adapter registration through dispatch. `agent/session-prefix` composes request-only prefix messages once per loop instance, and the header records the exact result used. Requests reaching `llm/stream` are deep-frozen, so mutation throws, and carry a process-local loop identity so observers do not confuse separately logged frozen auxiliary calls with conversation requests.
|
||||
|
||||
On the wire, a loop-built request reads in this order: the `system` slot (the rendered prompt assembly) → `messagePrefix` (the frozen session prefix) → the derived history — the boundary snapshot, whose tail is the newest `user/message` on a turn's first step and the previous step's tool results on later steps. The prefix never enters the derived history; its durable record is the header events, and the dev invariant recomputes exactly this equation against every loop-built request.
|
||||
|
||||
|
||||
@@ -331,7 +331,7 @@ interface ToolSchema {
|
||||
|
||||
循环从已记录状态构建每个请求。`EpochHeader` 通过完整的 `request/header` 快照记录调用配置、渲染后的提示词、权威返回工具顺序（由 `toolOrder` 配置；未配置时按字典序）以及会话前缀。结合派生历史，请求便可由会话日志重建。见 [session.md](session.md#the-request-header-event-requestheader) 与[可重建性 Agent Note（agent 决策记录）](../../.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md)。
|
||||
|
||||
`agent/request` 接收冻结的调用配置种子，并可返回替代值以切换提供方、模型、推理强度或采样参数。循环在 waterfall 之后解析确切模型的能力，拒绝显式指定但不受支持的推理强度 ID（不自动调整），填入适配器配置的默认值，并记录最终生效值。`agent/session-prefix` 为每个循环实例组合一次仅用于请求的 prefix 消息，header 记录实际使用的确切结果。到达 `llm/stream` 的请求会被深度冻结，因此变更会抛异常；请求还携带进程本地循环标识，使观察者不会把单独记录的冻结辅助调用误认成对话请求。
|
||||
`agent/request` 接收冻结的调用配置种子，并可返回替代值以切换提供方、模型、推理强度或采样参数。waterfall 结束后，循环会在轮次信号控制下完成确切模型的能力准备，拒绝显式指定但不受支持的推理强度 ID（不自动调整），填入适配器配置的默认值，并记录最终生效值。准备完成的调用直至分派完成始终持有同一项适配器注册。`agent/session-prefix` 为每个循环实例组合一次仅用于请求的 prefix 消息，header 记录实际使用的确切结果。到达 `llm/stream` 的请求会被深度冻结，因此变更会抛异常；请求还携带进程本地循环标识，使观察者不会把单独记录的冻结辅助调用误认成对话请求。
|
||||
|
||||
在协议格式上，循环构建的请求按此顺序读取：`system` 槽位（渲染后的提示词组装）→ `messagePrefix`（冻结的会话前缀）→ 派生历史——边界快照，其尾部在轮次首步是最新的 `user/message`，在后续步骤是上一步的工具结果。前缀从不进入派生历史；它的持久记录是 header 事件，开发不变式针对每个循环构建的请求精确重算此等式。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
llm-streaming.md: eec706d3b440833a78d7f5c24e716809fe961ffd
|
||||
llm-streaming.zh.md: ce93bb905234b6369389c7d9615ed621f224bc6d
|
||||
llm-streaming.md: 5155e680bfcc253d06742c7d3adcddc633a01c52
|
||||
llm-streaming.zh.md: 9b928253065da99c74f749b905375d790fab46c9
|
||||
|
||||
@@ -157,7 +157,23 @@ declare class BlockAssembler {
|
||||
|
||||
## The seam
|
||||
|
||||
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerInfo()` and asynchronous `listModels()` methods feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. The separate `resolveModelContext()` query exposes correctness-sensitive capacity, while `resolveModelReasoning()` exposes ordered model-owned effort ids and an optional deployment default; absence from either query means unavailable metadata or capability, not invalid catalog membership. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
|
||||
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerInfo()` and asynchronous `listModels()` methods feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. The separate `resolveModelContext()` query exposes correctness-sensitive capacity, while `resolveModelReasoning()` exposes ordered model-owned effort ids and an optional deployment default; absence from either query means unavailable metadata or capability, not invalid catalog membership. A reasoning resolver receives optional cancellation and must settle promptly after abort. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across capability resolution, durable header logging, and dispatch. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
|
||||
|
||||
```ts type-equiv
|
||||
/** One model call whose config and adapter registration were resolved together. */
|
||||
interface PreparedLlmCall {
|
||||
/** Detached, deep-frozen config with any adapter-owned default materialized. */
|
||||
readonly config: LlmCallConfig
|
||||
/**
|
||||
* Dispatch this call once through the registration captured during
|
||||
* preparation. The request's call-config fields must match {@link config};
|
||||
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
|
||||
* @param options - fully assembled request carrying the prepared config.
|
||||
* @returns the chunk stream, including the `llm/stream` waterfall.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
}
|
||||
```
|
||||
|
||||
```ts public-api
|
||||
/**
|
||||
@@ -197,11 +213,14 @@ declare abstract class LlmAdapter {
|
||||
* the model has no selectable reasoning-effort capability.
|
||||
* @param _provider - one provider route owned by this adapter.
|
||||
* @param _model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @param _signal - cancellation for this exact-model lookup; implementations
|
||||
* must settle promptly after it aborts.
|
||||
* @returns adapter-owned effort metadata, or `undefined` when unsupported.
|
||||
*/
|
||||
resolveModelReasoning(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo | undefined>;
|
||||
/**
|
||||
* Stream one model call as raw chunks. The only required method.
|
||||
|
||||
@@ -157,7 +157,23 @@ declare class BlockAssembler {
|
||||
|
||||
## seam
|
||||
|
||||
`LlmAdapter` 是提供方 seam：创建子类、实现 `stream()`，再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器；`GenerateOptions.model` 会传给该适配器，无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerInfo()` 与异步 `listModels()` 方法为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考，不是请求白名单：适配器仍是权威，并可接受未列出的模型 id。单独的 `resolveModelContext()` 查询暴露对正确性敏感的容量信息，`resolveModelReasoning()` 则暴露由模型持有的有序推理强度 ID 和可选的部署默认值；任一查询返回缺失都表示元数据或能力不可用，而不表示目录成员关系无效。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值，因此直接调用也无法绕过对不支持推理强度的拒绝。适配器查找发生在 `llm/stream` waterfall（瀑布式事件）的终端 continuation，因此 listener 可以在查找前短路调用，或路由一个可变的一次性请求。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface（`ctx.llm.stream()`）与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
|
||||
`LlmAdapter` 是提供方 seam：创建子类、实现 `stream()`，再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器；`GenerateOptions.model` 会传给该适配器，无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerInfo()` 与异步 `listModels()` 方法为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考，不是请求白名单：适配器仍是权威，并可接受未列出的模型 id。单独的 `resolveModelContext()` 查询暴露对正确性敏感的容量信息，`resolveModelReasoning()` 则暴露由模型持有的有序推理强度 ID 和可选的部署默认值；任一查询返回缺失都表示元数据或能力不可用，而不表示目录成员关系无效。推理能力解析器会接收可选的取消信号，并且必须在信号中止后迅速完成结算。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值，因此直接调用也无法绕过对不支持推理强度的拒绝；直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`，使能力解析、请求头持久记录和分派全程使用同一项注册。适配器查找发生在 `llm/stream` waterfall（瀑布式事件）的终端 continuation，因此 listener 可以在查找前短路调用，或路由一个可变的一次性请求。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface（`ctx.llm.stream()`）与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
|
||||
|
||||
```ts type-equiv
|
||||
/** One model call whose config and adapter registration were resolved together. */
|
||||
interface PreparedLlmCall {
|
||||
/** Detached, deep-frozen config with any adapter-owned default materialized. */
|
||||
readonly config: LlmCallConfig
|
||||
/**
|
||||
* Dispatch this call once through the registration captured during
|
||||
* preparation. The request's call-config fields must match {@link config};
|
||||
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
|
||||
* @param options - fully assembled request carrying the prepared config.
|
||||
* @returns the chunk stream, including the `llm/stream` waterfall.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
}
|
||||
```
|
||||
|
||||
```ts public-api
|
||||
/**
|
||||
@@ -197,11 +213,14 @@ declare abstract class LlmAdapter {
|
||||
* the model has no selectable reasoning-effort capability.
|
||||
* @param _provider - one provider route owned by this adapter.
|
||||
* @param _model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @param _signal - cancellation for this exact-model lookup; implementations
|
||||
* must settle promptly after it aborts.
|
||||
* @returns adapter-owned effort metadata, or `undefined` when unsupported.
|
||||
*/
|
||||
resolveModelReasoning(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo | undefined>;
|
||||
/**
|
||||
* Stream one model call as raw chunks. The only required method.
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
llm-adapter.md: f133a6ea748fd7d9bfa6a7adaa35b1cccbe40c8e
|
||||
llm-adapter.zh.md: 3a4fd9a516126d1f9fa8675fdc87be10c85f3489
|
||||
llm-adapter.md: 8b8201099e2b78385301daf14605be53436353c7
|
||||
llm-adapter.zh.md: 735b7a4babf3d31652374e91cbce794cdd580725
|
||||
|
||||
@@ -112,7 +112,7 @@ async function* exampleChunks(): AsyncIterable<StreamChunk> {
|
||||
|
||||
`stream()` receives the exported `GenerateOptions` type. It includes the model, adapter-owned reasoning-effort id, conversation history, system prompt, tool schemas, generation parameters, stop sequences, and abort signal; treat the TypeScript type exported by `@deepseek-ai/dsh-llm` as authoritative. Map supported fields to the provider API. If the provider cannot honor a field, throw `LlmError` with a stable code instead of silently dropping it.
|
||||
|
||||
Override `resolveModelReasoning(provider, model)` when an exact model exposes selectable reasoning strengths. Return ordered opaque ids and display names plus an optional configured default; do not promote provider names into a core enum. The service validates the metadata and rejects unsupported explicit values before `stream()`. Returning `undefined` means that model has no selectable reasoning-effort capability.
|
||||
Override `resolveModelReasoning(provider, model, signal?)` when an exact model exposes selectable reasoning strengths. Return ordered opaque ids and display names plus an optional configured default; do not promote provider names into a core enum. Honor the optional signal for asynchronous lookup so cancellation and disposal reach quiescence. The service validates the metadata and rejects unsupported explicit values before `stream()`. Returning `undefined` means that model has no selectable reasoning-effort capability.
|
||||
|
||||
## Register an adapter
|
||||
|
||||
|
||||
@@ -112,7 +112,7 @@ async function* exampleChunks(): AsyncIterable<StreamChunk> {
|
||||
|
||||
`stream()` 接收仓库导出的 `GenerateOptions`。它包含模型名、由适配器持有的推理强度 ID、对话历史、系统提示词、tool schema、生成参数、停止序列和中止信号；完整字段以 `@deepseek-ai/dsh-llm` 导出的 TypeScript 类型为准。适配器必须将支持的字段映射到具体 API；无法支持的字段应抛出带稳定 code 的 `LlmError`，不能静默丢弃。
|
||||
|
||||
当某个具体模型提供可选推理强度时，请覆写 `resolveModelReasoning(provider, model)`。返回有序的不透明 ID、展示名称，以及可选的配置默认值；不要将提供方使用的等级名称提升为核心枚举。服务会校验元数据，并在调用 `stream()` 前拒绝显式指定但不受支持的值。返回 `undefined` 表示该模型没有可选的推理强度能力。
|
||||
当某个具体模型提供可选推理强度时，请覆写 `resolveModelReasoning(provider, model, signal?)`。返回有序的不透明 ID、展示名称，以及可选的配置默认值；不要将提供方使用的等级名称提升为核心枚举。异步查询必须响应这个可选信号，让取消和资源释放都能达到完全停稳。服务会校验元数据，并在调用 `stream()` 前拒绝显式指定但不受支持的值。返回 `undefined` 表示该模型没有可选的推理强度能力。
|
||||
|
||||
## 注册适配器
|
||||
|
||||
|
||||
@@ -385,16 +385,20 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
|
||||
jsDoc: '/**\n * Resolve context capacity from the adapter that owns one exact route.\n * This query is independent of the advisory model catalog: an unlisted model\n * may return metadata, while `undefined` never rejects later routing.\n * @param provider - registered provider route to inspect.\n * @param model - exact model id passed to the adapter.\n * @returns detached context metadata, or `undefined` when the adapter has none.\n */',
|
||||
},
|
||||
{
|
||||
signature: 'async resolveModelReasoning( provider: string, model: string, ): Promise<LlmModelReasoningInfo | undefined>',
|
||||
jsDoc: '/**\n * Resolve selectable reasoning efforts from the adapter that owns one exact\n * route. Metadata is validated and detached; an absent result means an\n * effort selector is unsupported for that model.\n * @param provider - registered provider route to inspect.\n * @param model - exact model id passed to the adapter.\n * @returns detached reasoning metadata, or `undefined` when unsupported.\n */',
|
||||
signature: 'async resolveModelReasoning( provider: string, model: string, signal?: AbortSignal, ): Promise<LlmModelReasoningInfo | undefined>',
|
||||
jsDoc: '/**\n * Resolve selectable reasoning efforts from the adapter that owns one exact\n * route. Metadata is validated and detached; an absent result means an\n * effort selector is unsupported for that model.\n * @param provider - registered provider route to inspect.\n * @param model - exact model id passed to the adapter.\n * @param signal - optional cancellation for adapter-owned asynchronous lookup.\n * @returns detached reasoning metadata, or `undefined` when unsupported.\n */',
|
||||
},
|
||||
{
|
||||
signature: 'async resolveCallConfig(config: LlmCallConfig): Promise<LlmCallConfig>',
|
||||
jsDoc: '/**\n * Validate a conversation call config against its exact model capability and\n * materialize an adapter-configured default. Unsupported explicit efforts\n * reject before provider I/O; no clamping or aliasing is performed.\n * @param config - provider/model route and optional request controls.\n * @returns a detached config only when a default must be materialized.\n */',
|
||||
signature: 'async resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>',
|
||||
jsDoc: '/**\n * Validate a conversation call config against its exact model capability and\n * materialize an adapter-configured default. Unsupported explicit efforts\n * reject before provider I/O; no clamping or aliasing is performed. This\n * standalone query does not bind a later dispatch; use {@link prepareCall}\n * when logging and streaming must share one adapter registration.\n * @param config - provider/model route and optional request controls.\n * @param signal - optional cancellation for adapter-owned capability lookup.\n * @returns a detached config only when a default must be materialized.\n */',
|
||||
},
|
||||
{
|
||||
signature: 'async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>',
|
||||
jsDoc: '/**\n * Resolve one call under its current adapter registration. The returned\n * one-shot handle keeps that registration across header logging and dispatch,\n * so HMR cannot combine one adapter\'s capability result with another adapter.\n * @param config - provider/model route and optional request controls.\n * @param signal - optional cancellation for adapter-owned capability lookup.\n * @returns a prepared config and its registration-bound stream entry point.\n */',
|
||||
},
|
||||
{
|
||||
signature: 'stream(options: GenerateOptions): AsyncIterable<StreamChunk>',
|
||||
jsDoc: '/**\n * Stream one model call as raw chunks (token-level deltas). Throws\n * `LlmError` with code `NO_ADAPTER` if no adapter is registered for\n * `options.provider`. Replay state is retained only when the same adapter\n * instance owns its historical provider and the target provider. Final\n * adapter selection, dispatch, and iteration failures retain their original\n * Error identity and are tagged in a call-local scope for narrow agent-loop\n * request recovery; middleware and nested-call failures remain untagged for\n * the outer call.\n * @param options - the full request; `options.provider` selects the adapter.\n * @returns the chunk stream, possibly wrapped by `llm/stream` listeners.\n */',
|
||||
jsDoc: '/**\n * Stream one model call as raw chunks (token-level deltas). Throws\n * `LlmError` with code `NO_ADAPTER` if no adapter is registered for\n * `options.provider`. Replay state is retained only when the same adapter\n * instance owns its historical provider and the target provider. Final\n * adapter selection remains fixed through asynchronous reasoning resolution\n * and dispatch. Selection, dispatch, and iteration failures retain their\n * original Error identity and are tagged in a call-local scope for narrow\n * agent-loop request recovery; middleware and nested-call failures remain\n * untagged for the outer call.\n * @param options - the full request; `options.provider` selects the adapter.\n * @returns the chunk stream, possibly wrapped by `llm/stream` listeners.\n */',
|
||||
},
|
||||
],
|
||||
},
|
||||
@@ -1702,7 +1706,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'LlmAdapter',
|
||||
declaration: 'export abstract class LlmAdapter {\n providerInfo(provider: string): LlmProviderInfo;\n listModels(_provider: string): Promise<readonly LlmModelInfo[]>;\n resolveModelContext(_provider: string, _model: string): Promise<LlmModelContext | undefined>;\n resolveModelReasoning(_provider: string, _model: string): Promise<LlmModelReasoningInfo | undefined>;\n abstract stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
|
||||
declaration: 'export abstract class LlmAdapter {\n providerInfo(provider: string): LlmProviderInfo;\n listModels(_provider: string): Promise<readonly LlmModelInfo[]>;\n resolveModelContext(_provider: string, _model: string): Promise<LlmModelContext | undefined>;\n resolveModelReasoning(_provider: string, _model: string, _signal?: AbortSignal): Promise<LlmModelReasoningInfo | undefined>;\n abstract stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
|
||||
},
|
||||
{
|
||||
name: 'LlmCallConfig',
|
||||
@@ -1756,6 +1760,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
name: 'OutOfBandSessionEventType',
|
||||
declaration: 'export type OutOfBandSessionEventType = Exclude<Extract<SessionEventType, keyof OutOfBandSessionEventMap>, SurfaceEventType>;',
|
||||
},
|
||||
{
|
||||
name: 'PreparedLlmCall',
|
||||
declaration: 'export interface PreparedLlmCall {\n readonly config: LlmCallConfig;\n stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
|
||||
},
|
||||
{
|
||||
name: 'PreparedReferencedMessage',
|
||||
declaration: 'export interface PreparedReferencedMessage {\n content: ContentBlock[];\n contexts: HookContext[];\n}',
|
||||
|
||||
@@ -60,7 +60,7 @@ The driver owns one agent for its lifetime and runs inside `ctx.agents.withIniti
|
||||
|
||||
Every provider call that reaches a successful finish appends exactly one `assistant/message` completion anchor, including content-less calls and `max-tokens` finishes. A successful `agent/step-result` stores its transformed content; a rejected result records empty content before the original failure continues. The anchor retains exact chunk provenance (`[]` for a stream with no chunks) and usage when available, while empty content stays out of derived message history.
|
||||
|
||||
After `agent/request` returns a provider/model call config, the loop asks `ctx.llm.resolveCallConfig()` to validate any adapter-owned reasoning effort and materialize its configured default. The effective config is logged in the full `request/header` before dispatch, so a listener can change effort between steps without hidden request drift. A route with no registered adapter preserves the proposed config so an `llm/stream` listener can own and short-circuit it; unhandled terminal dispatch still fails with `NO_ADAPTER`. A new loop instance restores the last effort only when its initial provider/model route exactly matches the logged route; a route change discards that opaque model-owned ID and resolves the new model independently.
|
||||
After `agent/request` returns a provider/model call config, the loop asks `ctx.llm.prepareCall()` to validate any adapter-owned reasoning effort and materialize its configured default under the active turn signal. The prepared call retains the exact adapter registration across this asynchronous resolution, `request/header` logging, and terminal dispatch, so HMR cannot mix one adapter's capability result with another adapter's request. The effective config is logged before dispatch, so a listener can change effort between steps without hidden request drift. A route with no registered adapter preserves the proposed config so an `llm/stream` listener can own and short-circuit it; unhandled terminal dispatch still fails with `NO_ADAPTER`. A new loop instance restores the last effort only when its initial provider/model route exactly matches the logged route; a route change discards that opaque model-owned ID and resolves the new model independently.
|
||||
|
||||
Plugin failure ends the current turn, not the loop. Only final adapter dispatch/iteration failures and terminal in-band error or aborted finishes enter `agent/request-error`; middleware, result processing, tools, and `agent/post-step` remain ordinary turn failures. Recovery receives the exact live error, immutable provider facts, and immutable prior failures after the failed step closes. A retry rebuilds from the durable log in a new numbered step, success clears the consecutive history, and exhaustion records the structured failure once on `turn/end`. AgentLoop privately owns one cancellation holder whose explicit signal spans prompt policy, assembly, every step, model and tool work, recovery, continuation, and terminal stop; it retires the holder immediately before publishing `turn/end`, while the driver may remain `running` through the durability flush. An effective `cancel()` emits the typed runtime-only `user | parent` cause before clearing pending work and cooperatively aborting the holder; notification failures cannot veto cancellation, work queued by a notification observer is cleared, work queued by a later abort observer belongs to the next turn, and idle cancellation emits nothing. Durable `turn/end` remains coarse `aborted`; undispatched model tool calls receive synthetic `tool/call` and `ABORTED_BEFORE_DISPATCH` result pairs. Disposal wins terminal classification, and work that ignores the signal must settle before quiescence. The [explicit-cancellation decision](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md) owns the lifecycle and race contract. Terminal continuation stops remain authoritative through turn close and durability flush.
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
|
||||
import { randomUUID } from 'node:crypto'
|
||||
import type { Context } from 'cordis'
|
||||
import type { ContentBlock, FinishReason, GenerateOptions, LlmCallConfig, LlmFailure, Message } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, FinishReason, GenerateOptions, LlmCallConfig, LlmFailure, Message, PreparedLlmCall } from '@deepseek-ai/dsh-llm'
|
||||
import { isDeepStrictEqual } from 'node:util'
|
||||
import { BlockAssembler, HarnessError, LlmError, assertNever, deepFreeze, errorChain, llmFailureOf, markAgentLoopRequest } from '@deepseek-ai/dsh-llm'
|
||||
import { agentEvents, agentInterruptReasonOf, assembleContextFor, AgentMessageId } from '@deepseek-ai/dsh-agent'
|
||||
@@ -689,8 +689,10 @@ async function runStep(
|
||||
throw new Error(`agent "${agent.id}" has no provider/model: set AgentOptions.provider and AgentOptions.model or supply both via the agent/request waterfall`)
|
||||
}
|
||||
let config: LlmCallConfig
|
||||
let preparedCall: PreparedLlmCall | undefined
|
||||
try {
|
||||
config = await ctx.llm.resolveCallConfig(proposedConfig)
|
||||
preparedCall = await ctx.llm.prepareCall(proposedConfig, signal)
|
||||
config = preparedCall.config
|
||||
} catch (error: unknown) {
|
||||
// A waterfall listener may own and short-circuit a route with no adapter.
|
||||
// Terminal dispatch still raises NO_ADAPTER when no listener handles it.
|
||||
@@ -731,7 +733,7 @@ async function runStep(
|
||||
// --- Model call (streaming-first; raw chunks are the replay record) ---
|
||||
const assembler = new BlockAssembler()
|
||||
const chunkSeqs: number[] = []
|
||||
const stream = ctx.llm.stream(request)
|
||||
const stream = preparedCall?.stream(request) ?? ctx.llm.stream(request)
|
||||
try {
|
||||
for await (const chunk of stream) {
|
||||
interruptionCheckpoint(signal)
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { LlmError, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmModelReasoningInfo } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { Session, SessionId, foldRequestHeader } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry, { defineContentToolFixture } from '@deepseek-ai/dsh-tools'
|
||||
@@ -152,6 +152,88 @@ describe('request stability across the loop', () => {
|
||||
expect(resumedHeaders.at(-1)?.data.reason).toBe('resume')
|
||||
})
|
||||
|
||||
it('keeps reasoning resolution, request logging, and dispatch on one adapter registration', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SystemPrompt, { persona: 'stable base' })
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(AgentRegistry)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
const started = Promise.withResolvers<undefined>()
|
||||
const reasoning = Promise.withResolvers<LlmModelReasoningInfo>()
|
||||
const first = new class extends MockAdapter {
|
||||
override resolveModelReasoning(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
_signal?: AbortSignal,
|
||||
): typeof reasoning.promise {
|
||||
started.resolve(undefined)
|
||||
return reasoning.promise
|
||||
}
|
||||
}([textResponse('first')])
|
||||
const second = new MockAdapter([textResponse('second')], {
|
||||
efforts: [{ id: ReasoningEffortId('max'), name: 'Max' }],
|
||||
defaultEffort: ReasoningEffortId('max'),
|
||||
})
|
||||
const disposeFirst = ctx.llm.registerAdapter(['mock'], first)
|
||||
const agent = ctx.agentLoop.create(SessionId('effort-hmr'), { provider: 'mock', model: 'mock' })
|
||||
|
||||
send(agent, 'go')
|
||||
await started.promise
|
||||
disposeFirst()
|
||||
ctx.llm.registerAdapter(['mock'], second)
|
||||
reasoning.resolve({
|
||||
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
})
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(first.requests.map(request => request.reasoningEffort)).toEqual([
|
||||
ReasoningEffortId('high'),
|
||||
])
|
||||
expect(second.requests).toHaveLength(0)
|
||||
const headers = agent.session.events.filter(event => event.type === 'request/header')
|
||||
expect(headers.at(-1)?.data.header.config.reasoningEffort).toBe(ReasoningEffortId('high'))
|
||||
})
|
||||
|
||||
it('aborts a blocked reasoning lookup before quiescent disposal completes', async () => {
|
||||
const started = Promise.withResolvers<AbortSignal>()
|
||||
const adapter = new class extends MockAdapter {
|
||||
override resolveModelReasoning(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<never> {
|
||||
if (signal === undefined) return Promise.reject(new Error('missing reasoning signal'))
|
||||
started.resolve(signal)
|
||||
return new Promise((_resolve, reject) => {
|
||||
if (signal.aborted) {
|
||||
reject(signal.reason instanceof Error ? signal.reason : new Error('reasoning aborted'))
|
||||
return
|
||||
}
|
||||
signal.addEventListener('abort', () => {
|
||||
reject(signal.reason instanceof Error ? signal.reason : new Error('reasoning aborted'))
|
||||
}, { once: true })
|
||||
})
|
||||
}
|
||||
}([])
|
||||
const ctx = await harness(adapter)
|
||||
const handle = await ctx.agents.create({
|
||||
sessionId: SessionId('reasoning-dispose'),
|
||||
agentOptions: { provider: 'mock', model: 'mock' },
|
||||
})
|
||||
|
||||
send(handle.agent, 'go')
|
||||
const signal = await started.promise
|
||||
await handle.dispose()
|
||||
|
||||
expect(signal.aborted).toBe(true)
|
||||
expect(handle.agent.status).toBe('disposed')
|
||||
expect(adapter.requests).toHaveLength(0)
|
||||
expect(handle.agent.session.events.some(event => event.type === 'request/header')).toBe(false)
|
||||
})
|
||||
|
||||
it.each(['plain error', 'LLM error'] as const)(
|
||||
'does not swallow a %s from reasoning resolution',
|
||||
async (kind) => {
|
||||
|
||||
@@ -147,6 +147,7 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
override resolveModelReasoning(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo | undefined> {
|
||||
if (this.options.defaults?.thinking === 'disabled') return Promise.resolve(undefined)
|
||||
return Promise.resolve({
|
||||
|
||||
@@ -142,6 +142,7 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
override resolveModelReasoning(
|
||||
provider: string,
|
||||
model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo | undefined> {
|
||||
const profile = this.profiles.get(provider)
|
||||
if (profile === undefined) {
|
||||
|
||||
@@ -12,8 +12,9 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
|
||||
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` Resolve authoritative context capacity for one exact route from its owning adapter.
|
||||
- `ctx.llm.resolveModelReasoning(provider: string, model: string): Promise<LlmModelReasoningInfo | undefined>` Resolve ordered adapter-owned reasoning efforts and an optional deployment default for one exact route.
|
||||
- `ctx.llm.resolveCallConfig(config: LlmCallConfig): Promise<LlmCallConfig>` Validate an explicit effort and materialize an adapter-configured default without clamping.
|
||||
- `ctx.llm.resolveModelReasoning(provider: string, model: string, signal?: AbortSignal): Promise<LlmModelReasoningInfo | undefined>` Resolve ordered adapter-owned reasoning efforts and an optional deployment default for one exact route, with optional cancellation for asynchronous adapters.
|
||||
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` Validate an explicit effort and materialize an adapter-configured default without clamping.
|
||||
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` Resolve a config and capture its current adapter registration as one cancellable, one-shot call.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
|
||||
@@ -22,7 +23,7 @@ Provider and model metadata is a discovery surface, not a routing whitelist. `re
|
||||
|
||||
Context capacity is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelContext()` asks the adapter that owns the exact provider/model route; an adapter can describe an unlisted dynamic model, and `undefined` means only that capacity is unavailable. Invalid returned capacity fails with `INVALID_MODEL_CONTEXT`.
|
||||
|
||||
Reasoning effort is also an exact-route capability, but its identifiers are opaque adapter-owned strings rather than a core enum. `resolveModelReasoning()` validates and detaches the ordered display metadata; `undefined` means the model has no selectable effort. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Invalid capability metadata fails with `INVALID_MODEL_REASONING`; an unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
|
||||
Reasoning effort is also an exact-route capability, but its identifiers are opaque adapter-owned strings rather than a core enum. `resolveModelReasoning()` validates and detaches the ordered display metadata; `undefined` means the model has no selectable effort. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. Invalid capability metadata fails with `INVALID_MODEL_REASONING`; an unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
|
||||
|
||||
### Events
|
||||
|
||||
@@ -32,7 +33,7 @@ Reasoning effort is also an exact-route capability, but its identifiers are opaq
|
||||
|
||||
### Extension points
|
||||
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, `resolveModelContext()` when exact capacity is known, and `resolveModelReasoning()` when a model exposes selectable efforts; the defaults use the route id as its name, advertise no models, and return neither capacity nor reasoning metadata.
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, `resolveModelContext()` when exact capacity is known, and `resolveModelReasoning()` when a model exposes selectable efforts; an asynchronous reasoning resolver must honor its optional cancellation signal. The defaults use the route id as its name, advertise no models, and return neither capacity nor reasoning metadata.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
|
||||
|
||||
### Content-block vocabulary (`types.ts`)
|
||||
@@ -43,7 +44,7 @@ Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta
|
||||
|
||||
### Call configuration (`call-config.ts`)
|
||||
|
||||
`LlmCallConfig` is the provider, model, optional adapter-owned reasoning effort, and sampling scalars of one conversation's requests (`provider`, `model`, `reasoningEffort`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement, `resolveCallConfig()` validates and defaults it, and the loop logs the effective value before dispatch. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
|
||||
`LlmCallConfig` is the provider, model, optional adapter-owned reasoning effort, and sampling scalars of one conversation's requests (`provider`, `model`, `reasoningEffort`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement, `prepareCall()` validates and defaults it under the turn signal, and the loop logs the effective value before using the prepared call's registration-bound stream. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
|
||||
|
||||
### App attribution (`attribution.ts`)
|
||||
|
||||
|
||||
@@ -105,6 +105,20 @@ export class LlmError extends HarnessError {
|
||||
}
|
||||
}
|
||||
|
||||
/** One model call whose config and adapter registration were resolved together. */
|
||||
export interface PreparedLlmCall {
|
||||
/** Detached, deep-frozen config with any adapter-owned default materialized. */
|
||||
readonly config: LlmCallConfig
|
||||
/**
|
||||
* Dispatch this call once through the registration captured during
|
||||
* preparation. The request's call-config fields must match {@link config};
|
||||
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
|
||||
* @param options - fully assembled request carrying the prepared config.
|
||||
* @returns the chunk stream, including the `llm/stream` waterfall.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
}
|
||||
|
||||
/**
|
||||
* Provider-wire adapter for the harness message and stream vocabulary. Register implementations
|
||||
* with `ctx.llm.registerAdapter(providers, adapter)`. Every provider HTTP request must include
|
||||
@@ -151,11 +165,14 @@ export abstract class LlmAdapter {
|
||||
* the model has no selectable reasoning-effort capability.
|
||||
* @param _provider - one provider route owned by this adapter.
|
||||
* @param _model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @param _signal - cancellation for this exact-model lookup; implementations
|
||||
* must settle promptly after it aborts.
|
||||
* @returns adapter-owned effort metadata, or `undefined` when unsupported.
|
||||
*/
|
||||
resolveModelReasoning(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo | undefined> {
|
||||
return Promise.resolve(undefined)
|
||||
}
|
||||
@@ -173,7 +190,7 @@ export abstract class LlmAdapter {
|
||||
* surface, interceptable via the `llm/stream` waterfall.
|
||||
*/
|
||||
export class LlmService extends Service {
|
||||
private adapters = new Map<string, { adapter: LlmAdapter; provider: LlmProviderInfo }>()
|
||||
private adapters = new Map<string, AdapterRegistration>()
|
||||
|
||||
constructor(ctx: Context) {
|
||||
super(ctx, 'llm')
|
||||
@@ -191,7 +208,7 @@ export class LlmService extends Service {
|
||||
const dispose = this.ctx.effect(function* (this: LlmService) {
|
||||
if (providers.length === 0) throw new LlmError('an adapter must register at least one provider', 'INVALID_ADAPTER')
|
||||
const unique = new Set<string>()
|
||||
const registrations: { adapter: LlmAdapter; provider: LlmProviderInfo }[] = []
|
||||
const registrations: AdapterRegistration[] = []
|
||||
for (const provider of providers) {
|
||||
if (provider.length === 0) throw new LlmError('adapter provider names must be non-empty', 'INVALID_ADAPTER')
|
||||
if (unique.has(provider) || this.adapters.has(provider)) {
|
||||
@@ -284,13 +301,24 @@ export class LlmService extends Service {
|
||||
* effort selector is unsupported for that model.
|
||||
* @param provider - registered provider route to inspect.
|
||||
* @param model - exact model id passed to the adapter.
|
||||
* @param signal - optional cancellation for adapter-owned asynchronous lookup.
|
||||
* @returns detached reasoning metadata, or `undefined` when unsupported.
|
||||
*/
|
||||
async resolveModelReasoning(
|
||||
provider: string,
|
||||
model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo | undefined> {
|
||||
const reasoning = await this.registration(provider).adapter.resolveModelReasoning(provider, model)
|
||||
return this.resolveModelReasoningFor(this.registration(provider), model, signal)
|
||||
}
|
||||
|
||||
private async resolveModelReasoningFor(
|
||||
registration: AdapterRegistration,
|
||||
model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo | undefined> {
|
||||
const provider = registration.provider.id
|
||||
const reasoning = await registration.adapter.resolveModelReasoning(provider, model, signal)
|
||||
if (reasoning === undefined) return undefined
|
||||
if (reasoning.efforts.length === 0) {
|
||||
throw new LlmError(
|
||||
@@ -335,12 +363,23 @@ export class LlmService extends Service {
|
||||
/**
|
||||
* Validate a conversation call config against its exact model capability and
|
||||
* materialize an adapter-configured default. Unsupported explicit efforts
|
||||
* reject before provider I/O; no clamping or aliasing is performed.
|
||||
* reject before provider I/O; no clamping or aliasing is performed. This
|
||||
* standalone query does not bind a later dispatch; use {@link prepareCall}
|
||||
* when logging and streaming must share one adapter registration.
|
||||
* @param config - provider/model route and optional request controls.
|
||||
* @param signal - optional cancellation for adapter-owned capability lookup.
|
||||
* @returns a detached config only when a default must be materialized.
|
||||
*/
|
||||
async resolveCallConfig(config: LlmCallConfig): Promise<LlmCallConfig> {
|
||||
const reasoning = await this.resolveModelReasoning(config.provider, config.model)
|
||||
async resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig> {
|
||||
return this.resolveCallConfigFor(this.registration(config.provider), config, signal)
|
||||
}
|
||||
|
||||
private async resolveCallConfigFor(
|
||||
registration: AdapterRegistration,
|
||||
config: LlmCallConfig,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmCallConfig> {
|
||||
const reasoning = await this.resolveModelReasoningFor(registration, config.model, signal)
|
||||
const requested = config.reasoningEffort
|
||||
if (reasoning === undefined) {
|
||||
if (requested !== undefined) {
|
||||
@@ -362,7 +401,33 @@ export class LlmService extends Service {
|
||||
return requested === effective ? config : { ...config, reasoningEffort: effective }
|
||||
}
|
||||
|
||||
private registration(provider: string): { adapter: LlmAdapter; provider: LlmProviderInfo } {
|
||||
/**
|
||||
* Resolve one call under its current adapter registration. The returned
|
||||
* one-shot handle keeps that registration across header logging and dispatch,
|
||||
* so HMR cannot combine one adapter's capability result with another adapter.
|
||||
* @param config - provider/model route and optional request controls.
|
||||
* @param signal - optional cancellation for adapter-owned capability lookup.
|
||||
* @returns a prepared config and its registration-bound stream entry point.
|
||||
*/
|
||||
async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall> {
|
||||
const registration = this.registration(config.provider)
|
||||
const resolvedConfig = deepFreeze(structuredClone(
|
||||
await this.resolveCallConfigFor(registration, config, signal),
|
||||
))
|
||||
let dispatched = false
|
||||
return Object.freeze({
|
||||
config: resolvedConfig,
|
||||
stream: (options: GenerateOptions): AsyncIterable<StreamChunk> => {
|
||||
if (dispatched) {
|
||||
throw new LlmError('a prepared LLM call can only be dispatched once', 'INVALID_PREPARED_CALL')
|
||||
}
|
||||
dispatched = true
|
||||
return this.streamWithRegistration(options, { registration, config: resolvedConfig })
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
private registration(provider: string): AdapterRegistration {
|
||||
const registration = this.adapters.get(provider)
|
||||
if (!registration) throw new LlmError(`no adapter registered for provider "${provider}"`, 'NO_ADAPTER')
|
||||
return registration
|
||||
@@ -395,16 +460,26 @@ export class LlmService extends Service {
|
||||
private async * adapterStream(
|
||||
options: GenerateOptions,
|
||||
failures: AdapterFailureScope,
|
||||
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
|
||||
): AsyncGenerator<StreamChunk> {
|
||||
let iterator: AsyncIterator<StreamChunk>
|
||||
try {
|
||||
const resolvedConfig = await this.resolveCallConfig(options)
|
||||
const resolvedOptions = callConfigEquals(options, resolvedConfig)
|
||||
const registration = prepared?.registration ?? this.registration(options.provider)
|
||||
const resolvedConfig = prepared === undefined
|
||||
? await this.resolveCallConfigFor(registration, options, options.signal)
|
||||
: prepared.config
|
||||
if (prepared !== undefined && !callConfigEquals(options, resolvedConfig)) {
|
||||
throw new LlmError(
|
||||
'prepared LLM call config changed before adapter dispatch',
|
||||
'INVALID_PREPARED_CALL',
|
||||
)
|
||||
}
|
||||
const resolvedOptions = prepared !== undefined || callConfigEquals(options, resolvedConfig)
|
||||
? options
|
||||
: Object.isFrozen(options)
|
||||
? deepFreeze({ ...options, ...resolvedConfig })
|
||||
: { ...options, ...resolvedConfig }
|
||||
const adapter = this.registration(resolvedOptions.provider).adapter
|
||||
const adapter = registration.adapter
|
||||
const stream = adapter.stream(this.forAdapter(resolvedOptions, adapter))
|
||||
iterator = stream[Symbol.asyncIterator]()
|
||||
} catch (error: unknown) {
|
||||
@@ -445,18 +520,36 @@ export class LlmService extends Service {
|
||||
* `LlmError` with code `NO_ADAPTER` if no adapter is registered for
|
||||
* `options.provider`. Replay state is retained only when the same adapter
|
||||
* instance owns its historical provider and the target provider. Final
|
||||
* adapter selection, dispatch, and iteration failures retain their original
|
||||
* Error identity and are tagged in a call-local scope for narrow agent-loop
|
||||
* request recovery; middleware and nested-call failures remain untagged for
|
||||
* the outer call.
|
||||
* adapter selection remains fixed through asynchronous reasoning resolution
|
||||
* and dispatch. Selection, dispatch, and iteration failures retain their
|
||||
* original Error identity and are tagged in a call-local scope for narrow
|
||||
* agent-loop request recovery; middleware and nested-call failures remain
|
||||
* untagged for the outer call.
|
||||
* @param options - the full request; `options.provider` selects the adapter.
|
||||
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
return this.streamWithRegistration(options)
|
||||
}
|
||||
|
||||
private streamWithRegistration(
|
||||
options: GenerateOptions,
|
||||
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
|
||||
): AsyncIterable<StreamChunk> {
|
||||
const failures: AdapterFailureScope = new WeakMap<Error, LlmFailure>()
|
||||
const stream = this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options, failures))
|
||||
const stream = this.ctx.waterfall(
|
||||
this,
|
||||
'llm/stream',
|
||||
options,
|
||||
() => this.adapterStream(options, failures, prepared),
|
||||
)
|
||||
return bindAdapterFailureScope(stream, failures)
|
||||
}
|
||||
}
|
||||
|
||||
interface AdapterRegistration {
|
||||
readonly adapter: LlmAdapter
|
||||
readonly provider: LlmProviderInfo
|
||||
}
|
||||
|
||||
export default LlmService
|
||||
|
||||
@@ -799,6 +799,115 @@ describe('LlmService', () => {
|
||||
expect(Object.isFrozen(adapter.lastOptions)).toBe(true)
|
||||
})
|
||||
|
||||
it('pins one adapter registration across asynchronous reasoning resolution and dispatch', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const started = Promise.withResolvers<undefined>()
|
||||
const reasoning = Promise.withResolvers<LlmModelReasoningInfo>()
|
||||
const first = new class extends RecordingAdapter {
|
||||
override resolveModelReasoning(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo> {
|
||||
started.resolve(undefined)
|
||||
return reasoning.promise
|
||||
}
|
||||
}(SCRIPT)
|
||||
const disposeFirst = ctx.llm.registerAdapter(['route'], first)
|
||||
const draining = (async () => {
|
||||
for await (const _chunk of ctx.llm.stream({
|
||||
provider: 'route',
|
||||
model: 'model',
|
||||
messages: [],
|
||||
})) { /* drain */ }
|
||||
})()
|
||||
|
||||
await started.promise
|
||||
disposeFirst()
|
||||
const second = new RecordingAdapter(SCRIPT)
|
||||
ctx.llm.registerAdapter(['route'], second)
|
||||
reasoning.resolve({
|
||||
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
})
|
||||
await draining
|
||||
|
||||
expect(first.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('high'))
|
||||
expect(second.lastOptions).toBeUndefined()
|
||||
})
|
||||
|
||||
it('prepares a one-shot registration-bound call and rejects config drift', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{
|
||||
model: {
|
||||
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
},
|
||||
},
|
||||
)
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
const prepared = await ctx.llm.prepareCall({ provider: 'route', model: 'model' })
|
||||
expect(Object.isFrozen(prepared.config)).toBe(true)
|
||||
const stream = prepared.stream({
|
||||
...prepared.config,
|
||||
model: 'other',
|
||||
messages: [],
|
||||
})
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toMatchObject({ code: 'INVALID_PREPARED_CALL' })
|
||||
expect(() => prepared.stream({
|
||||
...prepared.config,
|
||||
messages: [],
|
||||
})).toThrow(expect.objectContaining({ code: 'INVALID_PREPARED_CALL' }))
|
||||
})
|
||||
|
||||
it('passes cancellation through reasoning capability resolution', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const started = Promise.withResolvers<undefined>()
|
||||
const adapter = new class extends ScriptedAdapter {
|
||||
override resolveModelReasoning(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo | undefined> {
|
||||
started.resolve(undefined)
|
||||
return new Promise((_resolve, reject) => {
|
||||
if (signal === undefined) {
|
||||
reject(new Error('missing reasoning signal'))
|
||||
return
|
||||
}
|
||||
if (signal.aborted) {
|
||||
reject(signal.reason instanceof Error ? signal.reason : new Error('reasoning aborted'))
|
||||
return
|
||||
}
|
||||
signal.addEventListener('abort', () => {
|
||||
reject(signal.reason instanceof Error ? signal.reason : new Error('reasoning aborted'))
|
||||
}, { once: true })
|
||||
})
|
||||
}
|
||||
}(SCRIPT)
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
const controller = new AbortController()
|
||||
const resolving = ctx.llm.resolveCallConfig(
|
||||
{ provider: 'route', model: 'model' },
|
||||
controller.signal,
|
||||
)
|
||||
|
||||
await started.promise
|
||||
const reason = new Error('cancel reasoning')
|
||||
controller.abort(reason)
|
||||
await expect(resolving).rejects.toBe(reason)
|
||||
})
|
||||
|
||||
it.each([0, -1, 1.5, Number.NaN])(
|
||||
'rejects invalid adapter model context %s',
|
||||
async (contextWindow) => {
|
||||
|
||||
@@ -93,6 +93,7 @@ export const LINK_MAP: Record<string, string> = {
|
||||
CommandResult: 'commands.md',
|
||||
CommandSurface: 'commands.md',
|
||||
LlmAdapter: 'llm-streaming.md',
|
||||
PreparedLlmCall: 'llm-streaming.md',
|
||||
LlmService: 'llm-streaming.md',
|
||||
StreamChunk: 'llm-streaming.md',
|
||||
CreateSessionOptions: 'persistence.md',
|
||||
|
||||
@@ -307,6 +307,11 @@
|
||||
"source": "packages/llm/llm/src/assembler.ts",
|
||||
"projection": "public-api"
|
||||
},
|
||||
{
|
||||
"doc": "docs/core-data-structures/llm-streaming.md",
|
||||
"symbol": "PreparedLlmCall",
|
||||
"source": "packages/llm/llm/src/index.ts"
|
||||
},
|
||||
{
|
||||
"doc": "docs/core-data-structures/llm-streaming.md",
|
||||
"symbol": "LlmAdapter",
|
||||
@@ -1465,6 +1470,11 @@
|
||||
"source": "packages/llm/llm/src/assembler.ts",
|
||||
"projection": "public-api"
|
||||
},
|
||||
{
|
||||
"doc": "docs/core-data-structures/llm-streaming.zh.md",
|
||||
"symbol": "PreparedLlmCall",
|
||||
"source": "packages/llm/llm/src/index.ts"
|
||||
},
|
||||
{
|
||||
"doc": "docs/core-data-structures/llm-streaming.zh.md",
|
||||
"symbol": "LlmAdapter",
|
||||
|
||||
Reference in New Issue
Block a user