mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
refactor(llm): simplify live request telemetry (round 2)
This commit is contained in:
Hypatia May
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-host-owned-web-session-metrics.md
|
||||
2026-07-28-host-owned-web-session-metrics.md: e37a7635cbdae65162308bf8a3a498b5071a4910
|
||||
2026-07-28-host-owned-web-session-metrics.zh.md: fd3e50c8cf8c0336a8cb2cd55f6629419bb8ad23
|
||||
2026-07-28-host-owned-web-session-metrics.md: bc66ee72dc278196af8bab65c5e118d8f5cf039c
|
||||
2026-07-28-host-owned-web-session-metrics.zh.md: a103a8e24bd806ded6131b94f4c123c8d745513d
|
||||
|
||||
@@ -12,7 +12,7 @@ A Web stats line derived from the currently loaded conversation nodes is window-
|
||||
|
||||
The Host owns one session-level metrics projection. It incrementally folds the complete durable event log, keys settled usage by `(turn, step)`, and replaces an earlier usage record for the same key instead of double-counting chunk and message forms. Uncached input, output, cache reads, and cache writes remain four disjoint cumulative buckets. Compaction can change the current prompt surface without erasing historical usage.
|
||||
|
||||
Current context pressure is the point-in-time `tokenMeter.measure(session).totalTokens`. Capacity instead belongs to the latest model request observed by the current live mux connection. `LlmService.prepareCall()` retains the context metadata obtained by the exact lookup that also validates reasoning/defaults, and the loop publishes it through one contained `agent/model-request` notification only after the final route has a successfully constructed stream handle. Failed or aborted iteration still counts as a dispatched request; preparation and synchronous construction failures do not.
|
||||
Current context pressure is the point-in-time `tokenMeter.measure(session).totalTokens`. Capacity instead belongs to the latest model request attempt observed by the current live mux connection. `LlmService.prepareCall()` retains the context metadata obtained by the exact lookup that also validates reasoning/defaults. After the final provider/model is fixed and the outer `llm/stream` call returns a handle, the loop publishes one contained `agent/model-request` notification. This boundary observes an attempt, not proof of provider I/O: preparation or a synchronous outer waterfall failure emits nothing, while short-circuit handles and later lazy adapter construction, iteration failure, or abort still count.
|
||||
|
||||
The tail `session.history` response carries durable usage and pressure, while older pages omit them. Live changes use `session/metrics` mux frames. Both forms carry a durable-log revision and a projection revision; the client accepts only nondecreasing revisions and preserves metrics across older-page prepend.
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ Web 统计行若根据当前加载的会话节点推导指标，其结果会随
|
||||
|
||||
Host 拥有一项会话级指标投影。它以增量方式归并完整的持久事件日志，按 `(turn, step)` 标识已结算用量；同一标识再次出现时，会替换较早的用量记录，而不会重复统计分片和消息两种形态。未缓存输入、输出、缓存读取与缓存写入保持为四个彼此独立的累计计数项。压缩可以改变当前提示词表层，但不会抹除历史用量。
|
||||
|
||||
当前上下文压力是即时的 `tokenMeter.measure(session).totalTokens`。容量则属于当前实时 mux 连接观察到的最新模型请求。`LlmService.prepareCall()` 会保留同一次精确查询取得的上下文元数据，该查询也负责校验推理设置与默认值；仅在最终路由的流句柄成功构造后，循环才会通过一条失败会被收容的 `agent/model-request` 通知发布这些元数据。后续迭代失败或中止仍算作已分派请求；准备阶段失败和同步构造失败则不算。
|
||||
当前上下文压力是即时的 `tokenMeter.measure(session).totalTokens`。容量则属于当前实时 mux 连接观察到的最新模型请求尝试。`LlmService.prepareCall()` 会保留同一次精确查询取得的上下文元数据，该查询也负责校验推理设置与默认值。最终提供方／模型确定且外层 `llm/stream` 调用返回句柄后，循环会发布一条失败会被收容的 `agent/model-request` 通知。这个边界观察到的是一次尝试，并不能证明提供方 I/O 已开始：准备阶段或外层 waterfall（瀑布式事件）的同步失败不会发出通知，而短路句柄以及之后的惰性适配器构造、迭代失败或中止仍会计入。
|
||||
|
||||
`session.history` 尾页响应携带持久用量与压力，较早页面则省略这两项。实时变更使用 `session/metrics` mux 帧。两种形式都携带持久日志修订号和投影修订号；客户端只接受不减小的修订号，并在向前加载较早页面时保留指标。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/architecture.md
|
||||
architecture.md: 52072633a0e81afce63c5e162dd1b0af7f6486ca
|
||||
architecture.zh.md: f0122ece146c17366aa316cfb4ea4196a1db74bd
|
||||
architecture.md: e9ae1e27d6f1f1170b5da4823f3e988f4e778022
|
||||
architecture.zh.md: 3a31a2e6694d39f38e9c88de44f59ce7f753871f
|
||||
|
||||
@@ -92,7 +92,7 @@ forever:
|
||||
assemble system prompt and tool schemas
|
||||
snapshot the derived messages (the reconstruction boundary)
|
||||
'step/start'
|
||||
agent/request (config only) -> prepare reasoning/default + context under turn signal -> log request/header -> construct llm/stream (frozen, registration-bound) -> agent/model-request (live, contained) -> iterate
|
||||
agent/request (config only) -> prepare reasoning/default + context under turn signal -> log request/header -> obtain outer llm/stream handle (frozen request; prepared calls registration-bound) -> agent/model-request (live, contained attempt) -> iterate
|
||||
'assistant/chunk'
|
||||
'assistant/message'
|
||||
schedule tool calls by ctx.tools.executionMode:
|
||||
@@ -151,7 +151,7 @@ Log-only events may sit between turns. Owners append through `Session`, flushing
|
||||
|
||||
Messages use typed blocks from merge-extensible `ContentBlockMap`; the pattern also types `MessageSource`, `FinishReason`, `TurnTrigger`, and `TurnEndReason`. New blocks coordinate adapters, UI, compaction, token metering, and persistence; replay measurements live in [token-meter.md](core-data-structures/token-meter.md).
|
||||
|
||||
Streaming uses raw chunks and `BlockAssembler`. After final-stream construction, the loop emits contained, non-durable, non-replayed `agent/model-request` metadata. Adapters normalize failures; `agent/request-error` may retry. Remote adapters use per-read idle watchdogs. Replay crosses routes only through a shared adapter ([contract](core-data-structures/llm-streaming.md)).
|
||||
Streaming uses chunks and `BlockAssembler`. When the outer `llm/stream` returns a handle, AgentLoop emits contained, non-durable, non-replayed `agent/model-request` attempt metadata—not proof of provider I/O. `agent/request-error` may retry. Replay crosses routes only through a shared adapter ([contract](core-data-structures/llm-streaming.md)).
|
||||
|
||||
## Extension And Composition
|
||||
|
||||
|
||||
@@ -92,7 +92,7 @@ forever:
|
||||
assemble system prompt and tool schemas
|
||||
snapshot the derived messages (the reconstruction boundary)
|
||||
'step/start'
|
||||
agent/request (config only) -> prepare reasoning/default + context under turn signal -> log request/header -> construct llm/stream (frozen, registration-bound) -> agent/model-request (live, contained) -> iterate
|
||||
agent/request (config only) -> prepare reasoning/default + context under turn signal -> log request/header -> obtain outer llm/stream handle (frozen request; prepared calls registration-bound) -> agent/model-request (live, contained attempt) -> iterate
|
||||
'assistant/chunk'
|
||||
'assistant/message'
|
||||
schedule tool calls by ctx.tools.executionMode:
|
||||
@@ -151,7 +151,7 @@ idle inject:
|
||||
|
||||
消息使用从可合并扩展的 `ContentBlockMap` 派生的类型化块；同一模式也为 `MessageSource`、`FinishReason`、`TurnTrigger` 和 `TurnEndReason` 定义类型。新增块会协调适配器、UI、压缩、token 计量和持久化；回放计量见 [token-meter.md](core-data-structures/token-meter.md)。
|
||||
|
||||
流式输出使用原始分片和 `BlockAssembler`。最终流构造完成后，循环会发出 `agent/model-request` 元数据；该通知的失败会被收容，元数据不会持久化或回放。适配器会规范化故障；`agent/request-error` 可以重试。远程适配器使用逐次读取空闲看门狗。回放仅通过共用适配器跨路由传递（[契约](core-data-structures/llm-streaming.md)）。
|
||||
流式输出使用分片和 `BlockAssembler`。外层 `llm/stream` 返回句柄时，AgentLoop 会发出 `agent/model-request` 尝试元数据；该通知的失败会被收容，元数据不会持久化或回放，但这并不能证明提供方 I/O 已开始。`agent/request-error` 可以重试。回放仅通过共用适配器跨路由传递（[契约](core-data-structures/llm-streaming.md)）。
|
||||
|
||||
## 扩展与组合
|
||||
|
||||
|
||||
@@ -166,15 +166,15 @@ Source: [`packages/core/agent/src/types.ts:286`](../../packages/core/agent/src/t
|
||||
|
||||
### `agent/model-request` — emit
|
||||
|
||||
One model request constructed its final stream handle and is about to iterate it. This live notification is not durable or replayed; failed or aborted iteration still has a dispatch, while preparation and synchronous stream-construction failures do not. Listener failures are contained and cannot affect the request.
|
||||
One model request obtained its outer `llm/stream` handle and is about to iterate it. This observes an Agent-loop request attempt, not proof that provider I/O began. The notification is live, contained, and not replayed. Preparation or a synchronous outer waterfall failure emits nothing; failures or abortion after the handle returns still count.
|
||||
|
||||
```ts cordis-catalog
|
||||
/**
|
||||
* One model request constructed its final stream handle and is about to
|
||||
* iterate it. This live notification is not durable or replayed; failed or
|
||||
* aborted iteration still has a dispatch, while preparation and
|
||||
* synchronous stream-construction failures do not. Listener failures are
|
||||
* contained and cannot affect the request.
|
||||
* One model request obtained its outer `llm/stream` handle and is about to
|
||||
* iterate it. This observes an Agent-loop request attempt, not proof that
|
||||
* provider I/O began. The notification is live, contained, and not replayed.
|
||||
* Preparation or a synchronous outer waterfall failure emits nothing;
|
||||
* failures or abortion after the handle returns still count.
|
||||
* @param agent - the agent dispatching the model request.
|
||||
* @param turn - the open turn number.
|
||||
* @param step - the request's step number.
|
||||
|
||||
@@ -780,16 +780,14 @@ async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<Prepared
|
||||
* agent-loop request recovery; middleware and nested-call failures remain
|
||||
* untagged for the outer call.
|
||||
* @param options - the full request; `options.provider` selects the adapter.
|
||||
* @param onDispatched - contained Agent-loop notification hook invoked after
|
||||
* a stream handle is constructed and before its adapter is iterated.
|
||||
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
|
||||
*/
|
||||
stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
```
|
||||
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmCallConfig](../core-data-structures/core.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [LlmResolvedModelInfo](../core-data-structures/core.md) · [PreparedLlmCall](../core-data-structures/llm-streaming.md) · [ResolvedRetryPolicy](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/llm/llm/src/index.ts:194`](../../packages/llm/llm/src/index.ts)
|
||||
Source: [`packages/llm/llm/src/index.ts:192`](../../packages/llm/llm/src/index.ts)
|
||||
|
||||
## `ctx.permission` — `PermissionService`
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/core-data-structures/llm-streaming.md
|
||||
llm-streaming.md: 89628ebb96a2e8eec5209635cd92859427df4a8d
|
||||
llm-streaming.zh.md: 9c8fcc1f24b970f3a7cdd7cd08d9ef3b934b4543
|
||||
llm-streaming.md: 30230d208c582463b3680d760c27f33a71b4cf6f
|
||||
llm-streaming.zh.md: 32b3485f967b849371f916f6a669b97c97701cc1
|
||||
|
||||
@@ -161,7 +161,7 @@ declare class BlockAssembler {
|
||||
|
||||
## The seam
|
||||
|
||||
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerRetryPolicy()` is captured per route with normal defaults, while `providerInfo()` and asynchronous `listModels()` feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or capability, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmService.resolveModelInfo()` validates and detaches the aggregate. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch, and to retain detached context metadata from that exact lookup. Its optional observer runs after a final stream handle is constructed and before adapter iteration. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
|
||||
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerRetryPolicy()` is captured per route with normal defaults, while `providerInfo()` and asynchronous `listModels()` feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or capability, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmService.resolveModelInfo()` validates and detaches the aggregate. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch, and to retain detached context metadata from that exact lookup. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. AgentLoop observes a request attempt once the outer waterfall returns a stream handle; that limited boundary does not prove a lazy terminal adapter was constructed or began provider I/O. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
|
||||
|
||||
```ts type-equiv
|
||||
/** One model call whose config and adapter registration were resolved together. */
|
||||
@@ -175,11 +175,9 @@ interface PreparedLlmCall {
|
||||
* preparation. The request's call-config fields must match {@link config};
|
||||
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
|
||||
* @param options - fully assembled request carrying the prepared config.
|
||||
* @param onDispatched - contained Agent-loop notification hook invoked after
|
||||
* a stream handle is constructed and before its adapter is iterated.
|
||||
* @returns the chunk stream, including the `llm/stream` waterfall.
|
||||
*/
|
||||
stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
@@ -161,7 +161,7 @@ declare class BlockAssembler {
|
||||
|
||||
## seam
|
||||
|
||||
`LlmAdapter` 是提供方 seam：创建子类、实现 `stream()`，再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器；`GenerateOptions.model` 会传给该适配器，无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerRetryPolicy()` 会按路由捕获并填入 normal 默认值，`providerInfo()` 与异步 `listModels()` 方法则为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考，不是请求白名单：适配器仍是权威，并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份，以及可选的对正确性敏感的上下文容量、由模型持有的有序推理强度 ID 和部署默认值；字段缺失表示元数据或能力不可用，而不表示目录成员关系无效。解析器会接收可选的取消信号，并且必须在信号中止后迅速完成结算。`LlmService.resolveModelInfo()` 会校验聚合结果并返回分离值。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值，因此直接调用也无法绕过对不支持推理强度的拒绝；直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`，使模型解析、请求头持久记录和分派全程使用同一项注册，并保留来自同一次精确查询的分离上下文元数据。其可选观察器在最终流句柄构造完成后、适配器开始迭代前运行。适配器查找发生在 `llm/stream` waterfall（瀑布式事件）的终端 continuation，因此 listener 可以在查找前短路调用，或路由一个可变的一次性请求。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface（`ctx.llm.stream()`）与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
|
||||
`LlmAdapter` 是提供方 seam：创建子类、实现 `stream()`，再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器；`GenerateOptions.model` 会传给该适配器，无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerRetryPolicy()` 会按路由捕获并填入 normal 默认值，`providerInfo()` 与异步 `listModels()` 方法则为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考，不是请求白名单：适配器仍是权威，并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份，以及可选的对正确性敏感的上下文容量、由模型持有的有序推理强度 ID 和部署默认值；字段缺失表示元数据或能力不可用，而不表示目录成员关系无效。解析器会接收可选的取消信号，并且必须在信号中止后迅速完成结算。`LlmService.resolveModelInfo()` 会校验聚合结果并返回分离值。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值，因此直接调用也无法绕过对不支持推理强度的拒绝；直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`，使模型解析、请求头持久记录和分派全程使用同一项注册，并保留来自同一次精确查询的分离上下文元数据。适配器查找发生在 `llm/stream` waterfall（瀑布式事件）的终端 continuation，因此 listener 可以在查找前短路调用，或路由一个可变的一次性请求。AgentLoop 在外层 waterfall 返回流句柄时观察到一次请求尝试；这个有限边界不能证明惰性终端适配器已构造完成或开始提供方 I/O。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface（`ctx.llm.stream()`）与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
|
||||
|
||||
```ts type-equiv
|
||||
/** One model call whose config and adapter registration were resolved together. */
|
||||
@@ -175,11 +175,9 @@ interface PreparedLlmCall {
|
||||
* preparation. The request's call-config fields must match {@link config};
|
||||
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
|
||||
* @param options - fully assembled request carrying the prepared config.
|
||||
* @param onDispatched - contained Agent-loop notification hook invoked after
|
||||
* a stream handle is constructed and before its adapter is iterated.
|
||||
* @returns the chunk stream, including the `llm/stream` waterfall.
|
||||
*/
|
||||
stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ export type { TodoItem }
|
||||
* connection overlaid for presentation.
|
||||
*/
|
||||
export interface ConversationMetrics extends SessionMetrics {
|
||||
/** Latest dispatched-request capacity; absent until observed or after reset/clear. */
|
||||
/** Latest observed request-attempt capacity; absent until observed or after reset/clear. */
|
||||
contextWindow?: number
|
||||
}
|
||||
|
||||
|
||||
@@ -397,8 +397,8 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
|
||||
jsDoc: '/**\n * Resolve one call under its current adapter registration. The returned\n * one-shot handle keeps that registration across header logging and dispatch,\n * so HMR cannot combine one adapter\'s capability result with another adapter.\n * @param config - provider/model route and optional request controls.\n * @param signal - optional cancellation for adapter-owned capability lookup.\n * @returns a prepared config and its registration-bound stream entry point.\n */',
|
||||
},
|
||||
{
|
||||
signature: 'stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>',
|
||||
jsDoc: '/**\n * Stream one model call as raw chunks (token-level deltas). Throws\n * `LlmError` with code `NO_ADAPTER` if no adapter is registered for\n * `options.provider`. Replay state is retained only when the same adapter\n * instance owns its historical provider and the target provider. Final\n * adapter selection remains fixed through asynchronous exact-model resolution\n * and dispatch. Selection, dispatch, and iteration failures retain their\n * original Error identity and are tagged in a call-local scope for narrow\n * agent-loop request recovery; middleware and nested-call failures remain\n * untagged for the outer call.\n * @param options - the full request; `options.provider` selects the adapter.\n * @param onDispatched - contained Agent-loop notification hook invoked after\n * a stream handle is constructed and before its adapter is iterated.\n * @returns the chunk stream, possibly wrapped by `llm/stream` listeners.\n */',
|
||||
signature: 'stream(options: GenerateOptions): AsyncIterable<StreamChunk>',
|
||||
jsDoc: '/**\n * Stream one model call as raw chunks (token-level deltas). Throws\n * `LlmError` with code `NO_ADAPTER` if no adapter is registered for\n * `options.provider`. Replay state is retained only when the same adapter\n * instance owns its historical provider and the target provider. Final\n * adapter selection remains fixed through asynchronous exact-model resolution\n * and dispatch. Selection, dispatch, and iteration failures retain their\n * original Error identity and are tagged in a call-local scope for narrow\n * agent-loop request recovery; middleware and nested-call failures remain\n * untagged for the outer call.\n * @param options - the full request; `options.provider` selects the adapter.\n * @returns the chunk stream, possibly wrapped by `llm/stream` listeners.\n */',
|
||||
},
|
||||
],
|
||||
},
|
||||
@@ -1052,8 +1052,8 @@ export const EVENT_API: readonly EventApiEntry[] = [
|
||||
name: 'agent/model-request',
|
||||
mode: 'emit',
|
||||
signature: '\'agent/model-request\'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, request: AgentModelRequest): void',
|
||||
jsDoc: '/**\n * One model request constructed its final stream handle and is about to\n * iterate it. This live notification is not durable or replayed; failed or\n * aborted iteration still has a dispatch, while preparation and\n * synchronous stream-construction failures do not. Listener failures are\n * contained and cannot affect the request.\n * @param agent - the agent dispatching the model request.\n * @param turn - the open turn number.\n * @param step - the request\'s step number.\n * @param request - final route plus registration-bound context capacity.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode emit\n */',
|
||||
summary: 'One model request constructed its final stream handle and is about to iterate it.',
|
||||
jsDoc: '/**\n * One model request obtained its outer `llm/stream` handle and is about to\n * iterate it. This observes an Agent-loop request attempt, not proof that\n * provider I/O began. The notification is live, contained, and not replayed.\n * Preparation or a synchronous outer waterfall failure emits nothing;\n * failures or abortion after the handle returns still count.\n * @param agent - the agent dispatching the model request.\n * @param turn - the open turn number.\n * @param step - the request\'s step number.\n * @param request - final route plus registration-bound context capacity.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode emit\n */',
|
||||
summary: 'One model request obtained its outer `llm/stream` handle and is about to iterate it.',
|
||||
},
|
||||
{
|
||||
name: 'agent/prompt-submit',
|
||||
@@ -1810,7 +1810,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'PreparedLlmCall',
|
||||
declaration: 'export interface PreparedLlmCall {\n readonly config: LlmCallConfig;\n readonly context?: LlmModelContext;\n stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>;\n}',
|
||||
declaration: 'export interface PreparedLlmCall {\n readonly config: LlmCallConfig;\n readonly context?: LlmModelContext;\n stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
|
||||
},
|
||||
{
|
||||
name: 'PreparedReferencedMessage',
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/core/agent-loop/README.md
|
||||
README.md: 39eaafd3abb2faef045c1a2f694d8dffe95c28b0
|
||||
README.zh.md: f8cc972e957fe95f652d54e859f8e5411288db51
|
||||
README.md: 6e23d9f543998d5c0266c73197ee52ede51260aa
|
||||
README.zh.md: cf561a03281f80b7afd5f245752e9ebf70333a8b
|
||||
|
||||
@@ -62,7 +62,7 @@ The driver owns one agent for its lifetime and runs inside `ctx.agents.withIniti
|
||||
|
||||
Every provider call that reaches a successful finish appends exactly one `assistant/message` completion anchor, including content-less calls and `max-tokens` finishes. The anchor records the assembled content as-is, retains exact chunk provenance (`[]` for a stream with no chunks), and includes usage when available; empty content stays out of derived message history.
|
||||
|
||||
After `agent/request` returns a provider/model call config, the loop asks `ctx.llm.prepareCall()` to validate any adapter-owned reasoning effort, materialize its configured default, and retain available context metadata from that same exact-model lookup under the active turn signal. The prepared call retains the exact adapter registration across this asynchronous resolution, `request/header` logging, and terminal dispatch, so HMR cannot mix one adapter's capability result with another adapter's request. After the final stream handle is constructed and before adapter iteration, the loop emits one contained live `agent/model-request` notification with turn, step, final provider/model, and optional registration-bound capacity. Preparation or synchronous stream-construction failures emit nothing; later failure or abortion remains an observed dispatch. The effective config is logged before dispatch, so a listener can change effort between steps without hidden request drift. A route with no registered adapter preserves the proposed config so an `llm/stream` listener can own and short-circuit it; unhandled terminal dispatch still fails with `NO_ADAPTER`. A new loop instance restores the last effort only when its initial provider/model route exactly matches the logged route; a route change discards that opaque model-owned ID and resolves the new model independently.
|
||||
After `agent/request` returns a provider/model call config, the loop asks `ctx.llm.prepareCall()` to validate any adapter-owned reasoning effort, materialize its configured default, and retain available context metadata from that same exact-model lookup under the active turn signal. The prepared call retains the exact adapter registration across this asynchronous resolution, `request/header` logging, and terminal dispatch, so HMR cannot mix one adapter's capability result with another adapter's request. Once the final provider/model is fixed and the outer `llm/stream` call returns a handle, the loop emits one contained live `agent/model-request` notification with turn, step, route, and optional registration-bound capacity. This is an observed Agent-loop attempt, not proof of provider I/O: preparation or a synchronous outer waterfall failure emits nothing, while a short-circuit handle or later lazy adapter construction, failure, or abortion still counts. The effective config is logged before dispatch, so a listener can change effort between steps without hidden request drift. A route with no registered adapter preserves the proposed config so an `llm/stream` listener can own and short-circuit it; unhandled terminal dispatch still fails with `NO_ADAPTER`. A new loop instance restores the last effort only when its initial provider/model route exactly matches the logged route; a route change discards that opaque model-owned ID and resolves the new model independently.
|
||||
|
||||
Plugin failure ends the current turn, not the loop. Only final adapter dispatch/iteration failures and terminal in-band error or aborted finishes enter `agent/request-error`; middleware, result processing, tools, and other extension failures close directly. Recovery receives the exact live error, immutable provider facts, immutable prior failures, the immutable retry policy of the adapter registration that served the request, and the turn signal after the failed step closes; the policy is absent if no final adapter served it. A handling listener returns `{ kind: 'retry' }`; the loop closes the failed turn with its error and opens one numbered retry turn without an intervening idle notification. Success clears the consecutive history, and an unhandled failure is terminal. AgentLoop owns one cancellation signal for the current admission or turn. An effective `cancel(cause)` clears pending work unless `keepInbox` is set and cooperatively aborts that signal; idle cancellation is a no-op. Durable `turn/end` records `aborted` for `user` and `parent`, while disposal records `disposed`; undispatched model tool calls receive synthetic `tool/call` and `ABORTED_BEFORE_DISPATCH` result pairs. The cancellation cause changes reporting, not how result context finalized after cancellation is handled. Disposal waits for signal-ignoring work before registry removal. The [explicit-cancellation decision](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md) owns the lifecycle and race contract.
|
||||
|
||||
|
||||
@@ -62,7 +62,7 @@ interface Config {
|
||||
|
||||
每次提供方调用成功结束时，都会恰好追加一个 `assistant/message` 完成锚点，包括无内容调用和以 `max-tokens` 结束的调用。该锚点原样记录组装后的内容，保留确切的 chunk 溯源（流没有 chunk 时为 `[]`），并在用量可用时包含用量；空内容不会进入派生消息历史。
|
||||
|
||||
在 `agent/request` 返回提供方／模型调用配置后，循环会调用 `ctx.llm.prepareCall()`，在活跃轮次信号的控制下校验由适配器持有的推理（reasoning）强度、填入其配置默认值，并从同一次精确模型查询中保留可用的上下文元数据。准备完成的调用会在这次异步解析、`request/header` 日志记录和最终分派期间保留同一项确切的适配器注册，因此 HMR（热模块替换）不会把某个适配器的能力解析结果与另一适配器的请求混用。最终流句柄构造完成后、适配器开始迭代前，循环会发出一条失败会被收容的实时 `agent/model-request` 通知，其中包含轮次、步骤、最终提供方／模型，以及可选的、与注册项绑定的容量。准备阶段失败或同步流构造失败不会发出通知；之后即使失败或中止，该请求仍视为已观察到的分派。生效配置会在分派前写入日志，因此监听器可以在步骤之间更改推理强度，而不会产生未记录的请求变化。没有已注册适配器的路由会保留原定配置，使 `llm/stream` 监听器可以接管并短路该请求；最终分派仍会以 `NO_ADAPTER` 拒绝未得到处理的路由。新循环实例仅在初始提供方／模型路由与日志路由完全一致时恢复上次的推理强度；路由变化会丢弃由前一模型持有的不透明 ID，并单独解析新模型。
|
||||
在 `agent/request` 返回提供方／模型调用配置后，循环会调用 `ctx.llm.prepareCall()`，在活跃轮次信号的控制下校验由适配器持有的推理（reasoning）强度、填入其配置默认值，并从同一次精确模型查询中保留可用的上下文元数据。准备完成的调用会在这次异步解析、`request/header` 日志记录和最终分派期间保留同一项确切的适配器注册，因此 HMR（热模块替换）不会把某个适配器的能力解析结果与另一适配器的请求混用。最终提供方／模型确定且外层 `llm/stream` 调用返回句柄后，循环会发出一条失败会被收容的实时 `agent/model-request` 通知，其中包含轮次、步骤、路由，以及可选的、与注册项绑定的容量。这是 agent loop 观察到的一次尝试，并不能证明提供方 I/O 已开始：准备阶段或外层 waterfall（瀑布式事件）的同步失败不会发出通知，而短路句柄或之后的惰性适配器构造、失败或中止仍会计入。生效配置会在分派前写入日志，因此监听器可以在步骤之间更改推理强度，而不会产生未记录的请求变化。没有已注册适配器的路由会保留原定配置，使 `llm/stream` 监听器可以接管并短路该请求；最终分派仍会以 `NO_ADAPTER` 拒绝未得到处理的路由。新循环实例仅在初始提供方／模型路由与日志路由完全一致时恢复上次的推理强度；路由变化会丢弃由前一模型持有的不透明 ID，并单独解析新模型。
|
||||
|
||||
插件失败会结束当前轮次，而不是结束循环。只有最终适配器分发／迭代失败以及带内的终止错误或中止结束才进入 `agent/request-error`；中间件、结果处理、工具及其他扩展失败会直接关闭轮次。失败步骤关闭后，恢复逻辑会接收确切的实时错误、不可变的提供方事实、不可变的先前失败、为请求提供服务的适配器注册所对应的不可变重试策略，以及轮次信号；如果没有最终适配器为其提供服务，则该策略缺失。处理失败的监听器返回 `{ kind: 'retry' }`；循环用其错误关闭失败轮次，并在不插入空闲通知的情况下开启一个编号重试轮次。成功会清除连续失败历史；未被处理的失败是终态。AgentLoop 为当前接纳或轮次拥有一个取消信号。有效的 `cancel(cause)` 在未设置 `keepInbox` 时清除待处理工作，并以协作方式中止该信号；空闲取消是空操作。持久 `turn/end` 为 `user` 和 `parent` 记录 `aborted`，dispose（资源释放）则记录 `disposed`；未分发的模型工具调用会收到合成的 `tool/call` 与 `ABORTED_BEFORE_DISPATCH` 结果对。取消原因只改变报告方式，不改变对取消后已定案结果上下文的处理。dispose 会等待忽略信号的工作完成，然后才从注册表移除。[显式取消决策](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md)规定生命周期与竞态契约。
|
||||
|
||||
@@ -89,7 +89,7 @@ interface Config {
|
||||
|
||||
#### Token 影响
|
||||
|
||||
每个步骤都会再次计入系统文本与 schema。逐 agent 作用域决定贡献，而权威组装 waterfall（瀑布式事件）可以改变最终请求，并使其监听器负责保持协议连贯。
|
||||
每个步骤都会再次计入系统文本与 schema。逐 agent 作用域决定贡献，而权威组装 waterfall 可以改变最终请求，并使其监听器负责保持协议连贯。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
|
||||
@@ -487,24 +487,21 @@ export class ReactLoopAgent implements Agent {
|
||||
|
||||
const assembler = new BlockAssembler()
|
||||
const chunkSeqs: number[] = []
|
||||
const onDispatched = (): void => {
|
||||
emitAgentEvent(
|
||||
this.loopCtx,
|
||||
this,
|
||||
'agent/model-request',
|
||||
turn,
|
||||
step,
|
||||
{
|
||||
provider: request.provider,
|
||||
model: request.model,
|
||||
...preparedCall?.context === undefined
|
||||
? {}
|
||||
: { contextWindow: preparedCall.context.contextWindow },
|
||||
},
|
||||
)
|
||||
}
|
||||
const stream = preparedCall?.stream(request, onDispatched)
|
||||
?? this.loopCtx.llm.stream(request, onDispatched)
|
||||
const stream = preparedCall?.stream(request) ?? this.loopCtx.llm.stream(request)
|
||||
emitAgentEvent(
|
||||
this.loopCtx,
|
||||
this,
|
||||
'agent/model-request',
|
||||
turn,
|
||||
step,
|
||||
{
|
||||
provider: request.provider,
|
||||
model: request.model,
|
||||
...preparedCall?.context === undefined
|
||||
? {}
|
||||
: { contextWindow: preparedCall.context.contextWindow },
|
||||
},
|
||||
)
|
||||
try {
|
||||
for await (const chunk of stream) {
|
||||
signal.throwIfAborted()
|
||||
|
||||
@@ -293,6 +293,7 @@ describe('request stability across the loop', () => {
|
||||
await ctx.plugin(AgentRegistry)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
let observed: GenerateOptions | undefined
|
||||
let observedRequest: { provider: string; model: string; contextWindow?: number } | undefined
|
||||
ctx.on('llm/stream', (options) => {
|
||||
observed = options
|
||||
return (async function* () {
|
||||
@@ -303,11 +304,15 @@ describe('request stability across the loop', () => {
|
||||
provider: 'listener',
|
||||
model: 'virtual',
|
||||
})
|
||||
ctx.on('agent/model-request', (subject, _turn, _step, request) => {
|
||||
if (subject === agent) observedRequest = { ...request }
|
||||
})
|
||||
|
||||
send(agent, 'go')
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(observed).toMatchObject({ provider: 'listener', model: 'virtual' })
|
||||
expect(observedRequest).toEqual({ provider: 'listener', model: 'virtual' })
|
||||
expect(agent.session.requestHeader()?.config).toEqual({
|
||||
provider: 'listener',
|
||||
model: 'virtual',
|
||||
@@ -318,7 +323,7 @@ describe('request stability across the loop', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('notifies one contained live model-request edge only after successful stream construction', async () => {
|
||||
it('notifies one contained request attempt after the outer stream handle returns', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(SessionStore)
|
||||
@@ -341,7 +346,9 @@ describe('request stability across the loop', () => {
|
||||
}
|
||||
|
||||
override stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
if (options.model === 'sync-failure') throw new LlmError('construction failed', 'CONSTRUCTION')
|
||||
if (options.model === 'lazy-sync-failure') {
|
||||
throw new LlmError('lazy construction failed', 'CONSTRUCTION')
|
||||
}
|
||||
if (options.model === 'async-failure') {
|
||||
return {
|
||||
[Symbol.asyncIterator]: () => ({
|
||||
@@ -359,6 +366,22 @@ describe('request stability across the loop', () => {
|
||||
provider: 'mock',
|
||||
model: 'capacity',
|
||||
})
|
||||
const returnedHandles = new Set<string>()
|
||||
ctx.on('llm/stream', (options, next) => {
|
||||
if (options.model === 'outer-failure') {
|
||||
throw new Error('outer waterfall failed before returning a handle')
|
||||
}
|
||||
if (options.model === 'lazy-sync-failure') {
|
||||
const stream = (async function* () {
|
||||
yield* next()
|
||||
})()
|
||||
returnedHandles.add(options.model)
|
||||
return stream
|
||||
}
|
||||
const stream = next()
|
||||
returnedHandles.add(options.model)
|
||||
return stream
|
||||
})
|
||||
const observed: {
|
||||
turn: number
|
||||
step: number
|
||||
@@ -366,18 +389,27 @@ describe('request stability across the loop', () => {
|
||||
model: string
|
||||
contextWindow?: number
|
||||
}[] = []
|
||||
const observedBeforeHandleReturn: string[] = []
|
||||
ctx.on('agent/model-request', (subject) => {
|
||||
if (subject === agent) throw new Error('observer failed')
|
||||
})
|
||||
ctx.on('agent/model-request', (subject, turn, step, request) => {
|
||||
if (subject === agent) observed.push({ turn, step, ...request })
|
||||
if (subject !== agent) return
|
||||
if (!returnedHandles.has(request.model)) observedBeforeHandleReturn.push(request.model)
|
||||
observed.push({ turn, step, ...request })
|
||||
})
|
||||
ctx.on('agent/request', async (_subject, turn, _step, _signal, next) => ({
|
||||
...await next(),
|
||||
model: ['capacity', 'unknown', 'async-failure', 'sync-failure'][turn - 1]!,
|
||||
model: [
|
||||
'capacity',
|
||||
'unknown',
|
||||
'async-failure',
|
||||
'lazy-sync-failure',
|
||||
'outer-failure',
|
||||
][turn - 1]!,
|
||||
}))
|
||||
|
||||
for (const prompt of ['one', 'two', 'three', 'four']) {
|
||||
for (const prompt of ['one', 'two', 'three', 'four', 'five']) {
|
||||
send(agent, prompt)
|
||||
await waitForIdle(ctx, agent)
|
||||
}
|
||||
@@ -402,8 +434,16 @@ describe('request stability across the loop', () => {
|
||||
provider: 'mock',
|
||||
model: 'async-failure',
|
||||
},
|
||||
{
|
||||
turn: 4,
|
||||
step: 1,
|
||||
provider: 'mock',
|
||||
model: 'lazy-sync-failure',
|
||||
},
|
||||
])
|
||||
expect(resolutions).toBe(4)
|
||||
expect(resolutions).toBe(5)
|
||||
expect(observedBeforeHandleReturn).toEqual([])
|
||||
expect(returnedHandles.has('outer-failure')).toBe(false)
|
||||
})
|
||||
|
||||
it('a compaction replace rewrites the resend, and the log explains it', async () => {
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/core/agent/README.md
|
||||
README.md: 304347d32df4546389ee2b45230d4e80acc60942
|
||||
README.zh.md: d9adbe20015aadbad26288d43df92f80a3f550bf
|
||||
README.md: ad79e101974eb187df3093092a282fc6e5850857
|
||||
README.zh.md: 84849df23dfdc668f0d18c05ed6d9fd4d8c315d6
|
||||
|
||||
@@ -48,7 +48,7 @@ Agent *creation* is provided by the plugin implementing `AgentFactory` (`dsh-age
|
||||
|
||||
The lifecycle edges have two important local caveats. `agent/created` runs after scoped setup and after both session and agent registry entries exist. Setup is trusted composition-only code; the immediately following non-vetoing `agent/session-start` notification is the first supported startup injection point. `agent/disposed` always means the exact agent has left the registry. AgentLoop emits it after its driver is quiescent, while ordered teardown may still be detaching the session and unwinding the scope; custom agents registered directly own any stronger driver-ordering contract themselves.
|
||||
|
||||
Most interception points are cooperative waterfalls. Turn-scoped asynchronous seams receive one explicit `AbortSignal`, with `signal` immediately before a waterfall's final `next`; listeners may cooperate but must not retain it as authority over another turn. `agent/step` is the serial checkpoint before request derivation, while the contained `agent/model-request` notification reports a final dispatched route and optional registration-bound context capacity without becoming durable state. `agent/request-error` is the failed-model-request recovery waterfall: it receives the exact error, normalized failure facts, and signal after the failed step closes. A listener returns `{ kind: 'retry' }` without calling `next()` when it owns recovery; the loop closes the failed turn and opens one numbered retry turn. `agent/turn-stopping` runs before an otherwise completed turn closes. Ordinary queued prompts remain intact. Effective broad cancellation first emits the observe-only `agent/cancel-requested` with its resolved typed cause, then clears queues and aborts; notification failures are contained and cannot veto the stop. The [explicit-cancellation decision](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md) owns signal lifetime; the [agent-scope runtime-design Agent Note](../../../.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.md#three-execution-boundaries-are-deliberately-one-way) owns scoped dispatch and terminal settlement.
|
||||
Most interception points are cooperative waterfalls. Turn-scoped asynchronous seams receive one explicit `AbortSignal`, with `signal` immediately before a waterfall's final `next`; listeners may cooperate but must not retain it as authority over another turn. `agent/step` is the serial checkpoint before request derivation, while the contained `agent/model-request` notification reports the route and optional registration-bound context capacity for an attempt whose outer stream handle returned. It is neither durable state nor proof of provider I/O. `agent/request-error` is the failed-model-request recovery waterfall: it receives the exact error, normalized failure facts, and signal after the failed step closes. A listener returns `{ kind: 'retry' }` without calling `next()` when it owns recovery; the loop closes the failed turn and opens one numbered retry turn. `agent/turn-stopping` runs before an otherwise completed turn closes. Ordinary queued prompts remain intact. Effective broad cancellation first emits the observe-only `agent/cancel-requested` with its resolved typed cause, then clears queues and aborts; notification failures are contained and cannot veto the stop. The [explicit-cancellation decision](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md) owns signal lifetime; the [agent-scope runtime-design Agent Note](../../../.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.md#three-execution-boundaries-are-deliberately-one-way) owns scoped dispatch and terminal settlement.
|
||||
|
||||
`PromptDecision.additionalContexts` is an array so every context keeps its own source. Allowed prompt content and every additional context become separate model-facing `user/message` events before the turn runs. A listener that wraps a downstream allow preserves its `content` and `additionalContexts` unless it intentionally replaces either field; the returned allow is authoritative.
|
||||
|
||||
|
||||
@@ -48,7 +48,7 @@ Agent *创建* 由实现 `AgentFactory` 的插件（`dsh-agent-loop`）提供，
|
||||
|
||||
生命周期边有两个重要的本地注意事项。`agent/created` 在作用域 setup 之后、会话与 agent 注册表条目都存在之后运行。Setup 是受信任、仅用于组合的代码；紧随其后且不可 veto 的 `agent/session-start` 通知是第一个受支持的启动注入点。`agent/disposed` 始终表示确切 agent 已离开注册表。AgentLoop 在其驱动器静默后发出该事件，而有序 teardown 此时可能仍在分离会话并撤销作用域；直接注册的自定义 agent 自行拥有任何更强的驱动器顺序契约。
|
||||
|
||||
大多数拦截点都是协作式 waterfall。轮次作用域的异步 seam 接收一个显式 `AbortSignal`，其中 `signal` 紧邻 waterfall 最终的 `next`；监听器可以配合，但不得将它保留为控制另一轮次的权限。`agent/step` 是派生请求前的串行检查点；`agent/model-request` 是失败会被收容的通知，它会报告最终已分派路由及可选的、与注册项绑定的上下文容量，但不会成为持久状态。`agent/request-error` 是失败模型请求的恢复 waterfall：失败步骤关闭后，它接收确切错误、规范化失败事实和信号。拥有恢复权的监听器返回 `{ kind: 'retry' }` 且不调用 `next()`；循环会关闭失败轮次，并打开一个编号重试轮次。`agent/turn-stopping` 在本可完成的轮次关闭前运行。普通排队提示词保持原样。有效的广义取消会先发出只观测的 `agent/cancel-requested` 及其解析后的类型化原因，再清空队列并中止；通知失败会被收容，不能 veto 停止。信号生命周期由[显式取消决策](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md)拥有；作用域分发与终止结算由 [agent 作用域 runtime 设计 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.md#three-execution-boundaries-are-deliberately-one-way)拥有。
|
||||
大多数拦截点都是协作式 waterfall。轮次作用域的异步 seam 接收一个显式 `AbortSignal`，其中 `signal` 紧邻 waterfall 最终的 `next`；监听器可以配合，但不得将它保留为控制另一轮次的权限。`agent/step` 是派生请求前的串行检查点；`agent/model-request` 是失败会被收容的通知，它会报告外层流句柄已返回的尝试所用路由，以及可选的、与注册项绑定的上下文容量。它既不是持久状态，也不能证明提供方 I/O 已开始。`agent/request-error` 是失败模型请求的恢复 waterfall：失败步骤关闭后，它接收确切错误、规范化失败事实和信号。拥有恢复权的监听器返回 `{ kind: 'retry' }` 且不调用 `next()`；循环会关闭失败轮次，并打开一个编号重试轮次。`agent/turn-stopping` 在本可完成的轮次关闭前运行。普通排队提示词保持原样。有效的广义取消会先发出只观测的 `agent/cancel-requested` 及其解析后的类型化原因，再清空队列并中止；通知失败会被收容，不能 veto 停止。信号生命周期由[显式取消决策](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md)拥有；作用域分发与终止结算由 [agent 作用域 runtime 设计 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.md#three-execution-boundaries-are-deliberately-one-way)拥有。
|
||||
|
||||
`PromptDecision.additionalContexts` 是数组，因此每个上下文都保留自己的来源。获准的提示词内容与每个附加上下文都会在轮次运行前成为各自独立、面向模型的 `user/message` 事件。包装下游允许决策的监听器会保留其 `content` 与 `additionalContexts`，除非有意替换任一字段；返回的允许决策是权威来源。
|
||||
|
||||
|
||||
@@ -116,13 +116,13 @@ export type PromptDecision =
|
||||
/** Model-request failure with an optional machine-routable provider code. */
|
||||
export type RequestError = Error & { code?: string }
|
||||
|
||||
/** Live metadata for one model request that reached adapter dispatch. */
|
||||
/** Live metadata for one model request whose outer stream handle was obtained. */
|
||||
export interface AgentModelRequest {
|
||||
/** Final registered provider route. */
|
||||
/** Final request provider route; a short-circuit listener may own it. */
|
||||
readonly provider: string
|
||||
/** Final adapter-owned model id. */
|
||||
/** Final request model id; a short-circuit listener may own it. */
|
||||
readonly model: string
|
||||
/** Registration-bound context capacity when the adapter exposed one. */
|
||||
/** Registration-bound context capacity when preparation exposed one. */
|
||||
readonly contextWindow?: number
|
||||
}
|
||||
|
||||
@@ -371,11 +371,11 @@ declare module 'cordis' {
|
||||
*/
|
||||
'agent/request'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, signal: AbortSignal, next: () => Promise<LlmCallConfig>): Promise<LlmCallConfig>
|
||||
/**
|
||||
* One model request constructed its final stream handle and is about to
|
||||
* iterate it. This live notification is not durable or replayed; failed or
|
||||
* aborted iteration still has a dispatch, while preparation and
|
||||
* synchronous stream-construction failures do not. Listener failures are
|
||||
* contained and cannot affect the request.
|
||||
* One model request obtained its outer `llm/stream` handle and is about to
|
||||
* iterate it. This observes an Agent-loop request attempt, not proof that
|
||||
* provider I/O began. The notification is live, contained, and not replayed.
|
||||
* Preparation or a synchronous outer waterfall failure emits nothing;
|
||||
* failures or abortion after the handle returns still count.
|
||||
* @param agent - the agent dispatching the model request.
|
||||
* @param turn - the open turn number.
|
||||
* @param step - the request's step number.
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/host/apiproxy/README.md
|
||||
README.md: d3d1711242f71832f63eb570242fdf9e149cc988
|
||||
README.zh.md: 4e52058c1795ea23a9ca8ed46b8890c85f129eb4
|
||||
README.md: 263cb2f661e2f5b91cc4885ac99fe3afe61bbfbc
|
||||
README.zh.md: 400031b7d6d668d6ec7d2922b8a6abb855fe5b7b
|
||||
|
||||
@@ -22,7 +22,7 @@ Workspace and Session lists are separate reconnect baselines. `workspace.create`
|
||||
|
||||
`session.history` pages on message boundaries, and its tail page (no `beforeSeq`) carries session-level projections the page window cannot supply: the in-flight partial's chunk events; `todos`, the latest `todo/write` whole-list projection; and `metrics`, full-log usage deduplicated by `(turn, step)` plus current token-meter pressure. Older pages omit the session-level projections. Live `session/metrics` mux frames carry monotonic log/projection revisions, so clients reject stale frames and preserve the counters while prepending older pages. Cache reads and writes remain disjoint buckets; the cache-hit denominator is uncached input plus cache reads.
|
||||
|
||||
Context capacity uses a distinct transient `session/model-request` mux frame emitted from the contained Agent notification after an actual request reaches dispatch. It carries turn, step, final provider/model, and optional capacity only to mux connections already open at that instant. `session.history`, mux subscription baselines, reconnects, and session restore never query or replay prior capacity; a frame without capacity explicitly clears the earlier connection-local value.
|
||||
Context capacity uses a distinct transient `session/model-request` mux frame emitted from the contained Agent notification after an observed request attempt returns its outer stream handle. This boundary does not prove provider I/O began. The frame carries turn, step, final provider/model, and optional capacity only to mux connections already open at that instant. `session.history`, mux subscription baselines, reconnects, and session restore never query or replay prior capacity; a frame without capacity explicitly clears the earlier connection-local value.
|
||||
|
||||
The `command.*` and `skill.*` domains expose the host command registry and skill catalog to clients. Every method addresses one session's agent by `sessionId` (a served session always has an Agent; `command.*` resumes cold sessions through the same path as `session.*`, while `skill.list` resolves the project root from the session header without touching the Agent registry). `command.execute` runs a slash-command line host-side and returns a detached result; the carrier's request signal cancels the running handler. `host/commands-changed` is the catalog invalidation frame: clients refetch `command.list` instead of diffing.
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ Workspace 列表与 Session 列表是相互独立的重连基线。`workspace.cr
|
||||
|
||||
`session.history` 按消息边界分页，其尾页（不带 `beforeSeq`）携带页窗口本身无法提供的会话级投影：进行中局部消息的分片事件；`todos`，即最后一次 `todo/write` 的整表投影；以及 `metrics`，即按 `(turn, step)` 去重的完整日志用量与当前 token 计量压力。较早的页面省略会话级投影。实时 `session/metrics` mux 帧携带单调递增的日志修订号与投影修订号，因此客户端会拒绝陈旧帧，并在向前加载较早页面时保留计数器。缓存读取与缓存写入保持为彼此独立的计数项；缓存命中率的分母是未缓存输入加缓存读取。
|
||||
|
||||
上下文容量使用独立的临时 `session/model-request` mux 帧；实际请求到达分派点后，该帧由失败会被收容的 Agent 通知发出。该帧携带轮次、步骤、最终提供方／模型与可选容量，且只发送给当时已经打开的 mux 连接。`session.history`、mux 订阅基线、重连和会话恢复绝不会查询或回放先前的容量；不带容量的帧会显式清除较早的连接本地值。
|
||||
上下文容量使用独立的临时 `session/model-request` mux 帧；观察到的请求尝试返回外层流句柄后，该帧由失败会被收容的 Agent 通知发出。这个边界不能证明提供方 I/O 已开始。该帧携带轮次、步骤、最终提供方／模型与可选容量，且只发送给当时已经打开的 mux 连接。`session.history`、mux 订阅基线、重连和会话恢复绝不会查询或回放先前的容量；不带容量的帧会显式清除较早的连接本地值。
|
||||
|
||||
`command.*` 与 `skill.*` 领域向客户端暴露宿主命令注册表和技能目录。每个方法都通过 `sessionId` 寻址一个会话的 Agent（被服务的会话必有 Agent；`command.*` 经由与 `session.*` 相同的路径恢复冷会话，而 `skill.list` 从会话头解析项目根目录，不触碰 Agent 注册表）。`command.execute` 在宿主侧运行一条斜杠命令行并返回脱耦结果；载体的请求信号可取消正在运行的处理器。`host/commands-changed` 是目录失效帧：客户端重新拉取 `command.list` 而不是做差分。
|
||||
|
||||
|
||||
@@ -60,11 +60,11 @@ export type MuxFrame =
|
||||
| { type: 'session/subscribed'; sessionId: SessionId; lastSeq: number }
|
||||
| { type: 'session/metrics'; sessionId: SessionId; metrics: SessionMetrics }
|
||||
/**
|
||||
* One model request observed by this already-open mux connection after its
|
||||
* final route and stream handle were resolved. This frame is transient: mux
|
||||
* baselines, reconnects, and session history never replay it. An absent
|
||||
* `contextWindow` explicitly clears a capacity observed from an earlier
|
||||
* request on the same connection.
|
||||
* One request attempt observed by this already-open mux connection after its
|
||||
* final route and outer `llm/stream` handle were obtained. This does not prove
|
||||
* provider I/O began. The frame is transient: mux baselines, reconnects, and
|
||||
* session history never replay it. An absent `contextWindow` explicitly clears
|
||||
* a capacity observed from an earlier request on the same connection.
|
||||
*/
|
||||
| {
|
||||
type: 'session/model-request'
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
|
||||
README.md: d28a5632a3fbdbf11c7dba2ee0c57a704f2ba6f4
|
||||
README.zh.md: fd93fa43d5bfabd6e8751d4efbad229096ced08d
|
||||
README.md: 5d0459b722c4c5f472231ee86e775bbc4ff7b7f2
|
||||
README.zh.md: 3dd7153d63c8c4b40b737ed58d5d6d1a29f3153a
|
||||
|
||||
@@ -25,7 +25,7 @@ Provider and model metadata is a discovery surface, not a routing whitelist. `re
|
||||
|
||||
Exact-model metadata is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelInfo()` asks the adapter that owns the exact provider/model route once; an adapter can describe an unlisted dynamic model, and absent `context` or `reasoning` fields mean only that those capabilities are unavailable. Invalid identity, context, or reasoning metadata fails with `INVALID_MODEL_INFO`, `INVALID_MODEL_CONTEXT`, or `INVALID_MODEL_REASONING`.
|
||||
|
||||
Reasoning identifiers are opaque adapter-owned strings rather than a core enum. An adapter publishes its ordered selectable list, including an `off` id when that model's capability API exposes one. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous model resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally exposes the detached context metadata from that same lookup and retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. Its dispatch observer runs after a final stream handle is constructed and before adapter iteration. An unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
|
||||
Reasoning identifiers are opaque adapter-owned strings rather than a core enum. An adapter publishes its ordered selectable list, including an `off` id when that model's capability API exposes one. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous model resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally exposes the detached context metadata from that same lookup and retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. An unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
|
||||
|
||||
### Events
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@
|
||||
|
||||
确切模型元数据是独立的正确性查询，不是 catalog 装饰或全局 LLM 设置。`resolveModelInfo()` 会向拥有精确提供方／模型路由的适配器查询一次；适配器可以描述未列出的动态模型，缺少 `context` 或 `reasoning` 字段只表示相应能力不可用。无效的身份、上下文或推理元数据会以 `INVALID_MODEL_INFO`、`INVALID_MODEL_CONTEXT` 或 `INVALID_MODEL_REASONING` 失败。
|
||||
|
||||
推理标识符是由适配器持有的不透明字符串，而非核心枚举。适配器会公布有序可选列表；模型能力 API 提供 `off` id 时，列表也会包含它。`resolveCallConfig()` 只接受与已公布标识符完全一致的值，在存在 `defaultEffort` 时填入它，否则保留提供方默认值。异步模型解析器会接收调用方的 signal，并且必须在取消后迅速完成结算。`prepareCall()` 还会公开同一次查询得到的脱耦上下文元数据，并让精确适配器注册跨越请求头记录和最终分派，因此 HMR（热模块替换）不会将一个适配器的能力结果与另一个适配器的请求混用；复用其一次性句柄或更改调用配置字段会以 `INVALID_PREPARED_CALL` 失败。其分派观察器在最终流句柄构造完成后、适配器开始迭代前运行。不支持的显式或配置推理强度会在提供方 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。
|
||||
推理标识符是由适配器持有的不透明字符串，而非核心枚举。适配器会公布有序可选列表；模型能力 API 提供 `off` id 时，列表也会包含它。`resolveCallConfig()` 只接受与已公布标识符完全一致的值，在存在 `defaultEffort` 时填入它，否则保留提供方默认值。异步模型解析器会接收调用方的 signal，并且必须在取消后迅速完成结算。`prepareCall()` 还会公开同一次查询得到的脱耦上下文元数据，并让精确适配器注册跨越请求头记录和最终分派，因此 HMR（热模块替换）不会将一个适配器的能力结果与另一个适配器的请求混用；复用其一次性句柄或更改调用配置字段会以 `INVALID_PREPARED_CALL` 失败。不支持的显式或配置推理强度会在提供方 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。
|
||||
|
||||
### 事件
|
||||
|
||||
|
||||
@@ -119,11 +119,9 @@ export interface PreparedLlmCall {
|
||||
* preparation. The request's call-config fields must match {@link config};
|
||||
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
|
||||
* @param options - fully assembled request carrying the prepared config.
|
||||
* @param onDispatched - contained Agent-loop notification hook invoked after
|
||||
* a stream handle is constructed and before its adapter is iterated.
|
||||
* @returns the chunk stream, including the `llm/stream` waterfall.
|
||||
*/
|
||||
stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -408,6 +406,7 @@ export class LlmService extends Service {
|
||||
const resolved = await this.resolveModelInfoFor(registration, config.model, signal)
|
||||
const reasoning = resolved.reasoning
|
||||
const requested = config.reasoningEffort
|
||||
let resolvedConfig = config
|
||||
if (reasoning === undefined) {
|
||||
if (requested !== undefined) {
|
||||
throw new LlmError(
|
||||
@@ -415,26 +414,20 @@ export class LlmService extends Service {
|
||||
'UNSUPPORTED_REASONING_EFFORT',
|
||||
)
|
||||
}
|
||||
return {
|
||||
config,
|
||||
...resolved.context === undefined ? {} : { context: resolved.context },
|
||||
} else {
|
||||
const effective = requested ?? reasoning.defaultEffort
|
||||
if (effective !== undefined) {
|
||||
if (!reasoning.efforts.some(effort => effort.id === effective)) {
|
||||
throw new LlmError(
|
||||
`provider "${config.provider}" model "${config.model}" does not support reasoning effort "${effective}"`,
|
||||
'UNSUPPORTED_REASONING_EFFORT',
|
||||
)
|
||||
}
|
||||
if (requested !== effective) resolvedConfig = { ...config, reasoningEffort: effective }
|
||||
}
|
||||
}
|
||||
const effective = requested ?? reasoning.defaultEffort
|
||||
if (effective === undefined) {
|
||||
return {
|
||||
config,
|
||||
...resolved.context === undefined ? {} : { context: resolved.context },
|
||||
}
|
||||
}
|
||||
if (!reasoning.efforts.some(effort => effort.id === effective)) {
|
||||
throw new LlmError(
|
||||
`provider "${config.provider}" model "${config.model}" does not support reasoning effort "${effective}"`,
|
||||
'UNSUPPORTED_REASONING_EFFORT',
|
||||
)
|
||||
}
|
||||
return {
|
||||
config: requested === effective ? config : { ...config, reasoningEffort: effective },
|
||||
config: resolvedConfig,
|
||||
...resolved.context === undefined ? {} : { context: resolved.context },
|
||||
}
|
||||
}
|
||||
@@ -458,16 +451,12 @@ export class LlmService extends Service {
|
||||
return Object.freeze({
|
||||
config: resolvedConfig,
|
||||
...context === undefined ? {} : { context },
|
||||
stream: (options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk> => {
|
||||
stream: (options: GenerateOptions): AsyncIterable<StreamChunk> => {
|
||||
if (dispatched) {
|
||||
throw new LlmError('a prepared LLM call can only be dispatched once', 'INVALID_PREPARED_CALL')
|
||||
}
|
||||
dispatched = true
|
||||
return this.streamWithRegistration(
|
||||
options,
|
||||
{ registration, config: resolvedConfig },
|
||||
onDispatched,
|
||||
)
|
||||
return this.streamWithRegistration(options, { registration, config: resolvedConfig })
|
||||
},
|
||||
})
|
||||
}
|
||||
@@ -502,50 +491,24 @@ export class LlmService extends Service {
|
||||
* so it cannot suppress the primary provider error. A downstream close awaits
|
||||
* adapter cleanup, whose failures remain ordinary untagged work.
|
||||
*/
|
||||
private adapterStream(
|
||||
private async * adapterStream(
|
||||
options: GenerateOptions,
|
||||
failures: AdapterFailureScope,
|
||||
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
|
||||
onDispatched?: () => void,
|
||||
): AsyncIterable<StreamChunk> {
|
||||
if (prepared === undefined) {
|
||||
return this.resolveAndStream(options, failures, onDispatched)
|
||||
}
|
||||
): AsyncGenerator<StreamChunk> {
|
||||
let iterator: AsyncIterator<StreamChunk>
|
||||
try {
|
||||
const registration = prepared.registration
|
||||
const registration = prepared?.registration ?? this.registration(options.provider)
|
||||
failures.retryPolicy = registration.retryPolicy
|
||||
const resolvedConfig = prepared.config
|
||||
if (!callConfigEquals(options, resolvedConfig)) {
|
||||
const resolvedConfig = prepared === undefined
|
||||
? (await this.resolveCallFor(registration, options, options.signal)).config
|
||||
: prepared.config
|
||||
if (prepared !== undefined && !callConfigEquals(options, resolvedConfig)) {
|
||||
throw new LlmError(
|
||||
'prepared LLM call config changed before adapter dispatch',
|
||||
'INVALID_PREPARED_CALL',
|
||||
)
|
||||
}
|
||||
const adapter = registration.adapter
|
||||
const stream = adapter.stream(this.forAdapter(options, adapter))
|
||||
iterator = stream[Symbol.asyncIterator]()
|
||||
} catch (error: unknown) {
|
||||
return this.failedAdapterStream(markLlmAdapterFailure(failures, error))
|
||||
}
|
||||
this.notifyDispatched(onDispatched)
|
||||
return this.iterateAdapter(iterator, failures)
|
||||
}
|
||||
|
||||
private async * resolveAndStream(
|
||||
options: GenerateOptions,
|
||||
failures: AdapterFailureScope,
|
||||
onDispatched?: () => void,
|
||||
): AsyncGenerator<StreamChunk> {
|
||||
let iterator: AsyncIterator<StreamChunk>
|
||||
try {
|
||||
const registration = this.registration(options.provider)
|
||||
failures.retryPolicy = registration.retryPolicy
|
||||
const resolvedConfig = (await this.resolveCallFor(
|
||||
registration,
|
||||
options,
|
||||
options.signal,
|
||||
)).config
|
||||
const resolvedOptions = callConfigEquals(options, resolvedConfig)
|
||||
? options
|
||||
: Object.isFrozen(options)
|
||||
@@ -557,19 +520,7 @@ export class LlmService extends Service {
|
||||
} catch (error: unknown) {
|
||||
throw markLlmAdapterFailure(failures, error)
|
||||
}
|
||||
this.notifyDispatched(onDispatched)
|
||||
yield* this.iterateAdapter(iterator, failures)
|
||||
}
|
||||
|
||||
private async * failedAdapterStream(error: Error): AsyncGenerator<StreamChunk> {
|
||||
await Promise.resolve()
|
||||
throw error
|
||||
}
|
||||
|
||||
private async * iterateAdapter(
|
||||
iterator: AsyncIterator<StreamChunk>,
|
||||
failures: AdapterFailureScope,
|
||||
): AsyncGenerator<StreamChunk> {
|
||||
let completed = false
|
||||
let iterationFailed = false
|
||||
try {
|
||||
@@ -599,15 +550,6 @@ export class LlmService extends Service {
|
||||
}
|
||||
}
|
||||
|
||||
private notifyDispatched(onDispatched: (() => void) | undefined): void {
|
||||
if (onDispatched === undefined) return
|
||||
try {
|
||||
onDispatched()
|
||||
} catch (error: unknown) {
|
||||
this.ctx.logger.warn(`llm dispatch observer threw: ${String(error)}`)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Stream one model call as raw chunks (token-level deltas). Throws
|
||||
* `LlmError` with code `NO_ADAPTER` if no adapter is registered for
|
||||
@@ -619,32 +561,23 @@ export class LlmService extends Service {
|
||||
* agent-loop request recovery; middleware and nested-call failures remain
|
||||
* untagged for the outer call.
|
||||
* @param options - the full request; `options.provider` selects the adapter.
|
||||
* @param onDispatched - contained Agent-loop notification hook invoked after
|
||||
* a stream handle is constructed and before its adapter is iterated.
|
||||
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
|
||||
*/
|
||||
stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk> {
|
||||
return this.streamWithRegistration(options, undefined, onDispatched)
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
return this.streamWithRegistration(options)
|
||||
}
|
||||
|
||||
private streamWithRegistration(
|
||||
options: GenerateOptions,
|
||||
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
|
||||
onDispatched?: () => void,
|
||||
): AsyncIterable<StreamChunk> {
|
||||
const failures: AdapterFailureScope = { failures: new WeakMap<Error, LlmFailure>() }
|
||||
let terminalEntered = false
|
||||
const stream = this.ctx.waterfall(
|
||||
this,
|
||||
'llm/stream',
|
||||
options,
|
||||
() => {
|
||||
terminalEntered = true
|
||||
return this.adapterStream(options, failures, prepared, onDispatched)
|
||||
},
|
||||
() => this.adapterStream(options, failures, prepared),
|
||||
)
|
||||
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- waterfall mutates this latch.
|
||||
if (!terminalEntered) this.notifyDispatched(onDispatched)
|
||||
return bindAdapterFailureScope(stream, failures)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1070,11 +1070,14 @@ describe('LlmService', () => {
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
description: 'Resolved model',
|
||||
context: source,
|
||||
reasoning: {
|
||||
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
},
|
||||
reasoning: model === 'no-default'
|
||||
? { efforts: [{ id: ReasoningEffortId('high'), name: 'High' }] }
|
||||
: {
|
||||
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
},
|
||||
})
|
||||
}
|
||||
}(SCRIPT)
|
||||
@@ -1090,6 +1093,11 @@ describe('LlmService', () => {
|
||||
messages: [],
|
||||
})) { /* drain */ }
|
||||
expect(resolutions).toBe(1)
|
||||
|
||||
const noDefault = await ctx.llm.prepareCall({ provider: 'route', model: 'no-default' })
|
||||
expect(noDefault.config).toEqual({ provider: 'route', model: 'no-default' })
|
||||
expect(noDefault.context).toEqual({ contextWindow: 64_000 })
|
||||
expect(resolutions).toBe(2)
|
||||
})
|
||||
|
||||
it('passes cancellation through exact-model resolution', async () => {
|
||||
|
||||
Reference in New Issue
Block a user