refactor(llm): simplify live request telemetry (round 2)

This commit is contained in:
Hypatia May
2026-07-28 19:03:45 +08:00
parent 35b9c454e5
commit e6ce6abd7d
31 changed files with 159 additions and 187 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/core-data-structures/llm-streaming.md
llm-streaming.md: 89628ebb96a2e8eec5209635cd92859427df4a8d
llm-streaming.zh.md: 9c8fcc1f24b970f3a7cdd7cd08d9ef3b934b4543
llm-streaming.md: 30230d208c582463b3680d760c27f33a71b4cf6f
llm-streaming.zh.md: 32b3485f967b849371f916f6a669b97c97701cc1

View File

@@ -161,7 +161,7 @@ declare class BlockAssembler {
## The seam
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerRetryPolicy()` is captured per route with normal defaults, while `providerInfo()` and asynchronous `listModels()` feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or capability, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmService.resolveModelInfo()` validates and detaches the aggregate. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch, and to retain detached context metadata from that exact lookup. Its optional observer runs after a final stream handle is constructed and before adapter iteration. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerRetryPolicy()` is captured per route with normal defaults, while `providerInfo()` and asynchronous `listModels()` feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or capability, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmService.resolveModelInfo()` validates and detaches the aggregate. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch, and to retain detached context metadata from that exact lookup. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. AgentLoop observes a request attempt once the outer waterfall returns a stream handle; that limited boundary does not prove a lazy terminal adapter was constructed or began provider I/O. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
```ts type-equiv
/** One model call whose config and adapter registration were resolved together. */
@@ -175,11 +175,9 @@ interface PreparedLlmCall {
* preparation. The request's call-config fields must match {@link config};
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
* @param options - fully assembled request carrying the prepared config.
* @param onDispatched - contained Agent-loop notification hook invoked after
* a stream handle is constructed and before its adapter is iterated.
* @returns the chunk stream, including the `llm/stream` waterfall.
*/
stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
}
```

View File

@@ -161,7 +161,7 @@ declare class BlockAssembler {
## seam
`LlmAdapter` 是提供方 seam创建子类、实现 `stream()`,再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器;`GenerateOptions.model` 会传给该适配器,无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerRetryPolicy()` 会按路由捕获并填入 normal 默认值,`providerInfo()` 与异步 `listModels()` 方法则为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考,不是请求白名单:适配器仍是权威,并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份,以及可选的对正确性敏感的上下文容量、由模型持有的有序推理强度 ID 和部署默认值;字段缺失表示元数据或能力不可用,而不表示目录成员关系无效。解析器会接收可选的取消信号,并且必须在信号中止后迅速完成结算。`LlmService.resolveModelInfo()` 会校验聚合结果并返回分离值。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值因此直接调用也无法绕过对不支持推理强度的拒绝直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`,使模型解析、请求头持久记录和分派全程使用同一项注册,并保留来自同一次精确查询的分离上下文元数据。其可选观察器在最终流句柄构造完成后、适配器开始迭代前运行。适配器查找发生在 `llm/stream` waterfall瀑布式事件的终端 continuation因此 listener 可以在查找前短路调用,或路由一个可变的一次性请求。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface`ctx.llm.stream()`)与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
`LlmAdapter` 是提供方 seam创建子类、实现 `stream()`,再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器;`GenerateOptions.model` 会传给该适配器,无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerRetryPolicy()` 会按路由捕获并填入 normal 默认值,`providerInfo()` 与异步 `listModels()` 方法则为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考,不是请求白名单:适配器仍是权威,并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份,以及可选的对正确性敏感的上下文容量、由模型持有的有序推理强度 ID 和部署默认值;字段缺失表示元数据或能力不可用,而不表示目录成员关系无效。解析器会接收可选的取消信号,并且必须在信号中止后迅速完成结算。`LlmService.resolveModelInfo()` 会校验聚合结果并返回分离值。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值因此直接调用也无法绕过对不支持推理强度的拒绝直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`,使模型解析、请求头持久记录和分派全程使用同一项注册,并保留来自同一次精确查询的分离上下文元数据。适配器查找发生在 `llm/stream` waterfall瀑布式事件的终端 continuation因此 listener 可以在查找前短路调用,或路由一个可变的一次性请求。AgentLoop 在外层 waterfall 返回流句柄时观察到一次请求尝试;这个有限边界不能证明惰性终端适配器已构造完成或开始提供方 I/O。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface`ctx.llm.stream()`)与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
```ts type-equiv
/** One model call whose config and adapter registration were resolved together. */
@@ -175,11 +175,9 @@ interface PreparedLlmCall {
* preparation. The request's call-config fields must match {@link config};
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
* @param options - fully assembled request carrying the prepared config.
* @param onDispatched - contained Agent-loop notification hook invoked after
* a stream handle is constructed and before its adapter is iterated.
* @returns the chunk stream, including the `llm/stream` waterfall.
*/
stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
}
```