mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
refactor(web): publish transient model request capacity (round 1)
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/core-data-structures/llm-streaming.md
|
||||
llm-streaming.md: db46deee28cd053d034f889eb7625c9f222b418b
|
||||
llm-streaming.zh.md: fbff50bf1f3b86afd313c2d5020c15dd88e0b449
|
||||
llm-streaming.md: 89628ebb96a2e8eec5209635cd92859427df4a8d
|
||||
llm-streaming.zh.md: 9c8fcc1f24b970f3a7cdd7cd08d9ef3b934b4543
|
||||
|
||||
@@ -161,21 +161,25 @@ declare class BlockAssembler {
|
||||
|
||||
## The seam
|
||||
|
||||
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerRetryPolicy()` is captured per route with normal defaults, while `providerInfo()` and asynchronous `listModels()` feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or capability, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmService.resolveModelInfo()` validates and detaches the aggregate. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
|
||||
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerRetryPolicy()` is captured per route with normal defaults, while `providerInfo()` and asynchronous `listModels()` feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or capability, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmService.resolveModelInfo()` validates and detaches the aggregate. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch, and to retain detached context metadata from that exact lookup. Its optional observer runs after a final stream handle is constructed and before adapter iteration. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
|
||||
|
||||
```ts type-equiv
|
||||
/** One model call whose config and adapter registration were resolved together. */
|
||||
interface PreparedLlmCall {
|
||||
/** Detached, deep-frozen config with any adapter-owned default materialized. */
|
||||
readonly config: LlmCallConfig
|
||||
/** Detached context metadata resolved with the registration-bound call. */
|
||||
readonly context?: LlmModelContext
|
||||
/**
|
||||
* Dispatch this call once through the registration captured during
|
||||
* preparation. The request's call-config fields must match {@link config};
|
||||
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
|
||||
* @param options - fully assembled request carrying the prepared config.
|
||||
* @param onDispatched - contained Agent-loop notification hook invoked after
|
||||
* a stream handle is constructed and before its adapter is iterated.
|
||||
* @returns the chunk stream, including the `llm/stream` waterfall.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
@@ -161,21 +161,25 @@ declare class BlockAssembler {
|
||||
|
||||
## seam
|
||||
|
||||
`LlmAdapter` 是提供方 seam:创建子类、实现 `stream()`,再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器;`GenerateOptions.model` 会传给该适配器,无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerRetryPolicy()` 会按路由捕获并填入 normal 默认值,`providerInfo()` 与异步 `listModels()` 方法则为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考,不是请求白名单:适配器仍是权威,并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份,以及可选的对正确性敏感的上下文容量、由模型持有的有序推理强度 ID 和部署默认值;字段缺失表示元数据或能力不可用,而不表示目录成员关系无效。解析器会接收可选的取消信号,并且必须在信号中止后迅速完成结算。`LlmService.resolveModelInfo()` 会校验聚合结果并返回分离值。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值,因此直接调用也无法绕过对不支持推理强度的拒绝;直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`,使模型解析、请求头持久记录和分派全程使用同一项注册。适配器查找发生在 `llm/stream` waterfall(瀑布式事件)的终端 continuation,因此 listener 可以在查找前短路调用,或路由一个可变的一次性请求。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface(`ctx.llm.stream()`)与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
|
||||
`LlmAdapter` 是提供方 seam:创建子类、实现 `stream()`,再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器;`GenerateOptions.model` 会传给该适配器,无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerRetryPolicy()` 会按路由捕获并填入 normal 默认值,`providerInfo()` 与异步 `listModels()` 方法则为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考,不是请求白名单:适配器仍是权威,并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份,以及可选的对正确性敏感的上下文容量、由模型持有的有序推理强度 ID 和部署默认值;字段缺失表示元数据或能力不可用,而不表示目录成员关系无效。解析器会接收可选的取消信号,并且必须在信号中止后迅速完成结算。`LlmService.resolveModelInfo()` 会校验聚合结果并返回分离值。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值,因此直接调用也无法绕过对不支持推理强度的拒绝;直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`,使模型解析、请求头持久记录和分派全程使用同一项注册,并保留来自同一次精确查询的分离上下文元数据。其可选观察器在最终流句柄构造完成后、适配器开始迭代前运行。适配器查找发生在 `llm/stream` waterfall(瀑布式事件)的终端 continuation,因此 listener 可以在查找前短路调用,或路由一个可变的一次性请求。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface(`ctx.llm.stream()`)与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
|
||||
|
||||
```ts type-equiv
|
||||
/** One model call whose config and adapter registration were resolved together. */
|
||||
interface PreparedLlmCall {
|
||||
/** Detached, deep-frozen config with any adapter-owned default materialized. */
|
||||
readonly config: LlmCallConfig
|
||||
/** Detached context metadata resolved with the registration-bound call. */
|
||||
readonly context?: LlmModelContext
|
||||
/**
|
||||
* Dispatch this call once through the registration captured during
|
||||
* preparation. The request's call-config fields must match {@link config};
|
||||
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
|
||||
* @param options - fully assembled request carrying the prepared config.
|
||||
* @param onDispatched - contained Agent-loop notification hook invoked after
|
||||
* a stream handle is constructed and before its adapter is iterated.
|
||||
* @returns the chunk stream, including the `llm/stream` waterfall.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user