mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
Merge remote-tracking branch 'origin/master' into docs/post-v3-release-proofreading
# Conflicts: # .agents/notes/implemented/architecture/2026-08-03-pi-ai-declared-provider-catalog.i18n.yaml # .agents/notes/implemented/architecture/2026-08-03-pi-ai-declared-provider-catalog.zh.md # packages/llm/llm-pi-ai/README.i18n.yaml # packages/llm/llm-pi-ai/README.zh.md
This commit is contained in:
xjt
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-03-pi-ai-declared-provider-catalog.md
|
||||
2026-08-03-pi-ai-declared-provider-catalog.md: 45de54f29e718ed1c9364b03da762700e1282e7b
|
||||
2026-08-03-pi-ai-declared-provider-catalog.zh.md: c8c9b74eefecab7c9116d612fbe874827e3737f0
|
||||
2026-08-03-pi-ai-declared-provider-catalog.md: 571986a94c8547538b44b11510609e0000923537
|
||||
2026-08-03-pi-ai-declared-provider-catalog.zh.md: c9a066a7c3fc99641c85a34d372ca41e7a042cf0
|
||||
|
||||
@@ -14,7 +14,7 @@ The adapter also streamed through `streamSimple` from `@earendil-works/pi-ai/com
|
||||
|
||||
A provider route is a **declaration**, and the installed catalog is its default. `resolveProfiles` no longer checks route keys against `getBuiltinProviders()`. Instead each route resolves to a materialized model list plus the pi-ai `Provider` that serves it:
|
||||
|
||||
- `catalog.ts` merges the installed catalog under the profile's own entries. A profile's `models` list *replaces* the route's catalog (an absent or empty list serves it unchanged), and each entry defaults its unset fields from the installed model of the same `id`. Only the fields the harness consumes are configurable — `id`, `name`, `contextWindow`, `maxTokens`; [[2026-08-08-pi-ai-per-model-reasoning-declarations]] later added `reasoningEfforts` and `compat`, which is also where the original "reasoning rides the installed entry or is absent" stance was revisited (a bare capability flag stays rejected; a full per-level declaration with wire spellings does not have its problem). Pricing and input modalities remain absent from the surface because nothing reads them: `replay.ts` zeroes pi-ai's cost metadata and `context.ts` keeps only text blocks. Materialization spreads the installed entry and overrides the configured fields, rather than enumerating the result: an enumerated rebuild silently drops every `Model` field this package does not model, which is how `headers` went missing from an nvidia route once already.
|
||||
- `catalog.ts` merges the installed catalog under the profile's own entries. A profile's `models` list *replaces* the route's catalog (an absent or empty list serves it unchanged), and each entry defaults its unset fields from the installed model of the same `id`. Only the fields the harness consumes are configurable — `id`, `name`, `contextWindow`, `maxTokens`; [[2026-08-08-pi-ai-per-model-reasoning-declarations]] later added `reasoningEfforts` and `compat`, which is also where the original "reasoning rides the installed entry or is absent" stance was revisited (a bare capability flag stays rejected; a full per-level declaration with wire spellings does not have its problem). Input modalities were opened later, as `input` on an entry plus a route-level `defaultInput`, once the image-admission points turned an unreported modality into a refusal a deployment could not lift ([[2026-08-12-pi-ai-route-default-input-modalities]]); the "nothing reads them" reasoning had described `llm-deepseek`'s serializer rather than this route, whose converter carries images. Pricing stays absent for the original reason: `replay.ts` zeroes pi-ai's cost metadata and no consumer reports spend. Materialization spreads the installed entry and overrides the configured fields, rather than enumerating the result: an enumerated rebuild silently drops every `Model` field this package does not model, which is how `headers` went missing from an nvidia route once already.
|
||||
- `provider.ts` builds the route's `Provider`. A catalog route that keeps its catalog protocol **reuses** the installed provider with `getModels()` replaced; every other route is built by `createProvider()` over a protocol table whose entries are the same `@earendil-works/pi-ai/api/*.lazy` factories pi-ai's own provider factories use. That table is narrower than pi-ai's full API set on purpose — it holds only protocols a profile can completely describe with a key, an endpoint, and headers, so Bedrock (SigV4 plus a region), Vertex (project, location, ADC), Azure (provider environment plus an api-version), and Codex (OAuth) are absent rather than offered as routes that cannot authenticate. Catalog routes still reach them through their own provider; only an explicit override is refused.
|
||||
- `adapter.ts` turns each resolution into an **immutable snapshot** — the profiles plus a `createModels()` collection holding those providers — and every operation captures a whole snapshot before its first `await`.
|
||||
- A model's **explicitly configured** `maxTokens` becomes the seam's `defaultMaxTokens`. The value inherited from the installed catalog does not: pi-ai requires `Model.maxTokens` as the model's output *capability*, while `defaultMaxTokens` is a cap the deployment chose to send on requests that name none, and materializing the former as the latter would start capping every request at a number nobody picked.
|
||||
@@ -50,7 +50,7 @@ A route's auth follows from that. A catalog route keeps the installed provider's
|
||||
- **Keep `createProvider()` but skip the `Models` collection**, streaming through `provider.streamSimple(model, ctx, {apiKey})`. Smallest diff and the credential path is untouched, but `createProvider`'s `auth` is a required field that this path never invokes — a required-by-signature implementation with no caller. It also leaves `refreshModels` needing a hand-built `RefreshModelsContext`, and keeps the adapter off the runtime pi-ai actually supports.
|
||||
- **Reuse the installed provider for catalog routes and `createProvider()` only for declared ones**, with no shared resolution. Zero risk to catalog behavior, but catalog materialization, endpoint override, and per-model configuration would each exist twice, and a catalog route that repoints its protocol would have to jump paths mid-resolution. The chosen split confines the asymmetry to provider construction, where it is forced by pi-ai not exposing a built provider's API implementations.
|
||||
- **Rebuild every route through `createProvider()`**, including catalog ones. Fully symmetric, but a built `Provider` does not expose its `api`, so the protocol table would become the ceiling on which providers work — Bedrock loads its Smithy module through a separate entry point and would silently stop working.
|
||||
- **Expose pi-ai's whole `Model` shape** (cost, input modalities, `thinkingLevelMap`, `compat`). Maximum configurability, but no current consumer read those fields then, so a configured price or modality would change nothing while reading as supported. The consumer-driven half of this arrived later: [[2026-08-08-pi-ai-per-model-reasoning-declarations]] opened reasoning (as `reasoningEfforts`, not a raw `thinkingLevelMap`) and the two reasoning-dispatch `compat` switches once selectors and dispatch actually consumed them; cost and modalities stay closed for the original reason.
|
||||
- **Expose pi-ai's whole `Model` shape** (cost, input modalities, `thinkingLevelMap`, `compat`). Maximum configurability, but no current consumer read those fields then, so a configured price or modality would change nothing while reading as supported. The consumer-driven half of this arrived later, field by field as a consumer appeared: [[2026-08-08-pi-ai-per-model-reasoning-declarations]] opened reasoning (as `reasoningEfforts`, not a raw `thinkingLevelMap`) and the two reasoning-dispatch `compat` switches once selectors and dispatch actually consumed them, and [[2026-08-12-pi-ai-route-default-input-modalities]] opened modalities (as `input` and `defaultInput`, not a raw `Model.input` passthrough) once the image-admission points read them. Cost stays closed for the original reason.
|
||||
|
||||
- **Keep one mutable `Models` collection and re-sync it.** Fewer allocations, and correct for every operation that resolves synchronously. It is exactly wrong for the one that does not: `stream()` awaits a credential between capturing its model and dispatching it.
|
||||
- **Simulate an atomic directory swap with dispose-then-register.** No seam change, and it works whenever the new set is valid — which is the case that never needed atomicity.
|
||||
|
||||
@@ -14,9 +14,10 @@ Status: implemented
|
||||
|
||||
提供方路由是一份**声明**，已安装 catalog 是它的默认值。`resolveProfiles` 不再拿路由键去核对 `getBuiltinProviders()`，而是把每条路由解析成一份物化模型列表，外加服务它的 pi-ai `Provider`：
|
||||
|
||||
- `catalog.ts` 把已安装 catalog 合并到 profile 自身条目之下。profile 的 `models` 列表*替换*该路由的 catalog（列表缺席或为空则原样服务），每个条目从同 `id` 的已安装模型继承自身未设置的字段。只有 harness 会消费的字段可配置——`id`、`name`、`contextWindow`、`maxTokens`；[[2026-08-08-pi-ai-per-model-reasoning-declarations]] 之后加入了 `reasoningEfforts` 与 `compat`，当初「推理（reasoning）沿用已安装条目或直接缺席」的立场也在那里被重新审视（孤立的能力布尔量仍被拒绝；带 wire 拼写的逐档位完整声明没有它那个问题）。定价与输入模态仍不出现在配置面，因为没有任何读取方：`replay.ts` 把 pi-ai 的成本元数据清零，`context.ts` 只保留文本块。物化时以已安装条目铺底、再覆盖已配置的字段，而不是逐字段枚举结果：枚举式重建会静默丢弃本包未建模的每一个 `Model` 字段——`headers` 就是这样从某条 nvidia 路由上消失过一次。
|
||||
- `provider.ts` 构造路由的 `Provider`。保持 catalog 协议不变的 catalog 路由会**复用**已安装提供方，只替换 `getModels()`；其余路由都由 `createProvider()` 基于一张协议表构造，表中条目正是 pi-ai 自己的提供方工厂所用的 `@earendil-works/pi-ai/api/*.lazy` factory。该表刻意窄于 pi-ai 的完整 API 集合——只保留 profile 能用密钥、端点与标头完整描述的协议，因此 Bedrock（SigV4 加 region）、Vertex（project、location、ADC）、Azure（提供方环境加 api-version）与 Codex（OAuth）不在其中，而不是被当作无法认证的路由提供出去。catalog 路由仍可经自己的提供方抵达它们；被拒的只有显式覆盖。
|
||||
- `adapter.ts` 把每次解析变成一份**不可变快照**——profiles 加上持有这些提供方的 `createModels()` 集合——每个操作都在自己第一个 `await` 之前整体捕获一份。
|
||||
- `catalog.ts` 把已安装 catalog 合并到 profile 自身条目之下。profile 的 `models` 列表*替换*该路由的 catalog（列表缺席或为空则原样服务），每个条目从同 `id` 的已安装模型继承自身未设置的字段。只有 harness 会消费的字段可配置——`id`、`name`、`contextWindow`、`maxTokens`；[[2026-08-08-pi-ai-per-model-reasoning-declarations]] 之后加入了 `reasoningEfforts` 与 `compat`，当初「推理（reasoning）沿用已安装条目或直接缺席」的立场也在那里被重新审视（孤立的能力布尔量仍被拒绝；带 wire 拼写的逐档位完整声明没有它那个问题）。输入模态后来被开放，形态是条目上的 `input` 加路由级 `defaultInput`——图片准入点使得「未被报告的模态」变成部署无法解除的拒绝之后（[[2026-08-12-pi-ai-route-default-input-modalities]]）；当初「没有任何读取方」那句论证描述的其实是 `llm-deepseek` 的序列化器，而不是这条路由，它的转换器能携带图片。定价仍因原有理由不出现在配置面：`replay.ts` 把 pi-ai 的成本元数据清零，且没有任何消费方报告开销。物化时以已安装条目铺底、再覆盖已配置的字段，而不是逐字段枚举结果：枚举式重建会静默丢弃本包未建模的每一个 `Model` 字段——`headers` 就是这样从某条 nvidia 路由上消失过一次。
|
||||
- `provider.ts` 构造路由的 `Provider`。保持 catalog 协议不变的 catalog 路由会**复用**已安装提供方，只替换 `getModels()`；其余路由都由 `createProvider()` 基于一张协议表构造，表中条目正是 pi-ai 自己的提供方工厂所用的 `@earendil-works/pi-ai/api/*.lazy` factory。该表刻意窄于 pi-ai 的完整 API 集合——只保留 profile 能用密钥、端点与标头完整描述的协议，因此 Bedrock（SigV4 加 region）、Vertex（project、location、ADC）、Azure（提供方环境加 api-version）与 Codex（OAuth）不在其中，而不是被当作无法认证的路由提供出去。catalog 路由仍可经自己的 provider 抵达它们；被拒的只有显式覆盖。
|
||||
- `adapter.ts` 把每次解析变成一份**不可变快照**——profiles 加上持有这些 provider 的 `createModels()` 集合——每个操作都在自己第一个 `await` 之前整体捕获一份。
|
||||
|
||||
- 模型**显式配置**的 `maxTokens` 会成为 seam 的 `defaultMaxTokens`；从已安装 catalog 继承来的那份不会：pi-ai 要求 `Model.maxTokens` 表示模型的输出*能力*，而 `defaultMaxTokens` 是部署选定、发给未点名上限的请求的那个值，把前者物化成后者会让每个请求都被一个无人选择的数字封顶。
|
||||
|
||||
### 快照，而不是共享集合
|
||||
@@ -50,7 +51,7 @@ pi-ai 的 `Models` 自带一套凭据概念——按提供方 ID 索引的 `Cred
|
||||
- **保留 `createProvider()` 但不建 `Models` 集合**，改由 `provider.streamSimple(model, ctx, {apiKey})` 发起。改动最小且凭据路径原封不动，但 `createProvider` 的 `auth` 是必填字段，这条路上它永远不会被调用——一份因签名而必填、却没有调用方的实现。它还让 `refreshModels` 需要手工构造 `RefreshModelsContext`，并使适配器始终不在 pi-ai 真正支持的运行时上。
|
||||
- **catalog 路由复用已安装提供方，只有声明式路由走 `createProvider()`**，且两者不共享解析。对 catalog 行为零风险，但 catalog 物化、端点覆盖与每模型配置这三件事都要各写两遍，而改指协议的 catalog 路由还得在解析中途跳到另一条路径。已采纳的拆法把不对称收敛在提供方构造这一处——那里的不对称是 pi-ai 不暴露已构造提供方的 API 实现所强加的。
|
||||
- **让每条路由都经 `createProvider()` 重建**，包括 catalog 路由。完全对称，但已构造的 `Provider` 不暴露自己的 `api`，于是协议表会成为「哪些提供方能用」的天花板——Bedrock 经独立入口加载其 Smithy 模块，会因此静默失效。
|
||||
- **完整暴露 pi-ai 的 `Model` 形状**（成本、输入模态、`thinkingLevelMap`、`compat`）。可配置性最大，但这些字段当时没有任何读取方，因此配了价格或模态什么也不会改变，却看起来像是受支持的。这条否决里由消费方驱动的那一半后来兑现了：[[2026-08-08-pi-ai-per-model-reasoning-declarations]] 在选择器与分派真正消费之后开放了推理（以 `reasoningEfforts` 的形态，而非裸 `thinkingLevelMap`）和两个推理分派 `compat` 开关；成本与模态仍因原有理由保持关闭。
|
||||
- **完整暴露 pi-ai 的 `Model` 形状**（成本、输入模态、`thinkingLevelMap`、`compat`）。可配置性最大，但这些字段当时没有任何读取方，因此配了价格或模态什么也不会改变，却看起来像是受支持的。这条否决里由消费方驱动的那一半后来逐字段兑现了，每次都等到出现真实读取方：[[2026-08-08-pi-ai-per-model-reasoning-declarations]] 在选择器与分派真正消费之后开放了推理（以 `reasoningEfforts` 的形态，而非裸 `thinkingLevelMap`）和两个推理分派 `compat` 开关；[[2026-08-12-pi-ai-route-default-input-modalities]] 在图片准入点开始读取之后开放了模态（以 `input` 与 `defaultInput` 的形态，而非裸 `Model.input` 直通）。成本仍因原有理由保持关闭。
|
||||
|
||||
- **保留单个可变 `Models` 集合并重新同步。** 分配更少，且对每个同步完成解析的操作都是正确的；唯独对那个不同步的操作恰恰是错的：`stream()` 会在捕获模型与派发模型之间 await 一次凭据。
|
||||
- **用「先 dispose 再注册」模拟目录原子替换。** 无需改 seam，且在新集合有效时确实可用——而那正是从不需要原子性的那种情形。
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-12-pi-ai-route-default-input-modalities.md
|
||||
2026-08-12-pi-ai-route-default-input-modalities.md: efd20b2cd73979208bb777fa42536bc5b918e29e
|
||||
2026-08-12-pi-ai-route-default-input-modalities.zh.md: 069a7916c8d4ffe738ff910a851e0a9cd1f66d0a
|
||||
@@ -0,0 +1,53 @@
|
||||
# Agent Note: A pi-ai model declares its own input modalities, and undeclared means text
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-12-pi-ai-route-default-input-modalities.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Nothing in `settings.yaml` could describe a hand-declared pi-ai model as accepting images, and the adapter assumed text-only for every model the installed pi-ai catalog does not describe. Every model a deployment adds through the web UI's "add a custom provider" card is such a model, so an OpenAI-compatible gateway serving a vision model reported `inputModalities: ['text']` no matter what it actually served.
|
||||
|
||||
The harness treats an omitted modality as negative capability, and three admission points act on it before any request is built: model selection refuses to switch into a session that already holds images, prompt admission refuses an image, and `read_image` refuses to read one. Their diagnostics tell the user to select an image-capable model — advice with no reachable referent, because no configuration key could make a hand-declared model image-capable. The route was closed at the metadata, not at the capability: the request converter and every pi-ai wire protocol carry images, and `llm-pi-ai`'s own stream guard is the only thing that would have stopped one.
|
||||
|
||||
The assumption was justified in the source as the adapter's real capability rather than a deployment choice, and [[2026-08-03-pi-ai-declared-provider-catalog]] recorded the same reasoning when it decided which `Model` fields the configuration surface would expose ("nothing reads them: … `context.ts` keeps only text blocks"). That justification described the DeepSeek chat-completions adapter, whose serializer genuinely rejects image blocks, and had never been true of the pi-ai route. This note supersedes that one on modalities alone; pricing stays closed there for its own, still-current reason.
|
||||
|
||||
## Decision
|
||||
|
||||
**Modalities resolve entry `input` → installed catalog entry → route `defaultInput`, which itself defaults to `[text]`.** That is the chain `contextWindow` and `maxTokens` already use, field for field. pi-ai types `Model.input` required and per-model, so the entry field mirrors upstream directly: one route can serve a vision model beside a text-only one, and an override can correct a catalog model whose gateway serves other modalities than the catalog records. The route field spares a gateway whose *undescribed* models all take images from repeating itself on every entry.
|
||||
|
||||
**The route value is a fallback, not an override — the catalog outranks it.** This is the `default*` ordering rather than `compat`'s, and the two are not interchangeable: `compat` shadows the catalog because a route-level protocol repoint invalidates the catalog's reasoning-dispatch facts wholesale, while a modality is a per-model property the catalog states accurately for the models it ships. Making the route value win would mean `defaultInput: [text]` silently strips images from every catalog vision model on the route — a footgun with no matching benefit, since narrowing one such model is what that model's own `input` is for.
|
||||
|
||||
**Undeclared means `[text]`, and that is the absence of a declaration rather than a guess at the endpoint.** Nothing can interrogate a gateway for its modalities — no OpenAI-compatible listing endpoint reports them — so the only honest floor is the modality every supported protocol certainly carries. This is where the modality fallback parts company with the capacity ones: 262,144 tokens is merely plausible and wrong in both directions (a gateway serving 8k overflows, one serving 1M is wasted), while text is safe in one direction. The two wrong answers do not cost the same either. Under-claiming refuses the image before it is attached, naming the model, and the remedy is one documented line. Over-claiming admits an image the provider then rejects mid-turn, *after* prompt admission has committed the message durably, so the session keeps re-sending a request that cannot succeed and model selection refuses a switch to any text-only model. A cheap refusal at the earliest resolvable point beats an expensive one at the latest.
|
||||
|
||||
**An entry's empty list means the same as an absent one; the route's is refused.** `[]` describes a model that accepts nothing and could serve no request, so it states no answer and resolution continues past it. That reading is not cosmetic: the config schema materializes `[]` for an absent array, so treating it as "accepts nothing" would silently strip images from every catalog vision model a `models` list happens to name. The route value has nothing below it to answer instead, so its empty list is refused where it is written. The route's `models` list already resolves absent-and-empty the same way for the same reason.
|
||||
|
||||
**No configuration surface edits `input`.** It joins `compat`, `reasoningEfforts`, `thinkingBudgets`, and `headers` as a settings-document field, and the model-list editor stays a hand-written form over id, name, and the two capacities. This costs nothing durable because that card was already built to carry fields it does not edit: its row patch spreads the stored row before applying changes, and adoption keeps an existing row over a rediscovered candidate, so a hand-written `input` survives both.
|
||||
|
||||
The DeepSeek chat-completions adapter is untouched. Its `['text']` is a fact about its serializer, not a missing declaration, and it keeps refusing before the send.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **An optimistic `[text, image]` default** — makes the motivating case work with zero configuration, and the web form writes no modality at all, so a conservative default leaves the remedy in a file a web-only user has no reason to open. Rejected on the severity of being wrong: a refused attachment is a speed bump with a documented fix, while a provider rejection poisons the session, presents as an unexplained repeating failure, and is escapable only by switching models or starting over. Documenting the remedy on the model-configuration page closes the discoverability gap; nothing closes the poisoned session.
|
||||
- **A route value that overrides the catalog** (`compat`'s ordering: entry → route → catalog) — lets a deployment that repoints a catalog route at its own gateway declare "no vision here" once. Rejected because the same sentence then silently disables every catalog vision model on a route where someone wrote it by analogy with the capacity fields, and the legitimate case is served by that model's own `input`. An override would also have to be named `input` at the route, since calling it `default*` beside two genuine fallbacks would misdescribe it.
|
||||
- **No route field at all, only the entry one** — closest to upstream, which has no route-level concept. Rejected on the bulk case the product's own flow produces: "fetch available models" adopts thirty ids with no modality, and an all-vision gateway would need `input` hand-written on each.
|
||||
- **A route-level `defaultInput` with no entry field** — cannot mix modalities on one route or correct a single catalog model, leaving "split the provider across two route keys" as the only workaround, at the cost of a second permanent provider id and a duplicate entry in every model selector.
|
||||
- **Probe the endpoint for its modalities** — no OpenAI-compatible listing endpoint reports them.
|
||||
- **Infer from the model id** (`*-vision`, `*-vl`) — a naming convention is not a capability, and a gateway renames freely.
|
||||
- **Keep refusing and improve the diagnostic** — the message was already accurate about the state and useless about the remedy; the missing thing was the remedy.
|
||||
|
||||
## Consequences
|
||||
|
||||
A vision model on a custom provider costs one line, `input: [text, image]`, written in the settings document — or one line at the route when every model it lists takes images. That is the whole of the fix: the three admission points then admit images on it and `read_image` works. A deployment that writes nothing keeps exactly the behavior it had, so no existing route changes what it reports.
|
||||
|
||||
The image-admission gate keeps its meaning everywhere, because every modality it reads is now either recorded by the installed catalog or written by a person. Nothing claims a capability on a deployment's behalf.
|
||||
|
||||
A model that declares images its endpoint does not serve is not caught locally — the claim is not verified — and the resulting failure is expensive. Prompt admission commits the user message durably (`agent/inbox/spliced`) before the request is built, so the rejected image stays in the session log: that model keeps re-sending it, and model selection refuses a switch to any text-only model. Recovery is to select a model that does serve images, fork before the image, or start a session. Making that failure non-destructive — rolling an unconsumed image message back out of the log when the send fails — is the change that would make an optimistic default reconsiderable, and is not attempted here.
|
||||
|
||||
## Testing
|
||||
|
||||
`packages/llm/llm-pi-ai/tests/catalog.spec.ts` covers each rung of the chain and both readings of an empty list at the resolver: one route mixing an undeclared model with entry-declared text-only and vision models, a route default answering an undeclared model while an entry still outranks it, a catalog vision model keeping its modalities under a narrower route default, an entry's `[]` inheriting rather than emptying, and the route's `[]` refused. A separate case re-asserts every rung end to end — a written settings section, the plugin's own registration, and `ctx.llm.listModels` / `resolveModelInfo` — so a break between the document and `LlmModelInfo` cannot pass.
|
||||
|
||||
`config.spec.ts` holds the schema boundary: an unknown modality refused at both levels, the empty route list accepted by the schema and refused by the namespace validator that the settings seam actually runs, and the `[]` materialization for an absent array that the inheritance rule depends on.
|
||||
|
||||
No keyless snapshot lane exercises a pi-ai route: the snapshot examples drive `dsh-llm-replay`, which declares modalities directly in its configuration, and a pi-ai route needs a live endpoint whose port a static `cordis.yml` cannot name. The admission points this change feeds are already covered there through that provider (`examples/acp-agent/image.cordis.snapshot.yml` and `image-text-route.cordis.snapshot.yml`) and are unaffected — what changed is what one adapter reports, not how a gate reads it.
|
||||
@@ -0,0 +1,53 @@
|
||||
# Agent Note：pi-ai 模型自行声明输入模态，未声明即为文本
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-12-pi-ai-route-default-input-modalities.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
`settings.yaml` 里没有任何写法能把一个手写的 pi-ai 模型描述成接受图片，而适配器对已安装 pi-ai catalog 未描述的每个模型都假定纯文本。部署通过 Web UI 的“添加自定义提供商”卡片新增的模型统统属于这一类，因此一个提供视觉模型的 OpenAI 兼容网关，无论实际提供什么，报出来的都是 `inputModalities: ['text']`。
|
||||
|
||||
Harness 把缺失的模态当作否定能力，并有三个准入点在构造任何请求之前就依据它行动：模型选择拒绝切换进已含图片的会话，prompt 准入拒绝图片，`read_image` 拒绝读取。它们的诊断让用户去选一个支持图片的模型——这条建议没有可达的指向对象，因为没有任何配置键能让手写模型变得支持图片。这条路封死在元数据上而非能力上：请求转换器和每种 pi-ai 线路协议都能携带图片，`llm-pi-ai` 自身的流前置检查才是唯一会拦下它的东西。
|
||||
|
||||
源码中把这一假定论证为适配器的真实能力而非部署选择，而 [[2026-08-03-pi-ai-declared-provider-catalog]] 在决定配置面公开哪些 `Model` 字段时记录了同一套论证（「没有任何读取方：…… `context.ts` 只保留文本块」）。该论证描述的是 DeepSeek chat-completions 适配器——它的序列化器确实拒绝 image block——对 pi-ai 路由从来就不成立。本 note 仅在模态一项上取代那一条；定价在那里依然因其自身、仍然成立的理由保持关闭。
|
||||
|
||||
## 决策
|
||||
|
||||
**模态的解析顺序是：条目的 `input` → 已安装 catalog 条目 → 路由的 `defaultInput`，后者自身默认为 `[text]`。** 这与 `contextWindow`、`maxTokens` 已经在用的链条逐字段一致。pi-ai 把 `Model.input` 定义为必填且按模型给出，因此条目字段直接镜像上游：一条路由可以让视觉模型与纯文本模型并存，而 override 可以修正那些网关实际提供的模态与 catalog 记录不符的模型。路由字段则让那些**未被 catalog 描述的**模型全都接受图片的网关，不必在每个条目上重复一遍。
|
||||
|
||||
**路由值是回退值而非覆盖值——catalog 的优先级更高。** 这采用的是 `default*` 的顺序而非 `compat` 的，两者不可互换：`compat` 之所以盖住 catalog，是因为路由级的协议改指会整体作废 catalog 关于推理分派的事实；而模态是按模型的属性，对 catalog 自己出货的那些模型，它记录得准确无误。让路由值获胜就意味着 `defaultInput: [text]` 会悄悄剥掉该路由上每一个 catalog 视觉模型的图片能力——一个没有对应收益的坑，因为收窄其中某个模型正是该模型自己的 `input` 要做的事。
|
||||
|
||||
**未声明即 `[text]`，而这是「尚未声明」，不是对端点的猜测。** 没有任何环节能去询问网关的模态——没有任何 OpenAI 兼容的列表端点会报告它们——因此唯一诚实的底线是每个受支持协议都确定携带的那个模态。这也正是模态回退值与容量回退值分道扬镳之处：262,144 只是个说得过去的数字，且两个方向都会错（网关只给 8k 会溢出，给 1M 则被浪费），而 text 在一个方向上是安全的。两种猜错的代价同样并不对等。少声明会在图片被附加之前就拒绝并点名该模型，补救办法是一行有文档可依的配置。多声明会接纳一张图片、再由提供方在轮次中途拒绝——而此时 prompt 准入**早已**把消息持久化提交，于是会话会不断重发一个不可能成功的请求，且模型选择拒绝切换到任何纯文本模型。在最早可解析点付出一次廉价的拒绝，胜过在最晚点付出一次昂贵的。
|
||||
|
||||
**条目的空列表与缺省同义；路由的空列表则被拒绝。** `[]` 描述的是一个什么都不接受、无法服务任何请求的模型，因此不作答，解析继续往下走。这个读法不是修辞：配置 schema 会为缺省数组物化出 `[]`，把它当作“什么都不接受”，会悄悄剥掉 `models` 列表恰好点到的每一个 catalog 视觉模型的图片能力。而路由值下面没有可以代为作答的层级，因此它的空列表在写入处即被拒绝。路由的 `models` 列表出于同样的理由，早已用同一种方式解析缺省与空。
|
||||
|
||||
**没有任何配置界面编辑 `input`。** 它和 `compat`、`reasoningEfforts`、`thinkingBudgets`、`headers` 一样是 settings 文档字段，而模型列表编辑器仍是一张只覆盖 id、名称和两个容量的手写表单。这不会带来持久代价，因为那张卡片本来就是按“承载自己并不编辑的字段”建造的：它的行 patch 会先展开已存储的行再应用改动，而采纳候选时已有行优先于重新发现的候选，因此手写的 `input` 在两条路径上都能存活。
|
||||
|
||||
DeepSeek chat-completions 适配器保持不动。它的 `['text']` 是关于其序列化器的事实，而不是一处缺失的声明，它继续在发送前拒绝。
|
||||
|
||||
## 备选方案
|
||||
|
||||
- **乐观的 `[text, image]` 默认值** —— 让触发本次变更的场景零配置即可工作；而且网页表单不会写入任何模态，因此保守默认值会把补救办法留在一个纯 Web 用户没有理由打开的文件里。被否决的理由是猜错时的严重程度：被拒绝的附件是一个有文档可依的减速带，而提供方拒绝会毒化整个会话、表现为一次无从解释的反复失败，且只能靠换模型或重开会话脱身。把补救办法写进配置模型页即可补上可发现性的缺口；而毒化的会话没有任何东西能补。
|
||||
- **让路由值盖住 catalog**（`compat` 的顺序：条目 → 路由 → catalog）—— 可以让把 catalog 路由改指到自家网关的部署，一句话声明「这里没有视觉能力」。被否决是因为同一句话也会在有人照着容量字段类比写下它的路由上，悄悄禁用每一个 catalog 视觉模型；而那个正当场景由该模型自己的 `input` 承担。覆盖值还必须在路由级改名叫 `input`，因为在两个货真价实的回退值旁边把它叫作 `default*` 是名不副实。
|
||||
- **完全不要路由字段，只要条目字段** —— 最贴近上游（上游没有路由级概念）。被否决的理由是产品自身流程会产生的批量场景：「获取可用模型」一次采纳三十个不带模态的 id，全是视觉模型的网关就得逐个手写 `input`。
|
||||
- **只要路由级 `defaultInput`，不要条目字段** —— 无法在一条路由上混合模态，也无法修正单个 catalog 模型，唯一的变通办法只剩「把该提供方拆成两个路由键」，代价是多一个永久的 provider id 和每个模型选择器里的一项重复。
|
||||
- **探测端点的模态** —— 没有任何 OpenAI 兼容的列表端点会报告它们。
|
||||
- **从模型 id 推断**（`*-vision`、`*-vl`）—— 命名约定不是能力，网关可以随意改名。
|
||||
- **继续拒绝，只改进诊断** —— 那条消息对状态的描述本来就准确，对补救办法则毫无用处；缺的正是补救办法。
|
||||
|
||||
## 影响
|
||||
|
||||
自定义提供方下的视觉模型只需在 settings 文档里写一行 `input: [text, image]`——或者当该路由列出的模型全都接受图片时，在路由上写一行。这就是修复的全部：三个准入点随即接纳它上面的图片，`read_image` 也可用。什么都不写的部署保持原有行为完全不变，因此没有任何既有路由改变它报出的内容。
|
||||
|
||||
图片准入门禁在各处都保住了自己的意义，因为它读到的每一个模态，如今要么由已安装 catalog 记录，要么由人写下。没有任何环节会替部署宣称一项能力。
|
||||
|
||||
声明了端点并不提供的图片能力的模型不会在本地被拦下——该断言不经验证——而由此产生的失败代价高昂。prompt 准入在构造请求之前就把用户消息持久化提交（`agent/inbox/spliced`），因此被拒绝的图片会留在会话日志里：该模型会不断重发它，而模型选择拒绝切换到任何纯文本模型。恢复途径是选择一个确实提供图片能力的模型、fork 到图片之前，或者开启新会话。让这次失败不具破坏性——发送失败时把尚未消费的图片消息从日志中回滚出去——才是能让乐观默认值重新可考虑的那项改动，本次未做尝试。
|
||||
|
||||
## 测试
|
||||
|
||||
`packages/llm/llm-pi-ai/tests/catalog.spec.ts` 在 resolver 层覆盖了这条链的每一级以及空列表的两种读法：一条路由上未声明的模型与条目声明的纯文本、视觉模型并存，路由默认值为未声明的模型作答而条目仍然压过它，catalog 视觉模型在更窄的路由默认值下保住自身模态，条目的 `[]` 走继承而非清空，以及路由的 `[]` 被拒绝。另有一条用例把每一级端到端复验一遍——写入的 settings 段、插件自身的注册、以及 `ctx.llm.listModels` / `resolveModelInfo`——因此文档与 `LlmModelInfo` 之间若有断点无法蒙混过关。
|
||||
|
||||
`config.spec.ts` 负责 schema 边界：两个层级上的未知模态拒绝、路由空列表被 schema 接受而由 settings seam 真正运行的命名空间校验器拒绝，以及继承规则所倚赖的「缺省数组物化为 `[]`」这一事实。
|
||||
|
||||
没有任何无密钥 snapshot 通道会跑 pi-ai 路由：snapshot 示例驱动的是 `dsh-llm-replay`，它在自己的配置里直接声明模态，而 pi-ai 路由需要一个真实端点，其端口是静态 `cordis.yml` 无法写出的。本次变更所供给的那些准入点已经通过该提供方在那里得到覆盖（`examples/acp-agent/image.cordis.snapshot.yml` 与 `image-text-route.cordis.snapshot.yml`）且不受影响——改变的是某个适配器报告什么，而非门禁如何读取它。
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/config-catalog.md
|
||||
config-catalog.md: 415f4a305caf52121aa3ae9396bfce5442782f71
|
||||
config-catalog.zh.md: 6272f2a31bb994f2d2b95c4db2e258bd3e108f25
|
||||
config-catalog.md: bd5b646d119828373d5f8c1a5160f2e85c049ce8
|
||||
config-catalog.zh.md: 9d36fd28bde7133dc857ef4a96d64ee8f6910659
|
||||
|
||||
@@ -858,6 +858,17 @@ export interface PiAiProviderProfile {
|
||||
* never becomes a per-request cap on its own.
|
||||
*/
|
||||
defaultMaxTokens?: number
|
||||
/**
|
||||
* Request modalities for a model this route lists that neither its entry's
|
||||
* {@link PiAiModelProfile.input} nor the installed catalog declares (default
|
||||
* `[text]`). A fallback like the capacities above, not an override: a
|
||||
* catalog model keeps the modalities the catalog records for it, and this
|
||||
* value never narrows one. A gateway serving vision models the catalog does
|
||||
* not describe declares `[text, image]` once here instead of on every entry.
|
||||
* Unlike an entry's list, this one may not be empty — nothing sits below it
|
||||
* to answer instead.
|
||||
*/
|
||||
defaultInput?: PiAiModality[]
|
||||
/** Provider request headers; Harness attribution wins reserved names. */
|
||||
headers?: Record<string, string>
|
||||
/** Provider-neutral pi-ai reasoning level. */
|
||||
@@ -893,6 +904,18 @@ export interface PiAiModelProfile {
|
||||
* default on its own.
|
||||
*/
|
||||
maxTokens?: number
|
||||
/**
|
||||
* Request modalities this model accepts. Absent — or empty, which describes
|
||||
* a model that accepts nothing and so states no answer either — keeps the
|
||||
* installed catalog entry's modalities, then the route's `defaultInput`.
|
||||
* Declaring images is what makes a hand-declared vision model usable, and
|
||||
* declaring text alone corrects a catalog model whose gateway does not serve
|
||||
* what the catalog records. This is a claim about the endpoint, not a check
|
||||
* of it: nothing interrogates a gateway for what it accepts, so a model
|
||||
* claiming images its endpoint refuses is refused by the provider instead,
|
||||
* mid-turn.
|
||||
*/
|
||||
input?: PiAiModality[]
|
||||
/**
|
||||
* Selectable reasoning efforts. Absent inherits the installed catalog
|
||||
* entry's capability (a hand-declared model has none and does not reason);
|
||||
@@ -930,6 +953,9 @@ export interface PiAiCompatProfile {
|
||||
supportsReasoningEffort?: boolean
|
||||
}
|
||||
|
||||
/** One request modality a pi-ai model may accept. */
|
||||
export type PiAiModality = Model<Api>['input'][number]
|
||||
|
||||
/**
|
||||
* Selectable reasoning efforts for one model: each key is a level the model
|
||||
* offers (and selectors show), and its value is the wire spelling dispatch
|
||||
@@ -953,9 +979,9 @@ type PiThinkingFormat = NonNullable<OpenAICompletionsCompat['thinkingFormat']>
|
||||
type WithheldThinkingFormat = 'chat-template' | 'qwen-chat-template'
|
||||
```
|
||||
|
||||
Depends on: `CacheRetention` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · `OpenAICompletionsCompat` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
|
||||
Depends on: `Api` (`@earendil-works/pi-ai`) · `CacheRetention` (`@earendil-works/pi-ai`) · `Model` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · `OpenAICompletionsCompat` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
|
||||
|
||||
Source: [`packages/llm/llm-pi-ai/src/config.ts:142`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
Source: [`packages/llm/llm-pi-ai/src/config.ts:172`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-llm-replay`
|
||||
|
||||
|
||||
@@ -860,6 +860,17 @@ export interface PiAiProviderProfile {
|
||||
* never becomes a per-request cap on its own.
|
||||
*/
|
||||
defaultMaxTokens?: number
|
||||
/**
|
||||
* Request modalities for a model this route lists that neither its entry's
|
||||
* {@link PiAiModelProfile.input} nor the installed catalog declares (default
|
||||
* `[text]`). A fallback like the capacities above, not an override: a
|
||||
* catalog model keeps the modalities the catalog records for it, and this
|
||||
* value never narrows one. A gateway serving vision models the catalog does
|
||||
* not describe declares `[text, image]` once here instead of on every entry.
|
||||
* Unlike an entry's list, this one may not be empty — nothing sits below it
|
||||
* to answer instead.
|
||||
*/
|
||||
defaultInput?: PiAiModality[]
|
||||
/** Provider request headers; Harness attribution wins reserved names. */
|
||||
headers?: Record<string, string>
|
||||
/** Provider-neutral pi-ai reasoning level. */
|
||||
@@ -895,6 +906,18 @@ export interface PiAiModelProfile {
|
||||
* default on its own.
|
||||
*/
|
||||
maxTokens?: number
|
||||
/**
|
||||
* Request modalities this model accepts. Absent — or empty, which describes
|
||||
* a model that accepts nothing and so states no answer either — keeps the
|
||||
* installed catalog entry's modalities, then the route's `defaultInput`.
|
||||
* Declaring images is what makes a hand-declared vision model usable, and
|
||||
* declaring text alone corrects a catalog model whose gateway does not serve
|
||||
* what the catalog records. This is a claim about the endpoint, not a check
|
||||
* of it: nothing interrogates a gateway for what it accepts, so a model
|
||||
* claiming images its endpoint refuses is refused by the provider instead,
|
||||
* mid-turn.
|
||||
*/
|
||||
input?: PiAiModality[]
|
||||
/**
|
||||
* Selectable reasoning efforts. Absent inherits the installed catalog
|
||||
* entry's capability (a hand-declared model has none and does not reason);
|
||||
@@ -932,6 +955,9 @@ export interface PiAiCompatProfile {
|
||||
supportsReasoningEffort?: boolean
|
||||
}
|
||||
|
||||
/** One request modality a pi-ai model may accept. */
|
||||
export type PiAiModality = Model<Api>['input'][number]
|
||||
|
||||
/**
|
||||
* Selectable reasoning efforts for one model: each key is a level the model
|
||||
* offers (and selectors show), and its value is the wire spelling dispatch
|
||||
@@ -955,9 +981,9 @@ type PiThinkingFormat = NonNullable<OpenAICompletionsCompat['thinkingFormat']>
|
||||
type WithheldThinkingFormat = 'chat-template' | 'qwen-chat-template'
|
||||
```
|
||||
|
||||
依赖：`CacheRetention`（`@earendil-works/pi-ai`）· `ModelThinkingLevel`（`@earendil-works/pi-ai`）· `OpenAICompletionsCompat`（`@earendil-works/pi-ai`）· [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets`（`@earendil-works/pi-ai`）· `Transport`（`@earendil-works/pi-ai`）
|
||||
依赖：`Api`（`@earendil-works/pi-ai`）· `CacheRetention`（`@earendil-works/pi-ai`）· `Model`（`@earendil-works/pi-ai`）· `ModelThinkingLevel`（`@earendil-works/pi-ai`）· `OpenAICompletionsCompat`（`@earendil-works/pi-ai`）· [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets`（`@earendil-works/pi-ai`）· `Transport`（`@earendil-works/pi-ai`）
|
||||
|
||||
来源：[`packages/llm/llm-pi-ai/src/config.ts:142`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
来源：[`packages/llm/llm-pi-ai/src/config.ts:172`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-llm-replay`
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/user/guide/providers.md
|
||||
providers.md: a3f94f0cc86401c0f9e5b94cfd823bf9f08e6bfc
|
||||
providers.zh.md: 7d74e0086e62d8a0c2fb39085207125b4b4354e7
|
||||
providers.md: 099f434ec4602aa402239e83c708d81fcadd7732
|
||||
providers.zh.md: 367c90b525ad628b3cd86b2d22045c25064e88a1
|
||||
|
||||
@@ -28,6 +28,57 @@ The Provider ID is permanent because requests, saved sessions, model defaults, a
|
||||
|
||||
Under **Model catalog**, choose **Fetch available models** to query the base URL and credential currently shown in the form. Selecting candidates updates the draft; the provider is not stored until you save. Catalog providers use their installed catalog without a network request.
|
||||
|
||||
### Image input
|
||||
|
||||
A model you enter by hand is treated as text-only until it says otherwise, because nothing can ask an endpoint which modalities it accepts. Attaching an image to such a model is refused before it is sent, naming the model.
|
||||
|
||||
A vision model on a custom provider therefore needs one line. The form has no field for it; add `input` to the model in `$DSH_HOME/settings.yaml`:
|
||||
|
||||
```yaml
|
||||
llm-pi-ai:
|
||||
providers:
|
||||
my-gateway:
|
||||
apiKeyEnv: GATEWAY_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: https://gateway.example/v1
|
||||
models:
|
||||
- id: legacy-chat
|
||||
- id: vision-preview
|
||||
input: [text, image]
|
||||
```
|
||||
|
||||
`input` accepts `text` and `image`, and applies to that model alone, so one route can serve both kinds. Omitting it — or writing an empty list, which means the same thing — keeps whatever the installed catalog records for that model, and falls back to the route's `defaultInput` for a model the catalog does not describe.
|
||||
|
||||
If every model you entered by hand takes images, set the fallback once on the route instead of on each of them:
|
||||
|
||||
```yaml
|
||||
llm-pi-ai:
|
||||
providers:
|
||||
vision-gateway:
|
||||
apiKeyEnv: GATEWAY_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: https://vision.example/v1
|
||||
defaultInput: [text, image]
|
||||
models:
|
||||
- id: first-model
|
||||
- id: second-model
|
||||
```
|
||||
|
||||
`defaultInput` is a fallback, not an override, and defaults to `[text]`: on a catalog provider it answers only for models the catalog does not describe, so it never removes images from a catalog model that has them. Narrow one of those with that model's own `input`. A catalog provider has no `models` list to put it in, so write it under `modelOverrides`, keyed by model id:
|
||||
|
||||
```yaml
|
||||
llm-pi-ai:
|
||||
providers:
|
||||
anthropic:
|
||||
modelOverrides:
|
||||
claude-sonnet-4-5:
|
||||
input: [text]
|
||||
```
|
||||
|
||||
Every list must name at least one modality except a model's own, where an empty list means the same as omitting it. An unknown modality is refused wherever it is written.
|
||||
|
||||
Both fields state a claim about your endpoint rather than checking it. A model that declares images its endpoint does not serve is not caught here; the provider rejects the request instead.
|
||||
|
||||
## Select a model
|
||||
|
||||
Configured providers appear in the model picker. Selecting a model also makes it the default for new sessions. A session that has already sent a request retains the model recorded in its own log.
|
||||
@@ -39,6 +90,8 @@ If a saved default names a provider that was deleted, the composer displays **Se
|
||||
- **`MISSING_CREDENTIAL`** — Store the provider key through the Models page or supply the referenced environment variable.
|
||||
- **`UNKNOWN_MODEL`** — Select a configured model or add the missing model to the custom provider.
|
||||
- **Fetching available models returns 401** — Check the key. Model discovery calls the OpenAI-compatible `GET /models` endpoint; enter models manually for endpoints that do not provide it.
|
||||
- **An image is refused before sending** — The model declares no image modality. Give a custom provider's model `input: [text, image]`; DeepSeek's own chat-completions route is text-only and cannot be configured otherwise.
|
||||
- **The provider rejects a request carrying an image** — The model declares images its endpoint does not actually serve. Remove `image` from whichever list granted it — the model's `input`, or the route's `defaultInput` — then start a new session: the attached image stays in the session log, so the same request repeats until the session moves off it.
|
||||
|
||||
## Advanced configuration
|
||||
|
||||
|
||||
@@ -28,6 +28,57 @@ Provider ID 是永久的，因为请求、已保存会话、模型默认值和
|
||||
|
||||
在**模型目录**中选择**获取可用模型**，可查询表单当前显示的基础 URL 和凭据。选择候选项只会更新草稿；保存前不会存储提供方。目录提供方使用已安装目录，不发起网络请求。
|
||||
|
||||
### 图片输入
|
||||
|
||||
手动输入的模型在自己声明之前一律按纯文本对待，因为没有任何环节能去询问端点接受哪些模态。给这类模型附加图片，会在发送前就被拒绝，并点名该模型。
|
||||
|
||||
因此自定义提供方下的视觉模型需要加一行。表单没有对应字段；请在 `$DSH_HOME/settings.yaml` 中给该模型加上 `input`：
|
||||
|
||||
```yaml
|
||||
llm-pi-ai:
|
||||
providers:
|
||||
my-gateway:
|
||||
apiKeyEnv: GATEWAY_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: https://gateway.example/v1
|
||||
models:
|
||||
- id: legacy-chat
|
||||
- id: vision-preview
|
||||
input: [text, image]
|
||||
```
|
||||
|
||||
`input` 接受 `text` 和 `image`，且只作用于该模型，因此一条路由可以同时服务两类模型。省略它——或写成空列表，两者同义——则保留已安装目录为该模型记录的模态；目录未描述的模型则回退到该路由的 `defaultInput`。
|
||||
|
||||
如果你手动录入的模型全都接受图片，可以在路由上设置一次回退值，不必逐个模型写：
|
||||
|
||||
```yaml
|
||||
llm-pi-ai:
|
||||
providers:
|
||||
vision-gateway:
|
||||
apiKeyEnv: GATEWAY_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: https://vision.example/v1
|
||||
defaultInput: [text, image]
|
||||
models:
|
||||
- id: first-model
|
||||
- id: second-model
|
||||
```
|
||||
|
||||
`defaultInput` 是回退值而不是覆盖值，默认为 `[text]`：在目录提供方上，它只为目录未描述的模型作答，因此绝不会把目录中本就具备图片能力的模型的该能力去掉。要收窄这类模型，请用它自己的 `input`。目录提供方没有可供填写的 `models` 列表，因此写在 `modelOverrides` 下，以模型 id 为键：
|
||||
|
||||
```yaml
|
||||
llm-pi-ai:
|
||||
providers:
|
||||
anthropic:
|
||||
modelOverrides:
|
||||
claude-sonnet-4-5:
|
||||
input: [text]
|
||||
```
|
||||
|
||||
除模型自身的列表外，每个列表都至少要写一项模态；模型自身的空列表与省略它同义。未知模态在任何位置写入都会被拒绝。
|
||||
|
||||
这两个字段都是对你端点的断言，而不是对它的检查。声明了端点并不提供的图片能力的模型不会在这里被拦下，改由提供方拒绝该请求。
|
||||
|
||||
## 选择模型
|
||||
|
||||
已配置的提供方会出现在模型选择器中。选择模型也会将其设为新会话的默认值。已发送过请求的会话会保留自身日志中记录的模型。
|
||||
@@ -39,6 +90,8 @@ Provider ID 是永久的，因为请求、已保存会话、模型默认值和
|
||||
- **`MISSING_CREDENTIAL`**：通过模型页存储提供方密钥，或提供被引用的环境变量。
|
||||
- **`UNKNOWN_MODEL`**：选择已配置的模型，或向自定义提供方添加缺失的模型。
|
||||
- **获取可用模型返回 401**：检查密钥。模型发现会调用 OpenAI 兼容的 `GET /models` 端点；对于不提供该端点的服务，请手动输入模型。
|
||||
- **图片在发送前被拒绝**：该模型未声明图片模态。请给自定义提供方的模型加上 `input: [text, image]`；DeepSeek 自身的 chat-completions 路由是纯文本的，且无法通过配置改变。
|
||||
- **提供方拒绝了带图片的请求**：该模型声明了其端点实际并不提供的图片能力。请从授予它图片能力的那个列表中移除 `image`——可能是模型的 `input`，也可能是路由的 `defaultInput`——然后开启新会话：附加的图片会留在会话日志里，因此在会话离开它之前，同一个请求会不断重复。
|
||||
|
||||
## 进阶配置
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: abf1924a1d1297a8fb7300710223886ccf8bc931
|
||||
README.zh.md: 6e6b8e755df56e69f9615fa40212ce93b5cc1411
|
||||
README.md: c68db532e94c1b0e956d933fbdcf51e5bb63dd29
|
||||
README.zh.md: a621fa67c58c6b185e1330668cb8fac05f17d157
|
||||
|
||||
@@ -91,6 +91,10 @@ How a thinking level travels — `reasoning_effort` alone, DeepSeek's `thinking:
|
||||
|
||||
A model neither the entry nor the installed catalog sizes takes the route's `defaultContextWindow` (262,144) and `defaultMaxTokens` (32,768), so a listing that discloses nothing but ids still yields a serviceable route. Both fallbacks are guesses by construction, which is why they are route fields a deployment whose gateway serves smaller models corrects once rather than constants buried in the adapter; the fallback sizes the model and never becomes a per-request cap.
|
||||
|
||||
Request modalities resolve entry `input` → installed catalog entry → route `defaultInput` (default `[text]`), the same order and the same fallback role the capacities above use. So a catalog model keeps the modalities the catalog records for it, and a narrower route default never strips them; a gateway whose *undescribed* models all take images declares `[text, image]` once at the route instead of on every entry. An entry's empty list means the same as an absent one — it describes a model accepting nothing, so it states no answer and resolution continues past it, which is what keeps a catalog model's own modalities when a `models` entry names it without declaring any. The route's may not be empty, since nothing sits below it to answer instead.
|
||||
|
||||
`[text]` is the absence of a declaration rather than a guess at the endpoint, which is why the fallback here is conservative where the capacity fallbacks are merely plausible. Nothing interrogates a gateway for what it accepts, and the two wrong answers do not cost the same: the harness refuses an image before it is attached when a model's modalities omit one, so under-claiming costs a refusal naming the model, while over-claiming admits an image the provider then rejects mid-turn — after the message is durable, which leaves the session repeating a request that cannot succeed.
|
||||
|
||||
Resolution still fails loud, naming the offending route and model, when a route cannot be served at all: a route the catalog does not ship needs `api`, `baseURL`, and a non-empty `models` list of uniquely-identified models. That resolution runs inside the section schema, so an unserviceable profile is refused **where it is written** — `settings.mutate` answers `settings-rejected` naming the route and model — rather than being stored and then quietly disabling every route in the namespace. The settings seam keeps a namespace's last good value for an already-stored section that fails, so this cannot strand a deployment. `api` accepts the protocols in `supportedProtocols()` and is only needed when the catalog cannot supply one: a model absent from the catalog inherits the protocol its shipped siblings agree on, so adding a model to a single-protocol catalog route restates nothing.
|
||||
|
||||
`baseURL` sets the endpoint of every model on the route, so private proxies such as `https://proxy.example.com:8443` remain supported; a catalog route that omits it keeps each catalog model's own endpoint. Naming `api` on a catalog route repoints the whole route at that protocol, which is how a deployment moves a provider between, say, Responses and Chat Completions.
|
||||
@@ -109,7 +113,7 @@ A model that carries reasoning metadata — from the installed catalog or from i
|
||||
|
||||
A model **without** that metadata — a hand-declared one whose entry declares no `reasoningEfforts`, and a catalog model pi-ai marks as non-reasoning — exposes no `reasoning` at all. pi-ai reports such a model as supporting the single level `off`, but `off` is translated to *omitting* the reasoning option, which is byte-for-byte the request that naming no effort already produces: selecting it could not disable anything, so a provider whose own default is to think would keep thinking with `off` shown as selected. Reporting the capability as unavailable leaves a surface offering the provider's default and nothing that misrepresents it. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and a level absent from the exact model capability fails the REQUEST with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. Describing a model never fails that way: the models under one provider disagree about which levels they accept, so `resolveModel` reports a profile level the exact model cannot take as no default at all rather than throwing. A throw there would take the whole provider out of every model catalog built over it — one mis-set profile field hiding even the models that do support the level — so a bad configuration surfaces where it is acted on, not where it is described. pi-ai's common stream options represent `off` by omitting `reasoning`.
|
||||
|
||||
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
|
||||
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
|
||||
|
||||
@@ -190,6 +194,7 @@ Recorded response content appends to the next request and does not invalidate it
|
||||
- **`headers` can carry a credential the redactor never sees** — the profile's `headers` dict is plain strings, so `Authorization` or `api-key` set there is returned verbatim by a redacted `describe()` and rendered by any configuration UI. Store credentials as `apiKeyEnv` references; making the dict write-only is deferred with the rest of the [wire-boundary work](../llm/README.md#known-limitations-and-deferred-work).
|
||||
- **A route's catalog never refreshes itself** — the catalog is whatever `settings.yaml` says, so a model list is only as current as its last edit. Nothing here queries a provider for the models it serves; a route gains a model when someone writes one.
|
||||
- **One wire protocol per route** — `api` applies to the whole route, so a mixed-protocol catalog route (an OpenAI-style catalog spanning Responses and Chat Completions) cannot host a model of the other protocol, and adding a model such a route does not describe requires naming `api` and moving every model onto it. Splitting the provider across two route keys is the workaround.
|
||||
- **A modality declaration is not verified, and over-claiming outlives the turn** — nothing interrogates an endpoint for what it accepts, so a model declaring `image` its gateway does not serve is refused by the provider mid-turn rather than here. Prompt admission commits the user message durably before the request is built, so the rejected image stays in the session log: that model keeps re-sending it, and model selection refuses a switch to any text-only model. Recovery is another image-capable model, a fork before the image, or a new session; rolling an unconsumed image message back out of the log on a failed send is deferred.
|
||||
- **An unauthenticated route depends on its protocol** — naming no credential resolves the route as configured-but-keyless, but pi-ai's OpenAI-compatible implementation still requires an API key or an `Authorization` header, so a keyless local server needs a placeholder credential referenced by `apiKeyEnv` or an `Authorization` entry in `headers`.
|
||||
- **`GenerateOptions.stop` is unsupported** — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers, so the adapter rejects the field.
|
||||
- **In-history `system` messages use pi-ai's common context conversion** — provider-specific placement follows pi-ai rather than a harness-owned wire override.
|
||||
|
||||
@@ -91,7 +91,12 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog，而不是扩
|
||||
|
||||
条目与已安装 catalog 都没有给出尺寸的模型，会采用该路由的 `defaultContextWindow`（262,144）与 `defaultMaxTokens`（32,768），因此一份只公布 id 的列表同样能产出可服务的路由。两个回退值本质上都是猜测，这正是它们作为路由字段、供网关服务更小模型的部署一次性更正的原因，而不是埋在适配器里的常量；回退值只用于给模型定尺寸，绝不会变成单次请求上限。
|
||||
|
||||
路由完全无法服务时解析仍会明确失败，并点名出问题的路由与模型：catalog 未提供的路由需要 `api`、`baseURL`，以及一个由具有唯一标识的模型组成的非空 `models` 列表。该解析在分节 schema 内部运行，因此无法服务的 profile 会在**写入之处**被拒绝——`settings.mutate` 以 `settings-rejected` 点名路由与模型——而不是先存下来、再悄悄让该 namespace 下每条路由失效。对于已经存下的、在此失败的分节，settings seam 会保留该 namespace 上一份可用值，因此这不会把部署卡死。`api` 接受 `supportedProtocols()` 中的协议，且仅在 catalog 无法提供协议时才需要：catalog 中不存在的模型会继承其同门模型一致同意的协议，因此向单协议 catalog 路由添加模型无需重述任何内容。
|
||||
请求模态的解析顺序是：条目的 `input` → 已安装 catalog 条目 → 路由的 `defaultInput`（默认 `[text]`），与上面两个容量字段的顺序和「回退值」定位完全一致。因此 catalog 模型保留 catalog 为它记录的模态，更窄的路由默认值也绝不会把它剥掉；而**未被 catalog 描述的**模型全都接受图片的网关，只需在路由上写一次 `[text, image]`，不必逐条目写。条目的空列表与缺省同义——它描述的是一个什么都不接受的模型，因此不作答，解析继续往下走——这正是当 `models` 条目点到某个 catalog 模型却不声明模态时，该模型仍保留 catalog 自有模态的原因。路由的那个则不得为空，因为它下面已经没有可以代为作答的层级。
|
||||
|
||||
`[text]` 是「尚未声明」，而不是对端点的猜测——这也是为什么这里的回退值取保守值，而两个容量回退值只是取一个说得过去的值。这里没有任何环节会去询问网关实际接受什么，而两种猜错的代价并不对等：模态中不含图片时，Harness 会在图片被附加之前就拒绝，因此少声明的代价是一次点名该模型的拒绝；而多声明会接纳一张图片、再由提供方在轮次中途拒绝——此时消息已经持久化，会话便会不断重复一个不可能成功的请求。
|
||||
|
||||
路由完全无法服务时解析仍会失败得响亮，并点名出问题的路由与模型：catalog 未提供的路由需要 `api`、`baseURL`，以及一个由唯一标识的模型组成的非空 `models` 列表。该解析在分节 schema 内部运行，因此无法服务的 profile 会在**写入之处**被拒绝——`settings.mutate` 以 `settings-rejected` 点名路由与模型——而不是先存下来、再悄悄让该 namespace 下每条路由失效。对于已经存下的、在此失败的分节，settings seam 会保留该 namespace 上一份可用值，因此这不会把部署卡死。`api` 接受 `supportedProtocols()` 中的协议，且仅在 catalog 无法提供协议时才需要：catalog 中不存在的模型会继承其同门模型一致同意的协议，因此向单协议 catalog 路由添加模型无需重述任何内容。
|
||||
|
||||
|
||||
`baseURL` 设定该路由下每个模型的端点，因此仍支持 `https://proxy.example.com:8443` 等私有 proxy；省略它的 catalog 路由会保留每个 catalog 模型自己的端点。在 catalog 路由上点名 `api` 会把整条路由改指到该协议，这正是部署把某个提供方在 Responses 与 Chat Completions 之间迁移的方式。
|
||||
|
||||
@@ -109,7 +114,7 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog，而不是扩
|
||||
|
||||
**没有**这份元数据的模型——条目未声明 `reasoningEfforts` 的手工声明模型，以及 pi-ai 标记为不具备推理能力的 catalog 模型——完全不公开 `reasoning`。pi-ai 会把这类模型报告为只支持 `off` 一档，但 `off` 会被翻译成*省略* reasoning 选项，而那与「不点名任何档位」产出的请求逐字节相同：选它关不掉任何东西，于是自身默认就在思考的提供方，会在界面显示 `off` 被选中的同时继续思考。把该能力报告为不可用，界面就只剩提供方默认这一项，不会再出现自相矛盾的控件。配置 profile 的 `reasoning` 值（包括 `off`）在存在时是部署默认值；省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先；未出现在确切模型能力中的档位会让**请求**在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败，而不会被自动调整。**描述**一个模型则从不这样失败：同一提供方下各模型接受的档位并不一致，因此 `resolveModel` 对该模型拿不下的 profile 档位报告为「没有默认值」，而不是抛错。在那里抛错会让整个提供方从任何基于它构建的模型目录中消失——一个配错的 profile 字段连支持该档位的模型也一并藏起来——所以坏配置暴露在被执行处，而不是被描述处。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`。
|
||||
|
||||
受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获；省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟，默认为五分钟，且只覆盖未完成提供方读取，不包括消费方思考时间。若已配置标头中有同名项，则以 Harness 应用归因为准。
|
||||
受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`defaultInput`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获；省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟，默认为五分钟，且只覆盖未完成提供方读取，不包括消费方思考时间。若已配置标头中有同名项，则以 Harness 应用归因为准。
|
||||
|
||||
适配器强制 pi-ai SDK `maxRetries` 为零，因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败，而不是静默倍增或隐藏单独组合的 agent（智能体）级重试预算。空闲超时会 abort SDK 的稳定请求信号，并以 `TIMEOUT` 呈现；较早的调用方 abort 仍为 `ABORTED`。
|
||||
|
||||
@@ -190,6 +195,7 @@ pi-ai 事件会变为 harness 推理、文本、工具调用、usage 与 finish
|
||||
- **`headers` 可能承载一条脱敏器看不见的凭据**：profile 的 `headers` 是纯字符串字典，因此设在其中的 `Authorization` 或 `api-key` 会被脱敏后的 `describe()` 原样返回，并被任何配置 UI 渲染出来。请把凭据存为 `apiKeyEnv` 引用；把该字典整体改为只写与其余[协议边界工作](../llm/README.md#known-limitations-and-deferred-work)一并暂缓。
|
||||
- **路由的 catalog 不会自我刷新**：catalog 就是 `settings.yaml` 所写的内容，因此模型列表的新鲜度只到最近一次编辑为止。这里没有任何环节会去问提供方它服务哪些模型；路由要多一个模型，得有人写进去。
|
||||
- **每条路由只有一种协议格式**：`api` 作用于整条路由，因此混合协议的 catalog 路由（跨 Responses 与 Chat Completions 的 OpenAI 式 catalog）无法承载另一种协议的模型，向这类路由添加它未描述的模型必须点名 `api` 并把全部模型一起迁过去。把该提供方拆成两个路由键是变通办法。
|
||||
- **模态声明不经验证，且多声明的后果超出本轮**：没有任何环节会去询问端点接受什么，因此声明了网关并不提供的 `image` 的模型不会在这里被拦下，而是由提供方在轮次中途拒绝。prompt 准入在构造请求之前就把用户消息持久化提交，于是被拒绝的图片留在会话日志里：该模型会不断重发它，而模型选择拒绝切换到任何纯文本模型。恢复途径是换一个确实支持图片的模型、fork 到图片之前，或开启新会话；发送失败时把尚未消费的图片消息从日志中回滚出去这件事已暂缓。
|
||||
- **未认证路由取决于其协议**：不点名凭据会让路由解析为「已配置但无密钥」，但 pi-ai 的 OpenAI 兼容实现仍要求 API key 或 `Authorization` 标头，因此无鉴权的本地服务需要一个由 `apiKeyEnv` 引用的占位凭据，或在 `headers` 中给出 `Authorization` 条目。
|
||||
- **不支持 `GenerateOptions.stop`**：pi-ai 的通用流选项无法保证所有提供方都支持 stop sequence，因此适配器会拒绝该字段。
|
||||
- **历史中的 `system` 消息使用 pi-ai 通用上下文转换**：提供方特定位置由 pi-ai 决定，而非由 harness 拥有的协议覆盖决定。
|
||||
|
||||
@@ -31,12 +31,34 @@ import type {
|
||||
*/
|
||||
const NO_COST: ModelCost = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }
|
||||
|
||||
/** One request modality a pi-ai model may accept. */
|
||||
export type PiAiModality = Model<Api>['input'][number]
|
||||
|
||||
/**
|
||||
* Input modalities for a model the installed catalog does not describe. The
|
||||
* request converter keeps only text blocks, so text is the adapter's actual
|
||||
* capability rather than a deployment choice.
|
||||
* Every pi-ai request modality. The `Record` key type is a drift gate: a pi-ai
|
||||
* upgrade that adds or removes a modality fails compilation here naming the
|
||||
* drifted key, instead of silently narrowing what a profile may declare.
|
||||
*/
|
||||
const TEXT_ONLY: Model<Api>['input'] = ['text']
|
||||
const MODALITY_GATE: Record<PiAiModality, true> = {
|
||||
text: true,
|
||||
image: true,
|
||||
}
|
||||
|
||||
/** Every request modality a profile may declare. */
|
||||
export const MODALITIES = Object.keys(MODALITY_GATE) as readonly PiAiModality[]
|
||||
|
||||
/**
|
||||
* One entry's modality list, or `undefined` when it states no answer. Absent
|
||||
* and empty mean the same thing — `[]` describes a model that accepts nothing
|
||||
* and could serve no request — which is what makes an entry naming a catalog
|
||||
* model without declaring modalities keep the catalog's, since the config
|
||||
* schema materializes `[]` for an absent array.
|
||||
* @param configured - the list a `models` or `modelOverrides` entry supplied.
|
||||
* @returns the declared modalities, or `undefined` to ask the next level.
|
||||
*/
|
||||
function declaredInput(configured: readonly PiAiModality[] | undefined): Model<Api>['input'] | undefined {
|
||||
return configured === undefined || configured.length === 0 ? undefined : [...configured]
|
||||
}
|
||||
|
||||
/**
|
||||
* Every pi-ai thinking level, in pi-ai's canonical escalation order. The
|
||||
@@ -171,6 +193,18 @@ export interface PiAiModelProfile {
|
||||
* default on its own.
|
||||
*/
|
||||
maxTokens?: number
|
||||
/**
|
||||
* Request modalities this model accepts. Absent — or empty, which describes
|
||||
* a model that accepts nothing and so states no answer either — keeps the
|
||||
* installed catalog entry's modalities, then the route's `defaultInput`.
|
||||
* Declaring images is what makes a hand-declared vision model usable, and
|
||||
* declaring text alone corrects a catalog model whose gateway does not serve
|
||||
* what the catalog records. This is a claim about the endpoint, not a check
|
||||
* of it: nothing interrogates a gateway for what it accepts, so a model
|
||||
* claiming images its endpoint refuses is refused by the provider instead,
|
||||
* mid-turn.
|
||||
*/
|
||||
input?: PiAiModality[]
|
||||
/**
|
||||
* Selectable reasoning efforts. Absent inherits the installed catalog
|
||||
* entry's capability (a hand-declared model has none and does not reason);
|
||||
@@ -210,6 +244,8 @@ export interface RouteCatalogRequest {
|
||||
defaultContextWindow: number
|
||||
/** Output capability for a model neither the entry nor the catalog sizes. */
|
||||
defaultMaxTokens: number
|
||||
/** Modalities for a model neither the entry nor the catalog declares. */
|
||||
defaultInput: Model<Api>['input']
|
||||
}
|
||||
|
||||
/** Report a route the deployment cannot serve, naming the settings key at fault. */
|
||||
@@ -474,7 +510,7 @@ export function resolveRouteModels(request: RouteCatalogRequest): RouteCatalog {
|
||||
api,
|
||||
provider,
|
||||
baseUrl,
|
||||
input: base?.input ?? TEXT_ONLY,
|
||||
input: declaredInput(entry.input) ?? base?.input ?? [...request.defaultInput],
|
||||
cost: base?.cost ?? NO_COST,
|
||||
contextWindow,
|
||||
maxTokens,
|
||||
|
||||
@@ -21,8 +21,14 @@ import type { CredentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm'
|
||||
import type { ResolvedRetryPolicy, RetryPolicyConfig } from '@deepseek-ai/dsh-llm'
|
||||
import { resolveRouteModels, SUPPORTED_THINKING_FORMATS, THINKING_LEVELS } from './catalog.ts'
|
||||
import type { PiAiCompatProfile, PiAiModelOverride, PiAiModelProfile, PiAiReasoningEfforts } from './catalog.ts'
|
||||
import { MODALITIES, resolveRouteModels, SUPPORTED_THINKING_FORMATS, THINKING_LEVELS } from './catalog.ts'
|
||||
import type {
|
||||
PiAiCompatProfile,
|
||||
PiAiModality,
|
||||
PiAiModelOverride,
|
||||
PiAiModelProfile,
|
||||
PiAiReasoningEfforts,
|
||||
} from './catalog.ts'
|
||||
import { buildProvider, supportedProtocols } from './provider.ts'
|
||||
|
||||
/** Default maximum idle interval while an adapter stream read is outstanding. */
|
||||
@@ -34,8 +40,21 @@ export const DEFAULT_CONTEXT_WINDOW = 262_144
|
||||
/** Output capability assumed for a model neither configuration nor the catalog sizes. */
|
||||
export const DEFAULT_MAX_TOKENS = 32_768
|
||||
|
||||
/**
|
||||
* Modalities assumed for a model neither configuration nor the catalog
|
||||
* declares. Text is the floor every supported protocol certainly carries, so
|
||||
* this is the absence of a declaration rather than a guess at the endpoint:
|
||||
* nothing can interrogate a gateway for its modalities, and the two wrong
|
||||
* answers do not cost the same. Under-claiming refuses the image before it is
|
||||
* attached, naming the model. Over-claiming admits one the provider then
|
||||
* rejects mid-turn, after the message is durable, leaving the session
|
||||
* repeating a request that cannot succeed.
|
||||
*/
|
||||
export const DEFAULT_INPUT: readonly PiAiModality[] = ['text']
|
||||
|
||||
export type {
|
||||
PiAiCompatProfile,
|
||||
PiAiModality,
|
||||
PiAiModelOverride,
|
||||
PiAiModelProfile,
|
||||
PiAiReasoningEfforts,
|
||||
@@ -90,6 +109,17 @@ export interface PiAiProviderProfile {
|
||||
* never becomes a per-request cap on its own.
|
||||
*/
|
||||
defaultMaxTokens?: number
|
||||
/**
|
||||
* Request modalities for a model this route lists that neither its entry's
|
||||
* {@link PiAiModelProfile.input} nor the installed catalog declares (default
|
||||
* `[text]`). A fallback like the capacities above, not an override: a
|
||||
* catalog model keeps the modalities the catalog records for it, and this
|
||||
* value never narrows one. A gateway serving vision models the catalog does
|
||||
* not describe declares `[text, image]` once here instead of on every entry.
|
||||
* Unlike an entry's list, this one may not be empty — nothing sits below it
|
||||
* to answer instead.
|
||||
*/
|
||||
defaultInput?: PiAiModality[]
|
||||
/** Provider request headers; Harness attribution wins reserved names. */
|
||||
headers?: Record<string, string>
|
||||
/** Provider-neutral pi-ai reasoning level. */
|
||||
@@ -180,6 +210,10 @@ const modelFields = {
|
||||
name: z.string(),
|
||||
contextWindow: z.number().step(1).min(1),
|
||||
maxTokens: z.number().step(1).min(1),
|
||||
// No explicit default, unlike the route's `defaultInput`: schemastery
|
||||
// materializes `[]` for an absent array, and resolution reads that as "no
|
||||
// answer here" so the catalog entry below still applies.
|
||||
input: z.array(z.union(MODALITIES)),
|
||||
// The union, not a bare dict: schemastery materializes an absent dict as
|
||||
// `{}`, and absent must stay distinguishable — it means "inherit the
|
||||
// installed catalog's capability", while `false` disables reasoning.
|
||||
@@ -205,6 +239,7 @@ const profile = z.object({
|
||||
compat: compatProfile,
|
||||
defaultContextWindow: z.number().step(1).min(1).default(DEFAULT_CONTEXT_WINDOW),
|
||||
defaultMaxTokens: z.number().step(1).min(1).default(DEFAULT_MAX_TOKENS),
|
||||
defaultInput: z.array(z.union(MODALITIES)).default([...DEFAULT_INPUT]),
|
||||
headers: z.dict(z.string()),
|
||||
reasoning: z.union(THINKING_LEVELS),
|
||||
thinkingBudgets,
|
||||
@@ -288,6 +323,15 @@ export function resolveProfiles(
|
||||
`llm-pi-ai: provider "${provider}" streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`,
|
||||
)
|
||||
}
|
||||
// Detached from the configuration object because pi-ai types `Model.input`
|
||||
// mutable. The schema's explicit default covers an absent key, so an empty
|
||||
// list here is always one someone typed — and unlike an entry's, nothing
|
||||
// below it can answer instead — so it is refused rather than read as "no
|
||||
// answer".
|
||||
const defaultInput = [...source.defaultInput ?? DEFAULT_INPUT]
|
||||
if (defaultInput.length === 0) {
|
||||
throw new Error(`llm-pi-ai: provider "${provider}" defaultInput must name at least one modality`)
|
||||
}
|
||||
// The route key, not the installed provider's own name: the directory has
|
||||
// always shown route keys, and a catalog route must not silently rename
|
||||
// itself on every configuration surface just because it gained a profile.
|
||||
@@ -299,6 +343,7 @@ export function resolveProfiles(
|
||||
...source.models === undefined ? {} : { models: source.models },
|
||||
...source.modelOverrides === undefined ? {} : { modelOverrides: source.modelOverrides },
|
||||
...source.compat === undefined ? {} : { compat: source.compat },
|
||||
defaultInput,
|
||||
defaultContextWindow: source.defaultContextWindow ?? DEFAULT_CONTEXT_WINDOW,
|
||||
defaultMaxTokens: source.defaultMaxTokens ?? DEFAULT_MAX_TOKENS,
|
||||
})
|
||||
|
||||
@@ -71,6 +71,7 @@ export type { PiAiAdapterOptions } from './adapter.ts'
|
||||
export { Config } from './config.ts'
|
||||
export type {
|
||||
PiAiCompatProfile,
|
||||
PiAiModality,
|
||||
PiAiModelOverride,
|
||||
PiAiModelProfile,
|
||||
PiAiProviderProfile,
|
||||
|
||||
@@ -186,6 +186,108 @@ describe('hand-declared providers', () => {
|
||||
expect(resolved.get('acme-gateway')?.configuredMaxTokens.get('sized')).toBe(512)
|
||||
})
|
||||
|
||||
it('takes a model’s declared modalities, then the catalog’s, then the route’s', () => {
|
||||
const vision = getBuiltinModels('anthropic').find(model => model.input.includes('image'))
|
||||
if (vision === undefined) throw new Error('the installed catalog ships no anthropic vision model')
|
||||
const resolved = resolveProfiles({
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
// One route, two modality sets: the entry field is what says so.
|
||||
models: [{ id: 'bare' }, { id: 'seeing', input: ['text', 'image'] }, { id: 'deaf', input: ['text'] }],
|
||||
},
|
||||
'seeing-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://seeing.test',
|
||||
// A gateway whose undescribed models all take images says so once
|
||||
// rather than on every entry; an entry still outranks it.
|
||||
defaultInput: ['text', 'image'],
|
||||
models: [{ id: 'bare' }, { id: 'deaf', input: ['text'] }],
|
||||
},
|
||||
// The route value is a fallback, never an override: a catalog model
|
||||
// keeps what the catalog records even under a narrower route default,
|
||||
// exactly as it keeps its own contextWindow.
|
||||
'anthropic': { defaultInput: ['text'] },
|
||||
})
|
||||
const inputOf = (route: string, id: string): readonly string[] | undefined =>
|
||||
resolved.get(route)?.piProvider.getModels().find(model => model.id === id)?.input
|
||||
|
||||
expect(inputOf('acme-gateway', 'bare')).toEqual(['text'])
|
||||
expect(inputOf('acme-gateway', 'seeing')).toEqual(['text', 'image'])
|
||||
expect(inputOf('acme-gateway', 'deaf')).toEqual(['text'])
|
||||
expect(inputOf('seeing-gateway', 'bare')).toEqual(['text', 'image'])
|
||||
expect(inputOf('seeing-gateway', 'deaf')).toEqual(['text'])
|
||||
expect(inputOf('anthropic', vision.id)).toEqual(vision.input)
|
||||
})
|
||||
|
||||
it('carries a written modality declaration all the way to the seam’s model metadata', async () => {
|
||||
// The resolver-level cases above cannot see a break between the settings
|
||||
// document and `LlmModelInfo`, so each rung is asserted once more through
|
||||
// a written section, the plugin's own registration, and `ctx.llm`.
|
||||
const dir = await home()
|
||||
const ctx = await bootWithSettings(dir, {})
|
||||
await ctx.settings.update(settingsNamespace('llm-pi-ai'), {
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test/v1',
|
||||
models: [{ id: 'bare' }, { id: 'seeing', input: ['text', 'image'] }],
|
||||
},
|
||||
'vision-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://vision.test/v1',
|
||||
defaultInput: ['text', 'image'],
|
||||
models: [{ id: 'bare' }, { id: 'deaf', input: ['text'] }],
|
||||
},
|
||||
'anthropic': { defaultInput: ['text'] },
|
||||
},
|
||||
})
|
||||
|
||||
const listed = async (provider: string): Promise<Record<string, readonly string[] | undefined>> =>
|
||||
Object.fromEntries((await ctx.llm.listModels(provider)).map(model => [model.id, model.inputModalities]))
|
||||
|
||||
expect(await listed('acme-gateway')).toEqual({ bare: ['text'], seeing: ['text', 'image'] })
|
||||
expect(await listed('vision-gateway')).toEqual({ bare: ['text', 'image'], deaf: ['text'] })
|
||||
expect((await ctx.llm.resolveModelInfo('acme-gateway', 'seeing')).inputModalities).toEqual(['text', 'image'])
|
||||
|
||||
// A catalog vision model keeps what the catalog records even under a
|
||||
// narrower route default: the route value is a fallback, not an override.
|
||||
const vision = getBuiltinModels('anthropic').find(model => model.input.includes('image'))
|
||||
if (vision === undefined) throw new Error('the installed catalog ships no anthropic vision model')
|
||||
expect((await ctx.llm.resolveModelInfo('anthropic', vision.id)).inputModalities).toEqual(vision.input)
|
||||
})
|
||||
|
||||
it('reads an entry’s empty modality list as no answer, and the route’s as unserviceable', () => {
|
||||
// Absent and empty are the same request on an entry, exactly as they are
|
||||
// for the route's `models` list — which matters because the config schema
|
||||
// materializes `[]` for an absent array, so an entry naming a catalog
|
||||
// model without declaring modalities must keep the catalog's rather than
|
||||
// describe a model that accepts nothing.
|
||||
const [catalogModel] = getBuiltinModels('deepseek')
|
||||
if (catalogModel === undefined) throw new Error('the installed catalog ships no deepseek model')
|
||||
const resolved = resolveProfiles({
|
||||
'deepseek': { baseURL: 'https://catalog.test', models: [{ id: catalogModel.id, input: [] }] },
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
models: [{ id: 'bare', input: [] }],
|
||||
},
|
||||
})
|
||||
expect(resolved.get('acme-gateway')?.piProvider.getModels()[0]?.input).toEqual(['text'])
|
||||
expect(resolved.get('deepseek')?.piProvider.getModels()[0]?.input).toEqual(catalogModel.input)
|
||||
|
||||
// Nothing sits below the route value, so its empty list states no answer
|
||||
// anything could take, and is refused where it is written.
|
||||
expect(() => resolveProfiles({
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
defaultInput: [],
|
||||
models: [{ id: 'bare' }],
|
||||
},
|
||||
})).toThrow(/defaultInput must name at least one modality/)
|
||||
})
|
||||
|
||||
it('rejects a model the route cannot identify', () => {
|
||||
const declare = (model: LlmPiAi.PiAiModelProfile): (() => unknown) =>
|
||||
() => resolveProfiles({ 'acme-gateway': { api: 'openai-completions', baseURL: 'https://acme.test', models: [model] } })
|
||||
|
||||
@@ -1,18 +1,24 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Config } from '../src/config.ts'
|
||||
import { assertServiceable, Config } from '../src/config.ts'
|
||||
|
||||
/** Validate one hand-declared route, with the caller's fields layered onto it. */
|
||||
const routeWith = (profile: Record<string, unknown>): (() => unknown) =>
|
||||
() => Config({
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
models: [{ id: 'm' }],
|
||||
...profile,
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
/** Validate that route with the caller's fields on its single model entry. */
|
||||
const configWith = (model: Record<string, unknown>): (() => unknown) =>
|
||||
routeWith({ models: [{ id: 'm', ...model }] })
|
||||
|
||||
describe('reasoning schema boundary', () => {
|
||||
const configWith = (model: Record<string, unknown>): (() => unknown) =>
|
||||
() => Config({
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
models: [{ id: 'm', ...model }],
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
it('rejects a level pi-ai does not know at the write that produced it', () => {
|
||||
expect(configWith({ reasoningEfforts: { ultra: 'x' } })).toThrow(/"off"/)
|
||||
expect(configWith({ reasoningEfforts: { high: 42 } })).toThrow()
|
||||
@@ -30,3 +36,31 @@ describe('reasoning schema boundary', () => {
|
||||
expect(configWith({ compat: { thinkingFormat: 'quantum' } })).toThrow(/expected/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('modality schema boundary', () => {
|
||||
it('rejects a modality pi-ai does not know, at either level', () => {
|
||||
expect(configWith({ input: ['audio'] })).toThrow(/expected/)
|
||||
expect(routeWith({ defaultInput: ['text', 'audio'] })).toThrow(/expected/)
|
||||
})
|
||||
|
||||
it('refuses a route whose models could accept nothing', () => {
|
||||
// The pair the settings seam runs: the schema accepts the empty list as
|
||||
// well-typed, and the namespace validator is what refuses it. Asserting
|
||||
// only the schema would report this route as writable.
|
||||
expect(routeWith({ defaultInput: [] })).not.toThrow()
|
||||
expect(() => { assertServiceable(routeWith({ defaultInput: [] })() as Config) })
|
||||
.toThrow(/defaultInput must name at least one modality/)
|
||||
})
|
||||
|
||||
type Materialized = {
|
||||
providers: Record<string, { defaultInput?: unknown; models?: { input?: unknown }[] }>
|
||||
}
|
||||
|
||||
it('materializes an absent entry list as empty and an absent route list as text', () => {
|
||||
// The empty-list inheritance rule exists because of exactly this: an entry
|
||||
// that declares nothing reaches resolution as `[]`, not as `undefined`.
|
||||
const absent = configWith({})() as Materialized
|
||||
expect(absent.providers['acme-gateway']?.models?.[0]?.input).toEqual([])
|
||||
expect(absent.providers['acme-gateway']?.defaultInput).toEqual(['text'])
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user