fix(llm): let an interrogation use the credential its route already stored

A configuration surface never holds a stored secret — it edits a redacted
descriptor — so once a key is saved, the draft it sends carries the route and
the endpoint and no credential at all. The interrogation went out
unauthenticated and the endpoint's 401 came back as "check the API key",
pointing at the one thing that was fine.

A named route now supplies its own credential, resolved exactly as a request
to it would be. A key typed into the form still wins: it is the one under
test, and may be the replacement for the stored one that is failing.

Resolution is a callback the probe invokes past the catalog short-circuit and
the protocol check, so a route answered from the installed registry costs no
credential lookup — and cannot fail over a credential the question never
needed.
This commit is contained in:
Yichen Jiang
2026-08-05 20:55:39 +08:00
parent b2d0e8972f
commit 66c2cb81d3
9 changed files with 94 additions and 15 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-04-draft-provider-endpoint-interrogation.md
2026-08-04-draft-provider-endpoint-interrogation.md: 49b863a3b923e9cdae34462c63fb2e2ed0968941
2026-08-04-draft-provider-endpoint-interrogation.zh.md: a6a74407b0713ffcadc734ecbca2b7d7363d7361
2026-08-04-draft-provider-endpoint-interrogation.md: 65545098cd1063c40081481c1ac8f0afdb4fb390
2026-08-04-draft-provider-endpoint-interrogation.zh.md: cb09042904f4ab1558c0c214d275a934234955ac

View File

@@ -17,7 +17,7 @@ The awkward part is that the question is about something that does not exist yet
Interrogation is keyed by **settings namespace**, not by provider route:
- `ctx.llm.registerModelDiscovery(settingsNs, discover)` lets an adapter plugin offer to interrogate endpoints for the namespace it owns, and `ctx.llm.discoverModels(settingsNs, request)` asks. There is no way to enumerate which namespaces registered: a surface that cannot interrogate learns it from the refusal, and a list nothing consumed would be a required wire field doing nothing. The namespace is the right key because a configuration surface already holds it from the configurable-provider directory, and because a provider being added has no route to name.
- `LlmModelDiscoveryRequest` carries the draft — an optional `provider`, an optional `baseURL`, an optional `api`, an optional `apiKey`, and a signal — and needs at least one of `provider` or `baseURL` to have anything to answer about. `provider` exists because a route the adapter already describes is answered from its own registry with no network call at all; only a route it does not describe reaches an endpoint. Nothing in this path reads or writes settings or credentials; the caller owns both.
- `LlmModelDiscoveryRequest` carries the draft — an optional `provider`, an optional `baseURL`, an optional `api`, an optional `apiKey`, and a signal — and needs at least one of `provider` or `baseURL` to have anything to answer about. `provider` exists because a route the adapter already describes is answered from its own registry with no network call at all; only a route it does not describe reaches an endpoint. Nothing in this path writes settings or credentials. The one read is the credential of a route the request names: a configuration surface holds a redacted descriptor rather than the stored secret, so the draft's `apiKey` is present only while the user is typing one, and without that read an already-configured route would be interrogated unauthenticated and answer 401. The typed key wins, being the one under test.
- `LlmDiscoveredModel` makes every field but `id` optional, because most listings disclose an id and nothing else. The reply is candidates, not a catalog: a surface adopting one still owes the capacities the adapter requires.
- `llm.discoverModels` carries the same draft over the wire. Its `apiKey` is the third and last payload on which a secret may ride, alongside `settings.update`/`mutate` and `credentials.set`, and it is never stored or echoed back. It does ride the client's outgoing envelope like every other secret-bearing payload, where a `subscribeEnvelopes()` observer can see it; redacting that tap is a configuration-plane-wide change, not this method's to make alone. The method is loopback-only for a second reason besides the key: it makes the host issue a GET to a caller-chosen URL and reports the outcome, which is a probe an anonymous LAN caller must not have. Every refusal folds into `model-discovery-failed`, whose message is the adapter's own text and whose details name the endpoint asked but never the credential offered.
@@ -25,7 +25,7 @@ Interrogation is keyed by **settings namespace**, not by provider route:
### Why not pi-ai's own refresh machinery
pi-ai supplies `createProvider({ fetchModels })` plus `Models.refresh()` and a `ModelsStore`, and the layer below already builds pi-ai `Provider` objects. Routing interrogation through them would have meant constructing a throwaway provider and collection per question, with a store whose entire purpose — persisting a catalog across runs — contradicts the decision that `settings.yaml` owns the catalog. It would also have bought nothing: **no built-in pi-ai provider implements `fetchModels`**, so the HTTP call and its response parsing are this package's code either way. A direct fetch says what is actually happening.
pi-ai supplies `createProvider({ fetchModels })` plus `Models.refresh()` and a `ModelsStore`, and the layer below already builds pi-ai `Provider` objects. Routing interrogation through them would have meant constructing a throwaway provider and collection per question, with a store whose entire purpose — persisting a catalog across runs — contradicts the decision that `settings.yaml` owns the catalog. It would also have bought nothing: **no built-in pi-ai provider implements `fetchModels`**, so the HTTP call and its response parsing are this package's code either way. A direct fetch says what is actually happening. The route's stored credential is resolved by the plugin's own per-request resolver, and only on the branch that reaches the network, so a catalog route answers without touching credentials and never fails over one the question did not need.
## Alternatives considered
@@ -33,7 +33,7 @@ pi-ai supplies `createProvider({ fetchModels })` plus `Models.refresh()` and a `
**Put the capability on `LlmAdapter`.** Adapters are reached through a route registration, so this has the same problem, plus it would make an adapter instance answer questions about endpoints it does not serve.
**Have the host read the stored profile instead of accepting a draft.** No secret would cross the wire for an already-configured provider. But adding a provider would then require saving an unusable configuration first, and a form whose endpoint was edited but not yet saved would silently interrogate the old one. Accepting the draft keeps what the user sees and what is asked identical.
**Have the host read the stored profile instead of accepting a draft.** No secret would cross the wire for an already-configured provider. But adding a provider would then require saving an unusable configuration first, and a form whose endpoint was edited but not yet saved would silently interrogate the old one. Accepting the draft keeps what the user sees and what is asked identical — with the credential as the one exception, because it is the one field a surface is never shown and so can never put in the draft.
**Interrogate every pi-ai protocol.** Anthropic's listing happens to share OpenAI's envelope, and Google's does not. Supporting the ones that are easy would make coverage arbitrary and, worse, make a wrong guess at a response shape indistinguishable from a provider with no models. A protocol that says it cannot be interrogated sends the user to hand-entry, which is the documented fallback.
@@ -47,4 +47,4 @@ What it costs: the wire gained a third secret-carrying payload, so the configura
## Testing
`packages/llm/llm/tests/topology.spec.ts` covers the registry: one offer per namespace, disposal with the fiber, normalization that drops duplicate and unusable ids without inventing capacities, and the `NO_DISCOVERY`/`INVALID_DISCOVERY` refusals. `packages/llm/llm-pi-ai/tests/discovery.spec.ts` drives the probe against local HTTP servers — a listing with and without disclosed capacities, a preserved deployment path, an absent credential, dropped rows, 401/403 versus a server fault, a non-listing and a non-JSON body, an unreachable endpoint, caller cancellation, an unsupported protocol, and the size ceiling in both its declared-length and streamed forms. `packages/host/apiproxy/tests/api-proxy-config.spec.ts` covers the RPC over a real proxy: the draft reaching its namespace whole, absent fields staying absent, no namespace or credential being written, and a failure surfacing as `model-discovery-failed` with the credential absent from the serialized error.
`packages/llm/llm/tests/topology.spec.ts` covers the registry: one offer per namespace, disposal with the fiber, normalization that drops duplicate and unusable ids without inventing capacities, and the `NO_DISCOVERY`/`INVALID_DISCOVERY` refusals. `packages/llm/llm-pi-ai/tests/discovery.spec.ts` drives the probe against local HTTP servers — a listing with and without disclosed capacities, a preserved deployment path, an absent credential, a configured route supplying its own where the draft has none and a typed key winning over it, a catalog route answering without resolving one at all, dropped rows, 401/403 versus a server fault, a non-listing and a non-JSON body, an unreachable endpoint, caller cancellation, an unsupported protocol, and the size ceiling in both its declared-length and streamed forms. `packages/host/apiproxy/tests/api-proxy-config.spec.ts` covers the RPC over a real proxy: the draft reaching its namespace whole, absent fields staying absent, no namespace or credential being written, and a failure surfacing as `model-discovery-failed` with the credential absent from the serialized error.

View File

@@ -17,7 +17,7 @@ Status: implemented
询问以 **settings namespace** 为键,而不是提供方路由:
- `ctx.llm.registerModelDiscovery(settingsNs, discover)` 让适配器插件为自己拥有的 namespace 提供「询问端点」的能力,`ctx.llm.discoverModels(settingsNs, request)` 发起询问。没有任何办法枚举哪些 namespace 注册过:询问不了的界面会从那句拒绝里知道,而一份无人消费的列表只会变成一个什么都不做的必填协议字段。以 namespace 为键是对的,因为配置界面已经从可配置提供方目录里拿到了它,也因为正在新增的提供方没有路由可点名。
- `LlmModelDiscoveryRequest` 携带草稿——可选的 `provider`、可选的 `baseURL`、可选的 `api`、可选的 `apiKey`,以及一个 signal——且 `provider``baseURL` 至少要有一个,才有东西可答。`provider` 之所以存在,是因为适配器已经描述过的路由直接由它自己的注册表作答、完全不联网;只有它未描述的路由才会抵达某个端点。这条路径既不读也不写 settings 与 credentials;两者都归调用方所有
- `LlmModelDiscoveryRequest` 携带草稿——可选的 `provider`、可选的 `baseURL`、可选的 `api`、可选的 `apiKey`,以及一个 signal——且 `provider``baseURL` 至少要有一个,才有东西可答。`provider` 之所以存在,是因为适配器已经描述过的路由直接由它自己的注册表作答、完全不联网;只有它未描述的路由才会抵达某个端点。这条路径不写 settings 与 credentials。唯一的读取是请求所点名路由的凭据:配置界面拿到的是脱敏描述符而非已存的机密,因此草稿里的 `apiKey` 只在用户正键入时才存在;没有这次读取,已配置好的路由就会被不带认证地询问,只换回一个 401。键入的密钥优先因为那正是被测试的那一把
- `LlmDiscoveredModel``id` 外每个字段都可选,因为大多数列表只公布 id。回复是候选而非 catalog采纳其中一条的界面仍要补上适配器所需的容量。
- `llm.discoverModels` 把同一份草稿送过协议层。它的 `apiKey` 是 secret 可以搭乘的第三个、也是最后一个载荷(另两个是 `settings.update`/`mutate``credentials.set`),且绝不被存储或回显。它确实会像其他承载机密的载荷一样随客户端外发信封同行,`subscribeEnvelopes()` 观察者看得到;把那个抽头脱敏是整个配置面的改动,不该由这一个方法独自决定。除密钥之外它被钉在回环还有第二个理由:它让宿主向调用方选定的 URL 发起 GET 并回报结果,这是匿名 LAN 调用者不该拥有的探测能力。每一种拒绝都折叠为 `model-discovery-failed`其消息是适配器自己的文本details 点名被询问的端点,绝不点名所提供的凭据。
@@ -25,7 +25,7 @@ Status: implemented
### 为什么不用 pi-ai 自己的 refresh 机制
pi-ai 提供了 `createProvider({ fetchModels })` 加上 `Models.refresh()``ModelsStore`,而下层本来就在构造 pi-ai `Provider` 对象。把询问接到它们上面,意味着每问一次就要构造一个用完即弃的 provider 与集合,而那个 store 的全部目的——跨运行持久化 catalog——恰恰与「`settings.yaml` 拥有 catalog」的决定相抵触。而且它什么也换不来**没有任何一个 pi-ai 内置 provider 实现了 `fetchModels`**,因此 HTTP 调用及其响应解析无论如何都是本包的代码。直接 fetch 才如实说出正在发生的事。
pi-ai 提供了 `createProvider({ fetchModels })` 加上 `Models.refresh()``ModelsStore`,而下层本来就在构造 pi-ai `Provider` 对象。把询问接到它们上面,意味着每问一次就要构造一个用完即弃的 provider 与集合,而那个 store 的全部目的——跨运行持久化 catalog——恰恰与「`settings.yaml` 拥有 catalog」的决定相抵触。而且它什么也换不来**没有任何一个 pi-ai 内置 provider 实现了 `fetchModels`**,因此 HTTP 调用及其响应解析无论如何都是本包的代码。直接 fetch 才如实说出正在发生的事。路由已存的凭据由本插件自己那套逐请求解析器取出,且只在真正要联网的那条分支上进行,因此 catalog 路由作答时既不触碰凭据,也不会因为一把这次询问根本用不上的密钥而失败。
## Alternatives considered
@@ -33,7 +33,7 @@ pi-ai 提供了 `createProvider({ fetchModels })` 加上 `Models.refresh()` 与
**把能力挂在 `LlmAdapter` 上。** 适配器要经由路由注册才能抵达,因此问题相同;而且这会让一个适配器实例去回答它并不服务的端点的问题。
**让 host 读已存 profile而不是接受草稿。** 对已配置好的提供方来说,不会有 secret 跨越协议层。但这样一来新增提供方就必须先保存一份不可用的配置,而端点已改却尚未保存的表单会静默地去询问旧地址。接受草稿让用户看见的与被询问的保持一致。
**让 host 读已存 profile而不是接受草稿。** 对已配置好的提供方来说,不会有 secret 跨越协议层。但这样一来新增提供方就必须先保存一份不可用的配置,而端点已改却尚未保存的表单会静默地去询问旧地址。接受草稿让用户看见的与被询问的保持一致——凭据是唯一的例外,因为它是界面从不被展示、因而永远无法放进草稿的那个字段
**询问 pi-ai 的每一种协议。** Anthropic 的列表恰好与 OpenAI 共用同一层信封,而 Google 的不是。只支持容易的那几种会让覆盖范围变得任意;更糟的是,猜错的响应形状会与「该提供方没有模型」无法区分。一个明说自己无法被询问的协议,会把用户送去手工填写——那正是既定的回退路径。
@@ -47,4 +47,4 @@ pi-ai 提供了 `createProvider({ fetchModels })` 加上 `Models.refresh()` 与
## Testing
`packages/llm/llm/tests/topology.spec.ts` 覆盖注册表:每个 namespace 一份、随 fiber dispose、丢弃重复与不可用 id 且不凭空补容量的归一化,以及 `NO_DISCOVERY`/`INVALID_DISCOVERY` 两种拒绝。`packages/llm/llm-pi-ai/tests/discovery.spec.ts` 针对本地 HTTP 服务器驱动探测——含与不含公布容量的列表、被保留的部署路径、无凭据、被丢弃的行、401/403 与服务器故障之别、非列表与非 JSON 响应、不可达端点、调用方取消、不支持的协议,以及尺寸上限的「声明长度」与「流式」两种形态。`packages/host/apiproxy/tests/api-proxy-config.spec.ts` 在真实 proxy 上覆盖该 RPC草稿完整抵达其 namespace、缺席字段保持缺席、没有 namespace 或凭据被写入,以及失败以 `model-discovery-failed` 呈现且序列化后的错误里不含凭据。
`packages/llm/llm/tests/topology.spec.ts` 覆盖注册表:每个 namespace 一份、随 fiber dispose、丢弃重复与不可用 id 且不凭空补容量的归一化,以及 `NO_DISCOVERY`/`INVALID_DISCOVERY` 两种拒绝。`packages/llm/llm-pi-ai/tests/discovery.spec.ts` 针对本地 HTTP 服务器驱动探测——含与不含公布容量的列表、被保留的部署路径、无凭据、草稿没带密钥时已配置路由自行取用凭据且键入的密钥压过它、catalog 路由完全不解析凭据即作答、被丢弃的行、401/403 与服务器故障之别、非列表与非 JSON 响应、不可达端点、调用方取消、不支持的协议,以及尺寸上限的「声明长度」与「流式」两种形态。`packages/host/apiproxy/tests/api-proxy-config.spec.ts` 在真实 proxy 上覆盖该 RPC草稿完整抵达其 namespace、缺席字段保持缺席、没有 namespace 或凭据被写入,以及失败以 `model-discovery-failed` 呈现且序列化后的错误里不含凭据。

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
README.md: 7cb575c9ed85a21f8dab7b37d2e76cf74fe5d16f
README.zh.md: a99d70aa7dd157f18332a1fa0e283fc01dd23a5d
README.md: af0e952dd8dbd9767b98229ee6b87262007d6738
README.zh.md: f8a19999f08aa8a6963874d57bf74370797b951c

View File

@@ -85,6 +85,8 @@ The plugin offers `ctx.llm.registerModelDiscovery('llm-pi-ai', …)`, which answ
A request naming a route the **installed catalog ships is answered from that catalog**, with no network call: pi-ai's registry is the authoritative list for its own providers, and it carries the context windows and output caps a listing endpoint would not disclose. Such a route needs no `baseURL` at all. Only a route the catalog does not describe — a gateway, a self-hosted server — is interrogated over the wire, and one that names no endpoint is told to set one or enter its models by hand.
A draft carries the credential the user typed, if any; a route that already stored one shows a configuration surface only a redacted descriptor, so the interrogation supplies that route's own credential — resolved exactly as a request to it would, `apiKey` then `apiKeyEnv` — rather than going out unauthenticated and reporting the endpoint's 401 as a wrong key. A typed key wins, being the one under test. Resolution happens only on the path that reaches the network, so a catalog route answers without touching credentials at all.
Interrogation reads `openai-completions` and `openai-responses`, whose `GET /models` shape with bearer auth is the one a gateway, a self-hosted server, and the official endpoints all agree on. Azure is excluded despite its OpenAI lineage — it authenticates with an `api-key` header and requires an `api-version` query — and Codex uses OAuth; every other protocol answers `DISCOVERY_UNSUPPORTED` so the surface falls back to hand-entry instead of an authentication failure being reported as a provider with no models. The `baseURL` is treated as a prefix rather than a URL to resolve against, so a deployment path such as `https://gateway.example/openai/v1` keeps its segments.
Most listings disclose an id and nothing else; `context_window`/`context_length` and `max_output_tokens`/`max_tokens` are read when a gateway supplies them, entries without a usable id are skipped rather than failing the whole listing, and everything else the adopting surface still owes. The reply is read under a four-megabyte ceiling enforced on the bytes actually received — the endpoint is a URL the user typed, so a declared length is checked first but never trusted as the bound. An unreachable endpoint, a refused credential, a non-JSON body, and a body with no `data` array all fail with `DISCOVERY_FAILED` and a message naming the endpoint and, for a 401 or 403 alone, the credential. Cancellation during the body read surfaces as `ABORTED`, like a cancellation before the request went out.

View File

@@ -85,6 +85,8 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog而不是扩
点名了**已安装 catalog 所提供路由**的请求,直接由该 catalog 作答完全不联网pi-ai 的注册表才是它自家提供方的权威列表,且携带列表端点不会公布的上下文窗口与输出上限。这类路由根本不需要 `baseURL`。只有 catalog 未描述的路由——网关、自建服务——才会经协议层询问;若它也没给端点,则会被告知去设置一个或手工填写模型。
草稿携带的是用户当下键入的凭据(如果有);已经存好凭据的路由,在配置界面上只呈现一个脱敏描述符,因此询问会自行取用该路由的凭据——解析方式与向它发请求时完全一致,先 `apiKey``apiKeyEnv`——而不是不带认证发出去、再把端点的 401 报成密钥不对。键入的密钥优先,因为那正是被测试的那一把。解析只发生在真正要联网的路径上,因此 catalog 路由作答时完全不会触碰凭据。
询问只读 `openai-completions``openai-responses`,它们「`GET /models` + bearer 认证」的形状是网关、自建服务与官方端点三方一致认可的那一种。Azure 尽管出身 OpenAI 也被排除——它用 `api-key` 标头认证并要求 `api-version` 查询参数——Codex 则走 OAuth其余协议一律以 `DISCOVERY_UNSUPPORTED` 回答,让界面回退到手工填写,而不是把认证失败报成一个没有模型的提供方。`baseURL` 按前缀而非待解析 URL 处理,因此 `https://gateway.example/openai/v1` 这类部署路径会保留其路径段。
多数列表只公布 id`context_window`/`context_length``max_output_tokens`/`max_tokens` 在网关提供时会被读取,没有可用 id 的条目会被跳过而不是让整份列表失败,其余仍由采纳方补齐。回复在四兆字节上限下读取,且上限落在实际收到的字节上——端点是用户自己填的 URL因此会先看声明长度但绝不把它当作边界。端点不可达、凭据被拒、响应非 JSON、以及响应没有 `data` 数组,都会以 `DISCOVERY_FAILED` 失败,消息点名端点;仅当 401 或 403 时才点名凭据。读取响应体期间被取消会呈现为 `ABORTED`,与请求发出之前被取消一致。

View File

@@ -164,12 +164,18 @@ function readListing(body: unknown): LlmDiscoveredModel[] {
/**
* Interrogate one draft provider endpoint for the models it advertises.
* @param request - the endpoint, protocol, and one-shot credential to use.
* @param storedApiKey - the credential the named route already stored, asked
* for only when the draft carries none and only on the path that reaches the
* network. A configuration surface never holds a stored secret — it edits a
* redacted descriptor — so without this an already-configured route would be
* interrogated unauthenticated and answer 401.
* @returns the advertised models in endpoint order.
* @throws LlmError when the protocol has no readable listing, the endpoint
* refuses or fails the request, or the reply is not a model listing.
*/
export async function discoverModels(
request: LlmModelDiscoveryRequest,
storedApiKey?: () => Promise<string | undefined>,
): Promise<readonly LlmDiscoveredModel[]> {
// A catalog route already has its answer, and a better one: the installed
// entries carry context windows and output caps no listing endpoint reports.
@@ -205,13 +211,19 @@ export async function discoverModels(
)
}
const url = listingUrl(request.baseURL)
// A key typed into the form wins: it is the one the user is testing, and it
// may be the replacement for exactly the stored key that is failing. The
// stored one is only asked for here, past the catalog short-circuit and the
// protocol check, so a route answered from the registry costs no credential
// lookup — and no diagnostic about a credential it never needed.
const apiKey = request.apiKey ?? await storedApiKey?.()
let response: Response
try {
response = await fetch(url, {
method: 'GET',
headers: {
accept: 'application/json',
...request.apiKey === undefined ? {} : { authorization: `Bearer ${request.apiKey}` },
...apiKey === undefined ? {} : { authorization: `Bearer ${apiKey}` },
...attributionHeaders(),
},
...request.signal === undefined ? {} : { signal: request.signal },

View File

@@ -177,10 +177,26 @@ export function apply(ctx: Context, config: Config): void {
directoryFacts = entries
}
ensureDirectory()
/**
* The credential a named route already resolves, for an interrogation whose
* draft carries none. A route being declared for the first time names no
* profile yet, and a profile that names no credential defers to pi-ai's own
* discovery, so both answer `undefined` and the endpoint is asked
* unauthenticated — the same posture a request to that route would take.
*/
const storedApiKey = async (provider: string | undefined): Promise<string | undefined> => {
if (provider === undefined) return undefined
const profile = profiles().get(provider)
if (profile === undefined) return undefined
return resolveApiKey(provider, profile)
}
// Interrogating an endpoint is a configuration-time action over a draft, so
// it is offered for the whole namespace rather than per route: the provider
// a surface is adding does not exist yet.
ctx.llm.registerModelDiscovery(NS, discoverModels)
// a surface is adding does not exist yet. The draft is the whole request
// except the credential: a configuration surface edits a redacted descriptor
// and never holds a stored secret, so an already-configured route supplies
// its own here rather than being interrogated unauthenticated.
ctx.llm.registerModelDiscovery(NS, request => discoverModels(request, () => storedApiKey(request.provider)))
// Route effects bind to this apply fiber via the stable `ctx` reference,
// even when a swap runs inside the scoped settings callback below. A bare
// mount (zero routes) is the dormant posture: nothing registers until a

View File

@@ -8,8 +8,11 @@ import { getBuiltinModels } from '@earendil-works/pi-ai/providers/all'
import { discoverModels } from '../src/discovery.ts'
const servers: Server[] = []
/** Credential variables a test set, cleared so the next one starts unset. */
const touchedEnv: string[] = []
afterEach(async () => {
for (const name of touchedEnv.splice(0)) Reflect.deleteProperty(process.env, name)
await Promise.all(servers.splice(0).map(server => new Promise(resolve => server.close(resolve))))
})
@@ -140,6 +143,50 @@ describe('draft-provider model discovery', () => {
expect(server.headers[0]?.authorization).toBeUndefined()
})
it('authenticates a configured route the draft cannot supply a key for', async () => {
// What the Models page actually sends after a key is saved: the form holds
// the redacted descriptor, so the draft names the route and the endpoint
// and no credential at all. Interrogating unauthenticated would answer 401
// and read as a wrong key.
const server = await listingServer({ body: JSON.stringify({ data: [{ id: 'm' }] }) })
const ctx = new Context()
await ctx.plugin(LlmService)
process.env['ACME_GATEWAY_KEY'] = 'stored-key'
touchedEnv.push('ACME_GATEWAY_KEY')
await ctx.plugin(LlmPiAi, {
providers: {
'acme-gateway': {
apiKeyEnv: 'ACME_GATEWAY_KEY',
api: 'openai-completions',
baseURL: server.url,
models: [{ id: 'acme-large' }],
},
},
})
await ctx.llm.discoverModels('llm-pi-ai', { provider: 'acme-gateway', baseURL: server.url })
// A key typed into the form is the one being tested — possibly the
// replacement for the stored one — so it wins.
await ctx.llm.discoverModels('llm-pi-ai', { provider: 'acme-gateway', baseURL: server.url, apiKey: 'typed' })
// A route no profile declares yet is the create case: nothing is stored.
await ctx.llm.discoverModels('llm-pi-ai', { provider: 'not-declared-yet', baseURL: server.url })
expect(server.headers.map(headers => headers.authorization))
.toEqual(['Bearer stored-key', 'Bearer typed', undefined])
})
it('leaves a catalog route\'s credential unresolved, having never reached the network', async () => {
// The catalog answers before any endpoint is asked, so a route whose
// profile names a credential that is not set must still answer rather than
// failing over a key the interrogation never needed.
const ctx = new Context()
await ctx.plugin(LlmService)
Reflect.deleteProperty(process.env, 'ABSENT_FOR_DISCOVERY')
await ctx.plugin(LlmPiAi, { providers: { deepseek: { apiKeyEnv: 'ABSENT_FOR_DISCOVERY' } } })
await expect(ctx.llm.discoverModels('llm-pi-ai', { provider: 'deepseek' })).resolves.not.toHaveLength(0)
})
it('drops unusable rows rather than failing the whole listing', async () => {
const server = await listingServer({
body: JSON.stringify({