mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
fix(host): pin model discovery to loopback and drop its unread wire field
llm.discoverModels was reachable from any declared trusted host. The method takes a caller-supplied baseURL and makes the host issue a GET to it, then reports the status or the parsed body — so on a LAN deployment an anonymous caller had a probe for whatever the host can reach and the browser cannot, plus a path that carries a draft credential. The PRIVILEGED_METHODS doc already states the rule this broke: trustedHosts is a DNS-rebinding fence, not authentication, so the configuration plane stays loopback-same-origin. It is in that set now, asserted both against the hand-built fence and over real HTTP beside the catalog reads that deliberately stay reachable. supportsDiscovery and listModelDiscoveryNamespaces are gone. The field was required on the wire and read by nobody: its own contract said a surface should offer the action "instead of naming an adapter family it would have to hardcode", while the surface hardcodes llm-pi-ai in two places and gates the button on whether there is anything to probe. Its shape did not fit the second caller either — the create card has no row to read a per-row field from. Keeping a required field alive for a consumer that may never arrive costs every producer and fixture a value nobody consults, which is exactly how the fixtures drifted. The registry that fed it had no other production consumer, so registration and disposal are now observed through the offer itself. The Agent Note claimed the key is never logged, which the wire schema beside it already contradicts, and predated both the provider field and the catalog-answer path. The two new public types pointed at core.md without a type-equiv block or manifest entry, so the generated service catalog named documentation that did not exist.
This commit is contained in:
Yichen Jiang
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-04-draft-provider-endpoint-interrogation.md
|
||||
2026-08-04-draft-provider-endpoint-interrogation.md: a09b971022986442b48bd7aa04a1dcabfa66eb8b
|
||||
2026-08-04-draft-provider-endpoint-interrogation.zh.md: 0f6a63385dc628938c702aca1895b608e4eeaf9a
|
||||
2026-08-04-draft-provider-endpoint-interrogation.md: 49b863a3b923e9cdae34462c63fb2e2ed0968941
|
||||
2026-08-04-draft-provider-endpoint-interrogation.zh.md: a6a74407b0713ffcadc734ecbca2b7d7363d7361
|
||||
|
||||
@@ -16,10 +16,10 @@ The awkward part is that the question is about something that does not exist yet
|
||||
|
||||
Interrogation is keyed by **settings namespace**, not by provider route:
|
||||
|
||||
- `ctx.llm.registerModelDiscovery(settingsNs, discover)` lets an adapter plugin offer to interrogate endpoints for the namespace it owns; `ctx.llm.listModelDiscoveryNamespaces()` lets a surface offer the action only where it works; `ctx.llm.discoverModels(settingsNs, request)` asks. The namespace is the right key because a configuration surface already holds it from the configurable-provider directory, and because a provider being added has no route to name.
|
||||
- `LlmModelDiscoveryRequest` carries the draft — `baseURL`, an optional `api`, an optional `apiKey`, and a signal. Nothing in this path reads or writes settings or credentials; the caller owns both.
|
||||
- `ctx.llm.registerModelDiscovery(settingsNs, discover)` lets an adapter plugin offer to interrogate endpoints for the namespace it owns, and `ctx.llm.discoverModels(settingsNs, request)` asks. There is no way to enumerate which namespaces registered: a surface that cannot interrogate learns it from the refusal, and a list nothing consumed would be a required wire field doing nothing. The namespace is the right key because a configuration surface already holds it from the configurable-provider directory, and because a provider being added has no route to name.
|
||||
- `LlmModelDiscoveryRequest` carries the draft — an optional `provider`, an optional `baseURL`, an optional `api`, an optional `apiKey`, and a signal — and needs at least one of `provider` or `baseURL` to have anything to answer about. `provider` exists because a route the adapter already describes is answered from its own registry with no network call at all; only a route it does not describe reaches an endpoint. Nothing in this path reads or writes settings or credentials; the caller owns both.
|
||||
- `LlmDiscoveredModel` makes every field but `id` optional, because most listings disclose an id and nothing else. The reply is candidates, not a catalog: a surface adopting one still owes the capacities the adapter requires.
|
||||
- `llm.discoverModels` carries the same draft over the wire. Its `apiKey` is the third and last payload on which a secret may ride, alongside `settings.update`/`mutate` and `credentials.set`, and it is never stored, logged, or echoed. Every refusal folds into `model-discovery-failed`, whose message is the adapter's own text and whose details name the endpoint asked but never the credential offered.
|
||||
- `llm.discoverModels` carries the same draft over the wire. Its `apiKey` is the third and last payload on which a secret may ride, alongside `settings.update`/`mutate` and `credentials.set`, and it is never stored or echoed back. It does ride the client's outgoing envelope like every other secret-bearing payload, where a `subscribeEnvelopes()` observer can see it; redacting that tap is a configuration-plane-wide change, not this method's to make alone. The method is loopback-only for a second reason besides the key: it makes the host issue a GET to a caller-chosen URL and reports the outcome, which is a probe an anonymous LAN caller must not have. Every refusal folds into `model-discovery-failed`, whose message is the adapter's own text and whose details name the endpoint asked but never the credential offered.
|
||||
|
||||
`dsh-llm-pi-ai` implements the wire path as a plain `GET {baseURL}/models`, reading `openai-completions` and `openai-responses`: their `GET /models` shape with bearer auth is the one a gateway, a self-hosted server, and the official endpoints all agree on. Azure is excluded despite its OpenAI lineage — it authenticates with an `api-key` header and requires an `api-version` query — and Codex uses OAuth; both would have reported an authentication failure as a provider with no models. Every other protocol answers `DISCOVERY_UNSUPPORTED`, so the surface falls back to hand-entry rather than reporting a guessed response shape as an empty provider. `baseURL` is treated as a prefix rather than a URL to resolve against, so a deployment path such as `https://gateway.example/openai/v1` keeps its segments. The reply is read under a four-megabyte ceiling enforced on the bytes actually received — the endpoint is a URL the user typed, so a declared `content-length` is checked first as a courtesy but never trusted as the bound, matching `dsh-web-fetch`'s two-stage shape for its own caller-supplied URLs.
|
||||
|
||||
|
||||
@@ -16,10 +16,10 @@ Status: implemented
|
||||
|
||||
询问以 **settings namespace** 为键，而不是提供方路由：
|
||||
|
||||
- `ctx.llm.registerModelDiscovery(settingsNs, discover)` 让适配器插件为自己拥有的 namespace 提供「询问端点」的能力；`ctx.llm.listModelDiscoveryNamespaces()` 让界面只在可用之处提供该动作；`ctx.llm.discoverModels(settingsNs, request)` 发起询问。以 namespace 为键是对的，因为配置界面已经从可配置提供方目录里拿到了它，也因为正在新增的提供方没有路由可点名。
|
||||
- `LlmModelDiscoveryRequest` 携带草稿——`baseURL`、可选的 `api`、可选的 `apiKey`，以及一个 signal。这条路径既不读也不写 settings 与 credentials；两者都归调用方所有。
|
||||
- `ctx.llm.registerModelDiscovery(settingsNs, discover)` 让适配器插件为自己拥有的 namespace 提供「询问端点」的能力，`ctx.llm.discoverModels(settingsNs, request)` 发起询问。没有任何办法枚举哪些 namespace 注册过：询问不了的界面会从那句拒绝里知道，而一份无人消费的列表只会变成一个什么都不做的必填协议字段。以 namespace 为键是对的，因为配置界面已经从可配置提供方目录里拿到了它，也因为正在新增的提供方没有路由可点名。
|
||||
- `LlmModelDiscoveryRequest` 携带草稿——可选的 `provider`、可选的 `baseURL`、可选的 `api`、可选的 `apiKey`，以及一个 signal——且 `provider` 与 `baseURL` 至少要有一个，才有东西可答。`provider` 之所以存在，是因为适配器已经描述过的路由直接由它自己的注册表作答、完全不联网；只有它未描述的路由才会抵达某个端点。这条路径既不读也不写 settings 与 credentials；两者都归调用方所有。
|
||||
- `LlmDiscoveredModel` 除 `id` 外每个字段都可选，因为大多数列表只公布 id。回复是候选而非 catalog：采纳其中一条的界面仍要补上适配器所需的容量。
|
||||
- `llm.discoverModels` 把同一份草稿送过协议层。它的 `apiKey` 是 secret 可以搭乘的第三个、也是最后一个载荷（另两个是 `settings.update`/`mutate` 与 `credentials.set`），且绝不被存储、记录或回显。每一种拒绝都折叠为 `model-discovery-failed`，其消息是适配器自己的文本，details 点名被询问的端点，绝不点名所提供的凭据。
|
||||
- `llm.discoverModels` 把同一份草稿送过协议层。它的 `apiKey` 是 secret 可以搭乘的第三个、也是最后一个载荷（另两个是 `settings.update`/`mutate` 与 `credentials.set`），且绝不被存储或回显。它确实会像其他承载机密的载荷一样随客户端外发信封同行，`subscribeEnvelopes()` 观察者看得到；把那个抽头脱敏是整个配置面的改动，不该由这一个方法独自决定。除密钥之外它被钉在回环还有第二个理由：它让宿主向调用方选定的 URL 发起 GET 并回报结果，这是匿名 LAN 调用者不该拥有的探测能力。每一种拒绝都折叠为 `model-discovery-failed`，其消息是适配器自己的文本，details 点名被询问的端点，绝不点名所提供的凭据。
|
||||
|
||||
`dsh-llm-pi-ai` 的实现只是一次朴素的 `GET {baseURL}/models`，且仅限 OpenAI 兼容协议。它们的列表形状是网关、自建服务与官方端点三方一致认可的那一种，而这正是该动作存在的场景。其余协议一律以 `DISCOVERY_UNSUPPORTED` 回答，让界面回退到手工填写，而不是把猜错的响应形状报成一个空提供方。`baseURL` 按前缀而非待解析 URL 处理，因此 `https://gateway.example/openai/v1` 这类部署路径会保留其路径段。回复在四兆字节上限下读取，且上限落在实际收到的字节上——端点是用户自己填的 URL，因此会先看声明的 `content-length` 作为善意提示，但绝不把它当作边界；这与 `dsh-web-fetch` 面对自己的调用方提供 URL 时所用的两段式形状一致。
|
||||
|
||||
|
||||
@@ -856,13 +856,6 @@ listConfigurableProviders(): LlmConfigurableProvider[]
|
||||
*/
|
||||
registerModelDiscovery( settingsNs: string, discover: (request: LlmModelDiscoveryRequest) => Promise<readonly LlmDiscoveredModel[]>, ): () => void
|
||||
|
||||
/**
|
||||
* List the settings namespaces that can interrogate a provider endpoint, so
|
||||
* a surface can offer the action only where it will work.
|
||||
* @returns the namespaces in registration order.
|
||||
*/
|
||||
listModelDiscoveryNamespaces(): string[]
|
||||
|
||||
/**
|
||||
* Interrogate one provider endpoint for the models it advertises. The
|
||||
* request describes a draft, not a stored route, so nothing here reads or
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/core-data-structures/core.md
|
||||
core.md: 1b5704384157688b45ae0900bf2d9924426bbd6b
|
||||
core.zh.md: 05802039920163ac8185703ab483c5603087ed96
|
||||
core.md: 97567226e25f06a7d97fe015995db11d94be7397
|
||||
core.zh.md: 8d06fa481a5a1e920f4e450f44dc26c28a1121a6
|
||||
|
||||
@@ -259,6 +259,55 @@ interface LlmModelInfo {
|
||||
}
|
||||
```
|
||||
|
||||
A provider a surface is still drafting has no route and no catalog, so interrogation is described separately: the request carries the draft the user is editing, and the reply is candidates a surface may adopt rather than a catalog it must serve.
|
||||
|
||||
```ts type-equiv
|
||||
/**
|
||||
* One interrogation of a provider endpoint that configuration has not stored
|
||||
* yet. Configuration surfaces send the draft a user is still editing, so the
|
||||
* request carries the endpoint and credential directly instead of naming a
|
||||
* route: a provider being added has no route to name.
|
||||
*/
|
||||
interface LlmModelDiscoveryRequest {
|
||||
/**
|
||||
* Route the draft is editing, when it edits an existing one. A route whose
|
||||
* adapter already knows its models answers from that knowledge instead of
|
||||
* asking the endpoint — the adapter's own registry is the better answer, and
|
||||
* it costs no network call.
|
||||
*/
|
||||
provider?: string
|
||||
/**
|
||||
* Endpoint to interrogate. Optional because a route the adapter already
|
||||
* describes needs none; a route it does not must supply one.
|
||||
*/
|
||||
baseURL?: string
|
||||
/** Wire protocol the endpoint speaks, when the draft names one. */
|
||||
api?: string
|
||||
/** Credential for this interrogation alone; the harness never stores it. */
|
||||
apiKey?: string
|
||||
/** Caller cancellation; implementations must settle promptly after it aborts. */
|
||||
signal?: AbortSignal
|
||||
}
|
||||
```
|
||||
|
||||
```ts type-equiv
|
||||
/**
|
||||
* One model an endpoint reports about itself. Every field but the id is
|
||||
* optional because most provider listings disclose an id and nothing else;
|
||||
* a surface adopting one of these still owes the capacities its adapter needs.
|
||||
*/
|
||||
interface LlmDiscoveredModel {
|
||||
/** Model id the endpoint accepts. */
|
||||
id: string
|
||||
/** Human-readable name when the endpoint supplies one. */
|
||||
name?: string
|
||||
/** Maximum combined request and response context, when disclosed. */
|
||||
contextWindow?: number
|
||||
/** Maximum output tokens, when disclosed. */
|
||||
maxTokens?: number
|
||||
}
|
||||
```
|
||||
|
||||
Correctness-sensitive metadata is resolved separately from the advisory catalog and is owned by the adapter serving the exact route. Context capacity, adapter call defaults, and reasoning choices share one exact-model result so consumers do not repeat authoritative model resolution.
|
||||
|
||||
```ts type-equiv
|
||||
|
||||
@@ -265,6 +265,55 @@ interface LlmModelInfo {
|
||||
}
|
||||
```
|
||||
|
||||
界面正在起草的提供方既没有路由也没有 catalog，因此询问被单独描述：请求携带用户正在编辑的草稿，回复是界面可以采纳的候选，而不是它必须服务的 catalog。
|
||||
|
||||
```ts type-equiv
|
||||
/**
|
||||
* One interrogation of a provider endpoint that configuration has not stored
|
||||
* yet. Configuration surfaces send the draft a user is still editing, so the
|
||||
* request carries the endpoint and credential directly instead of naming a
|
||||
* route: a provider being added has no route to name.
|
||||
*/
|
||||
interface LlmModelDiscoveryRequest {
|
||||
/**
|
||||
* Route the draft is editing, when it edits an existing one. A route whose
|
||||
* adapter already knows its models answers from that knowledge instead of
|
||||
* asking the endpoint — the adapter's own registry is the better answer, and
|
||||
* it costs no network call.
|
||||
*/
|
||||
provider?: string
|
||||
/**
|
||||
* Endpoint to interrogate. Optional because a route the adapter already
|
||||
* describes needs none; a route it does not must supply one.
|
||||
*/
|
||||
baseURL?: string
|
||||
/** Wire protocol the endpoint speaks, when the draft names one. */
|
||||
api?: string
|
||||
/** Credential for this interrogation alone; the harness never stores it. */
|
||||
apiKey?: string
|
||||
/** Caller cancellation; implementations must settle promptly after it aborts. */
|
||||
signal?: AbortSignal
|
||||
}
|
||||
```
|
||||
|
||||
```ts type-equiv
|
||||
/**
|
||||
* One model an endpoint reports about itself. Every field but the id is
|
||||
* optional because most provider listings disclose an id and nothing else;
|
||||
* a surface adopting one of these still owes the capacities its adapter needs.
|
||||
*/
|
||||
interface LlmDiscoveredModel {
|
||||
/** Model id the endpoint accepts. */
|
||||
id: string
|
||||
/** Human-readable name when the endpoint supplies one. */
|
||||
name?: string
|
||||
/** Maximum combined request and response context, when disclosed. */
|
||||
contextWindow?: number
|
||||
/** Maximum output tokens, when disclosed. */
|
||||
maxTokens?: number
|
||||
}
|
||||
```
|
||||
|
||||
对正确性敏感的元数据与参考目录分开解析，并归服务该确切路由的适配器所有。上下文容量、适配器调用默认值和推理选项共用同一个确切模型结果，消费方因而无需重复执行权威模型解析。
|
||||
|
||||
```ts type-equiv
|
||||
|
||||
@@ -2438,9 +2438,9 @@ export function createFixtureApi(options: FixtureOptions = {}): ApiProxy {
|
||||
llm: {
|
||||
providers: request => ok(request, {
|
||||
providers: [
|
||||
{ provider: 'deepseek-official', displayName: 'DeepSeek', settingsNs: 'llm-deepseek', settingsPath: [], active: true, supportsDiscovery: false },
|
||||
{ provider: 'openai', displayName: 'openai', settingsNs: 'llm-pi-ai', settingsPath: ['providers', 'openai'], active: true, supportsDiscovery: true },
|
||||
{ provider: 'anthropic', displayName: 'anthropic', settingsNs: 'llm-pi-ai', settingsPath: ['providers', 'anthropic'], active: false, supportsDiscovery: true },
|
||||
{ provider: 'deepseek-official', displayName: 'DeepSeek', settingsNs: 'llm-deepseek', settingsPath: [], active: true },
|
||||
{ provider: 'openai', displayName: 'openai', settingsNs: 'llm-pi-ai', settingsPath: ['providers', 'openai'], active: true },
|
||||
{ provider: 'anthropic', displayName: 'anthropic', settingsNs: 'llm-pi-ai', settingsPath: ['providers', 'anthropic'], active: false },
|
||||
],
|
||||
}),
|
||||
models: request => ok(request, { groups: fixtureModelGroups(), failures: [] }),
|
||||
|
||||
@@ -44,10 +44,15 @@ export const Config: z<ConnectionConfig> = z.object({
|
||||
* reconnaissance no anonymous caller should have. `trustedHosts` is a
|
||||
* DNS-rebinding fence, explicitly not authentication, so the whole
|
||||
* configuration plane stays loopback-same-origin until a real authentication
|
||||
* layer exists. The model catalog (`llm.providers`, `llm.models`) is
|
||||
* deliberately NOT here: it carries provider ids, display names, and model
|
||||
* lists — no endpoints, keys, or key state — and a LAN client's model picker
|
||||
* legitimately needs it.
|
||||
* layer exists. `llm.discoverModels` belongs to that plane on both counts: it
|
||||
* carries a draft credential, and it makes the HOST issue a GET to a URL the
|
||||
* caller chose and reports back the status or the parsed body — an anonymous
|
||||
* LAN caller would have a probe for whatever the host can reach and the
|
||||
* browser cannot.
|
||||
*
|
||||
* The model catalog (`llm.providers`, `llm.models`) is deliberately NOT here:
|
||||
* it carries provider ids, display names, and model lists — no endpoints,
|
||||
* keys, or key state — and a LAN client's model picker legitimately needs it.
|
||||
*/
|
||||
const PRIVILEGED_METHODS = new Set([
|
||||
'host.pickDirectory',
|
||||
@@ -60,6 +65,7 @@ const PRIVILEGED_METHODS = new Set([
|
||||
'credentials.describe',
|
||||
'credentials.set',
|
||||
'credentials.unset',
|
||||
'llm.discoverModels',
|
||||
])
|
||||
|
||||
/**
|
||||
|
||||
@@ -129,13 +129,15 @@ describe('connection node half', () => {
|
||||
it('pins privileged methods to loopback even for a declared trusted authority', async () => {
|
||||
const { routes, dispose } = await mounted({ trustedHosts: ['harness.example'] })
|
||||
// The privileged set: native dialogs plus the whole settings/credential
|
||||
// configuration plane, reads included. The same declared authority reaches
|
||||
// configuration plane, reads included, plus the one method that makes the
|
||||
// host fetch a caller-chosen URL. The same declared authority reaches
|
||||
// ordinary reads (carrier-level 404 from the empty proxy proves the fence
|
||||
// passed), but each privileged method stays loopback-only and 403s.
|
||||
for (const method of [
|
||||
'host.pickDirectory', 'host.openPath',
|
||||
'settings.describe', 'settings.openDocument', 'settings.update', 'settings.replace', 'settings.mutate',
|
||||
'credentials.describe', 'credentials.set', 'credentials.unset',
|
||||
'llm.discoverModels',
|
||||
]) {
|
||||
const denied = fakeResponse()
|
||||
await routes[0]!.handler(
|
||||
@@ -221,6 +223,9 @@ describe('connection node half over a real HTTP server', () => {
|
||||
'settings.describe', 'settings.openDocument', 'settings.update', 'settings.replace', 'settings.mutate',
|
||||
'credentials.describe', 'credentials.set', 'credentials.unset',
|
||||
'host.pickDirectory', 'host.openPath',
|
||||
// Carries a draft credential and turns the host into a fetcher for a
|
||||
// URL the caller picked: an anonymous LAN caller must not reach it.
|
||||
'llm.discoverModels',
|
||||
]) {
|
||||
expect([method, await call(port, method, 'harness.example')]).toEqual([method, 403])
|
||||
}
|
||||
|
||||
@@ -145,7 +145,7 @@ function scriptedFace(overrides: {
|
||||
llm: {
|
||||
providers: vi.fn(() => Promise.resolve(ok({
|
||||
providers: [
|
||||
{ provider: 'deepseek-official', displayName: 'DeepSeek', settingsNs: 'llm-deepseek', settingsPath: [], active: true, supportsDiscovery: false },
|
||||
{ provider: 'deepseek-official', displayName: 'DeepSeek', settingsNs: 'llm-deepseek', settingsPath: [], active: true },
|
||||
{ provider: 'openai', displayName: 'openai', settingsNs: 'llm-pi-ai', settingsPath: ['providers', 'openai'], active: true },
|
||||
{ provider: 'anthropic', displayName: 'anthropic', settingsNs: 'llm-pi-ai', settingsPath: ['providers', 'anthropic'], active: false },
|
||||
{ provider: 'zombie', displayName: 'zombie', settingsNs: 'llm-pi-ai', settingsPath: ['providers', 'zombie'], active: false },
|
||||
@@ -230,7 +230,7 @@ describe('ModelsSection', () => {
|
||||
})
|
||||
|
||||
it('decides setup need from the joined credential state and literal-key sidecar', () => {
|
||||
const entry = { provider: 'p', displayName: 'p', settingsNs: 'llm-deepseek', settingsPath: [], active: true, supportsDiscovery: false }
|
||||
const entry = { provider: 'p', displayName: 'p', settingsNs: 'llm-deepseek', settingsPath: [], active: true }
|
||||
const row = (
|
||||
credential: ProviderRow['credential'],
|
||||
literalApiKeyConfigured = false,
|
||||
|
||||
@@ -13,7 +13,7 @@ function row(overrides: Partial<ProviderRow> = {}): ProviderRow {
|
||||
displayName: 'DeepSeek',
|
||||
settingsNs: 'llm-deepseek',
|
||||
settingsPath: [],
|
||||
active: true, supportsDiscovery: false,
|
||||
active: true,
|
||||
},
|
||||
configured: true,
|
||||
removable: false,
|
||||
|
||||
@@ -432,10 +432,6 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
|
||||
signature: 'registerModelDiscovery( settingsNs: string, discover: (request: LlmModelDiscoveryRequest) => Promise<readonly LlmDiscoveredModel[]>, ): () => void',
|
||||
jsDoc: '/**\n * Offer to interrogate provider endpoints on behalf of the settings\n * namespace this plugin owns. The namespace is the key because that is what\n * a configuration surface already holds from the configurable-provider\n * directory, and because a provider being *added* has no route to name yet.\n * Disposed with the fiber.\n * @param settingsNs - the namespace whose profiles this discovery serves.\n * @param discover - interrogates one endpoint; must honor `request.signal`.\n * @returns the disposer that withdraws the offer.\n */',
|
||||
},
|
||||
{
|
||||
signature: 'listModelDiscoveryNamespaces(): string[]',
|
||||
jsDoc: '/**\n * List the settings namespaces that can interrogate a provider endpoint, so\n * a surface can offer the action only where it will work.\n * @returns the namespaces in registration order.\n */',
|
||||
},
|
||||
{
|
||||
signature: 'async discoverModels( settingsNs: string, request: LlmModelDiscoveryRequest, ): Promise<LlmDiscoveredModel[]>',
|
||||
jsDoc: '/**\n * Interrogate one provider endpoint for the models it advertises. The\n * request describes a draft, not a stored route, so nothing here reads or\n * writes settings or credentials — the caller owns both, and the reply is\n * candidate metadata a surface may offer for adoption.\n * @param settingsNs - namespace whose registered discovery serves this draft.\n * @param request - the endpoint, protocol, and one-shot credential to use.\n * @returns the advertised models, deduplicated in endpoint order.\n */',
|
||||
|
||||
@@ -2563,14 +2563,12 @@ export function createApiProxy(ctx: Context, defaults: ApiProxyDefaults): ApiPro
|
||||
const active = new Set(registered.map(provider => provider.id))
|
||||
const directory = ctx.llm.listConfigurableProviders()
|
||||
const declared = new Set(directory.map(entry => entry.provider))
|
||||
const discoverable = new Set(ctx.llm.listModelDiscoveryNamespaces())
|
||||
const views = directory.map(entry => ({
|
||||
provider: entry.provider,
|
||||
displayName: entry.displayName,
|
||||
settingsNs: entry.settingsNs,
|
||||
settingsPath: [...entry.settingsPath],
|
||||
active: active.has(entry.provider),
|
||||
supportsDiscovery: discoverable.has(entry.settingsNs),
|
||||
}))
|
||||
// Routes registered without a directory declaration still appear —
|
||||
// they exist and serve models — just with no settings address.
|
||||
@@ -2582,7 +2580,6 @@ export function createApiProxy(ctx: Context, defaults: ApiProxyDefaults): ApiPro
|
||||
settingsNs: '',
|
||||
settingsPath: [],
|
||||
active: true,
|
||||
supportsDiscovery: false,
|
||||
})
|
||||
}
|
||||
return Promise.resolve(ok(request, { providers: views }))
|
||||
|
||||
@@ -16,7 +16,6 @@ export const configurableProviderViewSchema = z.object({
|
||||
settingsNs: z.string(),
|
||||
settingsPath: z.array(z.string()),
|
||||
active: z.boolean(),
|
||||
supportsDiscovery: z.boolean(),
|
||||
}) satisfies z.ZodType<Wire<ConfigurableProviderView>>
|
||||
|
||||
/** llm.providers request payload. */
|
||||
|
||||
@@ -22,12 +22,6 @@ export interface ConfigurableProviderView {
|
||||
settingsPath: string[]
|
||||
/** Whether the route is currently registered (its models are requestable). */
|
||||
active: boolean
|
||||
/**
|
||||
* Whether `llm.discoverModels` can answer for this entry's namespace. A
|
||||
* surface offers the action only where it works instead of naming an adapter
|
||||
* family it would have to hardcode.
|
||||
*/
|
||||
supportsDiscovery: boolean
|
||||
}
|
||||
|
||||
/** Llm-domain unary methods (the map keys llm.* of RpcMethodMap). */
|
||||
|
||||
@@ -529,11 +529,11 @@ describe('llm domain', () => {
|
||||
const api = createApiProxy(ctx, DEFAULTS)
|
||||
const value = expectOk(await api.llm.providers(request({})))
|
||||
expect(value.providers).toEqual([
|
||||
{ provider: 'deepseek-official', displayName: 'DeepSeek', settingsNs: 'llm-deepseek', settingsPath: [], active: true, supportsDiscovery: false },
|
||||
{ provider: 'openai', displayName: 'openai', settingsNs: 'llm-pi-ai', settingsPath: ['providers', 'openai'], active: false, supportsDiscovery: true },
|
||||
{ provider: 'deepseek-official', displayName: 'DeepSeek', settingsNs: 'llm-deepseek', settingsPath: [], active: true },
|
||||
{ provider: 'openai', displayName: 'openai', settingsNs: 'llm-pi-ai', settingsPath: ['providers', 'openai'], active: false },
|
||||
// An undeclared live route has no settings address, so nothing can be
|
||||
// interrogated on its behalf either.
|
||||
{ provider: 'undeclared', displayName: 'Undeclared', settingsNs: '', settingsPath: [], active: true, supportsDiscovery: false },
|
||||
{ provider: 'undeclared', displayName: 'Undeclared', settingsNs: '', settingsPath: [], active: true },
|
||||
])
|
||||
})
|
||||
|
||||
|
||||
@@ -677,7 +677,6 @@ describe('config unary surface', () => {
|
||||
settingsNs: 'llm-pi-ai',
|
||||
settingsPath: ['providers', 'openai'],
|
||||
active: false,
|
||||
supportsDiscovery: true,
|
||||
}
|
||||
const group = { id: 'deepseek-official', name: 'DeepSeek', models: [{ id: 'deepseek-v4-flash', name: 'Flash' }] }
|
||||
const api = scriptedApi({
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: 07de1c0aceeccff5f3f14a43c4888b4481c0293a
|
||||
README.zh.md: 0e8895a0192c7f18e2b6ee8869896080f7ff49e2
|
||||
README.md: 6f3fa0bb0eb0236ab885ef803ace8069b1d52302
|
||||
README.zh.md: c0897d92da83084c5eeac02d5fd24c34a71e1af2
|
||||
|
||||
@@ -79,6 +79,16 @@ Supported profile fields are `apiKey`, `apiKeyEnv`, `displayName`, `api`, `baseU
|
||||
|
||||
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
|
||||
|
||||
## Endpoint interrogation
|
||||
|
||||
The plugin offers `ctx.llm.registerModelDiscovery('llm-pi-ai', …)`, which answers "which models can this provider serve?" for a route a configuration surface is editing or drafting. It is deliberately *not* a catalog refresh: nothing is stored, and the reply is candidates the surface offers for adoption. `settings.yaml` remains the only thing that decides what a route serves.
|
||||
|
||||
A request naming a route the **installed catalog ships is answered from that catalog**, with no network call: pi-ai's registry is the authoritative list for its own providers, and it carries the context windows and output caps a listing endpoint would not disclose. Such a route needs no `baseURL` at all. Only a route the catalog does not describe — a gateway, a self-hosted server — is interrogated over the wire, and one that names no endpoint is told to set one or enter its models by hand.
|
||||
|
||||
Interrogation reads `openai-completions` and `openai-responses`, whose `GET /models` shape with bearer auth is the one a gateway, a self-hosted server, and the official endpoints all agree on. Azure is excluded despite its OpenAI lineage — it authenticates with an `api-key` header and requires an `api-version` query — and Codex uses OAuth; every other protocol answers `DISCOVERY_UNSUPPORTED` so the surface falls back to hand-entry instead of an authentication failure being reported as a provider with no models. The `baseURL` is treated as a prefix rather than a URL to resolve against, so a deployment path such as `https://gateway.example/openai/v1` keeps its segments.
|
||||
|
||||
Most listings disclose an id and nothing else; `context_window`/`context_length` and `max_output_tokens`/`max_tokens` are read when a gateway supplies them, entries without a usable id are skipped rather than failing the whole listing, and everything else the adopting surface still owes. The reply is read under a four-megabyte ceiling enforced on the bytes actually received — the endpoint is a URL the user typed, so a declared length is checked first but never trusted as the bound. An unreachable endpoint, a refused credential, a non-JSON body, and a body with no `data` array all fail with `DISCOVERY_FAILED` and a message naming the endpoint and, for a 401 or 403 alone, the credential. Cancellation during the body read surfaces as `ABORTED`, like a cancellation before the request went out.
|
||||
|
||||
## Provider/model routing and replay
|
||||
|
||||
Each resolution produces one **immutable** snapshot — the profiles plus a `createModels()` collection holding the `Provider` each route built — and every operation captures a whole snapshot before its first `await`. A configuration change builds a *new* collection rather than mutating the one in use: `Models.streamSimple()` resolves its provider lazily, when the stream is first consumed, which is after the credential await, so a mutated collection would let a request that started under one configuration finish under another or fail on a provider that no longer exists. This is what makes the seam's per-step call freeze (`llm.prepareCall()`) hold end to end — switching models mid-reply takes effect on the next step, never inside the one in flight. Requests reach their provider through `Models.streamSimple()`. A catalog route that keeps its catalog protocol **reuses** the installed provider with its model list replaced, because that provider owns API implementations this package cannot reconstruct — Bedrock loads its Smithy module through a separate entry point — so rebuilding it from parts would silently narrow which providers work. Every other route is built by `createProvider()` over the protocol table behind `supportedProtocols()`, whose entries are the same factories pi-ai's own provider factories use.
|
||||
|
||||
@@ -79,6 +79,16 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog，而不是扩
|
||||
|
||||
适配器强制 pi-ai SDK `maxRetries` 为零，因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败，而不是静默倍增或隐藏单独组合的 agent（智能体）级重试预算。空闲超时会 abort SDK 的稳定请求信号，并以 `TIMEOUT` 呈现；较早的调用方 abort 仍为 `ABORTED`。
|
||||
|
||||
## 端点询问
|
||||
|
||||
插件提供 `ctx.llm.registerModelDiscovery('llm-pi-ai', …)`，用来回答「这个提供方能服务哪些模型？」——针对配置界面正在编辑或起草的路由。它刻意**不是** catalog 刷新：什么都不存储，回复是界面供用户采纳的候选。`settings.yaml` 始终是唯一决定路由服务什么的东西。
|
||||
|
||||
点名了**已安装 catalog 所提供路由**的请求，直接由该 catalog 作答，完全不联网：pi-ai 的注册表才是它自家提供方的权威列表，且携带列表端点不会公布的上下文窗口与输出上限。这类路由根本不需要 `baseURL`。只有 catalog 未描述的路由——网关、自建服务——才会经协议层询问；若它也没给端点，则会被告知去设置一个或手工填写模型。
|
||||
|
||||
询问只读 `openai-completions` 与 `openai-responses`，它们「`GET /models` + bearer 认证」的形状是网关、自建服务与官方端点三方一致认可的那一种。Azure 尽管出身 OpenAI 也被排除——它用 `api-key` 标头认证并要求 `api-version` 查询参数——Codex 则走 OAuth；其余协议一律以 `DISCOVERY_UNSUPPORTED` 回答，让界面回退到手工填写，而不是把认证失败报成一个没有模型的提供方。`baseURL` 按前缀而非待解析 URL 处理，因此 `https://gateway.example/openai/v1` 这类部署路径会保留其路径段。
|
||||
|
||||
多数列表只公布 id；`context_window`/`context_length` 与 `max_output_tokens`/`max_tokens` 在网关提供时会被读取，没有可用 id 的条目会被跳过而不是让整份列表失败，其余仍由采纳方补齐。回复在四兆字节上限下读取，且上限落在实际收到的字节上——端点是用户自己填的 URL，因此会先看声明长度，但绝不把它当作边界。端点不可达、凭据被拒、响应非 JSON、以及响应没有 `data` 数组，都会以 `DISCOVERY_FAILED` 失败，消息点名端点；仅当 401 或 403 时才点名凭据。读取响应体期间被取消会呈现为 `ABORTED`，与请求发出之前被取消一致。
|
||||
|
||||
## 提供方／模型路由与回放
|
||||
|
||||
每次解析产出一份**不可变**快照——profiles 加上一个持有各路由所建 `Provider` 的 `createModels()` 集合——每个操作都在自己第一个 `await` 之前整体捕获一份快照。配置变化会构造**新**集合，而不是改动正在被使用的那个：`Models.streamSimple()` 是惰性的，它在流首次被消费时才解析 provider，而那已在 credential await 之后，因此改动共享集合会让一个在旧配置下开始的请求在新配置下结束，或者撞上一个已不存在的 provider。这正是 seam 的每步调用冻结（`llm.prepareCall()`）能贯通到底的原因——回复途中切换模型会在下一步生效，绝不会影响在途的那一步。请求经 `Models.streamSimple()` 抵达提供方。保持 catalog 协议不变的 catalog 路由会**复用**已安装提供方，只替换其模型列表，因为该提供方持有本包无法重建的 API 实现——Bedrock 经由独立入口加载其 Smithy 模块——从零件重建会静默收窄可用提供方的范围。其余路由都由 `createProvider()` 基于 `supportedProtocols()` 背后的协议表构造，表中条目正是 pi-ai 自己的提供方工厂所用的同一批 factory。
|
||||
|
||||
@@ -191,6 +191,12 @@ export async function discoverModels(
|
||||
'DISCOVERY_FAILED',
|
||||
)
|
||||
}
|
||||
// A draft that has not chosen a protocol yet is asked as OpenAI Chat
|
||||
// Completions: it is the shape a gateway is overwhelmingly likely to speak,
|
||||
// and the alternative — refusing until the field is filled — would withhold
|
||||
// the action from the case it exists for. The cost is a misdirected message
|
||||
// when the endpoint speaks something else (an Anthropic gateway answers 401,
|
||||
// which reads as a credential problem), and hand-entry remains the way out.
|
||||
const api = request.api ?? 'openai-completions'
|
||||
if (!LISTABLE_PROTOCOLS.has(api)) {
|
||||
throw new LlmError(
|
||||
|
||||
@@ -245,7 +245,7 @@ describe('draft-provider model discovery', () => {
|
||||
it('is offered for the namespace, and refuses one it does not serve', async () => {
|
||||
const ctx = await harness()
|
||||
|
||||
expect(ctx.llm.listModelDiscoveryNamespaces()).toEqual(['llm-pi-ai'])
|
||||
await expect(ctx.llm.discoverModels('llm-pi-ai', { provider: 'openai' })).resolves.not.toHaveLength(0)
|
||||
await expect(ctx.llm.discoverModels('llm-deepseek', { baseURL: 'https://api.deepseek.com' }))
|
||||
.rejects.toMatchObject({ code: 'NO_DISCOVERY' })
|
||||
await expect(ctx.llm.discoverModels('llm-pi-ai', { baseURL: '' }))
|
||||
@@ -256,10 +256,11 @@ describe('draft-provider model discovery', () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const fiber = await ctx.plugin(LlmPiAi, {})
|
||||
expect(ctx.llm.listModelDiscoveryNamespaces()).toEqual(['llm-pi-ai'])
|
||||
await expect(ctx.llm.discoverModels('llm-pi-ai', { provider: 'openai' })).resolves.not.toHaveLength(0)
|
||||
|
||||
await fiber.dispose()
|
||||
|
||||
expect(ctx.llm.listModelDiscoveryNamespaces()).toEqual([])
|
||||
await expect(ctx.llm.discoverModels('llm-pi-ai', { provider: 'openai' }))
|
||||
.rejects.toMatchObject({ code: 'NO_DISCOVERY' })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -491,15 +491,6 @@ export class LlmService extends Service {
|
||||
return () => void dispose()
|
||||
}
|
||||
|
||||
/**
|
||||
* List the settings namespaces that can interrogate a provider endpoint, so
|
||||
* a surface can offer the action only where it will work.
|
||||
* @returns the namespaces in registration order.
|
||||
*/
|
||||
listModelDiscoveryNamespaces(): string[] {
|
||||
return [...this.discoveries.keys()]
|
||||
}
|
||||
|
||||
/**
|
||||
* Interrogate one provider endpoint for the models it advertises. The
|
||||
* request describes a draft, not a stored route, so nothing here reads or
|
||||
|
||||
@@ -212,14 +212,15 @@ describe('model discovery registry', () => {
|
||||
const discover = vi.fn(() => Promise.resolve([{ id: 'from-endpoint' }]))
|
||||
|
||||
const dispose = ctx.llm.registerModelDiscovery('llm-example', discover)
|
||||
expect(ctx.llm.listModelDiscoveryNamespaces()).toEqual(['llm-example'])
|
||||
|
||||
await expect(ctx.llm.discoverModels('llm-example', { baseURL: 'https://gateway.example/v1' }))
|
||||
.resolves.toEqual([{ id: 'from-endpoint' }])
|
||||
expect(discover).toHaveBeenCalledWith({ baseURL: 'https://gateway.example/v1' })
|
||||
|
||||
// Disposal is observed through the offer itself, which is the only thing
|
||||
// the registration ever produced.
|
||||
dispose()
|
||||
expect(ctx.llm.listModelDiscoveryNamespaces()).toEqual([])
|
||||
await expect(ctx.llm.discoverModels('llm-example', { baseURL: 'https://gateway.example/v1' }))
|
||||
.rejects.toThrow(/no model discovery is registered/)
|
||||
})
|
||||
|
||||
it('rejects an unnamed namespace and a second registration of the same one', async () => {
|
||||
@@ -229,7 +230,9 @@ describe('model discovery registry', () => {
|
||||
expect(() => ctx.llm.registerModelDiscovery('', discover)).toThrow(/non-empty settings namespace/)
|
||||
ctx.llm.registerModelDiscovery('llm-example', discover)
|
||||
expect(() => ctx.llm.registerModelDiscovery('llm-example', discover)).toThrow(/already registered/)
|
||||
expect(ctx.llm.listModelDiscoveryNamespaces()).toEqual(['llm-example'])
|
||||
// The refused second registration left the first one serving.
|
||||
await expect(ctx.llm.discoverModels('llm-example', { baseURL: 'https://gateway.example/v1' }))
|
||||
.resolves.toEqual([])
|
||||
})
|
||||
|
||||
it('normalizes what an interrogation returns without inventing capacities', async () => {
|
||||
|
||||
@@ -46,6 +46,16 @@
|
||||
"symbol": "LlmModelInfo",
|
||||
"source": "packages/llm/llm/src/types.ts"
|
||||
},
|
||||
{
|
||||
"doc": "docs/core-data-structures/core.md",
|
||||
"symbol": "LlmModelDiscoveryRequest",
|
||||
"source": "packages/llm/llm/src/types.ts"
|
||||
},
|
||||
{
|
||||
"doc": "docs/core-data-structures/core.md",
|
||||
"symbol": "LlmDiscoveredModel",
|
||||
"source": "packages/llm/llm/src/types.ts"
|
||||
},
|
||||
{
|
||||
"doc": "docs/core-data-structures/core.md",
|
||||
"symbol": "LlmModelContext",
|
||||
|
||||
Reference in New Issue
Block a user