fix(system-prompt): reject a toolOrder that names an unregistered tool

Review follow-up (#196): a listed name with no registered tool was silently
ignored; misconfiguration must block work instead. The check lives in the
assembly — the earliest moment the registered tool set exists (tool plugins
register after the service constructs) and the only universal one (cordis has
no "all plugins loaded" event; registrations change at any time). assemble()
is now async so the throw surfaces as a rejection rather than a synchronous
escape from a Promise-returning method.

Blast radius, pinned by a loop-level test: the rejection reaches the turn's
outer catch — the turn closes balanced with an `error` reason, agent/error
mirrors it, no step opens, no request/header is logged, no request reaches
the adapter, and the agent returns to idle; every turn fails identically
until the config is fixed. A boot-time validation pass was considered and
rejected (recorded in the RFC). The general principle — misconfiguration
fails loud, never a silent skip — is added to AGENTS.md.
This commit is contained in:
imccyu
2026-07-07 20:20:56 +08:00
parent 72933ec558
commit adbba0deb2
10 changed files with 106 additions and 35 deletions

View File

@@ -558,13 +558,17 @@ export interface Config {
persona?: string
/**
* Explicit model-facing tool order, as a list of `ToolSchema.name`s: listed
* tools take their listed position, names with no registered tool are
* ignored, and tools absent from the list are inserted at the
* {@link TOOL_ORDER_REST} (`'<unlisted-tools>'`) entry in lexicographic name order. A
* configured list must contain the rest entry exactly once and no duplicate names —
* anything else throws at load; a bad order config must never reach a
* model request. When omitted, tools are ordered lexicographically by name.
* Applied to the tools {@link SystemPrompt.assemble} collects, BEFORE the
* tools take their listed position, and tools absent from the list are
* inserted at the {@link TOOL_ORDER_REST} (`'<unlisted-tools>'`) entry in
* lexicographic name order. A configured list must contain the rest entry
* exactly once, no duplicate names, and no name without a registered tool
* a misconfigured order blocks work instead of silently reaching a model
* request: shape violations throw at load, and an unregistered name rejects
* every assembly (failing the turn before any model request — the earliest
* moment the registered tool set exists to check against, since tool
* plugins register after this service constructs). When omitted, tools are
* ordered lexicographically by name. Applied to the tools
* {@link SystemPrompt.assemble} collects, BEFORE the
* `system-prompt/assemble` waterfall — like the sections' `order` sort, it
* canonicalizes what the registry contributed (registration order is a
* plugin-load artifact); a waterfall listener that mutates the tool list
@@ -575,7 +579,7 @@ export interface Config {
}
```
Source: [`packages/core/system-prompt/src/index.ts:161`](../packages/core/system-prompt/src/index.ts)
Source: [`packages/core/system-prompt/src/index.ts:174`](../packages/core/system-prompt/src/index.ts)
## `@deepseek-ai/dsh-tool-fs`

View File

@@ -190,7 +190,7 @@ variable(name: string, provider: (context: AssembleContext) => string | undefine
assemble(context: AssembleContext = {}): Promise<PromptAssembly>
```
Source: [`packages/core/system-prompt/src/index.ts:262`](../../packages/core/system-prompt/src/index.ts)
Source: [`packages/core/system-prompt/src/index.ts:279`](../../packages/core/system-prompt/src/index.ts)
## `ctx.tools` — `ToolRegistry`

View File

@@ -8,10 +8,15 @@ The order of the tool list a model call carries — `request/header.tools` on th
## Decision
The system-prompt assembly owns the canonical model-facing tool order, exactly where it already owns section order:
The system-prompt assembly owns the canonical model-facing tool order, exactly where it already owns section order. `toolOrder?: string[]` on `dsh-system-prompt` is the optional explicit policy:
- **One config key on `dsh-system-prompt`.** `toolOrder?: string[]` names tools in the exact order to send. A listed tool takes its listed position; a listed name with no registered tool is ignored (a deployment may list optional tools it does not always load); tools absent from the list are inserted at the `'<unlisted-tools>'` rest entry (`TOOL_ORDER_REST`), in lexicographic name order among themselves. The list must contain the rest entry exactly once and no duplicate names — violations throw from the service constructor, failing the fiber at load, never mid-conversation. When `toolOrder` is unset, the canonical order is plain lexicographic name order (code-unit comparison, locale-independent) — determinism requires no configuration.
- **Applied where the list is born: `assemble()`, before the `system-prompt/assemble` waterfall.** The assembly canonicalizes the tools it collects from providers the same way it sorts sections by their `order` field — on the initial assembly, killing the registration-order entropy at its source. Everything downstream inherits the order untouched: the waterfall, the loop's `EpochHeader`, the `request/header` event, the deep-frozen request, and the dev invariant's cross-check all see one deterministic list, with no new service surface and no loop change.
- A listed tool that is registered takes its listed position.
- A listed name with no registered tool is a configuration error. Shape errors (rest entry missing or duplicate names) fail from the service constructor; an unregistered name rejects every `assemble()` — the earliest moment the registered tool set exists to check against (tool plugins register after the service constructs), and the only universal one (registrations can change at any time; cordis has no "all plugins loaded" event). Under the shipped loop the first turn fails before any model request — see the consequences below for the exact blast radius.
- A registered tool absent from the list is inserted at the `'<unlisted-tools>'` rest entry (`TOOL_ORDER_REST`), in lexicographic name order among the other unlisted tools.
- The list must contain the rest entry exactly once and no duplicate names.
- When `toolOrder` is unset, the canonical order is plain lexicographic name order (code-unit comparison, locale-independent), so determinism requires no configuration.
The policy is applied where the list is born: `assemble()`, before the `system-prompt/assemble` waterfall. The assembly canonicalizes the tools it collects from providers the same way it sorts sections by their `order` field — on the initial assembly, killing the registration-order entropy at its source. Everything downstream inherits the order untouched: the waterfall, the loop's `EpochHeader`, the `request/header` event, the deep-frozen request, and the dev invariant's cross-check all see one deterministic list, with no new loop change.
Scope is deliberately narrow: this fixes the REGISTRATION-ORDER race, not plugin behavior. A `system-prompt/assemble` listener may still add, remove, or rearrange tools — same as it may edit sections after their sort — and owns the determinism of what it emits; the waterfall contract already demands deterministic listeners (the reconstructability invariant would catch a listener that diverges between build and replay).
@@ -26,6 +31,7 @@ Config plumbing follows the `persona` precedent, and `toolOrder` sits beside it:
- **A `LlmService` config + `orderTools()` method the loop calls before logging the header** — works, but adds a public service method and a loop edit solely to apply a policy at a distance; every future request composer must remember the call. Canonicalizing where the list is born makes an unordered list unrepresentable, with zero new surface.
- **Normalizing inside `llm.stream()`** — runs after the header event is logged (the flake survives) and rebuilds the deep-frozen envelope, silently disarming the reconstruction invariant.
- **An exhaustive list (no rest entry)** — every newly loaded tool plugin would break boot; the mandatory rest entry keeps unlisted tools deterministic and their position explicit.
- **A boot-time validation pass (a `SystemPrompt.assertToolOrderSatisfied()` called by `dsh-app-boot` after `loader.await()`)** — would turn the misconfiguration into a startup death instead of a first-turn failure, but costs a public service method plus a structural coupling from the generic boot glue to one service, and cannot replace the assembly-time check anyway (embedded callers never run app boot; registrations change after boot). No existing event can host the check either: cordis v4 has no ready-like event, `loader/entry-init`/`internal/status` fire mid-load (racy against tool registration, the very entropy this RFC kills), and the agent lifecycle events are no earlier than the assembly. One enforcement point at `assemble()` was judged worth the later failure moment.
## Consequences
@@ -34,7 +40,8 @@ Config plumbing follows the `persona` precedent, and `toolOrder` sits beside it:
- The snapshot suite's single pinned request-header fixture (`text-turn`) carries the new canonical tool order; every other ACP snapshot keeps the header bulk scrubbed as `{{system}}`/`{{tools}}`, per the pinned-header design.
- A pure tool reordering between steps is representable only as a `request/header` `'fallback'` snapshot (the name-keyed `ToolsDelta` cannot express it); with a stable canonical order such reorders no longer occur in practice, so the fallback path stays a safety valve.
- The `toolOrder` key rides the app → `agent-core``SystemPrompt` forwarding chain, so deployments set it next to `persona` in the app config; `dsh-llm` and the agent loop are untouched.
- A misspelled or unloaded tool name in `toolOrder` fails the turn at prompt assembly, not the boot: the loop assembles inside the turn (after `turn/start`, before `step/start`), so the rejection reaches the turn's outer catch — the turn closes balanced with an `error` reason carrying the message, `agent/error` mirrors it, no step opens, no `request/header` is logged, no request reaches the adapter, and the agent returns to idle. Every turn fails identically until the config is fixed; the process itself stays up (matching the repo rule that explicit config references must not be silently ignored — the enforcement point is the assembly because no earlier universal moment exists).
## Testing
Unit tests on `dsh-system-prompt` pin the ordering semantics (lexicographic default, listed/ignored/rest placement, stable handling of shared names, provider-order independence), the pre-waterfall contract (listeners observe the canonical list; a listener-appended tool is not re-sorted), and each invalid-list rejection at load. Loop-level tests assert the `request/header` fold carries the canonical order for scrambled registration orders (identical across permutations), that a configured `toolOrder` reaches both the logged header and the dispatched deep-frozen request, and that the frozen loop-built envelope survives to the adapter. Forwarding is asserted at every level that exposes the key (`dsh-agent-core`, `dsh-stdio-agent`, `dsh-acp-agent`). The snapshot tier replays all scenarios while only the pinned `text-turn` header carries the full canonical tool list; non-pinning fixtures continue to compare through `{{tools}}`.
Unit tests on `dsh-system-prompt` pin the ordering semantics (lexicographic default, listed/rest placement, unknown-name rejection at assembly, stable handling of shared names, provider-order independence), the pre-waterfall contract (listeners observe the canonical list; a listener-appended tool is not re-sorted), and each invalid-list rejection at load. Loop-level tests assert the `request/header` fold carries the canonical order for scrambled registration orders (identical across permutations), that a configured `toolOrder` reaches both the logged header and the dispatched deep-frozen request, that the frozen loop-built envelope survives to the adapter, and that an unregistered `toolOrder` name fails the turn with a balanced `error` `turn/end`, an `agent/error`, no step, no logged header, and no dispatched request. Forwarding is asserted at every level that exposes the key (`dsh-agent-core`, `dsh-stdio-agent`, `dsh-acp-agent`). The snapshot tier replays all scenarios while only the pinned `text-turn` header carries the full canonical tool list; non-pinning fixtures continue to compare through `{{tools}}`.