mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
5a8234643a
GenerateOptions.prefill had no production setter and both adapters
rejected it with LlmError('UNSUPPORTED') — its entire observable
behavior was two throws, each pinned by one adapter test. DeepSeek's
chat-prefix completion is a Beta feature on a base URL neither adapter
targets. ToolSchema.strict was threaded through defineTool, the
registry's schemas() allowlist, the deepseek wire mapping, a per-tool
payload-patching pass in the pi-ai adapter, and a tool-catalog render
row, yet no shipped tool set it and the internal endpoint story for
strict mode was never built.
Remove both fields end-to-end: the vocabulary in dsh-llm, the adapter
guards and wire branches, the dsh-tools threading, the tool-catalog
Strict row, the pinning tests, the core.md pastes, the adapter README
rows, and the cookbook line that used prefill as the UNSUPPORTED
example (now stated generically). The pi-ai payload fixup keeps the
half with a job: pi-ai stamps strict:false on every serialized tool,
so the fixup scrubs it unconditionally for wire parity with the
hand-rolled twin (per-tool set/delete machinery gone). temperature/
stop/maxTokens are untouched — honored end-to-end by both adapters.
Each knob returns with its first real producer: prefill with an
adapter that implements chat-prefix completion, strict with a tool
that wants it and a beta-endpoint story.
RFC: docs/rfc/implemented/simplification/2026-07-04-drop-inert-request-knobs.md
(moved from proposed/, amended to shipped reality); the content-block
vocabulary RFC's consequence line now records prefill as producer-gated.
44 lines
3.5 KiB
Markdown
44 lines
3.5 KiB
Markdown
# Cookbook: adding an LLM adapter
|
||
|
||
How to connect a new model provider. Reference implementations: `packages/llm/llm-deepseek` (hand-rolled HTTP/SSE) and `packages/llm/llm-pi-ai` (wrapping an LLM library). Read the `StreamChunk` doc in `packages/llm/llm/src/types.ts` first — it records the protocol conventions both adapters were verified against.
|
||
|
||
## The shape
|
||
|
||
```ts ignore-check
|
||
class MyAdapter extends LlmAdapter {
|
||
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> { … }
|
||
}
|
||
|
||
export const name = 'llm-myprovider'
|
||
export const inject = ['llm']
|
||
export const Config: z<Config> = z.object({ apiKey: z.string(), … })
|
||
|
||
export function apply(ctx: Context, config: Config) {
|
||
ctx.llm.registerAdapter(['model-a', 'model-b'], new MyAdapter(…))
|
||
}
|
||
```
|
||
|
||
Registration is effect-based (HMR-safe); one adapter per model name — duplicates throw. Secrets are cordis-native: schemastery Config with env fallbacks, fed from cordis.yml via `!!js process.env.MY_KEY`. Never read ad-hoc key files in code.
|
||
|
||
## Protocol obligations (the contract two implementations verified)
|
||
|
||
- Emit `usage` BEFORE `finish`; emit NOTHING after `finish`. The robust way: buffer finish/usage until the provider's end-of-stream marker, then flush (handles providers that send trailing usage-only chunks).
|
||
- Tool-call `arguments` are RAW JSON strings end-to-end; stream fragments as `argumentsDelta`. If your provider hands back parsed objects, re-stringify at `block-end`.
|
||
- Allocate block `index`es in first-seen stream order; reuse the index for every delta of the same block.
|
||
- Errors have exactly two sanctioned paths: THROW from `stream()` (transport and protocol failures — use `LlmError` with a stable code), or end the stream with `finish {kind: 'error' | 'aborted'}` (provider in-band failures). Consumers handle both; pick per failure class and document it.
|
||
- Honor `options.signal` (pass it to fetch / your SDK).
|
||
- A `GenerateOptions` field your provider cannot honor (e.g. a `stop` list on a provider without stop sequences): throw `LlmError(..., 'UNSUPPORTED')` rather than silently dropping it.
|
||
|
||
Provider-specific request knobs (thinking modes, effort levels) belong in the ADAPTER's Config, not in `GenerateOptions` — the core vocabulary stays provider-neutral.
|
||
|
||
## Structure that worked
|
||
|
||
Split the adapter into testable stages (llm-deepseek's layout): wire types (`types.ts`, coverage-exempt) → request serializer → SSE/transport parser → chunk-translation state machine → a thin adapter class wiring them. Each stage gets its own unit suite.
|
||
|
||
## Testing
|
||
|
||
- **Unit: mock the provider, not the harness.** A scripted `node:http` server speaking the provider's wire format covers happy paths, every error status, malformed payloads, premature closes, and aborts — no network, and it drives the 100% per-file coverage gate. Works for SDK-backed adapters too (point the SDK's baseURL at the mock).
|
||
- **Hostile framing tests.** Split stream payloads at arbitrary byte positions (including mid-UTF-8) — real networks do.
|
||
- **E2E: `tests/*.e2e.ts`** under `pnpm run test:e2e`, gated with `describe.skipIf(!process.env.MY_KEY)` so CI (no secrets) stays green. Cover each model × each provider mode you map (thinking on/off, effort levels), a tool-call round trip INCLUDING the follow-up turn with results in history, and loose assertions only (substring/structure, bounded maxTokens — real models are nondeterministic).
|
||
- Register the e2e file pattern in `knip.json` (per-workspace `entry` override) or knip flags it unused.
|