deepseek-harness/packages/llm/llm/README.md

dsh-llm

Provider-neutral LLM vocabulary and abstract service. This package defines the canonical language spoken by the agent loop, session logs, and every plugin.

Service: LlmService (ctx key: llm)

An adapter registry plus a single streaming call surface, interceptable via a waterfall event.

Public API

• ctx.llm.registerAdapter(models: string[], adapter: LlmAdapter): () => void Register an adapter for the given model names. Disposed with the calling fiber.
• ctx.llm.models(): string[] — model names with a registered adapter.
• ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk> Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with BlockAssembler.

Events

Event	Mode	Purpose
llm/stream	waterfall	Intercept/wrap every streaming model call (retry, caching, routing)

Extension points

• Subclass LlmAdapter and call ctx.llm.registerAdapter(models, adapter) to add a new model provider.
• Wrap llm/stream via ctx.on() waterfall listeners for caching, retry, logging, rate-limiting, etc.

Content-block vocabulary (types.ts)

Messages are arrays of typed content blocks: text, reasoning, tool-call, tool-result. The union is derived from the merge-extensible ContentBlockMap, so plugins can add block types via declaration merging. The core set is limited to blocks every shipping path honors — multimodal content (images, audio, …) has no core block type; a feature that needs one adds it via the map together with the adapter/UI/compaction support that honors it.

Streaming is a raw chunk protocol (block-start, text-delta, reasoning-delta, tool-call-delta, block-end, usage, finish). BlockAssembler is the single shared implementation that assembles chunks into blocks/messages.

Call configuration (call-config.ts)

LlmCallConfig is the model + sampling scalars of one conversation's requests (model, temperature, maxTokens, stop — each mapping 1:1 onto the same-named GenerateOptions field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session request/header events), never a silently-adjustable per-call knob: the agent/request waterfall proposes a replacement and the loop logs a real change. callConfigEquals(a, b) is the field-wise real-change detector; deepFreeze(value) is the ownership helper the loop applies to every built request before dispatch (llm/stream listeners and adapters read, never rewrite).

App attribution (attribution.ts)

Every product adapter must identify the application on every provider HTTP request - attribution is part of the adapter contract, not an adapter-local nicety. attributionHeaders(identity?) builds the standard User-Agent header (product/version (+url), from userAgent()) for every request. The default APP_IDENTITY carries only static public product facts (its version is read from this package's manifest); a white-label deployment passes its own AppIdentity, and omission falls back to the default - nothing can suppress attribution. OpenRouter-specific app attribution headers are intentionally not supported by this contract. An adapter proves compliance with a wire-level test: a mock server asserting the received header (or, for a library-backed adapter, that the library's header hook delivers the same value). Policy and rationale: Mandatory User-Agent attribution.

Classes

• LlmAdapter — abstract base class for provider adapters. The only required method is stream().
• BlockAssembler — incrementally assembles raw chunks into complete content blocks and an assistant message. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks/message for history.
• HarnessError — base class for the harness error taxonomy: a stable code string (distinct from the human message) plus cause chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (LlmError, ToolArgsError, InvariantError, …) extend it. isHarnessError(value) narrows at seams.
• LlmError — extends HarnessError; code string (NO_ADAPTER, DUPLICATE_ADAPTER, and adapter codes like AUTH/RATE_LIMIT) plus an optional numeric status when the failure came from a non-2xx provider response.

Real adapters

Two adapters implement LlmAdapter against this vocabulary, deliberately built on different internals to keep the contract honest (see the twin LLM adapters): @deepseek-ai/dsh-llm-deepseek (hand-rolled fetch/SSE) and @deepseek-ai/dsh-llm-pi-ai (via @earendil-works/pi-ai). The pair pinned down the StreamChunk conventions now documented in types.ts (usage before finish, raw-string tool arguments, the two sanctioned error paths).