feat(llm): add replay token metering (PR2 round 1)

This commit is contained in:
Hypatia May
2026-07-15 14:47:29 +08:00
parent c9efdf68f9
commit f038780ff6
61 changed files with 3393 additions and 2369 deletions

View File

@@ -50,7 +50,7 @@ interface CompactionResult {
## The service
`CompactService` exposes `compactIfNeeded(...)` for pressure-triggered compaction, returning `null` when no compaction is needed, and `compactRegion(...)` for an explicit inclusive surface range. The pre-step caller supplies the agent, full prompt, session prefix, and abort signal; implementations must forward that signal to summarization. Estimation, retention, event sequencing, and summarization remain backend policy.
`CompactService` exposes `compactIfNeeded(...)` for pressure-triggered compaction, returning `null` when no compaction is needed, and `compactRegion(...)` for an explicit inclusive surface range. The pre-step caller supplies the agent, full prompt, session prefix, and abort signal; implementations must forward that signal to summarization. The seam owns no pricing API: `dsh-compact-basic` resolves the routed model through [`ctx.tokenMeter`](token-meter.md), whose model-bound handle owns estimation and replay, while the backend owns retention, event sequencing, and summarization.
Auto-compaction runs at serial `agent/pre-step`, before the step and request derivation, so it can replace surface nodes while keeping trace events outside the step. Region boundaries preserve tool-call/result pairing but do not preserve whole turns, allowing early closed steps of one oversized turn to compact. `dsh-compact-basic` owns the retention and failure details.

View File

@@ -16,6 +16,7 @@ Everything else is documented on a **sub-page**, not here. The rule that draws t
| Sub-page | Owns |
|---|---|
| [llm-streaming.md](llm-streaming.md) | the `StreamChunk` wire protocol + adapter contract, `BlockAssembler`, the `LlmAdapter` seam |
| [token-meter.md](token-meter.md) | immutable scalar and positional replay measurements with consumed-log revisions |
| [scope.md](scope.md) | scoped registration identity, dispatch carriers, and the owned `Scope` context |
| [session.md](session.md) | the full `SessionEventMap` variant catalog, `TurnTrigger`/`TurnEndReason`, `deriveMessages()`, the turn-enclosure invariant |
| [persistence.md](persistence.md) | the durability seam: `SessionPersistence`, JSONL + SQLite backends, `session/flush`, crash recovery, `SessionHeader` |

View File

@@ -150,6 +150,8 @@ type SessionEvent<T extends SessionEventType = SessionEventType> = {
`SessionEventType = keyof SessionEventMap`. Because `SessionEventMap` is merge-extensible, switches over `SessionEvent` must NOT use `assertNever` — a plugin-added variant is a valid unknown value; handle the known cases and fall through `default`.
For `assistant/message`, a present `sourceEventSeqs: []` is a complete known-empty provider stream, while an absent field means legacy or otherwise unrecorded provenance. The loop writes the field for every successful model call; every other surface event requires a non-empty list when the field is present.
## Surface types
The five message-producing types (`SurfaceEventType` — `user/message`, `assistant/message`, `tool/result`, `context/message`, `steering/message`) carry surface metadata declaring how they join the derived surface linked list. See the [session surface RFC](../rfc/implemented/architecture/2026-06-18-session-surface.md).
@@ -186,6 +188,8 @@ export interface SurfaceIntent {
Required for `SurfaceEventType` events — every message-producing event must declare how it joins the surface, the sole source of derived history. Non-surface types reject it at compile time.
The same provenance distinction applies here: only `assistant/message` may carry a present empty `sourceEventSeqs`; omission does not assert that its source stream was empty.
### `SurfaceNode` — a node in the surface linked list
```ts type-equiv

View File

@@ -0,0 +1,52 @@
# Token Meter
`@deepseek-ai/dsh-token-meter` exposes detached replay measurements for request pressure and positional surface pricing. Scalar and surface snapshots carry the number of durable events consumed as `logRevision`; consumers compare revisions before making a joint decision.
Source: [`packages/llm/token-meter/src/types.ts`](../../packages/llm/token-meter/src/types.ts)
## `TokenMeasurement`
```ts type-equiv
interface TokenMeasurement {
/** Model profile used for every heuristic component. */
readonly model: string
/** Number of durable events consumed; equal to the next unread event seq. */
readonly logRevision: number
/** Provider or heuristic anchor used for this measurement. */
readonly baseline: TokenMeasurementBaseline
/** Signed repricing of current surface content relative to the baseline anchor. */
readonly surfaceDeltaTokens: number
/** Non-negative current request-and-response pressure. */
readonly totalTokens: number
}
```
`baseline.kind === 'usage'` means a successful provider call has the same model and canonical envelope. `estimated` means the meter repriced the complete envelope and surface. Signed `surfaceDeltaTokens` preserves growth and shrinkage relative to a matching provider or estimated anchor.
## `TokenSurfaceNode`
```ts type-equiv
interface TokenSurfaceNode {
/** Durable sequence number of the surface event. */
readonly seq: number
/** Heuristic tokens for the exact message projected by this node. */
readonly tokens: number
}
```
## `TokenSurfaceMeasurement`
```ts type-equiv
interface TokenSurfaceMeasurement {
/** Model profile used to price every node. */
readonly model: string
/** Number of durable events consumed; equal to the next unread event seq. */
readonly logRevision: number
/** Total heuristic tokens across the current surface. */
readonly totalTokens: number
/** Current surface nodes in positional head-to-tail order. */
readonly nodes: readonly TokenSurfaceNode[]
}
```
Surface order is authoritative; replacement nodes can have higher durable seqs than later positional nodes. The snapshot is immutable and does not grow when the underlying replay fold advances.