mirror of https://github.com/deepseek-ai/deepseek-harness synced 2026-08-15 21:04:50 +00:00

Files

kingwl 6adb14b56d feat(telemetry): adopt from the construction boundary — constructor seeds never re-export

A cursor-less adoption (process restart + resume, fork, seam-module
reload) replayed the session's full log from seq 0, re-exporting
history that already left the process — a resume re-billed its entire
stored log on every restart, and a fork re-shipped the parent's prefix
under the child's id, doubling query-time counts on OTLP backends with
no native ingest dedupe.

dsh-session now exposes the fact the constructor already validated but
discarded: Session.firstLiveSeq, the constructor-seed length — the
first seq appended in this process. header.seedLength cannot serve
here: it is the durable fork-lineage boundary, and a resumed session's
constructor seed is its full stored log while the header keeps the
original fork value (llm-replay and session-query-sqlite depend on
that meaning). Constructor seeds also never publish on the
session/event firehose, so adoption replaying them was inconsistent
with the system's own publication semantics.

Adoption's cursor-less fallback starts at firstLiveSeq; seed events
still feed the chunk projection, so mid-step continuations re-drop
after a resume. Fork streams are no longer self-contained: records now
carry session.seed_length (with the existing session.parent_id) so
receivers stitch the child's stream onto the parent's. Accepted cost,
consistent with at-most-once delivery and recorded in the revival
Agent Note: a resume no longer backfills records a previous process
failed to deliver — a deployment with that requirement needs the
deferred outbox, not replay.

Pinned red-first: seeded adoption exports nothing (assertion reversed
from the prior seed-readback test, obsolete behavior changed with its
test), resume-shaped seed rebuilds the projection without exporting,
and fork records carry the stitch attributes.

2026-07-27 18:35:03 +08:00

7.0 KiB

Raw Blame History

Agent Note: Session telemetry seam with mandatory redaction and the OTel backend

Status: implemented

English | 中文

Problem

Every deployment that wants harness sessions in an observability stack must hand-roll a session-log consumer: subscription, lifecycle handoff, and — hardest — redaction, since the raw log carries file contents and command output that may embed credentials. A telemetry seam and OTel backend shipped once on the session-telemetry-otlp-rfc branch (PR #222/#231) but never reached master: the proposal exported raw session events verbatim, which legal review declined. The capture-side design (backend contract, coordinator, handoff cursor, chunk projection) was sound and reviewed; the export-side stance was the blocker.

Decision

packages/telemetry/ revives the two reviewed packages under the SDK stance — the harness provides the capability, the deployment configures where records go and owns what leaves in them:

@deepseek-ai/dsh-session-telemetry — the seam. TelemetryBackend (emit/flush?/shutdown), the service-registered Telemetry form, and TelemetryCoordinator owning capture: adoption with cursor read-back, the per-append firehose (project → structuredClone → redact → emit, zero I/O), the fixed first-chunk-per-(turn, step) projection, the agent/error relay, and dispose-time shutdown records.
The telemetry/redact waterfall — the delta over the branch version. Every record passes it before reaching any backend; the seam ships NO rules of its own — the innermost next() is a pass-through, deployments mount their rules as listeners (stacking by transforming next()'s return value), and a throwing rule withholds the record fail-closed. Redaction applies to the exported copy only; the canonical log is never rewritten.
@deepseek-ai/dsh-session-telemetry-otel — the reference backend: OTel JS SDK log pipeline (LoggerProvider → BatchLogRecordProcessor → OTLP/HTTP exporter), configured verbatim through exporter/processor passthroughs. exporter.url is required and validated at load; unmounted or unconfigured, nothing leaves the process.

The boundary axiom holds: the harness's aspect ends at emit(). Batching, retry, queueing, and loss policy are the reporting SDK's, configured through passthroughs — delivery is best-effort (at-most-once across a crash), which the READMEs state plainly.

Alternatives considered

Implement the runtime-telemetry RFC's outbox (durable spool, per-sink cursors, at-least-once, a readCommitted persistence-seam method). Deferred, not rejected: the SDK stance makes delivery semantics the reporting SDK's territory, and the OTel SDK's own batch pipeline is the honest default. The outbox is a pure additive layer (the emit() contract does not move); revive it when a deployment states a crash-loss requirement telemetry must satisfy.

No in-process redaction point, delegating to receiver-side collector processors. Rejected — receiver-side redaction ships the secret first and scrubs it second. The waterfall puts an auditable, stackable scrubbing point before bytes leave the process; where the branch version (what PR #222 shipped) had no redaction point at all, every record now passes one.

A built-in conservative rule set as the waterfall's innermost next(). Rejected: as an SDK we cannot know which patterns are secrets in a given deployment, a shipped list invites false confidence ("redaction is on") while catching only known shapes, and false positives would corrupt exported bodies for consumers who never asked. The seam owns the mechanism; the deployment owns the policy — the innermost next() is a pass-through, and rules mount as listeners.

Map onto OTel spans (GenAI semantic conventions) instead of logs. Rejected for this revival: the branch implementation's log mapping is reviewed and shipped-shaped; the span model is lossy for forkable, interruptible sessions and belongs to a future consumer with real span queries to serve.

Full-log replay when no handoff cursor survived (re-export constructor seeds). Shipped in the first revival round, then narrowed: adoption now replays from the session's construction boundary (Session.firstLiveSeq, the constructor-seed length — a fact the session already validated but did not expose; header.seedLength cannot serve, it is the durable fork-lineage value and a resumed session's constructor seed is its full stored log). A resumed session's history already shipped from the previous process under the same id, and a fork's inherited prefix already shipped in the parent's stream — re-exporting either re-billed every resume for its full history and doubled query-time counts on OTLP backends with no native ingest dedupe. Receivers stitch fork lineage via session.parent_id + session.seed_length. What the narrowing gives up, consistently with the at-most-once stance: a resume no longer backfills records the previous process failed to deliver (telemetry unmounted then, or queued at crash) — the full replay's only real benefit, bought at the common case's expense. A deployment that states a backfill requirement needs the deferred outbox above, not replay.

Forwarding the seam's turn-boundary flush() hint to the OTel provider's forceFlush(). Shipped in the first revival round, then removed after three review rounds each found a new silent-loss path in the same wrapper state: a dispose racing an in-flight flush (the SDK's concurrent-flush guard makes shutdown's internal drain skip), overlapping hints displacing the retained promise, and the provider's fixed 30-second flush timeout rejecting while the processor still drains. Every path exists only because the forwarding made this backend the process's second flusher against undocumented SDK internals from the upstream experimental tree; with no flush() implemented, the batch processor is the only flusher, its scheduledDelayMillis (already deployment-tunable through the processor passthrough) governs export cadence, and shutdown()'s drain is complete by construction. Reinstate only if a deployment states a turn-boundary latency requirement scheduledDelayMillis cannot meet — and then by calling the retained BatchLogRecordProcessor's own forceFlush(), never the provider's timeout-wrapped one.

Consequences

A deployment adds one cordis.yml entry with an OTLP endpoint and gets its session stream in any OTel-compatible stack; removing the entry is the opt-out, with no residual state. A rule-free deployment exports records exactly as captured — including any credentials embedded in file contents or command output — so a deployment crossing a trust boundary must mount telemetry/redact listeners, and both READMEs state this plainly. Where rules are mounted, exported bodies can differ from canonical log bytes, so receivers must not treat telemetry as a byte-exact replica; the log remains the source of truth. Crash durability is explicitly out of scope until the outbox decision above is revisited.

7.0 KiB Raw Blame History