mirror of https://github.com/deepseek-ai/deepseek-harness synced 2026-08-15 21:04:50 +00:00

Files

kingwl 95315386b3 refactor(telemetry): remove the OTel backend's flush forwarding

Three review rounds each found a new silent-loss path in the same
wrapper state (dispose racing an in-flight flush, overlapping hints
displacing the retained promise, the provider's fixed 30s flush timeout
rejecting while the processor still drains). Every path exists only
because forwarding the seam's turn-boundary hint to forceFlush() made
this backend the process's second flusher against undocumented SDK
internals from the upstream experimental tree.

The backend now implements no flush(): the batch processor is the only
flusher, its scheduledDelayMillis (already deployment-tunable through
the processor passthrough) governs export cadence, and shutdown()'s
drain is complete by construction. The two race-pin tests collapse into
one dispose-during-in-flight-batch drain pin; the seam's optional
flush() contract now tells implementers they own the concurrent-flush/
shutdown interaction. Removal rationale and the reinstatement trigger
(a stated turn-boundary latency requirement scheduledDelayMillis cannot
meet — and then via the processor's own forceFlush(), never the
provider's timeout-wrapped one) are recorded in the revival Agent Note,
both languages.

2026-07-26 17:55:54 +08:00

5.8 KiB

Raw Blame History

Agent Note: Session telemetry seam with mandatory redaction and the OTel backend

Status: implemented

English | 中文

Problem

Every deployment that wants harness sessions in an observability stack must hand-roll a session-log consumer: subscription, lifecycle handoff, and — hardest — redaction, since the raw log carries file contents and command output that may embed credentials. A telemetry seam and OTel backend shipped once on the session-telemetry-otlp-rfc branch (PR #222/#231) but never reached master: the proposal exported raw session events verbatim, which legal review declined. The capture-side design (backend contract, coordinator, handoff cursor, chunk projection) was sound and reviewed; the export-side stance was the blocker.

Decision

packages/telemetry/ revives the two reviewed packages under the SDK stance — the harness provides the capability, the deployment configures where records go and owns what leaves in them:

@deepseek-ai/dsh-session-telemetry — the seam. TelemetryBackend (emit/flush?/shutdown), the service-registered Telemetry form, and TelemetryCoordinator owning capture: adoption with cursor read-back, the per-append firehose (project → structuredClone → redact → emit, zero I/O), the fixed first-chunk-per-(turn, step) projection, the agent/error relay, and dispose-time shutdown records.
The telemetry/redact waterfall — the delta over the branch version. Every record passes it before reaching any backend; the seam ships NO rules of its own — the innermost next() is a pass-through, deployments mount their rules as listeners (stacking by transforming next()'s return value), and a throwing rule withholds the record fail-closed. Redaction applies to the exported copy only; the canonical log is never rewritten.
@deepseek-ai/dsh-session-telemetry-otel — the reference backend: OTel JS SDK log pipeline (LoggerProvider → BatchLogRecordProcessor → OTLP/HTTP exporter), configured verbatim through exporter/processor passthroughs. exporter.url is required and validated at load; unmounted or unconfigured, nothing leaves the process.

The boundary axiom holds: the harness's aspect ends at emit(). Batching, retry, queueing, and loss policy are the reporting SDK's, configured through passthroughs — delivery is best-effort (at-most-once across a crash), which the READMEs state plainly.

Alternatives considered

Implement the runtime-telemetry RFC's outbox (durable spool, per-sink cursors, at-least-once, a readCommitted persistence-seam method). Deferred, not rejected: the SDK stance makes delivery semantics the reporting SDK's territory, and the OTel SDK's own batch pipeline is the honest default. The outbox is a pure additive layer (the emit() contract does not move); revive it when a deployment states a crash-loss requirement telemetry must satisfy.

No in-process redaction point, delegating to receiver-side collector processors. Rejected — receiver-side redaction ships the secret first and scrubs it second. The waterfall puts an auditable, stackable scrubbing point before bytes leave the process; where the branch version (what PR #222 shipped) had no redaction point at all, every record now passes one.

A built-in conservative rule set as the waterfall's innermost next(). Rejected: as an SDK we cannot know which patterns are secrets in a given deployment, a shipped list invites false confidence ("redaction is on") while catching only known shapes, and false positives would corrupt exported bodies for consumers who never asked. The seam owns the mechanism; the deployment owns the policy — the innermost next() is a pass-through, and rules mount as listeners.

Map onto OTel spans (GenAI semantic conventions) instead of logs. Rejected for this revival: the branch implementation's log mapping is reviewed and shipped-shaped; the span model is lossy for forkable, interruptible sessions and belongs to a future consumer with real span queries to serve.

Forwarding the seam's turn-boundary flush() hint to the OTel provider's forceFlush(). Shipped in the first revival round, then removed after three review rounds each found a new silent-loss path in the same wrapper state: a dispose racing an in-flight flush (the SDK's concurrent-flush guard makes shutdown's internal drain skip), overlapping hints displacing the retained promise, and the provider's fixed 30-second flush timeout rejecting while the processor still drains. Every path exists only because the forwarding made this backend the process's second flusher against undocumented SDK internals from the upstream experimental tree; with no flush() implemented, the batch processor is the only flusher, its scheduledDelayMillis (already deployment-tunable through the processor passthrough) governs export cadence, and shutdown()'s drain is complete by construction. Reinstate only if a deployment states a turn-boundary latency requirement scheduledDelayMillis cannot meet — and then by calling the retained BatchLogRecordProcessor's own forceFlush(), never the provider's timeout-wrapped one.

Consequences

A deployment adds one cordis.yml entry with an OTLP endpoint and gets its session stream in any OTel-compatible stack; removing the entry is the opt-out, with no residual state. A rule-free deployment exports records exactly as captured — including any credentials embedded in file contents or command output — so a deployment crossing a trust boundary must mount telemetry/redact listeners, and both READMEs state this plainly. Where rules are mounted, exported bodies can differ from canonical log bytes, so receivers must not treat telemetry as a byte-exact replica; the log remains the source of truth. Crash durability is explicitly out of scope until the outbox decision above is revisited.

5.8 KiB Raw Blame History