Commit Graph

484 Commits

Author SHA1 Message Date
Hypatia May
d6da8ca29a fix(compact): decide step-alignment from surface tool-pairing, fire compaction pre-step (CBR-001)
Codex round 1 CBR-001: a head-anchored compaction checkpoint was
mis-classified by the log-position step-alignment scan, so a second
auto-compaction over a checkpoint-headed surface silently failed.

Root cause: `isStepAlignedStart/End` scanned the LOG by seq, but a
`replace` op lands a checkpoint at a high log seq whose SURFACE position
is the head — its log neighbours (the open step's assistant/message) are
not its surface neighbours, so the forward scan wrongly reported mid-step.

Fix, per the agreed direction:
- Replace the two log-position predicates with one surface-anchored
  helper `isToolPairingBalanced(nodes, events, beforeSeq)` in
  `dsh-session` (renamed step-boundary.ts → tool-pairing.ts). A cut is
  balanced when no unanswered tool-call precedes it on the surface; a
  region is collapsible iff both edges are balanced cuts. The open-tail
  and free-node cases fall out of the same counter. It also throws on a
  corrupt surface (a tool/result with no matching call).
- Move compaction off the in-step seam to a new "pre-step" seam fired
  after turn/start and before step/start, so a compaction's log-only
  compact/* records and its replacement node land cleanly OUTSIDE any
  step (the honest structure crash-safety relies on). Renamed the event
  agent/pre-request → agent/pre-step and switched its dispatch from
  parallel → serial (listeners mutate the surface as a side effect;
  serial isolates them so concurrent appends can't interleave). Extended
  the catalog generator to accept @mode serial.

Regression coverage: a real-loop test driving an auto-compaction asserts
the landed checkpoint is a balanced cut on both sides; unit tests pin the
checkpoint case, the mid-step injection case, multi-call steps, and the
corrupt-surface guard. Proven red on the old log-position logic.
2026-06-26 13:51:01 +08:00
Hypatia May
cec32faa4e refactor(compact): turn-agnostic retention + dedicated agent/pre-request seam
Reform the compaction blueprint so a runaway turn survives and the design
stops drifting across review rounds:

- Drop in-flight-turn protection ("layer 2"). Retention is a uniform tail→head
  whole-unit walk; the only structural guard is step-alignment. A single turn
  that alone exceeds the window now compacts its own early closed steps instead
  of being retained verbatim (the failure mode that motivated this).
- Move auto-compaction off the agent/request waterfall onto a new awaited
  agent/pre-request loop seam, fired before history derivation. Compaction
  mutates the surface; the loop derives once from the result — no double-derive,
  and a listener structurally cannot act on not-yet-derived messages.
- Tighten compactIfNeeded to required (session, system, model, signal).
- Enforce a single-pass convergence invariant in resolveConfig: reject configs
  where summarizationMaxTokens + retainTokens exceeds the threshold, so a
  compaction can never immediately re-trigger.
- Document the crash vs recoverable failure taxonomy; core session repair stays
  compaction-agnostic (a log-only orphaned compact/start is inert).
- Wire dsh-compact-basic into examples/coding-agent and add a with-key
  compaction e2e (compaction's first real-world exercise + runaway net).
- Rewrite the RFC to encode the blueprint and move it to implemented/.

The runaway-turn snapshot is a named deferred follow-up: dsh-llm-replay cannot
yet serve the interleaved summarization model call.
2026-06-26 08:59:33 +08:00
Hypatia May
aa9afcefc7 feat(compact-basic): baseline compaction backend (squashed from compact-basic)
Collapses the per-round review churn of the prior compact-basic branch into a
single clean baseline on top of compact-interface, so the upcoming retention
refactor lands as fresh, well-scoped commits rather than stacking on a history
of fixes that are being superseded.
2026-06-25 17:30:42 +08:00
Hypatia May
4c1b7191f3 Merge branch 'session-surface' into compact-interface 2026-06-25 10:07:37 +08:00
Hypatia May
b955c7d18d Merge remote-tracking branch 'origin/master' into session-surface 2026-06-25 09:50:37 +08:00
Hypatia May
cf70141486 Merge branch 'session-surface' into compact-interface
# Conflicts:
#	docs/core-data-structures/core.md
#	packages/README.md
#	scripts/type-equiv.manifest.json
#	tsconfig.base.json
#	tsconfig.typecheck.json
2026-06-25 09:10:50 +08:00
Tianyi Cui
329371a529 Merge pull request #61 from deepseek-ai/codex/fix-stdio-readline-terminal
fix(examples): enable terminal editing in demo agents
2026-06-24 22:09:45 +08:00
Hypatia May
828c3f85c9 fix review findings: skip collided SCHEMA_VERSION 3; reject marker-less surface events
P1: both merge parents shipped SCHEMA_VERSION=3 for different layouts (surface
columns vs seed_length), so an on-disk 3 was ambiguous and wrongly accepted.
Bump to 4 (merged layout) so the version check rejects both sibling v3s.

P2: a surface-eligible event with no surfaceOp lands in the log but vanishes
from deriveMessages() (surface is the sole derivation path). The typed append
overload enforces the marker only when the type arg is a literal; it collapses
to optional when widened to the union (a caller iterating raw events). Guard at
runtime in both append() and the seed constructor — no backward-compat for
surface-less logs. Shared seed fixtures carry surfaceOp explicitly and the
appendLog helper forwards it verbatim (no synthesized default). Exports
isSurfaceEligibleType. Regression tests for all three, each verified to fail
on the unfixed code.

Gates: typecheck, test (1115), snapshot (14), doc-sync, lint, build, hygiene green.
2026-06-24 17:45:48 +08:00
Hypatia May
9cc8dc371e Merge remote-tracking branch 'origin/master' into session-surface
Reconciles the session-surface work (surfaceOp/sourceEventSeqs provenance as
the sole derivation path) with master's worktree-subagent series (fork-seed
boundary + out-of-process subagent backends).

Semantic reconciliations beyond the textual auto-merge:
- SQLite SCHEMA_VERSION: both sides bumped 2->3. Merged to a single v3 carrying
  BOTH column families — master's seed_length on `sessions` and surface's
  source_event_seqs/surface_op on `events`. writeRow + both INSERT sites bind
  the full set; the schema doc lists all three added columns as the v2->v3 gap.
- agent-loop runStep request: master's `sessionId: session.id` and surface's
  per-append surfaceOp/sourceEventSeqs coexist (different regions).
- Fork seed + surface: a fork seeds the child from the parent's LIVE events,
  which now carry surfaceOp, so the child's surface rebuilds correctly. Verified
  end-to-end — the subagent-fork replay recalls the inherited "SAFFRON" codeword
  through the seeded prefix.
- Subagent snapshot fixtures (recorded pre-surface) re-enriched via KEYLESS
  deterministic replay: only surfaceOp/sourceEventSeqs added onto existing
  recorded lines (matched by seq), no recorded value changed. Not re-recorded
  against the live API.

Gates: typecheck, test (1112), test:snapshot (14), doc-sync, lint, build,
hygiene all green.
2026-06-24 10:48:01 +08:00
Tianyi Cui
4cfa22f997 Merge pull request #101 from deepseek-ai/worktree-subagent-fork-scenario
Subagent seam (follow-up): record fork + mixed spawn/fork snapshot scenarios
2026-06-23 23:40:36 +08:00
Tianyi Cui
e6de8d1e9d Merge remote-tracking branch 'origin/master' into worktree-subagent-fork-scenario 2026-06-23 23:40:05 +08:00
Tianyi Cui
ddb240f3b1 Merge pull request #100 from deepseek-ai/worktree-subagent-fork-replay
Subagent seam (follow-up): persist the seed boundary so fork-child replay routes correctly
2026-06-23 23:39:14 +08:00
Tianyi Cui
cdfbf6f631 Merge remote-tracking branch 'origin/master' into worktree-subagent-fork-replay 2026-06-23 23:38:54 +08:00
Tianyi Cui
91d9b5e415 Merge pull request #95 from deepseek-ai/worktree-subagent-seam-pr3
Subagent seam (PR3): ACP backend (out-of-process delegation)
2026-06-23 23:38:15 +08:00
Tianyi Cui
18a56f041d Merge remote-tracking branch 'origin/master' into worktree-subagent-seam-pr3 2026-06-23 23:37:57 +08:00
Tianyi Cui
3e2b90fe7d Merge pull request #94 from deepseek-ai/worktree-subagent-seam-pr2.5
Subagent seam (PR2.5): per-session snapshot replay for nested agents
2026-06-23 23:37:31 +08:00
Tianyi Cui
7152184342 Merge remote-tracking branch 'origin/master' into worktree-subagent-seam-pr2.5 2026-06-23 23:37:07 +08:00
Tianyi Cui
e945219df4 Merge pull request #92 from deepseek-ai/worktree-subagent-seam-pr2
Subagent seam (PR2): in-process spawn + fork backends
2026-06-23 23:36:49 +08:00
Tianyi Cui
921362364c Merge remote-tracking branch 'origin/master' into worktree-subagent-seam-pr2 2026-06-23 23:36:26 +08:00
Tianyi Cui
4a9449463b Merge pull request #90 from deepseek-ai/worktree-subagent-seam-pr1
Subagent capability seam (PR1): interface, mock backend, model-facing tool
2026-06-23 23:35:29 +08:00
Hypatia May
442f85469e refactor(compact): align compact/summary and CompactionResult on shadowed* naming 2026-06-23 16:51:20 +08:00
Hypatia May
894653763e docs(compact): add the compact group/service to package and architecture docs 2026-06-23 16:40:24 +08:00
Hypatia May
58d798492a docs(compact): catalog the compaction seam in core-data-structures 2026-06-23 16:33:05 +08:00
Yichen Jiang
01dea07922 Merge origin/master into codex/fix-stdio-readline-terminal 2026-06-23 16:27:49 +08:00
Hypatia May
bb013e934e docs(compact): bracket the surface mutation inside the compaction lock 2026-06-23 16:13:55 +08:00
Hypatia May
f4180bd764 feat(compact): add optional cancellation signal to the compact seam methods 2026-06-23 15:52:41 +08:00
Hypatia May
bdaf9f651d Merge branch 'session-surface' into compact-interface 2026-06-23 13:59:04 +08:00
Hypatia May
358ae02c56 feat(invariants): enforce replace provenance — sourceEventSeqs must cover every shadowed surface node 2026-06-23 13:37:13 +08:00
Hypatia May
1ec8c40d0d refactor(surface): use nodeBySeq map for lookup in _replace, drop dead params 2026-06-23 13:26:45 +08:00
Hypatia May
6089e226bc refactor(session): make surface the sole derivation path, drop legacy fallback 2026-06-23 13:05:59 +08:00
Hypatia May
09d497d5c5 docs(rfc): clarify compaction rides the replace op on an existing event type 2026-06-23 09:54:58 +08:00
Hypatia May
30805e1983 fix(sqlite): bump SCHEMA_VERSION to 3 for the new surface columns 2026-06-23 09:48:15 +08:00
Tianyi Cui
78d60366ec Record fork and mixed spawn+fork snapshot scenarios
The seed-boundary change made fork-child replay route correctly but shipped
with no recorded fork scenario — the seedLength slice was exercised only by
llm-replay unit tests and a persistence round-trip, never by the full-transcript
snapshot tier. Add two recorded scenarios that drive a real fork child through
it:

- subagent-fork: parent completes a turn, then forks one child (child fixture
  carries a non-zero seedLength, the boundary the replay slice consumes).
- subagent-mixed: parent completes a turn, then delegates once via spawn
  (seedLength 0) and once via fork (non-zero seedLength) in one transcript —
  the first scenario to drive two subagent backends at once, exercising both
  branches of the slice.

Both need a completed turn-1 so the fork seed is a non-empty completed-turn
prefix (a turn-1 fork seeds empty = spawn, which would not exercise the slice).
Removing the slice turns both scenarios red (the fork child receives the
parent's recorded chunks), proving the guard bites.

ACP (out-of-process) subagent replay remains a different shape, still tracked
as TODO(acp-subagent-replay).
2026-06-22 21:20:54 +08:00
Tianyi Cui
b3d40d427e Persist the seed boundary so fork-child replay routes correctly
A fork subagent seeds its child session with a prefix of the parent's log, and
that seed becomes the child's persisted log — so a fork child's .jsonl begins
with the PARENT's events, including the parent's assistant/chunk events. The
snapshot replay harness derived a child's script from its whole log, which would
replay the parent's recorded responses as the child's model calls. Spawn-only
scenarios never hit it, but a fork snapshot would mis-route silently.

Record the seed boundary and skip the inherited prefix at replay:

- SessionHeader gains an optional `seedLength` (how many leading events were
  inherited via a seed), threaded through CreateSessionOptions/CreateAgentOptions
  meta and stamped by the fork backend (= seeded-prefix length; absent for spawn).
  It is EXPLICIT, never inferred from seed.length: a resume seeds the whole stored
  log, so the resume path passes the persisted boundary back.
- Both persistence backends round-trip it: JSONL header line, SQLite seed_length
  column. The SQLite table change bumps SCHEMA_VERSION 2->3; per the pre-release
  stance the backend rejects an older user_version on open with NO migration.
- llm-replay's parseSessionHeader reads seedLength and loadSessionScripts derives
  a child script from events AFTER the boundary. seedLength is 0 for spawn, so
  spawn replay is byte-for-byte unchanged.

Closes the routing-correctness gap the per-session snapshot replay RFC under-
stated; a recorded fork scenario remains a future addition but now derives
correctly. RFC: docs/rfc/implemented/testing/2026-06-22-fork-child-replay-seed-boundary.md.

Regression coverage: a fork child fixture whose seeded prefix carries a parent
chunk (derived script must exclude it, proven red without the slice); a seedLength
persistence round-trip through the shared coordinator contract (both backends);
the fork backend stamping it; resume preserving it from the persisted header.
2026-06-22 20:55:32 +08:00
Tianyi Cui
c4ba1bd65a Merge branch 'worktree-subagent-seam-pr2.5' into worktree-subagent-seam-pr3 2026-06-22 17:07:43 +08:00
Tianyi Cui
8dc211beed Merge branch 'worktree-subagent-seam-pr2' into worktree-subagent-seam-pr2.5 2026-06-22 17:05:19 +08:00
Tianyi Cui
4e87a533a7 Merge branch 'worktree-subagent-seam-pr1' into worktree-subagent-seam-pr2 2026-06-22 17:02:12 +08:00
Tianyi Cui
34f6f28716 Make fork reachable by the model in the acp-agent demo (review feedback)
The acp-agent cordis configs loaded the fork backend but bound only one
dsh-tool-subagent (to spawn), so the comment's claim that a multi-child scenario
could exercise both transports was false — fork was loaded but unreachable by
the model. Register a second dsh-tool-subagent bound to fork with a distinct
toolName (subagent_fork), matching the coding-agent demo, in both cordis.yml
(record/demo) and cordis.snapshot.yml (replay). Snapshot goldens are unchanged
(the transcript does not capture the available-tool list).
2026-06-22 17:00:59 +08:00
Tianyi Cui
083af62785 Settle ACP cancel without the child's cooperation; preserve flattened errors (review feedback)
Two findings on the ACP backend:

Blocking: cancel() only sent session/cancel, so a child that ignores the notify
or wedges the prompt left result hung forever — the model-facing tool awaits
result before its finally disposes, so the parent cancellation hung and the
child stayed alive, violating the SubagentRun.cancel() contract (result settles
aborted). The result path now races the ACP drive against a cancelSettled
promise that requestCancel resolves, so result settles aborted the instant a
cancel is requested, regardless of the child. dispose() still kills+reaps the
process. New MOCK_IGNORE_CANCEL mock mode (receives cancel, never resolves the
prompt, never exits) drives a regression proven to hang without the race.

Nit: the drive-path catch was an empty broad catch that discarded the error
(AGENTS.md forbids). Because cancellation is now handled by the race arm, a
rejection reaching the catch is always a genuine child-level error — bind it,
flatten to error, and surface the original via a new AcpRunSpec.onError sink
that the provider wires to ctx.logger.warn, so a real fault is preserved.
2026-06-22 16:57:38 +08:00
Tianyi Cui
8723186398 Honor an already-aborted signal in the subagent tool bridge (review feedback)
addEventListener('abort') does not fire for a signal already aborted before the
listener is added, so a parent step cancelled before the subagent tool ran
would never reach the child — the tool leaned on each provider re-checking
request.signal itself, leaving the bridge's own claim incomplete for any
provider that relies on run.cancel(). Re-check exec.signal.aborted right after
registering and cancel explicitly. Regression test uses a spy provider that
only reacts to cancel() (never inspects the signal); proven to hang without the
fix (result never settles) and settle aborted with it.
2026-06-22 16:32:10 +08:00
Hypatia May
e45053f0f5 feat(compact): compaction capability seam — abstract CompactService interface
Adds the @deepseek-ai/dsh-compact interface package: the abstract
CompactService (ctx.compact) with compactIfNeeded / compactRegion, the
compact/* session-event types via SessionEventMap declaration merging, and the
capability-seam RFC. Wires the package into the three root tsconfigs and the
cordis catalog. A backend implementation lands separately.
2026-06-22 15:15:27 +08:00
Tianyi Cui
2086804b7e Merge branch 'worktree-subagent-seam-pr2.5' into worktree-subagent-seam-pr3
# Conflicts:
#	docs/module-graph.md
#	packages/subagent/README.md
2026-06-22 14:58:03 +08:00
Tianyi Cui
e46ebcec89 Merge branch 'worktree-subagent-seam-pr2' into worktree-subagent-seam-pr2.5 2026-06-22 14:45:09 +08:00
Tianyi Cui
67c7ef791f Apply the ts-build-config (lib/types) convention to the new subagent packages
Master's #36 moved declaration output to lib/types (and types/exports/files
point there). The merge applied that to all pre-existing packages, but the
subagent backends introduced on this stack (subagent-inprocess, subagent-spawn,
subagent-fork) still used the old lib/ layout. Bring them onto the new
convention and add them to the single typecheck tsconfig.json references.
2026-06-22 14:43:51 +08:00
Tianyi Cui
398b4f2ccf Merge branch 'worktree-subagent-seam-pr1' into worktree-subagent-seam-pr2 2026-06-22 14:36:57 +08:00
Tianyi Cui
8d111161de Merge remote-tracking branch 'origin/master' into worktree-subagent-seam-pr1
# Conflicts:
#	tsconfig.typecheck.json
2026-06-22 14:35:35 +08:00
Tianyi Cui
0c9ea3145f Extract the shared in-process driver into dsh-subagent-inprocess (review feedback)
The shared run driver lived inside dsh-subagent-spawn, so the spawn package
carried fork-aware seeding logic and dsh-subagent-fork depended backward on
dsh-subagent-spawn — the two in-process backends were not independent.

Move the driver (startInProcessRun, depthOf, SubagentDepthError,
InProcessRunOptions) into a new pure-library package
@deepseek-ai/dsh-subagent-inprocess that registers nothing. spawn and fork now
both depend only on that driver and neither knows about the other; spawn no
longer re-exports it and fork no longer imports from spawn.

Also wire BOTH backends in examples/coding-agent/cordis.yml (config-only): load
dsh-subagent-spawn + dsh-subagent-fork + two dsh-tool-subagent instances with
distinct toolNames (subagent → spawn, subagent_fork → fork), demonstrating that
exposing multiple transports needs no code change.
2026-06-22 14:27:38 +08:00
Tianyi Cui
98f272de25 Merge pull request #36 from deepseek-ai/feat/adr0016-type-build-check
RFC: Unified TypeScript Build and Config
2026-06-22 13:40:10 +08:00
Tianyi Cui
2ff112962b Widen the dispose EOF grace past nested-teardown headroom; prove the SIGTERM rung (Codex review round 3)
Two round-3 findings:

(A) The EOF-quiesce window reused the 3000ms SIGTERM grace, the SAME value as
dsh-bash-local's own SIGTERM->SIGKILL grace. The child acp-agent's EOF teardown
disposes its loop, which stops child-owned bash -- and a SIGTERM-trapping bash
grandchild can hold that for up to ~3s before its own SIGKILL, then the child
still owes a final flush. With both graces equal, the parent's SIGTERM fired
exactly as the child reached its own SIGKILL+flush, cutting it off. Split the
EOF grace into its own knob (disposeEofGraceMs, default 6000ms) that exceeds a
single signal-grace of nested-teardown headroom. The child is an arbitrary ACP
agent, so the value is a standalone generous default, NOT derived from any
child's internals. Tier-1 test now uses a flush that outlasts the SIGTERM grace
but fits the EOF grace, so it lands only because the EOF tier honors its own
wider window (proven RED when tier 1 reuses the small SIGTERM grace).

(B) The middle-tier (SIGTERM) test only asserted dispose returned in time, so
an EOF->SIGKILL ladder with the rung removed would still pass. The mock's
MOCK_IGNORE_EOF mode now installs a SIGTERM handler that touches an observable
marker before exiting; SIGKILL is uncatchable, so removing the SIGTERM rung
leaves the marker absent (proven RED). The test asserts the marker exists.
2026-06-22 13:29:23 +08:00
Tianyi Cui
3a67d0a882 docs: sync development CI gate docs 2026-06-22 12:54:11 +08:00