Commit Graph

134 Commits

Author SHA1 Message Date
Yichen Jiang
2688df99cd Merge remote-tracking branch 'origin/master' into codex/rfc-subagent-background-tasks
# Conflicts:
#	examples/acp-agent/tests/snapshots/both-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/code-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/skill-load/session.jsonl
#	examples/acp-agent/tests/snapshots/text-turn/session.jsonl
#	examples/sandbox-acp-agent/tests/snapshots/escalation-approved/session.jsonl
#	examples/sandbox-acp-agent/tests/snapshots/escalation-rejected/session.jsonl
#	examples/sandbox-acp-agent/tests/snapshots/mode-switching/session.jsonl
2026-07-12 16:09:22 +08:00
Yichen Jiang
df9617aaff feat(bash): generalize managed shell environment 2026-07-12 15:41:42 +08:00
Yichen Jiang
acdbe7b828 Merge remote-tracking branch 'origin/master' into codex/agent-session-jsonl-location
# Conflicts:
#	examples/acp-agent/tests/snapshots/both-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/code-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/skill-load/session.jsonl
#	examples/acp-agent/tests/snapshots/text-turn/session.jsonl
#	examples/sandbox-acp-agent/tests/snapshots/mode-switching/session.jsonl
2026-07-12 13:01:26 +08:00
Yichen Jiang
740106a51a Merge remote-tracking branch 'origin/master' into codex/rfc-subagent-background-tasks
# Conflicts:
#	docs/config-catalog.md
#	docs/cordis-catalog/services.md
#	docs/core-data-structures/bash.md
#	docs/core-data-structures/core.md
#	docs/event-producer-consumer.md
#	docs/module-graph.md
#	docs/tool-catalog.md
#	examples/acp-agent/tests/snapshots/both-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/code-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/text-turn/session.jsonl
#	packages/README.md
#	packages/bash/README.md
#	packages/bash/bash-local/src/index.ts
#	packages/bash/bash/README.md
#	packages/bash/bash/package.json
#	packages/bash/bash/src/index.ts
#	packages/bash/bash/src/types.ts
#	packages/bash/bash/tests/service.spec.ts
#	packages/bash/bash/tsconfig.json
#	packages/bash/tool-bash/README.md
#	packages/bash/tool-bash/src/index.ts
#	packages/bash/tool-bash/tests/tools.spec.ts
#	packages/bash/tool-bash/tsconfig.json
#	packages/cordis/tool-cordis/src/api-catalog.ts
#	packages/core/agent-core/README.md
#	packages/core/agent-core/package.json
#	packages/core/agent-core/src/index.ts
#	packages/core/agent-core/tests/agent-core.spec.ts
#	packages/core/tools/tests/gen-tool-catalog.spec.ts
#	packages/hooks/hook-protocol/tests/runner.spec.ts
#	packages/ui/acp-agent/tests/acp-agent.spec.ts
#	packages/ui/stdio-agent/tests/stdio-agent.spec.ts
#	pnpm-lock.yaml
#	scripts/doc-budgets.manifest.json
#	scripts/gen-tool-catalog.ts
2026-07-11 23:04:27 +08:00
Yichen Jiang
e1f317dcce Merge remote-tracking branch 'origin/master' into codex/agent-session-jsonl-location
# Conflicts:
#	docs/module-graph.md
#	docs/tool-catalog.md
#	examples/acp-agent/tests/snapshots/both-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/code-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/text-turn/session.jsonl
#	packages/bash/tool-bash/README.md
#	packages/bash/tool-bash/package.json
#	packages/bash/tool-bash/src/index.ts
#	packages/cordis/tool-cordis/src/api-catalog.ts
2026-07-11 22:57:27 +08:00
Tianyi Cui
da9cef381c Merge remote-tracking branch 'origin/master' into codex/system-prompt-snapshots
# Conflicts:
#	examples/acp-agent/tests/snapshots/both-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/code-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/text-turn/session.jsonl
#	packages/support/acp-snapshot/src/suite.ts
2026-07-11 22:48:14 +08:00
Tianyi Cui
169f86966a fix(review): lint prompt Markdown paragraphs 2026-07-11 22:33:33 +08:00
Tianyi Cui
3baaecc078 Merge remote-tracking branch 'origin/master' into codex/skill-system
# Conflicts:
#	docs/rfc/INDEX.md
#	examples/acp-agent/tests/snapshots/both-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/code-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/text-turn/session.jsonl
#	packages/README.md
2026-07-11 22:31:28 +08:00
Tianyi Cui
9339622d3b test(acp): snapshot system prompts as Markdown 2026-07-11 22:24:20 +08:00
Yichen Jiang
eea0a99985 feat: expose agent session log location 2026-07-10 20:52:27 +08:00
kingwl
2eed448acf feat(bash): the sandboxed executor — per-call policy carrier, denial facts, runner-failure classification
dsh-bash grows the per-call policy carrier: BashExecRequest.sandboxMode
(request-optional, spec required-but-nullable — the owner pattern; resolve()
is the one explicit defaulting step) and the BashExecutor.sandboxMode
capability fact (undefined in the base class — composition truth the tool
layer can read). dsh-bash-local carries the field verbatim and confines
nothing.

dsh-bash-sandbox extends LocalBashExecutor and hands ctx.sandbox the exact
argv it is about to spawn. A denial is a RESULT FACT (the command RAN;
result.sandbox.denied is orthogonal to exitCode/signal), classified
conservatively against the wrap own dialect; a RUNNER failure outranks
denial — foreground re-throws the structured SANDBOX_UNAVAILABLE, a settled
background task stamps sandbox.runnerFailed — so a broken sandbox never
reads as a failing command and the command never runs unconfined.
dsh-tool-bash renders the markers and teaches the model not to retry around
a policy denial; escalation and per-session switching are staged follow-ups.
2026-07-10 15:43:02 +08:00
Yichen Jiang
680ae53bb6 Merge remote-tracking branch 'origin/master' into codex/skill-system
# Conflicts:
#	docs/architecture.md
#	docs/config-catalog.md
#	docs/module-graph.md
#	docs/rfc/INDEX.md
#	examples/acp-agent/tests/snapshots/text-turn/session.jsonl
#	packages/core/agent-core/src/index.ts
#	packages/core/tools/tests/gen-tool-catalog.spec.ts
#	packages/support/acp-snapshot/src/suite.ts
#	packages/ui/acp-agent/src/index.ts
2026-07-10 14:43:33 +08:00
Yichen Jiang
6292d52236 feat(skill): move catalogs into session prefixes 2026-07-10 14:19:06 +08:00
Yichen Jiang
f42cd49337 Merge remote-tracking branch 'origin/master' into codex/rfc-subagent-background-tasks
# Conflicts:
#	docs/architecture.md
#	docs/module-graph.md
#	examples/acp-agent/tests/snapshots/both-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/code-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/text-turn/session.jsonl
#	packages/README.md
#	packages/core/tools/tests/gen-tool-catalog.spec.ts
#	scripts/doc-budgets.manifest.json
#	tsconfig.base.json
#	tsconfig.build.json
#	tsconfig.json
2026-07-10 10:13:13 +08:00
Yichen Jiang
b9bf67d0a7 Merge remote-tracking branch 'origin/master' into codex/skill-system 2026-07-09 23:11:10 +08:00
Yichen Jiang
184e164091 feat(tasks): background task runtime, generic task_* control tools, bash/subagent producers
One shared ctx.tasks registry (branded <kind>-N ids, owner-fenced
read/kill/wait/list, attachSurface misconfiguration fence, reported-flag
notice dedup, atomic register) + dsh-tool-tasks (task_output/task_list/
task_kill, completion-notice injection, background prompt habit).
Producers opt in via their own enableRunInBackground config: bash
(stream kind; seam slimmed to resolve/run/start returning a BashProcess
handle, bash_output/bash_kill deleted) and subagent (final-output kind;
done settles after run.dispose()). Owner disposal drains tasks through
the new awaited ctx.agents.onCleanup seam in the loop's disposal chain.
Both RFCs moved to implemented/; docs, catalogs, snapshots re-pinned.
2026-07-09 21:22:54 +08:00
imccyu
f91bfc1fcf test: re-record the header pin and workflow-run for the meta-parameter schema
The workflow tool's wire schema changed (required meta object
parameter; body-only script). Request-header content is pinned by
exactly ONE scenario (text-turn) and scrubbed to {{system}}/{{tools}}
tokens everywhere else, so the schema change re-records exactly two
fixtures:

- text-turn — the pinned header itself (the one committed copy of the
  tool schemas; every other scenario's live header is asserted equal to
  this pin by the uniformity guard).
- workflow-run — its recorded interaction used the removed call shape
  (meta embedded in the script), which the engine now rejects; the
  authored prompt is updated to dictate the new shape (meta as a
  parameter, body-only script) and the scenario re-recorded to a clean
  single call.

Every other fixture stays byte-identical and replays green against the
new pin. Known pre-existing exception: fs-policy-reject's recording
carries a GNU-only sed -i fallback that fails BSD/macOS replay — kept
out of this change deliberately (the snapshot CI lane is ubuntu).
2026-07-09 20:39:28 +08:00
Yichen Jiang
f453ba77a2 refactor: split skill providers 2026-07-08 15:50:38 +08:00
imccyu
773ecf03f5 workflow: drop the determinism bans (unimplemented-resume pre-support)
The Date.now()/Math.random()/argless-new-Date() bans existed solely to
keep scripts resume-compatible for the deferred journaling/resume
feature. Pre-support for an unimplemented feature is speculative cost:
scripts may now read the clock freely; implementing resume reintroduces
the bans as a script-contract tightening. The RFC's shipped-state
description is updated in place, the tool DESCRIPTION drops the
constraint sentence (the pinned text-turn header follows), and the
engine README's trust-premise paragraph now leans on absent globals
alone.
2026-07-08 00:51:19 +08:00
Tianyi Cui
737e1f7c0a chore: absorb master's explicit tool order into the stack
The tool-order feature canonicalizes the model-facing list (alphabetical
absent a configured toolOrder), so the header-pinning text-turn fixture is
re-recorded on the stacked tree — every other fixture stores the header as
scrubbed tokens and needed nothing. AGENTS.md condensed back under its
ceiling after the merge union.
2026-07-07 23:22:05 +08:00
imccyu
b67cda482a fix: update snaphsot 2026-07-07 16:45:27 +08:00
Tianyi Cui
7a35afa6fe Merge branch 'structured-output-subagent-seam' into worktree-dynamic-workflows
Restack on the carved-out foundation (#192), per review feedback on #170.
The seam files resolve to the carve-out's revision — its prompt-order
neutrality fix (backends no longer inject 'tools'; the structured runtime
gates its own capture-tool registration) restores the subagent tools to
master's front position, so every recorded fixture is re-recorded on the
stacked tree and the authored error-finish/cancel headers re-patched to the
stacked tool list ([subagent, subagent_fork, workflow, todo_write, ...]).
2026-07-06 23:43:13 +08:00
Tianyi Cui
ea58bdf1e8 Merge remote-tracking branch 'origin/master' into worktree-dynamic-workflows
Master's reconstructable-requests overhaul (#179) meets the workflow tool:
- subagent-inprocess structured-output nudge becomes a system-prompt section
  plus logged context (the injected-request waterfall shape is gone upstream)
- snapshot fixtures re-recorded on the merged tree so every request/header
  carries the workflow tool; authored error-finish/cancel headers patched to
  the merged tool list and system text
- architecture.md condensed back under its word ceiling; module graph regenerated
2026-07-06 22:28:57 +08:00
Yichen Jiang
be8874f8db test: reduce ACP skill snapshot churn 2026-07-06 17:22:17 +08:00
Yichen Jiang
0ba14f4a3c Merge remote-tracking branch 'origin/master' into codex/skill-system
# Conflicts:
#	docs/event-producer-consumer.md
2026-07-06 17:11:35 +08:00
Tianyi Cui
7376346246 test: re-record snapshot goldens — every transcript gains its request/header event
One re-record after the header events landed: recorded scenarios
re-harvested against the live API; the three fs-writing fixtures are
relativized (this recording's model happened to emit absolute
file_path arguments, which only round-trip through replay when the
path is cwd-independent) and, with the two never-re-recorded authored
scenarios (error-finish, cancel), rewritten in the normalized
authored-fixture form the harness documents — each now carrying the
request/header snapshot the loop logs before its first dispatch.
Keyless replay verified green across all 35 scenarios.
2026-07-06 03:39:48 +08:00
Hypatia May
0298f5c6f0 Merge remote-tracking branch 'origin/master' into session-surface
Reconcile the session-surface feature with master's package reorg and
simplifications:

- Adopt master's folded usage (assistant/message.usage; standalone `usage`
  event dropped) and re-attach surface metadata (surfaceOp/sourceEventSeqs).
- Add surface opts to master's new max-tokens assistant/message append.
- Port surface columns onto the coordinator-refactored SQLite backend at its
  new path; drop the dead v1->v2 migration (bump-and-reject, no migration per
  pre-release policy).
- Move the session-surface RFC into implemented/architecture/ and refresh its
  stale body (no migration, SESSION_FORMAT_VERSION=0, renamed package paths).
- Update the core-data-structures catalog SessionEvent blocks for the two new
  surface fields; regenerate the cordis catalog.
- Re-harvest ACP snapshot fixtures (keyless replay) to carry surface metadata.
2026-06-22 10:35:59 +08:00
Tianyi Cui
655206848d Merge PR5 (trace-event collapse + v0 format pin) into PR6
# Conflicts:
#	examples/acp-agent/tests/snapshots/cancel/session.golden.jsonl
#	examples/acp-agent/tests/snapshots/cancel/session.jsonl
#	examples/acp-agent/tests/snapshots/error-finish/session.golden.jsonl
#	examples/acp-agent/tests/snapshots/error-finish/session.jsonl
#	examples/acp-agent/tests/snapshots/multi-turn/session.golden.jsonl
#	examples/acp-agent/tests/snapshots/text-turn/session.golden.jsonl
#	examples/acp-agent/tests/snapshots/tool-call-turn/session.golden.jsonl
#	examples/acp-agent/tests/snapshots/workspace-edit/session.golden.jsonl
2026-06-21 11:12:48 +08:00
Tianyi Cui
b0422f2a50 fix review findings: bump session format version + restore late turn-end warn
Codex review of the trace-event fold found two merge-blockers.

Blocker #1 — format version. Folding usage onto assistant/message and removing
the standalone usage/error events changed the persisted SessionEventMap shape,
which per the AGENTS.md "bump the version and reject — don't migrate" policy
requires a backend to reject any non-current log. Centralize the version in an
exported SESSION_FORMAT_VERSION constant (dsh-session), read by both write sites
(Session constructor default, SessionStore.prepare header) and the coordinator's
load-time assertVersion check. The constant is pinned at 0: while unreleased the
on-disk format is unstable/pre-release, so breaking shape churn is absorbed at v0
(no monotonic bump until the first tagged release) and any non-0 log is rejected
on load — no migration. Update every test/fixture/doc that stamps a
currently-written header to the constant, bump the ACP snapshot fixture + golden
headers to v0, and keep the version-rejection test meaningful by switching its
bad value to a clearly non-current 99. AGENTS.md documents both the monotonic
(SQLite SCHEMA_VERSION) and pinned-0 (session log) pre-release stances.

Blocker #2 — restore the late turn-end warn. failTurn now sets the error reason
only while the turn is still open; once turn/end is appended (a throwing
agent/turn-end listener after closeTurn) the reason can no longer reach the
durable log, so the late throw is logged via ctx.logger.warn instead of
vanishing into a futile post-close assignment. A regression test asserts the
warn fires.

Also guard the normal-step assistant/message append with the same
content-or-usage condition as the max-tokens branch (a content-less, usage-less
step records no trace-only row), with a covering test.
2026-06-21 11:08:10 +08:00
Tianyi Cui
4209e4af3f test(snapshot): use session.jsonl as the only session-log artifact (drop session.golden.jsonl)
Model-driving ACP snapshot scenarios shipped both session.jsonl (the
replay fixture) and session.golden.jsonl (the expected re-persisted log).
For recorded scenarios the normalized fixture and golden were byte-identical
— pure duplication. Remove session.golden.jsonl entirely: every model
scenario now has at most one committed session-log artifact, session.jsonl,
which doubles as the replay source AND the expected produced log.

The snapshot test compares the replay run's persisted log against the
session.jsonl fixture, normalizing BOTH sides — but each against its OWN
volatile values, not a shared context. A raw harvested fixture bakes in the
recording run's session id / cwd / timestamps, distinct from the live replay
run's; since normalizeSessionLog scrubs cwd by exact string match, the
fixture must be normalized against its own header (new fixtureContext helper)
or its stale recorded cwd would leak unscrubbed and the compare would fail.
The session side uses a normalized-string toEqual, NOT toMatchFileSnapshot,
so a run never overwrites the fixture.

Authored override scenarios (error-finish, cancel) now hold their expected
produced log in session.jsonl. Verified llm-replay ignores the fixture for
model chunks when an override exists: loadReplayScript() returns the override
array and never reads config.file, so committing the full expected log there
does not affect replay behavior.

The required-fixture guard is now per-kind: every scenario needs input.json +
stdout.golden.jsonl; model scenarios need session.jsonl; authored ones
additionally need replay.override.json. Updates the ACP-snapshot-tests RFC to
the reduced fixture set and moves the proposing RFC proposed -> implemented.
2026-06-21 10:36:57 +08:00
Tianyi Cui
2be60b9a22 simplify(session): fold trace-only usage/error events into load-bearing events
The session event vocabulary carried two standalone trace-only events that
were not load-bearing as separate records. Fold their facts into nearby
load-bearing events and delete the standalone variants.

- Token usage now rides on `assistant/message` as an optional `usage` field —
  the assembled model output and its accounting travel together. The loop folds
  `assembler.usage` onto the append instead of emitting a separate `usage`
  event.
- The max-tokens path is the no-data-loss host: a step cut off with usage but
  EMPTY content (e.g. only a dropped tool call) previously emitted a standalone
  `usage`; it now records an empty-content `assistant/message { content: [],
  usage }`. `deriveMessages()` skips empty-content assistant messages, so the
  usage host never injects a spurious content-less assistant turn into the
  provider transcript. A step with neither content nor usage appends nothing.
- An operational error's step number now rides on `turn/end.reason` for
  `kind: 'error'` (`{ kind: 'error', step, message, code? }`) — the durable
  turn outcome ACP and resume already consume. `failTurn` sets the reason
  directly (no separate session `error` event). `agent/error` + logging are
  unchanged for live diagnostics.
- No format-version bump: pre-release, no persisted data, so per the format
  policy there is nothing to migrate or reject (the RFC's "refresh the format
  version" criterion over-reached). `version` stays 1.
- ACP fixtures + goldens re-recorded (keyless replay): dropped standalone
  usage/error lines, usage folded onto assistant/message, error step on
  turn/end.reason.

RFC moved proposed -> implemented with an implementation note recording the two
scope refinements.
2026-06-21 10:00:06 +08:00
Tianyi Cui
72df4d2e16 test(acp-example): re-record snapshot goldens after merging master
Master's "fix(acp): align prompt and workspace contracts" changed the editor-
facing transcript — `user/message` → `user_message_chunk` is now emitted during
session/load replay ONLY, not live streaming, so a live prompt no longer echoes
the user message back. The snapshot tier caught this (6 goldens shifted); this
re-records the four recorded scenarios against the API and re-accepts the two
authored goldens so they reflect the merged behavior. Full suite green and
deterministic; this is the tier working as designed.
2026-06-19 10:07:06 +08:00
Tianyi Cui
ec73e11c9b refactor(acp-example): snapshot goldens are JSONL, not pretty-printed .txt
The goldens now mirror the shape of the surfaces they capture — one compact
JSON record per line — matching the wire (NDJSON stdout) and disk (JSONL
session log) formats, renamed *.golden.jsonl. They stay grep/jq-able and
faithful to what the agent emits, where the prior pretty-printed .txt was a
reformatted representation. Both normalizers drop the 2-space indent; the
normalizer spec asserts the compact form. All 11 goldens regenerated; replay
remains deterministic (8/8 across runs).
2026-06-19 09:29:41 +08:00
Tianyi Cui
c94f1563f5 test(acp-example): five snapshot scenarios + cancel/error input ops
Adds the first cut of snapshot scenarios, each asserting a normalized stdout
transcript golden and (for model turns) a re-persisted session-log golden:

- text-turn, tool-call-turn, multi-turn: RECORDED against the real API — the
  committed session.jsonl is a genuine harvested log; replay derives the model
  script from it and reproduces deterministically with no key. tool-call-turn
  exercises the real bash executor (echo SNAPSHOT_OK → tool/call + tool/result
  + a post-tool answer step).
- error-finish, cancel: AUTHORED via a replay.override.json sidecar (the live
  API can't be coaxed into a deterministic 401 or mid-stream cancel). error-
  finish replays a {kind:throw} 401 → the bridge answers the prompt with a
  JSON-RPC error and the log records turn/end{kind:error}; cancel replays a
  {kind:hang} → stopReason:cancelled.

Two input-DSL ops support these: promptExpectError (awaits the prompt, asserts
it rejects — the editor's view of a failed turn — and swallows it) and
promptAndCancel (dispatches the prompt unawaited, waits until the client
OBSERVES the streamed agent_message_chunk, then cancels — pinning frame order
so the cancel transcript is deterministic; fixes a flake Codex caught where the
late chunk and the cancelled response could interleave either way).

Scenarios carry a `recorded` flag so test:snapshot:record only re-runs the
live-API ones. reasoning/max-tokens scenarios are deferred (hard to force
deterministically from the live model). Per docs/rfc/implemented/2026-06-19.
2026-06-19 04:10:49 +08:00