From 4dee9a8d9bc5bbce52dee79bd24ab2523a7f1309 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Wed, 8 Jul 2026 20:31:41 +0800 Subject: [PATCH 01/48] docs: propose background subagent tasks --- docs/rfc/INDEX.md | 1 + .../2026-07-08-background-subagent-tasks.md | 98 +++++++++++++++++++ 2 files changed, 99 insertions(+) create mode 100644 docs/rfc/proposed/feature/2026-07-08-background-subagent-tasks.md diff --git a/docs/rfc/INDEX.md b/docs/rfc/INDEX.md index d6d0ce747b..5f0da6a7ff 100644 --- a/docs/rfc/INDEX.md +++ b/docs/rfc/INDEX.md @@ -13,6 +13,7 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; | [Code Mode — the model writes TypeScript against the tool registry](proposed/feature/2026-06-15-code-mode.md) | 2026-06-15 | | [Pre-tool input rewrite — a consistent design](proposed/feature/2026-06-30-pre-tool-input-rewrite.md) | 2026-06-30 | | [Claude Code and Codex subagent backends (out-of-process delegation to external coding agents)](proposed/feature/2026-07-07-claude-code-and-codex-subagent-backends.md) | 2026-07-07 | +| [Background subagent tasks](proposed/feature/2026-07-08-background-subagent-tasks.md) | 2026-07-08 | | [Repeat-tool-call guard plugin](proposed/feature/2026-07-08-repeat-tool-guard.md) | 2026-07-08 | ### Simplification diff --git a/docs/rfc/proposed/feature/2026-07-08-background-subagent-tasks.md b/docs/rfc/proposed/feature/2026-07-08-background-subagent-tasks.md new file mode 100644 index 0000000000..7c1fe99624 --- /dev/null +++ b/docs/rfc/proposed/feature/2026-07-08-background-subagent-tasks.md @@ -0,0 +1,98 @@ +# RFC: Background subagent tasks + +Status: proposed + +## Problem + +The subagent seam ([the seam RFC](../../implemented/feature/2026-06-21-subagent-capability-seam.md)) exposes `start() -> SubagentRun`, and the model-facing `dsh-tool-subagent` consumer collects that run synchronously: the parent turn blocks until the child returns one final result. That shape is simple and transport-neutral, but it makes slow delegation expensive for the parent. A model that wants two independent investigations must either run them serially or hold the parent step open for the entire child duration. + +The harness already has one background-task precedent in bash. Bash has task ids, owner-token checks, output polling, stop, completion notifications, and prompt guidance. Subagents need the same user-facing habit, but not by copying bash's process-output semantics: a subagent does not expose an incremental stdout stream, and its child session remains the home for internal steps. The parent needs to start a child, keep working, later wait for or read the final answer, and stop the task when it is no longer relevant. + +The design also has two lifecycle constraints that the synchronous cut does not face. First, a background subagent can outlive the tool call that started it, so the tool-call abort signal must not stay wired to the child after the id is returned. Second, a completion notice can only be injected into a live owner agent: once an ACP session closes or an agent handle is disposed, `agent.inject()` cannot append to that session. The feature must therefore define whether background subagents survive owner disposal. + +## Proposal + +Add a background mode to the existing model-facing subagent tools and add three companion tools: `subagent_wait`, `subagent_output`, and `subagent_stop`. The background task registry lives in `@deepseek-ai/dsh-subagent`, keyed by branded task ids and owner tokens, while `@deepseek-ai/dsh-tool-subagent` owns the model-facing schemas, text rendering, completion notice injection, and prompt guidance. + +`dsh-tool-subagent` becomes a single multi-tool consumer plugin instead of one plugin instance per provider. Its config maps model-facing tool names to provider names, so one plugin instance can register `subagent`, `subagent_fork`, and any deployment-specific aliases such as `subagent_acp`, plus the shared background control tools. Providers remain named implementations on `ctx.subagents`: `spawn`, `fork`, `acp`, or future backends. This keeps provider implementation and model-facing exposure separate while avoiding a failure mode where one `subagent` tool exposes `run_in_background` but the companion wait/output/stop tools were never loaded. + +The background task is scoped to the owner session, not durable across session closure. A background subagent starts only from a model-driven call with `exec.agent`; the service stores the caller's `session.header.id` as the owner token. `subagent_output`, `subagent_wait`, and `subagent_stop` compare that stored token with the caller's session id and reject cross-session access. When the owner agent is disposed, the service cancels any running background subagent tasks for that owner and discards their retained snapshots after quiescence. Completion notices are best-effort: if the owner agent is still registered, `dsh-tool-subagent` injects a short `context/message`; if the owner is gone, no notice is written. + +## Tool surface + +Each configured delegation tool may expose `run_in_background?: boolean`. The deployment can disable background mode per tool; a disabled tool does not include the parameter in its schema. A foreground call keeps the synchronous semantics: it waits for `run.result`, returns final text on `completed`, maps non-clean terminal stop reasons to an errored tool result, and disposes the run in `finally`. + +A background call validates that a parent agent exists, starts the provider run through `ctx.subagents`, registers a task, and returns `started background subagent task `. It checks an already-aborted tool signal before starting, but after the id is returned it does not keep the tool-call signal connected to `run.cancel()`. The parent step may finish while the child continues. + +`subagent_output` is a non-blocking status read for a background subagent task. While the task is `running` or `stopping`, it returns only a status line. Once terminal, it returns the final text output or error message plus the terminal status. Reading output is idempotent and does not consume the result; v1 deliberately exposes no incremental transcript cursor because the child session remains the detailed trace. + +`subagent_wait` waits for a task to become terminal, bounded by a defaulted and capped timeout from `dsh-tool-subagent` config. A wait timeout returns `running` and leaves the child alive. Aborting the wait call cancels only the wait, not the background task. + +`subagent_stop` requests cancellation of a running or stopping task and returns immediately. The task registry remains responsible for observing the run settle, recording the terminal state, and disposing the run. Calling stop on an already-terminal task reports that terminal state rather than failing. + +## Runtime task model + +`@deepseek-ai/dsh-subagent` adds a runtime-global task registry to `SubagentService`. Task ids and owner tokens are branded types. A task snapshot records the task id, provider name, child run id, owner token, status, started/finished timestamps, final output, and error message. The status vocabulary is `running`, `stopping`, and the existing terminal `SubagentStopReason` values (`completed`, `aborted`, `error`, `max-tokens`, `refusal`, plus merge-extensible provider values). + +The registry owns task settlement. It attaches one continuation to `run.result`; on success it stores the final output and stop reason, on rejection it stores `error`, and in both cases it disposes the run and notifies task-done listeners. Listener failures are contained and logged so one consumer cannot starve cleanup. + +The registry is runtime-global because `ctx.subagents` is a service shared by all live agents in the Cordis context. Session isolation is therefore explicit owner-token authorization, not an assumption about separate service instances. This mirrors the bash background-task fence: predictable ids are safe only when read/stop operations check the caller's owner token. + +Owner disposal is a hard lifecycle boundary. `SubagentService` listens to `agent/disposed`, finds tasks owned by that agent's session id, and cancels running tasks. It does not attempt to persist unfinished task state, resume children, or inject into disposed sessions. A future durable job system can extend this boundary, but this feature intentionally keeps background subagents tied to live sessions. + +## Model guidance + +`dsh-tool-subagent` registers a system-prompt section that teaches the background-task habit: + +- Keep track of every task id returned by a background subagent call. +- Do not produce a final answer while a relevant background subagent is still running. +- While waiting, continue independent exploration or use other tools when useful. +- Before summarizing or handing work back, call `subagent_wait` or `subagent_output` to collect finished tasks. +- Call `subagent_stop` for a background task that is no longer needed. +- End without collecting a task only when its result is irrelevant or the task was explicitly stopped. + +This prompt guidance is not the enforcement boundary. Runtime enforcement is owner-token authorization and cancellation on owner disposal. The guidance keeps ordinary model behavior from accidentally abandoning relevant work while still allowing explicit stop or irrelevance. + +## Relationship to generic long-running tools + +The generic long-running tool runtime RFC ([Extract a generic long-running tool runtime](../../proposed/architecture/2026-06-20-generic-long-running-tool-runtime.md)) remains the larger direction for shared task ids, owner tokens, cancellation, completion notices, and presentation. Background subagents should not block on that extraction because subagents have a narrower result model than bash: no incremental stdout, no spill files, and no process exit markers. The implementation should keep the subagent registry small and shaped so it can later migrate into a generic runtime without changing the model-facing `run_in_background`, `subagent_wait`, `subagent_output`, and `subagent_stop` contract. + +## Alternatives considered + +### Why not keep one `dsh-tool-subagent` instance per provider? + +The existing one-instance-per-provider shape makes aliasing simple, but companion tools become ambiguous. If each instance registers `subagent_wait`, duplicate tool names collide. If only one instance registers them, deployments can accidentally expose `run_in_background` without the tools required to collect or stop the task. A single multi-tool consumer config keeps provider selection in deployment config and makes the background control plane atomic. + +### Why not put wait/output/stop in a separate plugin? + +A separate plugin has the same half-loaded failure mode: `subagent` could advertise background mode while the control tools are absent. The control tools are part of the model-facing subagent contract, so they should be registered by the same consumer plugin that adds `run_in_background`. + +### Why not let background subagents survive owner session closure? + +Survival after owner closure requires durable task state, child-session recovery, a way to surface late results into a reopened session, and policy for tasks whose owning client never returns. The current agent runtime unregisters disposed agents, and `agent.inject()` intentionally rejects disposed targets. Tying background tasks to live owner sessions makes the v1 lifecycle explicit and avoids orphaned child agents. + +### Why not skip owner-token checks because ACP sessions are isolated? + +ACP sessions isolate their logs and agents, but services such as `ctx.agents`, `ctx.tools`, and `ctx.subagents` are shared within the runtime. A background-task id is a global resource handle. Without an owner check, another live session in the same runtime could guess or receive a task id and read or stop it. + +### Why not expose incremental subagent transcript output? + +The child session is already the trace for internal reasoning, tool calls, and intermediate messages. Streaming that transcript into the parent would blur the parent/child log boundary that makes in-process and ACP providers equivalent. The first background surface returns status and final output only; richer observation belongs to UI/session tooling or a separate observation RFC. + +## Acceptance criteria + +- A deployment config can expose `subagent` and `subagent_fork` from one `dsh-tool-subagent` instance while binding them to different providers. +- A configured delegation tool exposes `run_in_background` only when that tool enables background mode. +- A background call returns a task id immediately and the parent can continue using other tools before collecting the result. +- `subagent_output`, `subagent_wait`, and `subagent_stop` enforce owner-token access and reject cross-session task ids. +- A task that finishes while the owner agent is live injects a durable completion notice into the owner session; a task whose owner is disposed does not throw while trying to notify. +- Disposing the owner agent cancels all of that owner's running background subagent tasks and reaches quiescence without leaking child agents. +- Snapshot coverage proves the changed tool schemas and the completion-notice path; unit coverage pins foreground compatibility, background settlement, timeout, stop, owner isolation, and owner-disposal cleanup. + +## Risks + +The multi-tool config reshapes how deployments expose provider aliases, so examples and generated tool catalogs must move together with the implementation. The pre-release policy allows this churn, but the migration must update every shipped config in one change. + +The prompt guidance can reduce abandoned tasks but cannot force a model to collect every background result. Runtime cleanup on owner disposal is the hard stop; a future planner or guard could enforce "no final answer with relevant running tasks" more strongly if the prompt proves insufficient. + +The task registry duplicates some concepts named by the generic long-running-tool RFC. Keeping the subagent registry final-output-only and service-local limits that duplication, but a later generic runtime extraction will still need a careful migration. From e7e382f9d1b499313c8d62d25e068b252798c90b Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Wed, 8 Jul 2026 21:16:56 +0800 Subject: [PATCH 02/48] docs: require awaited subagent owner cleanup --- .../feature/2026-07-08-background-subagent-tasks.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/rfc/proposed/feature/2026-07-08-background-subagent-tasks.md b/docs/rfc/proposed/feature/2026-07-08-background-subagent-tasks.md index 7c1fe99624..97e8ce1ddb 100644 --- a/docs/rfc/proposed/feature/2026-07-08-background-subagent-tasks.md +++ b/docs/rfc/proposed/feature/2026-07-08-background-subagent-tasks.md @@ -16,7 +16,7 @@ Add a background mode to the existing model-facing subagent tools and add three `dsh-tool-subagent` becomes a single multi-tool consumer plugin instead of one plugin instance per provider. Its config maps model-facing tool names to provider names, so one plugin instance can register `subagent`, `subagent_fork`, and any deployment-specific aliases such as `subagent_acp`, plus the shared background control tools. Providers remain named implementations on `ctx.subagents`: `spawn`, `fork`, `acp`, or future backends. This keeps provider implementation and model-facing exposure separate while avoiding a failure mode where one `subagent` tool exposes `run_in_background` but the companion wait/output/stop tools were never loaded. -The background task is scoped to the owner session, not durable across session closure. A background subagent starts only from a model-driven call with `exec.agent`; the service stores the caller's `session.header.id` as the owner token. `subagent_output`, `subagent_wait`, and `subagent_stop` compare that stored token with the caller's session id and reject cross-session access. When the owner agent is disposed, the service cancels any running background subagent tasks for that owner and discards their retained snapshots after quiescence. Completion notices are best-effort: if the owner agent is still registered, `dsh-tool-subagent` injects a short `context/message`; if the owner is gone, no notice is written. +The background task is scoped to the owner session, not durable across session closure. A background subagent starts only from a model-driven call with `exec.agent`; the service stores the caller's `session.header.id` as the owner token. `subagent_output`, `subagent_wait`, and `subagent_stop` compare that stored token with the caller's session id and reject cross-session access. When the owner agent is disposed, an awaited owner-cleanup path cancels any running background subagent tasks for that owner and waits for their settlement/dispose before the owner handle reports quiescence. Completion notices are best-effort: if the owner agent is still registered, `dsh-tool-subagent` injects a short `context/message`; if the owner is gone, no notice is written. ## Tool surface @@ -38,7 +38,7 @@ The registry owns task settlement. It attaches one continuation to `run.result`; The registry is runtime-global because `ctx.subagents` is a service shared by all live agents in the Cordis context. Session isolation is therefore explicit owner-token authorization, not an assumption about separate service instances. This mirrors the bash background-task fence: predictable ids are safe only when read/stop operations check the caller's owner token. -Owner disposal is a hard lifecycle boundary. `SubagentService` listens to `agent/disposed`, finds tasks owned by that agent's session id, and cancels running tasks. It does not attempt to persist unfinished task state, resume children, or inject into disposed sessions. A future durable job system can extend this boundary, but this feature intentionally keeps background subagents tied to live sessions. +Owner disposal is a hard lifecycle boundary, but `agent/disposed` alone is not the cleanup mechanism. The current agent registry emits `agent/disposed` synchronously after removing the agent, and `AgentHandle.dispose()` does not await asynchronous listener work. This feature therefore also adds an awaited owner-cleanup seam: background task registration attaches an owner-scoped disposer that runs in the owning agent's disposal chain before that handle resolves. That disposer finds tasks owned by the agent's session id, requests cancellation, waits for each task's settlement path to record the terminal snapshot, and awaits `run.dispose()`. The existing `agent/disposed` event may still be used as a best-effort notification/fallback, but it must not be the path that promises child quiescence. The service does not attempt to persist unfinished task state, resume children, or inject into disposed sessions. A future durable job system can extend this boundary, but this feature intentionally keeps background subagents tied to live sessions. ## Model guidance @@ -51,7 +51,7 @@ Owner disposal is a hard lifecycle boundary. `SubagentService` listens to `agent - Call `subagent_stop` for a background task that is no longer needed. - End without collecting a task only when its result is irrelevant or the task was explicitly stopped. -This prompt guidance is not the enforcement boundary. Runtime enforcement is owner-token authorization and cancellation on owner disposal. The guidance keeps ordinary model behavior from accidentally abandoning relevant work while still allowing explicit stop or irrelevance. +This prompt guidance is not the enforcement boundary. Runtime enforcement is owner-token authorization and the awaited owner-cleanup path. The guidance keeps ordinary model behavior from accidentally abandoning relevant work while still allowing explicit stop or irrelevance. ## Relationship to generic long-running tools @@ -69,7 +69,7 @@ A separate plugin has the same half-loaded failure mode: `subagent` could advert ### Why not let background subagents survive owner session closure? -Survival after owner closure requires durable task state, child-session recovery, a way to surface late results into a reopened session, and policy for tasks whose owning client never returns. The current agent runtime unregisters disposed agents, and `agent.inject()` intentionally rejects disposed targets. Tying background tasks to live owner sessions makes the v1 lifecycle explicit and avoids orphaned child agents. +Survival after owner closure requires durable task state, child-session recovery, a way to surface late results into a reopened session, and policy for tasks whose owning client never returns. The current agent runtime unregisters disposed agents, and `agent.inject()` intentionally rejects disposed targets. Tying background tasks to an awaited owner-cleanup path makes the v1 lifecycle explicit and avoids orphaned child agents. ### Why not skip owner-token checks because ACP sessions are isolated? @@ -86,13 +86,13 @@ The child session is already the trace for internal reasoning, tool calls, and i - A background call returns a task id immediately and the parent can continue using other tools before collecting the result. - `subagent_output`, `subagent_wait`, and `subagent_stop` enforce owner-token access and reject cross-session task ids. - A task that finishes while the owner agent is live injects a durable completion notice into the owner session; a task whose owner is disposed does not throw while trying to notify. -- Disposing the owner agent cancels all of that owner's running background subagent tasks and reaches quiescence without leaking child agents. +- Disposing the owner agent runs an awaited owner-cleanup path that cancels all of that owner's running background subagent tasks and reaches quiescence without leaking child agents; tests prove `agent/disposed` alone is not relied on for this guarantee. - Snapshot coverage proves the changed tool schemas and the completion-notice path; unit coverage pins foreground compatibility, background settlement, timeout, stop, owner isolation, and owner-disposal cleanup. ## Risks The multi-tool config reshapes how deployments expose provider aliases, so examples and generated tool catalogs must move together with the implementation. The pre-release policy allows this churn, but the migration must update every shipped config in one change. -The prompt guidance can reduce abandoned tasks but cannot force a model to collect every background result. Runtime cleanup on owner disposal is the hard stop; a future planner or guard could enforce "no final answer with relevant running tasks" more strongly if the prompt proves insufficient. +The prompt guidance can reduce abandoned tasks but cannot force a model to collect every background result. Runtime cleanup through the awaited owner-disposal path is the hard stop; a future planner or guard could enforce "no final answer with relevant running tasks" more strongly if the prompt proves insufficient. The task registry duplicates some concepts named by the generic long-running-tool RFC. Keeping the subagent registry final-output-only and service-local limits that duplication, but a later generic runtime extraction will still need a careful migration. From 184e164091d462cefc9dfe309c0e85d46ca6e231 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Thu, 9 Jul 2026 21:22:54 +0800 Subject: [PATCH 03/48] feat(tasks): background task runtime, generic task_* control tools, bash/subagent producers One shared ctx.tasks registry (branded -N ids, owner-fenced read/kill/wait/list, attachSurface misconfiguration fence, reported-flag notice dedup, atomic register) + dsh-tool-tasks (task_output/task_list/ task_kill, completion-notice injection, background prompt habit). Producers opt in via their own enableRunInBackground config: bash (stream kind; seam slimmed to resolve/run/start returning a BashProcess handle, bash_output/bash_kill deleted) and subagent (final-output kind; done settles after run.dispose()). Owner disposal drains tasks through the new awaited ctx.agents.onCleanup seam in the loop's disposal chain. Both RFCs moved to implemented/; docs, catalogs, snapshots re-pinned. --- docs/architecture.md | 4 +- docs/capability-seams.md | 8 + docs/config-catalog.md | 51 +- docs/cookbook/adding-a-tool.md | 4 +- docs/cordis-catalog/events.md | 8 +- docs/cordis-catalog/services.md | 43 +- docs/core-data-structures/bash.md | 65 +- docs/core-data-structures/core.md | 7 +- docs/core-data-structures/tasks.md | 126 +++ docs/event-producer-consumer.md | 8 +- docs/module-graph.md | 22 +- docs/rfc/INDEX.md | 4 +- ...06-20-generic-long-running-tool-runtime.md | 171 +++++ .../2026-07-08-background-subagent-tasks.md | 62 ++ ...06-20-generic-long-running-tool-runtime.md | 41 - .../2026-07-08-background-subagent-tasks.md | 98 --- ...2026-06-20-drop-bash-output-spill-files.md | 2 +- docs/tool-catalog.md | 128 ++-- .../tests/snapshots/text-turn/session.jsonl | 72 +- examples/coding-agent/README.md | 4 +- examples/coding-agent/cordis.yml | 3 +- packages/README.md | 1 + packages/bash/README.md | 4 +- packages/bash/bash-local/README.md | 2 +- packages/bash/bash-local/src/index.ts | 157 ++-- .../bash/bash-local/tests/executor.spec.ts | 346 ++++----- packages/bash/bash/README.md | 18 +- packages/bash/bash/package.json | 2 - packages/bash/bash/src/index.ts | 136 +--- packages/bash/bash/src/types.ts | 123 ++- packages/bash/bash/tests/service.spec.ts | 151 +--- packages/bash/bash/tsconfig.json | 3 - packages/bash/tool-bash/README.md | 36 +- packages/bash/tool-bash/package.json | 6 + packages/bash/tool-bash/src/index.ts | 286 +++---- .../bash/tool-bash/tests/integration.spec.ts | 76 +- packages/bash/tool-bash/tests/tools.spec.ts | 715 +++++++----------- packages/bash/tool-bash/tsconfig.json | 6 + packages/core/agent-core/README.md | 2 +- packages/core/agent-core/package.json | 4 + packages/core/agent-core/src/index.ts | 7 +- .../core/agent-core/tests/agent-core.spec.ts | 4 +- packages/core/agent-core/tsconfig.json | 6 + packages/core/agent-loop/README.md | 2 +- packages/core/agent-loop/src/index.ts | 11 +- .../agent-loop/tests/cleanup-drain.spec.ts | 57 ++ packages/core/agent/README.md | 5 +- packages/core/agent/src/index.ts | 68 ++ packages/core/agent/tests/agent.spec.ts | 125 ++- .../core/tools/tests/gen-tool-catalog.spec.ts | 2 +- .../hooks/hook-protocol/tests/runner.spec.ts | 1 - packages/subagent/subagent/README.md | 2 +- packages/subagent/subagent/src/index.ts | 15 +- packages/subagent/tool-subagent/README.md | 9 +- packages/subagent/tool-subagent/package.json | 5 +- packages/subagent/tool-subagent/src/index.ts | 125 ++- .../tool-subagent/tests/tool-subagent.spec.ts | 192 ++++- packages/subagent/tool-subagent/tsconfig.json | 3 + packages/tasks/README.md | 10 + packages/tasks/tasks/README.md | 25 + packages/tasks/tasks/package.json | 35 + packages/tasks/tasks/src/index.ts | 457 +++++++++++ packages/tasks/tasks/src/types.ts | 155 ++++ packages/tasks/tasks/tests/tasks.spec.ts | 465 ++++++++++++ packages/tasks/tasks/tsconfig.json | 24 + packages/tasks/tool-tasks/README.md | 24 + packages/tasks/tool-tasks/package.json | 43 ++ packages/tasks/tool-tasks/src/index.ts | 183 +++++ .../tasks/tool-tasks/tests/tool-tasks.spec.ts | 306 ++++++++ packages/tasks/tool-tasks/tsconfig.json | 33 + packages/ui/acp-agent/tests/acp-agent.spec.ts | 4 +- packages/ui/acp/README.md | 2 +- .../ui/stdio-agent/tests/stdio-agent.spec.ts | 4 +- packages/util/README.md | 2 +- packages/util/brand/README.md | 4 +- pnpm-lock.yaml | 68 +- scripts/doc-budgets.manifest.json | 4 +- scripts/gen-doc-graphs.ts | 9 + scripts/gen-tool-catalog.ts | 21 +- scripts/type-equiv.manifest.json | 9 +- tsconfig.base.json | 1 + tsconfig.build.json | 2 + tsconfig.json | 2 + 83 files changed, 3909 insertions(+), 1627 deletions(-) create mode 100644 docs/core-data-structures/tasks.md create mode 100644 docs/rfc/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.md create mode 100644 docs/rfc/implemented/feature/2026-07-08-background-subagent-tasks.md delete mode 100644 docs/rfc/proposed/architecture/2026-06-20-generic-long-running-tool-runtime.md delete mode 100644 docs/rfc/proposed/feature/2026-07-08-background-subagent-tasks.md create mode 100644 packages/core/agent-loop/tests/cleanup-drain.spec.ts create mode 100644 packages/tasks/README.md create mode 100644 packages/tasks/tasks/README.md create mode 100644 packages/tasks/tasks/package.json create mode 100644 packages/tasks/tasks/src/index.ts create mode 100644 packages/tasks/tasks/src/types.ts create mode 100644 packages/tasks/tasks/tests/tasks.spec.ts create mode 100644 packages/tasks/tasks/tsconfig.json create mode 100644 packages/tasks/tool-tasks/README.md create mode 100644 packages/tasks/tool-tasks/package.json create mode 100644 packages/tasks/tool-tasks/src/index.ts create mode 100644 packages/tasks/tool-tasks/tests/tool-tasks.spec.ts create mode 100644 packages/tasks/tool-tasks/tsconfig.json diff --git a/docs/architecture.md b/docs/architecture.md index 02dac4a2dd..eaeb86d913 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -31,6 +31,7 @@ The default distribution is a composition, not a hierarchy. `packages/core/` is | `ctx.web` | [`web/`](../packages/web/README.md) | search/fetch provider registries | | `ctx.compact` | [`compact/`](../packages/compact/README.md) | session-surface compaction | | `ctx.subagents` | [`subagent/`](../packages/subagent/README.md) | named delegation providers | +| `ctx.tasks` | [`tasks/`](../packages/tasks/README.md) | background task registry + generic `task_*` control tools | | `ctx.sessionPersistence` | [`session-persistence/`](../packages/session-persistence/README.md) | durable storage for session logs | ## Event Surface @@ -101,7 +102,7 @@ Every session event is turn-enclosed. Reloading a crashed session preserves the ### Agent Handles -`ctx.agents` owns live agents and returns an `AgentHandle { agent, dispose() }`. `Agent` is the surface other plugins drive: `send()` queues work, `steer()` injects mid-turn content, `inject()` appends context and opens a one-shot injection turn when idle, `cancel()` is the public stop primitive, and `whenIdle()` observes quiescence. Lifecycle owners tear down with `await dispose()`. +`ctx.agents` owns live agents and returns an `AgentHandle { agent, dispose() }`. `Agent` is the surface other plugins drive: `send()` queues work, `steer()` injects mid-turn content, `inject()` appends context and opens a one-shot injection turn when idle, `cancel()` is the public stop primitive, and `whenIdle()` observes quiescence. Lifecycle owners tear down with `await dispose()`, whose chain also awaits every `ctx.agents.onCleanup` registration — the seam tying resources (background tasks) to the owner's quiescence. ## State And Model Surface @@ -140,6 +141,7 @@ New behavior should attach to a documented seam; changing the shipped loop requi | Add a model provider | register an adapter on `ctx.llm` | | Add a model-facing capability | register a tool on `ctx.tools`; schemas flow into prompt assembly | | Add command execution | implement and register a `ctx.bash` backend | +| Add a long-running/background capability | register the work on `ctx.tasks`; the generic `task_*` tools collect/stop it | | Add filesystem access or policy | implement a `ctx.fs` provider or listen on `fs/*` policy events | | Intercept prompts, requests, tool use, or continuation | listen on the relevant `agent/*` or `tools/*` waterfall | | Add UI or editor integration | drive `ctx.agents` and render from `session/event` | diff --git a/docs/capability-seams.md b/docs/capability-seams.md index 339954c00f..54a271423e 100644 --- a/docs/capability-seams.md +++ b/docs/capability-seams.md @@ -56,6 +56,9 @@ flowchart LR pkg_subagent_fork["subagent-fork"] pkg_subagent_acp["subagent-acp"] pkg_subagent_mock["subagent-mock"] + pkg_tasks["tasks"] + svc_tasks["ctx.tasks
Background task registry"] + pkg_tool_tasks["tool-tasks"] pkg_web["web"] svc_web["ctx.web
Web access provider registry"] pkg_web_search_exa["web-search-exa"] @@ -85,6 +88,7 @@ flowchart LR pkg_subagent_mock --> svc_subagents pkg_subagent_spawn --> svc_subagents pkg_system_prompt --> svc_systemPrompt + pkg_tasks --> svc_tasks pkg_tools --> svc_tools pkg_web --> svc_web pkg_web_fetch_local --> svc_web @@ -116,6 +120,9 @@ flowchart LR svc_systemPrompt --> pkg_tool_fs svc_systemPrompt --> pkg_tool_web svc_systemPrompt --> pkg_tools + svc_tasks --> pkg_tool_bash + svc_tasks --> pkg_tool_subagent + svc_tasks --> pkg_tool_tasks svc_tools --> pkg_acp svc_tools --> pkg_agent_loop svc_tools --> pkg_tool_bash @@ -141,6 +148,7 @@ flowchart LR | `ctx.fs` | `seam` | [`fs`](../packages/fs/fs) | [`fs-local`](../packages/fs/fs-local) | [`tool-fs`](../packages/fs/tool-fs) | [`fs-policy`](../packages/fs/fs-policy) | tool-fs executes read/write/edit through ctx.fs; fs-policy contributes observed-state checks through the fs/* event gate. | | `ctx.compact` | `seam` | [`compact`](../packages/compact/compact) | [`compact-basic`](../packages/compact/compact-basic) | [`compact-basic`](../packages/compact/compact-basic) | - | The basic backend currently consumes the pre-step event directly; a model-facing compact tool remains deferred. | | `ctx.subagents` | `seam` | [`subagent`](../packages/subagent/subagent) | [`subagent-spawn`](../packages/subagent/subagent-spawn), [`subagent-fork`](../packages/subagent/subagent-fork), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-mock`](../packages/support/subagent-mock) | [`tool-subagent`](../packages/subagent/tool-subagent) | - | Providers implement transports; tool-subagent exposes one configured provider as a model-facing tool name. | +| `ctx.tasks` | `core` | [`tasks`](../packages/tasks/tasks) | - | [`tool-bash`](../packages/bash/tool-bash), [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-tasks`](../packages/tasks/tool-tasks) | - | Producers (tool-bash background commands, tool-subagent background delegations) register running work; tool-tasks is the model-facing control surface that reads, lists, and kills it. | | `ctx.web` | `seam` | [`web`](../packages/web/web) | [`web-search-exa`](../packages/web/web-search-exa), [`web-search-perplexity`](../packages/web/web-search-perplexity), [`web-search-deepseek`](../packages/web/web-search-deepseek), [`web-fetch-local`](../packages/web/web-fetch-local) | [`tool-web`](../packages/web/tool-web) | - | Search and fetch providers register into one ctx.web seam; tool-web owns the stable model-facing names. | Maintenance mode: hybrid: services are discovered from Cordis declarations; interface/implementation/consumer roles are classified in `scripts/gen-doc-graphs.ts` with a completeness guard. diff --git a/docs/config-catalog.md b/docs/config-catalog.md index e55066101f..f59260810f 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -83,7 +83,7 @@ export interface Config { Depends on: [`AgentLoopConfig`](#deepseek-aidsh-agent-loop) · [`SystemPromptConfig`](#deepseek-aidsh-system-prompt) -Source: [`packages/core/agent-core/src/index.ts:69`](../packages/core/agent-core/src/index.ts) +Source: [`packages/core/agent-core/src/index.ts:72`](../packages/core/agent-core/src/index.ts) ## `@deepseek-ai/dsh-agent-loop` @@ -139,7 +139,7 @@ export interface Config { } ``` -Source: [`packages/bash/bash-local/src/index.ts:28`](../packages/bash/bash-local/src/index.ts) +Source: [`packages/bash/bash-local/src/index.ts:29`](../packages/bash/bash-local/src/index.ts) ## `@deepseek-ai/dsh-compact-basic` @@ -602,6 +602,25 @@ export interface Config { Source: [`packages/core/system-prompt/src/index.ts:179`](../packages/core/system-prompt/src/index.ts) +## `@deepseek-ai/dsh-tool-bash` + +Requires: `tools` · `bash` · `systemPrompt` + +```ts config-catalog +/** Config: whether the model may background commands (the producer-opt-in flag). */ +export interface Config { + /** + * Expose `run_in_background` in the bash schema (default true). Disabled, + * the parameter is absent entirely — schema and capability never disagree. + * Backgrounding also needs the `ctx.tasks` runtime at call time; a missing + * one fails the call loud with the load-these-packages message. + */ + enableRunInBackground?: boolean +} +``` + +Source: [`packages/bash/tool-bash/src/index.ts:43`](../packages/bash/tool-bash/src/index.ts) + ## `@deepseek-ai/dsh-tool-fs` Requires: `tools` · `fs` · `systemPrompt` @@ -639,6 +658,14 @@ export interface Config { * `{ provider: 'acp', toolName: 'subagent_acp' }`. */ toolName?: string + /** + * Expose `run_in_background` in this instance's schema (default true). + * Disabled, the parameter is absent entirely — schema and capability never + * disagree; delegation through this instance stays strictly synchronous. + * Backgrounding also needs the `ctx.tasks` runtime at call time; a missing + * one fails the call loud with the load-these-packages message. + */ + enableRunInBackground?: boolean /** * Default per-child agent options (model) applied to every spawned child. * Omitted fields fall back to the child loop's own defaults. There is no @@ -651,7 +678,23 @@ export interface Config { Depends on: [`AgentOptions`](../packages/core/agent/src/index.ts) -Source: [`packages/subagent/tool-subagent/src/index.ts:44`](../packages/subagent/tool-subagent/src/index.ts) +Source: [`packages/subagent/tool-subagent/src/index.ts:56`](../packages/subagent/tool-subagent/src/index.ts) + +## `@deepseek-ai/dsh-tool-tasks` + +Requires: `tools` · `tasks` · `systemPrompt` + +```ts config-catalog +/** Config: the `task_output` wait bounds (defaulted, capped — never hardcoded). */ +export interface Config { + /** Wait duration applied when `task_output` sets `wait` without `timeout_ms` (default 30s). */ + waitTimeoutMs?: number + /** Hard cap on any single wait; a larger model-supplied `timeout_ms` is clamped down to it (default 10min). */ + maxWaitTimeoutMs?: number +} +``` + +Source: [`packages/tasks/tool-tasks/src/index.ts:34`](../packages/tasks/tool-tasks/src/index.ts) ## `@deepseek-ai/dsh-tool-web` @@ -793,7 +836,7 @@ These load from a `cordis.yml` entry with no `config:` block; they declare no co - `@deepseek-ai/dsh-llm` ([`packages/llm/llm/src/index.ts`](../packages/llm/llm/src/index.ts)) - `@deepseek-ai/dsh-session` ([`packages/core/session/src/index.ts`](../packages/core/session/src/index.ts)) - `@deepseek-ai/dsh-subagent` ([`packages/subagent/subagent/src/index.ts`](../packages/subagent/subagent/src/index.ts)) -- `@deepseek-ai/dsh-tool-bash` — requires `tools` · `bash` · `systemPrompt` ([`packages/bash/tool-bash/src/index.ts`](../packages/bash/tool-bash/src/index.ts)) +- `@deepseek-ai/dsh-tasks` ([`packages/tasks/tasks/src/index.ts`](../packages/tasks/tasks/src/index.ts)) - `@deepseek-ai/dsh-tool-todo` — requires `tools` ([`packages/todo/tool-todo/src/index.ts`](../packages/todo/tool-todo/src/index.ts)) - `@deepseek-ai/dsh-tools` — requires `systemPrompt` ([`packages/core/tools/src/index.ts`](../packages/core/tools/src/index.ts)) diff --git a/docs/cookbook/adding-a-tool.md b/docs/cookbook/adding-a-tool.md index 9180d88489..ba89caf387 100644 --- a/docs/cookbook/adding-a-tool.md +++ b/docs/cookbook/adding-a-tool.md @@ -41,9 +41,7 @@ Registration is effect-based: disposing the plugin fiber unregisters the tool (w ## Long-running work -Follow tool-bash's background pattern: a `run_in_background` flag returns a task id immediately; companion tools poll incrementally and kill; completion notices arrive via `agent.inject()`. Bound buffers and spill full output to disk so nothing is silently lost. - -> TODO: each tool reimplements this background pattern by hand today. At some point we need a generic long-running-tool layer that handles task ids, incremental polling, kill, and completion notices uniformly. +Register the running work with the shared task runtime instead of inventing a task protocol: gate a `run_in_background` parameter behind your plugin's own defaulted `enableRunInBackground`-style config, start the work, and hand it to `ctx.tasks.register({ kind, label, owner: exec.agent, cancel, done, readOutput? })` (`@deepseek-ai/dsh-tasks`). The runtime issues the `-N` id, fences access to the owning session, cancels-and-awaits your task when the owner disposes, and the generic `task_output`/`task_list`/`task_kill` tools plus the completion notice come from `@deepseek-ai/dsh-tool-tasks` — your tool returns `started background task ` and is done. Your producer keeps its execution concerns: `done` must settle at quiescence (resources released), and a stream-kind `readOutput` owns its own truncation/spill formatting (bound buffers, spill full output to disk so nothing is silently lost — see tool-bash's `renderProcessRead`). Do NOT wire `exec.signal` to the background work after the id is returned; check `exec.signal?.aborted` once before starting, then leave cancellation to `task_kill` and owner cleanup. **A failed `register()` must not orphan the work**: `register()` is atomic (a throw — the no-control-surface fence, a bad owner — mutates no registry state), so wrap it in try/catch, cancel the just-started work, AWAIT its quiescence, and rethrow — the model never learns an id, so nothing else could ever collect or kill what you started (tool-bash's `proc.kill(); await proc.done` and tool-subagent's `run.cancel(); await done` are the templates). ## Permissions / sandboxing diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index 8d776a75ef..ed6e7dad33 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -243,7 +243,7 @@ A subagent run settled — emitted when SubagentRun.result resolves (any stop re 'subagent/end'(info: SubagentRunEndInfo): void ``` -Source: [`packages/subagent/subagent/src/index.ts:98`](../../packages/subagent/subagent/src/index.ts) +Source: [`packages/subagent/subagent/src/index.ts:99`](../../packages/subagent/subagent/src/index.ts) ### `subagent/provider-added` — emit @@ -253,7 +253,7 @@ A provider became resolvable in the SubagentService registry. Consumers that der 'subagent/provider-added'(provider: SubagentProvider): void ``` -Source: [`packages/subagent/subagent/src/index.ts:72`](../../packages/subagent/subagent/src/index.ts) +Source: [`packages/subagent/subagent/src/index.ts:73`](../../packages/subagent/subagent/src/index.ts) ### `subagent/provider-removed` — emit @@ -263,7 +263,7 @@ A provider left the registry (its plugin's fiber was disposed — an unload or a 'subagent/provider-removed'(name: string): void ``` -Source: [`packages/subagent/subagent/src/index.ts:83`](../../packages/subagent/subagent/src/index.ts) +Source: [`packages/subagent/subagent/src/index.ts:84`](../../packages/subagent/subagent/src/index.ts) ### `subagent/start` — emit @@ -273,7 +273,7 @@ A subagent run started — emitted after the provider is resolved and its capabi 'subagent/start'(info: SubagentRunInfo): void ``` -Source: [`packages/subagent/subagent/src/index.ts:91`](../../packages/subagent/subagent/src/index.ts) +Source: [`packages/subagent/subagent/src/index.ts:92`](../../packages/subagent/subagent/src/index.ts) ## `system-prompt/*` diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 26126c4481..42179d4bd3 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -33,12 +33,14 @@ create(options: CreateAgentOptions): AgentHandle async resume(options: ResumeAgentOptions): Promise register(agent: Agent): () => void get(id: AgentId): Agent | undefined +onCleanup(agentId: AgentId, cleanup: () => Promise): () => void +async drainCleanups(agentId: AgentId): Promise list(): Agent[] ``` Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/index.ts:117`](../../packages/core/agent/src/index.ts) +Source: [`packages/core/agent/src/index.ts:124`](../../packages/core/agent/src/index.ts) ## `ctx.bash` — `BashExecutor` (abstract seam) @@ -47,25 +49,19 @@ Abstract bash execution service. Subclass, implement the abstract methods, and l Semantics every implementation must honor: - run REJECTS only for infrastructure failures (unusable workdir, missing shell, pre-aborted signal). Nonzero exits, timeout kills, and abort kills RESOLVE with a descriptive BashRunResult — reporting a failed command is the tool layer's job, not an exception. -- start returns immediately; no timeout applies to background tasks (callers stop them via kill or the spec's AbortSignal). Completion must fire the onTaskDone listeners exactly once per task, and must NOT fire after the service is disposed. -- readOutput is incremental: consecutive reads never re-deliver output. Implementations bound their buffers; reads that lost data flag `lossy` and point at full-stream spill files when available. -- Disposal kills every running task and awaits their exit (no orphan processes survive `fiber.dispose()`). +- start returns immediately; no timeout applies to background processes (callers stop them via BashProcess.kill or the spec's AbortSignal). The handle's `done` settles at process close and never rejects (a spawn failure settles as `killed` with the error readable on stderr). +- BashProcess.readOutput is incremental: consecutive reads never re-deliver output. Implementations bound their buffers; reads that lost data flag `lossy` and point at full-stream spill files when available. +- Disposal kills every running background process and awaits their exit (no orphan processes survive `fiber.dispose()`). ```ts cordis-catalog abstract resolve(request: BashExecRequest): BashExecSpec abstract run(spec: BashExecSpec): Promise -abstract start(spec: BashExecSpec): BashTask -abstract get(id: BashTaskId): BashTask | undefined -abstract ownerOf(id: BashTaskId): OwnerToken | undefined -abstract list(): BashTask[] -abstract readOutput(id: BashTaskId): BashTaskRead -abstract kill(id: BashTaskId): boolean -onTaskDone(listener: BashTaskListener): () => void +abstract start(spec: BashExecSpec): BashProcess ``` -Types: [BashExecRequest](../core-data-structures/bash.md) · [BashExecSpec](../core-data-structures/bash.md) · [BashRunResult](../core-data-structures/bash.md) · [BashTask](../core-data-structures/bash.md) · [BashTaskRead](../core-data-structures/bash.md) +Types: [BashExecRequest](../core-data-structures/bash.md) · [BashExecSpec](../core-data-structures/bash.md) · [BashRunResult](../core-data-structures/bash.md) -Source: [`packages/bash/bash/src/index.ts:59`](../../packages/bash/bash/src/index.ts) +Source: [`packages/bash/bash/src/index.ts:65`](../../packages/bash/bash/src/index.ts) ## `ctx.codeRuntime` — `CodeRuntime` (abstract seam) @@ -196,7 +192,7 @@ list(): string[] start(name: string, request: SubagentStartRequest): SubagentRun ``` -Source: [`packages/subagent/subagent/src/index.ts:144`](../../packages/subagent/subagent/src/index.ts) +Source: [`packages/subagent/subagent/src/index.ts:145`](../../packages/subagent/subagent/src/index.ts) ## `ctx.systemPrompt` — `SystemPrompt` @@ -211,6 +207,25 @@ async assemble(context: AssembleContext = {}): Promise Source: [`packages/core/system-prompt/src/index.ts:291`](../../packages/core/system-prompt/src/index.ts) +## `ctx.tasks` — `TaskService` + +The `tasks` service: the runtime-global background task registry. See the module doc for the ownership, isolation, and lifecycle contracts. + +```ts cordis-catalog +register(registration: TaskRegistration): TaskId +list(caller?: Agent): TaskSnapshot[] +get(id: TaskId, caller?: Agent): TaskSnapshot +read(id: TaskId, caller?: Agent): TaskRead +kill(id: TaskId, caller?: Agent, reason?: string): 'requested' | 'already-terminal' +async wait(id: TaskId, timeoutMs: number, caller?: Agent, signal?: AbortSignal): Promise +onTaskDone(listener: TaskDoneListener): () => void +attachSurface(name: string): () => void +``` + +Types: [Agent](../core-data-structures/core.md) + +Source: [`packages/tasks/tasks/src/index.ts:84`](../../packages/tasks/tasks/src/index.ts) + ## `ctx.tools` — `ToolRegistry` Tool registry (`ctx.tools`): tool plugins register definitions; the agent loop executes calls through the `tools/pre-execute` → dispatch → `tools/post-execute` pipeline. The registry contributes its schemas into the system-prompt assembly. diff --git a/docs/core-data-structures/bash.md b/docs/core-data-structures/bash.md index 273ba5ebe8..ffa83cd22a 100644 --- a/docs/core-data-structures/bash.md +++ b/docs/core-data-structures/bash.md @@ -1,6 +1,6 @@ # Bash Executor -The bash execution seam — the canonical [capability seam](../rfc/implemented/architecture/2026-06-13-capability-seams.md) example, split across three packages: interface ([dsh-bash](../../packages/bash/bash), `ctx.bash`), implementation ([dsh-bash-local](../../packages/bash/bash-local), local subprocesses), and consumer ([dsh-tool-bash](../../packages/bash/tool-bash), the `bash`/`bash_output`/`bash_kill` tool schemas). Bash is **one optional capability**, not part of the agent-loop spine — so its vocabulary lives here, not in [core.md](core.md). A sandboxed, containerized, or remote backend is a sibling package implementing the same interface. +The bash execution seam — the canonical [capability seam](../rfc/implemented/architecture/2026-06-13-capability-seams.md) example, split across three packages: interface ([dsh-bash](../../packages/bash/bash), `ctx.bash`), implementation ([dsh-bash-local](../../packages/bash/bash-local), local subprocesses), and consumer ([dsh-tool-bash](../../packages/bash/tool-bash), the `bash` tool schema). Bash is **one optional capability**, not part of the agent-loop spine — so its vocabulary lives here, not in [core.md](core.md). A sandboxed, containerized, or remote backend is a sibling package implementing the same interface. Source: [`packages/bash/bash/src/types.ts`](../../packages/bash/bash/src/types.ts) @@ -35,15 +35,6 @@ interface BashExecRequest { * uses shell syntax like `FOO=bar cmd`). */ env?: Record | undefined - /** - * Opaque OWNER token for a background task — the consumer's isolation key - * (the tool layer passes the owning agent's `session.header.id`). The - * executor stores it on the task and exposes it via {@link BashExecutor.ownerOf}; - * the executor itself NEVER interprets it (no access policy lives in the - * seam — that is the consumer's job). Absent for foreground runs and for an - * ownerless background start (a non-agent caller). - */ - owner?: OwnerToken | undefined } ``` @@ -56,10 +47,10 @@ interface BashExecSpec { signal?: AbortSignal | undefined /** * Bytes to write to the command's stdin (then close it), carried through - * verbatim from {@link BashExecRequest.stdin}. OPTIONAL on the resolved spec - * (unlike `owner`): it has no config default, so a missing one means "no - * stdin" — the safe, ordinary case — not a silent footgun, so it stays a - * plain optional rather than required-but-nullable (see the request field). + * verbatim from {@link BashExecRequest.stdin}. OPTIONAL on the resolved spec: + * it has no config default, so a missing one means "no stdin" — the safe, + * ordinary case — not a silent footgun, so it stays a plain optional rather + * than required-but-nullable (see the request field). */ stdin?: string | undefined /** @@ -70,23 +61,12 @@ interface BashExecSpec { * config default, absent means "no extra env". */ env?: Record | undefined - /** - * Opaque owner token, REQUIRED-but-nullable (mirrors `workdir`/`timeoutMs` - * being required on the resolved spec): {@link BashExecutor.resolve} carries - * the request's `owner` through, defaulting a missing one to `undefined`. A - * required field makes a forgotten owner a VISIBLE `undefined` rather than a - * silently-absent property that yields an unowned (cross-session-readable) - * task. `start()` stores it; `run()` (foreground) ignores it. - */ - owner: OwnerToken | undefined } ``` -The `owner` token is the isolation key: the executor stores it but never interprets it (access policy is the consumer's job), so a background task started by one agent isn't readable cross-session. A required-but-nullable field makes a forgotten owner a visible `undefined` rather than a silently-unowned task. +The seam is deliberately **task-free**: no task ids, no owner tokens, no polling protocol. Background-task semantics (ids, cross-session isolation, collect/stop tools, completion notices) live in the generic `ctx.tasks` runtime ([dsh-tasks](../../packages/tasks/tasks)); the tool layer adapts a `BashProcess` handle into a task registration, so a sandboxed or remote executor inherits no session or registry dependency. -`stdin` and `env` are set by in-process plugins (the hooks bridges, native plugins) to feed a hook command its JSON payload on stdin and its `CLAUDE_PROJECT_DIR`/`CLAUDE_PLUGIN_ROOT` env. The model-facing `dsh-tool-bash` tool does not expose them as parameters — its request is built from `command`/`workdir`/`timeoutMs`/`signal`/`owner` only — because a model already has equivalent power through shell syntax (`FOO=bar cmd`, a heredoc), so duplicating them as tool params would be redundant. This is NOT a security boundary: the credential scrub in `dsh-bash-local` is what stops the harness's ambient secrets reaching a spawned command, and it works regardless of these fields (a model cannot read a value the scrub removed, and tool-call args are static JSON, never shell-evaluated). A guard test asserts the tool doesn't forward model `env`/`stdin` — to catch a future `...args` spread, not to defend a trust wall. `env` is merged AFTER the scrub so an explicit caller entry (a value it already holds) wins even on a credential-shaped name. See [the bash-stdin-env RFC](../rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md). - -Both ids the seam handles are [branded](core.md) (zero-cost `string` brands, the same machinery as `SessionId`/`AgentId`): `BashTaskId` (a tracked background task, generated `bash-N` by the local executor) and `OwnerToken` (the opaque isolation key). `OwnerToken` is deliberately a DISTINCT brand from `SessionId`, not an alias: the bash seam is a capability seam that must not know what an owner token *means*, so it never imports `dsh-session`'s vocabulary — the `dsh-tool-bash` consumer is the single boundary that casts the owning agent's `SessionId` into an `OwnerToken`. Branding both stops a raw `string` (or a `BashTaskId` where an `OwnerToken` is expected, or vice versa) from slipping through the type checker on the model-facing `task_id` path. +`stdin` and `env` are set by in-process plugins (the hooks bridges, native plugins) to feed a hook command its JSON payload on stdin and its `CLAUDE_PROJECT_DIR`/`CLAUDE_PLUGIN_ROOT` env. The model-facing `dsh-tool-bash` tool does not expose them as parameters — its request is built from `command`/`workdir`/`timeoutMs`/`signal` only — because a model already has equivalent power through shell syntax (`FOO=bar cmd`, a heredoc), so duplicating them as tool params would be redundant. This is NOT a security boundary: the credential scrub in `dsh-bash-local` is what stops the harness's ambient secrets reaching a spawned command, and it works regardless of these fields (a model cannot read a value the scrub removed, and tool-call args are static JSON, never shell-evaluated). A guard test asserts the tool doesn't forward model `env`/`stdin` — to catch a future `...args` spread, not to defend a trust wall. `env` is merged AFTER the scrub so an explicit caller entry (a value it already holds) wins even on a credential-shaped name. See [the bash-stdin-env RFC](../rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md). ## Foreground runs: `BashRunResult` @@ -122,29 +102,40 @@ interface CollectedOutput { } ``` -## Background tasks: `BashTask` +## Background processes: `BashProcess` -A long-running command started with `start()` is tracked as a `BashTask`. `BashTaskStatus` is `'running' | 'completed' | 'killed'`; `done` resolves when the underlying process closes and never rejects. +A long-running command started with `start()` returns a `BashProcess` **handle** — the only access path (no executor-level id lookup). `BashProcessStatus` is `'running' | 'completed' | 'killed'`; `done` resolves when the underlying process closes and never rejects (a spawn failure settles as `killed` with the error readable on stderr). Reads stay valid after exit: the remaining buffered output is still consumable through the handle. ```ts type-equiv -interface BashTask { - readonly id: BashTaskId +interface BashProcess { + /** The command line this process runs. */ readonly command: string - status: BashTaskStatus + /** Process lifecycle state (settled exactly once). */ + status: BashProcessStatus /** Exit code once finished (null = killed by signal / still running). */ exitCode: number | null /** Terminating signal name, when signal-killed. */ signal: NodeJS.Signals | null - /** Resolves when the underlying process closes (never rejects). */ + /** Resolves when the underlying process closes (never rejects — a spawn failure settles as `killed` with the error on stderr). */ readonly done: Promise + /** + * Read output produced since the previous read (consuming — consecutive + * reads never re-deliver). Reads that lost data flag `lossy` and point at + * full-stream spill files when available. + */ + readOutput(): BashProcessRead + /** + * Kill the process group. Returns false when it had already finished + * (no-op); idempotent. + */ + kill(): boolean } ``` -`readOutput()` returns an incremental `BashTaskRead` — the output produced since the previous read, with a `lossy` flag when truncation dropped unread bytes: +`readOutput()` returns an incremental `BashProcessRead` — the output produced since the previous read, with a `lossy` flag when truncation dropped unread bytes: ```ts type-equiv -interface BashTaskRead { - task: BashTask +interface BashProcessRead { /** Output produced since the previous read (stderr in a marked section). */ delta: string /** True when truncation dropped unread bytes the delta cannot include. */ @@ -158,4 +149,4 @@ interface BashTaskRead { ## The service -`BashExecutor` (`ctx.bash`, abstract — defined in [`packages/bash/bash/src/index.ts`](../../packages/bash/bash/src/index.ts)) mirrors the `LlmService`/`LlmAdapter` split: `resolve` (request → spec), `run` (foreground), `start` (background), `get`/`ownerOf`/`list`/`readOutput`/`kill`, and `onTaskDone` (a `BashTaskListener` completion callback). Spawned commands get a **scrubbed env** (dropping `*KEY*`/`*SECRET*`/`*TOKEN*`) and spill files use a private 0700 dir with random names and owner-only opens — model output never gets the ambient environment or a predictable path. The implementation that provides all this is `dsh-bash-local`; the model-facing `bash`/`bash_output`/`bash_kill` schemas that call it are in `dsh-tool-bash` (and present as terminals via the [tool-presentation vocabulary](tools.md#tool-presentation-ui-vocabulary)). +`BashExecutor` (`ctx.bash`, abstract — defined in [`packages/bash/bash/src/index.ts`](../../packages/bash/bash/src/index.ts)) mirrors the `LlmService`/`LlmAdapter` split and is exactly three methods: `resolve` (request → spec), `run` (foreground), `start` (background, returning the `BashProcess` handle). Spawned commands get a **scrubbed env** (dropping `*KEY*`/`*SECRET*`/`*TOKEN*`) and spill files use a private 0700 dir with random names and owner-only opens — model output never gets the ambient environment or a predictable path. The implementation that provides all this is `dsh-bash-local`; the model-facing `bash` schema that calls it is in `dsh-tool-bash` (background runs register with [`ctx.tasks`](../../packages/tasks/README.md) and are collected via the generic `task_output`/`task_kill`), presenting as terminals via the [tool-presentation vocabulary](tools.md#tool-presentation-ui-vocabulary). diff --git a/docs/core-data-structures/core.md b/docs/core-data-structures/core.md index 615c222d94..91667a304e 100644 --- a/docs/core-data-structures/core.md +++ b/docs/core-data-structures/core.md @@ -19,7 +19,8 @@ Everything else is documented on a **sub-page**, not here. The rule that draws t | [session.md](session.md) | the full `SessionEventMap` variant catalog, `TurnTrigger`/`TurnEndReason`, `deriveMessages()`, the turn-enclosure invariant | | [persistence.md](persistence.md) | the durability seam: `SessionPersistence`, JSONL + SQLite backends, `session/flush`, crash recovery, `SessionHeader` | | [tools.md](tools.md) | `ToolDefinition` full fields, the schema DSL, `ToolExecution`/`ToolResult`, tool-presentation UI types, the `tools/pre-execute`/`tools/post-execute` pipeline | -| [bash.md](bash.md) | the bash executor seam: `BashExecRequest`/`Spec`, `BashRunResult`, background `BashTask`s | +| [bash.md](bash.md) | the bash executor seam: `BashExecRequest`/`Spec`, `BashRunResult`, the background `BashProcess` handle | +| [tasks.md](tasks.md) | the background task runtime: `TaskId`, `TaskRegistration`, `TaskOutcome`, `TaskSnapshot`/`TaskRead`, owner isolation, the control-tool surface | | [code-runtime.md](code-runtime.md) | the code-execution seam: `CodeRunRequest`/`Result`, binding namespaces, captured logs, the `CodeRunFailure` taxonomy | | [filesystem.md](filesystem.md) | the filesystem seam: `FsTarget`, read/write/edit outcomes, observed-file state, `FsErrorCode` | | [compaction.md](compaction.md) | the compaction seam: the `compact/*` session events, `CompactionResult`, the `CompactService` interface | @@ -68,7 +69,7 @@ Two large discriminated unions are the ones consumers `switch` over most: **`Str IDs that cross package boundaries are **branded** — structurally strings, but non-interchangeable at the type level (an `AgentId` can't be passed where a `CallId` is expected). Construction goes through a per-type factory; comparison, logging, and JSON behave as ordinary strings. -The `Branded` primitive lives in its own type-only package, [dsh-brand](../../packages/util/brand) (no runtime code, no harness-package dependency), so any package can brand the ids it owns without depending on an unrelated capability package (e.g. dsh-bash brands `BashTaskId`/`OwnerToken` via dsh-brand alone, never pulling in dsh-llm). +The `Branded` primitive lives in its own type-only package, [dsh-brand](../../packages/util/brand) (no runtime code, no harness-package dependency), so any package can brand the ids it owns without depending on an unrelated capability package (e.g. dsh-tasks brands `TaskId` via dsh-brand alone, never pulling in dsh-llm). Source: [`packages/util/brand/src/index.ts`](../../packages/util/brand/src/index.ts) @@ -76,7 +77,7 @@ Source: [`packages/util/brand/src/index.ts`](../../packages/util/brand/src/index type Branded = string & { readonly [BRAND]: B } ``` -The three core IDs: `CallId` (correlates a tool call with its result; dsh-llm), `SessionId` (dsh-session), `AgentId` (dsh-agent). Each is `Branded<'CallId'>` etc. plus a same-named factory function. Capability seams brand their own ids too — see `BashTaskId`/`OwnerToken` in [bash.md](bash.md). +The three core IDs: `CallId` (correlates a tool call with its result; dsh-llm), `SessionId` (dsh-session), `AgentId` (dsh-agent). Each is `Branded<'CallId'>` etc. plus a same-named factory function. Capability seams brand their own ids too — see `TaskId` in [tasks.md](tasks.md). ## Content blocks and messages diff --git a/docs/core-data-structures/tasks.md b/docs/core-data-structures/tasks.md new file mode 100644 index 0000000000..f229e402ee --- /dev/null +++ b/docs/core-data-structures/tasks.md @@ -0,0 +1,126 @@ +# Background Task Runtime + +The shared background-task vocabulary — what a producer (`dsh-tool-bash`, `dsh-tool-subagent`, any future long-running tool) hands to `ctx.tasks.register()` and what consumers (the `task_output`/`task_list`/`task_kill` tools, completion-notice injection) get back. The runtime is ONE concrete service ([dsh-tasks](../../packages/tasks/tasks), `ctx.tasks`), not an interface/implementation seam pair — see [the runtime RFC](../rfc/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.md) for the decision and [the tasks group README](../../packages/tasks/README.md) for the package split. + +Source: [`packages/tasks/tasks/src/types.ts`](../../packages/tasks/tasks/src/types.ts) + +## Ids and status + +`TaskId` is [branded](core.md#branded-ids) (`Branded<'TaskId'>` + a same-named factory), generated by the registry as `-N` with a per-kind counter (`bash-1`, `subagent-1`) — kind-prefixed so transcripts stay self-describing, sequential because the owner fence (not id secrecy) is the isolation boundary. `TaskStatus` is generic and CLOSED: `'running' | 'stopping' | 'completed' | 'killed' | 'failed'` — kind-specific meaning (exit codes, stop reasons) rides in `TaskSnapshot.detail`, so the registry never learns process or agent semantics. + +## The producer contract: `TaskRegistration` + +A producer starts its work, then hands the running work over. The producer stays the owner of its execution concerns (process streams, child agents); the registry owns ids, isolation, status, and completion fan-out. The optional `readOutput` marks a STREAM kind — the method presence is the capability, mirroring `SubagentRun.sendMessage`. + +```ts type-equiv +interface TaskRegistration { + /** Producer kind — also the id prefix (`bash`, `subagent`, …). Non-empty. */ + kind: string + /** One-line model-facing label (the command; the delegation description). */ + label: string + /** + * The spawning agent. Its `session.header.id` becomes the task's owner + * token (read/kill/wait/list are fenced to that session), and its disposal + * cancels and awaits the task through the `ctx.agents.onCleanup` seam. + * `undefined` registers an UNOWNED task: open to any caller, alive until the + * tasks service disposes. + */ + owner?: Agent | undefined + /** + * Request termination. Idempotent, synchronous, and must lead to + * {@link done} settling; a throw propagates to the killer (fail loud — a + * cancel that cannot even be requested is a producer bug). The optional + * reason is `task_kill`'s logged reason, forwarded verbatim. + */ + cancel(reason?: string): void + /** + * Settles with the terminal outcome at QUIESCENCE — after the producer has + * released the task's resources (process exited, child agent disposed) — + * not merely when the work finished. Must never reject; a rejection is + * contained as a `failed` outcome and logged as a producer contract + * violation. + */ + done: Promise + /** + * OPTIONAL incremental read (stream kinds): everything produced since the + * previous call, formatted by the producer (truncation/spill notices + * included). Consecutive calls never re-deliver output; the registry keeps + * ONE consuming cursor per task, so v1's single intended reader is the + * owning model. Absence marks a final-output-only kind (the method presence + * IS the capability). + */ + readOutput?(): string +} +``` + +`register()` is ATOMIC: a throw (the no-control-surface fence, an owner-cleanup attach failure) mutates no registry state, so the producer cancels and awaits its just-started work and rethrows — background work never runs without a collectable id. + +```ts type-equiv +interface TaskOutcome { + /** How the task ended: finished (`completed`), cancelled (`killed`), or broke (`failed`). */ + status: 'completed' | 'killed' | 'failed' + /** Kind-specific detail rendered into status lines ('exit code: 3', 'max-tokens'). */ + detail?: string + /** + * Final output for FINAL-OUTPUT-ONLY kinds (no {@link TaskRegistration.readOutput}), + * read idempotently after the task settles. Stream kinds leave it unset — + * their output is consumed incrementally through `readOutput`. + */ + output?: string +} +``` + +## What consumers see: `TaskSnapshot` and `TaskRead` + +Snapshots are fresh projections, never live registry state. `reported` is the notice-suppression flag: the completion-notice injector (`dsh-tool-tasks`) skips a task whose terminal state the model already saw. + +```ts type-equiv +interface TaskSnapshot { + /** The registry-issued id (`-N`). */ + id: TaskId + /** The producer kind the task was registered with. */ + kind: string + /** The producer-supplied one-line label. */ + label: string + /** + * The owner's session id (`session.header.id`), for surfaces that must + * reach the owning agent (the completion-notice injector); absent for + * unowned tasks. Session ids are runtime-shared identifiers, not secrets — + * the read/kill/wait/list FENCE is what isolation rests on. + */ + ownerSession?: string + /** Current lifecycle state. */ + status: TaskStatus + /** Kind-specific status detail, present once the producer supplied one (usually terminal). */ + detail?: string + /** Epoch ms when the task was registered. */ + startedAt: number + /** Epoch ms when the task settled; absent while `running`/`stopping`. */ + finishedAt?: number + /** + * True once the terminal state has been (or is being) reported to the owner + * through an explicit surface response — a `kill` call, or a `read`/`wait` + * that returned the terminal state (including a wait pending at settlement). + * Completion-notice surfaces suppress their notice when set, so the model + * never gets a redundant "finished" for a task it just collected or killed. + */ + reported: boolean +} +``` + +```ts type-equiv +interface TaskRead { + /** + * Stream kinds: the consuming delta since the previous read. Final-output + * kinds: empty while live, the terminal {@link TaskOutcome.output} (or + * empty) once settled — idempotent, never consumed. + */ + text: string + /** The task's state at read time. */ + snapshot: TaskSnapshot +} +``` + +## The service + +`TaskService` (`ctx.tasks` — [`packages/tasks/tasks/src/index.ts`](../../packages/tasks/tasks/src/index.ts)): `register` (atomic, fenced by `attachSurface`), non-consuming `get`/`list` (caller-scoped — owned-by-caller plus unowned only), `read` (consuming for stream kinds), `kill` (producer `cancel` first; a throw leaves the task untouched), `wait` (bounded, abort cancels the wait only), and `onTaskDone` (a `TaskDoneListener` per settlement, effect-scoped, contained). Every read/kill/wait/get compares the task's owner session with the caller's and rejects a foreign one. Owned tasks are cancelled and awaited when their owning agent disposes (the `ctx.agents.onCleanup` seam); the model-facing surface over all of this is [dsh-tool-tasks](../../packages/tasks/tool-tasks/README.md). diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index c10d7caa52..8aef863ee0 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -25,10 +25,10 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `session/created` | `emit` | [`packages/core/session/src/index.ts:39`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`emit`) | [`invariants`](../packages/support/invariants), [`session-persistence`](../packages/session-persistence/session-persistence) | | `session/event` | `emit` | [`packages/core/session/src/index.ts:47`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`emit`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`session-persistence`](../packages/session-persistence/session-persistence), [`stdio-agent`](../packages/ui/stdio-agent) | | `session/flush` | `parallel` | [`packages/core/session/src/index.ts:57`](../packages/core/session/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`parallel`) | [`session-persistence`](../packages/session-persistence/session-persistence) | -| `subagent/end` | `emit` | [`packages/subagent/subagent/src/index.ts:98`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude`](../packages/hooks/hooks-claude) | -| `subagent/provider-added` | `emit` | [`packages/subagent/subagent/src/index.ts:72`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`emit`) | [`tool-subagent`](../packages/subagent/tool-subagent) | -| `subagent/provider-removed` | `emit` | [`packages/subagent/subagent/src/index.ts:83`](../packages/subagent/subagent/src/index.ts) | - | [`tool-subagent`](../packages/subagent/tool-subagent) | -| `subagent/start` | `emit` | [`packages/subagent/subagent/src/index.ts:91`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude`](../packages/hooks/hooks-claude) | +| `subagent/end` | `emit` | [`packages/subagent/subagent/src/index.ts:99`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude`](../packages/hooks/hooks-claude) | +| `subagent/provider-added` | `emit` | [`packages/subagent/subagent/src/index.ts:73`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`emit`) | [`tool-subagent`](../packages/subagent/tool-subagent) | +| `subagent/provider-removed` | `emit` | [`packages/subagent/subagent/src/index.ts:84`](../packages/subagent/subagent/src/index.ts) | - | [`tool-subagent`](../packages/subagent/tool-subagent) | +| `subagent/start` | `emit` | [`packages/subagent/subagent/src/index.ts:92`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude`](../packages/hooks/hooks-claude) | | `system-prompt/assemble` | `waterfall` | [`packages/core/system-prompt/src/index.ts:38`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`waterfall`) | - | | `system-prompt/change` | `emit` | [`packages/core/system-prompt/src/index.ts:44`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`emit`) | - | | `tools/change` | `emit` | [`packages/core/tools/src/index.ts:97`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`emit`) | - | diff --git a/docs/module-graph.md b/docs/module-graph.md index 5e043d22d4..e7f77dce7a 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -82,6 +82,10 @@ flowchart TD subgraph group_code_runtime["packages/code-runtime"] pkg_code_runtime["code-runtime"] end + subgraph group_tasks["packages/tasks"] + pkg_tasks["tasks"] + pkg_tool_tasks["tool-tasks"] + end pkg_llm --> pkg_brand pkg_bash --> pkg_brand pkg_llm_deepseek --> pkg_llm @@ -124,6 +128,8 @@ flowchart TD pkg_invariants --> pkg_agent pkg_invariants --> pkg_llm pkg_invariants --> pkg_session + pkg_tasks --> pkg_agent + pkg_tasks --> pkg_brand pkg_agent_loop --> pkg_agent pkg_agent_loop --> pkg_llm pkg_agent_loop --> pkg_session @@ -134,6 +140,7 @@ flowchart TD pkg_tool_bash --> pkg_bash pkg_tool_bash --> pkg_llm pkg_tool_bash --> pkg_system_prompt + pkg_tool_bash --> pkg_tasks pkg_tool_bash --> pkg_tools pkg_tool_fs --> pkg_fs pkg_tool_fs --> pkg_llm @@ -160,13 +167,19 @@ flowchart TD pkg_acp --> pkg_session pkg_acp --> pkg_session_persistence pkg_acp --> pkg_tools + pkg_tool_tasks --> pkg_agent + pkg_tool_tasks --> pkg_system_prompt + pkg_tool_tasks --> pkg_tasks + pkg_tool_tasks --> pkg_tools pkg_agent_core --> pkg_agent pkg_agent_core --> pkg_agent_loop pkg_agent_core --> pkg_invariants pkg_agent_core --> pkg_llm pkg_agent_core --> pkg_session pkg_agent_core --> pkg_system_prompt + pkg_agent_core --> pkg_tasks pkg_agent_core --> pkg_tool_bash + pkg_agent_core --> pkg_tool_tasks pkg_agent_core --> pkg_tools pkg_subagent_acp --> pkg_agent pkg_subagent_acp --> pkg_llm @@ -180,6 +193,7 @@ flowchart TD pkg_tool_subagent --> pkg_agent pkg_tool_subagent --> pkg_llm pkg_tool_subagent --> pkg_subagent + pkg_tool_subagent --> pkg_tasks pkg_tool_subagent --> pkg_tools pkg_hooks_claude --> pkg_agent pkg_hooks_claude --> pkg_hook_protocol @@ -239,18 +253,20 @@ flowchart TD | [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl) | `session-persistence` | [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence) | | [`session-persistence-sqlite`](../packages/session-persistence/session-persistence-sqlite) | `session-persistence` | [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence) | | [`invariants`](../packages/support/invariants) | `support` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session) | +| [`tasks`](../packages/tasks/tasks) | `tasks` | [`agent`](../packages/core/agent), [`brand`](../packages/util/brand) | | [`agent-loop`](../packages/core/agent-loop) | `core` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | -| [`tool-bash`](../packages/bash/tool-bash) | `bash` | [`agent`](../packages/core/agent), [`bash`](../packages/bash/bash), [`llm`](../packages/llm/llm), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | +| [`tool-bash`](../packages/bash/tool-bash) | `bash` | [`agent`](../packages/core/agent), [`bash`](../packages/bash/bash), [`llm`](../packages/llm/llm), [`system-prompt`](../packages/core/system-prompt), [`tasks`](../packages/tasks/tasks), [`tools`](../packages/core/tools) | | [`tool-fs`](../packages/fs/tool-fs) | `fs` | [`fs`](../packages/fs/fs), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`subagent`](../packages/subagent/subagent) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`tools`](../packages/core/tools) | | [`tool-web`](../packages/web/tool-web) | `web` | [`llm`](../packages/llm/llm), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`web`](../packages/web/web) | | [`tool-todo`](../packages/todo/tool-todo) | `todo` | [`agent`](../packages/core/agent), [`session`](../packages/core/session), [`tools`](../packages/core/tools) | | [`hooks-codex`](../packages/hooks/hooks-codex) | `hooks` | [`agent`](../packages/core/agent), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`tools`](../packages/core/tools) | | [`acp`](../packages/ui/acp) | `ui` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`tools`](../packages/core/tools) | -| [`agent-core`](../packages/core/agent-core) | `core` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt), [`tool-bash`](../packages/bash/tool-bash), [`tools`](../packages/core/tools) | +| [`tool-tasks`](../packages/tasks/tool-tasks) | `tasks` | [`agent`](../packages/core/agent), [`system-prompt`](../packages/core/system-prompt), [`tasks`](../packages/tasks/tasks), [`tools`](../packages/core/tools) | +| [`agent-core`](../packages/core/agent-core) | `core` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt), [`tasks`](../packages/tasks/tasks), [`tool-bash`](../packages/bash/tool-bash), [`tool-tasks`](../packages/tasks/tool-tasks), [`tools`](../packages/core/tools) | | [`subagent-acp`](../packages/subagent/subagent-acp) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`subagent`](../packages/subagent/subagent) | | [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | -| [`tool-subagent`](../packages/subagent/tool-subagent) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`subagent`](../packages/subagent/subagent), [`tools`](../packages/core/tools) | +| [`tool-subagent`](../packages/subagent/tool-subagent) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`subagent`](../packages/subagent/subagent), [`tasks`](../packages/tasks/tasks), [`tools`](../packages/core/tools) | | [`hooks-claude`](../packages/hooks/hooks-claude) | `hooks` | [`agent`](../packages/core/agent), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent), [`tools`](../packages/core/tools) | | [`subagent-mock`](../packages/support/subagent-mock) | `support` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`subagent`](../packages/subagent/subagent) | | [`subagent-fork`](../packages/subagent/subagent-fork) | `subagent` | [`agent`](../packages/core/agent), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | diff --git a/docs/rfc/INDEX.md b/docs/rfc/INDEX.md index 5f0da6a7ff..8b09679f36 100644 --- a/docs/rfc/INDEX.md +++ b/docs/rfc/INDEX.md @@ -13,7 +13,6 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; | [Code Mode — the model writes TypeScript against the tool registry](proposed/feature/2026-06-15-code-mode.md) | 2026-06-15 | | [Pre-tool input rewrite — a consistent design](proposed/feature/2026-06-30-pre-tool-input-rewrite.md) | 2026-06-30 | | [Claude Code and Codex subagent backends (out-of-process delegation to external coding agents)](proposed/feature/2026-07-07-claude-code-and-codex-subagent-backends.md) | 2026-07-07 | -| [Background subagent tasks](proposed/feature/2026-07-08-background-subagent-tasks.md) | 2026-07-08 | | [Repeat-tool-call guard plugin](proposed/feature/2026-07-08-repeat-tool-guard.md) | 2026-07-08 | ### Simplification @@ -28,7 +27,6 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; | Title | First proposed | |---|---| | [Runtime schemas for the event vocabulary (Zod vs the merge-extensible-map pattern)](proposed/architecture/2026-06-16-typed-event-schemas.md) | 2026-06-16 | -| [Extract a generic long-running tool runtime](proposed/architecture/2026-06-20-generic-long-running-tool-runtime.md) | 2026-06-20 | ### Process @@ -64,6 +62,7 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; | [SessionStore fork API](implemented/feature/2026-06-30-session-store-fork-api.md) | 2026-06-30 | | [Subagent lifecycle enrichment — lastAssistantMessage (observe-only)](implemented/feature/2026-06-30-subagent-observe-enrich.md) | 2026-06-30 | | [Explicit model-facing tool order](implemented/feature/2026-07-06-explicit-tool-order.md) | 2026-07-06 | +| [Background subagent tasks](implemented/feature/2026-07-08-background-subagent-tasks.md) | 2026-07-08 | ### Simplification @@ -111,6 +110,7 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; | [Shared persistence write coordinator](implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md) | 2026-06-18 | | [Branded IDs everywhere they belong](implemented/architecture/2026-06-20-branded-ids.md) | 2026-06-20 | | [Extract example apps into packages](implemented/architecture/2026-06-20-extract-example-app-packages.md) | 2026-06-20 | +| [The background task runtime (`ctx.tasks`) and the generic task control tools](implemented/architecture/2026-06-20-generic-long-running-tool-runtime.md) | 2026-06-20 | | [Reorganize packages into a modular hierarchy](implemented/architecture/2026-06-20-package-hierarchy.md) | 2026-06-20 | | [Mandatory `User-Agent` attribution for provider requests](implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md) | 2026-06-21 | | [Web capability seam - stable tools over multiple providers](implemented/architecture/2026-06-24-web-capability-seam.md) | 2026-06-24 | diff --git a/docs/rfc/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.md b/docs/rfc/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.md new file mode 100644 index 0000000000..a049df012f --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-06-20-generic-long-running-tool-runtime.md @@ -0,0 +1,171 @@ +# RFC: The background task runtime (`ctx.tasks`) and the generic task control tools + +Status: implemented + +## Problem + +The bash capability seam supports both foreground commands and long-running background tasks. Background support was large: the abstract executor exposed `start`, `get`, `ownerOf`, `list`, `readOutput`, `kill`, and `onTaskDone`; the local executor tracked tasks, incremental reads, owner tokens, process cleanup, and completion listeners; the model saw three tools (`bash`, `bash_output`, `bash_kill`); the tool plugin injected completion notices back into the owning agent's session. The local executor fenced task access behind owner tokens because predictable global task ids are a cross-session read/kill hazard. + +The [tool cookbook](../../../cookbook/adding-a-tool.md) already pointed at the real design smell: background bash is really generic long-running-tool infrastructure living inside one tool. The pressure stopped being hypothetical with [background subagent tasks](../feature/2026-07-08-background-subagent-tasks.md), which needs the same task ids, owner isolation, polling, stop, completion notices, and prompt guidance, and whose first draft answered by cloning the protocol under new names (`subagent_wait`, `subagent_output`, `subagent_stop`) and reshaping `dsh-tool-subagent` into a multi-tool plugin solely so the cloned companion tools would not collide across instances. Every future long-running capability (dev servers, watchers, remote jobs) would clone it again, and the model would learn a new collect/stop habit per capability. + +The surveyed peer products converged on the opposite shape. Claude Code exposes one `TaskOutput`/`TaskStop` pair spanning seven task kinds (background shells, subagents, remote sessions, …), with its earlier per-capability `BashOutput`/`KillShell` names kept only as aliases; Kimi Code's `BackgroundManager` runs process, agent, and pending-question kinds behind the same two tools and a ~5-method producer interface; DeepSeek-Reasonix serves bash and delegation from one session-scoped jobs manager; OpenCode's `BackgroundJob` registry is kind-agnostic by construction. The lesson is that the task registry, the control tools, and the notification path are one capability, and the producers (bash, subagents) are plugins into it. + +## Decision + +The `tasks/` package group owns background-task semantics once, and bash and subagents are producers: + +- `@deepseek-ai/dsh-tasks` — the task registry service (`ctx.tasks`): branded task ids, owner-scoped authorization, status snapshots, incremental/final output reads, cancellation, wait-for-terminal, completion listeners, and the awaited owner-cleanup path. +- `@deepseek-ai/dsh-tool-tasks` — the model-facing control surface: `task_output`, `task_list`, `task_kill`, the completion-notice injection into the owning session, and the system-prompt section that teaches the background-task habit. + +Producers register running work into `ctx.tasks` and stay owners of their execution concerns: `dsh-tool-bash`'s `run_in_background` path registers the process it started (incremental stdout, spill formatting, kill), and `dsh-tool-subagent`'s background mode ([the feature RFC](../feature/2026-07-08-background-subagent-tasks.md)) registers the child run (final output only, cancel + dispose). The bash seam carries no registry: `bash_output`/`bash_kill` no longer exist (the generic tools replaced them), and the subagent companion tools were never created. The `dsh-agent-core` bundle loads the pair, so every shipped deployment has the control surface. + +The registry is a CONCRETE service, not an interface/implementation seam pair: there is exactly one sensible in-process implementation today, and the capability-seam convention says not to split preemptively. The pre-release stance lets a later durable/remote job system extract an interface when a second backend actually exists. + +## Task model + +`dsh-tasks` owns the vocabulary ([data-structure catalog](../../../core-data-structures/tasks.md)). `TaskId` is branded, generated by the registry as `-N` with a per-kind counter (`bash-1`, `subagent-1`) — the kind prefix keeps ids self-describing in transcripts and preserves the pre-runtime `bash-N` shape. Ids are runtime-global and predictable, so every access is authorized (below). + +A producer registers a task with: + +```ts ignore-check +interface TaskRegistration { + /** Producer kind — also the id prefix ('bash', 'subagent', …). */ + kind: string + /** One-line model-facing label (the command; the delegation description). */ + label: string + /** The spawning agent; undefined = unowned (open access, dies with the service). */ + owner?: Agent + /** Request termination; idempotent; must lead to `done` settling. The optional reason is `task_kill`'s logged reason, forwarded. */ + cancel(reason?: string): void + /** Settles at QUIESCENCE — after the producer has released the task's resources. Never rejects. */ + done: Promise + /** OPTIONAL incremental read (stream kinds). Consecutive calls never re-deliver output; the producer owns truncation/spill formatting. Absence = final-output-only kind. */ + readOutput?(): string +} + +interface TaskOutcome { + status: 'completed' | 'killed' | 'failed' + /** Kind-specific detail rendered into the status line ('exit code: 3', 'max-tokens'). */ + detail?: string + /** Final output for final-only kinds; read idempotently after the task settles. */ + output?: string +} +``` + +The task status vocabulary is generic and closed: `running`, `stopping` (cancel requested, not yet settled), and the three terminal values above. Kind-specific meaning rides in `detail`, so the registry never learns process or agent semantics — the method presence (`readOutput`) is the capability, mirroring `SubagentRun.sendMessage`. + +The registry attaches ONE continuation to `done`: record the terminal snapshot, then notify task-done listeners with per-listener containment (the guarantee the bash seam's `notifyTaskDone` used to give its own listener set). `done` settling at quiescence — not merely at completion — is what makes owner cleanup and service disposal awaitable without a second completion surface; this resolves the old seam's duplication of a per-task `done` promise AND a global `onTaskDone` registry by making the promise the producer contract and the listener registry the consumer surface. + +Registrations are NOT effect-scoped to the registering fiber: a task belongs to its owning agent and its producing backend, not to the tool plugin whose call started it, so an HMR reload of `dsh-tool-bash` or `dsh-tool-tasks` never orphans or kills a running task (the same argument that used to keep bash ownership in the executor). The registry's own disposal cancels every live task and awaits settlement — no orphans survive `fiber.dispose()`. + +## Authorization and the service surface + +Cross-session isolation lives IN the runtime so every consumer gets the same rule for free: read/kill/wait/get take the caller (`Agent | undefined`), and a task whose owner session differs from the caller's session is rejected (`!== undefined` comparison — an unowned task is open, a no-agent caller cannot match an owned task). `list(caller)` returns only the caller-visible tasks (owned-by-caller or unowned) — a global listing would leak other sessions' labels. Owner identity is `session.header.id`, the canonical id every other subsystem keys on; because both sides of the comparison come from live `Agent`s, the freestanding `OwnerToken` brand the bash seam used to carry became internal state rather than a seam type. + +```ts ignore-check +class TaskService extends Service { // ctx.tasks + register(reg: TaskRegistration): TaskId // throws when no control surface is attached; ATOMIC — a throw mutates nothing + get(id: TaskId, caller?: Agent): TaskSnapshot // non-consuming; throws: unknown id, foreign owner + list(caller?: Agent): TaskSnapshot[] // caller-visible only + read(id: TaskId, caller?: Agent): TaskRead // delta (stream kinds, consuming) or final output (final kinds, idempotent) + snapshot + kill(id: TaskId, caller?: Agent, reason?: string): 'requested' | 'already-terminal' + wait(id: TaskId, timeoutMs: number, caller?: Agent, signal?: AbortSignal): Promise + onTaskDone(listener: (snapshot: TaskSnapshot) => void): () => void // effect-scoped, contained, never fires after dispose + attachSurface(name: string): () => void // the misconfiguration fence, below +} +``` + +`TaskSnapshot` is the read-only projection: id, kind, label, owner session, status, detail, started/finished timestamps, and the `reported` notice-suppression flag (below). `wait` resolves with the terminal snapshot, or with the still-`running` snapshot on timeout; aborting the wait cancels only the wait. + +**Misconfiguration fails loud**: a deployment that loads a background-capable producer without any control surface would let the model start tasks it can never read or stop — the half-loaded failure mode the subagent RFC's first draft reshaped a whole plugin to avoid. The fence is `attachSurface()`: `dsh-tool-tasks` attaches (effect-scoped) on load, and `register()` throws `background tasks unavailable: no control surface is attached (load @deepseek-ai/dsh-tool-tasks)` when none is attached — the earliest self-contained moment, since concurrent plugin start makes a load-time check racy. The registry stays ignorant of tool names; a deployment with a custom (non-model) surface attaches its own. + +## The model-facing control tools + +`dsh-tool-tasks` registers three kind-agnostic tools (ACP render intent: `generic` cards, `kind: 'execute'` for kill and `'read'` for output/list, no `locations`): + +- `task_output(task_id, wait?, timeout_ms?)` — non-blocking by default: stream kinds return output produced since the previous read, final kinds return only a status line while running and the final output once terminal; every response ends with the status line (`[status: running]`, `[status: completed, exit code: 0]`, `[status: failed, max-tokens]` — generic status + producer detail). `wait: true` blocks until the task settles or the timeout expires (config: defaulted `waitTimeoutMs`, capped `maxWaitTimeoutMs`); a timed-out wait returns `[status: running]` and leaves the task alive. Polling-by-default preserves the established bash habit; `wait` is what a parent uses when it is genuinely blocked on a subagent's answer. +- `task_list()` — the caller's tasks, one line each: ` []