Files
deepseek-harness/packages/workflow/workflow-workerthread

@deepseek-ai/dsh-workflow-workerthread

The WorkflowService implementation, on node:worker_threads: each run gets its OWN worker thread (one run = one worker, no pooling — a run is heavyweight, so the ~tens-of-ms thread spin-up is noise), the script executes in a vm context INSIDE that worker with the workflow hooks injected, and every agent() call bridges back over the message port to ctx.subagents on the host. Child agents are I/O-bound LLM loops and stay on the host event loop; the thread isolates the SCRIPT, the only part that can spin synchronously.

Trust premise: what the thread buys (and what it does not)

Workflow scripts are model-written — the same trust level as the model's existing bash access — so this engine defends against buggy scripts, never hostile ones. A worker thread is NOT a security boundary: the vm context inside it is escapable by construction (node:vm shares object machinery with its surrounding realm, so a script can reach the Function constructor via globalThis.constructor.constructor and from it process and every Node builtin), and an escapee holds the same process privileges as the host — Node's permission model is process-wide. The absent globals are API surface that keeps honest scripts portable, not walls. What the thread concretely buys:

  • The host never blocks: start() returns without running any script code on the host; a synchronous spin anywhere in the script occupies the worker's loop, not the harness's.
  • Termination is real: a script that outlives its post-cancel grace is worker.terminate()d — nothing of it survives dispose(), where an in-process engine could only abandon the spin on its own loop.
  • No ambient credentials: the worker spawns with an EMPTY environment (env: {} plus hermetic execArgv, the same stance as dsh-code-runtime-worker; the unbuilt dev shape forwards exactly one loader variable, TSX_TSCONFIG_PATH — path plumbing, not a secret), so an escapee reading process.env finds no harness secrets — ambient-channel hardening only; the process-wide privileges above (fs and the rest) remain, so a genuine sandbox is still the engine swap.
  • Serialization by construction: everything crossing the thread is structured-clone data, and plain JSON before that — the materializeFromRealm walk rejects loud what JSON cannot carry, which is also what makes every postMessage hop total.

What the seam guarantees regardless, because benign scripts hit these constantly: result never rejects, a dropped hook promise never becomes an unhandled rejection, values JSON cannot carry are rejected loud instead of silently mangled, and hook misuse is fatal instead of dissolving into a per-item null. Genuine sandboxing (containing what an escaped script may touch) remains an isolated-vm/separate-process engine swap behind the seam, still deferred.

The script contract it executes

  • Meta as DATA (validateMeta, host-side): the workflow's identity arrives on the start request as plain JSON (the tool carries it as its schema-validated meta parameter — never as script text) and is shape-validated loud, every violation named (name/description required; unknown fields rejected). The engine deliberately evaluates NO script text to obtain meta: an evaluated meta literal could smuggle getters that run on the host outside any vm timeout — the exact spin the worker thread exists to isolate. A body that still opens with a Claude Code-style export const meta statement is rejected with a pointed SCRIPT_PARSE message.
  • Hooks: agent(prompt, {label, phase, schema, model}) (schema = the structured-output subset, forwarded as outputSchema; result = validated object, or final text without a schema; a failed child resolves null), parallel(thunks), pipeline(items, ...stages) with NO cross-stage barrier and (prev, item, index) stage callbacks, phase(title), log(message), and the args global. Anything else — effort/isolation/agentType, unknown options, malformed arguments, schemas outside the subset — throws a FATAL WorkflowError that parallel/pipeline re-throw rather than nulling (see the seam README's failure discipline).
  • No ambient APIs: no timers, filesystem, or Node APIs are injected into the context (absence is API surface, not containment — see the trust premise).

How a run executes

start() shape-validates the meta DATA host-side and parse-checks the body with the identical wrapper the worker compiles (new vm.Script, discarded), preserving the seam's synchronous META_INVALID/SCRIPT_PARSE throws; one redundant parse per run is the deliberate price. It then spawns the worker (src/worker.ts unbuilt via an explicit tsx execArgv; the sibling lib/worker.js bundle when built) with the meta, body, args, and worker-side limits as workerData.

Inside the worker, runWorkerSession builds the execution core (hooks, combinators, concurrency semaphore, caps, fatal-error discipline) over a child port. agent() sends child-start, and the host starts the child on ctx.subagents with parent attribution, the shared per-run abort signal, and outputSchema/model pass-through.

The host observes run.result immediately but buffers its snapshotted wire projection until run.started fulfills. It then replies child-started with the child id before forwarding settlement, so workflow/agent-start always names a ready child and precedes its end. A readiness rejection replies child-start-error, emits no workflow agent pair, and makes the host dispose the attempt because the worker never received a handle; the worker classifies it as fatal AGENT_START unless cancellation already owns the run. If readiness fulfills, an infrastructure result rejection crosses as child-failed/AGENT_RESULT regardless of whether that rejection settled before or after readiness. Child disposal acknowledgements complete the RPC.

Observer narration (phase/log/agent-start/agent-end) crosses as messages and re-emits as the seam's workflow/* events. A ready→go handshake gates the body: a cancellation racing worker boot arrives before go, so a run cancelled before start never executes the body at all.

The value boundary

Values LEAVING the script (hook options/schemas, the script's return) are materialized by materializeFromRealm: a plain recursive walk that rejects loud everything JSON cannot carry (exotic prototypes, functions, symbols, cycles, sparse arrays, non-finite numbers, nested undefined), copying into plain containers via defineProperty so a "__proto__" key becomes a data property, never a prototype mutation. Getters are read ordinarily — the RESULT is what crosses; a read that throws fails loud; both run in the WORKER, never on the host. Values ENTERING the realm (args, agent() results, hook promises and their failures, combinator arrays) are handed over directly as worker-realm values — the script is trusted, so outer prototypes are not a leak; args is cloned once at start so a script scribbling on it cannot mutate the caller's object. One script-visible consequence: an error thrown by a hook is built OUTSIDE the script's vm context, so e instanceof Error inside the script is false — branch on e.name/e.code instead (the combinators recognize fatality by instanceof against their own realm's class, which a script-built object can never pass, so fatal-vs-null cannot be forged or dissolved).

Cancellation, death, disposal

Per-run limits: a concurrency semaphore (maxConcurrentAgents), a total-agent() cap (maxTotalAgents), and a per-call item cap (maxItemsPerCall), all config. cancel() posts the cancel to the worker (its hooks start throwing CANCELLED; the script dies at its next await) and cancels every host-side child NOW on both seam channels — the shared request signal aborts AND each registered child's explicit cancel() is called host-side, because the seam leaves a provider free to honor either channel and a worker wedged in a synchronous spin could not relay its own per-child cancel RPCs (those later land as idempotent no-ops). The grace then arms: a run still unsettled disposeGraceMs later force-settles cancelled and the worker is terminated. A cancellation that lands before the body runs (the ready→go handshake) reports cancelled without executing anything; a worker result racing an in-flight host cancellation reports cancelled too (first-wins settlement — the seam-visible result had not settled when cancellation was requested); post-cancel phase/log narration is suppressed host-side, while cancelled children still deliver their paired agent-end.

A worker that dies unexpectedly (an OOM, a script reaching process.exit through the documented vm escape) settles the run stopReason: 'error' with the exit diagnostics — or 'cancelled' when a cancel was in flight — and the host-side child registry is what winds every surviving child down. dispose() = cancel + immediate host-driven disposal of every registered child (a wedged worker can relay no dispose RPC, so child teardown overlaps the grace instead of starting after it; the worker's own dispose RPCs join the same per-child disposal) + bounded wait (result, then child-registry quiescence, capped by the grace) + unconditional worker.terminate(): the thread never outlives its run. Once a run settles, stray children a script fired without awaiting are cancelled too, and dispose() waits for their disposal (bounded by the grace) before returning. agent-start/agent-end pairing is host-guaranteed the same way: forwarded starts live in a ledger, worker-reported ends pair them on the graceful paths, and the termination paths (grace force-settle, worker death) synthesize the missing ends (outcome cancelled) before the run settles — a start still in flight across the force-settle can surface after workflow/end, immediately paired the same way.

A returned promise or thenable resolves per JavaScript semantics BEFORE materialization — that is what makes an un-awaited return agent('x') work — with the value-boundary guard applying to the resolution.

Config

Key Default Meaning
provider spawn The ctx.subagents provider children run on (host-side).
maxConcurrentAgents 0 (auto) Concurrent agent() ceiling; 0 resolves to min(16, max(1, cores - 2)).
maxTotalAgents 1000 Total agent() calls one run may start (runaway-loop backstop).
maxItemsPerCall 4096 Items accepted by one parallel()/pipeline() call.
syncTimeoutMs 5000 vm timeout for the script's initial synchronous slice (in the worker).
disposeGraceMs 5000 How long a cancelled run may stay unsettled before force-settle + terminate; also bounds dispose().

Model Experience

Context surface What the model sees Token effect
Child-agent requests Every script agent() call sends its prompt and optional model or structured-output schema to a subagent provider. Each child sees that provider's own context; phase and log narration stays on observer events. Potentially many independent child contexts are paid, bounded by maxConcurrentAgents, maxTotalAgents, and maxItemsPerCall; they never join the parent history directly.
Parent tool result, indirectly The parent sees only the script's materialized final JSON value, child count, or error through dsh-tool-workflow. Intermediate child outputs are available to the script but not the parent model. Zero direct parent tokens from this engine. Final result size is capped by the tool consumer and retained until compaction.

Known Limitations and Deferred Work

  • The worker/vm is not a security boundary — model-written code can escape node:vm and reach the worker's process authority; a hostile-code deployment needs a separate-process or container engine.
  • One worker thread is paid per run — there is no pool, warm runtime, or cross-run script cache.
  • No ambient timers, filesystem, or network are injected, but escaped code can still reach Node — the missing globals are portability API, not containment.
  • Termination can only report host-observed startsagentsStarted excludes worker-side calls still queued behind concurrency when a forced termination makes them unknowable.
  • Cross-realm errors fail instanceof Error inside scripts — workflow authors must branch on stable fields such as name and code.