deepseek-harness/packages/workflow/workflow-workerthread

@deepseek-ai/dsh-workflow-workerthread

The WorkflowService implementation, on node:worker_threads: each run gets its OWN worker thread (one run = one worker, no pooling — a run is heavyweight, so the ~tens-of-ms thread spin-up is noise), the script executes in a vm context INSIDE that worker with the workflow hooks injected, and every agent() call bridges back over the message port to ctx.subagents on the host. Child agents are I/O-bound LLM loops and stay on the host event loop; the thread isolates the SCRIPT, the only part that can spin synchronously.

Trust premise: what the thread buys (and what it does not)

Workflow scripts are model-written — the same trust level as the model's existing bash access — so this engine defends against buggy scripts, never hostile ones. A worker thread is NOT a security boundary: the vm context inside it is escapable by construction (node:vm shares object machinery with its surrounding realm, so a script can reach the Function constructor via globalThis.constructor.constructor and from it process and every Node builtin), and an escapee holds the same process privileges as the host — Node's permission model is process-wide. The absent globals are API surface that keeps honest scripts portable, not walls. What the thread concretely buys:

• The host never blocks: start() returns without running any script code on the host; a synchronous spin anywhere in the script occupies the worker's loop, not the harness's.
• Termination is real: a script that outlives its post-cancel grace is worker.terminate()d — nothing of it survives dispose(), where an in-process engine could only abandon the spin on its own loop.
• Serialization by construction: everything crossing the thread is structured-clone data, and plain JSON before that — the materializeFromRealm walk rejects loud what JSON cannot carry, which is also what makes every postMessage hop total.

What the seam guarantees regardless, because benign scripts hit these constantly: result never rejects, a dropped hook promise never becomes an unhandled rejection, values JSON cannot carry are rejected loud instead of silently mangled, and hook misuse is fatal instead of dissolving into a per-item null. Genuine sandboxing (containing what an escaped script may touch) remains an isolated-vm/separate-process engine swap behind the seam, still deferred.

The script contract it executes

• Meta as DATA (validateMeta, host-side): the workflow's identity arrives on the start request as plain JSON (the tool carries it as its schema-validated meta parameter — never as script text) and is shape-validated loud, every violation named (name/description required; unknown fields rejected). The engine deliberately evaluates NO script text to obtain meta: an evaluated meta literal could smuggle getters that run on the host outside any vm timeout — the exact spin the worker thread exists to isolate. A body that still opens with a Claude Code-style export const meta statement is rejected with a pointed SCRIPT_PARSE message.
• Hooks: agent(prompt, {label, phase, schema, model}) (schema = the structured-output subset, forwarded as outputSchema; result = validated object, or final text without a schema; a failed child resolves null), parallel(thunks), pipeline(items, ...stages) with NO cross-stage barrier and (prev, item, index) stage callbacks, phase(title), log(message), and the args global. Anything else — effort/isolation/agentType, unknown options, malformed arguments, schemas outside the subset — throws a FATAL WorkflowError that parallel/pipeline re-throw rather than nulling (see the seam README's failure discipline).
• No ambient APIs: no timers, filesystem, or Node APIs are injected into the context (absence is API surface, not containment — see the trust premise).

How a run executes

start() shape-validates the meta DATA host-side and parse-checks the body with the identical wrapper the worker compiles (new vm.Script, discarded), preserving the seam's synchronous META_INVALID/SCRIPT_PARSE throws; one redundant parse per run is the deliberate price. It then spawns the worker (src/worker.ts unbuilt via an explicit tsx execArgv; the sibling lib/worker.js bundle when built) with the meta, body, args, and worker-side limits as workerData.

Inside the worker, runWorkerSession builds the execution core (hooks, combinators, concurrency semaphore, caps, fatal-error discipline) over a child port: agent() sends child-start and the host starts the child on ctx.subagents (parent attribution, the shared per-run abort signal, outputSchema/model pass-through), replying with the child id, its settlement (a JSON projection; an infrastructure REJECTION crosses as child-failed and stays the fatal AGENT_RESULT), and dispose acks. Observer narration (phase/log/agent-start/agent-end) crosses as messages and re-emits as the seam's workflow/* events. A ready→go handshake gates the body: a cancellation racing worker boot arrives before go, so a run cancelled before start never executes the body at all.

The value boundary

Values LEAVING the script (hook options/schemas, the script's return) are materialized by materializeFromRealm: a plain recursive walk that rejects loud everything JSON cannot carry (exotic prototypes, functions, symbols, cycles, sparse arrays, non-finite numbers, nested undefined), copying into plain containers via defineProperty so a "__proto__" key becomes a data property, never a prototype mutation. Getters are read ordinarily — the RESULT is what crosses; a read that throws fails loud; both run in the WORKER, never on the host. Values ENTERING the realm (args, agent() results, hook promises and their failures, combinator arrays) are handed over directly as worker-realm values — the script is trusted, so outer prototypes are not a leak; args is cloned once at start so a script scribbling on it cannot mutate the caller's object. One script-visible consequence: an error thrown by a hook is built OUTSIDE the script's vm context, so e instanceof Error inside the script is false — branch on e.name/e.code instead (the combinators recognize fatality by instanceof against their own realm's class, which a script-built object can never pass, so fatal-vs-null cannot be forged or dissolved).

Cancellation, death, disposal

Per-run limits: a concurrency semaphore (maxConcurrentAgents), a total-agent() cap (maxTotalAgents), and a per-call item cap (maxItemsPerCall), all config. cancel() posts the cancel to the worker (its hooks start throwing CANCELLED; the script dies at its next await) and cancels every host-side child NOW on both seam channels — the shared request signal aborts AND each registered child's explicit cancel() is called host-side, because the seam leaves a provider free to honor either channel and a worker wedged in a synchronous spin could not relay its own per-child cancel RPCs (those later land as idempotent no-ops). The grace then arms: a run still unsettled disposeGraceMs later force-settles cancelled and the worker is terminated. A cancellation that lands before the body runs (the ready→go handshake) reports cancelled without executing anything; a worker result racing an in-flight host cancellation reports cancelled too (first-wins settlement — the seam-visible result had not settled when cancellation was requested); post-cancel phase/log narration is suppressed host-side, while cancelled children still deliver their paired agent-end.

A worker that dies unexpectedly (an OOM, a script reaching process.exit through the documented vm escape) settles the run stopReason: 'error' with the exit diagnostics — or 'cancelled' when a cancel was in flight — and the host-side child registry is what winds every surviving child down. dispose() = cancel + bounded wait (result, then child-registry quiescence, capped by the grace) + unconditional worker.terminate(): the thread never outlives its run. Once a run settles, stray children a script fired without awaiting are cancelled too, and dispose() waits for their disposal (bounded by the grace) before returning.

Engine-specific limitations: worker startup is paid per run; on a termination path agentsStarted reports the HOST-observed count (accepted child-starts — calls still queued worker-side for a concurrency slot are unknowable then); and a returned promise or thenable resolves per JavaScript semantics BEFORE materialization — that is what makes an un-awaited return agent('x') work — with the value-boundary guard applying to the resolution.

Config

Key	Default	Meaning
provider	spawn	The ctx.subagents provider children run on (host-side).
maxConcurrentAgents	0 (auto)	Concurrent agent() ceiling; 0 resolves to min(16, max(1, cores - 2)).
maxTotalAgents	1000	Total agent() calls one run may start (runaway-loop backstop).
maxItemsPerCall	4096	Items accepted by one parallel()/pipeline() call.
syncTimeoutMs	5000	vm timeout for the script's initial synchronous slice (in the worker).
disposeGraceMs	5000	How long a cancelled run may stay unsettled before force-settle + terminate; also bounds dispose().