Commit Graph

868 Commits

Author SHA1 Message Date
Tianyi Cui
fff2e1f33d workflow: render thrown script values inside the realm's execution window
Codex code-review round 3: the round-2 'contained stack getter' still let a
script escape the vm sync-slice timeout — throw { get stack() { while(true){} } }
put the spin on the HOST catch path, where no timeout applies (verified: a
direct sync-slice spin dies by the timeout; the getter-hidden one hung the
process). Identity-trusting the native getter is also insufficient: V8 stack
formatting reads script-controllable hooks at format time (Error.prepareStackTrace,
a subclass name getter — both empirically confirmed), so ANY host-side
formatting of a realm error can run realm code.

The fix moves rendering into the realm itself: the compiled body (and the meta
literal) is wrapped in a realm-side catch that pre-renders the thrown value to
a string (REALM_THROWN_RENDERER_SOURCE) — a hostile accessor/toString now runs
as ordinary script code, killed by the sync-slice timeout or falling under the
documented post-await spin limitation; host WorkflowErrors pass through for
the CANCELLED mapping. The host catch descriptor-reads the pre-rendered string
(thrownRendering) or falls back to describeThrown, which invokes no getter
whose identity is not the host realm's own native stack getter.

Tests: hostile-table expectations updated for realm-side rendering; new
regressions for the getter-hidden sync spin dying by the vm timeout (engine +
meta paths) and for a hostile thenable rejection that bypasses the realm
wrapper (renders host-side, proxy labelled, traps never run); describeThrown/
thrownRendering unit tables including the realm-error identity-mismatch case.
2026-07-05 20:32:35 +08:00
Tianyi Cui
57b9910339 workflow: total, contained rendering of hostile thrown script values
Codex code-review round 2: errorText() read .stack/.message as plain property
gets and fell back to String(error) — a script throwing a value with a
throwing accessor (or toString/Symbol.toPrimitive) ran realm code in drive()'s
catch and made WorkflowRun.result REJECT, which the detached workflow/end hook
turned into an unhandledRejection (process death under dsh-app-boot).

Replaced with describeThrown in dsh-workflow-vm/realm: total (never throws),
proxy-labelling before any inspection, own-descriptor reads, String() only on
primitives, and a CONTAINED stack-getter invocation — modern V8 (Node >= 22)
makes stack an own ACCESSOR on genuine Errors, so refusing all accessors would
lose every real stack and the lineOffset line numbers; a hostile getter's
throw is swallowed and rendering falls back to message. The meta-literal
eval catch had the same String(error) exposure and now uses the same renderer.

Regression tests: a hostile-thrown-values table through the real engine
(throwing stack/message getters, data stack, setter-only stack, proxy,
Symbol.toPrimitive, function, null) asserting result resolves 'error' with the
expected rendering and NO unhandledRejection fires; a meta-path hostile throw
mapping to META_INVALID.
2026-07-05 19:39:19 +08:00
Tianyi Cui
e264a106fd workflow, subagent: fix Codex code-review round-1 blockers
Six verified A-findings from the code-stage review, each with a regression test:

- parallel()/pipeline() resolved to HOST arrays inside the vm realm, exposing
  host Array.prototype to scripts; combinator results are now realm-built
  (in-realm Array.from bound at context setup).
- materializeFromRealm ran proxy traps (ownKeys/getOwnPropertyDescriptor/
  getPrototypeOf) during the descriptor walk — realm code on the host stack,
  outside the vm timeout, escaping as raw errors; proxies (root, nested, and
  in the prototype position) are now rejected trap-free via util.types.isProxy
  before any inspection.
- an already-aborted signal or an immediate cancel() no longer reports
  'completed' for a hook-free script: drive() checks cancellation before
  running the body and again when the script settles.
- dispose() now waits (bounded by disposeGraceMs) for stray agent() children
  to FINISH disposing, not just for the script to settle: every agent() call
  is tracked and quiesce() drains the in-flight set.
- workflow/* event payloads were live mutable aliases shared across emissions;
  emitWorkflowEvent now hands each listener its own structural clone.
- the structured-output turn-continuation veto is now prepend: true, so an
  earlier-registered force-continue listener cannot short-circuit it.

Docs updated in the same change (READMEs, core-data-structures/workflow.md,
the dynamic-workflows RFC, regenerated cordis catalogs).
2026-07-05 19:04:38 +08:00
Tianyi Cui
1d43ea3cd5 workflow: dynamic workflows — script-driven multi-agent orchestration
A new capability family at packages/workflow/ in the bash seam shape,
modeled on Claude Code's dynamic workflows: the model writes a JavaScript
orchestration script (export const meta = {...} + plain-JS body), a runtime
executes it, and the script — not the conversation — holds the loop, the
branching, and the intermediate results.

- dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary
  (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events
  carrying data snapshots (id + meta, never the live run), per-listener
  contained like subagent/*.
- dsh-workflow-vm: in-process node:vm engine. Meta extraction via a
  string/comment-aware scanner (template interpolation rejected; literal
  evaluated alone in an empty timed context; statement blanked line-
  preservingly so stacks keep script line numbers). Hooks: agent(prompt,
  {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline()
  (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline:
  hook misuse (unknown/deferred options, bad arguments, unsupported
  schemas, tripped caps, seam start failures, cancellation) throws fatal
  WorkflowErrors the combinators RE-THROW — never dissolved into the
  per-item null reserved for child failures. Realm boundary: inbound values
  materialized by descriptor walks that never invoke accessors (defineProperty
  copies, __proto__-safe); outbound values rebuilt in-realm via the
  context's own JSON.parse. Determinism bans (Date.now/Math.random/argless
  new Date) kept so future resume support cannot break scripts. Caps and
  timeouts are validated Config. Every hook promise carries a no-op
  rejection consumer (app-boot exits on unhandled rejections).
- dsh-tool-workflow: the model-facing workflow tool, synchronous like
  dsh-tool-subagent (start → await → try/finally dispose; abort bridged;
  non-completed → isError). Generic render card titled by a textual
  meta.name sniff. The tool description carries the authoring contract.

Wired into examples/{coding-agent,acp-agent} with explicit-ask-only
guidance. Coverage at every tier: unit (meta scanner, materializer incl.
counting-getter and __proto__ regressions, combinator semantics,
concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon),
integration over the real spawn stack, with-key e2e (real two-phase run +
the tool through the registry pipeline), and a recorded ACP snapshot
scenario (workflow-run, 1 child session). RFC:
docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred
work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's
layout line.
2026-07-05 13:29:35 +08:00
Tianyi Cui
dafb81be7b subagent: implement structured output for in-process backends
The seam vocabulary (SubagentStartRequest.outputSchema, SubagentResult
.structured) existed but no in-process backend honored it — spawn/fork
advertised outputSchema: false. This lands the missing half:

- dsh-tools gains a structured-output JSON Schema subset (json-schema.ts):
  StructuredOutputSchema, assertSupportedOutputSchema (rejects loud outside
  the enforced subset, every violation listed), validateStructuredValue
  (path-qualified issues, total). outputSchema's seam type becomes this raw
  JSON-Schema subset instead of the author-facing SchemaSpec DSL — the schema
  travels verbatim to the model as a forced tool's parameters.
- dsh-subagent-inprocess gains the shared structured runtime: one global
  structured_output capture tool (placeholder parameters) + a prepend:true
  agent/request listener doing FINAL-REQUEST enforcement (strip for plain
  agents, per-run schema for structured children — survives downstream
  request-replacing listeners) + an agent/turn-continuation veto that stops
  a child's turn once captured (no wasted extra model step). Lifetime is
  refcounted by backends (plugin lifetime) AND live runs (start→settle).
- startInProcessRun drives the capture: subset asserted before the child
  exists, instruction appended to the child's system prompt, clean-finish
  nudge loop (structuredNudgeRetries, backend Config, default 1), captured
  value on result.structured; a clean finish without a capture settles
  'error' (never a silent success with a missing field).
- spawn + fork flip outputSchema: true and inject 'tools'.
2026-07-05 11:35:39 +08:00
Tianyi Cui
2bad139ece Merge pull request #163 from deepseek-harness/split-cordis-catalog
Split the cordis catalog into separate events and services documents
2026-07-05 01:38:38 +08:00
Tianyi Cui
816a9f15f6 Merge remote-tracking branch 'origin/master' into split-cordis-catalog
# Conflicts:
#	docs/cordis-catalog/events.md
2026-07-05 01:22:05 +08:00
Tianyi Cui
1e2efc861e Merge pull request #81 from deepseek-harness/codex/app-attribution-rfc
Mandatory User-Agent attribution for provider requests (RFC + implementation)
2026-07-05 01:08:25 +08:00
Tianyi Cui
e06807ce0e Merge remote-tracking branch 'origin/master' into codex/app-attribution-rfc 2026-07-05 01:00:40 +08:00
Tianyi Cui
f8e06fd5d7 Merge remote-tracking branch 'origin/master' into split-cordis-catalog 2026-07-05 00:48:30 +08:00
Tianyi Cui
b9e2d7ed85 Merge master into app attribution RFC 2026-07-05 00:45:39 +08:00
Tianyi Cui
25e02ae9a8 Split the cordis catalog into separate events and services documents
gen-cordis-catalog.ts now emits docs/cordis-catalog/events.md and
docs/cordis-catalog/services.md instead of the combined
events-and-services.md: a reader is either finding what to listen to or
what to call, and each axis now scans and deep-links as its own page.
Headings promote one level (scopes and ctx.<key> entries become H2), the
dispatch-mode legend lives on the events page, and the inherited tier
splits accordingly. --check verifies both files and names whichever is
stale.

Every reference updated in the same change (no compat redirects,
pre-release stance): architecture.md, AGENTS.md, docs/AGENTS.md tier row,
filesystem/subagent core-data-structures pages (the ctx.fs anchor
survives — slugs are heading-level-independent), fs README, four RFCs,
the tool-catalog and persistence-catalog generator intros (both
regenerated), and the bilingual development.md pair (re-recorded).
2026-07-05 00:45:06 +08:00
Tianyi Cui
34f9d00227 Merge pull request #161 from deepseek-harness/persistence-log-catalog
Auto-generate a persistence log event catalog
2026-07-05 00:30:18 +08:00
Tianyi Cui
a52c27ee07 Merge remote-tracking branch 'origin/master' into persistence-log-catalog
# Conflicts:
#	packages/hooks/hook-protocol/README.md
#	packages/hooks/hook-protocol/src/types.ts
2026-07-05 00:06:47 +08:00
Tianyi Cui
d91b82ea61 Merge pull request #152 from deepseek-harness/simpl-e-acp-trim
refactor(acp): trim unreachable bridge surface (branding knobs, kind-sniffing fallback)
2026-07-04 23:45:21 +08:00
Tianyi Cui
021dae58e2 Merge remote-tracking branch 'origin/master' into simpl-e-acp-trim
# Conflicts:
#	docs/rfc/README.md
2026-07-04 23:43:20 +08:00
Tianyi Cui
a23982b30a Regenerate catalog for post-merge line shifts; drop pruned continuation trigger from turn/start JSDoc
Merging master brought the producerless-vocabulary prune that removed
TurnTriggerMap's continuation variant: the source pointers above
SessionEventMap shifted by one line (the staleness CI caught on the
merge tree), and the new turn/start JSDoc still named the pruned
variant.
2026-07-04 23:42:47 +08:00
Tianyi Cui
f56b09c5ce Merge pull request #151 from deepseek-harness/simpl-g-hook-contract
refactor(hooks): tighten the hook-protocol contract surface
2026-07-04 23:42:30 +08:00
Tianyi Cui
2c9e5ee756 Merge remote-tracking branch 'origin/master' into persistence-log-catalog 2026-07-04 23:41:57 +08:00
Tianyi Cui
9857bb0c07 Merge remote-tracking branch 'origin/master' into simpl-g-hook-contract
# Conflicts:
#	docs/rfc/README.md
2026-07-04 23:40:05 +08:00
Tianyi Cui
b46e805207 Merge pull request #148 from deepseek-harness/simpl-d-fs-write-only
refactor(fs): prune write-only fields and the dead routing knob from the seam
2026-07-04 23:38:51 +08:00
Tianyi Cui
f90f1c358e Merge remote-tracking branch 'origin/master' into simpl-e-acp-trim
# Conflicts:
#	docs/rfc/README.md
2026-07-04 23:22:16 +08:00
Tianyi Cui
c01d8c0dfc Merge remote-tracking branch 'origin/master' into simpl-g-hook-contract
# Conflicts:
#	docs/rfc/README.md
2026-07-04 23:22:08 +08:00
Tianyi Cui
54fc3d65ab Merge remote-tracking branch 'origin/master' into simpl-d-fs-write-only
# Conflicts:
#	docs/rfc/README.md
2026-07-04 23:21:46 +08:00
Tianyi Cui
2728e3bd5b Close remaining keyof-join holes the review verification found
- The owning top-level interface SessionEventMap must now be the SINGLE
  EXPORTED declaration in @deepseek-ai/dsh-session: a non-exported local
  interface (even inside the owning package) and a second exported copy
  are hard errors, so a same-named helper can no longer be catalogued as
  the on-disk vocabulary.
- Any SessionEventMap declaration carrying an extends clause is a hard
  error: inherited keys join keyof SessionEventMap but have no catalog
  row, so heritage is a silent-skip path the gate must reject.

Three new spec cases; RFC and module doc updated to match.
2026-07-04 23:21:23 +08:00
Tianyi Cui
a8b6fb6989 Merge pull request #158 from deepseek-harness/simpl-a3-knobs
refactor(llm): drop the inert request knobs — prefill and strict
2026-07-04 23:20:19 +08:00
Tianyi Cui
c64cfbbe5d Merge remote-tracking branch 'origin/master' into simpl-a3-knobs
# Conflicts:
#	docs/rfc/README.md
2026-07-04 23:17:51 +08:00
Tianyi Cui
9cce15919f Merge pull request #157 from deepseek-harness/simpl-a2-vocab
refactor(vocab): prune producer-less variants — cache hints, agent source, continuation trigger
2026-07-04 23:16:42 +08:00
Tianyi Cui
53dc3c8a28 Harden generator against Codex review findings
- A SessionEventMap member that is not a property signature with an
  explicit payload type is now a hard error instead of silently skipped —
  a method-form or type-less member joins keyof SessionEventMap and must
  not escape the catalog.
- A top-level interface SessionEventMap outside @deepseek-ai/dsh-session
  (ownership read from the package manifest) is now a hard error — an
  unrelated same-named local interface was previously catalogued as the
  on-disk vocabulary.
- JSDoc tag detection runs on the trimmed line, so an extra-indented
  '*  @mode' can no longer bypass the forbidden-tag check and leak into
  prose.

Four new spec cases cover these; RFC and module doc updated to describe
the enforced (not just assumed) invariants.
2026-07-04 23:11:42 +08:00
Tianyi Cui
9862eae898 Merge remote-tracking branch 'origin/master' into simpl-a2-vocab
# Conflicts:
#	docs/rfc/README.md
2026-07-04 23:11:07 +08:00
Tianyi Cui
daa737fe9c Merge pull request #155 from deepseek-harness/simpl-b3-replay
test(acp): derive the replay config from the live config
2026-07-04 23:09:17 +08:00
Tianyi Cui
e13217ef49 Merge remote-tracking branch 'origin/master' into simpl-b3-replay 2026-07-04 23:06:59 +08:00
Tianyi Cui
2bd155291a Merge pull request #153 from deepseek-harness/simpl-b2-app-boot
feat(ui): share the app bins' boot glue in dsh-app-boot
2026-07-04 23:06:31 +08:00
Tianyi Cui
c961850302 Condense the ui/ tree line to fit the AGENTS.md word ceiling 2026-07-04 23:04:50 +08:00
Tianyi Cui
232f314c3a Add generated persistence log event catalog with freshness + completeness gates
docs/persistence-catalog/log-events.md enumerates every SessionEventMap
member — the owning dsh-session vocabulary plus the dsh-compact and
dsh-hook-protocol declaration merges — with payload, surface/log-only badge,
JSDoc prose, and declaration site. scripts/gen-persistence-catalog.ts is a
pure AST pass in the gen-cordis-catalog mold: verify-persistence-catalog
(--check) joins doc-sync, so a stale committed catalog fails pre-push and CI.

The walk enforces JSDoc completeness (every member needs description prose;
@mode is rejected as a category error — log events do not dispatch on the
cordis bus), derives the surface badge from the SurfaceEventType union with a
stale-member cross-check, and hard-errors on duplicate declarations. Payloads
render through the TypeScript printer so newline-separated multi-line type
literals still emit valid one-line fragments.

Documented the five previously JSDoc-less core events (turn/step boundaries,
tool/call), removed the two stray @mode tags on the hook/* merges, and
replaced the hand-restated event enumerations (session.md hook/* table,
compact README table, hook-protocol README bullets, session README name-list
— whose merge note had already drifted) with links to the catalog. RFC:
docs/rfc/implemented/process/2026-07-04-persistence-log-catalog.md.
2026-07-04 22:58:28 +08:00
Tianyi Cui
8f62086feb Merge remote-tracking branch 'origin/master' into simpl-b2-app-boot
# Conflicts:
#	docs/rfc/README.md
2026-07-04 22:58:26 +08:00
Tianyi Cui
31c2f49725 Merge pull request #147 from deepseek-harness/simpl-f-steering-mirror
refactor(events): remove the agent/steering mirror emit
2026-07-04 22:56:49 +08:00
Tianyi Cui
a6feec6d04 Merge remote-tracking branch 'origin/simpl-f-steering-mirror' into simpl-f-steering-mirror 2026-07-04 22:54:09 +08:00
Tianyi Cui
5f295de096 Merge remote-tracking branch 'origin/master' into simpl-f-steering-mirror
# Conflicts:
#	docs/rfc/README.md
2026-07-04 22:50:47 +08:00
Tianyi Cui
5d6b1fd24a Merge pull request #154 from deepseek-harness/simpl-a1-drop-image
refactor(llm): drop the image content block until a path can honor it
2026-07-04 22:49:14 +08:00
Tianyi Cui
544cc0439e Merge remote app attribution branch
Resolve the RFC and implementation to defer OpenRouter-specific attribution headers and keep mandatory attribution to User-Agent only.
2026-07-04 22:47:53 +08:00
Tianyi Cui
1ddd6a9eea Merge remote-tracking branch 'origin/master' into simpl-a1-drop-image
# Conflicts:
#	docs/rfc/README.md
2026-07-04 22:46:55 +08:00
Tianyi Cui
e72aa84eb3 Merge pull request #150 from deepseek-harness/simpl-c-web-observation
refactor(web): drop the unconsumed observation surface (providers-change, status methods)
2026-07-04 22:45:17 +08:00
Tianyi Cui
80609cca11 Merge remote-tracking branch 'origin/master' into simpl-c-web-observation
# Conflicts:
#	docs/rfc/README.md
2026-07-04 22:39:41 +08:00
Tianyi Cui
c0349051c6 Merge pull request #149 from deepseek-harness/simpl-b1-fold-ui-stdio
refactor(ui): fold the stdio UI helper into the stdio app
2026-07-04 22:36:43 +08:00
Tianyi Cui
8d4153d5fa Merge branch 'master' into simpl-f-steering-mirror 2026-07-04 22:33:24 +08:00
Tianyi Cui
1891cb52d5 Defer OpenRouter-specific attribution headers 2026-07-04 22:33:15 +08:00
Tianyi Cui
0b97b2f7c1 restore hook/result durationMs — review keeps wall-clock audit timing durable
Reverses item 3 of the tighten-hook-protocol-contract RFC per review:
a persistence log is written for future readers, and hook wall-clock
runtime is audit signal (which hook made a turn slow). runHook keeps
its injected now clock and RunHookResult wrapper, the bridges pass the
measured duration through HookResultRecord, the snapshot normalizer
keeps its replay scrub, and the hook fixtures carry the field again.
The RFC records the reversal; the other three prunes stand.
2026-07-04 22:00:12 +08:00
Tianyi Cui
663e57c6d6 Merge branch 'simpl-a2-vocab' into simpl-a3-knobs
# Conflicts:
#	packages/core/tools/src/index.ts
2026-07-04 21:20:55 +08:00
Tianyi Cui
060f272093 Merge branch 'simpl-b2-app-boot' into simpl-b3-replay 2026-07-04 21:18:39 +08:00