Commit Graph

664 Commits

Author SHA1 Message Date
Yichen Jiang
35acd34fd9 fix(tools): enforce a disabled run_in_background at execution time (review findings)
enableRunInBackground: false removed the parameter from the advertised
schema only — the arg validator deliberately allows undeclared keys, so
a caller (or a model that has seen the parameter elsewhere) could still
force run_in_background: true and start background work past the
deployment's opt-out, in both tool-bash and tool-subagent. Both
producers now refuse the forced key loud in execute(); tests pin the
refusal (and that nothing spawns) alongside the untouched foreground
path; the schema-omission-is-advertising rule is recorded in the
runtime RFC and both READMEs.
2026-07-09 23:37:51 +08:00
Yichen Jiang
3ad561862f Merge remote-tracking branch 'origin/master' into codex/rfc-subagent-background-tasks
Adopts #201 (session-prefix composition before pre-step + the pressure
gate). Only the generated RFC index conflicted; regenerated.
2026-07-09 23:17:16 +08:00
Yichen Jiang
43d60286e3 fix(tasks): an aborted wait must not leave a settled task notice-suppressed (review finding)
Two windows where settle() could read a live waiter count, mark the
task reported (suppressing the completion notice), and then watch that
waiter reject with 'wait aborted' delivering nothing — leaving the
owning session with no terminal notification at all:

- abort and settlement in the same tick, settle continuation ordered
  first: the waiter now un-counts itself SYNCHRONOUSLY inside onAbort
  (the finally decrement alone lands a microtask too late), so the
  settle path sees no live waiter and the notice fires;
- abort landing after settlement but before the wait's resolve
  microtask: the wait now resolves and DELIVERS the terminal snapshot
  it owes instead of rejecting (settlement suppressed the notice on
  this waiter's behalf).

Both windows pinned by deterministic tests that fail on the previous
implementation; wait()'s abort contract updated in JSDoc/README/RFC.
2026-07-09 23:08:26 +08:00
Tianyi Cui
c39fe5cd30 Merge remote-tracking branch 'origin/master' into agent-request-messages
# Conflicts:
#	docs/cordis-catalog/services.md
2026-07-09 22:43:26 +08:00
Yichen Jiang
032510a85a Merge remote-tracking branch 'origin/master' into codex/rfc-subagent-background-tasks
Adopts #220 (tool-cordis + the gen-cordis-api gate: the runtime API
catalog regenerated with ctx.tasks/onCleanup on it) and #225 (shared
transcript renderer). Tool-catalog expectations take the union
(cordis_* + task_*); packages/README budget adopts master's 660
ceiling, which absorbs both new group rows.
2026-07-09 22:42:19 +08:00
Tianyi Cui
d976767fa6 Merge branch 'master' into recall-renderer-shared 2026-07-09 22:34:56 +08:00
imccyu
fe4da9244f fix(tool-cordis): validate a dynamic tool's execute return shape after the realm round-trip
The sandbox execute wrapper JSON round-tripped the return and blindly cast it
to ToolExecuteReturn. A JSON-valid but wrong-shape return — a bare string,
{ content: 'ok' }, blocks without a type tag — sailed through: the registry
spreads result.content, so { content: 'ok' } became ['o','k'], passed the
session log's isJsonValue gate, and the DeepSeek serializer then flattened it
to '(no output)' — silent corruption of the next model request and every
replay, instead of a contained tool error.

The round-tripped value is now shape-checked against the two ToolExecuteReturn
forms (array of content blocks, or { content: blocks, meta? }); block checks
are structural only (plain object + string type tag) because the ContentBlock
union is merge-extensible. A wrong shape — and the formerly cryptic
forgot-return/bare-string cases — fails that one call with a teaching error
echoing a truncated preview of what was returned and the two valid forms.
New specs pin the object-form pass-through (meta included), six rejection
shapes, and the preview truncation; per-file 100% coverage holds.
2026-07-09 22:12:30 +08:00
Yichen Jiang
6771a73309 docs(tasks): fix review finding — TaskId JSDoc links TaskService.start, not the removed register() 2026-07-09 22:08:38 +08:00
imccyu
fda5784e16 Merge remote-tracking branch 'origin/master' into tool-cordis
# Conflicts:
#	docs/capability-seams.md
#	packages/core/tools/tests/gen-tool-catalog.spec.ts
#	scripts/gen-doc-graphs.ts
2026-07-09 22:02:06 +08:00
Yichen Jiang
bd59fddacd refactor(tasks): declare-then-execute — ctx.tasks.start() replaces register()
start({ kind, label, owner, run }) preflights everything that can fail
(the attachSurface fence, validation, the owner-cleanup attach) BEFORE
invoking the producer's run() starter, then commits atomically —
'work started but never got a collectable id' is now structurally
impossible instead of a producer try/catch rollback obligation (the
P1 review fix, rebuilt on #185's declare/execute split). Producers
lose their catch-wraps; the leak tests now pin the stronger property
that a failed preflight never spawns anything. TaskRegistration splits
into TaskStart (identity + run) and TaskHooks (cancel/done/readOutput);
docs, type-equiv manifest, catalogs, and both RFCs move with it.
2026-07-09 21:55:07 +08:00
Yichen Jiang
f858c647a6 refactor(tasks): drive wait() timing through dsh-timeout
ctx.tasks.wait arms a deadline() fusing the caller's abort with the
wait timeout and classifies the outcome with timeoutOf scoped to the
new TASK_WAIT_TIMEOUT code: a wait timeout resolves to the live
snapshot (the task keeps running), a caller abort rejects the wait —
same contract, no hand-rolled timer/listener plumbing, and a nested
foreign deadline can no longer misread as a wait timeout. task_output
deliberately declares NO ToolDefinition.timeoutMs: timeout-policy
turns a timed-out call into a structured TOOL_TIMEOUT failure, but a
timed-out wait is a SUCCESS that must still report [status: running]
(decision recorded in the runtime RFC alternatives).
2026-07-09 21:36:56 +08:00
Yichen Jiang
beaace5eb3 Merge remote-tracking branch 'origin/master' into codex/rfc-subagent-background-tasks
Adopts #185 (dsh-timeout: clampTimeout/deadline/timeoutOf drive bash
run() timeout classification; runBash loses its own timer) and #108
(ask_user_question) across the task-runtime rework: bash-local keeps
the BashProcess handle shape with master's deadline mechanics, tool
catalogs/expectations carry both the task_* and ask-user tools, and
generated docs are regenerated on the union.
2026-07-09 21:32:07 +08:00
Yichen Jiang
184e164091 feat(tasks): background task runtime, generic task_* control tools, bash/subagent producers
One shared ctx.tasks registry (branded <kind>-N ids, owner-fenced
read/kill/wait/list, attachSurface misconfiguration fence, reported-flag
notice dedup, atomic register) + dsh-tool-tasks (task_output/task_list/
task_kill, completion-notice injection, background prompt habit).
Producers opt in via their own enableRunInBackground config: bash
(stream kind; seam slimmed to resolve/run/start returning a BashProcess
handle, bash_output/bash_kill deleted) and subagent (final-output kind;
done settles after run.dispose()). Owner disposal drains tasks through
the new awaited ctx.agents.onCleanup seam in the loop's disposal chain.
Both RFCs moved to implemented/; docs, catalogs, snapshots re-pinned.
2026-07-09 21:22:54 +08:00
Yichen Jiang
0df497133c Merge branch 'master' into agent-request-messages 2026-07-09 21:12:31 +08:00
imccyu
f1e54d737b fix(tool-cordis): pass primitive provided service values through the façade
cordis provide() accepts any value and cross-mount composition advertises
ctx.provide('name', value), but the façade's readService unconditionally
proxied every declared service — new Proxy('42') throws "Cannot create proxy
with a non-object as target or handler", so a consumer of a primitive-valued
service crashed on first read with an error naming neither the service nor
the fix. A primitive or null value now passes through unwrapped (after the
denyContext check); only object- and function-valued services are proxied —
a primitive has no method that could hand back a Context, so nothing is lost.
New cross-mount spec pins both read paths (ctx.<name> and ctx.get) for a
number and a null provided value.
2026-07-09 19:30:49 +08:00
imccyu
db857f62c0 docs(tool-cordis): state the sandbox stance as steering, not containment
The sandbox docs overclaimed a containment contract the design never makes:
"capability access is routed through cordis services, never Node built-ins,
so everything a mounted plugin does stays inspectable and disposable". The
host-realm helpers on the sandbox global (harness, console, btoa) are
reachable functions, so mount code that goes looking can reach the host realm
through one of them — accepted under the trust stance, because the ctx a
mount ultimately receives is fully privileged anyway. Reword the sandbox
module doc, the README trust stance, and the RFC sandbox-semantics section to
say exactly that: the traps and small global surface STEER honest code onto
the cordis services; they are not a security boundary.
2026-07-09 18:51:44 +08:00
Yichen Jiang
e41dbe7303 docs: list the user-interaction packages in the ui/ group summaries 2026-07-09 18:42:51 +08:00
Yichen Jiang
1e06fdbb86 refactor: colocate user interaction with ui packages 2026-07-09 17:59:02 +08:00
imccyu
3e9527278a fix(tool-cordis): gate façade services on inject, and make tools.get read-only
Two review findings (#220) on the sandbox context façade:

- Undeclared services were reachable: the façade resolved any live global via
  ctx.get(name), so ctx.bash worked without inject: ['bash']. A cross-mount
  consumer could then depend on a provider cordis never saw — unmounting the
  provider would neither park the consumer nor unwind its registered tools,
  leaving a model-visible tool that fails only at execution. The façade now
  reads ctx.fiber.inject and refuses any service the mount did not declare
  (with a teaching error naming the inject fix), so the dependency is always
  visible to cordis and its activation/unload semantics bind.

- ctx.tools.get returned the live ToolDefinition, including execute — mount
  code could call another tool directly and bypass ToolRegistry.execute and
  its pre/post-execute hooks and accounting. get now returns the same
  read-only name/description/parameters view as schemas(), never an invocable.

Adds inject-gate and schema-view regression cases to sandbox-context.spec.ts
(undeclared property/get denied, declared allowed, the cross-mount zombie-tool
scenario refused at call time, get exposes no execute). Package stays at
per-file 100% coverage. RFC, mount description, and tool-catalog updated.
2026-07-09 13:57:03 +08:00
imccyu
1b1ba96d4f fix(tool-cordis): replace the pass-through ctx proxy with a whitelist façade
Review finding (#220): the guarded proxy only special-cased ctx.tools, so
mount code could reach an UNGUARDED context through ctx.root, ctx.extend(), or
a service instance's .ctx, then ctx.root.tools.register({…}) to bypass the
marker check and host-realm normalization — a raw vm-realm result would later
error a real agent turn at the session-log plainness check.

The sandbox ctx is now a whitelist façade, not a pass-through proxy: it exposes
only what a mount needs — tools.register (marker-guarded), on/once, provide, the
timer helpers, and injected services resolved through a guarded get — and denies
every framework-plumbing member (root, parent, fiber, reflect, registry, extend,
isolate, intercept, plugin, set, mixin, …) with a teaching error. Injected
services are wrapped so a method returning a Context is rejected on the way back
(the .ctx escape), closing the one indirect leak. There is no context-valued
member left to reach; cross-mount provide/inject is untouched (the plugin's own
inject and the fiber's pending/active gating are unchanged). ctx.plugin (child
plugins) and ctx.set are denied by design; ctx.effect is deferred (FIXME).

Adds tests/sandbox-context.spec.ts covering the escape class (root/extend/fiber/
plugin/set/… denied, the classic root.tools.register bypass, the .ctx escape,
read-only writes) plus the async-service and symbol/in-operator paths for 100%
coverage. RFC/README/tool-catalog/config-catalog updated; api-catalog.ts
regenerated (also picks up the codeRuntime service that entered on the master
merge and was left stale).
2026-07-09 13:57:03 +08:00
imccyu
ea66641b84 feat(tool-cordis): flatten the inspect plugins section to a capability list
The tree SHAPE was the wrong surface for the model: what it needs from
cordis_inspect is what services, plugins, and capabilities are loaded, not the
fiber hierarchy. The plugins section is now a flat name + lifecycle-state list
from ctx.registry (deterministically sorted, one line per instance); the ASCII
tree renderer, the parent→child rebuild, and the dyn-id tree annotation are
deleted — dynamic mounts keep their own richer dynamic section (id, state,
provides, waits). Net -49 lines; RFC and READMEs state the flat-list contract.
2026-07-09 13:57:03 +08:00
imccyu
a500c791f7 fix(tool-cordis): normalize the JSON-Schema dialect at the defineTool boundary
Field sessions showed models writing tool schemas in the JSON-Schema dialect
by strong prior — type: 'integer', required: false, then the full
{ type:'object', properties, required: [...] } wrapper — and the rejection
text itself pushed a nearly-correct DSL attempt BACK to raw JSON Schema: one
stats tool cost three consecutive schema errors before mounting. The boundary
now normalizes wherever the input has exactly one meaning (wrapper unwrapped
with the required array becoming per-property flags at any nesting level,
integer → number, required: false → optional, all rebuilt as fresh host-realm
objects) and rejects only genuinely meaningless input, enumerating the valid
vocabulary in the error. Re-running the failing session mounts first-try.
The mount description documents both accepted forms.
2026-07-09 13:57:03 +08:00
imccyu
db45769513 feat(tool-cordis): Node-API traps + fs/web capability routing
The sandbox deliberately provides no Node API, and now says so instead of
letting a bare ReferenceError teach nothing: require, the timers, and fetch
are callable traps whose error redirects to the cordis alternative (inject:
['fs'] + ctx.fs, ['web'] + ctx.web, ['bash'] + ctx.bash, ['timer'] +
ctx.setTimeout — a fiber effect, unwound on unmount). Only function-shaped
globals are trapped; process/Buffer stay undefined so typeof feature probes
stay inert. The mount description and the demo persona state the routing rule,
and the demo mounts ctx.fs (local provider) and ctx.web (seam + keyless local
fetch provider) so agent-built plugins have real capabilities to build on.
Live-validated: a model that reached for Node setTimeout self-corrected to
inject: ['timer'] in one step and built a working ctx.web fetch tool.
2026-07-09 13:57:03 +08:00
imccyu
e51e58e993 chore: register the cordis group across repo gates and docs
Everything outside the package and example that a new top-level group and a
new demo touch: GROUP_ORDER in gen-module-graph and gen-doc-graphs (plus the
tools-service consumers list, the APP_EXAMPLES entry, and the graph-atlas
label/mode rows), the knip e2e entries, the packages/README group row, the
AGENTS.md layout and demo lines, and the regenerated module-graph /
config-catalog / graph-atlas / capability-seams / composition artifacts.
AGENTS.md and examples/AGENTS.md word-budget ceilings rise to current+5%
(1802 / 653): the new group and demo rows are genuine additions to both docs,
not condensable restatements.
2026-07-09 13:57:03 +08:00
imccyu
ee1da1ce5b feat(cordis): @deepseek-ai/dsh-tool-cordis — inspect/mount/unmount over the live runtime
New top-level packages/cordis/ group with the self-referential toolset:
cordis_inspect (services / plugin tree / tools / dynamic mounts / api / events,
the api section intersecting the generated catalog with the live service store),
cordis_mount (model-written code evaluated in a node:vm sandbox, mounted under
one cordis-dynamic group fiber as dyn-<n>), cordis_unmount (awaited disposal to
quiescence). Boundary mechanisms: dual-realm instanceof, JSON realm
normalization of dynamic tool results, marker-guarded registration, SchemaSpec
teaching errors, parse failures surfaced with the offending line + caret and a
line-scoped TypeScript hint, and the unmount-first recipe on tool-name
collisions. Config: vmTimeoutMs (schemastery, default 5000). Design record:
docs/rfc/implemented/feature/2026-07-08-self-referential-cordis-toolset.md.

The tool-catalog boot manifest, its regenerated output, and the pinned
tool-name list land here rather than with the other repo registration: the
completeness guard globs packages/*/tool-* and fails the generator (and the
core/tools spec) the moment the package directory exists.
2026-07-09 13:57:03 +08:00
Ziya
fc6db791f1 refactor(compact): extract the shared transcript renderer into dsh-compact
Move compact-basic's private _extractText/_blocksToText into the interface
package as renderTranscript/renderContentBlocks, so the summarize path and a
future recall read path render one span identically. Byte-identical output
vs the private helpers it replaces; compact-basic delegates.
2026-07-08 22:39:55 -07:00
Yichen Jiang
bdab8c6a61 Merge remote-tracking branch 'origin/master' into codex/ask-user-question
# Conflicts:
#	docs/config-catalog.md
2026-07-09 13:34:13 +08:00
Yichen Jiang
e4331ca5e7 Merge branch 'master' into agent-request-messages 2026-07-09 13:30:11 +08:00
Dudu-0223
0a3ffe1b04 Merge remote-tracking branch 'origin/master' into timeout-design
# Conflicts:
#	docs/event-producer-consumer.md
#	packages/README.md
2026-07-09 11:52:18 +08:00
Yichen Jiang
c4aff62ef7 Merge remote-tracking branch 'origin/codex/ask-user-question' into codex/ask-user-question 2026-07-09 11:22:12 +08:00
Yichen Jiang
7db9a6c780 fix review findings: document ask-user recommendation convention 2026-07-09 11:20:17 +08:00
Yichen Jiang
a5ba05121a Merge branch 'master' into codex/ask-user-question 2026-07-09 11:11:07 +08:00
Yichen Jiang
1f6b67269e Merge branch 'master' into agent-request-messages 2026-07-09 09:52:33 +08:00
Tianyi Cui
43de115173 Merge branch 'master' into code-runtime-worker 2026-07-09 01:04:01 +08:00
Yichen Jiang
db9437a50e Merge remote-tracking branch 'origin/master' into codex/ask-user-question
# Conflicts:
#	packages/bash/bash-local/tests/executor.spec.ts
#	packages/bash/bash-local/tests/run.spec.ts
2026-07-09 00:48:45 +08:00
Yichen Jiang
56091d5b5d fix review findings: keep ask-user opt-in for acp app 2026-07-09 00:42:36 +08:00
Tianyi Cui
064d1c4ce1 docs: state advisory typing in the worker row (review)
A reviewer read "TypeScript via host-side type-strip" and reasonably
asked what typing buys if nothing checks it — the group README never
said the annotations are advisory by design. The row now states it; the
rationale stays in the RFC and the enforcement story (per-dispatch
validateArgs) in the dsh-tools README.
2026-07-09 00:38:21 +08:00
Tianyi Cui
46187ec3b6 Merge branch 'master' into agent-request-messages 2026-07-09 00:21:05 +08:00
Tianyi Cui
75ae8e38af Merge branch 'master' into codex/fix-bash-local-process-tests 2026-07-09 00:01:28 +08:00
Yichen Jiang
fd1071392c docs(compact): sync the compactIfNeeded prose signature with the sessionPrefix parameter
The seam gained sessionPrefix between fullSystemPrompt and signal in
18d478bc, but the compact README member table and the compaction
core-data-structures page still showed the 3-arg form and listed only
agent/system/signal as what pre-step supplies. The generated service
catalog was already correct; only these two prose homes drifted.
2026-07-08 22:40:38 +08:00
Yichen Jiang
0d023b9aaa Merge remote-tracking branch 'origin/master' into agent-request-messages 2026-07-08 22:26:25 +08:00
Yichen Jiang
b31399bdc4 Merge remote-tracking branch 'origin/master' into codex/ask-user-question
# Conflicts:
#	docs/module-graph.md
2026-07-08 22:22:25 +08:00
Tianyi Cui
90547f283b fix: byte-exact value/error caps + write-callback contract (agent review)
Two [P1] review findings on the worker runtime:

- maxValueBytes gated and sliced the rendered fallback by UTF-16 code
  units, so a multibyte string ("€€€€" under a 4-byte cap) crossed whole
  and a truncated multibyte rendering could still run ~3x over budget.
  New truncateUtf8Bytes cuts at code-point boundaries under a real byte
  budget; prepareValue's fallback and the host's forged-error-text bound
  both use it, and the VALUE_RENDER_SLACK comment drops its now-obsolete
  "sliced by characters" wrinkle.

- The patched stream write dropped Node's optional encoding/callback
  arguments, so a program awaiting flush completion
  (write(chunk, resolve)) hung to the wall ceiling and misreported as a
  timeout. The shim now fires the callback asynchronously once the chunk
  is admitted — including for writes the exhausted budget drops.
2026-07-08 21:56:28 +08:00
Yichen Jiang
959a3c3a8b fix(agent-loop): discard an interrupted prefix composition instead of caching it
A cancel/dispose landing inside the first agent/session-prefix waterfall
used to commit the listener chain's return value to the instance cache
before the interruption check dropped the turn; an abort-aware listener's
degraded fallback would then ship on every later request of the instance.
The commit now happens only after the composition survives the
interruption check — the cache only ever holds a fully composed prefix,
and the next turn recomposes under a live signal.
2026-07-08 21:46:03 +08:00
Yichen Jiang
9c133c644d test(bash-local): wait for process readiness 2026-07-08 21:45:10 +08:00
Yichen Jiang
765052a7d1 fix(agent-loop): compose the session prefix before pre-step; hand it to the pressure gate
ds-review-bot critical (follow-up): on the first step of a resumed or
seeded/forked instance, auto-compaction ran before runStep composed
this instance's prefix, so the gate read the PREVIOUS instance's logged
prefix from the header fold — a contributor that grew across
resume/fork (skills added, AGENTS.md grown: exactly the
environment-dependent case) could under-gate and ship an over-window
first request.

The loop now composes agent/session-prefix before the instance's first
agent/pre-step (still once per instance; runStep just reads the cache),
and agent/pre-step carries the composed prefix to its listeners.
CompactService.compactIfNeeded gains the sessionPrefix parameter;
BasicCompactService.estimatePressure gates on the handed value — the
header-fold read is gone, so the estimate is exact at every step
including a resumed/forked instance's first. Composition moving before
the boundary snapshot also means a composing listener's session append
now joins the CURRENT request (documented on the seam).

New coverage: composition precedes pre-step and the seam receives the
composed prefix; cancel and disposal landing inside the composition
window drop the step cleanly; the compact gate test hands the prefix
directly.
2026-07-08 20:30:10 +08:00
Yichen Jiang
e6bd6cc9be fix(compact): count the logged session prefix toward token pressure
ds-review-bot critical: compactIfNeeded estimated pressure from the
derived history + system prompt only, but every loop-built request also
carries EpochHeader.messagePrefix in front of the history — a
deployment at the window edge would under-estimate by exactly the
prefix, skip compaction, and ship an over-window request.

BasicCompactService now gates on estimatePressure(): the session prefix
read from the log's folded header + the derived history + the system
prompt. The fold is exact from the instance's second request on (and
from a resumed instance's first — the previous instance logged its
prefix); it is absent only before a fresh session's first request,
where the history is a single prompt and compaction is moot. Compaction
itself still shrinks history only — a prefix that alone approaches the
window is a configuration error no compactor fixes, same as the
documented single-unit-overflow stance.
2026-07-08 19:32:42 +08:00
Yichen Jiang
77333c0a19 fix(agent): correct session-prefix composition-order docs; scrub messagePrefix in snapshots
Two ds-review-bot findings:

The seam JSDoc claimed extending 'await next()' composes in
registration order — false for the append form: the waterfall unwinds
innermost-first, so appending places later-registered contributions
first. The canonical contribution is now documented as the PREPEND
'[mine, ...await next()]' (registration order on the wire), with the
append form's reverse-order behavior stated explicitly; the ordering
test now uses the canonical pattern in both listeners.

scrubRequestHeaders tokenized only system/tools, so a fixture recording
a composed session prefix would carry its raw text (workspace-specific
churn/leak). The scrubber now maps each header/delta messagePrefix
entry to a {{messagePrefix}} token — count stays a structural fact,
absence stays absent, the empty-array transition stays visible — with
normalize.spec coverage for the header, delta, absence, and odd-shape
paths.
2026-07-08 16:16:08 +08:00
Yichen Jiang
b04cfe3a41 Merge remote-tracking branch 'origin/master' into agent-request-messages 2026-07-08 15:54:36 +08:00
Dudu-0223
a83eb5d5c2 docs: fit packages/README budget after merging code-runtime + timeout rows
The master merge added a code-runtime/ package row while this branch adds
the timeout/ row; together they push packages/README.md over its 605-word
ceiling. Condense the timeout/ row to the terse sibling style and raise the
ceiling 605->610 for the genuinely-new package group, mirroring how the
code-runtime work raised architecture.md's ceiling in the same spirit.
2026-07-08 15:46:46 +08:00