review(tool-goal,llm-replay): address ds-review-bot round one

- fromRequest placeholder: the last two braces of a consecutive } run now
  terminate the placeholder, so patterns may end with a brace quantifier
  (bot warning; the truncated pattern could even silently mis-match since
  an unclosed { is literal in JS regexes)
- document that derived JSONL entries pass through the same resolution
- widen ToolRunContext/deferContext seam docs beyond composite-only usage
  (source JSDoc, README pair, core-data-structures type-equiv blocks)
- pin direct-human blocked as uninstructed, completing the
  goal-round/direct-human x complete/blocked test quadrant
This commit is contained in:
ZiyaZhang
2026-08-02 07:19:26 -07:00
parent 666ef95f81
commit 820a5a97f1
15 changed files with 58 additions and 35 deletions

View File

@@ -765,7 +765,7 @@ export interface ReplayModelConfig {
Depends on: [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts)
Source: [`packages/support/llm-replay/src/index.ts:701`](../packages/support/llm-replay/src/index.ts)
Source: [`packages/support/llm-replay/src/index.ts:707`](../packages/support/llm-replay/src/index.ts)
## `@deepseek-ai/dsh-llm-retry`
@@ -2039,7 +2039,7 @@ export interface Config {
export type ToolPresentationMode = 'native' | 'code' | 'both'
```
Source: [`packages/core/tools/src/index.ts:589`](../packages/core/tools/src/index.ts)
Source: [`packages/core/tools/src/index.ts:592`](../packages/core/tools/src/index.ts)
## `@deepseek-ai/dsh-tui`

View File

@@ -2428,7 +2428,7 @@ async execute(exec: ToolExecutionInput): Promise<ToolExecutionResult>
Types: [ScopeKey](../core-data-structures/scope.md) · [ToolDefinition](../core-data-structures/tools.md) · [ToolExecutionInput](../core-data-structures/tools.md) · [ToolExecutionMode](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) · [ToolGuard](../core-data-structures/tools.md) · [ToolRestriction](../core-data-structures/tools.md) · [ToolSchema](../core-data-structures/tools.md)
Source: [`packages/core/tools/src/index.ts:711`](../../packages/core/tools/src/index.ts)
Source: [`packages/core/tools/src/index.ts:714`](../../packages/core/tools/src/index.ts)
## `ctx.tui` — `TuiExtensionService` (abstract seam)

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/core-data-structures/tools.md
tools.md: 98b642b846b23e2b29e6c6d800fe4106235eda85
tools.zh.md: 1ef90c1e76ace7485ed6267de5ee82cbb4de6aa6
tools.md: acaf5d32dd5481aec495ac49f727c9b64f25211e
tools.zh.md: 5c39226fdf5d1406dab7383d40227c26ff1e447c

View File

@@ -200,20 +200,23 @@ interface ToolExecutionInput {
}
```
A tool body receives the runtime extension. `deferContext()` is the composite-tool channel: it records nested-dispatch context without injecting inside the still-open outer call.
A tool body receives the runtime extension. `deferContext()` attaches context to the execution's own result — the composite-tool nested-dispatch channel, also usable by a leaf tool minting a plugin-sourced instruction — without injecting inside the still-open outer call.
```ts type-equiv
/**
* Runtime context handed to a tool implementation after the registry has
* accepted a {@link ToolExecution}. A composite tool uses
* {@link deferContext} to ferry context produced by nested dispatches back to
* the outer result; the loop appends it only after the outer `tool/result`.
* accepted a {@link ToolExecution}. {@link deferContext} attaches context to
* this execution's own result — a composite tool ferries nested-dispatch
* context back to the outer result, and a leaf tool may mint a fresh
* plugin-sourced instruction; the loop appends it only after the
* `tool/result`.
*/
interface ToolRunContext extends ToolExecution {
/**
* Defer one nested-dispatch context until this tool's final result reaches
* the agent loop. Contexts retain their individual source and metadata and
* are emitted in call order.
* Defer one context — typically a nested-dispatch context ferried by a
* composite tool, or a fresh plugin-sourced instruction — until this tool's
* final result reaches the agent loop. Contexts retain their individual
* source and metadata and are emitted in call order.
*/
deferContext(context: UserMessage): void
/**

View File

@@ -200,20 +200,23 @@ interface ToolExecutionInput {
}
```
工具函数体接收运行时扩展。`deferContext()` 是组合工具的通道:它记录嵌套分派产生的上下文,而不会在外层调用尚未结束时注入这些上下文。
工具函数体接收运行时扩展。`deferContext()` 把上下文附着到本次执行自己的结果上——既是组合工具转运嵌套分派上下文的通道,也可供叶子工具铸造插件来源指令——而不会在外层调用尚未结束时注入这些上下文。
```ts type-equiv
/**
* Runtime context handed to a tool implementation after the registry has
* accepted a {@link ToolExecution}. A composite tool uses
* {@link deferContext} to ferry context produced by nested dispatches back to
* the outer result; the loop appends it only after the outer `tool/result`.
* accepted a {@link ToolExecution}. {@link deferContext} attaches context to
* this execution's own result — a composite tool ferries nested-dispatch
* context back to the outer result, and a leaf tool may mint a fresh
* plugin-sourced instruction; the loop appends it only after the
* `tool/result`.
*/
interface ToolRunContext extends ToolExecution {
/**
* Defer one nested-dispatch context until this tool's final result reaches
* the agent loop. Contexts retain their individual source and metadata and
* are emitted in call order.
* Defer one context — typically a nested-dispatch context ferried by a
* composite tool, or a fresh plugin-sourced instruction — until this tool's
* final result reaches the agent loop. Contexts retain their individual
* source and metadata and are emitted in call order.
*/
deferContext(context: UserMessage): void
/**

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/core/tools/README.md
README.md: 15fc5839a3b0e3fa2d20c5a9cc50577e9807ffda
README.zh.md: 8547ee4a796dcd93945dfa40373c14c10d7d0c8a
README.md: 80ea3cc93437d48a7ea0ffba0ff4d2ef2407755f
README.zh.md: 1f0791c5df7afd4a3479afdd827c4fc148cf8883

View File

@@ -43,7 +43,7 @@ The live registry pipeline has three transformable waterfalls, then the definiti
- `ToolExecutionInput` — the caller-supplied call description: `{ callId, name, arguments, signal, agent?, parent? }`; `signal` is required and readonly, callers may pass an enclosing execution's opaque token as `parent`, and callers never choose the new execution's own token.
- `ToolExecutionToken` — a fresh branded `Symbol` assigned by the registry. It supports equality correlation only and never crosses a model, log, or worker boundary.
- `ToolExecution` — the readonly pipeline view: immutable `{ token, callId, name, arguments, signal, agent?, parent? }`; the registry separately retains and re-fuses the original caller signal. `ToolDispatchExecution` is the `tools/execute`-only view whose required signal is mutable, so a wrapper may replace and restore it but cannot delete it. A nested call's `parent` is a `ToolExecutionToken`, not an execution object.
- `ToolRunContext` — the execution passed to a tool body, extending `ToolExecution` with `deferContext(context)`. Composite tools use it to ferry context produced by nested dispatches to the outer result even when the tool later throws or cancellation wins; it never injects immediately.
- `ToolRunContext` — the execution passed to a tool body, extending `ToolExecution` with `deferContext(context)`. It defers one context until the tool's final result reaches the loop — typically a nested-dispatch context ferried by a composite tool, or a fresh plugin-sourced instruction minted by a leaf tool (`tool-goal`'s wrap-up) — even when the tool later throws or cancellation wins; it never injects immediately.
- `ToolExecutionResult` — discriminated execution-local outcome. Success is `{ isError:false, value:JsonValue, content, meta?, additionalContexts? }`; failure is `{ isError:true, error:{ message, info? }, content, meta?, additionalContexts? }` and has no value. Call identity stays on the immutable `ToolExecution`. The registry snapshots, validates, and freezes the canonical value before rendering, then materializes the durable presentation fields before final observation. `ToolFailure.info` carries an internal `{ name, code }` for a `HarnessError`; `additionalContexts` preserves every deferred or post-execute identified `UserMessage` for the loop's post-result FIFO.
- `PreToolDecision``{kind:'allow'}` | `{kind:'deny', reason}` | `{kind:'ask', reason?}`. Input rewrite is deliberately not offered; `ask` is serviced by [`ctx.approval`](../../ui/user-approval/README.md) when mounted and otherwise degrades to deny.
- `PostToolDecision` — accept may replace `content` or `value`, never both, and may attach `additionalContexts`; block turns feedback into a valueless failure. Content replacement preserves the canonical value and metadata. Value replacement is revalidated and rerenders content/metadata. Accept preserves tool-deferred contexts before decision contexts; block discards tool-deferred contexts and exposes only contexts explicitly supplied by the blocking decision.

View File

@@ -43,7 +43,7 @@ tools:
- `ToolExecutionInput`:调用方提供的调用描述:`{ callId, name, arguments, signal, agent?, parent? }``signal` 必填且只读,调用方可以将外层执行的不透明 token 作为 `parent` 传入,但绝不能选择新执行自身的 token。
- `ToolExecutionToken`:注册表分配的全新带品牌 `Symbol`。它只支持通过相等性进行关联,绝不会跨越模型、日志或 worker 边界。
- `ToolExecution`:只读流水线视图:不可变的 `{ token, callId, name, arguments, signal, agent?, parent? }`;注册表会另行保留并重新融合调用方的原始信号。`ToolDispatchExecution` 是仅供 `tools/execute` 使用的视图,其必填信号可变,因此包装层可以替换并还原它,但不能删除它。嵌套调用的 `parent``ToolExecutionToken`,而不是执行对象。
- `ToolRunContext`:传给工具主体的执行上下文,在 `ToolExecution` 基础上增加 `deferContext(context)`。组合工具借此把嵌套分发产生的上下文传递到外层结果,即使工具后来抛出或取消胜出也不例外;该方法绝不会立即注入上下文。
- `ToolRunContext`:传给工具主体的执行上下文,在 `ToolExecution` 基础上增加 `deferContext(context)`它把一条上下文推迟到该工具的最终结果抵达循环时——通常是组合工具转运的嵌套分发上下文,也可以是叶子工具铸造的全新插件来源指令(如 `tool-goal` 的收尾注入)——即使工具后来抛出或取消胜出也不例外;该方法绝不会立即注入上下文。
- `ToolExecutionResult`:可辨识的执行局部结果。成功形态为 `{ isError:false, value:JsonValue, content, meta?, additionalContexts? }`;失败形态为 `{ isError:true, error:{ message, info? }, content, meta?, additionalContexts? }`,且不含值。调用身份保留在不可变的 `ToolExecution` 上。注册表会在呈现前快照、验证并冻结规范值,随后在最终观测前实体化持久呈现字段。`ToolFailure.info` 携带内部的 `{ name, code }`,用于表示 `HarnessError``additionalContexts` 会保留每个通过延迟或 post-execute 加入且带标识的 `UserMessage`,供循环在结果后按 FIFO 顺序处理。
- `PreToolDecision``{kind:'allow'}` | `{kind:'deny', reason}` | `{kind:'ask', reason?}`。该类型有意不提供输入改写;`ask` 在挂载 [`ctx.approval`](../../ui/user-approval/README.md) 时由它处理,否则退化为拒绝。
- `PostToolDecision`:接受决定可以替换 `content``value`(不能同时替换),并可附加 `additionalContexts`;阻止决定会把反馈变成无值失败。替换内容会保留规范值和元数据。替换值会重新验证,并重新呈现内容/元数据。接受决定会先保留工具延迟的上下文,再附加决定上下文;阻止决定会丢弃工具延迟的上下文,只公开阻止决定显式提供的上下文。

View File

@@ -344,15 +344,18 @@ export interface ToolDispatchExecution extends Omit<ToolExecution, 'signal'> {
/**
* Runtime context handed to a tool implementation after the registry has
* accepted a {@link ToolExecution}. A composite tool uses
* {@link deferContext} to ferry context produced by nested dispatches back to
* the outer result; the loop appends it only after the outer `tool/result`.
* accepted a {@link ToolExecution}. {@link deferContext} attaches context to
* this execution's own result — a composite tool ferries nested-dispatch
* context back to the outer result, and a leaf tool may mint a fresh
* plugin-sourced instruction; the loop appends it only after the
* `tool/result`.
*/
export interface ToolRunContext extends ToolExecution {
/**
* Defer one nested-dispatch context until this tool's final result reaches
* the agent loop. Contexts retain their individual source and metadata and
* are emitted in call order.
* Defer one context — typically a nested-dispatch context ferried by a
* composite tool, or a fresh plugin-sourced instruction — until this tool's
* final result reaches the agent loop. Contexts retain their individual
* source and metadata and are emitted in call order.
*/
deferContext(context: UserMessage): void
/**

View File

@@ -599,5 +599,7 @@ describe('goal tool state transitions', () => {
},
roundsStarted: 0,
})
expect(blocked.concludesTurn).toBeUndefined()
expect(blocked.additionalContexts).toBeUndefined()
})
})

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/support/llm-replay/README.md
README.md: ea52525ee85aae58006c852afe93291ea70807d5
README.zh.md: e8c0ec225df29fc6f5493776d75bc7f9e3e078de
README.md: 85aa56705929e7630e4cfb6c2a3c9cbbd0d843a6
README.zh.md: 751f75dea197ffb112cfa703e3a5dbfaffb8c0b2

View File

@@ -12,7 +12,7 @@ The fixture IS the persisted session log (`<scenario>/session.jsonl`). Its `assi
Two failure modes are not reconstructable from `assistant/chunk` alone — a pure throw before any chunk (e.g. an HTTP 401, where the log holds only a `turn/end {error}` and no chunks) and a cancel/hang (timing, not chunk content). A scenario that needs those supplies an optional sidecar (`<scenario>/replay.override.json`) that either replaces the derived script (a bare `ReplayEntry[]`) or augments it (`{ patches: [{ at, entry }] }`: keep every JSONL-derived call and swap the named 0-based call indexes; `at` equal to the derived length appends the retry attempt after an injected transient throw). Patch indexes must be unique. The override document, each patch and entry, and every chunk discriminant are validated when the file loads. A `hang` entry may name `readyFile`; replay writes that empty marker after its prefix chunks reach the loop and before it waits for cancellation, so an external driver can cancel deterministically without observing a presentation update.
A scripted string may embed `{{fromRequest:<regex>}}` to fill a value no static sidecar can know — for example a randomly minted goal id the model must echo back into `update_goal`. At stream time every placeholder resolves against the live request: the corpus is every string leaf of the request messages joined by newlines, the pattern's LAST corpus match wins, and its first capture group (or the whole match without one) substitutes in place. A pattern that matches nothing, an invalid pattern, and an unterminated placeholder each fail loud; the first `}}` ends the placeholder, so patterns cannot contain `}}`.
A scripted string may embed `{{fromRequest:<regex>}}` to fill a value no static sidecar can know — for example a randomly minted goal id the model must echo back into `update_goal`. At stream time every placeholder resolves against the live request: the corpus is every string leaf of the request messages joined by newlines, the pattern's LAST corpus match wins, and its first capture group (or the whole match without one) substitutes in place. A pattern that matches nothing, an invalid pattern, and an unterminated placeholder each fail loud. The last two braces of a consecutive `}` run terminate the placeholder, so a pattern may end with a brace quantifier (`[0-9a-f]{4}`) but cannot contain `}}` followed by further pattern content. Resolution applies to every scripted entry, including ones derived from the recorded JSONL — a recorded fixture whose text legitimately contains the literal marker must be expressed through a sidecar without it.
## Nested agents: per-session keying

View File

@@ -12,7 +12,7 @@ fixture 就是持久化的会话日志(`<scenario>/session.jsonl`)。其 `as
有两种失败模式无法仅根据 `assistant/chunk` 重建:在产生任何分片前直接抛出异常(例如 HTTP 401此时日志只有 `turn/end {error}` 而没有分片),以及取消或挂起(差异在时序,而非分片内容)。需要这些行为的场景可提供伴随文件(`<scenario>/replay.override.json`):它可以替换派生脚本(裸 `ReplayEntry[]`),也可以增补派生脚本(`{ patches: [{ at, entry }] }`:保留所有从 JSONL 派生的调用,只替换指定的从 0 开始计数的调用索引;当 `at` 等于派生长度时,则在注入瞬态异常后的重试位置追加一次调用)。补丁索引不得重复。文件加载时会校验覆写文档、每个补丁和条目,以及每个分片的判别标签。`hang` 条目可以指定 `readyFile`;当前缀分片到达循环后、开始等待取消前,回放会写入这个空标记,使外部驱动程序无需观察展示层更新即可确定性地取消。
脚本字符串可以内嵌 `{{fromRequest:<regex>}}`,用来填入静态伴随文件不可能预知的值——例如模型必须原样回填到 `update_goal` 的随机生成 goal id。回放时每个占位符针对实时请求解析语料是请求消息的所有字符串叶子按换行拼接的结果取该模式在语料中的最后一次匹配用其第一个捕获组无捕获组时用整个匹配原位替换。模式匹配不到内容、模式非法、占位符未闭合都会明确报错;第一个 `}}` 即结束占位符,因此模式本身不能包含 `}}`
脚本字符串可以内嵌 `{{fromRequest:<regex>}}`,用来填入静态伴随文件不可能预知的值——例如模型必须原样回填到 `update_goal` 的随机生成 goal id。回放时每个占位符针对实时请求解析语料是请求消息的所有字符串叶子按换行拼接的结果取该模式在语料中的最后一次匹配用其第一个捕获组无捕获组时用整个匹配原位替换。模式匹配不到内容、模式非法、占位符未闭合都会明确报错。连续右花括号串的最后两个花括号才是占位符结束符,因此模式可以以花括号量词收尾(如 `[0-9a-f]{4}`),但不能在 `}}` 之后还有后续模式内容。解析作用于所有脚本条目,包括从已记录 JSONL 派生的条目——若录制文本本身合法地含有该字面量标记,需改用不含标记的伴随文件表达
## 嵌套 agent每会话键控

View File

@@ -283,10 +283,13 @@ function substituteString(text: string, corpus: string): string {
while (true) {
const open = text.indexOf(FROM_REQUEST_OPEN, cursor)
if (open === -1) return result + text.slice(cursor)
const close = text.indexOf(FROM_REQUEST_CLOSE, open + FROM_REQUEST_OPEN.length)
let close = text.indexOf(FROM_REQUEST_CLOSE, open + FROM_REQUEST_OPEN.length)
if (close === -1) {
throw new Error(`llm-replay: fromRequest placeholder is unterminated in ${JSON.stringify(text)}`)
}
// The last two braces of a consecutive `}` run terminate the placeholder,
// so a pattern may end with a brace quantifier like `[0-9a-f]{4}`.
while (text[close + FROM_REQUEST_CLOSE.length] === '}') close += 1
const pattern = text.slice(open + FROM_REQUEST_OPEN.length, close)
result += text.slice(cursor, open) + resolveFromRequest(pattern, corpus)
cursor = close + FROM_REQUEST_CLOSE.length
@@ -313,7 +316,10 @@ function substituteValue(value: unknown, corpus: string): unknown {
* Scenario sidecars use this to script arguments no static file can know,
* such as a randomly minted goal id the model must echo back. A pattern that
* matches nothing, an invalid pattern, and an unterminated placeholder each
* fail loud. Patterns cannot contain `}}` — the first `}}` ends the placeholder.
* fail loud. The last two braces of a consecutive `}` run terminate the
* placeholder, so a pattern may end with a brace quantifier but cannot
* contain `}}` followed by further pattern content. Derived entries pass
* through the same resolution as sidecar entries.
* @param entry - the scripted entry about to replay.
* @param messages - the live request messages searched by the placeholders.
* @returns the entry itself when no placeholder appears, else a resolved deep copy.

View File

@@ -350,6 +350,12 @@ describe('installLlmReplay (through the real LlmService)', () => {
expect(delta).toMatchObject({ argumentsDelta: '{"goal_id":"goal-42ab"}' })
})
it('keeps a trailing brace quantifier inside the pattern (terminator is the run tail)', async () => {
const streamed = await streamScripted('{"goal_id":"{{fromRequest:goal-[0-9a-z]{4}}}"}')
const delta = streamed.find(chunk => chunk.type === 'tool-call-delta')
expect(delta).toMatchObject({ argumentsDelta: '{"goal_id":"goal-42ab"}' })
})
it('fails loud when a placeholder matches nothing in the request', async () => {
await expect(streamScripted('{"goal_id":"{{fromRequest:task-[0-9]+}}"}'))
.rejects.toThrow(/fromRequest.*matched nothing/)