mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
fix(goal): harden driver cancellation settlement
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-19-same-session-goal-round-driver.md: 2fc27ab28bff5e192e43d30afc6cd796fd3394e1
|
||||
2026-07-19-same-session-goal-round-driver.zh.md: e9610c94eca48fb4d06be505a8cdfffb6c272012
|
||||
2026-07-19-same-session-goal-round-driver.md: 94c0a870c46d6b7f618faf3f0a4b01c141bec358
|
||||
2026-07-19-same-session-goal-round-driver.zh.md: 9fdff05ffb722ef683bc06acee606f7897e5afc6
|
||||
|
||||
@@ -39,7 +39,7 @@ The driver classifies one closed goal-owned turn as follows:
|
||||
| Turn result | Action |
|
||||
|---|---|
|
||||
| durable `completed` | continue while active/armed and under cap |
|
||||
| broad cancellation / `aborted` | pause and disarm |
|
||||
| cancellation of a reserved/admitted goal round, or its `aborted` result | pause and disarm |
|
||||
| `error` with code `RATE_LIMIT` | mark `usage-limited` |
|
||||
| other `error` | block |
|
||||
| `max-tokens` | block |
|
||||
@@ -52,9 +52,9 @@ No abnormal outcome requests an automatic retry. A later human prompt can ask to
|
||||
|
||||
### Durability and cancellation seam
|
||||
|
||||
Every `goal/changed` notification creates a checkpoint obligation. The driver awaits `ctx.sessions.flush(session)` before reserving work, then checks for a newer mutation, agent lifecycle change, or competing prompt. Turn-end flush failure is reported by the existing `agent/error` notification after `turn/end`; the driver associates it with the exact attempt and disarms before the next idle decision.
|
||||
Every `goal/changed` notification creates a checkpoint obligation. The driver awaits `ctx.sessions.flush(session)` before reserving work, then checks for a newer mutation, agent lifecycle change, or competing prompt. Turn-end flush failure is reported by the existing `agent/error` notification after `turn/end`; the driver finds that exact closed turn even when a concurrent one-shot injection appended a later turn, associates the failure with the exact attempt, and disarms before the next idle decision.
|
||||
|
||||
Broad cancellation previously exposed only its effects after queues were cleared or the request aborted. The public agent vocabulary now includes observe-only `agent/cancel-requested(agent, reason)`. The concrete loop emits it for effective cancellation before either action; fused notification containment means a broken listener cannot veto cancellation. The goal driver uses this edge to clear its reservation and pause an active armed goal before the loop destroys the queued-work evidence.
|
||||
Broad cancellation previously exposed only its effects after queues were cleared or the request aborted. The public agent vocabulary now includes observe-only `agent/cancel-requested(agent, reason)`. The concrete loop emits it for effective cancellation before either action; fused notification containment means a broken listener cannot veto cancellation. The goal driver uses this edge to clear its reservation before the loop destroys the queued-work evidence. When that reservation is a queued or admitted goal attempt, cancellation durably pauses the goal; when cancellation belongs to unrelated human work with no goal attempt, it only removes process-local activation. If the pause mutation throws, the driver falls back to disarming rather than allowing cancelled automatic work to restart.
|
||||
|
||||
This is a coordination notification, not a second stop API. `Agent.cancel()` remains the only public broad cancellation verb, idle calls remain no-ops, and custom `Agent` implementations that claim the interface must honor the event ordering if consumers depend on it.
|
||||
|
||||
@@ -68,7 +68,7 @@ An inbox acceptance can win the microtask race immediately before plugin unload
|
||||
|
||||
## Testing
|
||||
|
||||
The unit suite uses the real agent loop and session service with only the model scripted. It covers exact sequential admission and cap enforcement, load/resume inertness, every outcome classification, rate limiting, request errors, max tokens, downstream prompt veto, pre-admission and in-flight cancellation, human-input ordering, queued and downstream revision races, forged goal attribution, failed mutation and turn checkpoints, scheduler and custom-agent failures, session-start reset, exact lifecycle retirement, and queued/running plugin teardown. The new driver source has per-file 100% statement, branch, function, and line coverage.
|
||||
The unit suite uses the real agent loop and session service with only the model scripted. It covers exact sequential admission and cap enforcement, load/resume inertness, every outcome classification, rate limiting, request errors, max tokens, downstream prompt veto, pre-admission and in-flight cancellation, unrelated-human cancellation, failed-pause fallback, human-input ordering, queued and downstream revision races, forged goal attribution, failed mutation and turn checkpoints including a later one-shot injection, scheduler and custom-agent failures, session-start reset, exact lifecycle retirement, and queued/running plugin teardown. The new driver source has per-file 100% statement, branch, function, and line coverage.
|
||||
|
||||
A keyless Loader/stdio process test mounts the real goal domain, goal tools, goal driver, agent loop, persistence, and deterministic adapter through `cordis.yml`. One human turn creates a two-round goal; round one stops normally; round two reads the exact ref and completes it. The external JSONL assertion proves one session, round sources `1, 2`, unchanged round revision, final complete revision, five model steps, and no extra request after the terminal completion tool.
|
||||
|
||||
|
||||
@@ -39,7 +39,7 @@ Status: implemented
|
||||
| 轮次结果 | 动作 |
|
||||
|---|---|
|
||||
| 持久的 `completed` | 目标仍 active/armed 且未到上限时继续 |
|
||||
| 广义取消 / `aborted` | 暂停并解除激活 |
|
||||
| 取消已预留/接纳的目标回合,或该回合产生 `aborted` 结果 | 暂停并解除激活 |
|
||||
| 代码为 `RATE_LIMIT` 的 `error` | 标记为 `usage-limited` |
|
||||
| 其他 `error` | 阻塞 |
|
||||
| `max-tokens` | 阻塞 |
|
||||
@@ -52,9 +52,9 @@ Status: implemented
|
||||
|
||||
### 持久性与取消接缝
|
||||
|
||||
每次 `goal/changed` 通知都会产生一个检查点义务。驱动器在预留工作前等待 `ctx.sessions.flush(session)`,随后检查是否出现了更新的变更、agent 生命周期变化或竞争提示词。轮次结束时的 flush 失败会在 `turn/end` 之后通过现有 `agent/error` 通知报告;驱动器把它关联到精确尝试,并在下一次空闲决策前解除激活。
|
||||
每次 `goal/changed` 通知都会产生一个检查点义务。驱动器在预留工作前等待 `ctx.sessions.flush(session)`,随后检查是否出现了更新的变更、agent 生命周期变化或竞争提示词。轮次结束时的 flush 失败会在 `turn/end` 之后通过现有 `agent/error` 通知报告;即使并发的一次性注入已追加后续轮次,驱动器仍会找到该精确的已关闭轮次,把失败关联到精确尝试,并在下一次空闲决策前解除激活。
|
||||
|
||||
广义取消此前只在队列已清除或请求已中止后暴露结果。公共 agent 词汇现在新增只观察的 `agent/cancel-requested(agent, reason)`。具体循环仅在取消有效时发出该事件,并且发生在清除队列和中止步骤之前;融合通知会隔离失败,因此损坏的监听器不能否决取消。目标驱动器利用该边沿清除预留并暂停 active 且 armed 的目标,之后循环才销毁排队工作证据。
|
||||
广义取消此前只在队列已清除或请求已中止后暴露结果。公共 agent 词汇现在新增只观察的 `agent/cancel-requested(agent, reason)`。具体循环仅在取消有效时发出该事件,并且发生在清除队列和中止步骤之前;融合通知会隔离失败,因此损坏的监听器不能否决取消。目标驱动器利用该边沿在循环销毁排队工作证据前清除预留。若该预留是排队中或已接纳的目标尝试,取消会持久暂停目标;若取消属于没有目标尝试的无关人类工作,则只移除进程内激活态。若暂停变更抛错,驱动器会回退到解除激活,避免已取消的自动工作重新启动。
|
||||
|
||||
该通知是协调事件,不是第二个停止 API。`Agent.cancel()` 仍是唯一的公共广义取消动词,空闲调用仍是无操作;若消费者依赖此接缝,自定义 `Agent` 实现就必须满足该事件顺序。
|
||||
|
||||
@@ -68,7 +68,7 @@ Status: implemented
|
||||
|
||||
## 测试
|
||||
|
||||
单元测试使用真实 agent loop 与会话服务,只对模型编写脚本。覆盖内容包括精确连续接纳和上限执行、加载与恢复的惰性、所有结果分类、限流、请求错误、最大 token、下游提示词否决、接纳前与执行中取消、人类输入排序、排队时与下游修订竞争、伪造目标来源、变更与轮次检查点失败、调度器与自定义 agent 失败、会话启动重置、精确生命周期退出,以及排队中和运行中的插件卸载。新驱动器源码达到逐文件 100% 语句、分支、函数和行覆盖率。
|
||||
单元测试使用真实 agent loop 与会话服务,只对模型编写脚本。覆盖内容包括精确连续接纳和上限执行、加载与恢复的惰性、所有结果分类、限流、请求错误、最大 token、下游提示词否决、接纳前与执行中取消、无关人类工作取消、暂停失败回退、人类输入排序、排队时与下游修订竞争、伪造目标来源、变更与轮次检查点失败(包括后续一次性注入)、调度器与自定义 agent 失败、会话启动重置、精确生命周期退出,以及排队中和运行中的插件卸载。新驱动器源码达到逐文件 100% 语句、分支、函数和行覆盖率。
|
||||
|
||||
无密钥 Loader/stdio 进程测试通过 `cordis.yml` 挂载真实目标领域、目标工具、目标驱动器、agent loop、持久化和确定性适配器。一个人类轮次创建两回合目标;第一回合正常停止;第二回合读取精确引用并完成目标。测试从外部检查 JSONL,证明只有一个会话、回合来源依次为 `1, 2`、回合修订号不变、最终完成修订正确、共有五个模型步骤,并且终止性的完成工具后没有额外请求。
|
||||
|
||||
|
||||
@@ -30,7 +30,8 @@ The retained prompt names the JSON-quoted objective and `round/maxGoalRounds`, t
|
||||
| Durable turn outcome | Goal action | Automatic retry |
|
||||
|---|---|---|
|
||||
| `completed` with goal still active and armed | admit the next round, or mark `budget-limited` at the cap | yes |
|
||||
| broad cancellation / `aborted` | `paused` | no |
|
||||
| cancellation of a reserved/admitted goal round, or its `aborted` outcome | `paused` | no |
|
||||
| cancellation with no goal-round attempt | keep durable phase; disarm activation | no |
|
||||
| `error` with `RATE_LIMIT` | `usage-limited` | no |
|
||||
| other `error`, `max-tokens`, or a non-stale prompt rejection | `blocked` | no |
|
||||
| durability failure, disposal, interruption, or unknown future outcome | disarm or block for inspection | no |
|
||||
@@ -39,11 +40,11 @@ A goal mutation made during its round supersedes settlement of the older revisio
|
||||
|
||||
## Lifecycle and durability
|
||||
|
||||
`goal/changed` creates a durability obligation. Before queuing work, the driver awaits `ctx.sessions.flush()` and rechecks both the goal revision and competing input after the await. A closing flush failure arrives through `agent/error`; the driver disarms before another round can start.
|
||||
`goal/changed` creates a durability obligation. Before queuing work, the driver awaits `ctx.sessions.flush()` and rechecks both the goal revision and competing input after the await. A closing flush failure arrives through `agent/error`; the driver associates it with the exact closed turn even if a later one-shot injection has appended another turn, then disarms before another round can start.
|
||||
|
||||
Activation is never inherited when this plugin loads over an existing agent. `GoalService.disarm()` removes process-local authority without changing durable phase, revision, or history; explicit human-authorized resume records the later reactivation. The same rule applies after session resume and fork through the goal domain's `agent/session-start` handling.
|
||||
|
||||
Cancellation is observe-before-act: the concrete loop emits `agent/cancel-requested` before clearing queues or aborting a step, allowing this plugin to pause and disarm the exact active goal. Plugin teardown closes admission, disarms every live goal, cancels an admitted round, and awaits the driver plus agent quiescence while its event fence remains installed.
|
||||
Cancellation is observe-before-act: the concrete loop emits `agent/cancel-requested` before clearing queues or aborting a step. The plugin durably pauses an active goal only when the cancellation owns a reserved or admitted goal attempt; cancellation of unrelated human work merely disarms process-local continuation. If the pause mutation fails, the driver falls back to disarming. Plugin teardown closes admission, disarms every live goal, cancels an admitted round, and awaits the driver plus agent quiescence while its event fence remains installed.
|
||||
|
||||
## Model Experience
|
||||
|
||||
|
||||
@@ -276,8 +276,8 @@ export function apply(ctx: Context): void {
|
||||
/** Mark a post-turn persistence failure before idle scheduling can run. */
|
||||
ctx.on('agent/error', (agent, turn) => {
|
||||
const state = stateFor(agent)
|
||||
const last = agent.session.events.at(-1)
|
||||
if (last?.type !== 'turn/end' || last.data.turn !== turn) return
|
||||
const closed = agent.session.events.some(event => event.type === 'turn/end' && event.data.turn === turn)
|
||||
if (!closed) return
|
||||
if (state.attempt?.turn === turn) state.flushFailedTurns.add(turn)
|
||||
disarm(state)
|
||||
})
|
||||
@@ -312,11 +312,21 @@ export function apply(ctx: Context): void {
|
||||
})
|
||||
ctx.on('agent/cancel-requested', (agent, reason) => {
|
||||
const state = stateFor(agent)
|
||||
const attempt = state.attempt
|
||||
state.attempt = undefined
|
||||
state.competingQueued = false
|
||||
const goal = currentGoal(state)
|
||||
if (goal?.phase === 'active' && goal.activation === 'armed') {
|
||||
applyOutcome(state, goal, { kind: 'pause', reason })
|
||||
if (attempt === undefined) {
|
||||
disarm(state)
|
||||
return
|
||||
}
|
||||
try {
|
||||
applyOutcome(state, goal, { kind: 'pause', reason })
|
||||
} catch (error: unknown) {
|
||||
ctx.logger.warn(`goal-session: could not pause cancelled goal for agent "${agent.id}": ${renderThrown(error)}`)
|
||||
disarm(state)
|
||||
}
|
||||
}
|
||||
})
|
||||
ctx.on('goal/changed', (agent) => {
|
||||
|
||||
@@ -402,12 +402,17 @@ describe('same-session goal driving', () => {
|
||||
expect(test.adapter.requests).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('disarms an admitted round whose closing durability checkpoint fails', async () => {
|
||||
it('disarms an admitted round when a later injection hides its failed closing checkpoint', async () => {
|
||||
const test = await harness([textResponse('not durable')])
|
||||
let injected = false
|
||||
test.ctx.on('session/flush', (session) => {
|
||||
const lastStart = session.events.findLast(event => event.type === 'turn/start')
|
||||
if (lastStart?.type === 'turn/start' && lastStart.data.trigger.kind === 'message'
|
||||
&& lastStart.data.trigger.source.kind === 'goal') {
|
||||
&& lastStart.data.trigger.source.kind === 'goal' && !injected) {
|
||||
injected = true
|
||||
test.agent.inject([{ type: 'text', text: 'concurrent completion notice' }], {
|
||||
source: { kind: 'plugin', plugin: 'test' },
|
||||
})
|
||||
return Promise.reject(new Error('round flush failed'))
|
||||
}
|
||||
})
|
||||
@@ -421,6 +426,10 @@ describe('same-session goal driving', () => {
|
||||
|
||||
expect(goal?.phase).toBe('active')
|
||||
expect(test.adapter.requests).toHaveLength(1)
|
||||
const turns = test.agent.session.events.filter(event => event.type === 'turn/start')
|
||||
const goalTurn = turns.findIndex(event => event.data.trigger.source.kind === 'goal')
|
||||
const injectedTurn = turns.findIndex(event => event.data.trigger.source.kind === 'plugin')
|
||||
expect(injectedTurn).toBeGreaterThan(goalTurn)
|
||||
})
|
||||
|
||||
it('blocks the goal when a custom agent rejects the otherwise valid send', async () => {
|
||||
@@ -559,6 +568,42 @@ describe('same-session goal driving', () => {
|
||||
expect(test.adapter.requests).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('disarms without durably pausing when cancellation belongs to unrelated human work', async () => {
|
||||
const test = await harness(['hang'])
|
||||
test.agent.send([{ type: 'text', text: 'inspect something first' }])
|
||||
await waitForRequests(test.adapter, 1)
|
||||
const created = test.ctx.goals.create(test.agent, { objective: 'continue after inspection' })
|
||||
|
||||
test.agent.cancel('cancel the inspection')
|
||||
await test.agent.whenIdle()
|
||||
|
||||
expect(test.ctx.goals.get(test.agent)).toMatchObject({
|
||||
id: created.id,
|
||||
revision: created.revision,
|
||||
phase: 'active',
|
||||
activation: 'disarmed',
|
||||
roundsStarted: 0,
|
||||
})
|
||||
})
|
||||
|
||||
it('falls back to disarming when a cancelled reservation cannot be paused', async () => {
|
||||
const test = await harness([])
|
||||
const cancel = test.ctx.on('agent/queued', (agent, _content, info) => {
|
||||
if (agent !== test.agent || info.source.kind !== 'goal') return
|
||||
cancel()
|
||||
vi.spyOn(test.ctx.goals, 'pause').mockImplementationOnce(() => {
|
||||
throw new Error('pause failed')
|
||||
})
|
||||
agent.cancel('cancel the reserved goal round')
|
||||
})
|
||||
test.ctx.goals.create(test.agent, { objective: 'fail closed after cancellation' })
|
||||
|
||||
const goal = await waitForGoal(test.ctx, test.agent, current => current?.activation === 'disarmed')
|
||||
|
||||
expect(goal).toMatchObject({ phase: 'active', revision: 1, roundsStarted: 0 })
|
||||
expect(test.adapter.requests).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('blocks admission when downstream cancellation clears the reservation', async () => {
|
||||
const test = await harness([])
|
||||
let cancelled = false
|
||||
|
||||
Reference in New Issue
Block a user