mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
fix(compact): decide step-alignment from surface tool-pairing, fire compaction pre-step (CBR-001)
Codex round 1 CBR-001: a head-anchored compaction checkpoint was mis-classified by the log-position step-alignment scan, so a second auto-compaction over a checkpoint-headed surface silently failed. Root cause: `isStepAlignedStart/End` scanned the LOG by seq, but a `replace` op lands a checkpoint at a high log seq whose SURFACE position is the head — its log neighbours (the open step's assistant/message) are not its surface neighbours, so the forward scan wrongly reported mid-step. Fix, per the agreed direction: - Replace the two log-position predicates with one surface-anchored helper `isToolPairingBalanced(nodes, events, beforeSeq)` in `dsh-session` (renamed step-boundary.ts → tool-pairing.ts). A cut is balanced when no unanswered tool-call precedes it on the surface; a region is collapsible iff both edges are balanced cuts. The open-tail and free-node cases fall out of the same counter. It also throws on a corrupt surface (a tool/result with no matching call). - Move compaction off the in-step seam to a new "pre-step" seam fired after turn/start and before step/start, so a compaction's log-only compact/* records and its replacement node land cleanly OUTSIDE any step (the honest structure crash-safety relies on). Renamed the event agent/pre-request → agent/pre-step and switched its dispatch from parallel → serial (listeners mutate the surface as a side effect; serial isolates them so concurrent appends can't interleave). Extended the catalog generator to accept @mode serial. Regression coverage: a real-loop test driving an auto-compaction asserts the landed checkpoint is a balanced cut on both sides; unit tests pin the checkpoint case, the mid-step injection case, multi-call steps, and the corrupt-surface guard. Proven red on the old log-position logic.
This commit is contained in:
Hypatia May
@@ -30,10 +30,13 @@
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent-loop": "workspace:^",
|
||||
"@deepseek-ai/dsh-compact": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-system-prompt": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,18 +5,18 @@
|
||||
* - **Token estimation** — char/4 heuristic with per-block structural overhead.
|
||||
* - **Retention policy** — walk surface nodes tail→head, keep recent nodes up
|
||||
* to a token budget, compact everything older. The cutoff is snapped forward
|
||||
* to the next step boundary so a compacted region never splits a step's
|
||||
* tool-call/result pair (an open tail step is never crossed — compaction
|
||||
* declines and retries once it closes).
|
||||
* to the next balanced tool-pairing boundary so a compacted region never
|
||||
* splits a step's tool-call/result pair (an open tail step is never crossed —
|
||||
* compaction declines and retries once it closes).
|
||||
* - **Summarization** — `ctx.llm.stream()` assembled via `BlockAssembler`
|
||||
* (the single model-call surface; same path the loop uses) with a fixed
|
||||
* condense-the-history system prompt.
|
||||
* - **Surface mutation** — a single `user/message` replace node carries the
|
||||
* summary; `compact/*` events are log-only lock + provenance records.
|
||||
* - **Auto-compaction** — an `agent/request` waterfall listener delegates to
|
||||
* {@link BasicCompactService.compactIfNeeded} before EVERY model call (every
|
||||
* step, so a tool-heavy turn that grows the surface mid-turn still compacts);
|
||||
* it owns the sole token-pressure check.
|
||||
* - **Auto-compaction** — an `agent/pre-step` listener delegates to
|
||||
* {@link BasicCompactService.compactIfNeeded} before EVERY step (so a
|
||||
* tool-heavy turn that grows the surface mid-turn still compacts); it owns the
|
||||
* sole token-pressure check.
|
||||
*
|
||||
* A different backend (real tokenizer, template summarizer, turn-count
|
||||
* retention) either subclasses this and overrides the {@link
|
||||
@@ -33,7 +33,7 @@ import type { CompactionResult } from '@deepseek-ai/dsh-compact'
|
||||
import { BlockAssembler } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, FinishReason, GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import { isStepAlignedStart, isStepAlignedEnd } from '@deepseek-ai/dsh-session'
|
||||
import { isToolPairingBalanced } from '@deepseek-ai/dsh-session'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import type { BasicCompactConfig, ResolvedConfig } from './types.ts'
|
||||
import { resolveConfig } from './types.ts'
|
||||
@@ -169,23 +169,26 @@ export class BasicCompactService extends CompactService {
|
||||
this.config = resolveConfig(config)
|
||||
|
||||
if (this.config.auto) {
|
||||
// Auto-compaction: delegate to compactIfNeeded before EVERY model call —
|
||||
// every step, not just the first. This is LOAD-BEARING for runaway-turn
|
||||
// survival: a tool-heavy ReAct turn appends an assistant/message and a
|
||||
// tool/result per step, so the surface (and the derived token count) grows
|
||||
// WITHIN a turn. The only moment to rescue a turn that alone approaches the
|
||||
// window is the next step's pre-request; gating to a turn's first step
|
||||
// would let a runaway turn overflow before the next turn's check. The
|
||||
// listener owns NO threshold logic — compactIfNeeded is the single place
|
||||
// that decides whether to compact, and its in-progress lock serializes
|
||||
// concurrent attempts.
|
||||
// Auto-compaction: delegate to compactIfNeeded before EVERY step. This is
|
||||
// LOAD-BEARING for runaway-turn survival: a tool-heavy ReAct turn appends
|
||||
// an assistant/message and a tool/result per step, so the surface (and the
|
||||
// derived token count) grows WITHIN a turn. The only moment to rescue a
|
||||
// turn that alone approaches the window is the next step's pre-step
|
||||
// checkpoint; gating to a turn's first step would let a runaway turn
|
||||
// overflow before the next turn's check. The listener owns NO threshold
|
||||
// logic — compactIfNeeded is the single place that decides whether to
|
||||
// compact, and its in-progress lock serializes concurrent attempts.
|
||||
//
|
||||
// It runs on `agent/pre-request` (a parallel surface-mutation checkpoint),
|
||||
// NOT `agent/request`: compaction mutates the session surface, and the loop
|
||||
// derives the request `messages` AFTER this fires — so a single derive
|
||||
// already reflects the compaction, with no double-derive and no need to
|
||||
// rewrite an already-assembled `messages` array.
|
||||
ctx.on('agent/pre-request', async (agent: Agent, _turn: number, _step: number, system: string, model: string, signal: AbortSignal) => {
|
||||
// It runs on `agent/pre-step` (a serial surface-mutation checkpoint fired
|
||||
// AFTER turn/start but BEFORE step/start), NOT `agent/request`: compaction
|
||||
// mutates the session surface, and the loop derives the request `messages`
|
||||
// AFTER this fires — so a single derive already reflects the compaction,
|
||||
// with no double-derive and no need to rewrite an already-assembled
|
||||
// `messages` array. Firing pre-step (outside any open step) keeps the
|
||||
// log-only `compact/*` records and the replacement node cleanly outside a
|
||||
// step, so a crash mid-compaction leaves an inert orphan the turn-repair
|
||||
// closes — never a half-open step.
|
||||
ctx.on('agent/pre-step', async (agent: Agent, _turn: number, _step: number, system: string, model: string, signal: AbortSignal) => {
|
||||
try {
|
||||
const result = await this.compactIfNeeded(agent.session, system, model, signal)
|
||||
if (result) {
|
||||
@@ -321,17 +324,19 @@ export class BasicCompactService extends CompactService {
|
||||
* Retention is a UNIFORM tail→head walk over the whole surface — turn
|
||||
* boundaries play NO role. Walking node-by-node from the tail and summing
|
||||
* token estimates, once the retained total reaches `retainTokens` the cutoff
|
||||
* is rounded to a step-aligned boundary: if the walk stopped INSIDE a step,
|
||||
* it continues head-ward past that step's `step/start` so the whole step is
|
||||
* retained (never splitting a step's tool-calls from their results); if it
|
||||
* stopped on a free node (a node belonging to no step), that is already a
|
||||
* clean boundary. This always rounds toward retaining MORE (retained ≥
|
||||
* `retainTokens`) and is step-aligned by construction — no separate snap pass.
|
||||
* is rounded to a balanced tool-pairing boundary: if the cut before the
|
||||
* retained node is unbalanced (an unanswered tool-call sits before it — i.e.
|
||||
* it is mid-step), the walk continues head-ward until the cut is balanced so
|
||||
* the whole step is retained (never splitting a step's tool-calls from their
|
||||
* results); if it stopped on a free node (a node belonging to no step), that
|
||||
* cut is already balanced. This always rounds toward retaining MORE (retained
|
||||
* ≥ `retainTokens`) and is boundary-safe by construction — no separate snap
|
||||
* pass.
|
||||
*
|
||||
* The compacted range is always anchored at the surface HEAD (`nodes[0]`):
|
||||
* auto-compaction re-consolidates any prior head checkpoint into one fresh
|
||||
* checkpoint. Declines (`null`) when nothing is over threshold, when the whole
|
||||
* surface fits the retain budget, or when no step-aligned cutoff exists in the
|
||||
* surface fits the retain budget, or when no balanced cutoff exists in the
|
||||
* compactable range (its only content is an open tail step — retry once it
|
||||
* closes).
|
||||
*/
|
||||
@@ -371,24 +376,26 @@ export class BasicCompactService extends CompactService {
|
||||
// The whole surface fits the retain budget — nothing to compact.
|
||||
if (keepFromIdx === 0) return null
|
||||
|
||||
// Round the cutoff to a step boundary: if `keepFromIdx` sits INSIDE a step,
|
||||
// extend the retained side head-ward until the boundary is a step-aligned
|
||||
// start, so the compacted range ends on a clean step edge. A node that
|
||||
// belongs to no step is already a valid start. Decline if no step-aligned
|
||||
// start exists at or below `keepFromIdx` (the compactable range is only an
|
||||
// un-splittable open tail step — retry once it closes).
|
||||
// Round the cutoff to a tool-pairing boundary: if the cut before
|
||||
// `nodes[keepFromIdx]` is unbalanced (an unanswered tool-call sits before
|
||||
// it — i.e. it is mid-step), extend the retained side head-ward until the
|
||||
// cut is balanced, so the compacted range ends without splitting an
|
||||
// assistant↔result pair. A node that belongs to no step is already a
|
||||
// balanced (free) boundary. Decline if no balanced cut exists at or below
|
||||
// `keepFromIdx` (the compactable range is only an un-splittable open tail
|
||||
// step — retry once it closes).
|
||||
while (keepFromIdx > 0) {
|
||||
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
|
||||
if (isStepAlignedStart(events, nodes[keepFromIdx]!.seq)) break
|
||||
if (isToolPairingBalanced(nodes, events, nodes[keepFromIdx]!.seq)) break
|
||||
keepFromIdx -= 1
|
||||
}
|
||||
if (keepFromIdx === 0) return null
|
||||
|
||||
// The compacted range is [head … keepFromIdx - 1], anchored at the head.
|
||||
// The cutoff node `nodes[keepFromIdx - 1]` is necessarily a step-aligned END:
|
||||
// the retained start `nodes[keepFromIdx]` is a step-aligned START (a boundary
|
||||
// marker sits between them in the log), and that same boundary makes the node
|
||||
// before it a step-aligned end — so no separate end check is needed.
|
||||
// The cutoff node `nodes[keepFromIdx - 1]` is necessarily a balanced END:
|
||||
// the retained start `nodes[keepFromIdx]` opens on a balanced cut, and that
|
||||
// same cut is the cut AFTER `nodes[keepFromIdx - 1]` — so no separate end
|
||||
// check is needed.
|
||||
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
|
||||
const firstSeq = nodes[0]!.seq
|
||||
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
|
||||
@@ -420,19 +427,24 @@ export class BasicCompactService extends CompactService {
|
||||
throw new Error(`compactRegion: start seq ${start} (position ${startIdx}) is after end seq ${end} (position ${endIdx}) on the surface`)
|
||||
}
|
||||
|
||||
// The region must contain whole steps, never split a step's
|
||||
// assistant-message tool-calls from their tool/results (which would orphan
|
||||
// one side and produce a transcript every provider rejects). A boundary is
|
||||
// valid when it sits on a step edge or on a node that belongs to no step
|
||||
// (pre-step user message, inter-step steering, injection context); an `end`
|
||||
// inside an open (unclosed) tail step is also rejected — its tool-calls have
|
||||
// no results yet. See dsh-session's step-boundary predicates.
|
||||
// The region must never split a step's assistant-message tool-calls from
|
||||
// their tool/results (which would orphan one side and produce a transcript
|
||||
// every provider rejects). A region is safe iff BOTH its edges are balanced
|
||||
// cuts: the cut before `start`, and the cut after `end`. A node that belongs
|
||||
// to no step (pre-step user message, inter-step steering, injection context)
|
||||
// is a balanced (free) boundary; an `end` inside an open (unclosed) tail step
|
||||
// leaves the cut after it unbalanced (the open tool-call has no result yet),
|
||||
// so it is rejected. See dsh-session's tool-pairing balance check.
|
||||
const events = session.events
|
||||
if (!isStepAlignedStart(events, start)) {
|
||||
throw new Error(`compactRegion: start seq ${start} is not on a step boundary (would split a step's tool-call/result pair)`)
|
||||
if (!isToolPairingBalanced(nodes, events, start)) {
|
||||
throw new Error(`compactRegion: start seq ${start} is not a balanced boundary (would split a step's tool-call/result pair)`)
|
||||
}
|
||||
if (!isStepAlignedEnd(events, end)) {
|
||||
throw new Error(`compactRegion: end seq ${end} is not on a step boundary (would split a step, or the step is still open)`)
|
||||
// The cut after `end` is named by `end`'s surface successor, or `null` when
|
||||
// `end` is the tail.
|
||||
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
|
||||
const afterEnd: number | null = nodes[endIdx]!.next
|
||||
if (!isToolPairingBalanced(nodes, events, afterEnd)) {
|
||||
throw new Error(`compactRegion: end seq ${end} is not a balanced boundary (would split a step, or the step is still open)`)
|
||||
}
|
||||
|
||||
if (this._isCompactionInProgress(session)) {
|
||||
@@ -441,10 +453,11 @@ export class BasicCompactService extends CompactService {
|
||||
|
||||
// Compaction's events (compact/* and the replacement user/message) must be
|
||||
// turn-enclosed: the session-log contract rejects any plugin event appended
|
||||
// outside an open turn. Auto-compaction satisfies this — it runs inside the
|
||||
// `agent/request` waterfall, strictly between a turn's start and end. A
|
||||
// manual call on a fully-closed session has no turn to enclose the events,
|
||||
// so reject rather than emit an un-enclosed run.
|
||||
// outside an open turn. Auto-compaction satisfies this — it runs on the
|
||||
// `agent/pre-step` seam, after `turn/start` and before `step/start`, so
|
||||
// strictly inside the open turn (but outside any step). A manual call on a
|
||||
// fully-closed session has no turn to enclose the events, so reject rather
|
||||
// than emit an un-enclosed run.
|
||||
const turn = this._openTurn(session)
|
||||
if (turn === null) {
|
||||
throw new Error('compactRegion: no open turn — compaction events must be enclosed in a turn')
|
||||
|
||||
@@ -49,9 +49,9 @@ function createTestService(config: BasicCompactConfig = {}): TestCompactService
|
||||
/**
|
||||
* Build a multi-turn session with surface markers (simulating real agent-loop
|
||||
* output). Compaction always runs inside an OPEN turn (the loop fires the
|
||||
* `agent/request` waterfall between a turn's start and its end), so by default
|
||||
* the session is left with a trailing open turn: turns `1..turns` close, then
|
||||
* one more `turn/start` opens with no matching `turn/end`. Pass
|
||||
* `agent/pre-step` seam after a turn's start and before a step's start), so by
|
||||
* default the session is left with a trailing open turn: turns `1..turns`
|
||||
* close, then one more `turn/start` opens with no matching `turn/end`. Pass
|
||||
* `{ leaveOpen: false }` for a fully-closed session (e.g. to assert that manual
|
||||
* compaction is rejected when no turn is open).
|
||||
*/
|
||||
@@ -219,7 +219,7 @@ describe('BasicCompactService step-alignment (never split a tool-call/result pai
|
||||
expect(s.events.some(e => e.type === 'compact/start')).toBe(false)
|
||||
})
|
||||
|
||||
it('compactRegion rejects a start that is not a step boundary (splits a step)', async () => {
|
||||
it('compactRegion rejects a start that splits a step (unbalanced boundary)', async () => {
|
||||
const svc = createTestService()
|
||||
const session = toolTurnSession(1)
|
||||
const nodes = session.surface.nodes // [user, asst(tool-call), result]
|
||||
@@ -228,11 +228,11 @@ describe('BasicCompactService step-alignment (never split a tool-call/result pai
|
||||
// start = the tool/result: its issuing assistant precedes it IN THE SAME STEP,
|
||||
// so starting here would orphan that assistant's tool-call. end is fine (user).
|
||||
await expect(svc.compactRegion(session, resultSeq, resultSeq, 'm'))
|
||||
.rejects.toThrow(/start seq .* is not on a step boundary/)
|
||||
.rejects.toThrow(/start seq .* is not a balanced boundary/)
|
||||
expect(userSeq).toBeLessThan(resultSeq) // sanity: ordering as expected
|
||||
})
|
||||
|
||||
it('compactRegion rejects an end that is not a step boundary (splits a step)', async () => {
|
||||
it('compactRegion rejects an end that splits a step (unbalanced boundary)', async () => {
|
||||
const svc = createTestService()
|
||||
const session = toolTurnSession(1)
|
||||
const nodes = session.surface.nodes
|
||||
@@ -241,7 +241,7 @@ describe('BasicCompactService step-alignment (never split a tool-call/result pai
|
||||
// end = the assistant/message: its tool/result follows IN THE SAME STEP, so
|
||||
// ending here would strand that result. start is fine (the pre-step user).
|
||||
await expect(svc.compactRegion(session, userSeq, asstSeq, 'm'))
|
||||
.rejects.toThrow(/end seq .* is not on a step boundary/)
|
||||
.rejects.toThrow(/end seq .* is not a balanced boundary/)
|
||||
})
|
||||
|
||||
it('compactRegion rejects an end inside an open tail step', async () => {
|
||||
@@ -258,7 +258,7 @@ describe('BasicCompactService step-alignment (never split a tool-call/result pai
|
||||
const userSeq = nodes[0]!.seq
|
||||
const asstSeq = nodes[1]!.seq
|
||||
await expect(svc.compactRegion(s, userSeq, asstSeq, 'm'))
|
||||
.rejects.toThrow(/end seq .* is not on a step boundary/)
|
||||
.rejects.toThrow(/end seq .* is not a balanced boundary/)
|
||||
})
|
||||
|
||||
it('compactRegion accepts step-aligned boundaries (pre-step user → last result of a closed step)', async () => {
|
||||
@@ -883,10 +883,10 @@ describe('BasicCompactService.summarize (real ctx.llm.stream)', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('BasicCompactService auto-compaction (agent/pre-request listener)', () => {
|
||||
/** Fire the agent/pre-request parallel checkpoint as the loop does. */
|
||||
function firePreRequest(ctx: Context, agent: Agent, step: number, system: string, model: string): Promise<unknown> {
|
||||
return ctx.parallel('agent/pre-request', agent, 1, step, system, model, SIGNAL)
|
||||
describe('BasicCompactService auto-compaction (agent/pre-step listener)', () => {
|
||||
/** Fire the agent/pre-step serial checkpoint as the loop does. */
|
||||
function firePreStep(ctx: Context, agent: Agent, step: number, system: string, model: string): Promise<unknown> {
|
||||
return ctx.serial('agent/pre-step', agent, 1, step, system, model, SIGNAL)
|
||||
}
|
||||
|
||||
it('compacts (mutating the surface) when over threshold', async () => {
|
||||
@@ -896,7 +896,7 @@ describe('BasicCompactService auto-compaction (agent/pre-request listener)', ()
|
||||
const agent = stubAgent(session, 'test-model')
|
||||
const before = session.surface.nodes.length
|
||||
|
||||
await firePreRequest(ctx, agent, 1, '', 'test-model')
|
||||
await firePreStep(ctx, agent, 1, '', 'test-model')
|
||||
|
||||
// The surface shrank in place, and a summary checkpoint landed.
|
||||
expect(session.surface.nodes.length).toBeLessThan(before)
|
||||
@@ -913,7 +913,7 @@ describe('BasicCompactService auto-compaction (agent/pre-request listener)', ()
|
||||
|
||||
// A step-2 checkpoint (a tool-heavy turn's later step) must still compact —
|
||||
// the surface accumulated assistant/message + tool/result nodes since step 1.
|
||||
await firePreRequest(ctx, agent, 2, '', 'test-model')
|
||||
await firePreStep(ctx, agent, 2, '', 'test-model')
|
||||
expect(session.events.some(e => e.type === 'compact/start')).toBe(true)
|
||||
})
|
||||
|
||||
@@ -923,7 +923,7 @@ describe('BasicCompactService auto-compaction (agent/pre-request listener)', ()
|
||||
const session = multiTurnSession(1, 1)
|
||||
const agent = stubAgent(session, 'test-model')
|
||||
|
||||
await firePreRequest(ctx, agent, 1, '', 'test-model')
|
||||
await firePreStep(ctx, agent, 1, '', 'test-model')
|
||||
expect(session.events.some(e => e.type === 'compact/start')).toBe(false)
|
||||
})
|
||||
|
||||
@@ -937,7 +937,7 @@ describe('BasicCompactService auto-compaction (agent/pre-request listener)', ()
|
||||
const agent = stubAgent(session, 'missing-model')
|
||||
const before = session.surface.nodes.length
|
||||
|
||||
await firePreRequest(ctx, agent, 1, '', 'missing-model')
|
||||
await firePreStep(ctx, agent, 1, '', 'missing-model')
|
||||
// No summary landed; the surface is unchanged.
|
||||
expect(session.events.some(e => e.type === 'compact/summary')).toBe(false)
|
||||
expect(session.surface.nodes.length).toBe(before)
|
||||
@@ -949,7 +949,7 @@ describe('BasicCompactService auto-compaction (agent/pre-request listener)', ()
|
||||
const session = multiTurnSession(3, 1)
|
||||
const agent = stubAgent(session, 'test-model')
|
||||
|
||||
await firePreRequest(ctx, agent, 1, '', 'test-model')
|
||||
await firePreStep(ctx, agent, 1, '', 'test-model')
|
||||
expect(session.events.some(e => e.type === 'compact/start')).toBe(false)
|
||||
})
|
||||
})
|
||||
@@ -992,6 +992,10 @@ describe('BasicCompactService._extractText branches', () => {
|
||||
s.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
s.append('step/start', { turn: 1, step: 1 })
|
||||
s.append('user/message', { content: [{ type: 'text', text: 'run it' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
s.append('assistant/message', {
|
||||
turn: 1, step: 1,
|
||||
content: [{ type: 'tool-call', id: CallId('c9'), name: 'bash', arguments: '{}' }],
|
||||
}, { surfaceOp: 'append' })
|
||||
s.append('tool/call', { turn: 1, step: 1, callId: CallId('c9'), name: 'bash', arguments: '{}' })
|
||||
s.append('tool/result', {
|
||||
turn: 1, step: 1, callId: CallId('c9'),
|
||||
@@ -1014,12 +1018,15 @@ describe('BasicCompactService edge cases', () => {
|
||||
const s = new Session(SessionId('toolresult'))
|
||||
s.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
s.append('step/start', { turn: 1, step: 1 })
|
||||
// assistant/message carrying a nested tool-result block and an unknown block.
|
||||
// assistant/message carrying a nested tool-result block, an unknown block,
|
||||
// and the tool-call that the following tool/result answers (so the surface
|
||||
// is tool-pairing balanced).
|
||||
s.append('assistant/message', {
|
||||
turn: 1, step: 1,
|
||||
content: [
|
||||
{ type: 'tool-result', toolCallId: CallId('n1'), content: [{ type: 'image', url: 'https://x/n.png' }] },
|
||||
{ type: 'custom-widget', payload: 'x' } as unknown as ContentBlock,
|
||||
{ type: 'tool-call', id: CallId('b1'), name: 'bash', arguments: '{}' },
|
||||
],
|
||||
}, { surfaceOp: 'append' })
|
||||
// tool/result whose content is itself only non-text → bare '[tool-result]'.
|
||||
@@ -1059,7 +1066,7 @@ describe('BasicCompactService edge cases', () => {
|
||||
const session = multiTurnSession(4, 1)
|
||||
const agent = stubAgent(session, 'test-model')
|
||||
|
||||
await ctx.parallel('agent/pre-request', agent, 1, 1, '', 'test-model', SIGNAL)
|
||||
await ctx.serial('agent/pre-step', agent, 1, 1, '', 'test-model', SIGNAL)
|
||||
expect(session.events.some(e => e.type === 'compact/summary')).toBe(true)
|
||||
// The surface was mutated; the head message is the framed summary checkpoint.
|
||||
expect(session.deriveMessages()[0]!.content).toContainEqual({ type: 'text', text: 'SUMMARY' })
|
||||
@@ -1140,7 +1147,7 @@ describe('BasicCompactService edge cases', () => {
|
||||
const agent = stubAgent(session, 'test-model')
|
||||
const before = session.surface.nodes.length
|
||||
|
||||
await ctx.parallel('agent/pre-request', agent, 1, 1, '', 'test-model', SIGNAL)
|
||||
await ctx.serial('agent/pre-step', agent, 1, 1, '', 'test-model', SIGNAL)
|
||||
// The failure was swallowed; the surface is untouched and a warning logged.
|
||||
expect(session.surface.nodes.length).toBe(before)
|
||||
expect(session.events.some(e => e.type === 'compact/summary')).toBe(false)
|
||||
@@ -1157,7 +1164,7 @@ describe('BasicCompactService edge cases', () => {
|
||||
const agent = stubAgent(session, 'test-model')
|
||||
const bigSystem = 'x'.repeat(900) // ceil(900/4)=225 > threshold 200
|
||||
|
||||
await ctx.parallel('agent/pre-request', agent, 1, 1, bigSystem, 'test-model', SIGNAL)
|
||||
await ctx.serial('agent/pre-step', agent, 1, 1, bigSystem, 'test-model', SIGNAL)
|
||||
expect(session.events.some(e => e.type === 'compact/start')).toBe(false)
|
||||
expect(svc.summarizeCalls.length).toBe(0)
|
||||
})
|
||||
@@ -1166,23 +1173,37 @@ describe('BasicCompactService edge cases', () => {
|
||||
const svc = createTestService()
|
||||
const s = new Session(SessionId('empties'))
|
||||
s.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
// Step 1: an empty-text user, an empty-reasoning assistant with NO tool-call
|
||||
// (balanced: nothing to answer), and empty context/steering — all extract to
|
||||
// nothing and are skipped.
|
||||
s.append('step/start', { turn: 1, step: 1 })
|
||||
// Empty-text text/reasoning blocks contribute nothing → message skipped.
|
||||
s.append('user/message', { content: [{ type: 'text', text: '' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
s.append('assistant/message', { turn: 1, step: 1, content: [{ type: 'reasoning', text: '' }] }, { surfaceOp: 'append' })
|
||||
// tool/result with empty content → empty extraction → skipped.
|
||||
s.append('tool/call', { turn: 1, step: 1, callId: CallId('z1'), name: 'bash', arguments: '{}' })
|
||||
s.append('tool/result', { turn: 1, step: 1, callId: CallId('z1'), content: [], isError: false }, { surfaceOp: 'append' })
|
||||
s.append('context/message', { content: [], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
s.append('steering/message', { turn: 1, content: [{ type: 'text', text: '' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
s.append('step/end', { turn: 1, step: 1 })
|
||||
// Step 2: a tool exchange whose tool/result has empty content → empty
|
||||
// extraction → skipped. The assistant carries the matching tool-call so the
|
||||
// surface stays tool-pairing balanced; its text extracts to the tool-call
|
||||
// placeholder (the one surviving line).
|
||||
s.append('step/start', { turn: 1, step: 2 })
|
||||
s.append('assistant/message', {
|
||||
turn: 1, step: 2,
|
||||
content: [{ type: 'tool-call', id: CallId('z1'), name: 'bash', arguments: '{}' }],
|
||||
}, { surfaceOp: 'append' })
|
||||
s.append('tool/call', { turn: 1, step: 2, callId: CallId('z1'), name: 'bash', arguments: '{}' })
|
||||
s.append('tool/result', { turn: 1, step: 2, callId: CallId('z1'), content: [], isError: false }, { surfaceOp: 'append' })
|
||||
s.append('step/end', { turn: 1, step: 2 })
|
||||
s.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
s.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
|
||||
const nodes = s.surface.nodes
|
||||
await svc.compactRegion(s, nodes[0]!.seq, nodes[nodes.length - 1]!.seq, 'm')
|
||||
// Every message extracted to empty text — the conversation is empty.
|
||||
expect(svc.summarizeCalls[0]!.text).toBe('')
|
||||
// Every empty-content message (user text, empty reasoning, empty-content
|
||||
// tool/result, empty context, empty steering) extracted to nothing and was
|
||||
// skipped — the only surviving line is the assistant's tool-call (which a
|
||||
// balanced surface requires to answer the tool/result).
|
||||
expect(svc.summarizeCalls[0]!.text).toBe('Assistant: [tool-call: bash({})]')
|
||||
})
|
||||
|
||||
it('renders non-text blocks as type-tagged placeholders across all message kinds', async () => {
|
||||
@@ -1192,8 +1213,15 @@ describe('BasicCompactService edge cases', () => {
|
||||
s.append('step/start', { turn: 1, step: 1 })
|
||||
// user/message with only an image block → '[image]' placeholder.
|
||||
s.append('user/message', { content: [{ type: 'image', url: 'https://x/y.png' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
// assistant/message with only an image block → '[image]' placeholder.
|
||||
s.append('assistant/message', { turn: 1, step: 1, content: [{ type: 'image', url: 'https://x/z.png' }] }, { surfaceOp: 'append' })
|
||||
// assistant/message with an image block AND the tool-call its tool/result
|
||||
// answers (so the surface is tool-pairing balanced) → '[image]' placeholder.
|
||||
s.append('assistant/message', {
|
||||
turn: 1, step: 1,
|
||||
content: [
|
||||
{ type: 'image', url: 'https://x/z.png' },
|
||||
{ type: 'tool-call', id: CallId('e1'), name: 'bash', arguments: '{}' },
|
||||
],
|
||||
}, { surfaceOp: 'append' })
|
||||
// tool/result with an image block → '[image]' placeholder.
|
||||
s.append('tool/call', { turn: 1, step: 1, callId: CallId('e1'), name: 'bash', arguments: '{}' })
|
||||
s.append('tool/result', { turn: 1, step: 1, callId: CallId('e1'), content: [{ type: 'image', url: 'https://x/r.png' }], isError: false }, { surfaceOp: 'append' })
|
||||
|
||||
155
packages/compact/compact-basic/tests/compact-loop-repro.spec.ts
Normal file
155
packages/compact/compact-basic/tests/compact-loop-repro.spec.ts
Normal file
@@ -0,0 +1,155 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId, LlmAdapter } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore from '@deepseek-ai/dsh-session'
|
||||
import { isToolPairingBalanced } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop'
|
||||
import * as Invariants from '@deepseek-ai/dsh-invariants'
|
||||
import { BasicCompactService } from '@deepseek-ai/dsh-compact-basic'
|
||||
import type { SurfaceEvent } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/**
|
||||
* CBR-001 regression: a compaction checkpoint that the REAL loop lands is a
|
||||
* free surface boundary (it carries no tool-call/result pair), so it must be a
|
||||
* valid region edge on BOTH sides. A surface-anchored balance check sees that;
|
||||
* the abandoned log-position scan did not.
|
||||
*
|
||||
* The loop fires the compaction seam mid-flight, so the landed checkpoint
|
||||
* `user/message{replace}` sits at a HIGH log seq positioned beside the current
|
||||
* step even though its SURFACE position is the head. A log-position forward scan
|
||||
* from the checkpoint reaches the step's own later `assistant/message` and
|
||||
* wrongly reports the checkpoint as mid-step — refusing it as a region end. A
|
||||
* SECOND compaction that re-summarizes just that head checkpoint (region end ==
|
||||
* checkpoint) therefore throws and is swallowed, so the surface never
|
||||
* re-consolidates.
|
||||
*
|
||||
* This drives a real auto-compaction through the agent-loop and asserts the
|
||||
* landed checkpoint balances on both sides AND that re-compacting it (end ==
|
||||
* checkpoint) succeeds. RED on the log-position predicates; GREEN once alignment
|
||||
* is decided from surface tool-pairing balance.
|
||||
*/
|
||||
|
||||
const TOKENS_PER_BLOCK = 10
|
||||
|
||||
class ReproCompactService extends BasicCompactService {
|
||||
override estimateContentTokens(blocks: readonly ContentBlock[]): number {
|
||||
return blocks.length * TOKENS_PER_BLOCK
|
||||
}
|
||||
|
||||
override async summarize(): Promise<ContentBlock[]> {
|
||||
return [{ type: 'text', text: 'CHECKPOINT SUMMARY' }]
|
||||
}
|
||||
}
|
||||
|
||||
/** Each call emits one tool-call until exhausted, then a final text answer. */
|
||||
class StepwiseToolAdapter extends LlmAdapter {
|
||||
calls = 0
|
||||
constructor(private toolSteps: number) {
|
||||
super()
|
||||
}
|
||||
|
||||
async * stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
const n = this.calls
|
||||
this.calls += 1
|
||||
if (n < this.toolSteps) {
|
||||
const id = CallId(`c${n}`)
|
||||
const args = `{"i":${n}}`
|
||||
yield { type: 'block-start', index: 0, blockType: 'text' }
|
||||
yield { type: 'block-end', index: 0, block: { type: 'text', text: `step ${n}` } }
|
||||
yield { type: 'block-start', index: 1, blockType: 'tool-call' }
|
||||
yield { type: 'block-end', index: 1, block: { type: 'tool-call', id, name: 'work', arguments: args } }
|
||||
yield { type: 'finish', reason: { kind: 'tool-calls' } }
|
||||
return
|
||||
}
|
||||
yield { type: 'block-start', index: 0, blockType: 'text' }
|
||||
yield { type: 'block-end', index: 0, block: { type: 'text', text: 'all done' } }
|
||||
yield { type: 'finish', reason: { kind: 'stop' } }
|
||||
}
|
||||
}
|
||||
|
||||
async function harness(toolSteps: number): Promise<{ ctx: Context; compact: ReproCompactService }> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(Invariants, {})
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(AgentRegistry)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
ctx.llm.registerAdapter(['mock'], new StepwiseToolAdapter(toolSteps))
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'work',
|
||||
description: 'does work',
|
||||
parameters: { i: { type: 'number' } },
|
||||
async execute() {
|
||||
return [{ type: 'text', text: 'work result' }]
|
||||
},
|
||||
}))
|
||||
// Tiny window so a couple of tool steps cross the threshold and compaction
|
||||
// fires within the runaway turn. Convergence invariant holds:
|
||||
// summarizationMaxTokens(1) + retainTokens(20) = 21 <= floor(60*0.5) = 30.
|
||||
const compact = new ReproCompactService(ctx, {
|
||||
auto: true,
|
||||
contextWindow: 60,
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 20,
|
||||
summarizationMaxTokens: 1,
|
||||
})
|
||||
return { ctx, compact }
|
||||
}
|
||||
|
||||
function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
const dispose = ctx.on('agent/status', (subject, status) => {
|
||||
if (subject === agent && status === 'idle') {
|
||||
dispose()
|
||||
resolve()
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
describe('CBR-001: a real-loop checkpoint is a valid boundary on both sides', () => {
|
||||
it('the head checkpoint the loop lands is a balanced cut on both sides', async () => {
|
||||
const { ctx } = await harness(8)
|
||||
try {
|
||||
const agent = ctx.agentLoop.create(AgentId('repro'), { model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'do a long multi-step task' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const events = [...agent.session.events]
|
||||
// A compaction ran: at least one checkpoint landed on the surface.
|
||||
const checkpoints = events.filter(
|
||||
(e): e is SurfaceEvent =>
|
||||
e.type === 'user/message'
|
||||
&& typeof (e as SurfaceEvent).surfaceOp === 'object',
|
||||
)
|
||||
expect(checkpoints.length).toBeGreaterThan(0)
|
||||
|
||||
// The loop fired compaction mid-flight, so each landed checkpoint sits at a
|
||||
// high log seq beside the step it landed in, even though its SURFACE
|
||||
// position is the head of the range it shadowed. A checkpoint carries no
|
||||
// tool-call/result pair (only summarized prose), so every checkpoint still
|
||||
// on the surface must be a balanced cut on BOTH sides — the cut before it
|
||||
// (region START) and the cut after it (region END). The abandoned
|
||||
// log-position scan reported the END as mis-aligned because the forward log
|
||||
// scan reached the neighbouring step's assistant/message.
|
||||
const nodes = agent.session.surface.nodes
|
||||
for (const cp of checkpoints) {
|
||||
const node = nodes.find(n => n.seq === cp.seq)
|
||||
if (!node) continue // shadowed by a later checkpoint — no longer an edge.
|
||||
expect(isToolPairingBalanced(nodes, events, node.seq),
|
||||
`checkpoint seq ${node.seq} must be a balanced region START`).toBe(true)
|
||||
expect(isToolPairingBalanced(nodes, events, node.next),
|
||||
`checkpoint seq ${node.seq} must be a balanced region END`).toBe(true)
|
||||
}
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
})
|
||||
Reference in New Issue
Block a user