Merge branch 'codex/canonical-tool-output' into codex/code-mode-typed-results

docs/core-data-structures/compaction.md

@@ -58,7 +58,7 @@ Automatic callers state why policy is running; implementations may treat confirm
 type CompactionTrigger = 'pressure' | 'context-overflow'
 ```
 
-`CompactService` exposes `compactIfNeeded(agent, trigger, signal)` for automatic `pressure` or `context-overflow` policy, returning `null` when no safe work exists, and `compactRegion(...)` for an explicit inclusive surface range. Implementations must forward the supplied signal to summarization. The seam owns no pricing API: the singleton [`ctx.tokenMeter`](token-meter.md) directly owns estimation and replay, while `dsh-compact-basic` owns retention, event sequencing, routed summarization calls, and their configuration.
+`CompactService` exposes `compactIfNeeded(agent, trigger, signal)` for automatic `pressure` or `context-overflow` policy, returning `null` when no safe work exists, and `compactRegion(...)` for an explicit inclusive surface range. Every backend marks its replacement `user/message` with the package-exported `COMPACT_CHECKPOINT_SOURCE`; consumers call `isCompactCheckpointSource()` instead of coupling checkpoint recognition to one backend. Implementations must forward the supplied signal to summarization. The seam owns no pricing API: the singleton [`ctx.tokenMeter`](token-meter.md) directly owns estimation and replay, while `dsh-compact-basic` owns retention, event sequencing, routed summarization calls, and their configuration.
 
 Pressure compaction runs at serial `agent/post-step`, after successful assistant output, tool results, buffered context, and steering are durable but before `step/end`. Once pressure or canonical overflow qualifies, compact-basic invokes optional [`ctx.toolResultPrune`](../../packages/compact/compact-tool-result-prune/README.md) before range selection, remeasures through `ctx.tokenMeter`, and can advance the surface without a summary. Failed-request recovery runs through `agent/request-error` after the failed step closes and authorizes a fresh numbered-step retry only when the surface replacement generation advances, even if later summary work throws after pruning; cancellation still wins. Region boundaries preserve tool-call/result pairing but not whole turns, allowing early closed steps of one oversized turn to compact. `dsh-compact-basic` owns thresholds, retained-tail policy, overflow caps, and failure handling.
 

docs/core-data-structures/core.md

@@ -29,6 +29,7 @@ Everything else is documented on a **sub-page**, not here. The rule that draws t
 | [user-interaction.md](user-interaction.md) | the UI-backed human question/answer seam: `AskUserQuestionRequest`, answer/options vocabulary, provider API, error taxonomy |
 | [approval.md](approval.md) | the one-shot user-approval seam: `ApprovalRequest`, `ApprovalOutcome`, per-session policy, audit and answerer contracts |
 | [bash.md](bash.md) | the bash executor seam: `BashExecRequest`/`Spec`, `BashRunResult`, background `BashProcess` handles |
+| [pty.md](pty.md) | persistent terminal ids, backend/session contracts, send readiness, bounded reads, and owner-visible snapshots |
 | [sandbox.md](sandbox.md) | per-session policy resolution and the process-confinement seam: file-effect modes, execution/provider policies, `ConfinedArgv`, enforcement and fail-closed errors |
 | [code-runtime.md](code-runtime.md) | the code-execution seam: `CodeRunRequest`/`Result`, binding namespaces, captured logs, the `CodeRunFailure` taxonomy |
 | [filesystem.md](filesystem.md) | the filesystem seam: `FsTarget`, read/write/edit outcomes, observed-file state, `FsErrorCode` |
@@ -358,11 +359,27 @@ The fourteen event variants (`turn/start`, `turn/end`, `step/start`, `step/end`,
 
 Source: [`packages/core/agent/src/types.ts`](../../packages/core/agent/src/types.ts)
 
-`InjectOptions` extends ordinary message attribution with durable model-hidden JSON metadata:
+```ts type-equiv
+/**
+ * Message options. An omitted source attests direct human input as `{ kind: 'user' }`
+ * and may authorize policy consumers, so non-human producers must label their content.
+ */
+interface SendOptions {
+  source?: MessageSource
+  /**
+   * Model-facing contexts captured with this inbox item. A queued prompt exposes
+   * them through the default `agent/prompt-submit` allow decision, while steering
+   * records them directly at its next checkpoint.
+   */
+  contexts?: HookContext[]
+}
+```
+
+`InjectOptions` accepts ordinary message attribution and durable model-hidden JSON metadata. Attached contexts belong only to queued or steering input, so synthetic injection cannot accept them:
 
 ```ts type-equiv
 /** Options specific to durable synthetic context injection. */
-interface InjectOptions extends SendOptions {
+interface InjectOptions extends Omit<SendOptions, 'contexts'> {
   /** Opaque JSON state retained in the session event but hidden from the model. */
   meta?: JsonValue
 }
@@ -390,7 +407,8 @@ interface Agent {
    * Queue one detached, frozen lossless-JSON item. If claimed, it is the sole
    * ordinary message in its FIFO-ordered turn; the next claimed item waits for
    * that turn's checkpoint.
-   * Invalid input throws synchronously before notification or enqueue.
+   * Attached contexts share the same snapshot and ownership boundary. Invalid
+   * input throws synchronously before notification or enqueue.
    */
   send(content: ContentBlock[], options?: SendOptions): void
 
@@ -442,15 +460,21 @@ The process-local initiator carried by `ctx.agents` is the exact `Agent` above,
 
 ## Interception decisions
 
-Each `agent/*` interception waterfall returns a small, seam-specific typed union — the unified Decision idiom (the tool seams' `PreToolDecision`/`PostToolDecision` in [tools.md](tools.md) follow the same shape). A CC/Codex hook bridge maps its `permissionDecision`/`decision`/`continue`/`additionalContext` fields onto these; a native plugin returns them directly. Prompt and post-tool decisions share one model-facing context shape, `HookContext`, which is `inject()`ed as a `context/message` and therefore carries a REQUIRED `source` (a missing source would default to `{kind:'user'}` and mislabel plugin context as a user prompt). Its `content` reaches the model verbatim as a user-role message, while JSON `meta` persists plugin state without exposing it to the model. Both decisions carry `additionalContexts[]` so every entry preserves its own provenance and metadata. Continuation reasons are steering messages instead and deliberately use the narrower content/source shape.
+Each `agent/*` interception waterfall returns a small, seam-specific typed union — the unified Decision idiom (the tool seams' `PreToolDecision`/`PostToolDecision` in [tools.md](tools.md) follow the same shape). A CC/Codex hook bridge maps its `permissionDecision`/`decision`/`continue`/`additionalContext` fields onto these; a native plugin returns them directly. Prompt and post-tool decisions share one model-facing context shape, `HookContext`, which carries a REQUIRED `source` (a missing source would default to `{kind:'user'}` and mislabel plugin context as a user prompt). Its `content` reaches the model verbatim as user-role input, while JSON `meta` persists plugin state without exposing it to the model. Absent or `separate` placement becomes `context/message`; `prompt-prefix` placement is available to prompt and steering inbox attachments and bakes the context before the effective request in the same message. Both decisions carry `additionalContexts[]` so every entry preserves its own provenance, metadata, and placement. Continuation reasons are steering messages instead and deliberately use the narrower content/source shape.
 
 Source: [`packages/core/agent/src/types.ts`](../../packages/core/agent/src/types.ts)
 
 ```ts type-equiv
-/** Model-facing context injected by a listener; `source` prevents plugin text from being labeled as user input. */
+/** Model-facing context injected by a listener or atomically attached to one inbox message. */
 interface HookContext {
   content: ContentBlock[]
   source: MessageSource
+  /**
+   * Model placement. Absent or `separate` records an independent
+   * `context/message`; `prompt-prefix` prepends this context and a stable
+   * request delimiter to the same user-role message as its attached prompt.
+   */
+  placement?: 'separate' | 'prompt-prefix'
   /** Opaque JSON state retained in the session event but hidden from the model. */
   meta?: JsonValue
 }
@@ -460,10 +484,13 @@ interface HookContext {
 
 ```ts type-equiv
 /**
- * Prompt interception result. `allow.content` replaces the prompt and each
- * `additionalContexts` entry becomes a separate context message. `block`
- * records a durable `prompt/blocked` and ends the claimed prompt's zero-step
- * turn as rejected.
+ * Prompt interception result. `allow.content` replaces the prompt. Each
+ * `additionalContexts` entry follows its declared placement: separate context
+ * message by default, or a prefix inside the prompt's user-role message.
+ * `block` records a durable `prompt/blocked` and ends the claimed prompt's
+ * zero-step turn as rejected. An `allow` returned by a listener is
+ * authoritative: a listener wrapping `next()` preserves downstream `content`
+ * and `additionalContexts` unless it intentionally replaces them.
  */
 type PromptDecision =
   | { kind: 'allow'; content?: ContentBlock[]; additionalContexts?: HookContext[] }

docs/core-data-structures/pty.md

@@ -0,0 +1,89 @@
+# Persistent PTY Sessions
+
+Types shared by PTY backends, `ctx.pty`, and the model-facing consumer. The [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) owns the rationale; this page records the cross-package vocabulary from [`packages/pty/pty/src/types.ts`](../../packages/pty/pty/src/types.ts).
+
+## Identity and readiness
+
+`PtySessionId` is a service-minted branded id. Optional names are owner-local display metadata; authorization compares the exact owning `Agent`, not a name or guessed id.
+
+`PtyWaitReason` says why one send returned. It is independent from `PtySessionStatus`: silence or timeout may return while the top-level shell remains alive, while `session_exit` means that shell exited rather than an arbitrary foreground child.
+
+```ts type-equiv
+/** Why one interactive send returned control to its caller. */
+type PtyWaitReason = 'stdin_read' | 'inferred_idle' | 'timeout' | 'session_exit'
+```
+
+```ts type-equiv
+/** Top-level PTY process status, independent of a send's wait reason. */
+type PtySessionStatus =
+  | { kind: 'running' }
+  | { kind: 'exited'; exitCode: number | null; signal: NodeJS.Signals | null }
+```
+
+## Backend and live session
+
+A backend owns how one registered type starts and detects readiness. `PtyService` publishes the returned session only after setup succeeds, then owns id authorization and cleanup. A backend session owns terminal state and captured-resource quiescence.
+
+```ts type-equiv
+/** Replaceable provider for one PTY session type. */
+interface PtyBackend {
+  /** Stable type selected by {@link PtySpawnRequest.type}. */
+  readonly type: string
+  /** Create an unpublished session or reject after cleaning partial resources. */
+  spawn(spec: PtyBackendSpawnSpec): Promise<PtyBackendSession>
+}
+```
+
+```ts type-equiv
+/** Backend-owned live session retained by {@link PtyService}. */
+interface PtyBackendSession {
+  /** Initial bounded terminal output returned from `terminal_open`. */
+  readonly motd: string
+  /** Top-level process id when one exists. */
+  readonly pid?: number
+  /** Start one exclusive send operation. */
+  startSend(request: PtySendRequest): PtySendOperation
+  /** Read one bounded page from retained scrollback. */
+  read(request: PtyReadRequest): PtyReadResult
+  /** Signal the verified foreground process group. */
+  signal(signal: PtySignal): Promise<PtySignalResult>
+  /** Observe top-level process status. */
+  status(): PtySessionStatus
+  /** Idempotently close the captured owned process tree and await quiescence. */
+  close(reason: string): Promise<void>
+}
+```
+
+## Send and retained output
+
+One live session accepts one active send. Its operation exposes a consuming output cursor for generic background tasks and one terminal result for a foreground caller. `PtyReadResult` separately pages the bounded session scrollback.
+
+```ts type-equiv
+/** Live backend-owned send; exactly one may be active per PTY session. */
+interface PtySendOperation {
+  /** Resolves after readiness, timeout, cancellation, or top-level process exit. */
+  done: Promise<PtySendResult>
+  /** Consume output produced since the prior call. */
+  readOutput(): PtySendRead
+  /** Request `SIGINT`; returns false after the operation settled. */
+  cancel(): boolean
+}
+```
+
+```ts type-equiv
+/** Settled result for one foreground or background send. */
+interface PtySendResult {
+  /** Bounded rendered terminal delta remaining at settlement. */
+  viewport: string
+  /** Why the wait returned; this does not imply arbitrary child-process exit. */
+  waitReason: PtyWaitReason
+  /** Top-level session status observed at settlement. */
+  sessionStatus: PtySessionStatus
+  /** Whether output was dropped from the operation or retained scrollback. */
+  truncated: boolean
+}
+```
+
+## Ownership and durability
+
+`PtyService` attaches one awaited cleanup to the exact owner scope, rejects foreign operations, and keeps sessions alive across backend or tool-plugin reload. PTY state and raw bytes remain process-local. Model input and bounded returned output are durable through the existing `tool/call`, `tool/result`, and task-result paths rather than duplicate PTY session events.

docs/core-data-structures/session-query.md

@@ -25,6 +25,20 @@ interface SessionRecord {
 }
 ```
 
+`SessionSurfaceSnapshot` is one exact-read observation rather than a retained subscription. Its raw-log boundary and folded events come from the same live-preferred load.
+
+```ts type-equiv
+/** One atomic live-preferred observation of a session's current model surface. */
+interface SessionSurfaceSnapshot {
+  /** Cloned session header selected from the same corpus observation as `events`. */
+  session: SessionHeader
+  /** Highest raw-log seq included in the observation, or `null` for an empty log. */
+  capturedThroughSeq: number | null
+  /** Cloned current surface events in model-history order. */
+  events: SurfaceEvent[]
+}
+```
+
 ```ts type-equiv
 /** Lightweight metadata for one event within a logical session. */
 interface SessionEventRecord {

docs/core-data-structures/session-reference.md

@@ -0,0 +1,65 @@
+# Session References
+
+Structured cross-session reference requests and prepared message contexts. The [package contract](../../packages/context/session-reference) owns canonical URIs, current-surface projection, tag-safe JSON and byte retention, stable errors, and the untrusted model prompt. Host adapters use these types instead of passing their UI mention syntax into the agent core.
+
+Source: [`packages/context/session-reference/src/types.ts`](../../packages/context/session-reference/src/types.ts)
+
+## Inputs and candidates
+
+`SessionReferenceInput` is the host-independent selection. The id is authoritative; the label is display metadata carried into the snapshot.
+
+```ts type-equiv
+/** One source session selected by a host. */
+interface SessionReferenceInput {
+  /** Opaque source session identity. */
+  sessionId: SessionId
+  /** Optional user-facing mention label. */
+  label?: string
+}
+```
+
+`SessionReferenceCandidate` is host-facing discovery output. Its label uses the latest session title when present, while filtering still searches only session id and cwd and never transcript text.
+
+```ts type-equiv
+/** One host-facing candidate from exact session metadata. */
+interface SessionReferenceCandidate {
+  /** Opaque source session identity. */
+  sessionId: SessionId
+  /** Latest log-backed title, falling back to the opaque session id. */
+  label: string
+  /** Source session working directory, when recorded. */
+  cwd?: string
+  /** Source session creation time in Unix epoch milliseconds. */
+  createdAt: number
+}
+```
+
+## Prepared messages
+
+Preparation preserves readable current-message content and returns at most one aggregated context. The host binds `contexts` to that exact `send()` or `steer()` call.
+
+```ts type-equiv
+/** Message payload and the zero-or-one durable snapshot contexts bound to it. */
+interface PreparedReferencedMessage {
+  /** Readable message content after host mention tokens are removed. */
+  content: ContentBlock[]
+  /** Empty without references; otherwise one aggregated untrusted context. */
+  contexts: HookContext[]
+}
+```
+
+## Errors
+
+`SessionReferenceError.code` separates invalid configuration or input, self-reference, count limits, source-read failure, budget failure, and cancellation. Host protocols map these codes to their own error envelopes without inspecting prompt bytes.
+
+```ts type-equiv
+/** Stable failure codes exposed to host adapters. */
+type SessionReferenceErrorCode =
+  | 'SESSION_REFERENCE_INVALID_CONFIG'
+  | 'SESSION_REFERENCE_INVALID_REFERENCE'
+  | 'SESSION_REFERENCE_SELF_REFERENCE'
+  | 'SESSION_REFERENCE_TOO_MANY'
+  | 'SESSION_REFERENCE_READ_FAILED'
+  | 'SESSION_REFERENCE_BUDGET_EXCEEDED'
+  | 'SESSION_REFERENCE_CANCELLED'
+```

docs/core-data-structures/session.md

@@ -8,6 +8,18 @@ Source: [`packages/core/session/src/types.ts`](../../packages/core/session/src/t
 
 The append-only event types. Merge-extensible: a plugin declares extra event types via declaration merging — e.g. the [compaction seam](compaction.md) adds `compact/start` / `compact/summary` / `compact/end`, and `@deepseek-ai/dsh-hook-protocol` adds log-only `hook/invoked` / `hook/result` provenance for a hook bridge. Like `compact/*`, these are NOT `SurfaceEventType`s (no `surfaceOp`). The generated [persistence log event catalog](../persistence-catalog.md) enumerates every member — core and merged — with its payload, surface badge, and declaration site.
 
+```ts type-equiv
+/** Shared payload for ordinary and steering prompt messages. */
+interface PromptMessageData {
+  /** Exact model-facing blocks, including any baked prompt-prefix contexts. */
+  content: ContentBlock[]
+  /** Producer provenance for the direct prompt. */
+  source: MessageSource
+  /** Present only when prompt-prefix contexts were baked into `content`. */
+  envelope?: PromptMessageEnvelope
+}
+```
+
 ```ts type-equiv
 /**
  * The merge-extensible, append-only source of truth for an agent interaction.
@@ -35,7 +47,7 @@ interface SessionEventMap {
   /** Closes step `step` of turn `turn`. */
   'step/end': { turn: number; step: number }
   /** A user-visible prompt (the queued message claimed for this turn). */
-  'user/message': { content: ContentBlock[]; source: MessageSource }
+  'user/message': PromptMessageData
   /**
    * Durable record of a prompt veto and its reason. It is log-only: the blocked
    * prompt never enters the model-visible surface, and its turn runs zero steps.
@@ -93,7 +105,7 @@ interface SessionEventMap {
     meta?: JsonValue
   }
   /** Steering content injected between steps of a running turn. */
-  'steering/message': { turn: number; content: ContentBlock[]; source: MessageSource }
+  'steering/message': PromptMessageData & { turn: number }
   /** Whole-list snapshot; latest write wins on replay. Log-only UI state; never derived history. */
   'todo/write': { todos: TodoItem[] }
   /**
@@ -104,6 +116,8 @@ interface SessionEventMap {
 }
 ```
 
+`PromptMessageData.content` is always the exact model-facing content. When attached context declares `prompt-prefix` placement, AgentLoop concatenates its blocks, a `## My request:` delimiter, and the effective direct prompt into that array. The optional model-hidden `envelope` retains `displayContent` plus ordered prefix-context source/metadata descriptors, so transcript, title, and re-reference consumers can present the human prompt without changing reconstructable history. `displayPromptContent()` performs that selection and falls back to `content` for ordinary and older events.
+
 ### `OutOfBandSessionEventMap` — narrow late-append opt-in
 
 `SessionEventMap` membership alone does not authorize an event outside the agent loop's ordinary lifecycle. An event owner declaration-merges the same key into this empty marker map before `ctx.sessions.appendOutOfBand()` accepts it; the derived type additionally excludes every surface event. An accepted update joins an open turn or receives a balanced, flushed zero-step turn.
@@ -448,11 +462,11 @@ declare class Session {
 
 `Session.deriveMessages()` projects the event log into the `Message[]` the model sees — cached (each surface node projected once, when first seen; a surface rewrite rebuilds) and frozen (a fresh array per call over shared, deep-frozen messages, so mutating logged history through a projection is unrepresentable). `deriveEventMessage(event)` is the per-node pure function the fold applies — public so external reconstructors and the dev invariant project a log prefix with exactly the same rules and cannot disagree with the cache. The projection rules:
 
-- `user/message` → a user message.
+- `user/message` → a user message carrying exact `content`; an optional envelope remains log-only display metadata.
 - `assistant/message` → an assistant message with the event's provider/model provenance and optional adapter-private replay state. Raw `assistant/chunk` events are replay/UI data and are **skipped** in derivation (the assembled message is authoritative). An **empty-content** `assistant/message` is also skipped — a max-tokens step cut off with no content still records an `assistant/message` to host its usage/provenance, but a content-less assistant turn must not enter the provider transcript.
 - `tool/result` → a user message carrying a `tool-result` block.
 - `context/message` → a user-role message carrying its `content` verbatim at its chronological position. Optional JSON `meta` remains in the event log and is never rendered.
-- `steering/message` → a user-role message carrying its content verbatim at its chronological position.
+- `steering/message` → a user-role message carrying exact `content` at its chronological position; an optional envelope remains log-only display metadata.
 
 Everything else (`turn/*`, `step/*`, plugin-owned `llm/retry`) is structural and does not project into a message. Token accounting reads per-step `assistant/chunk { type: 'usage' }` records and treats `assistant/message.usage` as the committed-step fallback when no usage chunk exists; failed model-request attempts have no assistant message, so their usage chunk is the durable accounting record. An operational error's step number is on `turn/end.reason` for `kind: 'error'`, with normalized `LlmFailure` facts for a final model-request failure and message/code for other live errors. Because this unreleased format intentionally has no compatibility promise, seed/load validation rejects request headers without provider+model and assistant messages without provider/model provenance instead of guessing a route for historical data.