diff --git a/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.i18n.yaml b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.i18n.yaml index 2282e1dace..1832263d0f 100644 --- a/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md -2026-07-31-code-mode-language-dispatch.md: bc56736c1582b89b4c16b76c49762eeaf0c3fc39 -2026-07-31-code-mode-language-dispatch.zh.md: 9a224cbda75ce530f18498a8e1b0ca42ab540ee1 +2026-07-31-code-mode-language-dispatch.md: 52ec905b871d4a4954e1b33d3422a797307b75bc +2026-07-31-code-mode-language-dispatch.zh.md: 94361744fbfcd7b7fb5d6bc94e3da3aae3403aeb diff --git a/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md index bc56736c15..52ec905b87 100644 --- a/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md +++ b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md @@ -43,4 +43,4 @@ The cost is that the Python branch of both tables is unreachable on this base: ` Two runtime contracts the Python SDK text asserts are owed by that same backend PR. First, the instructions tell the model that exactly `tools` and `ToolCallError` are bound and that the declared `TypedDict` classes are not, so the backend must inject those two names — with `ToolCallError.toolName` populated per the seam's `errorClass` contract — and must NOT bind the declared class names into the program's globals; injecting them "helpfully" would make the SDK text false. Second, the language has to be bound to the request: `requireCodeRuntime` resolves `ctx.codeRuntime` separately at assembly and at `run_code` execution, so a reload that swapped the runtime between those two points would hand a program written against one flavor to the other. The split is finer than those two points — `run_code`'s `description` and `parameters` getters each call `resolveFlavor(peekRuntime())`, and `schemaOf` destructures both, so one projection reads the runtime twice; both reads are for `run_code`'s own schema, since the getters are installed on that one definition and every other definition carries plain data properties. A reload between those two reads yields a single schema whose two halves name different languages. Neither is reachable here — one published backend means both reads return the same flavor and no program ever runs against this renderer's output — and the cross-language rejection is not testable until a second language exists. -Third, that PR owns the CPython floor, and with it the Unicode-table skew in `isBareIdentifier`. This renderer decides whether a field or tool name can be emitted bare using the running engine's `\p{XID_Start}`/`\p{XID_Continue}` tables (Node 22.23.1: Unicode 17.0), while the interpreter uses its own (CPython 3.9.6: 13.0.0). An interpreter older than the engine is the failing direction: a character added to `XID_Start` in between is emitted bare and its tokenizer refuses the whole block. The exposure window is exactly the characters added between the two versions, so the PR that names a supported CPython range must decide explicitly between accepting it and tightening the predicate against pinned tables for that floor. Nothing here can decide it: the floor does not exist yet, and a table pinned to a guess would be a deployment-varying constant with no configurability behind it. +Third, that PR owns the CPython floor, and with it the renderer's Unicode-table skew. Three regexes read the running engine's tables (Node 22.23.1: Unicode 17.0) while the interpreter uses its own (CPython 3.9.6: 13.0.0): `isBareIdentifier`'s `IDENTIFIER`, and `camelCase`'s split set and head test. An interpreter older than the engine is the failing direction — a character added to `XID_Start` or `XID_Continue` in between is emitted and its tokenizer refuses the whole block — and it arrives by two independent paths. Through the predicate, a bare method or field name. Through `camelCase`, a class name, which is emitted for every tool including one the predicate rejected: `zz-` plus U+1E4D0 never reaches the predicate's skew, since the `-` rejects it outright, yet it still declares `class Zz𞓐xArgs`. The exposure window is exactly the characters added between the two versions, so the PR that names a supported CPython range must decide explicitly between accepting it and pinning all three read points to tables for that floor — pinning the predicate alone leaves the class-name path open. Nothing here can decide it: the floor does not exist yet, and a table pinned to a guess would be a deployment-varying constant with no configurability behind it. diff --git a/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.zh.md b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.zh.md index 9a224cbda7..94361744fb 100644 --- a/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.zh.md +++ b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.zh.md @@ -43,4 +43,4 @@ Code Mode 只生成一种 SDK 形态:TypeScript。`ToolRegistry` 为 `tools:sd Python SDK 文本断言的两条运行时契约同样归属那个 backend PR。其一,说明文字告诉模型运行时恰好绑定 `tools` 与 `ToolCallError` 两个名字、所声明的 `TypedDict` 类不绑定,因此后端必须注入这两个名字(并按 seam 的 `errorClass` 契约填充 `ToolCallError.toolName`),且**不得**把所声明的类名绑进程序全局——「好心」注入会使这段 SDK 文本变成假话。其二,语言必须绑定到请求上:`requireCodeRuntime` 在组装时与 `run_code` 执行时分别解析 `ctx.codeRuntime`,若在这两点之间发生重载并换掉运行时,就会把针对一种形态写成的程序交给另一种形态执行。分裂比这两点更细——`run_code` 的 `description` 与 `parameters` 两个 getter 各自调用 `resolveFlavor(peekRuntime())`,而 `schemaOf` 会解构这两个字段,因此一次投影读两次运行时;两次都属于 `run_code` 自己的 schema,因为这两个 getter 只装在那一个 definition 上,其余 definition 携带的都是普通数据属性。在这两次读取之间重载会产出单个 schema 的两半分属不同语言。两者在此处都不可达——只有一个已发布后端意味着两次读取返回同一形态,且没有任何程序会针对本渲染器的输出运行——而跨语言拒绝在第二门语言存在之前也无法测试。 -其三,那个 PR 拥有 CPython 版本下限,连带拥有 `isBareIdentifier` 里的 Unicode 表偏斜。本渲染器用所运行引擎的 `\p{XID_Start}`/`\p{XID_Continue}` 表(Node 22.23.1:Unicode 17.0)决定某个字段名或工具名能否裸发,而解释器用它自己的表(CPython 3.9.6:13.0.0)。解释器旧于引擎是会失败的那个方向:在两者之间被加进 `XID_Start` 的字符会被裸发,其 tokenizer 拒收,整个块随之不可解析。暴露窗口恰是两个版本之间新增的那些字符,所以宣布支持某个 CPython 范围的那个 PR 必须在「接受该暴露」与「按该下限的固定表收紧判据」之间显式作出决定。此处无法决定:下限尚不存在,而按猜测钉死一张表会成为一个随部署而变、却没有可配置性支撑的常量。 +其三,那个 PR 拥有 CPython 版本下限,连带拥有本渲染器的 Unicode 表偏斜。有三个正则读所运行引擎的表(Node 22.23.1:Unicode 17.0),而解释器用它自己的表(CPython 3.9.6:13.0.0):`isBareIdentifier` 的 `IDENTIFIER`,以及 `camelCase` 的切分集与头部测试。解释器旧于引擎是会失败的那个方向——在两者之间被加进 `XID_Start` 或 `XID_Continue` 的字符会被发出,其 tokenizer 拒收,整个块随之不可解析——而它经两条独立路径抵达。经判据抵达的是裸发的方法名或字段名。经 `camelCase` 抵达的是类名,而类名对每个工具都发出,包括被判据拒绝的那些:工具名 `zz-` 加 U+1E4D0 因 `-` 被判据直接拒绝、从不触及那里的偏斜,却照样声明 `class Zz𞓐xArgs`。暴露窗口恰是两个版本之间新增的那些字符,所以宣布支持某个 CPython 范围的那个 PR 必须在「接受该暴露」与「按该下限的表钉住全部三个读取点」之间显式作出决定——只钉判据会留下类名那条路径。此处无法决定:下限尚不存在,而按猜测钉死一张表会成为一个随部署而变、却没有可配置性支撑的常量。 diff --git a/packages/core/tools/src/py-types.ts b/packages/core/tools/src/py-types.ts index 9ed7d75d4d..c50ba657c1 100644 --- a/packages/core/tools/src/py-types.ts +++ b/packages/core/tools/src/py-types.ts @@ -5,11 +5,12 @@ * store, keyed by the loaded {@link @deepseek-ai/dsh-code-runtime#CodeRuntime.language | code * runtime's language}. * - * In Code Mode the native tool schemas are omitted from the request, so this generated SDK is - * the model's ONLY source for each tool's argument names, required fields, types, descriptions, - * and canonical output shapes. Object-shaped arguments and outputs therefore render as one named - * `TypedDict` per tool (and per nested object), not an opaque `dict[str, Any]`, so the shape - * survives into the program. + * Under `mode: 'code'` the native tool schemas are omitted from the request, so this generated + * SDK is the model's ONLY source for each tool's argument names, required fields, types, + * descriptions, and canonical output shapes; under `mode: 'both'` the native schemas ship + * alongside it and it is one of two. Object-shaped arguments and outputs therefore render as one + * named `TypedDict` per tool (and per nested object), not an opaque `dict[str, Any]`, so the + * shape survives into the program under the mode that has nothing else to carry it. * @module @deepseek-ai/dsh-tools/src/py-types */ @@ -30,8 +31,8 @@ const IDENTIFIER = /^[\p{XID_Start}_]\p{XID_Continue}*$/u * * Python identifiers are not ASCII: `路径` is as legal a field name as `path`, * and rejecting it would degrade the whole enclosing object, dropping every - * field's name, requiredness, and type — and in Code Mode the native schemas - * are omitted, so this text is the model's only source for them. + * field's name, requiredness, and type — which under `mode: 'code'` is the + * model's only source for them. * * NFKC stability is a second and separate condition, because CPython * normalizes identifiers at compile time while JSON keys are compared as @@ -40,23 +41,37 @@ const IDENTIFIER = /^[\p{XID_Start}_]\p{XID_Continue}*$/u * that normalize together would collapse into one declaration. Those names * take the subscript path, which carries their exact bytes. * - * Both conditions are evaluated against the ENGINE's Unicode tables, and the - * two sides are versioned independently — `\p{XID_Start}` follows the running - * engine (Node 22.23.1 reports Unicode 17.0) while CPython follows its own - * (3.9.6 reports 13.0.0). The skew is not symmetric. A CPython older than the - * engine is the dangerous direction: a character added to `XID_Start` since its - * tables (U+1C89, U+10570, U+1E290, U+1E4D0 are all NFKC-stable and accepted - * here, and all rejected by that 3.9.6) is emitted bare and its tokenizer - * refuses the character, taking the whole SDK block down — the same - * parseability invariant {@link UNPRINTABLE}, {@link LONE_SURROGATE} and - * {@link MAX_LIST_NESTING} exist for. A CPython newer than the engine only - * routes a legal name to the subscript path: less readable, still correct. The - * NFKC condition reduces to the same skew, since normalization stability - * guarantees an assigned character's normalization never changes afterwards. + * The equivalence to `str.isidentifier()` was measured across 21 samples with + * zero divergence, on Node 22.23.1 against CPython 3.9.6 — the halves the two + * conditions are proxies for, both tested by that run. * - * Closing the exposure needs the target interpreter's version, which the - * backend reporting `language: 'python'` owns and which is unpublished on this - * base; the note records it as that PR's decision. + * Both conditions are evaluated against the ENGINE's Unicode tables, and the + * two sides are versioned independently — `\p{XID_Start}`/`\p{XID_Continue}` + * follow the running engine (Node 22.23.1 reports Unicode 17.0) while CPython + * follows its own (3.9.6 reports 13.0.0). The skew is not symmetric. A CPython + * older than the engine is the dangerous direction: a character added to + * either property since its tables (U+1C89, U+10570, U+1E290, U+1E4D0 are all + * NFKC-stable and accepted here, and all rejected by that 3.9.6) is emitted + * bare and its tokenizer refuses the character, taking the whole SDK block + * down — the same parseability invariant {@link UNPRINTABLE}, + * {@link LONE_SURROGATE} and {@link MAX_LIST_NESTING} exist for. Both + * properties carry it: a character added only to `XID_Continue` passes the + * trailing `\p{XID_Continue}*` in a tail position and fails the same way. A + * CPython newer than the engine only routes a legal name to the + * subscript/`dict[str, Any]` path: less readable, still correct. The NFKC + * condition reduces to the same skew, since normalization stability guarantees + * an assigned character's normalization never changes afterwards. + * + * This predicate is not the only reader of those tables. {@link camelCase} + * reads them too, through its split set and its head test, and its output is + * emitted for EVERY tool — including one this predicate rejected, whose + * `TypedDict` is still declared and named. A tool named `zz-\u{1E4D0}x` never + * reaches the skew here (the `-` rejects it outright) yet emits + * `class Zz\u{1E4D0}xArgs`, which that same 3.9.6 refuses. Closing the + * exposure therefore covers all three read points, not this predicate alone; + * it needs the target interpreter's version, which the backend reporting + * `language: 'python'` owns and which is unpublished on this base, so the note + * records it as that PR's decision. * * The `ts-types` sibling keeps its own ASCII rule rather than sharing this * one: ECMAScript identifiers are a different set (`$`, ZWJ/ZWNJ) and are @@ -221,6 +236,11 @@ function docLines(description: unknown, indent: number): string[] { * a combining-mark head composes there (`U+0301` gives `Tooĺ`, U+013A), so * normalizing only the un-prefixed part would emit a name CPython compiles to * a different symbol. The second call is idempotent on the un-prefixed arm. + * + * The split set and the head test read the engine's Unicode tables, so this + * function carries the same version skew {@link isBareIdentifier} documents, + * by an independent path: a class name derived here is emitted for every tool, + * including one the predicate rejected. * @param raw - the schema field or tool name to derive from. * @returns a class-name segment safe to emit. */ diff --git a/packages/core/tools/tests/py-types.spec.ts b/packages/core/tools/tests/py-types.spec.ts index 2b73ec5795..a5e2f660f2 100644 --- a/packages/core/tools/tests/py-types.spec.ts +++ b/packages/core/tools/tests/py-types.spec.ts @@ -453,8 +453,11 @@ describe('renderToolsSdkPy', () => { // ligature name cannot, because `async def find` would define `find`. The // subscript comment quotes the name, so its exact bytes survive, and its // TypedDict is still named and referenced — the name is only unusable as a - // method, not as a class-name source (`camelCase` normalizes what it - // derives, since a generated name is never matched against a JSON key). + // method, not as a class-name source. The `FInd` spelling comes from `fi`'s + // multi-character full case mapping (`'fi'.toUpperCase()` is `'FI'`), not + // from `camelCase`'s NFKC step, which is the identity on `FInd`: the + // ligature is XID_Start, so the split set keeps it and only the + // capitalization of the head transforms it. const of = (name: string): ToolSdkSchema => ({ name, description: `Tool ${name}.`, @@ -559,6 +562,64 @@ describe('renderToolsSdkPy', () => { expect(text).toContain('class XArgs\uAC00\u1100(TypedDict):') }) + it('routes a fold collision through the counter that raw-byte dedup would miss', () => { + // The other half of the `childClassName` normalization: two joins that are + // byte-distinct before NFKC and identical after. Field `\uAC00` allocates + // `XArgs\uAC00`; the sibling `\u1100` allocates `XArgs\u1100`, and ITS child + // `\u1161` joins to `XArgs\u1100\u1161` — the same `XArgs\uAC00` once composed. + // Normalizing at the join is what lets `usedClassNames`, which dedupes by raw + // bytes, see the collision at all; unnormalized, both would be declared and + // CPython would compile the second as a shadow of the first. + const text = renderToolsSdkPy([ + { + name: 'x', + description: 'Colliding jamo joins.', + parameters: { + type: 'object', + additionalProperties: false, + required: ['\uAC00', '\u1100'], + properties: { + '\uAC00': { type: 'object', additionalProperties: false, required: ['q'], properties: { q: { type: 'string' } } }, + '\u1100': { + type: 'object', + additionalProperties: false, + required: ['\u1161'], + properties: { + '\u1161': { type: 'object', additionalProperties: false, required: ['q'], properties: { q: { type: 'string' } } }, + }, + }, + }, + }, + output: { type: 'string' }, + }, + ]) + expect(text).toContain('class XArgs\uAC00(TypedDict):') + expect(text).toContain('class XArgs\uAC002(TypedDict):') + expect(text).toContain(' \u1161: XArgs\uAC002') + }) + + it('names both branches of a oneOf of objects on the argument side', () => { + // The output side is pinned elsewhere; arguments reach the same + // `childClassName(frame.className, index + 1)` path, and the annotation is + // the union of the two derived names rather than a degraded dict. + const text = renderToolsSdkPy([ + { + name: 'x', + description: 'Union arguments.', + parameters: { + oneOf: [ + { type: 'object', additionalProperties: false, required: ['a'], properties: { a: { type: 'string' } } }, + { type: 'object', additionalProperties: false, required: ['b'], properties: { b: { type: 'number' } } }, + ], + }, + output: { type: 'string' }, + }, + ]) + expect(text).toContain('class XArgs1(TypedDict):') + expect(text).toContain('class XArgs2(TypedDict):') + expect(text).toContain('async def x(self, args: XArgs1 | XArgs2) -> str:') + }) + it('declares a closed empty object with omitted properties as an empty TypedDict, not dict[str, Any]', () => { // `{ type: 'object', additionalProperties: false }` with no `properties` // is a closed empty object — no key accepted — exactly as the validator