fix(tools): widen the Unicode-skew obligation past isBareIdentifier

The predicate is not the only reader of the engine's XID tables. camelCase
reads them through its split set and its head test, and the class name it
derives is emitted for EVERY tool -- including one the predicate rejected,
whose TypedDict is still declared and named. A tool named `zz-` + U+1E4D0
never reaches the skew in the predicate, since the `-` rejects it outright,
yet still emits `class Zz<U+1E4D0>xArgs`, which CPython 3.9.6 refuses the
same way. A backend PR executing "pin the predicate against tables for the
floor" literally would leave that path open, so the note and the docstring
now name all three read points.

Two corrections in the same paragraph. The failing direction is a character
added to XID_Start OR XID_Continue -- one added only to the latter passes
the trailing `\p{XID_Continue}*` in a tail position and fails identically.
And the safe direction routes a name to the subscript/`dict[str, Any]`
path: a rejected FIELD name degrades its whole enclosing object rather than
just itself, which the predicate's opening paragraph already said.

Also qualify the module header's "ONLY source" claim, which holds under
`mode: 'code'` but not `both`, where wireSchemas ships every native schema
alongside the SDK section; record the measured str.isidentifier()
equivalence (21 samples, zero divergence, Node 22.23.1 vs CPython 3.9.6)
where the versions it is relative to already live; and attribute the
`FInd` spelling in the ligature test to full case mapping rather than to
the NFKC step, which is the identity there.

Two tests. The fold-collision half of the childClassName fix: sibling joins
that are byte-distinct before NFKC and equal after, so `usedClassNames`
dedupes by raw bytes and the counter only sees the collision because the
join is normalized. And the argument-side oneOf-of-objects branch naming,
which reaches the same childClassName path the output side already pins.
This commit is contained in:
Chinesezjc
2026-08-05 21:08:24 +08:00
parent af51e68875
commit 2914a87eda
5 changed files with 110 additions and 29 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md
2026-07-31-code-mode-language-dispatch.md: bc56736c1582b89b4c16b76c49762eeaf0c3fc39
2026-07-31-code-mode-language-dispatch.zh.md: 9a224cbda75ce530f18498a8e1b0ca42ab540ee1
2026-07-31-code-mode-language-dispatch.md: 52ec905b871d4a4954e1b33d3422a797307b75bc
2026-07-31-code-mode-language-dispatch.zh.md: 94361744fbfcd7b7fb5d6bc94e3da3aae3403aeb

View File

@@ -43,4 +43,4 @@ The cost is that the Python branch of both tables is unreachable on this base: `
Two runtime contracts the Python SDK text asserts are owed by that same backend PR. First, the instructions tell the model that exactly `tools` and `ToolCallError` are bound and that the declared `TypedDict` classes are not, so the backend must inject those two names — with `ToolCallError.toolName` populated per the seam's `errorClass` contract — and must NOT bind the declared class names into the program's globals; injecting them "helpfully" would make the SDK text false. Second, the language has to be bound to the request: `requireCodeRuntime` resolves `ctx.codeRuntime` separately at assembly and at `run_code` execution, so a reload that swapped the runtime between those two points would hand a program written against one flavor to the other. The split is finer than those two points — `run_code`'s `description` and `parameters` getters each call `resolveFlavor(peekRuntime())`, and `schemaOf` destructures both, so one projection reads the runtime twice; both reads are for `run_code`'s own schema, since the getters are installed on that one definition and every other definition carries plain data properties. A reload between those two reads yields a single schema whose two halves name different languages. Neither is reachable here — one published backend means both reads return the same flavor and no program ever runs against this renderer's output — and the cross-language rejection is not testable until a second language exists.
Third, that PR owns the CPython floor, and with it the Unicode-table skew in `isBareIdentifier`. This renderer decides whether a field or tool name can be emitted bare using the running engine's `\p{XID_Start}`/`\p{XID_Continue}` tables (Node 22.23.1: Unicode 17.0), while the interpreter uses its own (CPython 3.9.6: 13.0.0). An interpreter older than the engine is the failing direction: a character added to `XID_Start` in between is emitted bare and its tokenizer refuses the whole block. The exposure window is exactly the characters added between the two versions, so the PR that names a supported CPython range must decide explicitly between accepting it and tightening the predicate against pinned tables for that floor. Nothing here can decide it: the floor does not exist yet, and a table pinned to a guess would be a deployment-varying constant with no configurability behind it.
Third, that PR owns the CPython floor, and with it the renderer's Unicode-table skew. Three regexes read the running engine's tables (Node 22.23.1: Unicode 17.0) while the interpreter uses its own (CPython 3.9.6: 13.0.0): `isBareIdentifier`'s `IDENTIFIER`, and `camelCase`'s split set and head test. An interpreter older than the engine is the failing direction a character added to `XID_Start` or `XID_Continue` in between is emitted and its tokenizer refuses the whole block — and it arrives by two independent paths. Through the predicate, a bare method or field name. Through `camelCase`, a class name, which is emitted for every tool including one the predicate rejected: `zz-` plus U+1E4D0 never reaches the predicate's skew, since the `-` rejects it outright, yet it still declares `class Zz𞓐xArgs`. The exposure window is exactly the characters added between the two versions, so the PR that names a supported CPython range must decide explicitly between accepting it and pinning all three read points to tables for that floor — pinning the predicate alone leaves the class-name path open. Nothing here can decide it: the floor does not exist yet, and a table pinned to a guess would be a deployment-varying constant with no configurability behind it.

View File

@@ -43,4 +43,4 @@ Code Mode 只生成一种 SDK 形态TypeScript。`ToolRegistry` 为 `tools:sd
Python SDK 文本断言的两条运行时契约同样归属那个 backend PR。其一说明文字告诉模型运行时恰好绑定 `tools``ToolCallError` 两个名字、所声明的 `TypedDict` 类不绑定,因此后端必须注入这两个名字(并按 seam 的 `errorClass` 契约填充 `ToolCallError.toolName`),且**不得**把所声明的类名绑进程序全局——「好心」注入会使这段 SDK 文本变成假话。其二,语言必须绑定到请求上:`requireCodeRuntime` 在组装时与 `run_code` 执行时分别解析 `ctx.codeRuntime`,若在这两点之间发生重载并换掉运行时,就会把针对一种形态写成的程序交给另一种形态执行。分裂比这两点更细——`run_code``description``parameters` 两个 getter 各自调用 `resolveFlavor(peekRuntime())`,而 `schemaOf` 会解构这两个字段,因此一次投影读两次运行时;两次都属于 `run_code` 自己的 schema因为这两个 getter 只装在那一个 definition 上,其余 definition 携带的都是普通数据属性。在这两次读取之间重载会产出单个 schema 的两半分属不同语言。两者在此处都不可达——只有一个已发布后端意味着两次读取返回同一形态,且没有任何程序会针对本渲染器的输出运行——而跨语言拒绝在第二门语言存在之前也无法测试。
其三,那个 PR 拥有 CPython 版本下限,连带拥有 `isBareIdentifier` 里的 Unicode 表偏斜。本渲染器用所运行引擎的 `\p{XID_Start}`/`\p{XID_Continue}` Node 22.23.1Unicode 17.0决定某个字段名或工具名能否裸发而解释器用它自己的表CPython 3.9.613.0.0)。解释器旧于引擎是会失败的那个方向在两者之间被加进 `XID_Start` 的字符会被发,其 tokenizer 拒收,整个块随之不可解析。暴露窗口恰是两个版本之间新增的那些字符,所以宣布支持某个 CPython 范围的那个 PR 必须在「接受该暴露」与「按该下限的固定表收紧判据」之间显式作出决定。此处无法决定:下限尚不存在,而按猜测钉死一张表会成为一个随部署而变、却没有可配置性支撑的常量。
其三,那个 PR 拥有 CPython 版本下限,连带拥有本渲染器的 Unicode 表偏斜。有三个正则读所运行引擎的Node 22.23.1Unicode 17.0而解释器用它自己的表CPython 3.9.613.0.0`isBareIdentifier``IDENTIFIER`,以及 `camelCase` 的切分集与头部测试。解释器旧于引擎是会失败的那个方向——在两者之间被加进 `XID_Start``XID_Continue` 的字符会被发,其 tokenizer 拒收,整个块随之不可解析——而它经两条独立路径抵达。经判据抵达的是裸发的方法名或字段名。经 `camelCase` 抵达的是类名,而类名对每个工具都发出,包括被判据拒绝的那些:工具名 `zz-` 加 U+1E4D0 因 `-` 被判据直接拒绝、从不触及那里的偏斜,却照样声明 `class Zz𞓐xArgs`。暴露窗口恰是两个版本之间新增的那些字符,所以宣布支持某个 CPython 范围的那个 PR 必须在「接受该暴露」与「按该下限的表钉住全部三个读取点」之间显式作出决定——只钉判据会留下类名那条路径。此处无法决定:下限尚不存在,而按猜测钉死一张表会成为一个随部署而变、却没有可配置性支撑的常量。

View File

@@ -5,11 +5,12 @@
* store, keyed by the loaded {@link @deepseek-ai/dsh-code-runtime#CodeRuntime.language | code
* runtime's language}.
*
* In Code Mode the native tool schemas are omitted from the request, so this generated SDK is
* the model's ONLY source for each tool's argument names, required fields, types, descriptions,
* and canonical output shapes. Object-shaped arguments and outputs therefore render as one named
* `TypedDict` per tool (and per nested object), not an opaque `dict[str, Any]`, so the shape
* survives into the program.
* Under `mode: 'code'` the native tool schemas are omitted from the request, so this generated
* SDK is the model's ONLY source for each tool's argument names, required fields, types,
* descriptions, and canonical output shapes; under `mode: 'both'` the native schemas ship
* alongside it and it is one of two. Object-shaped arguments and outputs therefore render as one
* named `TypedDict` per tool (and per nested object), not an opaque `dict[str, Any]`, so the
* shape survives into the program under the mode that has nothing else to carry it.
* @module @deepseek-ai/dsh-tools/src/py-types
*/
@@ -30,8 +31,8 @@ const IDENTIFIER = /^[\p{XID_Start}_]\p{XID_Continue}*$/u
*
* Python identifiers are not ASCII: `路径` is as legal a field name as `path`,
* and rejecting it would degrade the whole enclosing object, dropping every
* field's name, requiredness, and type — and in Code Mode the native schemas
* are omitted, so this text is the model's only source for them.
* field's name, requiredness, and type — which under `mode: 'code'` is the
* model's only source for them.
*
* NFKC stability is a second and separate condition, because CPython
* normalizes identifiers at compile time while JSON keys are compared as
@@ -40,23 +41,37 @@ const IDENTIFIER = /^[\p{XID_Start}_]\p{XID_Continue}*$/u
* that normalize together would collapse into one declaration. Those names
* take the subscript path, which carries their exact bytes.
*
* Both conditions are evaluated against the ENGINE's Unicode tables, and the
* two sides are versioned independently — `\p{XID_Start}` follows the running
* engine (Node 22.23.1 reports Unicode 17.0) while CPython follows its own
* (3.9.6 reports 13.0.0). The skew is not symmetric. A CPython older than the
* engine is the dangerous direction: a character added to `XID_Start` since its
* tables (U+1C89, U+10570, U+1E290, U+1E4D0 are all NFKC-stable and accepted
* here, and all rejected by that 3.9.6) is emitted bare and its tokenizer
* refuses the character, taking the whole SDK block down — the same
* parseability invariant {@link UNPRINTABLE}, {@link LONE_SURROGATE} and
* {@link MAX_LIST_NESTING} exist for. A CPython newer than the engine only
* routes a legal name to the subscript path: less readable, still correct. The
* NFKC condition reduces to the same skew, since normalization stability
* guarantees an assigned character's normalization never changes afterwards.
* The equivalence to `str.isidentifier()` was measured across 21 samples with
* zero divergence, on Node 22.23.1 against CPython 3.9.6 — the halves the two
* conditions are proxies for, both tested by that run.
*
* Closing the exposure needs the target interpreter's version, which the
* backend reporting `language: 'python'` owns and which is unpublished on this
* base; the note records it as that PR's decision.
* Both conditions are evaluated against the ENGINE's Unicode tables, and the
* two sides are versioned independently — `\p{XID_Start}`/`\p{XID_Continue}`
* follow the running engine (Node 22.23.1 reports Unicode 17.0) while CPython
* follows its own (3.9.6 reports 13.0.0). The skew is not symmetric. A CPython
* older than the engine is the dangerous direction: a character added to
* either property since its tables (U+1C89, U+10570, U+1E290, U+1E4D0 are all
* NFKC-stable and accepted here, and all rejected by that 3.9.6) is emitted
* bare and its tokenizer refuses the character, taking the whole SDK block
* down — the same parseability invariant {@link UNPRINTABLE},
* {@link LONE_SURROGATE} and {@link MAX_LIST_NESTING} exist for. Both
* properties carry it: a character added only to `XID_Continue` passes the
* trailing `\p{XID_Continue}*` in a tail position and fails the same way. A
* CPython newer than the engine only routes a legal name to the
* subscript/`dict[str, Any]` path: less readable, still correct. The NFKC
* condition reduces to the same skew, since normalization stability guarantees
* an assigned character's normalization never changes afterwards.
*
* This predicate is not the only reader of those tables. {@link camelCase}
* reads them too, through its split set and its head test, and its output is
* emitted for EVERY tool — including one this predicate rejected, whose
* `TypedDict` is still declared and named. A tool named `zz-\u{1E4D0}x` never
* reaches the skew here (the `-` rejects it outright) yet emits
* `class Zz\u{1E4D0}xArgs`, which that same 3.9.6 refuses. Closing the
* exposure therefore covers all three read points, not this predicate alone;
* it needs the target interpreter's version, which the backend reporting
* `language: 'python'` owns and which is unpublished on this base, so the note
* records it as that PR's decision.
*
* The `ts-types` sibling keeps its own ASCII rule rather than sharing this
* one: ECMAScript identifiers are a different set (`$`, ZWJ/ZWNJ) and are
@@ -221,6 +236,11 @@ function docLines(description: unknown, indent: number): string[] {
* a combining-mark head composes there (`U+0301` gives `Tooĺ`, U+013A), so
* normalizing only the un-prefixed part would emit a name CPython compiles to
* a different symbol. The second call is idempotent on the un-prefixed arm.
*
* The split set and the head test read the engine's Unicode tables, so this
* function carries the same version skew {@link isBareIdentifier} documents,
* by an independent path: a class name derived here is emitted for every tool,
* including one the predicate rejected.
* @param raw - the schema field or tool name to derive from.
* @returns a class-name segment safe to emit.
*/

View File

@@ -453,8 +453,11 @@ describe('renderToolsSdkPy', () => {
// ligature name cannot, because `async def find` would define `find`. The
// subscript comment quotes the name, so its exact bytes survive, and its
// TypedDict is still named and referenced — the name is only unusable as a
// method, not as a class-name source (`camelCase` normalizes what it
// derives, since a generated name is never matched against a JSON key).
// method, not as a class-name source. The `FInd` spelling comes from `fi`'s
// multi-character full case mapping (`'fi'.toUpperCase()` is `'FI'`), not
// from `camelCase`'s NFKC step, which is the identity on `FInd`: the
// ligature is XID_Start, so the split set keeps it and only the
// capitalization of the head transforms it.
const of = (name: string): ToolSdkSchema => ({
name,
description: `Tool ${name}.`,
@@ -559,6 +562,64 @@ describe('renderToolsSdkPy', () => {
expect(text).toContain('class XArgs\uAC00\u1100(TypedDict):')
})
it('routes a fold collision through the counter that raw-byte dedup would miss', () => {
// The other half of the `childClassName` normalization: two joins that are
// byte-distinct before NFKC and identical after. Field `\uAC00` allocates
// `XArgs\uAC00`; the sibling `\u1100` allocates `XArgs\u1100`, and ITS child
// `\u1161` joins to `XArgs\u1100\u1161` — the same `XArgs\uAC00` once composed.
// Normalizing at the join is what lets `usedClassNames`, which dedupes by raw
// bytes, see the collision at all; unnormalized, both would be declared and
// CPython would compile the second as a shadow of the first.
const text = renderToolsSdkPy([
{
name: 'x',
description: 'Colliding jamo joins.',
parameters: {
type: 'object',
additionalProperties: false,
required: ['\uAC00', '\u1100'],
properties: {
'\uAC00': { type: 'object', additionalProperties: false, required: ['q'], properties: { q: { type: 'string' } } },
'\u1100': {
type: 'object',
additionalProperties: false,
required: ['\u1161'],
properties: {
'\u1161': { type: 'object', additionalProperties: false, required: ['q'], properties: { q: { type: 'string' } } },
},
},
},
},
output: { type: 'string' },
},
])
expect(text).toContain('class XArgs\uAC00(TypedDict):')
expect(text).toContain('class XArgs\uAC002(TypedDict):')
expect(text).toContain(' \u1161: XArgs\uAC002')
})
it('names both branches of a oneOf of objects on the argument side', () => {
// The output side is pinned elsewhere; arguments reach the same
// `childClassName(frame.className, index + 1)` path, and the annotation is
// the union of the two derived names rather than a degraded dict.
const text = renderToolsSdkPy([
{
name: 'x',
description: 'Union arguments.',
parameters: {
oneOf: [
{ type: 'object', additionalProperties: false, required: ['a'], properties: { a: { type: 'string' } } },
{ type: 'object', additionalProperties: false, required: ['b'], properties: { b: { type: 'number' } } },
],
},
output: { type: 'string' },
},
])
expect(text).toContain('class XArgs1(TypedDict):')
expect(text).toContain('class XArgs2(TypedDict):')
expect(text).toContain('async def x(self, args: XArgs1 | XArgs2) -> str:')
})
it('declares a closed empty object with omitted properties as an empty TypedDict, not dict[str, Any]', () => {
// `{ type: 'object', additionalProperties: false }` with no `properties`
// is a closed empty object — no key accepted — exactly as the validator