docs(tools): scope the Python SDK validity standard to the grammar

The list-nesting cap guards against a tokenizer SyntaxError, which makes the
text not Python. A long `A | B | …` union is valid at any length and only
defeats CPython's compile-time C recursion (measured: 1,000 branches compile,
5,000 raise RecursionError); nothing compiles this block, and capping would
retire the deep-chain tests pinning the walk's linear time. Records that
boundary at the `oneOf` arm and in the Agent Note (both languages).

Also documents that the context-free degrade marker reads the call's
className rather than the frame's — frames propagate a derived name, so a
per-frame read would declare classes the caller cannot receive — and pins
that path with oneOf-of-objects and array-of-oneOf assertions.
This commit is contained in:
Chinesezjc
2026-08-05 15:59:15 +08:00
parent 0d17baae01
commit cc6e4d59fc
5 changed files with 38 additions and 5 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md
2026-07-31-code-mode-language-dispatch.md: 6245891651aece73d5a51a6341bc4f76b98fad12
2026-07-31-code-mode-language-dispatch.zh.md: 23dbd1c2a9d049d0648109c474b09feaae28886e
2026-07-31-code-mode-language-dispatch.md: 5785565296cd06e8e1b4761969449e51d1e3af0d
2026-07-31-code-mode-language-dispatch.zh.md: 6e9d39bb117b2b18c0291bc047c4972e140a0b6e

View File

@@ -25,6 +25,8 @@ Both tables are read with `Object.hasOwn` before use so a language named `toStri
`py-types.ts` renders the same unified tool-schema vocabulary `jsonSchemaToTs` covers, targeting Python: `jsonSchemaToPy` emits a type expression per JSON-schema node, and `renderToolsSdkPy` assembles named `TypedDict`s for each visible tool's arguments and canonical output plus a `tools` object with usage instructions equivalent to the TypeScript flavor. Unsupported raw constructs degrade rather than throwing during assembly, matching the TypeScript renderer's contract. The output is deterministic — lexicographic tool order, byte-identical text for an unchanged tool set — so the prompt stays prefix-cache-friendly. Lexicographic means one ordered member stream: a tool whose name is not a legal attribute is listed as a `tools[name]` comment in its sorted position rather than partitioned to the end, matching how the TypeScript flavor quotes an exotic key in place. That stream forces one thing directly — comment lines are not statements, so a tool set that emits no method at all still needs an explicit `pass`. Two further rules are Python-specific rather than consequences of the ordering. A description becomes the method's docstring emitted as the FIRST statement of its body: above the `async def` the first one would document the `Tools` class and the rest would be dead expressions, leaving every method undocumented. And a `list[…]` chain degrades to `Any` past `MAX_LIST_NESTING`, because CPython's tokenizer rejects a line with more than 200 open brackets and the block must stay parseable Python — the same reason `docLines` escapes quotes and backslashes. `ts-types` needs neither: TypeScript attaches a leading `/** … */` to the member that follows it and bounds nesting nowhere in its grammar.
The standard that cap serves is grammatical validity, and the boundary is deliberate: a long `A | B | …` union is valid Python at any length and is left uncapped, even though CPython's `compile()` exhausts its C recursion walking the left-nested `BinOp` spine (measured on 3.9: 1,000 branches compile, 5,000 raise `RecursionError`). Nothing compiles this block — it is prompt text — so that limit costs nothing, whereas capping union length would retire the deep-chain tests that pin the walk's linear time and the class-name propagation cap. A future renderer that does need compilable output should flatten unions rather than truncate them.
`renderType` validates the whole schema once (`assertSupportedJsonSchema`) and then trusts it, wrapping the walk in one `try/catch` that degrades to `Any` — the same trusted-after-validation stance the sibling `ts-types` renderer takes at this typed same-process seam ([Trust TypeScript at typed same-process seams](../../../../AGENTS.md)). It deliberately carries NO defenses against a schema whose accessors mutate between reads (post-validation cycles, TOCTOU on `const`/`enum`, self-referential functions): the input is a first-party registration (a `defineTool` literal or a raw registration) or a wire-derived plain JSON schema — the former is trusted per AGENTS.md, the latter is a `JSON.parse` product that physically cannot carry accessors, and `renderType` re-validates the whole tree on every call regardless — so such inputs are unreachable, and adding per-shape guards here would break symmetry with `ts-types` (which has none) for values the static interface forbids. `jsonSchemaToPy(schema: unknown)` accepts `unknown` and returns `Any` on a malformed schema — the Python counterpart of the TS flavor's `unknown` — but its contract is "degrade an unsupported schema", not "survive an adversarial mutating one".
## Alternatives considered

View File

@@ -25,6 +25,8 @@ Code Mode 只生成一种 SDK 形态TypeScript。`ToolRegistry` 为 `tools:sd
`py-types.ts` 渲染 `jsonSchemaToTs` 所覆盖的同一套统一工具 schema 词汇,目标为 Python`jsonSchemaToPy` 为每个 JSON-schema 节点发出一个类型表达式,`renderToolsSdkPy` 为每个可见工具的参数与规范输出装配具名 `TypedDict`,再加一个带用法说明的 `tools` 对象,与 TypeScript 形态等价。不支持的原始构造在装配时降级而非抛错,与 TypeScript 渲染器的契约一致。输出是确定性的——工具按字典序排列,工具集不变时文本逐字节相同——因此 prompt 保持 prefix-cache 友好。字典序意味着单一有序的成员流:名字不是合法属性的工具以 `tools[name]` 注释出现在它排序后的位置上,而不是被分拣到末尾,与 TypeScript 形态就地为异常键加引号的做法一致。这个成员流直接决定了一件事:注释行不是语句,所以一个不发出任何方法的工具集仍需显式 `pass`。另有两条规则并非源自排序,而是 Python 特有。其一,描述会成为方法的 docstring且必须作为方法体的**第一条语句**发出:放在 `async def` 之上,第一条会变成 `Tools` 的类文档、其余都是无效果表达式,导致每个方法都没有文档。其二,`list[…]` 链超过 `MAX_LIST_NESTING` 后降级为 `Any`,因为 CPython 的 tokenizer 拒绝一行中超过 200 个同时未闭合的括号,而这个块必须是可解析的 Python——与 `docLines` 转义引号和反斜杠是同一个理由。`ts-types` 两者都不需要TypeScript 会把前置的 `/** … */` 附着到其后的成员上,其语法也不对嵌套设限。
该上限服务的标准是**语法合法性**,这条边界是有意划定的:长的 `A | B | …` union 在任何长度下都是合法 Python故不设上限——尽管 CPython 的 `compile()` 在沿左嵌套 `BinOp` 脊柱下降时会耗尽 C 递归(在 3.9 上实测1,000 个分支可编译5,000 个抛 `RecursionError`)。没有任何东西会编译这个块——它是提示词文本——所以那条限制在这里没有代价;而给 union 长度封顶会作废那几个钉住 walk 线性时间与类名传播上限的深链测试。将来若有渲染器确实需要可编译的输出,应当把 union 拍平,而不是截断。
`renderType` 先用 `assertSupportedJsonSchema` 整树校验一次、随后信任它,用单个 `try/catch` 把整个遍历兜住并降级为 `Any`——与姊妹渲染器 `ts-types` 在这个 typed 同进程 seam 上采取的「校验后信任」姿态一致([Trust TypeScript at typed same-process seams](../../../../AGENTS.md))。它有意不设任何针对「访问器在多次读取间变值」的防御(校验后成环、`const`/`enum` 的 TOCTOU、自引用函数输入是第一方注册`defineTool` 字面量或 raw 注册)或从 wire 桥接而来的纯 JSON——前者按 AGENTS.md 受信任,后者是 `JSON.parse` 产物、物理上不可能携带访问器,且每次调用 `renderType` 都会整树重新校验——这类输入不可达,而在此加逐形态守卫会为静态接口所禁止的值破坏与 `ts-types`(没有这类守卫)的对称。`jsonSchemaToPy(schema: unknown)` 接受 `unknown` 并对畸形 schema 返回 `Any`——TypeScript 形态 `unknown` 的对应物——但它的契约是「降级不支持的 schema」而非「扛住对抗性的可变 schema」。
## Alternatives considered

View File

@@ -136,14 +136,18 @@ const MAX_CLASS_NAME_BASE = 120
* nested parentheses`), so an array chain deeper than that would render an SDK
* block that is not valid Python at all — the same failure the docstring
* escaping in {@link docLines} exists to prevent. 180 leaves headroom for the
* one bracket an annotation can add around the chain (`NotRequired[…]`).
* few brackets an annotation can add around the chain: `NotRequired[…]`, a
* `Literal[…]` item, and the `def` parameter list an argument annotation sits
* inside, for a worst case of 182.
*
* A CPython grammar limit, not a deployment choice, so it is fixed rather than
* configurable. The sibling `ts-types` renderer needs no counterpart: nothing
* in the TypeScript grammar bounds nesting, and its SDK block is never type-
* checked. Only bracket nesting counts — a `oneOf` renders as a flat `A | B`
* chain and nested objects render as separate `class` statements, so neither
* accumulates open brackets at any depth.
* accumulates open brackets at any depth. The invariant this cap serves is
* grammatical validity; see the `oneOf` arm in {@link renderType} for the one
* interpreter limit deliberately left uncapped.
*/
const MAX_LIST_NESTING = 180
@@ -367,6 +371,18 @@ function renderType(schema: unknown, className: string, state: RenderState): str
frame.kind = 'oneOf'
// A union renders as `A | B` — no brackets of its own, so the branches
// inherit the enclosing depth unchanged.
//
// Union LENGTH is deliberately uncapped, unlike list nesting. The two
// limits are different in kind: >200 open brackets is a SyntaxError
// from the tokenizer, so the text is not Python; a long `A | B | …`
// chain is grammatically valid at any length and only defeats CPython's
// C-recursion when `compile()` walks the left-nested BinOp spine
// (measured: 1,000 branches compile, 5,000 raise RecursionError). This
// block is prompt text — nothing compiles it — so that limit costs
// nothing here, while capping would retire the deep-chain tests that
// pin the walk's linear time and the class-name propagation cap. The
// standard this renderer holds is grammatical validity, not
// compilability under one interpreter's stack.
frame.children = node.oneOf.map((branch, index) => ({ schema: branch, className: childClassName(frame.className, `${index + 1}`), listDepth: frame.listDepth }))
continue
}
@@ -409,7 +425,11 @@ function renderType(schema: unknown, className: string, state: RenderState): str
// than a permissive `dict[str, Any]`.
const entries = Object.entries(node.properties ?? {})
// An empty `className` marks the context-free `jsonSchemaToPy` entry:
// there is no naming context to declare into, so degrade. A field
// there is no naming context to declare into, so degrade. This reads
// the CALL's className, not `frame.className`: the marker belongs to
// the whole walk, and frames propagate a derived name (a `oneOf`
// branch of the context-free root gets `Tool1`), so a per-frame read
// would declare classes the caller has no way to receive. A field
// name that is not a legal Python attribute is inexpressible as a
// class-syntax `TypedDict` field, so such an object degrades whole.
// A leading-double-underscore non-dunder field (`__token`) would be

View File

@@ -249,6 +249,12 @@ describe('renderToolsSdkPy', () => {
],
})
expect(type).toBe('dict[str, Any] | str')
// Both branches objects, and the same shape reached through an array: the
// marker is the CALL's className, so a propagated frame name (`Tool1`) does
// not revive class declaration on a walk that has nowhere to declare into.
const object = { type: 'object', additionalProperties: false, properties: { ok: { type: 'boolean' } }, required: ['ok'] }
expect(jsonSchemaToPy({ oneOf: [object, object] })).toBe('dict[str, Any] | dict[str, Any]')
expect(jsonSchemaToPy({ type: 'array', items: { oneOf: [object, { type: 'string' }] } })).toBe('list[dict[str, Any] | str]')
})
it('suffixes a counter when two tools CamelCase to the same class base', () => {
@@ -539,6 +545,9 @@ describe('renderToolsSdkPy', () => {
// depth the quadratic path (~100,000^2 char copies) blows past vitest's 5s
// default, so this fails loud on a regression; the `+`/ConsString path is
// milliseconds. (Guard the depth explicitly so the assertions stay exact.)
// The resulting chain is intentionally uncapped, unlike list nesting: it is
// grammatically valid Python at any length, and only CPython's `compile()`
// recursion would reject it — see the `oneOf` arm in py-types.ts.
const depth = 100000
let deep: Record<string, unknown> = { type: 'string' }
for (let i = 0; i < depth; i++) deep = { oneOf: [deep, { type: 'null' }] }