docs: enforce bilingual README coverage

This commit is contained in:
Tianyi Cui
2026-07-26 03:29:11 +08:00
parent 0fb392e42f
commit 37bfac749b
12 changed files with 126 additions and 28 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-07-02-bilingual-docs-and-pairing-gate.md: ece3ccc183893a85335a36eb9b00cb42d32a1a37
2026-07-02-bilingual-docs-and-pairing-gate.zh.md: dc8c648cc4ce9dd7739383c620ee2b1c3794f2ee
2026-07-02-bilingual-docs-and-pairing-gate.md: 45a587586b1387d7c351f9c268bd038fcd549ed5
2026-07-02-bilingual-docs-and-pairing-gate.zh.md: 4875e48b43f2324ae117bb7aec1bf81dce3bc2bb

View File

@@ -13,7 +13,7 @@ This repo's README and docs tree are read by people and agents inside and outsid
- **Paired sibling files with equal authority.** A documentation pair is three sibling files: English `foo.md`, Chinese `foo.zh.md`, and a consistency record `foo.i18n.yaml`. Neither language is canonical — a document may be authored and reviewed Chinese-first and translated to English afterwards, or the reverse; what binds the pair is that both sides must say the same thing, and pairs merge whole (both languages plus the record, never one alone). Policy: [docs/i18n/README.md](../../../../docs/i18n/README.md); translation rules: [docs/i18n/translation-rules.md](../../../../docs/i18n/translation-rules.md); terminology source of truth: [docs/i18n/terminology.md](../../../../docs/i18n/terminology.md).
- **A sidecar record of both blob hashes makes consistency checkable.** `foo.i18n.yaml` holds the full git blob hash of each side as of the last confirmed-consistent state. An edit to either side without re-confirming the pair is then mechanically detectable as a pure content comparison — no history lookup — and the hashes are computable for files edited in the same PR, which a commit-hash record is not. Re-recording (`verify-translation-pairing --write`) produces a reviewable yaml diff: confirming consistency is an explicit, visible act in the PR.
- **`verify-translation-pairing` joins `doc-sync`.** The gate ([scripts/verify-translation-pairing.ts](../../../../scripts/verify-translation-pairing.ts)) enforces: sources selected explicitly, by document class, or by the manifest's `requiredSince` cutoff have complete pairs; every existing pair is complete (all three files) and consistent (both hashes match, switcher links both ways, structural signatures identical); and excluded generated, instruction, or bilingual-by-construction files stay unpaired. The `requiredClasses` set in [scripts/translation-pairing.manifest.json](../../../../scripts/translation-pairing.manifest.json) closes a translated class for all present and future files instead of relying on an enumerated snapshot. Source-oriented code gates consume a `.zh.md` fence sequence as a derivative only when its unsuffixed sibling has the same tracked fences in the same order with byte-identical bodies; an incomplete, reordered, reclassified, or changed sequence stays independent, so the owning code gate or pairing gate reports the mismatch.
- **The enforcement frontier advances in coherent review batches, then closes by class.** Explicit `required` entries admit related files while their back-catalog is still being reviewed; after that catalog is complete, its `non-readme` or `readme` class enters `requiredClasses` and can no longer grow a backlog. A site-published pair uses `pairedPages()` so the root locale projects `.zh.md` and `/en/` projects `.md`; creating a counterpart alone does not publish it.
- **The enforcement frontier advances in coherent review batches, then closes by class.** Explicit `required` entries admit related files while their back-catalog is still being reviewed; after that catalog is complete, its `non-readme` or `readme` class enters `requiredClasses` and can no longer grow a backlog. Both classes are closed. README discovery covers every case-insensitive README basename outside vendored, dependency, and ignored build-output trees, including future top-level directories. A site-published pair uses `pairedPages()` so the root locale projects `.zh.md` and `/en/` projects `.md`; creating a counterpart alone does not publish it.
- **Pairing records are metadata, not Cordis Loader configuration.** Cordis configuration discovery accepts actual `.cordis.yml` and `.cordis.yaml` files while excluding `*.i18n.yaml`, even when the document name contains `cordis`. This preserves validation of executable Loader entries without parsing translation hashes as configuration.
- **Translation is agent work with human review.** The committed workflow is [.agents/skills/dsh-translate-docs](../../../skills/dsh-translate-docs/SKILL.md), following the same pattern as [dsh-code-review](../../../skills/dsh-code-review/SKILL.md): the skill carries the workflow and defers to the docs as sources of truth. The skill directs the orchestrating agent to delegate translation writing to a subagent.
@@ -40,5 +40,5 @@ Paired sibling files with locale suffixes are the dominant Chinese big-tech conv
- Every pair adds a third file to the tree. The record is machine-written (`--write`), so the cost is directory noise, not maintenance effort; in exchange, "who confirmed these consistent, and when" is answerable from git blame on the yaml.
- When the two sides disagree, no mechanical rule picks a winner — the PR review does. That is the price of equal authority, accepted deliberately: the alternative (a canonical language) forbids Chinese-first authoring.
- Generated docs (`cordis-catalog/`, `tool-catalog/`, `module-graph.md`) are excluded for now; the planned follow-up is to teach their generators to emit Chinese alongside English, at which point they leave the exclusion list.
- Rollout remains incremental until a document class is complete: explicit `required` entries and the date cutoff prevent regression during review batches, while a closed class makes every current and future member mandatory. The non-README class is closed, so only the README class can still appear as backlog.
- Explicit `required` entries and the date cutoff preserve the reviewed rollout history, while the two closed classes make every current and future in-scope document mandatory. No document class can grow a new backlog.
- The recorded hashes double as the update tool (`git cat-file -p <hash>` recovers either side's last-confirmed text for a minimal diff-based update), so re-translation of whole files is never forced by the mechanism.

View File

@@ -13,7 +13,7 @@ Status: implemented
- **配对兄弟文件,两种语言同权。** 一对文档由三个兄弟文件组成:英文 `foo.md`、中文 `foo.zh.md`,以及一份一致性记录 `foo.i18n.yaml`。没有哪种语言是正典:一篇文档可以先用中文撰写和评审、之后再译成英文,反之亦可;约束配对的是:两侧必须表达相同的内容,且配对整体合并(两种语言加记录,绝不单独落一侧)。政策见 [docs/i18n/README.md](../../../../docs/i18n/README.md);翻译规则见 [docs/i18n/translation-rules.md](../../../../docs/i18n/translation-rules.md);术语真源见 [docs/i18n/terminology.md](../../../../docs/i18n/terminology.md)。
- **伴随记录保存两侧 blob hash使一致性可检查。** `foo.i18n.yaml` 保存两侧文件在上一次确认一致时各自的完整 git blob hash。此后修改了任一侧而未重新确认配对都能被机械检测出来纯内容比较无需查询历史而且同一个 PRPull Request内改动的文件也能计算出 hashcommit hash 式的记录做不到这一点。重新记录(`verify-translation-pairing --write`)会产生一份可评审的 yaml diff确认一致在 PR 中是一个显式、可见的动作。
- **`verify-translation-pairing` 加入 `doc-sync`。** 门禁([scripts/verify-translation-pairing.ts](../../../../scripts/verify-translation-pairing.ts))强制执行以下规则:通过显式指定、文档类别或 manifest元数据清单`requiredSince` 分界日期选中的源文档必须有完整配对;任何已存在的配对必须完整(三个文件齐全)且一致(两个 hash 匹配、切换行双向互链、结构签名一致);被排除的生成文档、指令文档或本身即双语的文档不得配对。[scripts/translation-pairing.manifest.json](../../../../scripts/translation-pairing.manifest.json) 中的 `requiredClasses` 集合会将已完成翻译的类别纳入强制范围,对其当前及今后所有文件强制执行契约,而不再依赖一份枚举式快照。只有当 `.zh.md` 围栏序列与其无后缀兄弟文件拥有顺序相同、正文按字节一致的同一组受跟踪围栏时,面向源码的代码门禁才会将其作为派生内容消费;不完整、顺序变更、重分类或已改动的序列仍会独立受检,因此由其所属的代码门禁或配对门禁报告不匹配。
- **执行红线按连贯的评审批次推进,再以类别为单位完成强制覆盖。** 在存量文档仍处于评审阶段时,显式 `required` 条目会纳入相关文件;存量文档全部完成后,其 `non-readme``readme` 类别进入 `requiredClasses`,不再产生新的 backlog。发布到文档站的配对使用 `pairedPages()`,由根 locale 投影 `.zh.md`,由 `/en/` 投影 `.md`;仅创建对侧文件并不会发布它。
- **执行红线按连贯的评审批次推进,再以类别为单位完成强制覆盖。** 在存量文档仍处于评审阶段时,显式 `required` 条目会纳入相关文件;存量文档全部完成后,其 `non-readme``readme` 类别进入 `requiredClasses`,不再产生新的 backlog。这两个类别均已纳入强制范围。README 发现会覆盖 vendor 源码、依赖目录与被忽略的构建产物目录之外所有文件名不区分大小写匹配 README 的文件,包括今后新增的顶层目录。发布到文档站的配对使用 `pairedPages()`,由根 locale 投影 `.zh.md`,由 `/en/` 投影 `.md`;仅创建对侧文件并不会发布它。
- **配对记录是元数据,而不是 Cordis Loader 配置。** Cordis 配置发现会接受实际的 `.cordis.yml``.cordis.yaml` 文件,同时排除 `*.i18n.yaml`,即使文档名中包含 `cordis` 也不例外。这样既能继续校验可执行的 Loader 配置项,又不会把翻译 hash 当作配置来解析。
- **翻译是 agent 的工作,由人评审。** 仓库内置的工作流是 [.agents/skills/dsh-translate-docs](../../../skills/dsh-translate-docs/SKILL.md),与 [dsh-code-review](../../../skills/dsh-code-review/SKILL.md) 模式相同skill技能承载工作流并将文档作为真源。该 skill 要求编排 agent 把翻译写作委派给 subagent。
@@ -40,5 +40,5 @@ Status: implemented
- 每个配对给目录树多添一个文件。记录由机器写入(`--write`),代价是目录噪音而非维护负担;换来的是「谁在何时确认过这对文档一致」可以从 yaml 的 git blame 直接回答。
- 两侧说法冲突时,没有机械规则裁决谁赢,由 PR 评审裁决。这是同权的代价,且是有意接受的:另一个选项(正典语言)会禁止中文先行撰写。
- 生成文档(`cordis-catalog/``tool-catalog/``module-graph.md`)暂被排除;计划中的后续工作是让生成器在输出英文的同时输出中文,届时将这些文件移出排除清单。
- 在文档类别全部完成之前,推进仍然是渐进的:显式 `required` 条目与日期分界可在评审批次期间防止回退,已纳入强制范围的类别则将其当前及今后的每个成员都列为必选项。非 README 类别已纳入强制范围,因此只有 README 类别仍可能出现 backlog待翻清单
- 显式 `required` 条目与日期分界保留已经评审的推进历史,而两个已纳入强制范围的类别会将当前及今后所有范围内的文档列为必选项。任何文档类别都不能新增 backlog待翻清单
- 记录的 hash 兼作更新工具(`git cat-file -p <hash>` 能还原任一侧上次确认的文本,用于基于 diff 的最小更新),因此这套机制从不强迫整篇重译。

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
README.md: 053453bc622f58083a5e0e2992f1b8a820e3f3f9
README.zh.md: 7f6242e8ff2a3ec69e2b84402dab447297047991
README.md: 25c4698b2efacbb0cb1dd5b8f27ad94be051c558
README.zh.md: e5faefff97d4ef8de9bf05613f257ede06c5e4a7

View File

@@ -38,7 +38,7 @@ The gate's limit, stated plainly: **a green gate means the pair was confirmed co
## Scope, exclusions, and rollout
**Scope**: the root `README.md`, everything under `.agents/notes/**`, `docs/**`, and `python/**`. Package READMEs (`packages/**`) join the scope in a later batch.
**Scope**: every non-vendor README, plus every document under `.agents/notes/**`, `docs/**`, and `python/**`. README matching is case-insensitive on the basename and covers future directories without another manifest edit. Dependency and ignored build-output trees are discovery exclusions, not source documentation.
**Excluded** (never paired, and the gate rejects a `.zh.md` or `.i18n.yaml` for them):
@@ -47,7 +47,7 @@ The gate's limit, stated plainly: **a green gate means the pair was confirmed co
- `docs/i18n/terminology.md` and [style-samples.md](style-samples.md) — both are bilingual by construction.
- [translation-prompt.md](translation-prompt.md) — the automated pipeline's prompt template; its body is machine-consumed verbatim, so a paired translation would change pipeline behavior.
**Enforcement frontier**: `requiredClasses` closes a whole document class after its back-catalog has been translated. `non-readme` is closed: every current or future in-scope non-README document must merge bilingual. README coverage remains an explicit-file rollout until `readme` joins the closed set. The manifest's `required` list retains already-admitted files, and a date-named document (`yyyy-mm-dd-*.md`, i.e. an Agent Note) dated on or after `requiredSince` must merge with its pair regardless of class. `--list` reports any unclosed-class backlog while every existing pair remains governed by the full contract.
**Enforcement frontier**: `requiredClasses` closes a whole document class after its back-catalog has been translated. Both `non-readme` and `readme` are closed: every current or future in-scope document must merge bilingual. The manifest's `required` list retains already-admitted files, and a date-named document (`yyyy-mm-dd-*.md`, i.e. an Agent Note) dated on or after `requiredSince` must merge with its pair regardless of class. `--list` reports any unclosed-class backlog while every existing pair remains governed by the full contract.
## Division of labor

View File

@@ -38,7 +38,7 @@
## 范围、排除与推进
**范围**根 `README.md`,以及 `.agents/notes/**`、`docs/**` 与 `python/**` 下的全部内容。包packageREADME`packages/**`)在后续批次加入范围
**范围**除 vendor 源码外的全部 README以及 `.agents/notes/**`、`docs/**` 与 `python/**` 下的全部文档。匹配 README 时只看文件名且不区分大小写,因此今后新增的目录无需再修改 manifest。依赖目录和被忽略的构建产物目录只在发现阶段排除并非源文档
**排除**(永不配对,门禁拒绝为它们建 `.zh.md` 或 `.i18n.yaml`
@@ -47,7 +47,7 @@
- `docs/i18n/terminology.md` 与 [style-samples.md](style-samples.md):二者本身即为中英对照文档。
- [translation-prompt.md](translation-prompt.md):自动翻译流水线的提示词模板;正文逐字进入模型请求,配对翻译会改变流水线行为。
**执行红线**:某个文档类别的存量文档全部翻译完成后,`requiredClasses` 会将整个类别纳入强制范围。`non-readme` 已纳入强制范围:当前及今后所有纳入范围的非 README 文档,合并时都必须配齐双语文件。README 覆盖仍按显式文件逐步推进,直到 `readme` 加入这一强制范围。manifest 的 `required` 列表保留已纳入的文件;以日期命名的文档(`yyyy-mm-dd-*.md`,即 Agent Note只要日期不早于 `requiredSince`,就无论所属类别都必须与对侧文件一同合并。`--list` 会报告尚未纳入强制范围的类别中的任何 backlog待翻清单而每个已存在的配对仍受完整契约约束。
**执行红线**:某个文档类别的存量文档全部翻译完成后,`requiredClasses` 会将整个类别纳入强制范围。`non-readme` 与 `readme` 均已纳入强制范围当前及今后所有纳入范围的文档合并时都必须配齐双语文件。manifest 的 `required` 列表保留已纳入的文件;以日期命名的文档(`yyyy-mm-dd-*.md`,即 Agent Note只要日期不早于 `requiredSince`,就无论所属类别都必须与对侧文件一同合并。`--list` 会报告尚未纳入强制范围的类别中的任何 backlog待翻清单而每个已存在的配对仍受完整契约约束。
## 分工

View File

@@ -70,9 +70,9 @@
## ⑦ 推进策略(长段拆分示范)
> **Enforcement frontier**: a document class enters the manifest's `requiredClasses` set only after its back-catalog has been translated and reviewed. The `non-readme` class is closed, so every current or future in-scope non-README document must merge bilingual; README coverage remains an explicit-file rollout until that class is ready to close. […] Pairing a document is a commitment: every later edit to either side must carry the counterpart along, so close a class only when translation review can sustain it.
> **Enforcement frontier**: a document class enters the manifest's `requiredClasses` set only after its back-catalog has been translated and reviewed. The `non-readme` and `readme` classes are closed, so every current or future in-scope document must merge bilingual. […] Pairing a document is a commitment: every later edit to either side must carry the counterpart along, so close a class only when translation review can sustain it.
**执行红线**:只有在某个文档类别的存量文档全部完成翻译和评审后,该类别才会进入 manifest元数据清单`requiredClasses` 集合。`non-readme` 类别已纳入强制范围,因此当前及今后所有纳入范围的非 README 文档,合入时都必须配齐双语文件README 覆盖仍按显式文件逐步推进,直到该类别具备整体纳入强制范围的条件。(……)一旦文档完成配对,后续修改任一侧都必须同步更新另一侧。因此,只有在翻译评审能力足以持续支撑时,才应将整个类别纳入强制范围。
**执行红线**:只有在某个文档类别的存量文档全部完成翻译和评审后,该类别才会进入 manifest元数据清单`requiredClasses` 集合。`non-readme``readme` 类别已纳入强制范围,因此当前及今后所有纳入范围的文档,合入时都必须配齐双语文件。(……)一旦文档完成配对,后续修改任一侧都必须同步更新另一侧。因此,只有在翻译评审能力足以持续支撑时,才应将整个类别纳入强制范围。
## 从样例提炼的要点

File diff suppressed because one or more lines are too long

View File

@@ -1,6 +1,7 @@
{
"requiredClasses": [
"non-readme"
"non-readme",
"readme"
],
"requiredSince": "2026-07-14",
"required": [

View File

@@ -4,6 +4,7 @@ import { describe, expect, it } from 'vitest'
import {
datedDocumentDate,
isIsoDate,
isTranslationScopeFile,
parseTranslationMarkdown,
parseTranslationPairingManifest,
requiresPairByDate,
@@ -82,6 +83,45 @@ describe('document-class pairing frontier', () => {
expect(requiresTranslationPair('docs/legacy/README.md', manifest)).toBe(true)
expect(requiresTranslationPair('docs/new/README.md', manifest)).toBe(false)
})
it('requires both document classes after the README frontier closes', () => {
const closed = parseTranslationPairingManifest(JSON.stringify({
...manifest,
requiredClasses: ['non-readme', 'readme'],
}))
expect(requiresTranslationPair('docs/guide.md', closed)).toBe(true)
expect(requiresTranslationPair('future/subtree/README.md', closed)).toBe(true)
})
})
describe('translation scope discovery', () => {
it.each([
'README.md',
'apps/cli/README.md',
'future/subtree/readme.md',
'packages/example/README.zh.md',
'native/example/README.i18n.yaml',
'.agents/notes/proposed/feature.md',
'docs/guide.md',
'python/guide.md',
])('includes %s', (file) => {
expect(isTranslationScopeFile(file)).toBe(true)
})
it.each([
'packages/example/guide.md',
'examples/tutorial.md',
'website/reference.md',
'packages/example/README.txt',
'vendor/example/README.md',
'packages/example/node_modules/dependency/README.md',
'packages/example/lib/README.md',
'coverage/report/README.md',
'python/sdk-runtime/src/deepseek_harness_runtime/runtime/dsh-jsonrpc-agent-macos-arm64/README.md',
'python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/README.md',
])('excludes non-source or non-README path %s', (file) => {
expect(isTranslationScopeFile(file)).toBe(false)
})
})
describe('date-based pairing frontier', () => {

View File

@@ -26,6 +26,63 @@ const TRANSLATION_DOCUMENT_CLASSES: TranslationDocumentClass[] = ['readme', 'non
const ISO_DATE = /^\d{4}-\d{2}-\d{2}$/
const DATED_DOCUMENT = /(?:^|\/)(\d{4}-\d{2}-\d{2})-[^/]*\.md$/
const README_ARTIFACT = /(?:^|\/)readme(?:\.md|\.zh\.md|\.i18n\.yaml)$/i
const NON_SOURCE_DIRECTORIES = new Set([
'node_modules',
'lib',
'.pnpm-store',
'.cache',
'coverage',
'.sessions',
'.storages',
'tmp',
'dist-exe',
'__pycache__',
'.pytest_cache',
'.artifacts',
'vendor',
])
/** Glob traversal exclusions corresponding to the non-source path predicate. */
export const TRANSLATION_SCOPE_GLOB_EXCLUDES = [
'**/node_modules/**',
'**/lib/**',
'**/.pnpm-store/**',
'**/.cache/**',
'**/coverage/**',
'**/.doc-typecheck-*/**',
'**/.node-next-types-*/**',
'**/.sessions/**',
'**/.storages/**',
'**/tmp/**',
'**/dist-exe/**',
'**/__pycache__/**',
'**/.pytest_cache/**',
'apps/web/dist/**',
'.artifacts/**',
'python/sdk-runtime/src/deepseek_harness_runtime/runtime/dsh-jsonrpc-agent-*/**',
'python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/**',
'vendor/**',
]
/** Whether a repository-relative path belongs to a dependency or generated tree. */
function isTranslationSourceExcluded(file: string): boolean {
const segments = file.split('/')
return segments.some(segment => NON_SOURCE_DIRECTORIES.has(segment)
|| segment.startsWith('.doc-typecheck-')
|| segment.startsWith('.node-next-types-'))
|| file.startsWith('apps/web/dist/')
|| file.startsWith('python/sdk-runtime/src/deepseek_harness_runtime/runtime/dsh-jsonrpc-agent-')
|| file.startsWith('python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/')
}
/** Whether one discovered Markdown or sidecar path belongs to the bilingual source corpus. */
export function isTranslationScopeFile(file: string): boolean {
return !isTranslationSourceExcluded(file) && (README_ARTIFACT.test(file)
|| file.startsWith('.agents/notes/')
|| file.startsWith('docs/')
|| file.startsWith('python/'))
}
/** Whether a string names one real calendar day in canonical ISO form. */
export function isIsoDate(value: string): boolean {

View File

@@ -15,7 +15,9 @@ import {
linksTo,
parseTranslationMarkdown,
parseTranslationPairingManifest,
isTranslationScopeFile,
requiresTranslationPair,
TRANSLATION_SCOPE_GLOB_EXCLUDES,
translationDocumentClass,
translationStructureDiff,
translationStructureSignature,
@@ -25,17 +27,12 @@ const root = resolve(import.meta.dirname, '..')
const listMode = process.argv.includes('--list')
const writeMode = process.argv.includes('--write')
/** Scope of the bilingual contract: root docs, Agent Notes, the docs tree, and the Python SDK tree. */
/** Discover source Markdown and pairing sidecars before applying the corpus predicate. */
const SCOPE_PATTERNS = [
'README.md',
'README.zh.md',
'README.i18n.yaml',
'**/*.md',
'**/*.i18n.yaml',
'.agents/notes/**/*.md',
'.agents/notes/**/*.i18n.yaml',
'docs/**/*.md',
'docs/**/*.i18n.yaml',
'python/**/*.md',
'python/**/*.i18n.yaml',
]
const manifest = parseTranslationPairingManifest(readFileSync(join(root, 'scripts/translation-pairing.manifest.json'), 'utf8'))
@@ -93,7 +90,10 @@ function renderMeta(source: string, sourceHash: string, zh: string, zhHash: stri
// Enumerate the scope once.
const files = new Set<string>()
for (const pattern of SCOPE_PATTERNS) {
for (const match of globSync(pattern, { cwd: root })) files.add(match.split(sep).join('/'))
for (const match of globSync(pattern, { cwd: root, exclude: TRANSLATION_SCOPE_GLOB_EXCLUDES })) {
const normalized = match.split(sep).join('/')
if (isTranslationScopeFile(normalized)) files.add(normalized)
}
}
const translations = [...files].filter(f => f.endsWith('.zh.md')).sort()
const metas = [...files].filter(f => f.endsWith('.i18n.yaml')).sort()