Merge branch 'config-catalog' into catalog-flatten

This commit is contained in:
Tianyi Cui
2026-07-06 23:38:40 +08:00
22 changed files with 846 additions and 164 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
development.md: f032764fff29baaca007211db8b69d9a5129078f
development.zh.md: 3a650d03ce7cafd0e34290ae918e5a303c2ad8a9
development.md: 97ca3f6b9fc9653ab658e480e6155fc1e121854f
development.zh.md: e837afb6a01ed4d0c4801886bd6ca6a7602ac573

View File

@@ -59,7 +59,7 @@ DEEPSEEK_BASE_URL=https://... # optional
lefthook is configured in `lefthook.yml` as an early local checkpoint before review:
- `pre-commit` runs staged-file ESLint fixes, `pnpm run typecheck`, and the vendor manifest guard.
- `pre-push` runs `pnpm run test`, `pnpm run test:snapshot`, `pnpm run hygiene`, `pnpm run doc-sync`, and `pnpm run verify-module-graph`.
- `pre-push` runs `pnpm run check:pre-push`, whose scheduler runs unit tests, snapshot tests, build, module-graph freshness, and the member gates of `pnpm run hygiene` and `pnpm run doc-sync` concurrently.
The vendor manifest guard checks that changes under `vendor/*/src` are staged with the matching `vendor/README.md` manifest update. See `vendor/README.md` before editing vendored code.
@@ -67,22 +67,9 @@ These hooks do not exactly mirror CI. Notably, `pre-push` runs unit tests withou
## CI gates
The GitHub workflow runs these gates on each pull request:
The keyless GitHub workflow has six jobs: five Node 24 lanes run static gates, lint, coverage, snapshot replay, and artifact gates separately, and the Node 26 compatibility job runs `pnpm run check:node-compat`. The lane schedulers fan out independent gates from `package.json`: constraints, typecheck, lint, coverage, snapshot replay, `doc-sync` members, module-graph freshness, `knip`, and the echo-agent smoke test.
- `pnpm install --frozen-lockfile`
- `pnpm run constraints`
- `pnpm run typecheck`
- `pnpm run lint`
- `pnpm run doc-sync`
- `pnpm run verify-module-graph`
- `pnpm run test:coverage`
- `pnpm run test:snapshot`
- `pnpm run build`
- `pnpm run hygiene`
- an echo-agent smoke test that checks the demo's tool call, tool result, and JSONL output
- built-bin smoke tests that run the published `lib/bin.js` entrypoints under plain `node`
`pnpm run hygiene` is the local shorthand for `pnpm run knip && pnpm run publint && pnpm run constraints && pnpm run verify-node-next-types`; CI also runs `pnpm run constraints` as an earlier fail-fast step, then runs the full hygiene script after `pnpm run build`.
`pnpm run build` feeds the artifact lane, and `publint`, `verify-node-next-types`, and built-bin smoke tests wait for build output. The separate real-API workflow runs `pnpm run test:e2e` with a secret and `DSH_E2E_MAX_WORKERS=14`.
## Daily commands

View File

@@ -59,7 +59,7 @@ DEEPSEEK_BASE_URL=https://... # optional
lefthook 在 `lefthook.yml` 中配置,作为评审前的本地早期检查点:
- `pre-commit` 运行对暂存文件的 ESLint 修复、`pnpm run typecheck` 和 vendor manifest 守卫。
- `pre-push` 运行 `pnpm run test``pnpm run test:snapshot``pnpm run hygiene``pnpm run doc-sync` `pnpm run verify-module-graph`
- `pre-push` 运行 `pnpm run check:pre-push`其调度器并发运行单元测试、快照测试、build、module graph 新鲜度,以及 `pnpm run hygiene``pnpm run doc-sync` 的成员门禁
vendor manifest 守卫检查 `vendor/*/src` 下的改动是否连同对应的 `vendor/README.md` manifest 更新一起暂存。编辑 vendor 代码前先看 `vendor/README.md`
@@ -67,22 +67,9 @@ vendor manifest 守卫检查 `vendor/*/src` 下的改动是否连同对应的 `v
## CI 门禁
GitHub 工作流在每个 pull request 上运行这些门禁:
keyless GitHub 工作流有六个 job五个 Node 24 lane 分别运行 static gates、lint、coverage、snapshot replay 和 artifact gatesNode 26 兼容性 job 运行 `pnpm run check:node-compat`。各 lane 调度器并发运行来自 `package.json` 的独立门禁constraints、typecheck、lint、coverage、snapshot replay、`doc-sync` 成员、module graph 新鲜度、`knip` 和 echo-agent 冒烟测试。
- `pnpm install --frozen-lockfile`
- `pnpm run constraints`
- `pnpm run typecheck`
- `pnpm run lint`
- `pnpm run doc-sync`
- `pnpm run verify-module-graph`
- `pnpm run test:coverage`
- `pnpm run test:snapshot`
- `pnpm run build`
- `pnpm run hygiene`
- 一个 echo-agent 冒烟测试,检查演示的工具调用、工具结果和 JSONL 输出
- built-bin 冒烟测试,用纯 `node` 运行发布产物 `lib/bin.js` 入口
`pnpm run hygiene``pnpm run knip && pnpm run publint && pnpm run constraints && pnpm run verify-node-next-types` 的本地简写CI 还会把 `pnpm run constraints` 作为更早的快速失败步骤单独跑一次,然后在 `pnpm run build` 之后跑完整的 hygiene 脚本。
`pnpm run build` 供给 artifact lane`publint``verify-node-next-types` 和 built-bin 冒烟测试等待 build 输出。单独的真实 API 工作流带密钥运行 `pnpm run test:e2e`,并设置 `DSH_E2E_MAX_WORKERS=14`
## 日常命令

View File

@@ -143,6 +143,8 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand;
| [Generated persistence log event catalog](implemented/process/2026-07-04-persistence-log-catalog.md) | 2026-07-04 |
| [One gated in-file format for RFCs](implemented/process/2026-07-05-uniform-rfc-format.md) | 2026-07-05 |
| [Generated plugin config catalog](implemented/process/2026-07-06-generated-config-catalog.md) | 2026-07-06 |
| [Parallel GitHub CI gates](implemented/process/2026-07-06-parallel-github-ci-gates.md) | 2026-07-06 |
| [Parallel pre-push gates](implemented/process/2026-07-06-parallel-pre-push-gates.md) | 2026-07-06 |
### Testing

View File

@@ -0,0 +1,39 @@
# RFC: Parallel GitHub CI gates
Status: implemented
## Problem
The keyless GitHub CI gates are mostly orthogonal: typecheck, lint, documentation freshness, coverage, snapshot replay, build, package-publication hygiene, demo smoke, and built-bin smoke fail for different reasons and do not need each other's runtime state. Running them as one ordered command chain makes the workflow wall clock equal the sum of those gates, while splitting every leaf gate into its own GitHub job repeats checkout, Node setup, pnpm restore, and install work until orchestration overhead becomes the bottleneck.
The hard part is the artifact boundary. `publint`, `verify-node-next-types`, and built-bin smoke tests need the built `lib/` outputs, while most gates only need source and dependencies. A blind fan-out either races those artifact consumers before `pnpm run build` has emitted declarations and bundles, or repeats the build in every artifact-dependent job.
## Decision
[CI](../../../../.github/workflows/ci.yml) keeps the keyless workflow to a few broad jobs instead of one job per gate. The Node 24 matrix has five lanes: static gates (`pnpm run check:ci:static`), lint (`pnpm run check:ci:lint`), coverage (`pnpm run check:ci:coverage`), snapshot replay (`pnpm run check:ci:snapshot`), and artifact gates (`pnpm run check:ci:artifacts`). The Node 26 compatibility job installs once and runs `pnpm run check:node-compat`.
Each lane delegates to [scripts/run-gates.ts](../../../../scripts/run-gates.ts), an in-process scheduler with bounded concurrency (`DSH_GATE_CONCURRENCY`). The static lane fans out constraints, the echo-agent demo smoke, `doc-sync` leaf gates, module-graph freshness, and `knip`; the lint lane runs ESLint with its own Node heap cap and a content-strategy ESLint cache; the coverage lane runs Vitest coverage with bounded file workers (`DSH_COVERAGE_MAX_WORKERS`); the snapshot lane isolates replay; the artifact lane builds once and then fans out the artifact consumers; the Node 26 compatibility job owns the TypeScript typecheck. The scheduler buffers each gate's output and prints a named result block with duration, so independent failures stay attributable inside each broad job log.
Generated `.sessions/` logs and `.doc-typecheck-*` temp directories are ignored by lint. The aggregate local CI mode still runs demo smoke after lint, while the split GitHub static lane can run demo smoke directly because lint is isolated in its own lane.
Build output is produced once inside the Node 24 artifact lane. The artifact consumers (`publint`, `verify-node-next-types`, and built-bin smoke) declare a dependency on `build`, so there is no upload/download handoff and no consumer can race ahead of declarations or bundles. The CI coverage reporter is text-only while local coverage keeps the HTML report.
Both CI workflows cache the pnpm store after enabling Corepack. The real-API e2e workflow also uses the shared `vitest.e2e.config.ts` bounded file pool (`DSH_E2E_MAX_WORKERS=14` in CI), so its speedup comes from dependency-cache reuse plus lower-level test-file fan-out instead of a separate GitHub job split.
## Alternatives considered
- **Keep the full serial chain in a Node matrix** - simplest to reason about, but it duplicates repo-wide gates that do not produce Node-version-specific signal and leaves every PR waiting for the sum of all gates.
- **Run every gate as a separate GitHub job** - maximizes GitHub-visible fan-out, but it creates too many checks and pays repeated setup/install overhead for gates whose runtime is shorter than the runner preparation.
- **Upload build artifacts to artifact-dependent jobs** - preserves correctness across many jobs, but it adds artifact upload/download time and keeps the workflow wide when the artifact consumers can run behind a local dependency in the primary job.
- **Run `typecheck` and `build` concurrently** - exposes more work to the scheduler, but both commands invoke `tsc -b`; sharing incremental build state between them is a needless race for a small wall-clock gain.
- **Use unbounded real-API e2e parallelism** - rejected because the suite includes many live model/tool scenarios; the worker pool needs an explicit `DSH_E2E_MAX_WORKERS` cap so CI and local runs can fan out without hiding quota or resource problems behind flaky rate-limit failures.
## Consequences
PR feedback arrives as a few GitHub checks with structured per-gate log blocks inside each broad job. That keeps runner setup overhead bounded and the Actions UI compact, at the cost of losing one status check per leaf gate.
The broad-lane split repeats checkout, setup, and install more often than a single primary job. That setup cost is intentional: on GitHub's hosted runner, running lint, coverage, and snapshot replay in one process pool oversubscribes CPU badly enough that the single-job critical path is longer than the repeated setup.
The split introduces a maintenance obligation: when `package.json` adds or removes a gate that belongs in CI, [scripts/run-gates.ts](../../../../scripts/run-gates.ts) needs the matching leaf. That obligation is intentional because the runner is the parallel execution plan for the same gate vocabulary, not a separate quality policy.
The Node 26 signal is narrower than the primary Node 24 signal. It proves the source graph on the newer runtime without doubling documentation, coverage, publication, snapshot, and smoke checks whose failures are not expected to vary by Node minor version.

View File

@@ -0,0 +1,40 @@
# RFC: Parallel pre-push gates
Status: implemented
## Problem
The pre-push hook is the last local checkpoint before a branch leaves the machine, so its wall clock directly shapes whether contributors keep it enabled and trust its signal. Lefthook already runs top-level jobs in parallel, but aggregate jobs such as `pnpm run hygiene` and `pnpm run doc-sync` hide long sequential chains inside one job. The hook can therefore be configured as parallel while still waiting on serial subcommands whose members are independent.
Flattening those members directly into `lefthook.yml` solves the local hook only. CI has the same scheduling problem, and duplicating a long leaf list in YAML gives future script changes two places to drift.
`publint` has the same shape one level lower. Each package is linted independently against its own manifest and built output, but the runner loops through every package in order. On this repo that makes one package-publication gate consume time proportional to the number of packages even though the checks do not share mutable state.
## Decision
[lefthook.yml](../../../../lefthook.yml) keeps one pre-push job named `full check` and runs `pnpm run check:pre-push`. That package script delegates to [scripts/run-gates.ts](../../../../scripts/run-gates.ts), the same bounded scheduler CI uses.
The `pre-push` mode expands into leaf gates for the unit suite, snapshot suite, build, `hygiene` members, `doc-sync` members, and module-graph freshness. The leaf list keeps the same gate vocabulary as the package scripts, including RFC classification and RFC format, while the runner schedules independent checks concurrently and prints one timing/output block per gate.
The build gate makes the hook self-contained from a clean worktree. `publint` and `verify-node-next-types` wait for that build output, while source-only gates continue in parallel.
[scripts/publint-all.ts](../../../../scripts/publint-all.ts) discovers the package list from `packages/<group>/<pkg>` and runs `publint` with a worker pool sized from `availableParallelism()`. `DSH_PUBLINT_CONCURRENCY` can cap or raise the worker count for local machines and CI runners with different resource profiles. Results are buffered per package and printed in deterministic package order, so parallel execution does not scramble each package's log block.
The aggregate package scripts remain the source of truth for ad hoc local runs. The scheduler is a parallel execution plan over their member gates, not a replacement vocabulary.
## Alternatives considered
- **Keep aggregate `hygiene` and `doc-sync` jobs in the hook** - simpler config, but it leaves most of the pre-push wall clock inside serial command chains that lefthook cannot see or schedule.
- **Declare one lefthook job per leaf gate** - exposes parallelism through lefthook's native job model, but it makes the hook file carry a long member list that CI cannot reuse.
- **Require developers to build before pushing** - avoids one hook gate, but it makes `publint` fail in a clean worktree and turns the final local checkpoint into a convention instead of a runnable check.
- **Background subcommands inside shell scripts** - can parallelize work, but it loses lefthook's job names, per-job timing, and failure grouping, and makes signal handling harder to reason about.
- **Declare one publint lefthook job per package** - exposes maximum parallelism, but it turns the hook into a hand-maintained package inventory that drifts exactly when new packages are added.
- **Run publint with unbounded concurrency** - minimizes elapsed time on small machines only by gambling with process count, memory pressure, package tarball creation, and readable logs.
## Consequences
The hook's critical path becomes the slowest real gate instead of the sum of hidden gate chains. Lefthook reports one `full check` job, and the runner reports per-gate timing inside that job, so a slow local checkpoint still points at the gate that dominates the run.
The hook file stays short, and the duplicated member list lives in [scripts/run-gates.ts](../../../../scripts/run-gates.ts), where CI and pre-push can share it. The cost is a custom scheduler script instead of pure lefthook configuration, plus a build in the local pre-push path.
`publint-all.ts` becomes asynchronous code and buffers command output instead of inheriting stdio live. The payoff is package-level parallelism with stable output order and one environment variable for resource tuning.

View File

@@ -12,7 +12,7 @@ This RFC records the decision to add a **second, secret-consuming workflow** tha
## Decision
Add a dedicated workflow, [.github/workflows/e2e.yml](../../../../.github/workflows/e2e.yml), separate from ci.yml. It runs only `pnpm run test:e2e` against the external API using a repo secret, on trusted events, with a preflight that converts a missing secret into a loud failure instead of a false green. ci.yml is left untouched.
Add a dedicated workflow, [.github/workflows/e2e.yml](../../../../.github/workflows/e2e.yml), separate from ci.yml. It runs only `pnpm run test:e2e` against the external API using a repo secret, on trusted events, with a preflight that converts a missing secret into a loud failure instead of a false green. The keyless workflow remains separate so forkable quality gates and secret-consuming real-API gates keep different trigger and credential policies.
### A separate workflow, not a job in ci.yml
@@ -20,7 +20,7 @@ ci.yml's value is that it is keyless, forkable, and always-green: any contributo
### Cost is not the constraint; reliability is
The usual reason to ration real-API CI — token cost — does not apply here: we are DeepSeek and internal inference is effectively free. So the design optimizes for *coverage and signal*, not for minimizing calls. The suite runs in full (all six `*.e2e.ts` files), on multiple triggers, on every trusted PR. This is the CI embodiment of the [docs/testing.md](../../../testing.md) with-key policy.
The usual reason to ration real-API CI — token cost — does not apply here: we are DeepSeek and internal inference is effectively free. So the design optimizes for *coverage and signal*, not for minimizing calls. The suite runs in full (all matching `*.e2e.ts` files), on multiple triggers, on every trusted PR. This is the CI embodiment of the [docs/testing.md](../../../testing.md) with-key policy.
### Triggers: trusted events only
@@ -54,7 +54,7 @@ The repo secret is named `DEEPSEEK_API_KEY_EXTERNAL`; it is mapped to the `DEEPS
### Scope, runtime shape
Run **only** `test:e2e`. The keyless gates (typecheck/lint/coverage/snapshot/build/hygiene) already run in ci.yml on every push and PR; repeating them here would duplicate signal and slow the real-API job. No build step — e2e tests run unbuilt via tsx + the tsconfig paths map. Single Node 24 (the `engines` floor): these tests exercise *API integration*, not node-version compat, which ci.yml's `[24, 26]` matrix already owns; a second Node version would double real-API calls for no added signal. `timeout-minutes: 45` bounds a wedged run given serial files (`fileParallelism: false`), 120s/test, and `retry: 2`. `cancel-in-progress` is enabled only for `pull_request` runs — a superseded PR run is on a stale commit and worth cancelling, whereas a push/schedule run is already producing the post-merge/nightly signal and is never cancelled.
Run **only** `test:e2e`. The keyless gates (typecheck/lint/coverage/snapshot/build/hygiene) already run in ci.yml on every push and PR; repeating them here would duplicate signal and slow the real-API job. No build step — e2e tests run unbuilt via tsx + the tsconfig paths map. Single Node 24 (the `engines` floor): these tests exercise *API integration*, not node-version compat, which ci.yml's Node 24/26 jobs already own; a second Node version would double real-API calls for no added signal. `vitest.e2e.config.ts` runs files through a bounded worker pool (`DSH_E2E_MAX_WORKERS`, default `4`, CI value `14`) so CI and local with-key runs parallelize independent files while retaining a one-line serial escape hatch for quota investigations. `timeout-minutes: 45` bounds a wedged run given 120s/test and `retry: 2`. `cancel-in-progress` is enabled only for `pull_request` runs — a superseded PR run is on a stale commit and worth cancelling, whereas a push/schedule run is already producing the post-merge/nightly signal and is never cancelled.
## Security