ci: enforce bounded build lanes

This commit is contained in:
Tianyi Cui
2026-07-21 20:15:09 +08:00
parent 99f67adb69
commit f1202d0dd8
16 changed files with 769 additions and 191 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-07-19-package-invariant-runtime-contracts.md: 7d1fb1ad5a2e7563bdddffde1f49368b9f0c13f7
2026-07-19-package-invariant-runtime-contracts.zh.md: 669eb02221aea4b0654497bb81327d725648dabe
2026-07-19-package-invariant-runtime-contracts.md: 40d152b2320ac65f9ea7d8732b1a667236d2780a
2026-07-19-package-invariant-runtime-contracts.zh.md: bd2f440d5dce15b352e7bcea0d1243400d290f11

View File

@@ -59,7 +59,7 @@ Session-backed companions validate existing durable events when they load, using
`verify-package-invariants` discovers every workspace package and enforces companion source, exact-name registration, named-only Loader shape, `./invariant` exports, publication files, dependencies, TypeScript references, and bundle entries. Its AST rule rejects generated markers, default exports, and unexplained empty installers. A non-empty installer must accept and use the failure reporter, and registration must pass that checked local `install` function. The gate deliberately does not infer semantic quality from method names or helper calls.
Vitest mounts `InvariantService` with `{ enabled: true }` for every package test topology and loads the owning companion. The invariant subpath path mapping resolves source companions instead of stale built output. Focused suites cover every executable companion's valid and invalid observations, and the exhaustive topology runs every source companion through the real Loader namespace normalization. An artifact gate stages each package's exact `npm pack` file inventory, imports its compiled `./invariant` self-reference under plain Node, and repeats that Loader-shape check, so an unpublished shared runtime chunk fails before release. Tests that synthesize event streams must produce a valid surrounding lifecycle unless the test is intentionally asserting a violation.
Vitest mounts `InvariantService` with `{ enabled: true }` for every package test topology and loads the owning companion. The invariant subpath path mapping resolves source companions instead of stale built output. Focused suites cover every executable companion's valid and invalid observations, and the exhaustive topology runs every source companion through the real Loader namespace normalization. After the structural gate validates each publication map, an artifact gate stages its manifest-declared `lib/` files, imports the compiled `./invariant` self-reference under plain Node, and repeats that Loader-shape check, so a companion that imports an undeclared runtime chunk fails before release. Tests that synthesize event streams must produce a valid surrounding lifecycle unless the test is intentionally asserting a violation.
## Alternatives considered

View File

@@ -59,7 +59,7 @@ Status: implemented
`verify-package-invariants` 发现每个 workspace 包,并强制 companion 源文件、完整名称注册、仅含具名 export 的 Loader 形状、`./invariant` export、发布文件、依赖、TypeScript reference 和 bundle entry 完整。其 AST 规则拒绝生成标记、默认导出和没有解释的空安装器。非空安装器必须接收并使用失败报告器,注册时还必须传入该经检查的本地 `install` 函数。门禁不会通过方法名或 helper 调用推断语义质量。
Vitest 为每个包测试拓扑使用 `{ enabled: true }` 挂载 `InvariantService`,并加载所有者 companion。不变量 subpath 的 path mapping 会解析源 companion而不是陈旧的构建输出。聚焦 suite 覆盖每个可执行 companion 的有效和无效观测;穷举拓扑通过真实 Loader 命名空间归一化运行每个源 companion。产物门禁会按每个包的精确 `npm pack` 文件清单暂存文件,在 plain Node 下导入该包已编译的 `./invariant` 自引用,并重复执行该 Loader 形状检查;这样,未发布的共享运行时分片会在正式发布前导致门禁失败。合成事件流的测试必须构造有效的外围生命周期,除非测试本身就是在断言违规。
Vitest 为每个包测试拓扑使用 `{ enabled: true }` 挂载 `InvariantService`,并加载所有者 companion。不变量 subpath 的 path mapping 会解析源 companion而不是陈旧的构建输出。聚焦 suite 覆盖每个可执行 companion 的有效和无效观测;穷举拓扑通过真实 Loader 命名空间归一化运行每个源 companion。结构门禁验证每个包的发布映射后,产物门禁会暂存其 manifest元数据清单声明的 `lib/` 文件,在 plain Node 下导入已编译的 `./invariant` 自引用,并重复执行该 Loader 形状检查;这样,若 companion 导入未声明的运行时分片,门禁就会在发布前失败。合成事件流的测试必须构造有效的外围生命周期,除非测试本身就是在断言违规。
## 考虑过的替代方案

View File

@@ -4,36 +4,43 @@ Status: implemented
## Problem
The keyless GitHub CI gates are mostly orthogonal: typecheck, lint, documentation freshness, coverage, snapshot replay, build, package-publication hygiene, demo smoke, and built-bin smoke fail for different reasons and do not need each other's runtime state. Running them as one ordered command chain makes the workflow wall clock equal the sum of those gates, while splitting every leaf gate into its own GitHub job repeats checkout, Node setup, pnpm restore, and install work until orchestration overhead becomes the bottleneck.
The keyless GitHub CI gates are mostly orthogonal: typecheck, lint, documentation freshness, coverage, snapshot replay, build, package-publication hygiene, demo smoke, and built-bin smoke fail for different reasons and do not need each other's runtime state. Running them as one ordered command chain makes the workflow wall clock equal the sum of those gates, while splitting every short leaf into its own GitHub job repeats checkout, Node setup, pnpm restore, and install work until orchestration overhead becomes the bottleneck.
The hard part is the artifact boundary. `publint`, `verify-node-next-types`, and built-bin smoke tests need the built `lib/` outputs, while most gates only need source and dependencies. A blind fan-out either races those artifact consumers before `pnpm run build` has emitted declarations and bundles, or repeats the build in every artifact-dependent job.
The original broad-lane split stopped meeting that balance as the workspace grew. On the merge of PR #404, Linux static, coverage, snapshot, and artifact jobs took 148, 195, 94, and 230 seconds; Windows static and artifacts took 251 and 482 seconds. Package-manager packing once per package dominated both artifact validators, coverage needlessly rebuilt output before a source-only suite, and CPU-heavy gates contended inside the static and coverage lanes.
The artifact boundary remains load-bearing. `publint`, `verify-node-next-types`, compiled invariant loading, and built-bin smoke tests need emitted `lib/` output. Sharding cannot race those consumers ahead of build or replace their published-artifact signal with source execution.
## Decision
[CI](../../../../.github/workflows/ci.yml) groups keyless checks into broad primary-runtime lanes plus a compatibility matrix. The workflow file owns the current lane and runtime inventory.
[CI](../../../../.github/workflows/ci.yml) bounds every non-Windows job to one minute and every Windows job to three minutes. The timeout is an executable regression ceiling; the lane design leaves headroom below it rather than treating a timeout as normal control flow.
Each lane delegates to [scripts/run-gates.ts](../../../../scripts/run-gates.ts), which schedules independent gates with bounded concurrency and prints an attributable result block for each one. Artifact consumers depend on one build within their lane, while compatibility jobs combine typechecking with a real unbuilt worker launch to cover runtime-specific loader behavior.
[scripts/run-gates.ts](../../../../scripts/run-gates.ts) remains the common bounded scheduler, but GitHub supplies explicit shard names for the expensive gate families. [scripts/static-shards.ts](../../../../scripts/static-shards.ts) partitions static gates into foundation, API-contract, catalog, prose, and documentation-site lanes and rejects a missing or duplicate gate assignment. [scripts/coverage-shards.ts](../../../../scripts/coverage-shards.ts) assigns every workspace package to exactly one source-coverage lane; its test expands the live package tree, so a new package makes CI red until it has an owner. Each coverage lane includes only its owned source files, repeats the exhaustive companion topology test, and runs without a preceding build because the complete coverage suite passes from a tree with every generated `lib/` removed.
Generated `.sessions/` logs and `.doc-typecheck-*` temp directories are ignored by lint. The aggregate local CI mode still runs demo smoke after lint, while the split GitHub static lane can run demo smoke directly because lint is isolated in its own lane.
Snapshot replay is four Vitest file shards. Each snapshot job builds the shipped runtime while its Linux runner installs bubblewrap, then runs only its assigned replay files. Static, coverage, and snapshot sharding changes only GitHub scheduling: the ordinary local package scripts still run their complete suites.
Build output is produced once inside the Node 24 artifact lane. The artifact consumers (`publint`, `verify-node-next-types`, and built-bin smoke) declare a dependency on `build`, so there is no upload/download handoff and no consumer can race ahead of declarations or bundles. The CI coverage reporter is text-only while local coverage keeps the HTML report.
Artifacts use three lanes: one metadata lane for `publint`, NodeNext declarations, and compiled invariant loading, plus two Vitest shards for built-bin smoke. Each lane produces its own build before its consumers. Repeating the short build costs runner minutes but avoids an upload/download dependency and keeps each job's critical path bounded.
Both workflows cache the pnpm store. The real-API workflow uses the shared bounded Vitest file pool rather than a separate job per test group.
[scripts/publint-all.ts](../../../../scripts/publint-all.ts) calls publint's supported API in-process against an in-memory publication view made from each manifest's declared files and npm's mandatory metadata files. This preserves the distinction between workspace files and published files without spawning a package-manager pack command 103 times. [scripts/verify-built-package-invariants.mjs](../../../../scripts/verify-built-package-invariants.mjs) stages those structurally validated manifest-declared `lib/` files below the real package, then imports the compiled self-reference through plain Node and Cordis Loader normalization. A companion that reaches an undeclared runtime chunk still fails.
Compatibility lanes run the source worker and Zstandard runtime smokes on every advertised Node line. TypeScript checks the source graph once on the primary Node 24 lane; repeating the same compiler analysis on Node 22 and 26 added time without runtime-specific signal.
The workflow caches the pnpm store, preserves native PowerShell for Windows measurements, and retains one aggregate `all checks passed` status for branch protection. Windows build remains blocking; the wider Windows static, lint, and artifact matrix remains observational while carrying the same three-minute ceiling.
## Alternatives considered
- **Keep the full serial chain in a Node matrix** - simplest to reason about, but it duplicates repo-wide gates that do not produce Node-version-specific signal and leaves every PR waiting for the sum of all gates.
- **Run every gate as a separate GitHub job** - maximizes GitHub-visible fan-out, but it creates too many checks and pays repeated setup/install overhead for gates whose runtime is shorter than the runner preparation.
- **Upload build artifacts to artifact-dependent jobs** - preserves correctness across many jobs, but it adds artifact upload/download time and keeps the workflow wide when the artifact consumers can run behind a local dependency in the primary job.
- **Run `typecheck` and `build` concurrently** - exposes more work to the scheduler, but both commands invoke `tsc -b`; sharing incremental build state between them is a needless race for a small wall-clock gain.
- **Use unbounded real-API e2e parallelism** - rejected because the suite includes many live model/tool scenarios; the worker pool needs an explicit `DSH_E2E_MAX_WORKERS` cap so CI and local runs can fan out without hiding quota or resource problems behind flaky rate-limit failures.
- **Keep the broad lanes and raise timeouts** - minimizes workflow YAML, but it preserves the measured multi-minute feedback loop and offers no regression budget.
- **Run every leaf gate as a separate GitHub job** - maximizes fan-out, but short generators and prose checks would spend more time preparing a runner than checking the repository.
- **Upload one build to artifact consumers** - avoids repeated compilation, but upload/download and dependency scheduling lengthen wall time; the clean build is short enough to repeat inside bounded lanes.
- **Keep package-manager packing in both publication gates** - delegates inventory selection to pnpm, but repeats more than 200 package-manager processes. The manifest structural gate plus publication-view fixtures make the optimized inventory contract explicit and fail on an on-disk but unpublished dependency.
- **Keep build before coverage** - provides emitted output the source suite no longer consumes; a clean-tree coverage proof showed it was pure latency.
- **Typecheck on every Node version** - repeats compiler work while the compatibility smokes already exercise actual Node-specific loading and compression behavior.
## Consequences
PR feedback arrives as a few GitHub checks with structured per-gate log blocks inside each broad job. That keeps runner setup overhead bounded and the Actions UI compact, at the cost of losing one status check per leaf gate.
The Actions UI contains more matrix checks and total runner time can exceed a serial workflow, but PR wall time is the slowest bounded lane instead of the sum of unrelated work. Repeated setup and builds are the deliberate price of sub-minute non-Windows feedback and sub-three-minute Windows feedback.
The broad-lane split repeats checkout, setup, and install more often than a single primary job. That setup cost is intentional: on GitHub's hosted runner, running lint, coverage, and snapshot replay in one process pool oversubscribes CPU badly enough that the single-job critical path is longer than the repeated setup.
Shard inventories are repository contracts. Static selection validates the complete live gate list at runtime, coverage tests validate exhaustive package ownership, and Vitest owns deterministic file sharding for snapshots and built-bin smokes. Adding a gate or package therefore requires an explicit scheduling decision instead of silently lengthening an existing lane.
The split introduces a maintenance obligation: when `package.json` adds or removes a gate that belongs in CI, [scripts/run-gates.ts](../../../../scripts/run-gates.ts) needs the matching leaf. That obligation is intentional because the runner is the parallel execution plan for the same gate vocabulary, not a separate quality policy.
The optimized publication validators rely on the manifest `files` contract enforced by `verify-package-invariants`. If publication rules grow beyond that contract, the structural gate and both staged views must change together.
The compatibility signal is narrower than the primary Node 24 signal. It proves that the source graph typechecks and that the real unbuilt workflow-worker launch path executes on every advertised runtime line without doubling documentation, coverage, publication, snapshot replay, and unrelated smoke checks whose failures are not expected to vary by Node version.
Compatibility jobs no longer claim that TypeScript itself was exercised under every Node runtime. They prove runtime-sensitive source loading on Node 22, 24, and 26, while the primary runtime owns the single source-graph typecheck.

View File

@@ -8,7 +8,7 @@ The pre-push hook is the last local checkpoint before a branch leaves the machin
Flattening those members directly into `lefthook.yml` solves the local hook only. CI has the same scheduling problem, and duplicating a long leaf list in YAML gives future script changes two places to drift.
`publint` has the same shape one level lower. Each package is linted independently against its own manifest and built output, but the runner loops through every package in order. On this repo that makes one package-publication gate consume time proportional to the number of packages even though the checks do not share mutable state.
`publint` has the same shape one level lower. Each package is linted independently against its own manifest and built output, but invoking the CLI separately also asks the package manager to compute the same manifest-bounded publication view once per package. On this repo process and packing overhead dominate the publication checks.
## Decision
@@ -18,7 +18,7 @@ The `pre-push` mode expands into leaf gates for the unit suite, snapshot suite,
The build gate makes the hook self-contained from a clean worktree. `publint`, `verify-node-next-types`, and the pre-push form of `doc-typecheck` wait for that build output, while source-only gates continue in parallel.
[scripts/publint-all.ts](../../../../scripts/publint-all.ts) discovers the package list from `packages/<group>/<pkg>` and runs `publint` with a worker pool sized from `availableParallelism()`. `DSH_PUBLINT_CONCURRENCY` can cap or raise the worker count for local machines and CI runners with different resource profiles. Results are buffered per package and printed in deterministic package order, so parallel execution does not scramble each package's log block.
[scripts/publint-all.ts](../../../../scripts/publint-all.ts) discovers the package list from `packages/<group>/<pkg>` and calls publint's supported API against an in-memory view of each manifest's declared publication files plus npm's mandatory metadata files. That keeps unpublished workspace files invisible to publint without a package-manager subprocess per package. A worker pool sized from `availableParallelism()` bounds parallel file loading and linting; `DSH_PUBLINT_CONCURRENCY` can cap or raise it, and results print in deterministic package order.
The aggregate package scripts remain the source of truth for ad hoc local runs. The scheduler is a parallel execution plan over their member gates, not a replacement vocabulary.
@@ -29,7 +29,7 @@ The aggregate package scripts remain the source of truth for ad hoc local runs.
- **Require developers to build before pushing** - avoids one hook gate, but it makes `publint` fail in a clean worktree and turns the final local checkpoint into a convention instead of a runnable check.
- **Background subcommands inside shell scripts** - can parallelize work, but it loses lefthook's job names, per-job timing, and failure grouping, and makes signal handling harder to reason about.
- **Declare one publint lefthook job per package** - exposes maximum parallelism, but it turns the hook into a hand-maintained package inventory that drifts exactly when new packages are added.
- **Run publint with unbounded concurrency** - minimizes elapsed time on small machines only by gambling with process count, memory pressure, package tarball creation, and readable logs.
- **Run publint with unbounded concurrency** - minimizes elapsed time on small machines only by gambling with file descriptors, memory pressure, and readable logs.
## Consequences
@@ -37,4 +37,4 @@ The hook's critical path becomes the slowest real gate instead of the sum of hid
The hook file stays short, and the duplicated member list lives in [scripts/run-gates.ts](../../../../scripts/run-gates.ts), where CI and pre-push can share it. The cost is a custom scheduler script instead of pure lefthook configuration, plus a build in the local pre-push path.
`publint-all.ts` becomes asynchronous code and buffers command output instead of inheriting stdio live. The payoff is package-level parallelism with stable output order and one environment variable for resource tuning.
`publint-all.ts` becomes asynchronous code and formats API results after each package completes. The payoff is package-level parallelism with stable output order, one environment variable for resource tuning, and no repeated package-manager packing.

View File

@@ -19,45 +19,128 @@ jobs:
node-24:
runs-on: ubuntu-latest
name: node 24 / ${{ matrix.lane }}
timeout-minutes: 1
env:
DSH_GATE_CONCURRENCY: ${{ matrix.gate_concurrency }}
DSH_PUBLINT_CONCURRENCY: ${{ matrix.publint_concurrency }}
DSH_ARTIFACT_SHARD: ${{ matrix.artifact_shard }}
DSH_COVERAGE_MAX_WORKERS: ${{ matrix.coverage_max_workers }}
DSH_COVERAGE_SHARD: ${{ matrix.coverage_shard }}
DSH_STATIC_SHARD: ${{ matrix.static_shard }}
DSH_SNAPSHOT_PREBUILT: ${{ matrix.snapshot_prebuilt }}
DSH_SNAPSHOT_SHARD: ${{ matrix.snapshot_shard }}
DSH_ESLINT_CACHE: ${{ matrix.eslint_cache }}
strategy:
fail-fast: false
matrix:
include:
- lane: static
- lane: static-foundation
command: pnpm run check:ci:static
gate_concurrency: '4'
publint_concurrency: '8'
coverage_max_workers: ''
eslint_cache: ''
static_shard: foundation
- lane: static-api-contracts
command: pnpm run check:ci:static
gate_concurrency: '4'
static_shard: api-contracts
- lane: static-catalogs
command: pnpm run check:ci:static
gate_concurrency: '4'
static_shard: catalogs
- lane: static-prose
command: pnpm run check:ci:static
gate_concurrency: '4'
static_shard: prose
- lane: static-site
command: pnpm run check:ci:static
gate_concurrency: '1'
static_shard: site
- lane: lint
command: pnpm run check:ci:lint
gate_concurrency: '1'
publint_concurrency: '8'
coverage_max_workers: ''
eslint_cache: '1'
- lane: coverage
- lane: coverage-spine
command: pnpm run check:ci:coverage
gate_concurrency: '1'
publint_concurrency: '8'
coverage_max_workers: '4'
eslint_cache: ''
- lane: snapshot
coverage_shard: spine
- lane: coverage-sdk
command: pnpm run check:ci:coverage
gate_concurrency: '1'
coverage_max_workers: '4'
coverage_shard: sdk
- lane: coverage-interfaces
command: pnpm run check:ci:coverage
gate_concurrency: '1'
coverage_max_workers: '4'
coverage_shard: interfaces
- lane: coverage-execution
command: pnpm run check:ci:coverage
gate_concurrency: '1'
coverage_max_workers: '4'
coverage_shard: execution
- lane: coverage-orchestration
command: pnpm run check:ci:coverage
gate_concurrency: '1'
coverage_max_workers: '4'
coverage_shard: orchestration
- lane: coverage-infrastructure
command: pnpm run check:ci:coverage
gate_concurrency: '1'
coverage_max_workers: '4'
coverage_shard: infrastructure
- lane: coverage-session-state
command: pnpm run check:ci:coverage
gate_concurrency: '1'
coverage_max_workers: '4'
coverage_shard: session-state
- lane: coverage-hooks-claude
command: pnpm run check:ci:coverage
gate_concurrency: '1'
coverage_max_workers: '4'
coverage_shard: hooks-claude
- lane: coverage-hooks-codex
command: pnpm run check:ci:coverage
gate_concurrency: '1'
coverage_max_workers: '4'
coverage_shard: hooks-codex
- lane: coverage-capabilities
command: pnpm run check:ci:coverage
gate_concurrency: '1'
coverage_max_workers: '4'
coverage_shard: capabilities
- lane: snapshot-1
command: pnpm run check:ci:snapshot
gate_concurrency: '1'
publint_concurrency: '8'
coverage_max_workers: ''
eslint_cache: ''
- lane: artifacts
snapshot_prebuilt: '1'
snapshot_shard: '1/4'
- lane: snapshot-2
command: pnpm run check:ci:snapshot
gate_concurrency: '1'
snapshot_prebuilt: '1'
snapshot_shard: '2/4'
- lane: snapshot-3
command: pnpm run check:ci:snapshot
gate_concurrency: '1'
snapshot_prebuilt: '1'
snapshot_shard: '3/4'
- lane: snapshot-4
command: pnpm run check:ci:snapshot
gate_concurrency: '1'
snapshot_prebuilt: '1'
snapshot_shard: '4/4'
- lane: artifacts-metadata
command: pnpm run check:ci:artifacts
gate_concurrency: '3'
publint_concurrency: '8'
coverage_max_workers: ''
eslint_cache: ''
artifact_shard: metadata
- lane: artifacts-smoke-1
command: pnpm run check:ci:artifacts
gate_concurrency: '1'
artifact_shard: smoke-1
- lane: artifacts-smoke-2
command: pnpm run check:ci:artifacts
gate_concurrency: '1'
artifact_shard: smoke-2
steps:
- uses: actions/checkout@v6
@@ -82,19 +165,30 @@ jobs:
- name: Install (immutable)
run: pnpm install --frozen-lockfile
# The snapshot lane REPLAYS the sandbox example's recorded scenarios,
# The snapshot lanes REPLAY the sandbox example's recorded scenarios,
# re-executing their bash calls under a real runner. ubuntu-latest has
# no bubblewrap preinstalled and no built Landlock launcher, so without
# this the confined executions fail closed (SANDBOX_UNAVAILABLE). Same
# install as sandbox.yml's bwrap leg (incl. the Ubuntu 24.04 AppArmor
# userns knob).
- name: Install bubblewrap (unrestrict userns)
if: matrix.lane == 'snapshot'
# userns knob). Building does not depend on bubblewrap, so overlap them.
- name: Prepare built snapshot runtime and bubblewrap
if: startsWith(matrix.lane, 'snapshot-')
run: |
sudo apt-get update -q
sudo apt-get install -yq bubblewrap
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 \
|| echo "apparmor userns knob absent — the functional probe decides"
pnpm run build &
build_pid=$!
(
sudo apt-get update -q
sudo apt-get install -yq bubblewrap
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 \
|| echo "apparmor userns knob absent — the functional probe decides"
) &
sandbox_pid=$!
build_status=0
sandbox_status=0
wait "$build_pid" || build_status=$?
wait "$sandbox_pid" || sandbox_status=$?
if (( build_status != 0 )); then exit "$build_status"; fi
exit "$sandbox_status"
- uses: actions/cache@v4
if: matrix.lane == 'lint'
@@ -110,12 +204,20 @@ jobs:
node-compat:
runs-on: ubuntu-latest
name: node ${{ matrix.node }}
timeout-minutes: 1
env:
DSH_GATE_CONCURRENCY: '2'
DSH_NODE_COMPAT_SKIP_TYPECHECK: ${{ matrix.skip_typecheck }}
strategy:
fail-fast: false
matrix:
node: ['22.19', 24, 26]
include:
- node: '22.19'
skip_typecheck: '1'
- node: 24
skip_typecheck: ''
- node: 26
skip_typecheck: '1'
steps:
- uses: actions/checkout@v6
@@ -146,6 +248,7 @@ jobs:
python-sdk:
runs-on: ubuntu-latest
name: python 3.10 / keyless SDK
timeout-minutes: 1
steps:
- uses: actions/checkout@v6
@@ -166,6 +269,7 @@ jobs:
windows-build:
runs-on: windows-2025
name: windows / build
timeout-minutes: 3
steps:
- uses: actions/checkout@v6
@@ -191,29 +295,54 @@ jobs:
continue-on-error: true
runs-on: windows-2025
name: windows node 24 / ${{ matrix.lane }}
timeout-minutes: 3
env:
DSH_GATE_CONCURRENCY: ${{ matrix.gate_concurrency }}
DSH_PUBLINT_CONCURRENCY: ${{ matrix.publint_concurrency }}
DSH_ARTIFACT_SHARD: ${{ matrix.artifact_shard }}
DSH_STATIC_SHARD: ${{ matrix.static_shard }}
DSH_ESLINT_CACHE: ${{ matrix.eslint_cache }}
strategy:
fail-fast: false
matrix:
include:
- lane: static
- lane: static-foundation
command: pnpm run check:ci:static
gate_concurrency: '4'
publint_concurrency: '8'
eslint_cache: ''
static_shard: foundation
- lane: static-api-contracts
command: pnpm run check:ci:static
gate_concurrency: '4'
static_shard: api-contracts
- lane: static-catalogs
command: pnpm run check:ci:static
gate_concurrency: '4'
static_shard: catalogs
- lane: static-prose
command: pnpm run check:ci:static
gate_concurrency: '4'
static_shard: prose
- lane: static-site
command: pnpm run check:ci:static
gate_concurrency: '1'
static_shard: site
- lane: lint
command: pnpm run check:ci:lint
gate_concurrency: '1'
publint_concurrency: '8'
eslint_cache: '1'
- lane: artifacts
- lane: artifacts-metadata
command: pnpm run check:ci:artifacts
gate_concurrency: '3'
publint_concurrency: '8'
eslint_cache: ''
artifact_shard: metadata
- lane: artifacts-smoke-1
command: pnpm run check:ci:artifacts
gate_concurrency: '1'
artifact_shard: smoke-1
- lane: artifacts-smoke-2
command: pnpm run check:ci:artifacts
gate_concurrency: '1'
artifact_shard: smoke-2
steps:
- uses: actions/checkout@v6
@@ -271,6 +400,7 @@ jobs:
all-checks-passed:
name: all checks passed
runs-on: ubuntu-latest
timeout-minutes: 1
needs: [node-24, node-compat, python-sdk, windows-build]
if: always()
steps:

View File

@@ -391,6 +391,9 @@ describe('LocalSkillProvider', () => {
await empty.plugin(SkillService)
SkillLocal.apply(empty, {})
expect(await empty.skills.list()).toEqual([])
delete process.env.DSH_AGENTS_HOME
expect(new SkillLocal.LocalSkillProvider(empty, { dshHome: join(envHome, 'empty-dsh') }).name).toBe('local')
} finally {
if (previousDshHome === undefined) {
delete process.env.DSH_HOME

View File

@@ -0,0 +1,43 @@
import { readdirSync } from 'node:fs'
import { resolve } from 'node:path'
import { describe, expect, it } from 'vitest'
import { coverageArgs, coverageShards } from './coverage-shards.ts'
const repositoryRoot = resolve(import.meta.dirname, '..')
describe('coverage shards', () => {
it('assigns every workspace package to exactly one lane', () => {
const packagesRoot = resolve(repositoryRoot, 'packages')
const workspacePackages = readdirSync(packagesRoot, { withFileTypes: true })
.filter(group => group.isDirectory())
.flatMap(group => readdirSync(resolve(packagesRoot, group.name), { withFileTypes: true })
.filter(entry => entry.isDirectory())
.map(entry => `${group.name}/${entry.name}`))
.sort()
const assignedPackages = coverageShards.flatMap(shard => shard.packageRoots.flatMap((packageRoot) => {
if (packageRoot.includes('/')) return [packageRoot]
return readdirSync(resolve(packagesRoot, packageRoot), { withFileTypes: true })
.filter(entry => entry.isDirectory())
.map(entry => `${packageRoot}/${entry.name}`)
}))
expect([...assignedPackages].sort()).toEqual(workspacePackages)
expect(new Set(assignedPackages).size).toBe(assignedPackages.length)
})
it.each(coverageShards)('selects tests and source includes for $name', (shard) => {
const args = coverageArgs(shard.name)
for (const packageRoot of shard.packageRoots) {
expect(args).toContain(`packages/${packageRoot}`)
expect(args).toContain(packageRoot.includes('/')
? `--coverage.include=packages/${packageRoot}/src/**/*.ts`
: `--coverage.include=packages/${packageRoot}/*/src/**/*.ts`)
}
expect(args).toContain('scripts/test-invariants.spec.ts')
expect(new Set(args).size).toBe(args.length)
})
it('rejects an unknown lane', () => {
expect(() => coverageArgs('missing')).toThrow('unknown DSH_COVERAGE_SHARD')
})
})

View File

@@ -0,0 +1,68 @@
/** Coverage shard definitions for the GitHub Actions source-test lanes. */
/** A coverage lane that owns complete package roots and optional cross-package tests. */
export interface CoverageShard {
/** Stable lane identifier passed through `DSH_COVERAGE_SHARD`. */
name: string
/** Group or package paths below `packages/` whose tests and source coverage belong to the lane. */
packageRoots: readonly string[]
/** Additional test roots needed for cross-package behavior or repository scripts. */
extraTestRoots?: readonly string[]
}
/** Exhaustive, non-overlapping ownership of workspace packages in coverage CI. */
export const coverageShards = [
{
name: 'spine',
packageRoots: ['core', 'llm', 'compact', 'context'],
extraTestRoots: ['packages/examples/cli-demo/tests'],
},
{ name: 'sdk', packageRoots: ['sdk'] },
{
name: 'interfaces',
packageRoots: ['ui', 'examples', 'goal'],
extraTestRoots: ['examples'],
},
{ name: 'execution', packageRoots: ['fs', 'bash', 'sandbox', 'code-runtime'] },
{ name: 'orchestration', packageRoots: ['workflow', 'subagent', 'tasks'] },
{
name: 'infrastructure',
packageRoots: ['cordis', 'support', 'lsp', 'mcp'],
extraTestRoots: ['scripts'],
},
{
name: 'session-state',
packageRoots: ['session-persistence', 'session-query'],
},
{ name: 'hooks-claude', packageRoots: ['hooks/hook-protocol', 'hooks/hooks-claude'] },
{ name: 'hooks-codex', packageRoots: ['hooks/hooks-codex'] },
{
name: 'capabilities',
packageRoots: ['web', 'skill', 'spill', 'util', 'guard', 'todo', 'timeout'],
},
] as const satisfies readonly CoverageShard[]
/**
* Build Vitest filters and coverage include globs for one source-test lane.
*
* @param name Stable shard name from {@link coverageShards}.
* @returns Positional test roots followed by per-group coverage include flags.
*/
export function coverageArgs(name: string): string[] {
const shard = coverageShards.find(candidate => candidate.name === name)
if (shard === undefined) {
throw new Error(`run-gates: unknown DSH_COVERAGE_SHARD ${JSON.stringify(name)}.`)
}
const testRoots = new Set([
...shard.packageRoots.map(packageRoot => `packages/${packageRoot}`),
...('extraTestRoots' in shard ? shard.extraTestRoots : []),
'scripts/test-invariants.spec.ts',
])
return [
...testRoots,
...shard.packageRoots.map(packageRoot => packageRoot.includes('/')
? `--coverage.include=packages/${packageRoot}/src/**/*.ts`
: `--coverage.include=packages/${packageRoot}/*/src/**/*.ts`),
]
}

View File

@@ -0,0 +1,61 @@
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { spawnSync } from 'node:child_process'
import { afterEach, describe, expect, it } from 'vitest'
const repositoryRoot = fileURLToPath(new URL('..', import.meta.url))
const runner = fileURLToPath(new URL('./publint-all.ts', import.meta.url))
const roots: string[] = []
afterEach(() => {
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true })
})
function fixture(exportPath = './lib/index.js'): string {
const root = mkdtempSync(join(tmpdir(), 'dsh-publint-all-'))
roots.push(root)
const packageDir = join(root, 'packages/core/probe')
mkdirSync(join(packageDir, 'lib'), { recursive: true })
writeFileSync(join(packageDir, 'package.json'), `${JSON.stringify({
name: '@deepseek-ai/dsh-probe',
version: '0.0.1',
type: 'module',
license: 'MIT',
engines: { node: '>=22.19' },
sideEffects: false,
files: ['lib'],
exports: { '.': { default: exportPath } },
}, null, 2)}\n`)
writeFileSync(join(packageDir, 'README.md'), '# Probe\n')
writeFileSync(join(packageDir, 'lib/index.js'), 'export const probe = true\n')
writeFileSync(join(packageDir, 'unpublished.js'), 'export const hidden = true\n')
return root
}
function run(root: string) {
return spawnSync(process.execPath, [
'--import', 'tsx', runner,
'--packages-root', root,
], {
cwd: repositoryRoot,
encoding: 'utf8',
timeout: 5_000,
})
}
describe('publint package runner', () => {
it('lints recursively declared files from an in-memory publication view', () => {
const result = run(fixture())
expect(result.status, result.stderr).toBe(0)
expect(result.stdout).toContain('linting 1 package(s)')
expect(result.stdout).toContain('All good!')
})
it('rejects an export that exists in the workspace but is not published', () => {
const result = run(fixture('./unpublished.js'))
expect(result.status).toBe(1)
expect(result.stdout).toContain('unpublished.js')
})
})

View File

@@ -1,46 +1,53 @@
import { execFile } from 'node:child_process'
import { existsSync, readdirSync } from 'node:fs'
/** Run publint over the exact manifest-declared publication view of every package. */
import {
globSync,
readFileSync,
readdirSync,
statSync,
} from 'node:fs'
import { availableParallelism } from 'node:os'
import { resolve } from 'node:path'
import { promisify } from 'node:util'
import { dirname, relative, resolve, sep } from 'node:path'
import { publint, type Message, type PackFile } from 'publint'
import { formatMessage } from 'publint/utils'
const execFileAsync = promisify(execFile)
const CONCURRENCY_ENV = 'DSH_PUBLINT_CONCURRENCY'
const repositoryRoot = resolve(import.meta.dirname, '..')
const options = parseOptions(process.argv.slice(2))
const packagesRoot = resolve(options.get('--packages-root') ?? repositoryRoot)
// Discover harness packages at packages/<group>/<pkg>; group containers,
// examples, and private vendored sources are not package targets.
const root = resolve(import.meta.dirname, '..')
const packagesRoot = resolve(root, 'packages')
interface PackageTarget {
path: string
directory: string
manifest: PackageManifest
}
// Run publint's JS CLI through the current node, not the .bin shim: the
// extensionless shim isn't spawnable on Windows (CVE-2024-27980) and the .cmd
// variant needs shell:true, which space-joins args UNESCAPED (DEP0190) and
// breaks when the repo path contains spaces. The JS entry is identical on every
// platform (`bin` is `./src/cli.js` per publint's package.json).
const publintCli = resolve(root, 'node_modules/publint/src/cli.js')
interface PackageManifest {
name?: string
files?: unknown
}
type PublintResult =
| { path: string; status: 'passed'; stdout: string; stderr: string }
| { path: string; status: 'failed'; stdout: string; stderr: string; message: string }
| { path: string; status: 'passed'; messages: Message[]; manifest: Record<string, unknown> }
| { path: string; status: 'failed'; messages: Message[]; manifest: Record<string, unknown>; failure?: string }
function workspacePackages(): string[] {
return readdirSync(packagesRoot, { withFileTypes: true })
.filter(group => group.isDirectory())
.flatMap(group =>
readdirSync(resolve(packagesRoot, group.name), { withFileTypes: true })
.filter(pkg => pkg.isDirectory())
.filter(pkg => existsSync(resolve(packagesRoot, group.name, pkg.name, 'package.json')))
.map(pkg => `packages/${group.name}/${pkg.name}`),
)
function workspacePackages(): PackageTarget[] {
return globSync('packages/*/*/package.json', { cwd: packagesRoot })
.sort()
.map((manifestPath) => {
const absoluteManifestPath = resolve(packagesRoot, manifestPath)
const manifest = JSON.parse(readFileSync(absoluteManifestPath, 'utf8')) as PackageManifest
return { path: dirname(manifestPath), directory: dirname(absoluteManifestPath), manifest }
})
}
function publintConcurrency(total: number): number {
if (total === 0) return 0
const raw = process.env[CONCURRENCY_ENV]
if (raw !== undefined) {
if (raw !== undefined && raw !== '') {
const parsed = Number.parseInt(raw, 10)
if (!Number.isSafeInteger(parsed) || parsed < 1) {
if (!Number.isSafeInteger(parsed) || parsed < 1 || String(parsed) !== raw) {
throw new Error(`publint-all: ${CONCURRENCY_ENV} must be a positive integer, got ${JSON.stringify(raw)}.`)
}
return Math.min(total, parsed)
@@ -49,57 +56,106 @@ function publintConcurrency(total: number): number {
return Math.min(total, availableParallelism())
}
function outputText(value: unknown): string {
if (typeof value === 'string') return value
if (Buffer.isBuffer(value)) return value.toString()
return ''
function publicationFiles(target: PackageTarget): PackFile[] {
const paths = new Set<string>()
addPath(resolve(target.directory, 'package.json'), paths)
const declared = Array.isArray(target.manifest.files)
? target.manifest.files.filter((value): value is string => typeof value === 'string')
: []
for (const pattern of [
...declared,
'README*',
'LICENSE*',
'LICENCE*',
'CHANGELOG*',
'CHANGES*',
'HISTORY*',
'NOTICE*',
]) {
for (const match of globSync(pattern, { cwd: target.directory })) {
addPath(resolve(target.directory, match), paths)
}
}
return [...paths]
.sort()
.map(path => ({
name: `package/${relative(target.directory, path).split(sep).join('/')}`,
data: readFileSync(path),
}))
}
async function runPublint(path: string): Promise<PublintResult> {
function addPath(path: string, paths: Set<string>): void {
const stat = statSync(path)
if (stat.isDirectory()) {
for (const entry of readdirSync(path)) addPath(resolve(path, entry), paths)
} else if (stat.isFile()) {
paths.add(path)
}
}
async function runPublint(target: PackageTarget): Promise<PublintResult> {
try {
const { stdout, stderr } = await execFileAsync(process.execPath, [publintCli, path], {
cwd: root,
encoding: 'utf8',
maxBuffer: 10 * 1024 * 1024,
const result = await publint({
pkgDir: 'package',
pack: { files: publicationFiles(target) },
})
return { path, status: 'passed', stdout, stderr }
const manifest = result.pkg as Record<string, unknown>
return result.messages.some(message => message.type === 'error')
? { path: target.path, status: 'failed', messages: result.messages, manifest }
: { path: target.path, status: 'passed', messages: result.messages, manifest }
} catch (error: unknown) {
const failed = error as { stdout?: unknown; stderr?: unknown; message?: string }
return {
path,
path: target.path,
status: 'failed',
stdout: outputText(failed.stdout),
stderr: outputText(failed.stderr),
message: failed.message ?? 'publint failed',
messages: [],
manifest: target.manifest as Record<string, unknown>,
failure: error instanceof Error ? error.message : String(error),
}
}
}
async function runAll(paths: string[], concurrency: number): Promise<PublintResult[]> {
async function runAll(targets: PackageTarget[], concurrency: number): Promise<PublintResult[]> {
let next = 0
const results: Array<PublintResult | undefined> = []
await Promise.all(Array.from({ length: concurrency }, async () => {
for (;;) {
const index = next
next += 1
const path = paths[index]
if (path === undefined) return
results[index] = await runPublint(path)
const target = targets[index]
if (target === undefined) return
results[index] = await runPublint(target)
}
}))
return paths.map((path, index) => {
return targets.map((target, index) => {
const result = results[index]
if (result === undefined) throw new Error(`publint-all: missing result for ${path}.`)
if (result === undefined) throw new Error(`publint-all: missing result for ${target.path}.`)
return result
})
}
function printResult(result: PublintResult): void {
console.log(`Running publint for ${result.path}...`)
process.stdout.write(result.stdout)
process.stderr.write(result.stderr)
if (result.status === 'failed') console.error(result.message)
if ('failure' in result) console.error(result.failure)
for (const message of result.messages) {
console.log(formatMessage(message, result.manifest, { color: false }) ?? message.code)
}
if (result.status === 'passed' && result.messages.length === 0) console.log('All good!')
}
function parseOptions(args: string[]): Map<string, string> {
const parsed = new Map<string, string>()
for (let index = 0; index < args.length; index += 2) {
const name = args[index]
const value = args[index + 1]
if (name !== '--packages-root' || value === undefined || value.startsWith('--')) {
throw new Error(`publint-all: expected [--packages-root PATH], got ${JSON.stringify(args)}.`)
}
if (parsed.has(name)) throw new Error(`publint-all: duplicate option ${name}.`)
parsed.set(name, value)
}
return parsed
}
const packages = workspacePackages()

View File

@@ -8,6 +8,8 @@ import { spawn } from 'node:child_process'
import { availableParallelism } from 'node:os'
import { resolve } from 'node:path'
import { performance } from 'node:perf_hooks'
import { coverageArgs } from './coverage-shards.ts'
import { selectStaticGates } from './static-shards.ts'
type Mode =
| 'ci-primary'
@@ -161,20 +163,16 @@ function gatesForMode(selected: Mode): Gate[] {
pnpmScript('duplication', 'duplication'),
]
case 'ci-coverage':
return [
pnpmScript('build', 'build'),
coverageGate(),
]
return [coverageGate()]
case 'ci-snapshot':
return [
pnpmScript('build', 'build'),
snapshotGate(),
]
return flagEnabled('DSH_SNAPSHOT_PREBUILT')
? [snapshotGate([])]
: [pnpmScript('build', 'build'), snapshotGate()]
case 'ci-artifacts':
return ciArtifactGates()
case 'node-compat':
return [
pnpmScript('typecheck', 'typecheck'),
...flagEnabled('DSH_NODE_COMPAT_SKIP_TYPECHECK') ? [] : [pnpmScript('typecheck', 'typecheck')],
pnpmExec('source-worker-smoke', [
'vitest',
'run',
@@ -230,7 +228,7 @@ function ciPrimaryGates(): Gate[] {
}
function ciStaticGates(): Gate[] {
return [
const gates = [
pnpmScript('runtime-closure', 'verify-runtime-closure', { label: 'runtime closure' }),
pnpmScript('constraints', 'constraints'),
pnpmScript('package-invariants', 'verify-package-invariants', { label: 'package invariants' }),
@@ -239,10 +237,12 @@ function ciStaticGates(): Gate[] {
pnpmScript('module-graph', 'verify-module-graph', { label: 'module graph' }),
pnpmScript('knip', 'knip'),
]
return selectStaticGates(gates, process.env.DSH_STATIC_SHARD)
}
function ciArtifactGates(): Gate[] {
return [
const shard = process.env.DSH_ARTIFACT_SHARD
const metadataGates = [
pnpmScript('build', 'build'),
pnpmScript('publint', 'publint', { needs: ['build'] }),
pnpmScript('node-next-types', 'verify-node-next-types', {
@@ -250,8 +250,14 @@ function ciArtifactGates(): Gate[] {
needs: ['build'],
}),
builtPackageInvariantsGate(['build']),
builtBinSmokeGate(),
]
if (shard === 'metadata') return metadataGates
if (shard === 'smoke-1') return [pnpmScript('build', 'build'), builtBinSmokeGate('1/2')]
if (shard === 'smoke-2') return [pnpmScript('build', 'build'), builtBinSmokeGate('2/2')]
if (shard !== undefined && shard !== '') {
throw new Error(`run-gates: unknown DSH_ARTIFACT_SHARD ${JSON.stringify(shard)}.`)
}
return [...metadataGates, builtBinSmokeGate()]
}
function lintGate(): Gate {
@@ -275,25 +281,36 @@ function lintGate(): Gate {
}
function coverageGate(): Gate {
const shard = process.env.DSH_COVERAGE_SHARD
return pnpmExec('coverage', [
'vitest',
'run',
'--coverage',
...(shard === undefined || shard === '' ? [] : coverageArgs(shard)),
...positiveIntArg('DSH_COVERAGE_MAX_WORKERS', '--maxWorkers'),
], {
label: 'test:coverage',
env: { DSH_EXAMPLE_MODE: 'lib' },
needs: ['build'],
})
}
// The snapshot suite boots the example bins in `lib` mode (built artifact under plain Node,
// plugins via real exports) — CI and pre-push already build, so they exercise what ships rather
// than the tsx/source path dev uses. It therefore waits on `build`.
function snapshotGate(): Gate {
return pnpmScript('snapshot', 'test:snapshot', {
function snapshotGate(needs: string[] = ['build']): Gate {
const shard = process.env.DSH_SNAPSHOT_SHARD
if (shard !== undefined && shard !== '' && !/^\d+\/\d+$/.test(shard)) {
throw new Error(`run-gates: DSH_SNAPSHOT_SHARD must be INDEX/TOTAL, got ${JSON.stringify(shard)}.`)
}
return pnpmExec('snapshot', [
'vitest',
'run',
'--config',
'vitest.snapshot.config.ts',
...(shard === undefined || shard === '' ? [] : [`--shard=${shard}`]),
], {
label: 'test:snapshot',
env: { DSH_EXAMPLE_MODE: 'lib' },
needs: ['build'],
...needs.length === 0 ? {} : { needs },
})
}
@@ -314,6 +331,13 @@ function positiveIntArg(envName: string, flag: string): string[] {
return [`${flag}=${raw}`]
}
function flagEnabled(envName: string): boolean {
const raw = process.env[envName]
if (raw === undefined || raw === '') return false
if (raw !== '1') throw new Error(`run-gates: ${envName} must be 1 when set, got ${JSON.stringify(raw)}.`)
return true
}
function hygieneLeafGates(options: { artifactNeeds?: string[] } = {}): Gate[] {
const artifactOptions = options.artifactNeeds === undefined ? {} : { needs: options.artifactNeeds }
return [
@@ -363,7 +387,7 @@ function docSyncLeafGates(options: {
]
}
function builtBinSmokeGate(): Gate {
function builtBinSmokeGate(shard?: string): Gate {
return pnpmExec('built-bin-smoke', [
'vitest',
'run',
@@ -379,6 +403,7 @@ function builtBinSmokeGate(): Gate {
// (the e2e lane runs unbuilt, so these files self-skip there).
'packages/workflow/workflow-workerthread/tests/built-worker.e2e.ts',
'packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts',
...(shard === undefined ? [] : [`--shard=${shard}`]),
], {
label: 'built-bin smoke',
needs: ['build'],

View File

@@ -0,0 +1,16 @@
import { describe, expect, it } from 'vitest'
import { selectStaticGates, staticShards } from './static-shards.ts'
const completeInventory = staticShards.flatMap(shard => shard.gateIds).map(id => ({ id }))
describe('static gate shards', () => {
it.each(staticShards)('selects only the gates owned by $name', (shard) => {
expect(selectStaticGates(completeInventory, shard.name).map(gate => gate.id)).toEqual(shard.gateIds)
})
it('rejects missing, duplicate, and unknown assignments', () => {
expect(() => selectStaticGates(completeInventory.slice(1))).toThrow('assign every static gate exactly once')
expect(() => selectStaticGates([...completeInventory, completeInventory[0]!])).toThrow('static gate IDs must be unique')
expect(() => selectStaticGates(completeInventory, 'missing')).toThrow('unknown DSH_STATIC_SHARD')
})
})

77
scripts/static-shards.ts Normal file
View File

@@ -0,0 +1,77 @@
/** Static-gate shard definitions for GitHub Actions. */
/** A static CI lane identified by the gate IDs it owns. */
export interface StaticShard {
/** Stable lane identifier passed through `DSH_STATIC_SHARD`. */
name: string
/** Gate IDs selected from the static gate inventory. */
gateIds: readonly string[]
}
/** Exhaustive, non-overlapping ownership of static CI gates. */
export const staticShards = [
{
name: 'foundation',
gateIds: [
'runtime-closure',
'constraints',
'package-invariants',
'cordis-config',
'module-graph',
'knip',
],
},
{
name: 'api-contracts',
gateIds: ['doc-typecheck', 'export-jsdoc', 'scoped-events', 'type-equivalence'],
},
{
name: 'catalogs',
gateIds: ['cordis-catalog', 'tool-catalog', 'config-catalog', 'persistence-catalog', 'doc-graphs'],
},
{
name: 'prose',
gateIds: [
'markdown-wrap',
'markdown-links',
'doc-refs',
'package-paths',
'package-readme-model-experience',
'mermaid',
'agent-note-classification',
'agent-note-format',
'translation-prompt',
'translation-pairing',
'doc-budgets',
'package-readme-limitations',
],
},
{ name: 'site', gateIds: ['docs-site'] },
] as const satisfies readonly StaticShard[]
/**
* Validate the complete gate partition and optionally select one lane.
*
* @param gates Complete static gate inventory.
* @param name Optional stable shard name.
* @returns All gates when no shard is requested, otherwise the selected lane.
*/
export function selectStaticGates<T extends { id: string }>(gates: readonly T[], name?: string): T[] {
const gateIds = gates.map(gate => gate.id)
const assignedIds = staticShards.flatMap(shard => shard.gateIds)
const uniqueGateIds = new Set<string>(gateIds)
const uniqueAssignedIds = new Set<string>(assignedIds)
if (uniqueGateIds.size !== gateIds.length) throw new Error('run-gates: static gate IDs must be unique.')
if (uniqueAssignedIds.size !== assignedIds.length) throw new Error('run-gates: static shard gate IDs must be unique.')
if (gateIds.length !== assignedIds.length
|| gateIds.some(id => !uniqueAssignedIds.has(id))
|| assignedIds.some(id => !uniqueGateIds.has(id))) {
throw new Error('run-gates: static shards must assign every static gate exactly once.')
}
if (name === undefined || name === '') return [...gates]
const shard = staticShards.find(candidate => candidate.name === name)
if (shard === undefined) throw new Error(`run-gates: unknown DSH_STATIC_SHARD ${JSON.stringify(name)}.`)
const selectedIds = new Set<string>(shard.gateIds)
return gates.filter(gate => selectedIds.has(gate.id))
}

View File

@@ -1,102 +1,106 @@
/** Verify every packed companion through its package self-reference under plain Node. */
/** Verify every compiled companion through its staged package self-reference under plain Node. */
import { spawnSync } from 'node:child_process'
import {
copyFileSync,
cpSync,
existsSync,
globSync,
mkdirSync,
mkdtempSync,
readFileSync,
rmSync,
writeFileSync,
} from 'node:fs'
import { dirname, resolve } from 'node:path'
import { pathToFileURL } from 'node:url'
const root = resolve(import.meta.dirname, '..')
const loaderUrl = pathToFileURL(resolve(root, 'vendor/loader/lib/index.js')).href
const repositoryRoot = resolve(import.meta.dirname, '..')
const options = parseOptions(process.argv.slice(2))
const packagesRoot = resolve(options.get('--packages-root') ?? repositoryRoot)
const loaderUrl = options.get('--loader-url')
?? pathToFileURL(resolve(repositoryRoot, 'vendor/loader/lib/index.js')).href
const failures = []
const manifests = globSync('packages/*/*/package.json', { cwd: root }).sort()
const packArgs = ['pack', '--dry-run', '--json', '--ignore-scripts']
// Windows cannot spawn npm's .cmd shim directly; setup-node installs this JS
// entrypoint beside node.exe, so the probe stays shell-free on every runner.
const npmInvocation = process.platform === 'win32'
? [process.execPath, [resolve(dirname(process.execPath), 'node_modules/npm/bin/npm-cli.js'), ...packArgs]]
: ['npm', packArgs]
const manifests = globSync('packages/*/*/package.json', { cwd: packagesRoot }).sort()
const { default: Loader } = await import(loaderUrl)
const loader = Object.create(Loader.prototype)
for (const manifestPath of manifests) {
const packageDir = dirname(resolve(root, manifestPath))
const manifest = JSON.parse(readFileSync(resolve(root, manifestPath), 'utf8'))
const packageDir = dirname(resolve(packagesRoot, manifestPath))
const manifest = JSON.parse(readFileSync(resolve(packagesRoot, manifestPath), 'utf8'))
const packageName = manifest.name
if (typeof packageName !== 'string' || packageName.length === 0) {
failures.push(`${manifestPath}: missing package name`)
continue
}
const pack = spawnSync(npmInvocation[0], npmInvocation[1], {
cwd: packageDir,
encoding: 'utf8',
})
if (pack.status !== 0) {
const detail = pack.error?.message
?? (pack.stderr.trim() || pack.stdout.trim() || `npm pack exited ${pack.status}`)
failures.push(`${packageName}: ${detail}`)
const invariantExport = manifest.exports?.['./invariant']
if (typeof invariantExport !== 'object'
|| invariantExport.default !== './lib/invariant.js'
|| !manifest.files?.includes('lib/invariant.js')) {
failures.push(`${packageName}: manifest does not publish ./lib/invariant.js as ./invariant`)
continue
}
let files
try {
const result = JSON.parse(pack.stdout)
files = result[0]?.files
if (!Array.isArray(files)) throw new Error('npm pack returned no file inventory')
} catch (error) {
failures.push(`${packageName}: cannot parse npm pack inventory: ${String(error)}`)
continue
}
// Keep the packed view below its owning package so Node reaches the real
// Keep the staged view below its owning package so Node reaches the real
// pnpm dependency links. Junctioning node_modules elsewhere breaks pnpm's
// relative workspace links on Windows.
const stagedPackageDir = mkdtempSync(resolve(packageDir, '.dsh-packed-invariant-'))
// relative workspace links on Windows. Copy only the statically required
// runtime entry so a companion that imports an undeclared chunk fails here.
const stagedPackageDir = mkdtempSync(resolve(packageDir, '.dsh-built-invariant-'))
try {
for (const file of files) {
if (typeof file.path !== 'string'
|| (file.path !== 'package.json' && !file.path.startsWith('lib/'))) continue
const target = resolve(stagedPackageDir, file.path)
mkdirSync(dirname(target), { recursive: true })
copyFileSync(resolve(packageDir, file.path), target)
}
const probe = `
const companion = await import(${JSON.stringify(`${packageName}/invariant`)});
const { default: Loader } = await import(${JSON.stringify(loaderUrl)});
if ('default' in companion) throw new Error('companion has a default export');
const loader = Object.create(Loader.prototype);
const unwrapped = loader.unwrapExports(companion);
if (unwrapped !== companion) throw new Error('Loader collapsed the companion namespace');
if (typeof unwrapped.name !== 'string') throw new Error('companion name is missing');
if (!Array.isArray(unwrapped.inject) || !unwrapped.inject.includes('invariants')) {
throw new Error('companion does not inject invariants');
}
if (typeof unwrapped.apply !== 'function') throw new Error('companion apply is missing');
`
const result = spawnSync(process.execPath, ['--input-type=module', '--eval', probe], {
cwd: stagedPackageDir,
encoding: 'utf8',
})
if (result.status !== 0) {
const detail = result.error?.message
?? (result.stderr.trim() || result.stdout.trim() || `node exited ${result.status}`)
failures.push(`${packageName}: ${detail}`)
copyFileSync(resolve(packageDir, 'package.json'), resolve(stagedPackageDir, 'package.json'))
copyDeclaredLibFiles(packageDir, stagedPackageDir, manifest.files)
const probePath = resolve(stagedPackageDir, 'probe.mjs')
writeFileSync(
probePath,
`import * as companion from ${JSON.stringify(`${packageName}/invariant`)}\nexport default companion\n`,
)
const { default: companion } = await import(pathToFileURL(probePath).href)
if ('default' in companion) throw new Error('companion has a default export')
const unwrapped = loader.unwrapExports(companion)
if (unwrapped !== companion) throw new Error('Loader collapsed the companion namespace')
if (typeof unwrapped.name !== 'string') throw new Error('companion name is missing')
if (!Array.isArray(unwrapped.inject) || !unwrapped.inject.includes('invariants')) {
throw new Error('companion does not inject invariants')
}
if (typeof unwrapped.apply !== 'function') throw new Error('companion apply is missing')
} catch (error) {
failures.push(`${packageName}: ${error instanceof Error ? error.message : String(error)}`)
} finally {
rmSync(stagedPackageDir, { recursive: true, force: true })
}
}
if (failures.length > 0) {
console.error('verify-built-package-invariants: packed companion failures:')
console.error('verify-built-package-invariants: compiled companion failures:')
for (const failure of failures) console.error(` ${failure}`)
process.exit(1)
}
console.log(`verify-built-package-invariants: ${manifests.length} packed companion(s) passed plain-Node Loader checks.`)
console.log(`verify-built-package-invariants: ${manifests.length} compiled companion(s) passed plain-Node Loader checks.`)
function parseOptions(args) {
const allowed = new Set(['--packages-root', '--loader-url'])
const parsed = new Map()
for (let index = 0; index < args.length; index += 2) {
const name = args[index]
const value = args[index + 1]
if (!allowed.has(name) || value === undefined || value.startsWith('--')) {
throw new Error(`verify-built-package-invariants: expected [--packages-root PATH] [--loader-url URL], got ${JSON.stringify(args)}.`)
}
if (parsed.has(name)) throw new Error(`verify-built-package-invariants: duplicate option ${name}.`)
parsed.set(name, value)
}
return parsed
}
function copyDeclaredLibFiles(packageDir, stagedPackageDir, files) {
for (const pattern of files) {
if (!pattern.startsWith('lib/')) continue
for (const relativePath of globSync(pattern, { cwd: packageDir })) {
const source = resolve(packageDir, relativePath)
if (!existsSync(source)) continue
const target = resolve(stagedPackageDir, relativePath)
mkdirSync(dirname(target), { recursive: true })
cpSync(source, target, { recursive: true })
}
}
}

View File

@@ -0,0 +1,88 @@
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { fileURLToPath, pathToFileURL } from 'node:url'
import { spawnSync } from 'node:child_process'
import { afterEach, describe, expect, it } from 'vitest'
const verifier = fileURLToPath(new URL('./verify-built-package-invariants.mjs', import.meta.url))
const roots: string[] = []
afterEach(() => {
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true })
})
function fixture(options: {
invariantSource?: string
invariantExport?: string
runtimeChunk?: string
} = {}): { root: string; loaderUrl: string } {
const root = mkdtempSync(join(tmpdir(), 'dsh-built-package-invariants-'))
roots.push(root)
const packageDir = join(root, 'packages/core/probe')
mkdirSync(join(packageDir, 'lib'), { recursive: true })
writeFileSync(join(packageDir, 'package.json'), `${JSON.stringify({
name: '@deepseek-ai/dsh-probe',
type: 'module',
files: ['lib/invariant.js'],
exports: {
'./invariant': {
default: options.invariantExport ?? './lib/invariant.js',
},
},
}, null, 2)}\n`)
writeFileSync(
join(packageDir, 'lib/invariant.js'),
options.invariantSource ?? "export const name = 'probe-invariant'\nexport const inject = ['invariants']\nexport const apply = () => {}\n",
)
if (options.runtimeChunk !== undefined) {
writeFileSync(join(packageDir, 'lib/chunk.js'), options.runtimeChunk)
}
const loaderPath = join(root, 'loader.mjs')
writeFileSync(loaderPath, 'export default class Loader { unwrapExports(value) { return value } }\n')
return { root, loaderUrl: pathToFileURL(loaderPath).href }
}
function verify(root: string, loaderUrl: string) {
return spawnSync(process.execPath, [
verifier,
'--packages-root', root,
'--loader-url', loaderUrl,
], {
encoding: 'utf8',
timeout: 5_000,
})
}
describe('built package invariant verifier', () => {
it('loads the staged compiled self-reference through plain Node and Loader normalization', () => {
const { root, loaderUrl } = fixture()
const result = verify(root, loaderUrl)
expect(result.status, result.stderr).toBe(0)
expect(result.stdout).toContain('1 compiled companion(s) passed plain-Node Loader checks')
})
it('rejects a default export and a broken invariant export map', () => {
const withDefault = fixture({
invariantSource: "export default {}\nexport const name = 'probe-invariant'\nexport const inject = ['invariants']\nexport const apply = () => {}\n",
})
const defaultResult = verify(withDefault.root, withDefault.loaderUrl)
expect(defaultResult.status).toBe(1)
expect(defaultResult.stderr).toContain('companion has a default export')
const brokenExport = fixture({ invariantExport: './lib/missing.js' })
const exportResult = verify(brokenExport.root, brokenExport.loaderUrl)
expect(exportResult.status).toBe(1)
expect(exportResult.stderr).toContain('@deepseek-ai/dsh-probe')
})
it('rejects an invariant bundle that needs an unstaged runtime chunk', () => {
const { root, loaderUrl } = fixture({
invariantSource: "export * from './chunk.js'\n",
runtimeChunk: "export const name = 'probe-invariant'\nexport const inject = ['invariants']\nexport const apply = () => {}\n",
})
const result = verify(root, loaderUrl)
expect(result.status).toBe(1)
expect(result.stderr).toContain('chunk.js')
})
})