Merge pull request #640 from deepseek-harness/fix-update-builderror

fix(build): clean stale workspace residue
This commit is contained in:
Tianyi Cui
2026-07-26 02:45:21 +08:00
committed by GitHub
12 changed files with 258 additions and 11 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-06-17-ts-build-config.md: 1275a635242ea887c941db9dc0554fd33acd4102
2026-06-17-ts-build-config.zh.md: 7691118dd152874e2849f1ace686069c9ea3f61f
2026-06-17-ts-build-config.md: 17036438b83a77f72b49f55abf29632af3f4ffef
2026-06-17-ts-build-config.zh.md: 70e49c61deba418894a48be3016898d1d85c78a0

View File

@@ -43,6 +43,8 @@ In-package relative imports use explicit `.ts` specifiers.
- Referenced package/vendor projects keep the same emit behavior as build, so typecheck refreshes their `lib/types` outputs instead of using a separate no-emit graph. Project-specific strictness changes live in the owning `packages/*/*/tsconfig.json` or `vendor/*/tsconfig.json`.
- The no-emit aggregates disable `rewriteRelativeImportExtensions`; they emit nothing and include tests that import helpers across project-reference boundaries. Package/vendor emit projects keep the rewrite enabled.
Composite projects keep their incremental build information inside their project-local `lib/` output. `pnpm run clean` derives live output directories from the root TypeScript project-reference graph, removes legacy root build information, and removes deleted `packages/*/*` directories that contain only known generated residue. It preserves `node_modules` for every package that still has a `package.json`, and refuses to remove a manifest-less directory containing unknown files. Build does not invoke clean automatically, so ordinary builds retain incremental state.
The command orchestration shape is:
```sh
@@ -55,6 +57,9 @@ tsx scripts/verify-node-next-types.ts
pnpm run typecheck:
tsc -b
pnpm run clean:
tsx scripts/clean.ts
```
`pnpm run demo:*` still runs `src` directly through tsx and root paths, without a compile step.
@@ -63,6 +68,8 @@ tsc -b
- **Keep `tsdown`/oxc as the TypeScript transformer** — oxc's transform is not `tsc` behavior (decorator transform differs, bundled JS differs from per-file emit), and its bundled `.d.ts` conflicts with Cordis' internal relative module augmentation shape.
- **One root strict program over packages, vendor, examples, tests, and scripts** — vendor source triggers type errors outside this project's ownership under the root strict flags; project references with per-project strictness are the boundary that works.
- **Clean before every build** — this would discard the incremental state owned by `tsc` and the bundler even when the workspace layout is unchanged.
- **Remove every package-level `node_modules`** — valid package dependency links do not cause the workspace-discovery failure, and deleting them would turn build cleanup into dependency reinstallation.
## Consequences
@@ -76,5 +83,6 @@ Build responsibilities are clearer:
- `lib/index.*` is the publish runtime output and is generated by the bundler, currently `tsdown`.
- `pnpm run verify-node-next-types` scans built declarations for relative specifiers without file extensions, then typechecks a temporary external ESM consumer with `moduleResolution: "NodeNext"` against the built `types`/`exports` surface, so declaration specifier regressions fail before publish.
- The `typecheck` command uses `tsconfig.json`. Examples, tests, and scripts are checked by the root no-emit project, while packages and vendor modules keep the same emit behavior as `build`. Package and vendor source stays behind project-reference boundaries.
- After changing branches or updating a checkout that deleted packages, contributors can run `pnpm run clean` to remove stale package directories before rebuilding. Unknown files in a manifest-less package directory require manual classification instead of being deleted.
The Cordis vendor copy now has one more type-structure divergence from upstream. During upstream sync, that divergence must be reapplied or explicitly retired.

View File

@@ -43,6 +43,8 @@ Status: implemented
- 被引用的包/vendor 项目保持与构建相同的输出行为,因此类型检查会刷新它们的 `lib/types` 输出,而无需使用独立的 no-emit 图。项目特定的严格度变更放在各自的 `packages/*/*/tsconfig.json``vendor/*/tsconfig.json` 中。
- 两个 no-emit 聚合禁用 `rewriteRelativeImportExtensions`;它们不输出任何文件,且包含跨 project-reference 边界导入 helper 的测试。包/vendor 的 emit 项目保持重写开启。
复合项目将增量构建信息保存在各项目本地的 `lib/` 输出中。`pnpm run clean` 会根据根 TypeScript project-reference 图确定当前有效的输出目录,删除遗留的根目录构建信息,并删除已删除包留下且仅包含已知生成残留的 `packages/*/*` 目录。对于仍有 `package.json` 的每个包,该命令都会保留 `node_modules`;如果不含 `package.json` 的目录中存在未知文件,则拒绝删除。构建不会自动调用 clean因此常规构建会保留增量状态。
命令编排结构如下:
```sh
@@ -55,6 +57,9 @@ tsx scripts/verify-node-next-types.ts
pnpm run typecheck:
tsc -b
pnpm run clean:
tsx scripts/clean.ts
```
`pnpm run demo:*` 仍通过 tsx 和根路径直接运行 `src`,无需编译步骤。
@@ -63,6 +68,8 @@ tsc -b
- **继续使用 `tsdown`/oxc 作为 TypeScript 转换器**oxc 的转换行为与 `tsc` 不同(装饰器转换有差异、打包 JS 与逐文件输出不同),且其打包 `.d.ts` 与 Cordis 内部的相对模块增强结构冲突。
- **用一个根目录严格程序覆盖包、vendor、示例、测试和脚本**vendor 源码在根目录严格标志下会触发不属于本项目所有权范围的类型错误;带有逐项目严格度的 project references 才是可行的边界。
- **每次构建前都执行清理**:即使工作区布局没有变化,这也会丢弃 `tsc` 和打包器拥有的增量状态。
- **删除所有包级 `node_modules`**:有效的包依赖链接不会导致工作区发现失败,而删除这些链接会使构建清理变成重新安装依赖。
## 后果
@@ -76,5 +83,6 @@ tsc -b
- `lib/index.*` 是发布用的运行时输出,由打包器(当前为 `tsdown`)生成。
- `pnpm run verify-node-next-types` 扫描构建出的声明文件,检查是否存在缺少文件扩展名的相对说明符,然后以 `moduleResolution: "NodeNext"` 对构建出的 `types`/`exports` 接口进行临时外部 ESM 消费方的类型检查,确保声明说明符的回归在发布前被捕获。
- `typecheck` 命令使用 `tsconfig.json`。示例、测试和脚本由根 no-emit 项目检查,包和 vendor 模块保持与 `build` 相同的输出行为。包和 vendor 源码始终处于 project-reference 边界之后。
- 切换分支或更新工作副本后,如果其中删除了包,贡献者可在重新构建前运行 `pnpm run clean`,删除残留的包目录。不含 `package.json` 的包目录如果存在未知文件,必须手动判定其类别,不能直接删除。
Cordis 的 vendor 副本现在与上游多了一处类型结构差异。在上游同步时,该差异必须被重新应用或明确废弃。

View File

@@ -50,6 +50,7 @@ Package groups: [packages/README.md](packages/README.md).
```sh
pnpm install # pnpm workspaces, node ^22.19 || >=24
pnpm run clean # remove build outputs and safe residue from deleted packages
pnpm run test # vitest unit tests
pnpm run test:coverage # CI coverage gate: per-file 100% on packages/*/*/src
pnpm run test:e2e # real-API tests; self-skip without DEEPSEEK_API_KEY

View File

@@ -16,7 +16,7 @@
"scripts": {
"build": "tsc -b && tsdown",
"build:web": "pnpm --filter @deepseek-ai/dsh-frontend run build",
"clean:build": "rm -rf .typecheck packages/*/*/lib vendor/*/lib *.tsbuildinfo",
"clean": "tsx scripts/clean.ts",
"typecheck": "tsc -b",
"lint": "eslint .",
"lint:fix": "eslint . --fix",

View File

@@ -3,8 +3,7 @@
"compilerOptions": {
"composite": true,
"rootDir": "src",
"outDir": "lib/types",
"tsBuildInfoFile": "../../../.typecheck/cli-demo.tsbuildinfo"
"outDir": "lib/types"
},
"include": ["src/**/*.ts"],
"references": [

View File

@@ -6,7 +6,7 @@ Local Linux/macOS `node-pty` backend for `ctx.pty`; loading it on another platfo
The plugin injects `pty`, `sandbox`, and `sandboxPolicy`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a local-provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade.
Linux readiness combines a foreground-verified private bash prompt marker, foreground-process-group syscall inspection, silence fallback, and absolute timeout. macOS uses the verified prompt marker plus silence/timeout because it has no `/proc` syscall surface. A marker is not ready until printable prompt text arrives, including when the OSC marker and `PS1` are split across data callbacks; when bash prints the marker before the kernel publishes its return to the foreground process group, polling retains the candidate until bash ownership is observable or the ordinary silence bound expires. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unrecognized or unreadable process state is never a positive exact-idle signal. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason even when its foreground process group is not observable yet; if that close fails, `PtyBackendCleanupError` separately preserves the cleanup failure for registry disposal. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline.
Linux readiness combines a foreground-verified private bash prompt marker, foreground-process-group syscall inspection, silence fallback, and absolute timeout. macOS uses the verified prompt marker plus silence/timeout because it has no `/proc` syscall surface. A marker is not ready until printable prompt text arrives, including when the OSC marker and `PS1` are split across data callbacks; when bash prints the marker before the kernel publishes its return to the foreground process group, polling retains the candidate through one final poll after the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unrecognized or unreadable process state is never a positive exact-idle signal. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason even when its foreground process group is not observable yet; if that close fails, `PtyBackendCleanupError` separately preserves the cleanup failure for registry disposal. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline.
Send cancellation resolves the current foreground process group and delivers a real `SIGINT`; it never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. Close sends `SIGTERM` to descendants, waits, then sends `SIGKILL` to the union of captured survivors and newly scanned descendants so reparenting cannot hide a process from teardown. It verifies that every retained identity is gone or, on Linux, a non-executing zombie before stopping the shell; zombie entries are quiescent and are reaped as the shell exits. A survivor failure does not cache a permanently rejected close; a later close retries the teardown.

View File

@@ -331,8 +331,11 @@ export class LocalPtySession implements PtyBackendSession {
// A prompt candidate can race bash's foreground handoff, but an interactive
// child also inherits PROMPT_COMMAND. Silence therefore remains the bound
// on waiting for shell ownership instead of letting a child marker suppress
// readiness until the absolute timeout.
if (startupHasOutput && Date.now() - this.lastOutputAt >= this.config.idleSilenceMs) {
// readiness until the absolute timeout. One final poll lets a foreground
// handoff coincident with that boundary win before the fallback settles.
const idleFor = Date.now() - this.lastOutputAt
const handoffGrace = this.promptSeen ? this.config.pollIntervalMs : 0
if (startupHasOutput && idleFor >= this.config.idleSilenceMs && idleFor - this.config.idleSilenceMs >= handoffGrace) {
this.settleActive('inferred_idle')
return
}

View File

@@ -298,7 +298,7 @@ describe('LocalPtySession readiness and output', () => {
void operation.done.then(() => { settled = true })
inspector.pgid = 789
terminal.emitData('\x1b]133;D;0\x07dsh> ')
await vi.advanceTimersByTimeAsync(40)
await vi.advanceTimersByTimeAsync(50)
expect(settled).toBe(false)
inspector.pgid = 456

62
scripts/clean.spec.ts Normal file
View File

@@ -0,0 +1,62 @@
import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { dirname, join } from 'node:path'
import { afterEach, describe, expect, it } from 'vitest'
import { RepositoryCleaner } from './clean.ts'
const roots: string[] = []
function fixture(): string {
const root = mkdtempSync(join(tmpdir(), 'dsh-clean-'))
roots.push(root)
return root
}
function write(path: string, content = ''): void {
mkdirSync(dirname(path), { recursive: true })
writeFileSync(path, content)
}
function addProject(root: string, path: string): void {
write(join(root, 'tsconfig.json'), JSON.stringify({ files: [], references: [{ path }] }))
write(join(root, path, 'tsconfig.json'), JSON.stringify({
compilerOptions: { composite: true, outDir: 'lib/types' },
include: ['src'],
}))
write(join(root, path, 'src/index.ts'), 'export {}\n')
}
afterEach(() => {
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true })
})
describe('RepositoryCleaner', () => {
it('derives live build outputs from project references and removes safe stale package residue', async () => {
const root = fixture()
addProject(root, 'products/shell')
write(join(root, 'products/shell/lib/types/index.js'))
write(join(root, 'products/shell/lib/index.js'))
write(join(root, '.typecheck/legacy.tsbuildinfo'))
write(join(root, 'root.tsbuildinfo'))
write(join(root, 'packages/removed/ghost/node_modules/.bin/tool'))
await new RepositoryCleaner(root).clean()
expect(existsSync(join(root, 'products/shell/lib'))).toBe(false)
expect(existsSync(join(root, 'products/shell/src/index.ts'))).toBe(true)
expect(existsSync(join(root, '.typecheck'))).toBe(false)
expect(existsSync(join(root, 'root.tsbuildinfo'))).toBe(false)
expect(existsSync(join(root, 'packages/removed/ghost'))).toBe(false)
})
it('does not delete any target when a manifest-less package contains an unknown file', async () => {
const root = fixture()
addProject(root, 'products/shell')
write(join(root, 'products/shell/lib/types/index.js'))
write(join(root, 'packages/removed/ghost/notes.txt'))
await expect(new RepositoryCleaner(root).clean()).rejects.toThrow('packages/removed/ghost/notes.txt')
expect(existsSync(join(root, 'products/shell/lib'))).toBe(true)
})
})

165
scripts/clean.ts Normal file
View File

@@ -0,0 +1,165 @@
import { lstat, readdir, rm } from 'node:fs/promises'
import { basename, dirname, isAbsolute, join, relative, resolve, sep } from 'node:path'
import { fileURLToPath } from 'node:url'
import ts from 'typescript'
import { repositoryConfigHost } from './ts-project.ts'
const knownOrphanEntries = new Set(['node_modules', 'lib', '.typecheck'])
function isMissing(error: unknown): boolean {
return error instanceof Error && 'code' in error && error.code === 'ENOENT'
}
async function exists(path: string): Promise<boolean> {
try {
await lstat(path)
return true
} catch (error) {
if (isMissing(error)) return false
throw error
}
}
async function childDirectories(path: string): Promise<string[]> {
try {
const entries = await readdir(path, { withFileTypes: true })
return entries.filter(entry => entry.isDirectory()).map(entry => join(path, entry.name))
} catch (error) {
if (isMissing(error)) return []
throw error
}
}
function repositoryPath(root: string, path: string): string {
return relative(root, path).split(sep).join('/')
}
function parseConfig(configPath: string): ts.ParsedCommandLine {
const parsed = ts.getParsedCommandLineOfConfigFile(configPath, {}, repositoryConfigHost)
if (!parsed) throw new Error(`clean: cannot parse TypeScript config ${configPath}`)
if (parsed.errors.length > 0) {
throw new Error(parsed.errors.map(error => ts.flattenDiagnosticMessageText(error.messageText, '\n')).join('\n'))
}
return parsed
}
/** Plans and removes repository-owned build output without crossing the repository boundary. */
export class RepositoryCleaner {
constructor(private readonly root: string) {}
/**
* Remove generated build state and package directories containing only known residue.
* @returns Repository-relative paths that were removed.
*/
async clean(): Promise<string[]> {
const targets = await this.plan()
// Planning validates every target first, so an unsafe orphan prevents all deletion.
for (const target of targets) await rm(target, { recursive: true, force: true })
return targets.map(target => repositoryPath(this.root, target))
}
private async plan(): Promise<string[]> {
const targets = new Set<string>()
const unsafeOrphans: string[] = []
// These checks cover legacy root-level incremental state emitted by older configs.
await this.addIfPresent(targets, join(this.root, '.typecheck'))
for (const entry of await readdir(this.root, { withFileTypes: true })) {
if (entry.isFile() && entry.name.endsWith('.tsbuildinfo')) targets.add(join(this.root, entry.name))
}
// The root project-reference graph is the source of truth for live build targets.
// Each emitting project declares lib/types as outDir; its parent lib also owns
// the sibling runtime bundles, so the complete build output root is removed.
for (const outputDirectory of this.buildOutputDirectories()) {
await this.addIfPresent(targets, outputDirectory)
}
for (const groupDirectory of await childDirectories(join(this.root, 'packages'))) {
for (const packageDirectory of await childDirectories(groupDirectory)) {
// A package.json marks a live package; its output was discovered from the
// project graph above, and its package-local node_modules must be preserved.
if (await exists(join(packageDirectory, 'package.json'))) {
continue
}
// A manifest-less package directory is stale only when every remaining
// entry is known generated residue; unknown files make the whole clean fail.
const entries = await readdir(packageDirectory)
const unknown = entries.filter(entry => !knownOrphanEntries.has(entry) && !entry.endsWith('.tsbuildinfo'))
if (unknown.length > 0) {
unsafeOrphans.push(...unknown.map(entry => repositoryPath(this.root, join(packageDirectory, entry))))
} else {
targets.add(packageDirectory)
}
}
}
if (unsafeOrphans.length > 0) {
throw new Error([
'clean: refusing to remove package directories without package.json; unknown entries remain:',
...unsafeOrphans.sort().map(path => ` ${path}`),
].join('\n'))
}
return [...targets].sort()
}
private buildOutputDirectories(): string[] {
const outputs = new Set<string>()
const pending = [join(this.root, 'tsconfig.json')]
const visited = new Set<string>()
while (pending.length > 0) {
const nextConfigPath = pending.pop()
if (nextConfigPath === undefined) break
const configPath = resolve(nextConfigPath)
if (visited.has(configPath)) continue
visited.add(configPath)
const parsed = parseConfig(configPath)
if (parsed.options.outDir !== undefined) {
const typesDirectory = resolve(parsed.options.outDir)
if (basename(typesDirectory) !== 'types') {
throw new Error(`clean: expected TypeScript outDir to end in /types: ${repositoryPath(this.root, typesDirectory)}`)
}
const outputDirectory = dirname(typesDirectory)
this.assertRepositoryTarget(outputDirectory)
outputs.add(outputDirectory)
}
for (const reference of parsed.projectReferences ?? []) {
pending.push(ts.resolveProjectReferencePath(reference))
}
}
return [...outputs]
}
private assertRepositoryTarget(path: string): void {
const repositoryRelative = relative(this.root, path)
if (repositoryRelative === '' || repositoryRelative === '..' || repositoryRelative.startsWith(`..${sep}`) || isAbsolute(repositoryRelative)) {
throw new Error(`clean: refusing build output outside repository: ${path}`)
}
}
private async addIfPresent(targets: Set<string>, path: string): Promise<void> {
// Missing outputs are normal on a clean checkout; only existing paths become deletion targets.
if (await exists(path)) targets.add(path)
}
}
const scriptPath = fileURLToPath(import.meta.url)
if (process.argv[1] !== undefined && resolve(process.argv[1]) === scriptPath) {
try {
const removed = await new RepositoryCleaner(resolve(dirname(scriptPath), '..')).clean()
if (removed.length === 0) {
console.log('clean: already clean')
} else {
console.log(`clean: removed ${removed.length} paths`)
}
} catch (error) {
console.error(error instanceof Error ? error.message : error)
process.exitCode = 1
}
}

View File

@@ -11,7 +11,8 @@ interface ProjectGraph {
options: ts.CompilerOptions
}
const configHost: ts.ParseConfigFileHost = {
/** TypeScript config host shared by repository scripts. */
export const repositoryConfigHost: ts.ParseConfigFileHost = {
useCaseSensitiveFileNames: ts.sys.useCaseSensitiveFileNames,
readDirectory: (...args) => ts.sys.readDirectory(...args),
fileExists: fileName => ts.sys.fileExists(fileName),
@@ -52,7 +53,7 @@ function loadProjectGraph(projectRoot: string): ProjectGraph {
/** Parse one config file and fail loud on any config diagnostic. */
function parseConfig(configPath: string): ts.ParsedCommandLine {
const parsed = ts.getParsedCommandLineOfConfigFile(configPath, {}, configHost)
const parsed = ts.getParsedCommandLineOfConfigFile(configPath, {}, repositoryConfigHost)
if (!parsed) throw new Error(`cannot parse TypeScript config ${configPath}`)
if (parsed.errors.length > 0) {
throw new Error(parsed.errors.map(error => ts.flattenDiagnosticMessageText(error.messageText, '\n')).join('\n'))