mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
cleanup(build): drop helper architecture parsing
This commit is contained in:
Tianyi Cui
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-29-persistent-bash-str-replace-editor.md
|
||||
2026-07-29-persistent-bash-str-replace-editor.md: aee566f2adee94cea88b2ab802d95e405ee8b26c
|
||||
2026-07-29-persistent-bash-str-replace-editor.zh.md: 86a4b11ff18ec8c4ef276565f242f60270c4bfb0
|
||||
2026-07-29-persistent-bash-str-replace-editor.md: a97af750bdd80ddf38dc2d126e70c245ed035f35
|
||||
2026-07-29-persistent-bash-str-replace-editor.zh.md: 0c2ab26693d90c91d5c41a128ebb77a3c6cc2e7f
|
||||
|
||||
@@ -16,7 +16,7 @@ Some deployments need a one-call Bash schema whose shell state survives across m
|
||||
|
||||
`dsh-system-prompt` accepts `includeHarnessIdentity: false`, while `dsh-agent-spine-demo` forwards that setting and accepts `toolBash: false`. A deployment can therefore own an exact persona and replace the spine's native Bash without duplicate prompt or tool registrations. Existing defaults remain unchanged.
|
||||
|
||||
Both plugins are included in the Python runtime closure. The persistent Bash closure also includes the PTY service/local backend and the sandbox services required by that backend. Because `node-pty` executes a native `spawn-helper` on macOS, each packaged macOS runtime executable ships with an architecture-matched `-spawn-helper` sibling; Linux uses `forkpty` directly. A pinned `node-pty` patch checks `DSH_NODE_PTY_SPAWN_HELPER` first, so it remains a true override for a current external consumer that supplies a non-sibling helper. When the override is unset, the patch resolves the packaged executable sibling if present and otherwise preserves upstream lookup in ordinary Node runs. The macOS executable and runtime-wheel builders inspect the thin Mach-O header and fail before publication when the helper is absent, mismatched, or not executable.
|
||||
Both plugins are included in the Python runtime closure. The persistent Bash closure also includes the PTY service/local backend and the sandbox services required by that backend. Because `node-pty` executes a native `spawn-helper` on macOS, each packaged macOS runtime executable ships with a `-spawn-helper` sibling; Linux uses `forkpty` directly. A pinned `node-pty` patch checks `DSH_NODE_PTY_SPAWN_HELPER` first, so it remains a true override for a current external consumer that supplies a non-sibling helper. When the override is unset, the patch resolves the packaged executable sibling if present and otherwise preserves upstream lookup in ordinary Node runs. The macOS builders fail before publication when the helper is absent or not executable.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@
|
||||
|
||||
`dsh-system-prompt` 接受 `includeHarnessIdentity: false`；`dsh-agent-spine-demo` 会转发该设置，并接受 `toolBash: false`。因此部署可以拥有精确 persona，并替换 spine 的原生 Bash，而不会重复注册提示词或工具。既有默认值不变。
|
||||
|
||||
两个插件都进入 Python runtime 闭包。持久 Bash 的闭包还包含 PTY 服务／本地后端，以及该后端要求的沙箱服务。由于 `node-pty` 在 macOS 上会执行原生 `spawn-helper`，每个打包后的 macOS 运行时可执行文件都会携带一个架构匹配的 `-spawn-helper` 伴随文件；Linux 直接使用 `forkpty`。固定版本的 `node-pty` 补丁会先检查 `DSH_NODE_PTY_SPAWN_HELPER`，因此对当前提供非伴随 helper 的外部消费方而言，该变量仍是真正的覆盖项。未设置该覆盖时，补丁会在打包可执行文件的伴随文件存在时解析它，否则在普通 Node 运行中保留上游查找方式。macOS 可执行文件与运行时 wheel 包的构建器会检查 thin Mach-O 文件头；若 helper 缺失、架构不匹配或不可执行，构建会在发布前失败。
|
||||
两个插件都进入 Python runtime 闭包。持久 Bash 的闭包还包含 PTY 服务／本地后端，以及该后端要求的沙箱服务。由于 `node-pty` 在 macOS 上会执行原生 `spawn-helper`，每个打包后的 macOS 运行时可执行文件都会携带一个 `-spawn-helper` 伴随文件；Linux 直接使用 `forkpty`。固定版本的 `node-pty` 补丁会先检查 `DSH_NODE_PTY_SPAWN_HELPER`，因此对当前提供非伴随 helper 的外部消费方而言，该变量仍是真正的覆盖项。未设置该覆盖时，补丁会在打包可执行文件的伴随文件存在时解析它，否则在普通 Node 运行中保留上游查找方式。若 helper 缺失或不可执行，macOS 构建器会在发布前失败。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write python/sdk-runtime/README.md
|
||||
README.md: efdb5cf9f87e0831ef09a47e6ffdb99254a17f36
|
||||
README.zh.md: cafac7418608c416a9f291d62442c32b8787b1fc
|
||||
README.md: 07bb3c574b3cd49f1dc74f0e9d9bd1bb7ca9b216
|
||||
README.zh.md: 9eb03352505f7cad3e6bf6f253ed6564fdb06ca0
|
||||
|
||||
@@ -8,7 +8,7 @@ Runtime carrier package for the Python SDK (dist `deepseek-harness-runtime-bin`,
|
||||
|
||||
Two carriers coexist under `src/deepseek_harness_runtime/runtime/`, both injected by the repo's `scripts/build-exe-for-python-sdk.ts` build and both gitignored:
|
||||
|
||||
- **exe (production)** — a single-file Node executable `dsh-jsonrpc-agent-pkg-<platform>-<arch>` (platform: `linux`/`macos`; arch: `x64`/`arm64`). macOS builds also ship the native `-spawn-helper` sibling that `node-pty` uses there, and its thin Mach-O header must match the target. No Node installation is needed on the target machine. This is the only carrier that ships in wheel distributions; this package does not publish sdists.
|
||||
- **exe (production)** — a single-file Node executable `dsh-jsonrpc-agent-pkg-<platform>-<arch>` (platform: `linux`/`macos`; arch: `x64`/`arm64`). macOS builds also ship the native `-spawn-helper` sibling that `node-pty` uses there. No Node installation is needed on the target machine. This is the only carrier that ships in wheel distributions; this package does not publish sdists.
|
||||
- **node (dev-only)** — the full deploy closure under `runtime/node/` (`package.json` + `node_modules/`), executed as `node runtime/node/node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/bin.js` on a system Node >= 22.19. It is the current checkout's source build, meant for repo-local development and verification only; it is never selected automatically and is excluded from distributions.
|
||||
|
||||
Both carriers hold the same content, defined once: the [package.json](package.json) at this package's root is the deploy root of the single-exe pipeline — a pure dependency manifest (no code of its own) whose dependency closure IS both the plugin set compiled into the exe and the tree materialized into `runtime/node/`. Adding a plugin to the distribution means adding one dependency line there and rebuilding.
|
||||
|
||||
@@ -8,7 +8,7 @@ Python SDK 的运行时载体包（分发名 `deepseek-harness-runtime-bin`，
|
||||
|
||||
两种载体并存于 `src/deepseek_harness_runtime/runtime/` 之下，均由仓库的 `scripts/build-exe-for-python-sdk.ts` 构建注入，且均被 git 忽略：
|
||||
|
||||
- **exe（生产）**——单文件 Node 可执行程序 `dsh-jsonrpc-agent-pkg-<platform>-<arch>`（platform：`linux`/`macos`；arch：`x64`/`arm64`）。macOS 构建还会随附 `node-pty` 在该平台使用的原生 `-spawn-helper` 伴随文件，其 thin Mach-O 文件头必须与目标匹配。目标机器无需安装 Node。这是唯一随 wheel 包分发的载体；本包不发布 sdist。
|
||||
- **exe（生产）**——单文件 Node 可执行程序 `dsh-jsonrpc-agent-pkg-<platform>-<arch>`（platform：`linux`/`macos`；arch：`x64`/`arm64`）。macOS 构建还会随附 `node-pty` 在该平台使用的原生 `-spawn-helper` 伴随文件。目标机器无需安装 Node。这是唯一随 wheel 包分发的载体；本包不发布 sdist。
|
||||
- **`node`（仅限开发）**——`runtime/node/` 下的完整部署闭包（`package.json` + `node_modules/`），在系统 Node >= 22.19 上以 `node runtime/node/node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/bin.js` 执行。它是当前检出的源码构建，仅用于仓库本地的开发与验证；不会被自动选中，也不进入分发物。
|
||||
|
||||
两种载体承载相同的内容，且只定义一次：本包根目录的 [package.json](package.json) 是 single-exe 流水线的部署根目录——一份零代码的纯依赖 manifest，其依赖闭包既是编译进 exe 的插件集，也是物化到 `runtime/node/` 的文件树。往分发物里加插件，就是在那里加一行依赖再重新构建。
|
||||
|
||||
@@ -16,26 +16,6 @@ _PLATFORMS = {
|
||||
_SPAWN_HELPER_SUFFIX = "-spawn-helper"
|
||||
|
||||
|
||||
def _spawn_helper_binary_target(header: bytes) -> str | None:
|
||||
if len(header) >= 8 and header[:4] == b"\xcf\xfa\xed\xfe":
|
||||
cpu_type = int.from_bytes(header[4:8], "little")
|
||||
if cpu_type == 0x01000007:
|
||||
return "macos-x64"
|
||||
if cpu_type == 0x0100000C:
|
||||
return "macos-arm64"
|
||||
return None
|
||||
|
||||
|
||||
def _validate_spawn_helper(path: Path, expected_target: str) -> None:
|
||||
with path.open("rb") as helper:
|
||||
actual_target = _spawn_helper_binary_target(helper.read(8))
|
||||
if actual_target != expected_target:
|
||||
raise RuntimeError(
|
||||
f"runtime spawn helper binary mismatch: expected {expected_target}, "
|
||||
f"found {actual_target or 'unsupported format or architecture'} at {path}"
|
||||
)
|
||||
|
||||
|
||||
def _host_platform_tag() -> str:
|
||||
machine = platform.machine().lower()
|
||||
arch = "arm64" if machine in {"arm64", "aarch64"} else "x64" if machine in {"x86_64", "amd64"} else machine
|
||||
@@ -86,9 +66,6 @@ class RuntimeBuildHook(BuildHookInterface):
|
||||
for executable in [executables[0], *helpers]:
|
||||
if executable.stat().st_mode & stat.S_IXUSR == 0:
|
||||
raise RuntimeError(f"runtime executable is not executable: {executable}")
|
||||
if helpers:
|
||||
_validate_spawn_helper(helpers[0], expected_target)
|
||||
|
||||
build_data["pure_python"] = False
|
||||
build_data["infer_tag"] = False
|
||||
build_data["tag"] = f"py3-none-{platform_tag}"
|
||||
|
||||
@@ -16,14 +16,6 @@ SCRIPT = ROOT / "scripts" / "build-python-release.py"
|
||||
build_python_release = SimpleNamespace(**runpy.run_path(str(SCRIPT)))
|
||||
|
||||
|
||||
def helper_header(target: str) -> bytes:
|
||||
header = bytearray(8)
|
||||
header[:4] = b"\xcf\xfa\xed\xfe"
|
||||
cpu_type = 0x01000007 if target == "macos-x64" else 0x0100000C
|
||||
header[4:8] = cpu_type.to_bytes(4, "little")
|
||||
return bytes(header)
|
||||
|
||||
|
||||
def test_repository_version_matches_root_package_json() -> None:
|
||||
expected = json.loads((ROOT / "package.json").read_text())["version"]
|
||||
|
||||
@@ -53,7 +45,7 @@ def test_stage_runtime_copies_executable_and_spawn_helper(tmp_path: Path) -> Non
|
||||
executable.write_bytes(b"runtime")
|
||||
executable.chmod(0o755)
|
||||
spawn_helper = Path(f"{executable}-spawn-helper")
|
||||
spawn_helper.write_bytes(helper_header("macos-arm64"))
|
||||
spawn_helper.write_bytes(b"helper")
|
||||
spawn_helper.chmod(0o751)
|
||||
destination = tmp_path / "staging"
|
||||
|
||||
@@ -67,7 +59,7 @@ def test_stage_runtime_copies_executable_and_spawn_helper(tmp_path: Path) -> Non
|
||||
runtime_dir = destination / "src" / "deepseek_harness_runtime" / "runtime"
|
||||
assert (runtime_dir / executable.name).read_bytes() == b"runtime"
|
||||
copied_helper = runtime_dir / spawn_helper.name
|
||||
assert copied_helper.read_bytes() == helper_header("macos-arm64")
|
||||
assert copied_helper.read_bytes() == b"helper"
|
||||
assert copied_helper.stat().st_mode & stat.S_IXUSR
|
||||
|
||||
|
||||
@@ -120,42 +112,3 @@ def test_stage_runtime_copies_linux_executable_without_spawn_helper(
|
||||
runtime_dir = destination / "src" / "deepseek_harness_runtime" / "runtime"
|
||||
runtime_files = [path.name for path in runtime_dir.glob("dsh-jsonrpc-agent-pkg-*")]
|
||||
assert runtime_files == [executable.name]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("target", ["macos-x64", "macos-arm64"])
|
||||
def test_spawn_helper_binary_target(target: str) -> None:
|
||||
assert build_python_release.spawn_helper_binary_target(helper_header(target)) == target
|
||||
|
||||
|
||||
def test_stage_runtime_rejects_mismatched_spawn_helper(tmp_path: Path) -> None:
|
||||
executable = tmp_path / "dsh-jsonrpc-agent-pkg-macos-arm64"
|
||||
executable.write_bytes(b"runtime")
|
||||
executable.chmod(0o755)
|
||||
spawn_helper = Path(f"{executable}-spawn-helper")
|
||||
spawn_helper.write_bytes(helper_header("macos-x64"))
|
||||
spawn_helper.chmod(0o755)
|
||||
|
||||
with pytest.raises(ValueError, match="expected macos-arm64, found macos-x64"):
|
||||
build_python_release.stage_runtime(
|
||||
tmp_path / "staging",
|
||||
"1.2.3",
|
||||
executable,
|
||||
executable.name,
|
||||
)
|
||||
|
||||
|
||||
def test_stage_runtime_rejects_non_binary_spawn_helper(tmp_path: Path) -> None:
|
||||
executable = tmp_path / "dsh-jsonrpc-agent-pkg-macos-arm64"
|
||||
executable.write_bytes(b"runtime")
|
||||
executable.chmod(0o755)
|
||||
spawn_helper = Path(f"{executable}-spawn-helper")
|
||||
spawn_helper.write_bytes(b"helper")
|
||||
spawn_helper.chmod(0o755)
|
||||
|
||||
with pytest.raises(ValueError, match="unsupported format or architecture"):
|
||||
build_python_release.stage_runtime(
|
||||
tmp_path / "staging",
|
||||
"1.2.3",
|
||||
executable,
|
||||
executable.name,
|
||||
)
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
*/
|
||||
|
||||
import { spawn } from 'node:child_process'
|
||||
import { existsSync, mkdirSync, readFileSync, statSync } from 'node:fs'
|
||||
import { existsSync, mkdirSync, statSync } from 'node:fs'
|
||||
import { chmod, copyFile, mkdir, readFile, rm, writeFile } from 'node:fs/promises'
|
||||
import { basename, dirname, join, resolve, sep } from 'node:path'
|
||||
import { parseArgs } from 'node:util'
|
||||
@@ -58,16 +58,6 @@ interface RuntimeProduct {
|
||||
spawnHelper?: string
|
||||
}
|
||||
|
||||
function spawnHelperBinaryTarget(path: string): string | undefined {
|
||||
const header = readFileSync(path).subarray(0, 8)
|
||||
if (header.length >= 8 && header.readUInt32LE(0) === 0xfeedfacf) {
|
||||
const cpuType = header.readUInt32LE(4)
|
||||
if (cpuType === 0x01000007) return 'macos-x64'
|
||||
if (cpuType === 0x0100000c) return 'macos-arm64'
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
function runtimeProductFiles(product: RuntimeProduct): string[] {
|
||||
return [product.executable, ...(product.spawnHelper === undefined ? [] : [product.spawnHelper])]
|
||||
}
|
||||
@@ -393,17 +383,7 @@ class SingleExeBuild {
|
||||
+ `checked ${candidates.join(', ')}. Build each runtime on its target platform and architecture.`,
|
||||
)
|
||||
}
|
||||
if (statSync(helper).mode & 0o111) {
|
||||
const expected = `${target.platform}-${target.arch}`
|
||||
const actual = spawnHelperBinaryTarget(helper)
|
||||
if (actual !== expected) {
|
||||
throw new Error(
|
||||
`build-exe-for-python-sdk: node-pty spawn-helper binary mismatch: expected ${expected}, `
|
||||
+ `found ${actual ?? 'unsupported format or architecture'} at ${helper}`,
|
||||
)
|
||||
}
|
||||
return helper
|
||||
}
|
||||
if (statSync(helper).mode & 0o111) return helper
|
||||
throw new Error(`build-exe-for-python-sdk: node-pty spawn-helper is not executable: ${helper}`)
|
||||
}
|
||||
|
||||
|
||||
@@ -36,26 +36,6 @@ def executable_target(executable_name: str) -> str:
|
||||
) from error
|
||||
|
||||
|
||||
def spawn_helper_binary_target(header: bytes) -> str | None:
|
||||
if len(header) >= 8 and header[:4] == b"\xcf\xfa\xed\xfe":
|
||||
cpu_type = int.from_bytes(header[4:8], "little")
|
||||
if cpu_type == 0x01000007:
|
||||
return "macos-x64"
|
||||
if cpu_type == 0x0100000C:
|
||||
return "macos-arm64"
|
||||
return None
|
||||
|
||||
|
||||
def validate_spawn_helper(path: Path, expected_target: str) -> None:
|
||||
with path.open("rb") as helper:
|
||||
actual_target = spawn_helper_binary_target(helper.read(8))
|
||||
if actual_target != expected_target:
|
||||
raise ValueError(
|
||||
f"runtime spawn helper binary mismatch: expected {expected_target}, "
|
||||
f"found {actual_target or 'unsupported format or architecture'} at {path}"
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--package", choices=("sdk", "runtime"), required=True)
|
||||
@@ -175,7 +155,6 @@ def stage_runtime(destination: Path, version: str, executable: Path, executable_
|
||||
raise FileNotFoundError(f"runtime spawn helper does not exist: {spawn_helper}")
|
||||
if spawn_helper.stat().st_mode & stat.S_IXUSR == 0:
|
||||
raise PermissionError(f"runtime spawn helper is not executable: {spawn_helper}")
|
||||
validate_spawn_helper(spawn_helper, expected_target)
|
||||
copy_package(ROOT / "python" / "sdk-runtime", destination)
|
||||
rewrite_version(destination / "pyproject.toml", version)
|
||||
runtime_dir = destination / "src" / "deepseek_harness_runtime" / "runtime"
|
||||
@@ -228,13 +207,6 @@ def verify_wheel(
|
||||
mode = archive.getinfo(executable).external_attr >> 16
|
||||
if mode & stat.S_IXUSR == 0:
|
||||
raise RuntimeError(f"{wheel} runtime executable lost its executable bit: {executable}")
|
||||
if helpers:
|
||||
actual_target = spawn_helper_binary_target(archive.read(helpers[0])[:8])
|
||||
if actual_target != expected_target:
|
||||
raise RuntimeError(
|
||||
f"{wheel} spawn helper binary mismatch: expected {expected_target}, "
|
||||
f"found {actual_target or 'unsupported format or architecture'}"
|
||||
)
|
||||
elif runtime_files:
|
||||
raise RuntimeError(f"SDK wheel unexpectedly contains runtime executables: {runtime_files}")
|
||||
if package == "sdk":
|
||||
|
||||
Reference in New Issue
Block a user