diff --git a/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.i18n.yaml new file mode 100644 index 0000000000..fe6bb60e68 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.md +2026-08-01-packaged-ripgrep-search.md: e43354ff8e4dde0480a6c07816fc112810197234 +2026-08-01-packaged-ripgrep-search.zh.md: 55498d366a2171a178cf9d0d1004b6fe946a7281 diff --git a/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.md b/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.md new file mode 100644 index 0000000000..e43354ff8e --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.md @@ -0,0 +1,34 @@ +# Agent Note: Packaged ripgrep spawn for glob/grep + +Status: implemented + +English | [中文](2026-08-01-packaged-ripgrep-search.zh.md) + +> Supersedes [bash-backed grep/glob discovery](../../archived/feature/2026-07-09-bash-backed-grep-glob-discovery.md): the v1 decision's explicitly deferred alternative — directly spawning ripgrep — is now what ships. + +## Problem + +The `glob`/`grep` tools ran through the bash executor seam, which made a system `rg` install a host dependency. On Windows and container images there is no `rg` on `PATH` by default, so the tools silently vanished there; a deployment could only discover that from the load-time probe warning. The bash seam also forced the whole model-visible argument surface through one shell-quoting helper, because a shell sat between the tool and ripgrep — the [bash-backed note](../../archived/feature/2026-07-09-bash-backed-grep-glob-discovery.md) recorded that coupling as the v1 trade-off and named direct spawn as the reasonable follow-up if the shell-string domain ever proved too sensitive. It did: every model value had to survive POSIX single-quoting, the probe had to be scripted in tests, and the executor's own timeout classification duplicated what the cooperative tool-timeout policy already owns. + +## Decision + +`@deepseek-ai/dsh-tool-fs-search` now runs the PACKAGED ripgrep binary (`@vscode/ripgrep`, an npm dependency whose optional platform packages ship the binary) through the `ctx.subprocess` seam: `runRipgrep()` spawns `rgPath` with a plain argv vector, collect-mode stdout/stderr, `graceMs`, and `exec.signal` forwarded. There is no shell layer, so the shell-quoting boundary is gone from execution; `singleQuote` stays exported as a compatibility surface with its tests. Registration is unconditional — the load-time `command -v rg` probe and the conditional registration decision are deleted, and with them the "rg not found" warning. The package injects `tools`, `systemPrompt`, and `subprocess`. + +Exit semantics stay tool-owned: exit 0 is success with results, exit 1 is a successful empty search, anything else classifies into the existing `SEARCH_*` vocabulary (invalid pattern, launch failure, signal kill, raw-output overflow). Timeout is the cooperative tool-call budget attached to the tool definitions: `@deepseek-ai/dsh-timeout-policy` aborts `exec.signal`, the subprocess seam's terminate escalation provides the hard kill, and the tool reports `SEARCH_ABORTED`. The working directory is the session header cwd when present, else `process.cwd()` — there is no executor config to default through anymore, so the tool owns the fallback. + +The `fs-glob-sampling` ACP snapshot scenario now executes the real packaged binary against a prepared workspace whose fixed mtimes pin the `--sort=modified` order, replacing the PATH-injected `rg` stand-in (POSIX-only, because the displayed paths carry `/` separators the session-log comparison cannot normalize). + +## Alternatives considered + +**Keep the bash seam and probe, but document `rg` as a required host dependency.** Rejected: the host dependency is exactly the failure this change removes, and Windows support for the discovery tools was the point of the exercise; a documented requirement is still a requirement. + +**Make `rgPath` injectable (a config field or env override) so tests and snapshots keep substituting a stand-in binary.** Rejected: it adds a public deployment surface whose only consumer would be test seams, and the real binary is deterministic enough to pin directly through fixture mtimes — the packaged binary is the deployment, so tests should exercise it. + +**Switch to a pure-JS glob/search engine (e.g. `picomatch`/`tinyglobby`).** Rejected: the [dependency-swaps audit](../../rejected/simplification/2026-07-26-dependency-swaps-rejected-by-nih-audit.md) already rejected that on the "no glob engine exists" evidence; ripgrep semantics (`--sort=modified`, VCS pruning, JSON transport, regex dialect) are the tool contract. + +## Consequences + +- The discovery tools work on every platform the packaged binary covers (darwin/linux/win32, x64/arm64) with no host install; the shipped TUI/Web rosters gain `glob`/`grep` as fixed members ([even-out-shipped-tool-rosters](../feature/2026-07-31-even-out-shipped-tool-rosters.md)). +- The shell-string attack surface is gone: hostile patterns are inert argv elements, pinned by the integration suite, which now runs on Windows too (it previously self-skipped without a system `rg`). +- Load-time failure modes changed: a broken subprocess seam now fails the first search call (`SEARCH_FAILED`) instead of failing plugin load through the probe; a missing binary is a launch failure with the packaged path, not a PATH problem. +- The integration suite's fixture dropped a filename Windows cannot represent (`"` in a name), keeping the suite replayable on every platform. diff --git a/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.zh.md b/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.zh.md new file mode 100644 index 0000000000..55498d366a --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.zh.md @@ -0,0 +1,34 @@ +# Agent Note: glob/grep 改用打包的 ripgrep 二进制直接 spawn + +Status: implemented + +[English](2026-08-01-packaged-ripgrep-search.md) | 中文 + +> 取代 [bash 承载的 grep/glob 发现工具](../../archived/feature/2026-07-09-bash-backed-grep-glob-discovery.md):v1 决策中明确延期的方案——直接 spawn ripgrep——现在成为实际交付的实现。 + +## 问题 + +`glob`/`grep` 工具经由 bash 执行器 seam 运行,这使系统 `rg` 安装成为宿主依赖。Windows 和容器镜像的 `PATH` 默认没有 `rg`,工具在那里会静默消失;部署方只能从加载期探针警告里发现这一点。bash seam 还迫使整个模型可见参数面经过一个 shell 引号工具,因为工具与 ripgrep 之间隔着一层 shell——[bash 承载决策](../../archived/feature/2026-07-09-bash-backed-grep-glob-discovery.md) 把这种耦合记为 v1 的取舍,并把直接 spawn 列为 shell 字符串域一旦被证明过于敏感时的合理后续。它确实被证明了:每个模型值都要经受 POSIX 单引号转义,探针要在测试里脚本化,执行器自身的超时分类还与协作式工具超时策略已有的职责重复。 + +## 决策 + +`@deepseek-ai/dsh-tool-fs-search` 现在运行 PACKAGED(打包的)ripgrep 二进制(`@vscode/ripgrep`,一个 npm 依赖,其可选平台包随附二进制),经由 `ctx.subprocess` seam:`runRipgrep()` 以纯 argv 向量 spawn `rgPath`,配以 collect 模式 stdout/stderr、`graceMs` 与转发的 `exec.signal`。不再有 shell 层,执行路径上的 shell 引号边界随之消失;`singleQuote` 作为兼容导出与其测试保留。注册变为无条件——加载期 `command -v rg` 探针与条件注册决策被删除,连同那条 "rg not found" 警告。本包注入 `tools`、`systemPrompt` 与 `subprocess`。 + +退出语义仍由工具拥有:退出码 0 为有结果的成功,1 为成功的空搜索,其余归入既有 `SEARCH_*` 词汇(无效模式、启动失败、信号杀死、原始输出溢出)。超时是挂在工具定义上的协作式工具调用预算:`@deepseek-ai/dsh-timeout-policy` 中止 `exec.signal`,subprocess seam 的终止升级提供硬终止,工具报告 `SEARCH_ABORTED`。工作目录为会话 header cwd(存在时),否则为 `process.cwd()`——不再有执行器配置可供默认化,因此回退由工具自己拥有。 + +`fs-glob-sampling` ACP 快照场景改为执行真实的打包二进制,作用于一个用固定 mtime 钉住 `--sort=modified` 顺序的预制工作区,取代 PATH 注入的 `rg` 替身(仅 POSIX:展示路径携带 `/` 分隔符,会话日志比较无法归一化)。 + +## 备选方案 + +**保留 bash seam 与探针,仅把 `rg` 记为必需宿主依赖。** 否决:宿主依赖正是本次改动要消除的失败模式,而让发现工具支持 Windows 正是此举的目的;写进文档的依赖仍是依赖。 + +**让 `rgPath` 可注入(配置字段或环境变量覆盖),让测试与快照继续替换替身二进制。** 否决:这会新增一个只有测试 seam 会消费的公开部署面,而真实二进制本身足够确定——通过 fixture mtime 即可直接钉住;打包二进制就是部署形态,测试应当拿它来测。 + +**改用纯 JS 的 glob/搜索引擎(如 `picomatch`/`tinyglobby`)。** 否决:[依赖替换审计](../../rejected/simplification/2026-07-26-dependency-swaps-rejected-by-nih-audit.md) 已基于"不存在 glob 引擎"的证据否决过该方向;ripgrep 语义(`--sort=modified`、VCS 剪枝、JSON 传输、正则方言)就是工具契约。 + +## 后果 + +- 发现工具在打包二进制覆盖的每个平台(darwin/linux/win32,x64/arm64)上开箱即用,无需宿主安装;交付的 TUI/Web 工具清单把 `glob`/`grep` 变为固定成员(见 [拉平交付的工具清单](../feature/2026-07-31-even-out-shipped-tool-rosters.md))。 +- shell 字符串攻击面消失:恶意模式只是惰性 argv 元素,由集成套件钉住;该套件现在也在 Windows 上运行(此前没有系统 `rg` 时它自行跳过)。 +- 加载期失败模式改变:subprocess seam 损坏现在让首次搜索调用失败(`SEARCH_FAILED`),而非通过探针使插件加载失败;二进制缺失是带打包路径的启动失败,而不是 PATH 问题。 +- 集成套件的 fixture 去掉了 Windows 无法表示的文件名(名称含 `"`),保证套件在每个平台都能重放。 diff --git a/.agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.i18n.yaml b/.agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.i18n.yaml index 83e965b391..60be2bc25f 100644 --- a/.agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.md -2026-07-31-even-out-shipped-tool-rosters.md: 316e5045e559e2da162c53d64989ccecfd18b857 -2026-07-31-even-out-shipped-tool-rosters.zh.md: ed39212dc4877f4df1dc1c6e84142b61a866c548 +2026-07-31-even-out-shipped-tool-rosters.md: d5f1d714ab538b740c25bdf148df285567c616ca +2026-07-31-even-out-shipped-tool-rosters.zh.md: b09fb43ab66f3784f1a3a3f2a6ee74e185fbdad8 diff --git a/.agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.md b/.agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.md index 316e5045e5..d5f1d714ab 100644 --- a/.agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.md +++ b/.agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.md @@ -12,7 +12,7 @@ The result was a user-visible difference nobody had decided: the same model, ask ## Decision -The rows that are not surface-specific move into [`base.cordis.yml`](../../../../apps/cli/config/base.cordis.yml), and three more join them: `tool-session-query`, `tool-str-replace-editor`, and `repeat-tool-guard`. Web search moves there too; its [deployment decision](2026-07-31-web-default-search.md) owns the security boundary while the shared base owns its surface-neutral mount. Both surfaces now assemble the same roster: twenty-five tools on every host, plus `glob` and `grep` when ripgrep is available. +The rows that are not surface-specific move into [`base.cordis.yml`](../../../../apps/cli/config/base.cordis.yml), and three more join them: `tool-session-query`, `tool-str-replace-editor`, and `repeat-tool-guard`. Web search moves there too; its [deployment decision](2026-07-31-web-default-search.md) owns the security boundary while the shared base owns its surface-neutral mount. Both surfaces now assemble the same roster: twenty-seven tools on every host — the twenty-five shared rows plus `glob` and `grep`, which are fixed members because `dsh-tool-fs-search` spawns the [packaged ripgrep binary](../architecture/2026-08-01-packaged-ripgrep-search.md). Two rows stay surface-specific. `tmux-context` is TUI-only because a browser surface has no terminal multiplexer to describe. `session-reference` is TUI-only because it drives the shared session-query index from the launcher's process-local path, and the browser sidebar reconciles that index on its own first search. @@ -46,7 +46,7 @@ The same smoke pins the TUI's unchanged execution posture from the same artifact [`apps/web/tests/shipped-composition.e2e.ts`](../../../../apps/web/tests/shipped-composition.e2e.ts) covers the Web surface in the built lane, asserting its catalog, that its access default is untouched, and that `workspace-write`'s writable roots include the temp directories — a trap that makes sandbox tests lie when the workspace sits under `/tmp` ([`roots.ts`](../../../../packages/sandbox/sandbox/src/roots.ts)). -`glob` and `grep` are asserted as an all-or-nothing pair rather than fixed members: `dsh-tool-fs-search` probes `command -v rg` at load and registers neither tool without ripgrep, which is a host dependency. +`glob` and `grep` are asserted as fixed members rather than a host-dependent pair: `dsh-tool-fs-search` spawns the packaged ripgrep binary and registers both tools unconditionally, so the pair is always present. Beyond the committed tests, both surfaces were driven against a real key from the built `apps/cli/lib/bin.js` under plain Node. Every mounted tool executed successfully, including `ralph` and `web_search`; the model never reached `cordis_*` or `mcp_*`, fell back to `grep` when asked for LSP navigation, and used a background `bash` task when asked for a persistent terminal. @@ -62,7 +62,7 @@ Beyond the committed tests, both surfaces were driven against a real key from th ## Consequences -The same model gets the same tools on both surfaces, and the difference that existed for no recorded reason is gone. The tests assert the twenty-five unconditional names exactly and require the ripgrep-dependent pair to be either present together or absent together on both sides, so a later change that alters only one surface fails a check instead of shipping quietly. +The same model gets the same tools on both surfaces, and the difference that existed for no recorded reason is gone. The tests assert all twenty-seven names exactly on both sides, so a later change that alters only one surface fails a check instead of shipping quietly. `apps/cli` gains five workspace dependencies: four the shipped tree now mounts, plus `dsh-mcp-client`, which it does not mount and which exists so an installed `dsh` can. diff --git a/.agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.zh.md b/.agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.zh.md index ed39212dc4..b09fb43ab6 100644 --- a/.agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.zh.md +++ b/.agents/notes/implemented/feature/2026-07-31-even-out-shipped-tool-rosters.zh.md @@ -12,7 +12,7 @@ Status: implemented ## 决策 -那些并非 surface 专属的行移入 [`base.cordis.yml`](../../../../apps/cli/config/base.cordis.yml),另有三行加入:`tool-session-query`、`tool-str-replace-editor` 和 `repeat-tool-guard`。Web 搜索也一并移入;其[部署决策](2026-07-31-web-default-search.md)负责安全边界,共享 base 则负责与 surface 无关的挂载。两个 surface 现在组装同一份清单:每台宿主上都有二十五个工具,ripgrep 可用时再加上 `glob` 和 `grep`。 +那些并非 surface 专属的行移入 [`base.cordis.yml`](../../../../apps/cli/config/base.cordis.yml),另有三行加入:`tool-session-query`、`tool-str-replace-editor` 和 `repeat-tool-guard`。Web 搜索也一并移入;其[部署决策](2026-07-31-web-default-search.md)负责安全边界,共享 base 则负责与 surface 无关的挂载。两个 surface 现在组装同一份清单:每台宿主上都有二十七个工具——二十五个共享行加上 `glob` 和 `grep`,它们成为固定成员是因为 `dsh-tool-fs-search` 直接 spawn [打包的 ripgrep 二进制](../architecture/2026-08-01-packaged-ripgrep-search.md)。 有两行仍是 surface 专属。`tmux-context` 只在 TUI,因为浏览器 surface 没有终端复用器可描述。`session-reference` 只在 TUI,因为它以 launcher 的进程本地路径驱动共享的 session-query 索引,而浏览器侧边栏会在自己的首次搜索里重建该索引。 @@ -62,7 +62,7 @@ Status: implemented ## 后果 -同一个模型在两个 surface 上拿到同样的工具,那处没有记录理由的差异消失了。测试会精确断言二十五个无条件提供的名称,并要求依赖 ripgrep 的一对工具在两侧要么同时存在、要么同时缺席,因此日后只改一个 surface 都会让检查失败而不是悄悄发出去。 +同一个模型在两个 surface 上拿到同样的工具,那处没有记录理由的差异消失了。测试会精确断言两侧全部二十七个名称,因此日后只改一个 surface 都会让检查失败而不是悄悄发出去。 `apps/cli` 增加五个 workspace 依赖:四个是交付树现在挂载的,外加 `dsh-mcp-client`——它并不被挂载,存在的意义是让已安装的 `dsh` 能挂。 diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 754ae93d82..1b14996c3f 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -47,6 +47,9 @@ External packages that a workspace package resolves at runtime. `scripts/install | [`@opentelemetry/sdk-logs`](https://github.com/open-telemetry/opentelemetry-js) | Apache-2.0 | | [`@shikijs/langs`](https://github.com/shikijs/shiki) | MIT | | [`@standard-schema/spec`](https://github.com/standard-schema/standard-schema) | MIT | +| [`@testing-library/dom`](https://github.com/testing-library/dom-testing-library) | MIT | +| [`@testing-library/react`](https://github.com/testing-library/react-testing-library) | MIT | +| [`@vscode/ripgrep`](https://github.com/microsoft/vscode-ripgrep) | MIT | | [`anser`](https://github.com/IonicaBizau/anser) | MIT | | [`chokidar`](https://github.com/paulmillr/chokidar) | MIT | | [`clsx`](https://github.com/lukeed/clsx) | MIT | @@ -54,6 +57,7 @@ External packages that a workspace package resolves at runtime. `scripts/install | [`diff`](https://github.com/kpdecker/jsdiff) | BSD-3-Clause | | [`dotenv`](https://github.com/motdotla/dotenv) | BSD-2-Clause | | [`eventsource-parser`](https://github.com/rexxars/eventsource-parser) | MIT | +| [`execa`](https://github.com/sindresorhus/execa) | MIT | | [`handlebars`](https://github.com/handlebars-lang/handlebars.js) | MIT | | [`immer`](https://github.com/immerjs/immer) | MIT | | [`js-yaml`](https://github.com/nodeca/js-yaml) | MIT | @@ -79,6 +83,7 @@ External packages that a workspace package resolves at runtime. `scripts/install | [`turndown`](https://github.com/mixmark-io/turndown) | MIT | | [`typescript`](https://github.com/microsoft/TypeScript) | Apache-2.0 | | [`use-sync-external-store`](https://github.com/facebook/react) | MIT | +| [`vitest`](https://github.com/vitest-dev/vitest) | MIT | | [`yaml`](https://github.com/eemeli/yaml) | ISC | | [`zod`](https://github.com/colinhacks/zod) | MIT | | [`zustand`](https://github.com/pmndrs/zustand) | MIT | @@ -98,8 +103,6 @@ External packages **directly declared** only by repository tooling, test infrast | [`@modelcontextprotocol/server-everything`](https://github.com/modelcontextprotocol/servers) | MIT / Apache-2.0 | | [`@modelcontextprotocol/server-filesystem`](https://github.com/modelcontextprotocol/servers) | MIT / Apache-2.0 | | [`@stylistic/eslint-plugin`](https://github.com/eslint-stylistic/eslint-stylistic) | MIT | -| [`@testing-library/dom`](https://github.com/testing-library/dom-testing-library) | MIT | -| [`@testing-library/react`](https://github.com/testing-library/react-testing-library) | MIT | | [`@types/babel__code-frame`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT | | [`@types/js-yaml`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT | | [`@types/jsdom`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT | @@ -122,7 +125,6 @@ External packages **directly declared** only by repository tooling, test infrast | [`esbuild`](https://github.com/evanw/esbuild) | MIT | | [`eslint`](https://github.com/eslint/eslint) | MIT | | [`eslint-plugin-sonarjs`](https://github.com/SonarSource/SonarJS) | LGPL-3.0-only | -| [`execa`](https://github.com/sindresorhus/execa) | MIT | | [`fast-check`](https://github.com/dubzzz/fast-check) | MIT | | [`jscpd`](https://github.com/kucherenko/jscpd) | MIT | | [`jsdom`](https://github.com/jsdom/jsdom) | MIT | @@ -142,7 +144,6 @@ External packages **directly declared** only by repository tooling, test infrast | [`vite-tsconfig-paths`](https://github.com/aleclarson/vite-tsconfig-paths) | MIT | | [`vitepress`](https://github.com/vuejs/vitepress) | MIT | | [`vitepress-plugin-mermaid`](https://github.com/emersonbottero/vitepress-plugin-mermaid) | MIT | -| [`vitest`](https://github.com/vitest-dev/vitest) | MIT | `eslint-plugin-sonarjs` (LGPL-3.0-only) and `lightningcss` (MPL-2.0) run only as development tooling; their code is not linked into or distributed with any DeepSeek Harness artifact. diff --git a/apps/cli/tests/shipped-composition.e2e.ts b/apps/cli/tests/shipped-composition.e2e.ts index ac1e9d5456..82bf698c17 100644 --- a/apps/cli/tests/shipped-composition.e2e.ts +++ b/apps/cli/tests/shipped-composition.e2e.ts @@ -54,10 +54,10 @@ const EXPECTED_TUI_TOOLS = [ ] /** - * `glob` and `grep` come from `dsh-tool-fs-search`, which probes `command -v rg` - * through the mounted bash executor at load and registers neither tool when - * ripgrep is absent. That is a host dependency, not a composition decision, so the - * pair is asserted separately — present together or absent together. + * `glob` and `grep` come from `dsh-tool-fs-search`, which spawns the PACKAGED + * ripgrep binary (`@vscode/ripgrep`) through the subprocess seam, so the pair + * is always present on every host — asserted as fixed members, not a host + * dependency. */ const RIPGREP_TOOLS = ['glob', 'grep'] @@ -117,7 +117,9 @@ describe('shipped dsh composition (real Loader tree in a PTY)', () => { }) expect(output).toContain(COMPOSITION_REPLY_TEXT) expect(observed?.names.filter(name => !RIPGREP_TOOLS.includes(name))).toEqual(EXPECTED_TUI_TOOLS) - expect([[], RIPGREP_TOOLS]).toContainEqual(observed?.names.filter(name => RIPGREP_TOOLS.includes(name))) + // The packaged ripgrep binary ships with the dependency, so the pair is a + // fixed roster member on every host. + expect(observed?.names.filter(name => RIPGREP_TOOLS.includes(name))).toEqual(RIPGREP_TOOLS) // The TUI mounts the unrestricted local executors, so `tool-bash` emits no // escalation pair. Pinning its absence keeps a later sandbox change from // arriving here unannounced. diff --git a/apps/web/tests/shipped-composition.e2e.ts b/apps/web/tests/shipped-composition.e2e.ts index 0cad833303..80adcdd1de 100644 --- a/apps/web/tests/shipped-composition.e2e.ts +++ b/apps/web/tests/shipped-composition.e2e.ts @@ -49,10 +49,10 @@ const EXPECTED_TOOLS = [ ] /** - * `glob` and `grep` come from `dsh-tool-fs-search`, which probes `command -v rg` - * through the mounted bash executor at load and registers neither tool when - * ripgrep is absent. That is a host dependency, not a composition decision, so the - * pair is asserted separately — present together or absent together. + * `glob` and `grep` come from `dsh-tool-fs-search`, which spawns the PACKAGED + * ripgrep binary (`@vscode/ripgrep`) through the subprocess seam, so the pair + * is always present on every host — asserted as fixed members, not a host + * dependency. */ const RIPGREP_TOOLS = ['glob', 'grep'] @@ -67,7 +67,9 @@ it('assembles the shipped Web catalog and keeps its access default', async () => scaffold = await launchWebScaffold() const names = scaffold.ctx.tools.schemas().map(schema => schema.name).sort() expect(names.filter(name => !RIPGREP_TOOLS.includes(name))).toEqual(EXPECTED_TOOLS) - expect([[], RIPGREP_TOOLS]).toContainEqual(names.filter(name => RIPGREP_TOOLS.includes(name))) + // The packaged ripgrep binary ships with the dependency, so the pair is a + // fixed roster member on every host. + expect(names.filter(name => RIPGREP_TOOLS.includes(name))).toEqual(RIPGREP_TOOLS) // `workspace-write` is not "the workspace and nothing else": the shared roots // helper always admits the temp directories too. Pinning it against an // explicit mode keeps the claim independent of this surface's default, and diff --git a/docs/config-catalog.md b/docs/config-catalog.md index a356a151e3..20f835bb7f 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -1716,7 +1716,7 @@ Source: [`packages/fs/tool-fs/src/index.ts:24`](../packages/fs/tool-fs/src/index ## `@deepseek-ai/dsh-tool-fs-search` -Requires: `tools` · `systemPrompt` · `bash` +Requires: `tools` · `systemPrompt` · `subprocess` ```ts config-catalog /** Plugin config; over-cap glob sampling is an explicit deployment choice and the remaining fields have defaults. */ @@ -1738,7 +1738,7 @@ export interface Config { } ``` -Source: [`packages/fs/tool-fs-search/src/index.ts:71`](../packages/fs/tool-fs-search/src/index.ts) +Source: [`packages/fs/tool-fs-search/src/index.ts:70`](../packages/fs/tool-fs-search/src/index.ts) ## `@deepseek-ai/dsh-tool-goal` diff --git a/docs/module-graph.md b/docs/module-graph.md index f0f58a18c1..87789c9edb 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -713,12 +713,12 @@ flowchart TD pkg_tool_fs --> pkg_system_prompt pkg_tool_fs --> pkg_tools pkg_tool_fs --> pkg_user_approval - pkg_tool_fs_search --> pkg_bash pkg_tool_fs_search --> pkg_invariants pkg_tool_fs_search --> pkg_llm pkg_tool_fs_search --> pkg_retention pkg_tool_fs_search --> pkg_session pkg_tool_fs_search --> pkg_spill + pkg_tool_fs_search --> pkg_subprocess pkg_tool_fs_search --> pkg_system_prompt pkg_tool_fs_search --> pkg_tools pkg_tool_str_replace_editor --> pkg_fs @@ -1176,7 +1176,7 @@ flowchart TD | [`tool-goal`](../packages/goal/tool-goal) | `goal` | [`agent`](../packages/core/agent), [`goal`](../packages/goal/goal), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`tool-bash`](../packages/bash/tool-bash) | `bash` | [`agent`](../packages/core/agent), [`bash`](../packages/bash/bash), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`paths`](../packages/util/paths), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`session-persistence`](../packages/session-persistence/session-persistence), [`system-prompt`](../packages/core/system-prompt), [`tasks`](../packages/tasks/tasks), [`tools`](../packages/core/tools), [`user-approval`](../packages/ui/user-approval) | | [`tool-fs`](../packages/fs/tool-fs) | `fs` | [`fs`](../packages/fs/fs), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`user-approval`](../packages/ui/user-approval) | -| [`tool-fs-search`](../packages/fs/tool-fs-search) | `fs` | [`bash`](../packages/bash/bash), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`retention`](../packages/util/retention), [`session`](../packages/core/session), [`spill`](../packages/spill/spill), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | +| [`tool-fs-search`](../packages/fs/tool-fs-search) | `fs` | [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`retention`](../packages/util/retention), [`session`](../packages/core/session), [`spill`](../packages/spill/spill), [`subprocess`](../packages/subprocess/subprocess), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) | `fs` | [`fs`](../packages/fs/fs), [`invariants`](../packages/support/invariants), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`tools`](../packages/core/tools) | | [`tool-skill`](../packages/skill/tool-skill) | `skill` | [`agent`](../packages/core/agent), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`skill`](../packages/skill/skill), [`tools`](../packages/core/tools) | | [`subagent`](../packages/subagent/subagent) | `subagent` | [`agent`](../packages/core/agent), [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`tools`](../packages/core/tools) | diff --git a/docs/tool-catalog.md b/docs/tool-catalog.md index 7d6fa79dea..d19a7adf52 100644 --- a/docs/tool-catalog.md +++ b/docs/tool-catalog.md @@ -23,7 +23,7 @@ This table connects model-visible tool names to the plugin package and service s | `@deepseek-ai/dsh-tool-bash-persistent` | `bash` | `ctx.tools`, `ctx.pty`, `an owning Agent at execution time` | `tool/call`, `PTY shell state`, `tool/result` | - | One owner-isolated persistent bash tool; deployment composition supplies the PTY backend and may override the model-facing environment description. | | `@deepseek-ai/dsh-tool-str-replace-editor` | `str_replace_editor` | `ctx.tools`, `ctx.fs` | `tool/call`, `fs/observed after successful file operations`, `tool/result` | - | Standalone view/create/unique literal replace/line insert tool over the filesystem seam; it composes with any shell or terminal surface. | | `@deepseek-ai/dsh-tool-fs` | `edit`, `read`, `write` | `ctx.tools`, `ctx.fs`, `ctx.systemPrompt` | `tool/call`, `fs/write-intent or fs/edit-intent for mutations`, `fs/observed after successful file operations`, `tool/result` | - | The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. The tool schemas above are identical with or without the policy plugin. | -| `@deepseek-ai/dsh-tool-fs-search` | `glob`, `grep` | `ctx.tools`, `ctx.bash`, `ctx.systemPrompt` | `tool/call`, `tool/result` | - | glob and grep are conditional bash-backed discovery tools: they register only when ctx.bash can find `rg`, then run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). The catalog uses `sampleOverCapGlobResults: true`; deployments must choose that behavior explicitly. Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. | +| `@deepseek-ai/dsh-tool-fs-search` | `glob`, `grep` | `ctx.tools`, `ctx.subprocess`, `ctx.systemPrompt` | `tool/call`, `tool/result` | - | glob and grep are unconditional discovery tools that spawn the packaged ripgrep binary (`@vscode/ripgrep`) through ctx.subprocess as ordinary foreground calls (never background tasks) — no host `rg` install and no shell layer. The catalog uses `sampleOverCapGlobResults: true`; deployments must choose that behavior explicitly. Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. | | `@deepseek-ai/dsh-tool-pty` | `terminal_close`, `terminal_list`, `terminal_open`, `terminal_read`, `terminal_send`, `terminal_signal` | `ctx.tools`, `ctx.pty`, `ctx.systemPrompt`, `ctx.tasks at call time for run_in_background` | `tool/call`, `tool/result` | - | The six terminal tools are opt-in and complement one-shot bash/filesystem tools. `terminal_send(run_in_background: true)` registers with `ctx.tasks`; TUI, named key sequences, BEL, resize, auto-start, and cross-agent sharing are absent from the schema. | | `@deepseek-ai/dsh-tool-goal` | `create_goal`, `get_goal`, `update_goal` | `ctx.tools`, `ctx.agents`, `ctx.goals`, `ctx.systemPrompt`, `a calling Agent in an authorized open turn` | `tool/call`, `user/message goal snapshot for mutations`, `tool/result` | - | create, edit, pause, and resume require direct-human root authority; complete and blocked also accept the exact current goal round. The default blocked lower bound is three admitted rounds. | | `@deepseek-ai/dsh-tool-lsp` | `lsp` | `ctx.tools`, `ctx.lsp`, `ctx.systemPrompt` | `tool/call`, `tool/result` | - | The lsp tool keeps provider selection and language-server subprocesses behind ctx.lsp, so its model-visible schema stays stable across providers. Requires a registered provider (e.g. `@deepseek-ai/dsh-lsp-local`) at runtime; without one, a query returns the structured `LSP_UNAVAILABLE` error rather than changing the schema. | @@ -524,7 +524,7 @@ Search file contents with a ripgrep regular expression. Returns matching lines w Source: [`packages/fs/tool-fs-search/src/index.ts`](../packages/fs/tool-fs-search/src/index.ts) -glob and grep are conditional bash-backed discovery tools: they register only when ctx.bash can find `rg`, then run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). The catalog uses `sampleOverCapGlobResults: true`; deployments must choose that behavior explicitly. Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. +glob and grep are unconditional discovery tools that spawn the packaged ripgrep binary (`@vscode/ripgrep`) through ctx.subprocess as ordinary foreground calls (never background tasks) — no host `rg` install and no shell layer. The catalog uses `sampleOverCapGlobResults: true`; deployments must choose that behavior explicitly. Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. ## `@deepseek-ai/dsh-tool-pty` diff --git a/examples/acp-agent/tests/acp.snapshot.ts b/examples/acp-agent/tests/acp.snapshot.ts index b652f09931..277d0b20ca 100644 --- a/examples/acp-agent/tests/acp.snapshot.ts +++ b/examples/acp-agent/tests/acp.snapshot.ts @@ -1,6 +1,6 @@ import { fileURLToPath } from 'node:url' import { readFileSync } from 'node:fs' -import { mkdir, writeFile } from 'node:fs/promises' +import { mkdir, utimes, writeFile } from 'node:fs/promises' import { dirname, join } from 'node:path' import { homedir } from 'node:os' import { expect, it } from 'vitest' @@ -44,7 +44,6 @@ const SESSION_TITLE_CONFIG = fileURLToPath(new URL('../session-title.cordis.yml' const LSP_CONFIG = fileURLToPath(new URL('./lsp.cordis.yml', import.meta.url)) const WEB_CONFIG = fileURLToPath(new URL('../web.cordis.yml', import.meta.url)) const FS_SEARCH_CONFIG = fileURLToPath(new URL('./fs-search.cordis.yml', import.meta.url)) -const FS_SEARCH_BIN = fileURLToPath(new URL('./fixtures/fs-search-bin', import.meta.url)) const SNAPSHOTS_DIR = join(dirname(fileURLToPath(import.meta.url)), 'snapshots') const PACKED_CHUNKS_SOURCE = 'hook-cc-pretool-deny' @@ -57,6 +56,33 @@ async function prepareDelimiterPathWorkspace(cwd: string): Promise { ]) } +/** + * Seed the over-cap glob fixture: eight files under `tree/` with fixed mtimes, + * so the packaged ripgrep's `--sort=modified` order is deterministic — three + * files under `archive/`, one each under `docs/`, `src/`, and `test/`, plus + * two flat files (six top-level entries). Scoping the search to `tree/` keeps + * the harness's own session artifacts out of the listing. + */ +async function prepareFsSearchWorkspace(cwd: string): Promise { + const tree = join(cwd, 'tree') + const files: Array<[relative: string, mtime: Date]> = [ + [join('archive', 'a.ts'), new Date(2000, 0, 1, 0, 0, 0, 1)], + [join('archive', 'b.ts'), new Date(2000, 0, 1, 0, 0, 0, 2)], + [join('archive', 'c.ts'), new Date(2000, 0, 1, 0, 0, 0, 3)], + [join('docs', 'guide.md'), new Date(2000, 0, 1, 0, 0, 0, 4)], + [join('src', 'index.ts'), new Date(2000, 0, 1, 0, 0, 0, 5)], + [join('test', 'spec.ts'), new Date(2000, 0, 1, 0, 0, 0, 6)], + ['top.txt', new Date(2000, 0, 1, 0, 0, 0, 7)], + ['notes.md', new Date(2000, 0, 1, 0, 0, 0, 8)], + ] + for (const [relative, mtime] of files) { + const target = join(tree, relative) + await mkdir(dirname(target), { recursive: true }) + await writeFile(target, 'fixture\n') + await utimes(target, mtime, mtime) + } +} + // FIXME: Migrate backend-oriented scenarios to the headless stream-json suite; // this ACP suite should eventually retain only automation-protocol contracts. @@ -150,9 +176,12 @@ const SCENARIOS: Scenario[] = [ hasModelTurn: true, recorded: true, }, - // The real Loader/app/bash path executes a deterministic rg stand-in at the - // external-process seam, pinning over-cap glob sampling without depending on - // a host-installed ripgrep binary. + // The real Loader/app/subprocess path executes the PACKAGED ripgrep binary + // against a prepared workspace whose fixed mtimes pin the + // `--sort=modified` order, pinning over-cap glob sampling without depending + // on a host-installed ripgrep binary or a PATH stand-in. POSIX-only because + // the displayed paths carry `/` separators the session-log comparison + // cannot normalize. { name: 'fs-glob-sampling', hasModelTurn: true, @@ -160,7 +189,7 @@ const SCENARIOS: Scenario[] = [ pinsHeader: true, headerClass: 'fs-search', configPath: FS_SEARCH_CONFIG, - env: { PATH: `${FS_SEARCH_BIN}:${process.env.PATH ?? ''}` }, + prepareWorkspace: prepareFsSearchWorkspace, posixOnly: true, }, { name: 'fs-read', hasModelTurn: true, recorded: true }, diff --git a/examples/acp-agent/tests/fixtures/fs-search-bin/rg b/examples/acp-agent/tests/fixtures/fs-search-bin/rg deleted file mode 100755 index 181ad68837..0000000000 --- a/examples/acp-agent/tests/fixtures/fs-search-bin/rg +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/sh -printf '%s\n' \ - 'archive/a.ts' \ - 'archive/b.ts' \ - 'archive/c.ts' \ - 'old\one' \ - 'old\two' \ - 'src/index.ts' \ - 'docs/guide.md' \ - 'test/spec.ts' diff --git a/examples/acp-agent/tests/snapshots/fs-glob-sampling/input.json b/examples/acp-agent/tests/snapshots/fs-glob-sampling/input.json index cc5fc95e59..d615bd4840 100644 --- a/examples/acp-agent/tests/snapshots/fs-glob-sampling/input.json +++ b/examples/acp-agent/tests/snapshots/fs-glob-sampling/input.json @@ -2,6 +2,6 @@ "steps": [ { "op": "initialize" }, { "op": "newSession" }, - { "op": "prompt", "text": "Call glob exactly once with pattern * and no path. Then reply with exactly GLOB_SAMPLED and nothing else." } + { "op": "prompt", "text": "Call glob exactly once with pattern * and path tree. Then reply with exactly GLOB_SAMPLED and nothing else." } ] } diff --git a/examples/acp-agent/tests/snapshots/fs-glob-sampling/session.jsonl b/examples/acp-agent/tests/snapshots/fs-glob-sampling/session.jsonl index b66ffc36b4..ca51259632 100644 --- a/examples/acp-agent/tests/snapshots/fs-glob-sampling/session.jsonl +++ b/examples/acp-agent/tests/snapshots/fs-glob-sampling/session.jsonl @@ -1,18 +1,18 @@ {"type":"session","version":0,"id":"f5a99d52-3eaa-4ce7-858d-61d4fd77df2a","createdAt":1785218400000,"cwd":"{{cwd}}","delegationDepth":0} {"type":"turn/start","seq":0,"time":1785218400001,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}} -{"type":"user/message","seq":1,"time":1785218400002,"data":{"content":[{"type":"text","text":"Call glob exactly once with pattern * and no path. Then reply with exactly GLOB_SAMPLED and nothing else."}],"source":{"kind":"user"},"role":"user","id":"6790985f-1de2-42f8-a7f1-24e46d6439c7"},"surfaceOp":"append"} +{"type":"user/message","seq":1,"time":1785218400002,"data":{"content":[{"type":"text","text":"Call glob exactly once with pattern * and path tree. Then reply with exactly GLOB_SAMPLED and nothing else."}],"source":{"kind":"user"},"role":"user","id":"6790985f-1de2-42f8-a7f1-24e46d6439c7"},"surfaceOp":"append"} {"type":"session/title","seq":2,"time":1785218400003,"data":{"title":"Call glob exactly once with","messageSeqs":[1],"source":{"kind":"fallback"}}} {"type":"step/start","seq":3,"time":1785218400004,"data":{"turn":1,"step":1}} {"type":"request/header","seq":4,"time":1785218400005,"data":{"header":{"config":{"provider":"deepseek","model":"deepseek-v4-pro"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} {"type":"request/context","seq":5,"time":1785483397569,"data":{"provider":"deepseek","model":"deepseek-v4-pro"}} {"type":"assistant/chunk","seq":6,"time":1785218400007,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} -{"type":"assistant/chunk","seq":7,"time":1785218400008,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":0,"id":"glob-sampling-call","name":"glob","argumentsDelta":"{\"pattern\":\"*\"}"}}} -{"type":"assistant/chunk","seq":8,"time":1785218400009,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"glob-sampling-call","name":"glob","arguments":"{\"pattern\":\"*\"}"}}}} +{"type":"assistant/chunk","seq":7,"time":1785218400008,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":0,"id":"glob-sampling-call","name":"glob","argumentsDelta":"{\"pattern\":\"*\",\"path\":\"tree\"}"}}} +{"type":"assistant/chunk","seq":8,"time":1785218400009,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"glob-sampling-call","name":"glob","arguments":"{\"pattern\":\"*\",\"path\":\"tree\"}"}}}} {"type":"assistant/chunk","seq":9,"time":1785218400010,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":1,"outputTokens":1}}}} {"type":"assistant/chunk","seq":10,"time":1785483397579,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} -{"type":"assistant/message","seq":11,"time":1785483397579,"data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"tool-call","id":"glob-sampling-call","name":"glob","arguments":"{\"pattern\":\"*\"}"}],"source":{"kind":"model","provider":"deepseek","model":"deepseek-v4-pro"},"id":"a127cfe5-39fb-462c-8e5a-a8c79bd0e52b"},"usage":{"inputTokens":1,"outputTokens":1}},"sourceEventSeqs":[6,7,8,9,10],"surfaceOp":"append"} -{"type":"tool/call","seq":12,"time":1785483397579,"data":{"turn":1,"step":1,"callId":"glob-sampling-call","name":"glob","arguments":"{\"pattern\":\"*\"}"}} -{"type":"tool/result","seq":13,"time":1785483398062,"data":{"turn":1,"step":1,"message":{"source":{"kind":"tool","callId":"glob-sampling-call"},"content":[{"type":"tool-result","toolCallId":"glob-sampling-call","content":[{"type":"text","text":"archive/a.ts\nold\\one\nold\\two\nsrc/index.ts\n\n(Showing 4 of 8 paths, sampled across 4 of the 6 top-level entries this pattern matched instead of taken in modification-time order. Narrow path to inspect a specific subtree. The complete result could not be saved; narrow pattern or path to see more.)"}],"isError":false}],"role":"user","id":"2beecb2e-627d-43dc-a936-03e1dc874093"},"meta":{"shape":"paths","paths":["archive/a.ts","old\\one","old\\two","src/index.ts"],"truncated":true,"total":8}},"sourceEventSeqs":[12],"surfaceOp":"append"} +{"type":"assistant/message","seq":11,"time":1785483397579,"data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"tool-call","id":"glob-sampling-call","name":"glob","arguments":"{\"pattern\":\"*\",\"path\":\"tree\"}"}],"source":{"kind":"model","provider":"deepseek","model":"deepseek-v4-pro"},"id":"a127cfe5-39fb-462c-8e5a-a8c79bd0e52b"},"usage":{"inputTokens":1,"outputTokens":1}},"sourceEventSeqs":[6,7,8,9,10],"surfaceOp":"append"} +{"type":"tool/call","seq":12,"time":1785483397579,"data":{"turn":1,"step":1,"callId":"glob-sampling-call","name":"glob","arguments":"{\"pattern\":\"*\",\"path\":\"tree\"}"}} +{"type":"tool/result","seq":13,"time":1785483398062,"data":{"turn":1,"step":1,"message":{"source":{"kind":"tool","callId":"glob-sampling-call"},"content":[{"type":"tool-result","toolCallId":"glob-sampling-call","content":[{"type":"text","text":"tree/archive/a.ts\ntree/docs/guide.md\ntree/src/index.ts\ntree/test/spec.ts\n\n(Showing 4 of 8 paths, sampled across 4 of the 6 top-level entries this pattern matched instead of taken in modification-time order. Narrow path to inspect a specific subtree. The complete result could not be saved; narrow pattern or path to see more.)"}],"isError":false}],"role":"user","id":"2beecb2e-627d-43dc-a936-03e1dc874093"},"meta":{"shape":"paths","paths":["tree/archive/a.ts","tree/docs/guide.md","tree/src/index.ts","tree/test/spec.ts"],"truncated":true,"total":8}},"sourceEventSeqs":[12],"surfaceOp":"append"} {"type":"step/end","seq":14,"time":1785483398062,"data":{"turn":1,"step":1}} {"type":"step/start","seq":15,"time":1785483398072,"data":{"turn":1,"step":2}} {"type":"assistant/chunk","seq":16,"time":1785218400017,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} diff --git a/packages/fs/tool-fs-search/README.i18n.yaml b/packages/fs/tool-fs-search/README.i18n.yaml index bedb8289c1..5ed7bfb8bb 100644 --- a/packages/fs/tool-fs-search/README.i18n.yaml +++ b/packages/fs/tool-fs-search/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/tool-fs-search/README.md -README.md: b12ffda9869c7d6bef5ea5b54594781ecf555ff4 -README.zh.md: 7dd6cdf9a209f2fe357b4ffe48d20d574266ce60 +README.md: 0152be017ae15fc83a3d5cb7df927f25d04d2d53 +README.zh.md: 69ce49f3ad1621021dc1d0938cdc07a900d1cda8 diff --git a/packages/fs/tool-fs-search/README.md b/packages/fs/tool-fs-search/README.md index b12ffda986..0152be017a 100644 --- a/packages/fs/tool-fs-search/README.md +++ b/packages/fs/tool-fs-search/README.md @@ -2,21 +2,21 @@ English | [中文](README.zh.md) -The **model-facing filesystem discovery tools**—`glob`, `grep`—are backed by the **bash executor seam**, not by `ctx.fs` provider methods. At load, the package probes `command -v rg` through `ctx.bash`; if the executor cannot find ripgrep on its `PATH`, it logs a warning and registers no tools or prompt sections. Each call assembles a fixed ripgrep command (every model-controlled value through one package-private shell-quoting helper), runs it via `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` as an ordinary foreground tool call, parses the raw `rg` output, and returns a workdir-relative canonical value. The package injects `tools`, `systemPrompt`, and `bash`—deliberately **not** `fs`; `ctx.spillStore` is read opportunistically with `ctx.get()` because formatted-result spill is optional. +The **model-facing filesystem discovery tools**—`glob`, `grep`—are backed by the **packaged ripgrep binary** (`@vscode/ripgrep`), not by `ctx.fs` provider methods and not by a system `rg` install. Registration is unconditional: the binary ships inside the npm dependency, so there is no load-time availability probe. Each call spawns the binary through the `ctx.subprocess` seam with a fixed argv vector (model-controlled values are plain argv elements — no shell layer exists, so no quoting applies), parses the raw `rg` output, and returns a workdir-relative canonical value. The package injects `tools`, `systemPrompt`, and `subprocess`—deliberately **not** `fs`; `ctx.spillStore` is read opportunistically with `ctx.get()` because formatted-result spill is optional. ```ts ignore-check // A deployment chooses how over-cap glob pages are selected. -await ctx.plugin(LocalBashExecutor, { cwd: process.cwd() }) // @deepseek-ai/dsh-bash-local +await ctx.plugin(LocalSubprocessService) // @deepseek-ai/dsh-subprocess-local await ctx.plugin(ToolFsSearch, { sampleOverCapGlobResults: false }) // Optional: a spill backend makes capped results fully recoverable. await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-spill-local ``` -Why bash-backed: local workspace discovery is naturally a process-backed `rg` workflow, and putting search on `ctx.fs` would force every filesystem backend to grow a search API. The bash executor owns request defaulting/capping, subprocess execution, process-group termination, environment scrubbing, raw output capture, and backend substitution (local, sandboxed, remote); this package owns schemas, argument validation, shell quoting, parsing, retention, formatted-result spill, and timeout declaration. The tools never call `ctx.bash.start()` and never expose a bash task id — the call returns only after `rg` exits, times out, is aborted, or fails. +Why spawn-backed: local workspace discovery is naturally a process-backed `rg` workflow, and putting search on `ctx.fs` would force every filesystem backend to grow a search API. The subprocess seam owns spawn execution, process-tree termination, environment scrubbing, and bounded output capture; this package owns schemas, argument validation, argv construction, parsing, retention, formatted-result spill, and timeout declaration. The tools never expose a background task — the call returns only after `rg` exits, is terminated by the cooperative timeout, is aborted, or fails. -## Deployment requirement: rg + co-located bash/filesystem +## Deployment requirement: no host rg, co-located workdir/filesystem -The mounted bash executor must be able to resolve `rg` from its `PATH` at plugin load; otherwise `glob` and `grep` are absent from the model-visible tool schema. Returned paths are displayed relative to the resolved bash workdir (the calling agent's session cwd when present, else the executor's configured default) and are follow-up-readable with `read` only when the bash workdir and the filesystem root are the same workspace. v1 documents that co-location requirement and performs no runtime cross-service validation; remote or virtual filesystem search waits for a shared workspace contract or a provider-specific search backend. +The binary ships with the package on every supported platform (macOS/Linux/Windows, x64/arm64), so no host `rg` install is required and the tools register on every deployment. Returned paths are displayed relative to the resolved workdir (the calling agent's session cwd when present, else `process.cwd()`) and are follow-up-readable with `read` only when that workdir and the filesystem root are the same workspace. v1 documents that co-location requirement and performs no runtime cross-service validation; remote or virtual filesystem search waits for a shared workspace contract or a provider-specific search backend. ## Config @@ -29,24 +29,24 @@ The mounted bash executor must be able to resolve `rg` from its `PATH` at plugin | `grepMaxMatches` | `250` | Max flat matches one `grep` call retains inline (matches Claude Code's `GrepTool` `head_limit`); later matches go to the formatted spill artifact. | | `grepMaxLineBytes` | `2000` | Byte cap per matched-line preview; the cut preserves UTF-8 boundaries and is marked `(line truncated)`. | | `rawOutputMaxBytes` | `20000000` | Max complete raw `rg` stdout a search will parse (matches Claude Code's ripgrep raw buffer); larger raw output fails with `SEARCH_RAW_OUTPUT_OVERFLOW`. | -| `timeoutMs` | `30000` | Cooperative tool-call budget attached to both tool definitions, enforced by `@deepseek-ai/dsh-timeout-policy` through `exec.signal`; the bash backend's own timeout stays a second safety cap. | +| `timeoutMs` | `30000` | Cooperative tool-call budget attached to both tool definitions, enforced by `@deepseek-ai/dsh-timeout-policy` through `exec.signal`; the subprocess seam's terminate escalation is the hard kill. | ## Tools | Tool | Arguments | Behavior | |---|---|---| -| `glob` | `pattern`, `path?` | `rg --files --glob --sort=modified --no-ignore --hidden` plus VCS metadata excludes (`.git`, `.svn`, `.hg`, `.bzr`, `.jj`, `.sl`). `path` is an optional **directory** search root; omitted means the resolved bash workdir. Returns one FILE path per line; `rg --files` never emits directory entries. The pattern keeps ripgrep semantics: without a `/` it matches the basename at any depth, so `*` matches the whole tree. Complete results stay modification-time ordered; over-cap presentation follows `sampleOverCapGlobResults`. | +| `glob` | `pattern`, `path?` | `rg --files --glob --sort=modified --no-ignore --hidden` plus VCS metadata excludes (`.git`, `.svn`, `.hg`, `.bzr`, `.jj`, `.sl`). `path` is an optional **directory** search root; omitted means the resolved workdir. Returns one FILE path per line; `rg --files` never emits directory entries. The pattern keeps ripgrep semantics: without a `/` it matches the basename at any depth, so `*` matches the whole tree. Complete results stay modification-time ordered; over-cap presentation follows `sampleOverCapGlobResults`. | | `grep` | `pattern`, `path?`, `include?` | Line-oriented `rg --json` parse (no colon-splitting ambiguity). `pattern` is a ripgrep regex; `path` is an optional **file or directory** target; `include` is ONE positive glob filter — a comma-separated list or a negated (`!…`) value is rejected up front (brace alternation like `*.{ts,tsx}` is fine). Returns matches grouped by file as `Line N: `. | Routine budgets stay out of the model-facing schema (no `head_limit`/`offset`/`case_insensitive`/output modes): a model that needs surrounding context reads the matched file with `read`; one that needs later results follows the returned spill locator's retrieval hint. ## Two budgets, two artifacts -Raw `rg` stdout is an internal transport detail. Each search requests `stdoutMaxBytes: rawOutputMaxBytes` from the bash seam and parses only complete retained stdout; if the executor still returns `stdout.truncated`, the search fails with `SEARCH_RAW_OUTPUT_OVERFLOW` and tells the model to narrow the query. A successful `glob` keeps the displayed search root and every acquired path in `{ root, paths }`; when sampling is enabled, `root` lets the Native renderer group an explicit relative or absolute search path by entries beneath that root rather than by its workdir prefix. `grep` keeps every acquired `{ path, lineNumber, line }` in `{ matches }`. Inline item and per-line preview caps apply only in the Native renderer. For a direct surface call with more logical results than the inline cap, post-policy best-effort saves the complete formatted preview through `ctx.spillStore.saveText()` and replaces only presentation with the configured page plus locator. Nested Code dispatches skip that spill because their full canonical value does not enter model context. Missing/failed spill keeps the inline page and reports that the complete result could not be saved—never an `isError`. +Raw `rg` stdout is an internal transport detail. Each search requests a collect-mode stdout budget of `rawOutputMaxBytes` from the subprocess seam and parses only complete retained stdout; if the seam still reports a lossy read, the search fails with `SEARCH_RAW_OUTPUT_OVERFLOW` and tells the model to narrow the query. A successful `glob` keeps the displayed search root and every acquired path in `{ root, paths }`; when sampling is enabled, `root` lets the Native renderer group an explicit relative or absolute search path by entries beneath that root rather than by its workdir prefix. `grep` keeps every acquired `{ path, lineNumber, line }` in `{ matches }`. Inline item and per-line preview caps apply only in the Native renderer. For a direct surface call with more logical results than the inline cap, post-policy best-effort saves the complete formatted preview through `ctx.spillStore.saveText()` and replaces only presentation with the configured page plus locator. Nested Code dispatches skip that spill because their full canonical value does not enter model context. Missing/failed spill keeps the inline page and reports that the complete result could not be saved—never an `isError`. ## Errors -Search failures carry the package-owned `SearchError` (a `HarnessError` subclass), surfaced as `{ name, code }` on `isError` results: `SEARCH_INVALID_PATTERN` (ripgrep rejected the regex/glob), `SEARCH_FAILED` (runtime `rg` disappearance after registration, inaccessible target, signal kill, malformed `--json` output), `SEARCH_RAW_OUTPUT_OVERFLOW` (raw output over `rawOutputMaxBytes`, or still truncated after the requested stdout capture budget), and `SEARCH_ABORTED` (tool timeout, caller cancellation, or the bash executor's own timeout). ripgrep exit semantics are tool-owned: exit 0 is success with results, exit 1 is a successful empty search (`No files found` / `No matches found`), and only other exits are failures. Model argument mistakes (blank pattern, a list-valued `include`) stay ordinary tool argument errors. +Search failures carry the package-owned `SearchError` (a `HarnessError` subclass), surfaced as `{ name, code }` on `isError` results: `SEARCH_INVALID_PATTERN` (ripgrep rejected the regex/glob), `SEARCH_FAILED` (a failed `rg` launch, inaccessible target, signal kill, malformed `--json` output), `SEARCH_RAW_OUTPUT_OVERFLOW` (raw output over `rawOutputMaxBytes`, or still lossy after the requested stdout capture budget), and `SEARCH_ABORTED` (cooperative tool timeout or caller cancellation). ripgrep exit semantics are tool-owned: exit 0 is success with results, exit 1 is a successful empty search (`No files found` / `No matches found`), and only other exits are failures. Model argument mistakes (blank pattern, a list-valued `include`) stay ordinary tool argument errors. ## Model Experience @@ -54,7 +54,7 @@ Search failures carry the package-owned `SearchError` (a `HarnessError` subclass #### What the model sees -After the load-time `rg` probe succeeds, every request in this plugin's registration scope contains the independently registered glob and grep guidance below. Agent-scoped tool restrictions can hide either schema without removing its prompt section. +Every request in this plugin's registration scope contains the independently registered glob and grep guidance below. Agent-scoped tool restrictions can hide either schema without removing its prompt section. ##### Glob guidance with `sampleOverCapGlobResults: true` @@ -86,7 +86,7 @@ Prefix-stable while the plugin scope, sampling choice, and guidance text are unc #### What the model sees -The glob description states the configured over-cap ordering. The generated [`glob` and `grep` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs-search) use `sampleOverCapGlobResults: true`; schemas are visible only after the load-time `rg` probe succeeds. +The glob description states the configured over-cap ordering. The generated [`glob` and `grep` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs-search) use `sampleOverCapGlobResults: true`; the tools are registered unconditionally. #### Token effect @@ -126,7 +126,7 @@ Append-only; newly visible content follows the reusable request prefix and does ## Known Limitations and Deferred Work -- **Search and file access have no shared-workspace proof** — returned paths are follow-up-readable only when the bash workdir and filesystem root denote the same workspace; the package performs no runtime cross-service validation. -- **Ripgrep is a deployment dependency** — a missing `rg` executable makes the package register no tools or guidance; an incompatible executable or one that disappears after registration fails calls with `SEARCH_FAILED`. Remote or virtual filesystems need a co-located executor or another search consumer. +- **Search and file access have no shared-workspace proof** — returned paths are follow-up-readable only when the workdir and filesystem root denote the same workspace; the package performs no runtime cross-service validation. +- **The packaged binary is fixed at dependency version** — `@vscode/ripgrep` covers the platforms it ships (macOS/Linux/Windows, x64/arm64); an unsupported platform or a corrupted install fails calls with `SEARCH_FAILED`. Remote or virtual filesystems need a co-located workspace or another search consumer. - **The schemas expose one bounded page** — offset pagination, case-mode switches, alternate output modes, and provider-backed discovery remain outside this package; capped complete output requires a spill backend. - **Sampling, when enabled, groups by first path segment beneath the search root only** — an over-cap `glob` page balances across those top-level entries, so a result concentrated deeper (one busy directory inside an otherwise even tree) is still shown unevenly below that level; recursive balancing is deferred. diff --git a/packages/fs/tool-fs-search/README.zh.md b/packages/fs/tool-fs-search/README.zh.md index 7dd6cdf9a2..69ce49f3ad 100644 --- a/packages/fs/tool-fs-search/README.zh.md +++ b/packages/fs/tool-fs-search/README.zh.md @@ -2,21 +2,21 @@ [English](README.md) | 中文 -**面向模型的文件系统发现工具**(`glob`、`grep`)由 **bash 执行器 seam** 支持,而不是由 `ctx.fs` 提供方方法支持。加载时,本包(package)探测 `command -v rg`,探测通过 `ctx.bash` 进行;如果执行器无法在其 `PATH` 上找到 ripgrep,就记录警告,并且不注册工具或提示词段。每次调用都会组装固定的 ripgrep 命令(所有模型控制的值都经过同一个包私有 shell 引用辅助函数),通过 `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` 作为普通前台工具调用运行,解析原始 `rg` 输出,并返回相对于工作目录的规范值。本包注入 `tools`、`systemPrompt` 和 `bash`,有意**不**注入 `fs`;格式化结果 spill 为可选功能,因此机会性读取 `ctx.spillStore`,调用方式为 `ctx.get()`。 +**面向模型的文件系统发现工具**(`glob`、`grep`)由 **打包的 ripgrep 二进制**(`@vscode/ripgrep`)支持,而不是由 `ctx.fs` 提供方方法或系统 `rg` 安装支持。注册是无条件的:二进制随 npm 依赖一起交付,因此没有加载期可用性探针。每次调用都通过 `ctx.subprocess` seam 以固定 argv 向量 spawn 该二进制(模型控制的值是普通 argv 元素——不存在 shell 层,因此无需引号),解析原始 `rg` 输出,并返回相对于工作目录的规范值。本包注入 `tools`、`systemPrompt` 和 `subprocess`,有意**不**注入 `fs`;格式化结果 spill 为可选功能,因此机会性读取 `ctx.spillStore`,调用方式为 `ctx.get()`。 ```ts ignore-check // A deployment chooses how over-cap glob pages are selected. -await ctx.plugin(LocalBashExecutor, { cwd: process.cwd() }) // @deepseek-ai/dsh-bash-local +await ctx.plugin(LocalSubprocessService) // @deepseek-ai/dsh-subprocess-local await ctx.plugin(ToolFsSearch, { sampleOverCapGlobResults: false }) // Optional: a spill backend makes capped results fully recoverable. await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-spill-local ``` -采用 bash 支持的原因:本地工作区发现天然是由进程支持的 `rg` 工作流;如果把搜索放到 `ctx.fs` 上,就会迫使每个文件系统后端扩展搜索 API。bash 执行器负责请求默认值/上限、子进程执行、进程组终止、环境清理、原始输出捕获和后端替换(本地、沙箱化、远程);本包负责 schema、参数校验、shell 引用、解析、保留、格式化结果 spill 和超时声明。工具绝不调用 `ctx.bash.start()`,也不公开 bash task id;只有在 `rg` 退出、超时、中止或失败后,调用才会返回。 +采用 spawn 支持的原因:本地工作区发现天然是由进程支持的 `rg` 工作流;如果把搜索放到 `ctx.fs` 上,就会迫使每个文件系统后端扩展搜索 API。subprocess seam 负责 spawn 执行、进程树终止、环境清理和有界输出捕获;本包负责 schema、参数校验、argv 构造、解析、保留、格式化结果 spill 和超时声明。工具绝不暴露后台任务——只有在 `rg` 退出、被协作式超时终止、被中止或失败后,调用才会返回。 -## 部署要求:rg 与共置的 bash/文件系统 +## 部署要求:无需宿主 rg,但工作目录与文件系统需共置 -已挂载的 bash 执行器必须能在插件加载时解析 `rg`,其来源是执行器的 `PATH`;否则面向模型的工具 schema 中不会出现 `glob` 和 `grep`。返回路径会相对于解析后的 bash 工作目录显示(调用方 agent(智能体)有会话 cwd 时使用该 cwd,否则使用执行器配置的默认值);只有 bash 工作目录与文件系统根目录是同一工作区时,才能用 `read` 继续读取。v1 只记录这项共置要求,不执行运行时跨服务校验;远程或虚拟文件系统搜索需等待共享工作区契约或特定提供方的搜索后端。 +二进制随包交付,覆盖所有受支持平台(macOS/Linux/Windows,x64/arm64),因此无需宿主 `rg` 安装,工具在每个部署上都注册。返回路径会相对于解析后的工作目录显示(调用方 agent(智能体)有会话 cwd 时使用该 cwd,否则使用 `process.cwd()`);只有该工作目录与文件系统根目录是同一工作区时,才能用 `read` 继续读取。v1 只记录这项共置要求,不执行运行时跨服务校验;远程或虚拟文件系统搜索需等待共享工作区契约或特定提供方的搜索后端。 ## 配置 @@ -29,24 +29,24 @@ await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh- | `grepMaxMatches` | `250` | 一次 `grep` 调用内联保留的最大平铺匹配数(与 Claude Code 的 `GrepTool` `head_limit` 相同);后续匹配写入格式化 spill 产物。 | | `grepMaxLineBytes` | `2000` | 每条匹配行预览的字节上限;截断会保留 UTF-8 边界,并标记为 `(line truncated)`。 | | `rawOutputMaxBytes` | `20000000` | 搜索将解析的完整原始 `rg` stdout 上限(与 Claude Code 的 ripgrep 原始 buffer 相同);更大的原始输出以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败。 | -| `timeoutMs` | `30000` | 附加到两个工具定义上的协作式工具调用预算,由 `@deepseek-ai/dsh-timeout-policy` 通过 `exec.signal` 强制执行;bash 后端自身的超时仍作为第二道安全上限。 | +| `timeoutMs` | `30000` | 附加到两个工具定义上的协作式工具调用预算,由 `@deepseek-ai/dsh-timeout-policy` 通过 `exec.signal` 强制执行;subprocess seam 的终止升级提供硬终止。 | ## 工具 | 工具 | 参数 | 行为 | |---|---|---| -| `glob` | `pattern`、`path?` | 运行 `rg --files --glob --sort=modified --no-ignore --hidden`,并排除 VCS 元数据(`.git`、`.svn`、`.hg`、`.bzr`、`.jj`、`.sl`)。`path` 是可选的**目录**搜索根;省略时使用解析后的 bash 工作目录。每行返回一个**文件**路径;`rg --files` 从不输出目录条目。pattern 保留 ripgrep 语义:不含 `/` 时匹配任意深度的基名,因此 `*` 匹配整棵树。完整结果保持按修改时间排序;超过上限时的呈现方式遵循 `sampleOverCapGlobResults`。 | +| `glob` | `pattern`、`path?` | 运行 `rg --files --glob --sort=modified --no-ignore --hidden`,并排除 VCS 元数据(`.git`、`.svn`、`.hg`、`.bzr`、`.jj`、`.sl`)。`path` 是可选的**目录**搜索根;省略时使用解析后的工作目录。每行返回一个**文件**路径;`rg --files` 从不输出目录条目。pattern 保留 ripgrep 语义:不含 `/` 时匹配任意深度的基名,因此 `*` 匹配整棵树。完整结果保持按修改时间排序;超过上限时的呈现方式遵循 `sampleOverCapGlobResults`。 | | `grep` | `pattern`、`path?`、`include?` | 按行解析 `rg --json`,避免按冒号拆分的歧义。`pattern` 是 ripgrep 正则表达式;`path` 是可选的**文件或目录**目标;`include` 是一个正向 glob 过滤器,前置拒绝逗号分隔列表或否定值(`!…`),但允许 `*.{ts,tsx}` 等花括号交替。返回按文件分组、形如 `Line N: ` 的匹配。 | 常规预算不进入面向模型的 schema(没有 `head_limit`/`offset`/`case_insensitive`/输出模式):模型需要周边上下文时,用 `read` 读取匹配文件;需要后续结果时,遵循返回的 spill locator 检索提示。 ## 两类预算、两类产物 -原始 `rg` stdout 是内部传输细节。每次搜索从 bash seam 请求 `stdoutMaxBytes: rawOutputMaxBytes`,且只解析完整保留的 stdout;如果执行器仍返回 `stdout.truncated`,搜索会以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败,并要求模型缩小查询。成功的 `glob` 在 `{ root, paths }` 中保留所显示的搜索根及所有已取得路径;启用采样时,借助 `root`,原生渲染器能以显式的相对或绝对搜索路径为根,按该根下的条目分组,而不是按其工作目录前缀分组。`grep` 保留所有已取得的 `{ path, lineNumber, line }`,并将其存入 `{ matches }`。内联条目和每行预览上限只应用于原生渲染器。直接接口调用的逻辑结果超过内联上限时,后置策略会尽力通过 `ctx.spillStore.saveText()` 保存完整格式化预览,并只把呈现替换为配置指定的页面与 locator。嵌套 Code 分派会跳过 spill,因为其完整规范值不会进入模型上下文。spill 缺失/失败时保留内联页面,并报告完整结果无法保存,绝不会成为 `isError`。 +原始 `rg` stdout 是内部传输细节。每次搜索从 subprocess seam 请求 `rawOutputMaxBytes` 的 collect 模式 stdout 预算,且只解析完整保留的 stdout;如果 seam 仍报告 lossy 读取,搜索会以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败,并要求模型缩小查询。成功的 `glob` 在 `{ root, paths }` 中保留所显示的搜索根及所有已取得路径;启用采样时,借助 `root`,原生渲染器能以显式的相对或绝对搜索路径为根,按该根下的条目分组,而不是按其工作目录前缀分组。`grep` 保留所有已取得的 `{ path, lineNumber, line }`,并将其存入 `{ matches }`。内联条目和每行预览上限只应用于原生渲染器。直接接口调用的逻辑结果超过内联上限时,后置策略会尽力通过 `ctx.spillStore.saveText()` 保存完整格式化预览,并只把呈现替换为配置指定的页面与 locator。嵌套 Code 分派会跳过 spill,因为其完整规范值不会进入模型上下文。spill 缺失/失败时保留内联页面,并报告完整结果无法保存,绝不会成为 `isError`。 ## 错误 -搜索失败携带本包拥有的 `SearchError`(`HarnessError` 子类),以 `{ name, code }` 公开在 `isError` 结果上:`SEARCH_INVALID_PATTERN`(ripgrep 拒绝正则/glob)、`SEARCH_FAILED`(注册后 `rg` 在运行时消失、目标不可访问、信号终止、`--json` 输出格式错误)、`SEARCH_RAW_OUTPUT_OVERFLOW`(原始输出超过 `rawOutputMaxBytes`,或在请求 stdout 捕获预算后仍被截断)和 `SEARCH_ABORTED`(工具超时、调用方取消或 bash 执行器自身超时)。ripgrep 退出语义由工具拥有:退出 0 表示成功且有结果,退出 1 表示成功的空搜索(`No files found` / `No matches found`),只有其他退出值表示失败。模型参数错误(空白 pattern、列表值 `include`)仍是普通工具参数错误。 +搜索失败携带本包拥有的 `SearchError`(`HarnessError` 子类),以 `{ name, code }` 公开在 `isError` 结果上:`SEARCH_INVALID_PATTERN`(ripgrep 拒绝正则/glob)、`SEARCH_FAILED`(`rg` 启动失败、目标不可访问、信号终止、`--json` 输出格式错误)、`SEARCH_RAW_OUTPUT_OVERFLOW`(原始输出超过 `rawOutputMaxBytes`,或在请求 stdout 捕获预算后仍 lossy)和 `SEARCH_ABORTED`(协作式工具超时或调用方取消)。ripgrep 退出语义由工具拥有:退出 0 表示成功且有结果,退出 1 表示成功的空搜索(`No files found` / `No matches found`),只有其他退出值表示失败。模型参数错误(空白 pattern、列表值 `include`)仍是普通工具参数错误。 ## 模型体验 @@ -54,7 +54,7 @@ await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh- #### 模型看到的内容 -加载时 `rg` 探测成功后,该插件注册作用域内的每个请求都包含下方独立注册的 glob 与 grep 指导。agent 作用域的工具限制可以隐藏任一 schema,而不移除其提示词段。 +该插件注册作用域内的每个请求都包含下方独立注册的 glob 与 grep 指导。agent 作用域的工具限制可以隐藏任一 schema,而不移除其提示词段。 ##### 启用 `sampleOverCapGlobResults: true` 时的 Glob 指导 @@ -76,57 +76,57 @@ Use the grep tool — not shell grep or rg — to search file contents. Use read #### Token 影响 -工具注册期间,每个请求支付固定指导成本;必填的采样选项决定采用哪个 glob 变体。 +工具注册期间每个请求有固定的指导成本;必填的采样选择决定采用哪一个 glob 变体。 #### KV Cache 影响 -只要插件作用域、采样选项和指导文本不变,前缀就保持稳定。启用、dispose(资源释放)或更改该选项,可能从该提示词段开始使复用失效。 +插件作用域、采样选择与指导文本不变时前缀稳定。激活、销毁或改变选择可能使该提示词段的复用失效。 ### 工具 schema #### 模型看到的内容 -glob 描述会说明配置所指定的超限结果排序方式。已生成的 [`glob` 和 `grep` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs-search) 使用 `sampleOverCapGlobResults: true`;只有加载时 `rg` 探测成功后,这些 schema 才可见。 +glob 描述声明了配置的超过上限排序方式。生成的 [`glob` 和 `grep` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs-search) 使用 `sampleOverCapGlobResults: true`;工具无条件注册。 #### Token 影响 -工具可见的每个请求都支付固定 schema 成本。 +工具可见时每个请求有固定的 schema 成本。 #### KV Cache 影响 -只要工具可见性和定义不变,前缀就保持稳定。注册生命周期或作用域限制可能从首个变化的 schema token 开始使复用失效。 +工具可见性与定义不变时前缀稳定。注册生命周期或作用域限制可能从第一个改变的 schema token 起使复用失效。 -### 结果与 spill 通知 +### 结果与 spill 提示 #### 模型看到的内容 -`glob` 每行返回一个路径;`grep` 在每个路径下对 `Line : ` 匹配分组。空搜索返回 `No files found` 或 `No matches found`。达到上限的结果末尾会附加省略数量、spill locator 和后端检索提示,或说明完整结果无法保存。`sampleOverCapGlobResults: true` 时,超过上限的 `glob` 页面会在实际搜索根正下方的条目之间按轮转方式取路径,footer 会说明采样依据和触达的顶层条目数;若无法触达全部条目,footer 会要求模型缩小 `path`。设为 `false` 时,页面保留按修改时间排序的前部,并沿用通常用于达到上限结果的 footer。未超过上限的结果原样不动;扁平的采样结果也沿用普通 footer,因为其样本等同于按修改时间排序的前部。spill 产物始终保存按修改时间排序的完整列表。 +`glob` 每行返回一个路径;`grep` 在每个路径下分组展示 `Line : ` 匹配。空搜索返回 `No files found` 或 `No matches found`。达到上限的结果以省略计数结尾,并附 spill locator 与后端检索提示;否则说明完整结果无法保存。启用 `sampleOverCapGlobResults: true` 时,超过上限的 `glob` 页面按实际搜索根正下方的条目轮转取路径,页脚说明采样依据及其覆盖的顶层条目数;无法覆盖全部条目时,页脚提示模型收窄 `path`。`false` 时页面是按修改时间排序的前部,并保留普通的上限结果页脚。未超过上限的结果原样呈现;扁平采样的结果也保留普通页脚,因为其采样等于按修改时间排序的前部。spill 产物始终持有按修改时间排序的完整列表。 #### Token 影响 -内联路径和匹配受 `globMaxResults`、`grepMaxMatches` 与 `grepMaxLineBytes` 限制;调用和保留结果会留在历史中,直到上下文压缩(compaction)。 +内联路径与匹配受 `globMaxResults`、`grepMaxMatches` 与 `grepMaxLineBytes` 约束;调用与保留结果在压缩前留在历史中。 #### KV Cache 影响 -仅追加;新增可见内容位于可复用请求前缀之后,不会使现有 KV-cache 条目失效。 +只追加;新可见内容跟在可复用请求前缀之后,不会使既有 KV-cache 条目失效。 ### 工具错误 #### 模型看到的内容 -失败会规范化为 `Error: `,并向调用方提供结构化的 `SEARCH_INVALID_PATTERN`、`SEARCH_FAILED`、`SEARCH_RAW_OUTPUT_OVERFLOW` 或 `SEARCH_ABORTED` 元数据。 +失败被规范化为 `Error: `,并携带结构化 `SEARCH_INVALID_PATTERN`、`SEARCH_FAILED`、`SEARCH_RAW_OUTPUT_OVERFLOW` 或 `SEARCH_ABORTED` 元数据供调用方使用。 #### Token 影响 -只有失败调用会添加这些保留 token。 +只有失败的调用会增加这些保留 token。 #### KV Cache 影响 -仅追加;新增可见内容位于可复用请求前缀之后,不会使现有 KV-cache 条目失效。 +只追加;新可见内容跟在可复用请求前缀之后,不会使既有 KV-cache 条目失效。 -## 已知限制与暂缓事项 +## 已知局限与延期工作 -- **搜索和文件访问没有共享工作区证明**:只有 bash 工作目录和文件系统根目录表示同一工作区时,返回路径才能继续读取;本包不执行运行时跨服务校验。 -- **Ripgrep 是部署依赖**:缺失 `rg` 可执行文件时,本包不注册工具或指导;可执行文件不兼容或注册后消失时,调用以 `SEARCH_FAILED` 失败。远程或虚拟文件系统需要共置执行器或其他搜索消费方。 -- **schema 只公开一个有界页面**:offset 分页、大小写模式开关、其他输出模式和提供方支持的发现均不在本包内;达到上限的完整输出需要 spill 后端。 -- **启用采样时,只按搜索根下的路径首段分组**:超过上限的 `glob` 页面在这些顶层条目之间做均衡,因此集中在更深层的结果(一棵总体均匀的树里某个特别庞大的子目录)在该层级以下仍然分布不均;递归均衡已延期。 +- **搜索与文件访问没有共享工作区证明**——只有当工作目录与文件系统根目录指向同一工作区时,返回路径才保证可继续读取;本包不执行运行时跨服务校验。 +- **打包二进制固定在依赖版本上**——`@vscode/ripgrep` 覆盖其随附的平台(macOS/Linux/Windows,x64/arm64);不支持的平台或损坏的安装会以 `SEARCH_FAILED` 使调用失败。远程或虚拟文件系统需要共置的工作区或另一个搜索消费方。 +- **schema 只暴露一个有界页面**——偏移分页、大小写开关、替代输出模式与提供方支撑的发现仍不在本包范围内;达到上限的完整输出需要 spill 后端。 +- **启用采样时仅按搜索根正下方的第一段路径分组**——超过上限的 `glob` 页面在这些顶层条目之间平衡,因此集中在更深处的结果(一棵均匀树里某个繁忙目录)在该层级之下仍会呈现不均;递归平衡被延期。 diff --git a/packages/fs/tool-fs-search/package.json b/packages/fs/tool-fs-search/package.json index bf9cf15aa0..8953aea77a 100644 --- a/packages/fs/tool-fs-search/package.json +++ b/packages/fs/tool-fs-search/package.json @@ -1,6 +1,6 @@ { "name": "@deepseek-ai/dsh-tool-fs-search", - "description": "Model-facing filesystem discovery tools (glob, grep) backed by the DeepSeek Harness bash seam (ctx.bash)", + "description": "Model-facing filesystem discovery tools (glob, grep) backed by the packaged ripgrep binary (@vscode/ripgrep)", "version": "0.0.1", "private": true, "type": "module", @@ -27,23 +27,23 @@ ], "license": "BSD-3-Clause", "dependencies": { + "@vscode/ripgrep": "^1.18.0", "schemastery": "^3.18.0" }, "peerDependencies": { - "@deepseek-ai/dsh-bash": "^0.0.1", "@deepseek-ai/dsh-invariants": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", "@deepseek-ai/dsh-retention": "^0.0.1", "@deepseek-ai/dsh-session": "^0.0.1", "@deepseek-ai/dsh-spill": "^0.0.1", + "@deepseek-ai/dsh-subprocess": "^0.0.1", "@deepseek-ai/dsh-system-prompt": "^0.0.1", "@deepseek-ai/dsh-tools": "^0.0.1", "cordis": "^4.0.0-rc.6" }, "devDependencies": { "@deepseek-ai/dsh-agent": "workspace:^", - "@deepseek-ai/dsh-bash": "workspace:^", - "@deepseek-ai/dsh-bash-local": "workspace:^", + "@deepseek-ai/dsh-subprocess": "workspace:^", "@deepseek-ai/dsh-subprocess-local": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", diff --git a/packages/fs/tool-fs-search/src/glob.ts b/packages/fs/tool-fs-search/src/glob.ts index 2670ab57c4..6e7d411a01 100644 --- a/packages/fs/tool-fs-search/src/glob.ts +++ b/packages/fs/tool-fs-search/src/glob.ts @@ -1,10 +1,11 @@ /** * The model-facing `glob` tool: discover files whose paths match a glob - * pattern, sorted by modification time. Execution goes through the bash seam - * (`ctx.bash`) with a fixed `rg --files` command — this module owns the - * model-facing schema, argument validation, shell-safe command construction, - * result parsing, inline sampling, and formatting; process concerns (defaulting, - * scrubbing, kill, backend substitution) stay behind `ctx.bash`. + * pattern, sorted by modification time. Execution spawns the packaged + * ripgrep binary (`@vscode/ripgrep`) directly through the subprocess seam + * with a plain argv vector — this module owns the model-facing schema, + * argument validation, argv construction, result parsing, inline sampling, + * and formatting; process concerns (spawn execution, tree termination, + * environment scrubbing, output capture) stay behind `ctx.subprocess`. * @module @deepseek-ai/dsh-tool-fs-search/glob */ @@ -13,11 +14,9 @@ import { sep } from 'node:path' import { defineTool } from '@deepseek-ai/dsh-tools' import type { GenericCallView, SearchResultView, ToolResult } from '@deepseek-ai/dsh-tools' import type { SpillRef } from '@deepseek-ai/dsh-spill' -import type {} from '@deepseek-ai/dsh-bash' import type {} from '@deepseek-ai/dsh-system-prompt' import { runRipgrep, toWorkdirRelative, trySaveFormattedResult } from './search-core.ts' import { globSearchMeta, searchViewFromMeta } from './presentation.ts' -import { singleQuote } from './shell-quote.ts' import { acceptedSurfaceValue } from './surface.ts' /** @@ -73,32 +72,35 @@ export function parseGlobArgs(args: { pattern: string; path?: string }): GlobInp } /** - * Build the fixed `rg --files` command for one `glob` call. Every + * Build the fixed `rg --files` argv for one `glob` call. Every * model-controlled value ({@link GlobInput.pattern}, {@link GlobInput.path}) - * passes through {@link singleQuote}; the search root rides behind `--` so a - * leading-dash path can never be parsed as a flag. `--sort=modified` orders by - * modification time, `--no-ignore --hidden` searches ignored and hidden files, - * and {@link GLOB_VCS_EXCLUDES} keeps VCS metadata out. + * is a plain argv element — no shell layer exists, so no quoting applies; the + * search root rides behind `--` so a leading-dash path can never be parsed as + * a flag. `--sort=modified` orders by modification time, `--no-ignore + * --hidden` searches ignored and hidden files, and + * {@link GLOB_VCS_EXCLUDES} keeps VCS metadata out. * * @param input - the validated arguments. - * @returns the complete, shell-safe command string. + * @returns the complete ripgrep argument vector (excluding the binary itself). */ -export function buildGlobCommand(input: GlobInput): string { +export function buildGlobCommand(input: GlobInput): string[] { const parts = [ - 'rg --files', - `--glob=${singleQuote(input.pattern)}`, - '--sort=modified --no-ignore --hidden', + '--files', + `--glob=${input.pattern}`, + '--sort=modified', + '--no-ignore', + '--hidden', // Two negated globs per VCS name: the bare form prunes the directory // during traversal; the /** form still excludes the contents when the // search root is AT or INSIDE the directory (where the bare form, // matched against root-prefixed paths, never fires). ...GLOB_VCS_EXCLUDES.flatMap(name => [ - `--glob=${singleQuote(`!**/${name}`)}`, - `--glob=${singleQuote(`!**/${name}/**`)}`, + `--glob=!**/${name}`, + `--glob=!**/${name}/**`, ]), ] - if (input.path !== undefined) parts.push('--', singleQuote(input.path)) - return parts.join(' ') + if (input.path !== undefined) parts.push('--', input.path) + return parts } /** @@ -285,7 +287,7 @@ export function presentGlobResult(_args: { pattern: string; path?: string }, res * Register the `glob` tool and its system-prompt guidance. * * @param ctx - the plugin context; registrations are effects scoped to it, and - * execution uses its `bash` service. + * execution uses its `subprocess` service. * @param caps - the deployment's resolved glob caps (plugin config after defaulting). */ export function applyGlobTool(ctx: Context, caps: GlobToolCaps): void { diff --git a/packages/fs/tool-fs-search/src/grep.ts b/packages/fs/tool-fs-search/src/grep.ts index b7e67ea153..03b49f01e8 100644 --- a/packages/fs/tool-fs-search/src/grep.ts +++ b/packages/fs/tool-fs-search/src/grep.ts @@ -1,11 +1,12 @@ /** * The model-facing `grep` tool: search file contents with a ripgrep regular - * expression. Execution goes through the bash seam (`ctx.bash`) with a fixed - * line-oriented `rg --json` command so file path, line number, and line text - * parse without colon-splitting ambiguity — this module owns the model-facing - * schema, argument validation, shell-safe command construction, `--json` - * record parsing, per-line preview retention, match retention, grouping, and - * formatting; process concerns stay behind `ctx.bash`. + * expression. Execution spawns the packaged ripgrep binary + * (`@vscode/ripgrep`) directly through the subprocess seam with a plain argv + * vector using a fixed line-oriented `rg --json` command so file path, line + * number, and line text parse without colon-splitting ambiguity — this module + * owns the model-facing schema, argument validation, argv construction, + * `--json` record parsing, per-line preview retention, match retention, + * grouping, and formatting; process concerns stay behind `ctx.subprocess`. * * @module @deepseek-ai/dsh-tool-fs-search/grep */ @@ -15,12 +16,10 @@ import { defineTool } from '@deepseek-ai/dsh-tools' import type { GenericCallView, SearchResultView, ToolResult } from '@deepseek-ai/dsh-tools' import type { RetainedItems } from '@deepseek-ai/dsh-retention' import type { SpillRef } from '@deepseek-ai/dsh-spill' -import type {} from '@deepseek-ai/dsh-bash' import type {} from '@deepseek-ai/dsh-system-prompt' import type { GrepMatch } from './search-core.ts' import { SearchError, previewLine, retainGrepMatches, runRipgrep, toWorkdirRelative, trySaveFormattedResult } from './search-core.ts' import { grepSearchMeta, searchViewFromMeta } from './presentation.ts' -import { singleQuote } from './shell-quote.ts' import { acceptedSurfaceValue } from './surface.ts' /** @@ -96,20 +95,21 @@ export function parseGrepArgs(args: { pattern: string; path?: string; include?: } /** - * Build the fixed line-oriented `rg --json` command for one `grep` call. Every + * Build the fixed line-oriented `rg --json` argv for one `grep` call. Every * model-controlled value ({@link GrepInput.pattern}, {@link GrepInput.path}, - * {@link GrepInput.include}) passes through {@link singleQuote}; the pattern - * and include ride in `--flag=value` form and the target behind `--`, so a - * leading-dash value can never be parsed as a flag. + * {@link GrepInput.include}) is a plain argv element — no shell layer exists, + * so no quoting applies; the pattern and include ride in `--flag=value` form + * and the target behind `--`, so a leading-dash value can never be parsed as + * a flag. * * @param input - the validated arguments. - * @returns the complete, shell-safe command string. + * @returns the complete ripgrep argument vector (excluding the binary itself). */ -export function buildGrepCommand(input: GrepInput): string { - const parts = ['rg --json', `--regexp=${singleQuote(input.pattern)}`] - if (input.include !== undefined) parts.push(`--glob=${singleQuote(input.include)}`) - if (input.path !== undefined) parts.push('--', singleQuote(input.path)) - return parts.join(' ') +export function buildGrepCommand(input: GrepInput): string[] { + const parts = ['--json', `--regexp=${input.pattern}`] + if (input.include !== undefined) parts.push(`--glob=${input.include}`) + if (input.path !== undefined) parts.push('--', input.path) + return parts } /** diff --git a/packages/fs/tool-fs-search/src/index.ts b/packages/fs/tool-fs-search/src/index.ts index 072865d568..e596ae3ca1 100644 --- a/packages/fs/tool-fs-search/src/index.ts +++ b/packages/fs/tool-fs-search/src/index.ts @@ -1,28 +1,27 @@ /** * The model-facing filesystem discovery tool suite (`glob`, `grep`) over the - * bash executor seam (`ctx.bash`). This single plugin registers both tools - * only when the mounted bash executor can find `rg` on its `PATH`. + * packaged ripgrep binary (`@vscode/ripgrep`). This single plugin registers + * both tools; the binary ships inside the npm dependency, so no system `rg` + * install and no shell layer is involved. * - * ## Bash-backed, not a `ctx.fs` provider method + * ## Spawn-backed, not a `ctx.fs` provider method * * Local workspace discovery is a process-backed `rg` workflow, so these tools - * execute through `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` with fixed - * ripgrep command templates — never `ctx.bash.start()`, never a model-visible - * background task. The tool layer owns schemas, argument validation, shell - * quoting ({@link module:@deepseek-ai/dsh-tool-fs-search/shell-quote}), result - * parsing, retention, formatted-result spill, and timeout declaration; the - * bash executor owns request defaulting/capping, subprocess execution, - * process-group termination, environment scrubbing, raw output capture, and - * backend substitution. At load, the package probes `command -v rg` through the - * same bash seam; if ripgrep is absent, `glob` / `grep` and their prompt - * sections are not registered. The package injects `tools`, `systemPrompt`, - * and `bash` — deliberately NOT `fs`, and `ctx.spillStore` is read + * execute through `ctx.subprocess.spawn()` with fixed ripgrep argv templates — + * never `ctx.bash`, never `ctx.bash.start()`, never a model-visible background + * task. The tool layer owns schemas, argument validation, argv construction + * ({@link module:@deepseek-ai/dsh-tool-fs-search/glob} / + * {@link module:@deepseek-ai/dsh-tool-fs-search/grep}), result parsing, + * retention, formatted-result spill, and timeout declaration; the subprocess + * seam owns spawn execution, process-tree termination, environment scrubbing, + * and raw output capture. The package injects `tools`, `systemPrompt`, and + * `subprocess` — deliberately NOT `fs`, and `ctx.spillStore` is read * opportunistically with `ctx.get()` because formatted-result spill is optional. * - * Returned paths are displayed relative to the resolved bash workdir and are - * follow-up-readable only in co-located deployments where the bash workdir and - * the filesystem `read` root are the same workspace — a documented v1 - * deployment requirement, not runtime-validated. + * Returned paths are displayed relative to the resolved workdir and are + * follow-up-readable only in co-located deployments where the workdir and the + * filesystem `read` root are the same workspace — a documented v1 deployment + * requirement, not runtime-validated. * * @module @deepseek-ai/dsh-tool-fs-search */ @@ -65,7 +64,7 @@ export { singleQuote } from './shell-quote.ts' export const name = 'tool-fs-search' /** Services required by the search tool suite (`spillStore` is optional, read via `ctx.get()`). */ -export const inject = ['tools', 'systemPrompt', 'bash'] +export const inject = ['tools', 'systemPrompt', 'subprocess'] /** Plugin config; over-cap glob sampling is an explicit deployment choice and the remaining fields have defaults. */ export interface Config { @@ -98,9 +97,6 @@ export const Config: z = z.object({ /** The shape after schemastery applied the defaults. */ type ResolvedConfig = Required -/** POSIX-shell builtin probe for the ripgrep binary in the bash executor environment. */ -const RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1' - /** Every search cap counts items/bytes/milliseconds — a positive integer, or retention and timeout arithmetic misbehaves silently. */ function assertPositiveInteger(name: string, value: number): void { if (!Number.isInteger(value) || value < 1) { @@ -109,36 +105,14 @@ function assertPositiveInteger(name: string, value: number): void { } /** - * Check whether the mounted bash executor can find `rg`. - * - * Nonzero exit means "not available" and disables this optional tool suite. - * Infrastructure failures stay loud: a deployment with a broken bash executor - * should not silently lose tools in a way that looks like a deliberate skip. - * - * @param ctx - plugin context whose `bash` service is the executor the tools will use. - * @returns true when `command -v rg` exits 0, false when it exits nonzero. - */ -async function ripgrepAvailable(ctx: Context): Promise { - const spec = ctx.bash.resolve({ command: RG_PROBE_COMMAND }) - let result - try { - result = await ctx.bash.run(spec) - } catch (error: unknown) { - throw new Error(`tool-fs-search: ripgrep availability probe could not start: ${String(error)}`, { cause: error }) - } - if (result.aborted || result.timedOut || result.signal !== null || result.exitCode === null) { - throw new Error('tool-fs-search: ripgrep availability probe did not complete') - } - return result.exitCode === 0 -} - -/** - * Register the `glob`/`grep` filesystem discovery tool suite when `rg` exists. + * Register the `glob`/`grep` filesystem discovery tool suite. The packaged + * ripgrep binary is always available (an npm dependency), so registration is + * unconditional. * * @param ctx - plugin context; registrations are effects scoped to this plugin. * @param config - resolved plugin configuration from schemastery. - * @returns when ripgrep is unavailable, resolves without registering any tools. */ +// oxlint-disable-next-line typescript/require-await -- async keeps a load-time config rejection a rejection, not a synchronous throw export async function apply(ctx: Context, config: Config): Promise { // schemastery (Config) has already filled every defaulted field. const resolved = config as ResolvedConfig @@ -148,10 +122,6 @@ export async function apply(ctx: Context, config: Config): Promise { assertPositiveInteger('searchMetaMaxBytes', resolved.searchMetaMaxBytes) assertPositiveInteger('rawOutputMaxBytes', resolved.rawOutputMaxBytes) assertPositiveInteger('timeoutMs', resolved.timeoutMs) - if (!await ripgrepAvailable(ctx)) { - ctx.logger.warn('tool-fs-search: ripgrep (rg) not found on the bash executor PATH; glob/grep tools not registered') - return - } applyGlobTool(ctx, { sampleOverCapGlobResults: resolved.sampleOverCapGlobResults, maxResults: resolved.globMaxResults, diff --git a/packages/fs/tool-fs-search/src/ripgrep.d.ts b/packages/fs/tool-fs-search/src/ripgrep.d.ts new file mode 100644 index 0000000000..25d268471e --- /dev/null +++ b/packages/fs/tool-fs-search/src/ripgrep.d.ts @@ -0,0 +1,12 @@ +/** + * Minimal type surface for the `@vscode/ripgrep` package: an ESM module that + * resolves the platform ripgrep binary (`@vscode/ripgrep--` + * optional dependency) and exports its absolute path as the named export + * `rgPath` (no bundled type declarations). + * @module @deepseek-ai/dsh-tool-fs-search/ripgrep-types + */ + +declare module '@vscode/ripgrep' { + /** Absolute path to the packaged ripgrep executable for the current platform. */ + export const rgPath: string +} diff --git a/packages/fs/tool-fs-search/src/search-core.ts b/packages/fs/tool-fs-search/src/search-core.ts index 402fc9d655..c9f5f80b5b 100644 --- a/packages/fs/tool-fs-search/src/search-core.ts +++ b/packages/fs/tool-fs-search/src/search-core.ts @@ -1,16 +1,19 @@ /** * Shared execution plumbing for the `glob` / `grep` search tools: the - * package-owned `SEARCH_*` error vocabulary, one bash-seam run helper that - * turns a fixed `rg` command into complete raw stdout, the best-effort - * formatted-result spill handoff, and workdir-relative path display. + * package-owned `SEARCH_*` error vocabulary, one spawn helper that runs the + * PACKAGED ripgrep binary (`@vscode/ripgrep`) with a plain argv vector and + * returns complete raw stdout, the best-effort formatted-result spill handoff, + * and workdir-relative path display. * - * Both tools execute through `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` - * as ordinary foreground tool calls — never `ctx.bash.start()`, never a - * model-visible background task. Raw `rg` stdout is an internal transport - * detail: the tools request a per-run stdout capture budget from the bash seam, - * parse only complete in-memory stdout within `rawOutputMaxBytes`, and never - * read executor spill files. The model-facing recovery artifact is the - * formatted result saved through `ctx.spillStore.saveText()` + * Both tools execute as ordinary foreground spawns through `ctx.subprocess` — + * never `ctx.bash`, never `ctx.bash.start()`, never a model-visible background + * task. The ripgrep binary ships inside the npm package, so no system `rg` + * install is required, and no shell layer exists between the argv vector and + * ripgrep, so no shell quoting is involved. Raw `rg` stdout is an internal + * transport detail: the tools request a per-run stdout capture budget from the + * subprocess seam, parse only complete in-memory stdout within + * `rawOutputMaxBytes`, and never read spill files. The model-facing recovery + * artifact is the formatted result saved through `ctx.spillStore.saveText()` * ({@link trySaveFormattedResult}). * * @module @deepseek-ai/dsh-tool-fs-search/search-core @@ -18,10 +21,11 @@ import { isAbsolute, relative, sep } from 'node:path' import type { Context } from 'cordis' +import { rgPath } from '@vscode/ripgrep' import { HarnessError } from '@deepseek-ai/dsh-llm' import { ItemRetainer, TextRetainer } from '@deepseek-ai/dsh-retention' import type { RetainedItems } from '@deepseek-ai/dsh-retention' -import type { BashRunResult, CollectedOutput } from '@deepseek-ai/dsh-bash' +import type { SubprocessCollect, SubprocessOutcome, SubprocessOutputRead, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' import type { SaveTextSpill, SpillRef } from '@deepseek-ai/dsh-spill' import type { ToolExecution } from '@deepseek-ai/dsh-tools' @@ -38,6 +42,18 @@ export const RAW_OUTPUT_MAX_BYTES = 20_000_000 */ export const SEARCH_TIMEOUT_MS = 30_000 +/** + * Default cap in bytes on the retained stderr tail of one search run — a + * diagnostic excerpt only (the tool never reads `stderr.spillPath`). + */ +const SEARCH_STDERR_MAX_BYTES = 64 * 1024 + +/** Default whole-stream spill cap for search output (the subprocess seam requires an explicit budget). */ +const SEARCH_SPILL_MAX_BYTES = 64 * 1024 * 1024 + +/** Default terminate grace period for a search process (ms). */ +const SEARCH_GRACE_MS = 3_000 + /** * Default cap in bytes on one search's serialized `presentationMeta` (the * `searchMetaMaxBytes` config). The inline match/path caps already bound the item @@ -52,14 +68,14 @@ export const SEARCH_META_MAX_BYTES = 65_536 /** * Stable, machine-routable codes for search failures. Package-owned (not - * `FsErrorCode`) because these tools are bash-backed discovery, not `ctx.fs` + * `FsErrorCode`) because these tools are spawn-backed discovery, not `ctx.fs` * provider operations: `SEARCH_INVALID_PATTERN` — ripgrep rejected the regex or * glob; `SEARCH_FAILED` — the search could not run or its output could not be - * parsed (missing `rg`, inaccessible target, signal kill, malformed `--json`); - * `SEARCH_RAW_OUTPUT_OVERFLOW` — raw `rg` output exceeded `rawOutputMaxBytes` - * or stayed truncated after that requested stdout budget; `SEARCH_ABORTED` — the tool - * timeout, caller cancellation, or the bash executor's own timeout cut the - * search short. + * parsed (a failed `rg` launch, inaccessible target, signal kill, malformed + * `--json`); `SEARCH_RAW_OUTPUT_OVERFLOW` — raw `rg` output exceeded + * `rawOutputMaxBytes` or stayed truncated after that requested stdout budget; + * `SEARCH_ABORTED` — the cooperative tool timeout or caller cancellation cut + * the search short. */ export type SearchErrorCode = | 'SEARCH_INVALID_PATTERN' @@ -84,7 +100,7 @@ export class SearchError extends HarnessError { /** The completed acquisition of one `rg` run: complete stdout plus the resolved workdir. */ export interface RipgrepRun { - /** Complete raw stdout retained by the bash executor within the requested cap. */ + /** Complete raw stdout retained by the subprocess seam within the requested cap. */ stdout: string /** True when ripgrep exited 1: a successful search with zero results. */ noMatches: boolean @@ -94,73 +110,71 @@ export interface RipgrepRun { /** * The retained stderr tail as a diagnostic excerpt, with a truncation note when - * the executor dropped bytes (the tool never reads `stderr.spillPath`). + * the subprocess seam dropped bytes (the tool never reads `stderr.spillPath`). */ -function stderrExcerpt(stderr: CollectedOutput): string { - const text = stderr.text.trim() +function stderrExcerpt(stderrText: string, truncated: boolean): string { + const text = stderrText.trim() if (text.length === 0) return '' - return stderr.truncated ? `${text} [stderr truncated]` : text + return truncated ? `${text} [stderr truncated]` : text } /** Classify a nonzero-exit `rg` run into the search error vocabulary (invalid pattern vs missing `rg` vs everything else). */ -function classifyRunFailure(toolName: string, result: BashRunResult): SearchError { - const stderr = stderrExcerpt(result.stderr) +function classifyRunFailure(toolName: string, exitCode: number, stderrText: string, stderrTruncated: boolean): SearchError { + const stderr = stderrExcerpt(stderrText, stderrTruncated) if (/regex parse error|error parsing glob/i.test(stderr)) { return new SearchError(`${toolName} pattern rejected by ripgrep: ${stderr}`, 'SEARCH_INVALID_PATTERN') } - if (result.exitCode === 127 || /command not found/i.test(stderr)) { - return new SearchError(`${toolName} requires ripgrep (rg) on the bash executor's PATH${stderr.length > 0 ? `: ${stderr}` : ''}`, 'SEARCH_FAILED') + if (exitCode === 127 || /command not found/i.test(stderr)) { + return new SearchError(`${toolName} requires ripgrep (rg) to launch${stderr.length > 0 ? `: ${stderr}` : ''}`, 'SEARCH_FAILED') } - return new SearchError(`${toolName} search failed (exit ${result.exitCode})${stderr.length > 0 ? `: ${stderr}` : ''}`, 'SEARCH_FAILED') + return new SearchError(`${toolName} search failed (exit ${exitCode})${stderr.length > 0 ? `: ${stderr}` : ''}`, 'SEARCH_FAILED') } /** * Acquire the COMPLETE raw stdout of a finished run, enforcing * `rawOutputMaxBytes` on the in-memory transport. A truncated result means the - * bash backend could not retain complete stdout within the requested budget, so - * the tool fails clearly instead of parsing a silently-partial stream. + * subprocess seam could not retain complete stdout within the requested + * budget, so the tool fails clearly instead of parsing a silently-partial + * stream. */ -function completeStdout(toolName: string, result: BashRunResult, rawOutputMaxBytes: number): string { +function completeStdout(toolName: string, stdout: SubprocessOutputRead, rawOutputMaxBytes: number): string { const narrow = 'narrow pattern, path, or include and retry' - if (!result.stdout.truncated) { - const inlineBytes = Buffer.byteLength(result.stdout.text, 'utf8') + if (!stdout.lossy) { + const inlineBytes = Buffer.byteLength(stdout.text, 'utf8') if (inlineBytes > rawOutputMaxBytes) { throw new SearchError( `${toolName} produced ${inlineBytes} bytes of raw output, over the ${rawOutputMaxBytes}-byte cap; ${narrow}`, 'SEARCH_RAW_OUTPUT_OVERFLOW', ) } - return result.stdout.text + return stdout.text } throw new SearchError( - `${toolName} produced more raw output than the bash executor retained within the ${rawOutputMaxBytes}-byte cap; ${narrow}`, + `${toolName} produced more raw output than the subprocess seam retained within the ${rawOutputMaxBytes}-byte cap; ${narrow}`, 'SEARCH_RAW_OUTPUT_OVERFLOW', ) } /** - * Run one fixed `rg` command through the bash seam and return its complete raw - * stdout. The bash request workdir is the calling agent's session cwd - * (`exec.agent.session.header.cwd`) when available — mirroring `dsh-tool-bash` / - * `dsh-tool-fs` — else omitted so the implementation's `resolve()` applies its - * configured default. `exec.signal` is forwarded so the cooperative tool - * timeout (`@deepseek-ai/dsh-timeout-policy`) and caller cancellation kill the - * command; the bash backend's own timeout stays a second safety cap. + * Run the packaged ripgrep binary with a plain argv vector and return its + * complete raw stdout. The working directory is the calling agent's session + * cwd (`exec.agent.session.header.cwd`) when available, else + * `process.cwd()`. `exec.signal` is forwarded so the cooperative tool timeout + * (`@deepseek-ai/dsh-timeout-policy`) and caller cancellation terminate the + * process tree. * * Exit semantics are tool-owned: exit 0 is success with results, exit 1 is * success with zero results (`noMatches`), anything else throws a * {@link SearchError} (abort/timeout → `SEARCH_ABORTED`, invalid pattern → * `SEARCH_INVALID_PATTERN`, the rest → `SEARCH_FAILED` / - * `SEARCH_RAW_OUTPUT_OVERFLOW`). A `run()` REJECTION — the seam's - * infrastructure failures (pre-aborted signal, unusable workdir, missing - * shell) — is translated into the same taxonomy: a pre-aborted signal becomes - * `SEARCH_ABORTED`, everything else `SEARCH_FAILED`, with the original as - * `cause`. + * `SEARCH_RAW_OUTPUT_OVERFLOW`). A spawn REJECTION — the seam's + * infrastructure failures — is translated into `SEARCH_FAILED` with the + * original as `cause`; a pre-aborted signal becomes `SEARCH_ABORTED`. * - * @param ctx - the plugin context; execution uses its `bash` service. + * @param ctx - the plugin context; execution uses its `subprocess` service. * @param exec - the tool-execution context; supplies the session cwd and the abort signal. * @param toolName - `glob` or `grep`, used in error messages. - * @param command - the fully-quoted `rg` command string (every model value already through `singleQuote`). + * @param argv - the ripgrep arguments (every model value an unquoted argv element; no shell layer exists). * @param rawOutputMaxBytes - cap on the complete raw stdout the tool will parse. * @returns the complete stdout, the zero-result flag, and the resolved workdir. */ @@ -168,54 +182,64 @@ export async function runRipgrep( ctx: Context, exec: ToolExecution, toolName: string, - command: string, + argv: readonly string[], rawOutputMaxBytes: number, ): Promise { - const cwd = exec.agent?.session.header.cwd - const spec = ctx.bash.resolve({ - command, - stdoutMaxBytes: rawOutputMaxBytes, - ...cwd !== undefined ? { workdir: cwd } : {}, - signal: exec.signal, - }) - let result: BashRunResult - try { - result = await ctx.bash.run(spec) - } catch (error: unknown) { - // The seam contract: run() REJECTS only for infrastructure failures — a - // pre-aborted signal, an unusable workdir, a missing shell. Translate them - // so these failures stay machine-routable under the SEARCH_* taxonomy. - if (spec.signal?.aborted === true) { - throw new SearchError(`${toolName} was aborted before completion (tool timeout or caller cancellation)`, 'SEARCH_ABORTED', { cause: error }) - } - throw new SearchError(`${toolName} could not start its search command (unusable working directory or missing shell)`, 'SEARCH_FAILED', { cause: error }) - } - if (result.aborted) { + if (exec.signal.aborted) { throw new SearchError(`${toolName} was aborted before completion (tool timeout or caller cancellation)`, 'SEARCH_ABORTED') } - if (result.timedOut) { - throw new SearchError(`${toolName} timed out after ${result.timeoutMs}ms in the bash executor; narrow pattern, path, or include and retry`, 'SEARCH_ABORTED') + const cwd = exec.agent?.session.header.cwd + const workdir = cwd ?? process.cwd() + const collect = (maxBytes: number): SubprocessCollect => + ({ maxBytes, spill: { maxBytes: SEARCH_SPILL_MAX_BYTES } }) + const handle = ctx.subprocess.spawn({ + argv: [rgPath, ...argv], + cwd: workdir, + stdio: { + stdin: 'ignore', + stdout: collect(rawOutputMaxBytes), + stderr: collect(SEARCH_STDERR_MAX_BYTES), + }, + graceMs: SEARCH_GRACE_MS, + signal: exec.signal, + } satisfies SubprocessSpawnSpec) + let outcome: SubprocessOutcome + try { + outcome = await handle.done + } catch (error: unknown) { + throw new SearchError(`${toolName} could not start its search command (ripgrep launch failed)`, 'SEARCH_FAILED', { cause: error }) } - if (result.signal !== null || result.exitCode === null) { - throw new SearchError(`${toolName} search command was killed by signal ${result.signal ?? '(unknown)'}`, 'SEARCH_FAILED') + const stdout = handle.collected.stdout?.readFrom(0) + const stderr = handle.collected.stderr?.readFrom(0) + if (stdout === undefined || stderr === undefined) { + throw new SearchError(`${toolName} search command produced no collected output streams`, 'SEARCH_FAILED') } - if (result.exitCode !== 0 && result.exitCode !== 1) { - throw classifyRunFailure(toolName, result) + // The signal can abort while the spawn is awaited; the static narrowing that + // proves this re-check "always false" cannot see AbortSignal state changes. + // oxlint-disable-next-line typescript/no-unnecessary-condition + if (exec.signal.aborted) { + throw new SearchError(`${toolName} was aborted before completion (tool timeout or caller cancellation)`, 'SEARCH_ABORTED') } - const stdout = completeStdout(toolName, result, rawOutputMaxBytes) - return { stdout, noMatches: result.exitCode === 1, workdir: spec.workdir } + if (outcome.signal !== null || outcome.exitCode === null) { + throw new SearchError(`${toolName} search command was killed by signal ${outcome.signal ?? '(unknown)'}`, 'SEARCH_FAILED') + } + if (outcome.exitCode !== 0 && outcome.exitCode !== 1) { + throw classifyRunFailure(toolName, outcome.exitCode, stderr.text, stderr.lossy) + } + const text = completeStdout(toolName, stdout, rawOutputMaxBytes) + return { stdout: text, noMatches: outcome.exitCode === 1, workdir } } /** * Map an `rg` output path to its display form: absolute paths inside the - * resolved bash workdir become workdir-relative; everything else (relative - * output, paths outside the workdir) passes through unchanged. Display-only — - * returned paths are follow-up-readable in co-located bash/filesystem + * resolved workdir become workdir-relative; everything else (relative output, + * paths outside the workdir) passes through unchanged. Display-only — + * returned paths are follow-up-readable in co-located workdir/filesystem * deployments where both resolve the same workspace (the documented v1 * deployment requirement). * * @param path - one path as ripgrep printed it. - * @param workdir - the resolved bash workdir the command ran in. + * @param workdir - the resolved workdir the command ran in. * @returns the workdir-relative display path when possible, else `path` unchanged. */ export function toWorkdirRelative(path: string, workdir: string): string { diff --git a/packages/fs/tool-fs-search/src/shell-quote.ts b/packages/fs/tool-fs-search/src/shell-quote.ts index 9453b8e255..ea67abf449 100644 --- a/packages/fs/tool-fs-search/src/shell-quote.ts +++ b/packages/fs/tool-fs-search/src/shell-quote.ts @@ -1,12 +1,9 @@ /** - * The one shell-quoting helper both search tools MUST route every - * model-controlled value through before it enters an `rg` command string. The - * bash seam (`ctx.bash`) accepts a command STRING, not an argv vector, so this - * is the safety boundary that stops a `pattern`, `path`, or `include` from - * breaking out of its argument and injecting shell syntax. - * - * Command builders in `glob.ts` / `grep.ts` must never hand-roll quoting or - * concatenate an unquoted model value — they call {@link singleQuote}. + * POSIX single-quoting helper retained for compatibility with older + * deployments and tests. The current `glob`/`grep` command builders spawn the + * packaged ripgrep binary with a plain argv vector — no shell layer exists — + * so no quoting is involved; this module is kept because its export is part + * of the package surface. * * @module @deepseek-ai/dsh-tool-fs-search/shell-quote */ diff --git a/packages/fs/tool-fs-search/tests/integration.spec.ts b/packages/fs/tool-fs-search/tests/integration.spec.ts index 8cb96e7e66..dc7a88b30f 100644 --- a/packages/fs/tool-fs-search/tests/integration.spec.ts +++ b/packages/fs/tool-fs-search/tests/integration.spec.ts @@ -1,15 +1,16 @@ /** - * Integration tests: the REAL local bash executor (`dsh-bash-local`) plus a - * REAL ripgrep binary, exercised through `ctx.tools.execute()`. These verify - * the WORLD — actual files on disk are discovered and grepped, hostile - * patterns stay inert in a real shell, and real `rg` stderr classifies into - * the `SEARCH_*` vocabulary. The whole suite self-skips when `rg` is not on - * PATH (a CI accommodation mirroring the keyless e2e skip); the fake-executor - * suite (tools.spec.ts) carries the coverage gate. + * Integration tests: the REAL local subprocess service plus the PACKAGED + * ripgrep binary (`@vscode/ripgrep`), exercised through `ctx.tools.execute()`. + * These verify the WORLD — actual files on disk are discovered and grepped, + * hostile patterns stay inert (they are plain argv elements; there is no + * shell layer to escape), and real `rg` stderr classifies into the + * `SEARCH_*` vocabulary. The binary ships inside the npm dependency, so the + * suite runs on every platform without a system `rg` install; the + * fake-service suite (tools.spec.ts) carries the coverage gate. */ import { afterEach, beforeEach, describe, expect, it } from 'vitest' -import { spawnSync } from 'node:child_process' +import { existsSync } from 'node:fs' import { mkdir, mkdtemp, rm, utimes, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' @@ -17,14 +18,11 @@ import { Context } from 'cordis' import { CallId } from '@deepseek-ai/dsh-llm' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry, { TOOL_ABORTED_BEFORE_DISPATCH } from '@deepseek-ai/dsh-tools' -import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import * as ToolFsSearch from '@deepseek-ai/dsh-tool-fs-search' const testToolSignal = new AbortController().signal -const hasRg = spawnSync('rg', ['--version'], { encoding: 'utf8' }).status === 0 - let dir: string let ctx: Context @@ -43,7 +41,10 @@ function text(result: { content: { type: string; text?: string }[] }): string { return result.content.filter(b => b.type === 'text').map(b => b.text).join('') } -describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', () => { +/** The fixture workspace as a session cwd, so relative paths resolve inside `dir`. */ +const agent = () => ({ session: { header: { id: 'session-int', cwd: dir } } }) + +describe('search tools over the real subprocess service + the packaged rg', () => { beforeEach(async () => { dir = await mkdtemp(join(tmpdir(), 'dsh-search-int-')) await mkdir(join(dir, 'src'), { recursive: true }) @@ -54,7 +55,7 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', () await writeFile(join(dir, 'notes.md'), 'alpha appears here too\n') await writeFile(join(dir, '.hidden.ts'), 'export const hidden = 3\n') await writeFile(join(dir, '.git', 'config.ts'), 'never listed\n') - await writeFile(join(dir, 'spaced dir', "wei'rd \"name\".ts"), 'const inside = true\n') + await writeFile(join(dir, 'spaced dir', "wei'rd name.ts"), 'const inside = true\n') // Deterministic --sort=modified order: alpha oldest, beta newest. await utimes(join(dir, 'src', 'alpha.ts'), new Date(2000, 0, 1), new Date(2000, 0, 1)) await utimes(join(dir, 'src', 'beta.ts'), new Date(2020, 0, 1), new Date(2020, 0, 1)) @@ -63,7 +64,6 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', () await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) await ctx.plugin(LocalSubprocessService) - await ctx.plugin(LocalBashExecutor, { cwd: dir, timeoutMs: 20_000 }) await ctx.plugin(ToolFsSearch, { sampleOverCapGlobResults: true }) }) @@ -73,33 +73,33 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', () describe('glob', () => { it('discovers files by pattern, sorted by modification time, hidden included, .git excluded', async () => { - const result = await call('glob', { pattern: '**/*.ts' }) + const result = await call('glob', { pattern: '**/*.ts' }, agent()) expect(result.isError).toBe(false) const paths = text(result).split('\n') - expect(paths.indexOf('src/alpha.ts')).toBeLessThan(paths.indexOf('src/beta.ts')) + expect(paths.indexOf(join('src', 'alpha.ts'))).toBeLessThan(paths.indexOf(join('src', 'beta.ts'))) expect(paths).toContain('.hidden.ts') - expect(paths).toContain("spaced dir/wei'rd \"name\".ts") - expect(paths).not.toContain('.git/config.ts') + expect(paths).toContain(join('spaced dir', "wei'rd name.ts")) + expect(paths).not.toContain(join('.git', 'config.ts')) expect(paths).not.toContain('notes.md') }) it('scopes to a directory search root (path arg)', async () => { - const result = await call('glob', { pattern: '*.ts', path: 'src' }) - expect(text(result).split('\n').sort()).toEqual(['src/alpha.ts', 'src/beta.ts']) + const result = await call('glob', { pattern: '*.ts', path: 'src' }, agent()) + expect(text(result).split('\n').sort()).toEqual([join('src', 'alpha.ts'), join('src', 'beta.ts')]) }) it('reports zero discoveries as No files found', async () => { - expect(text(await call('glob', { pattern: '*.nomatch' }))).toBe('No files found') + expect(text(await call('glob', { pattern: '*.nomatch' }, agent()))).toBe('No files found') }) it('excludes VCS internals even when the search root IS the VCS directory', async () => { // The prune glob alone never matches root-prefixed paths when rg is // rooted at .git; the paired contents glob keeps the exclusion airtight. - expect(text(await call('glob', { pattern: '*', path: '.git' }))).toBe('No files found') + expect(text(await call('glob', { pattern: '*', path: '.git' }, agent()))).toBe('No files found') }) it('classifies an invalid glob as SEARCH_INVALID_PATTERN', async () => { - const result = await call('glob', { pattern: '[' }) + const result = await call('glob', { pattern: '[' }, agent()) expect(result.isError).toBe(true) expect(result.error).toMatchObject({ info: { name: 'SearchError', code: 'SEARCH_INVALID_PATTERN' } }) }) @@ -107,37 +107,42 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', () describe('grep', () => { it('greps a directory tree with grouped, line-numbered output', async () => { - const result = await call('grep', { pattern: 'alpha' }) + const result = await call('grep', { pattern: 'alpha' }, agent()) expect(result.isError).toBe(false) const output = text(result) expect(output).toContain('Found 3 matches') - expect(output).toContain('src/alpha.ts\nLine 1: export const alpha = 1\nLine 2: // TODO: refit alpha') + expect(output).toContain(`${join('src', 'alpha.ts')}\nLine 1: export const alpha = 1\nLine 2: // TODO: refit alpha`) expect(output).toContain('notes.md\nLine 1: alpha appears here too') }) it('greps a single FILE target', async () => { - const result = await call('grep', { pattern: 'alpha', path: 'notes.md' }) + const result = await call('grep', { pattern: 'alpha', path: 'notes.md' }, agent()) expect(text(result)).toBe('Found 1 match\n\nnotes.md\nLine 1: alpha appears here too') }) it('greps a directory target with an include filter', async () => { - const result = await call('grep', { pattern: 'alpha', path: '.', include: '*.ts' }) + const result = await call('grep', { pattern: 'alpha', path: '.', include: '*.ts' }, agent()) const output = text(result) expect(output).toContain('alpha.ts') expect(output).not.toContain('notes.md') }) - it('a hostile pattern stays inert (no command substitution, the world untouched)', async () => { + it('a hostile pattern stays inert (a plain argv element, the world untouched)', async () => { + // There is no shell layer between the argv vector and rg, so the pattern + // is a literal regex — but the world-untouched guarantee is the shipped + // contract, and a future shell-wrapping change must not reintroduce it. + // The canary name carries no path so the regex stays valid on every + // platform (a Windows path's backslashes would be regex escapes). const canary = join(dir, 'pwned') - const result = await call('grep', { pattern: `$(touch ${canary})` }) + const result = await call('grep', { pattern: '$(touch pwned)' }, agent()) expect(result.isError).toBe(false) // exit 1: found nothing, executed nothing expect(text(result)).toBe('No matches found') - expect(spawnSync('test', ['-e', canary]).status).not.toBe(0) + expect(existsSync(canary)).toBe(false) }) it('a leading-dash pattern is a pattern, not a flag', async () => { await writeFile(join(dir, 'dashes.txt'), 'value --flag value\n') - const result = await call('grep', { pattern: '--flag', path: 'dashes.txt' }) + const result = await call('grep', { pattern: '--flag', path: 'dashes.txt' }, agent()) expect(text(result)).toBe('Found 1 match\n\ndashes.txt\nLine 1: value --flag value') }) @@ -155,7 +160,7 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', () }) describe('per-session cwd', () => { - it('resolves the search in the SESSION workspace, not the executor config cwd', async () => { + it('resolves the search in the SESSION workspace, not the process cwd', async () => { const sessionDir = await mkdtemp(join(tmpdir(), 'dsh-search-session-')) try { await writeFile(join(sessionDir, 'only-here.ts'), 'const sessionFile = true\n') @@ -170,7 +175,7 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', () }) }) - describe('pre-dispatch cancellation and bash-start failures', () => { + describe('pre-dispatch cancellation and spawn failures', () => { it('a pre-aborted registry call is ABORTED_BEFORE_DISPATCH', async () => { const controller = new AbortController() controller.abort() diff --git a/packages/fs/tool-fs-search/tests/load-path.spec.ts b/packages/fs/tool-fs-search/tests/load-path.spec.ts index 71022720fc..1d1e348c09 100644 --- a/packages/fs/tool-fs-search/tests/load-path.spec.ts +++ b/packages/fs/tool-fs-search/tests/load-path.spec.ts @@ -3,14 +3,15 @@ * a NAMESPACE plugin with `inject` — so a stray `export default apply` would * make the cordis Loader's `unwrapExports` (`exports.default ?? exports`) * collapse the module to the bare `apply` function, DROPPING `inject`. The - * plugin would then read `ctx.bash` without having injected it and throw + * plugin would then read `ctx.subprocess` without having injected it and throw * `cannot get property … without inject` the moment it loads (postmortem 0001). * * A hand-built `ctx.plugin({ apply, inject })` mount CANNOT catch that — it * bypasses `unwrapExports`. So this test unwraps the module through the REAL - * `Loader.prototype.unwrapExports` and mounts the result over a bash executor, - * exercising the exact path the Loader uses. Prove the guard bites: add - * `export default apply` to `src/index.ts`, watch this go red, revert. + * `Loader.prototype.unwrapExports` and mounts the result over the real local + * subprocess service, exercising the exact path the Loader uses. Prove the + * guard bites: add `export default apply` to `src/index.ts`, watch this go + * red, revert. */ import { describe, expect, it } from 'vitest' @@ -18,48 +19,9 @@ import { Context } from 'cordis' import Loader from '@cordisjs/plugin-loader' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' -import { BashExecutor } from '@deepseek-ai/dsh-bash' -import type { BashExecRequest, BashExecSpec, BashProcess, BashRunResult } from '@deepseek-ai/dsh-bash' +import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import * as toolFsSearch from '@deepseek-ai/dsh-tool-fs-search' -const RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1' - -/** - * Deterministic bash service for this Loader guard: the test wants to exercise - * the real unwrap/inject path, not depend on whether the host image has rg. - */ -class ProbeSuccessBashExecutor extends BashExecutor { - override resolve(request: BashExecRequest): BashExecSpec { - return { - command: request.command, - workdir: request.workdir ?? '/work', - timeoutMs: request.timeoutMs ?? 60_000, - stdoutMaxBytes: request.stdoutMaxBytes ?? 64_000, - signal: request.signal, - sandboxPolicy: request.sandboxPolicy, - } - } - - override run(spec: BashExecSpec): Promise { - if (spec.command !== RG_PROBE_COMMAND) { - throw new Error(`unexpected command in load-path guard: ${spec.command}`) - } - return Promise.resolve({ - exitCode: 0, - signal: null, - timedOut: false, - aborted: false, - timeoutMs: spec.timeoutMs, - stdout: { text: '', truncated: false }, - stderr: { text: '', truncated: false }, - }) - } - - override start(): BashProcess { - throw new Error('load-path guard must not start background processes') - } -} - describe('dsh-tool-fs-search real-load-path guard', () => { it('has no default export and keeps name/inject/Config through unwrapExports', () => { expect('default' in toolFsSearch).toBe(false) @@ -68,16 +30,16 @@ describe('dsh-tool-fs-search real-load-path guard', () => { const unwrapped = loader.unwrapExports(toolFsSearch) as Record expect(unwrapped).toBe(toolFsSearch) expect(unwrapped.name).toBe('tool-fs-search') - expect(unwrapped.inject).toEqual(['tools', 'systemPrompt', 'bash']) + expect(unwrapped.inject).toEqual(['tools', 'systemPrompt', 'subprocess']) expect(typeof unwrapped.Config).toBe('function') expect(typeof unwrapped.apply).toBe('function') }) - it('boots over ctx.bash through the unwrapped module without an inject error', async () => { + it('boots over ctx.subprocess through the unwrapped module without an inject error', async () => { const ctx = new Context() await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) - await ctx.plugin(ProbeSuccessBashExecutor) + await ctx.plugin(LocalSubprocessService) const loader = Object.create(Loader.prototype) as Loader const unwrapped = loader.unwrapExports(toolFsSearch) as Parameters[0] diff --git a/packages/fs/tool-fs-search/tests/tools.spec.ts b/packages/fs/tool-fs-search/tests/tools.spec.ts index 389d8dbe05..d786fa3181 100644 --- a/packages/fs/tool-fs-search/tests/tools.spec.ts +++ b/packages/fs/tool-fs-search/tests/tools.spec.ts @@ -1,23 +1,24 @@ /** - * Consumer-surface tests for the search tools over a FAKE bash executor and a - * FAKE spill backend, exercised through `ctx.tools.execute()` so nothing - * bypasses the tool registry. The fake executor makes every seam outcome - * scriptable — registration-time `rg` probing, truncated stdout with/without a - * raw spill path, abort/timeout, signal kills, ripgrep exit codes — so these - * tests verify schemas, argument validation, shell-safe command construction, - * workdir derivation, signal forwarding, `SEARCH_*` error classification, - * retention, formatted-result spill handoff, and the no-background-task - * invariant. Real-`rg` behavior is pinned separately in integration.spec.ts. + * Consumer-surface tests for the search tools over a FAKE subprocess service + * and a FAKE spill backend, exercised through `ctx.tools.execute()` so nothing + * bypasses the tool registry. The fake service makes every seam outcome + * scriptable — spawn failure, truncated stdout with/without a raw spill path, + * abort/timeout kills, signal kills, ripgrep exit codes — so these tests + * verify schemas, argument validation, argv construction, workdir derivation, + * signal forwarding, `SEARCH_*` error classification, retention, + * formatted-result spill handoff, and the no-background-task invariant. + * Real-`rg` behavior is pinned separately in integration.spec.ts. */ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' import { join, sep } from 'node:path' -import { createUserMessage, CallId } from '@deepseek-ai/dsh-llm' +import { createUserMessage, CallId } from '@deepseek-ai/dsh-llm' import SystemPrompt, { renderPrompt } from '@deepseek-ai/dsh-system-prompt' -import ToolRegistry, { TOOL_ABORTED_BEFORE_DISPATCH, type ToolExecutionToken } from '@deepseek-ai/dsh-tools' -import { BashExecutor } from '@deepseek-ai/dsh-bash' -import type { BashExecRequest, BashExecSpec, BashProcess, BashRunResult } from '@deepseek-ai/dsh-bash' +import ToolRegistry, { TOOL_ABORTED_BEFORE_DISPATCH, type ToolExecution, type ToolExecutionToken } from '@deepseek-ai/dsh-tools' +import { SubprocessService } from '@deepseek-ai/dsh-subprocess' +import type { SubprocessCollectedOutputs, SubprocessHandle, SubprocessOutcome, SubprocessOutputRead, SubprocessOutputReader, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import { rgPath } from '@vscode/ripgrep' import { SpillLocator, SpillStore } from '@deepseek-ai/dsh-spill' import type { SaveTextSpill, SpillRef } from '@deepseek-ai/dsh-spill' import * as ToolFsSearch from '@deepseek-ai/dsh-tool-fs-search' @@ -31,68 +32,124 @@ import { presentGrepCall, presentGrepResult, previewLine, + runRipgrep, sampleAcrossTopLevel, toWorkdirRelative, } from '@deepseek-ai/dsh-tool-fs-search' const testToolSignal = new AbortController().signal -const RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1' -/** A successful run result over the given stdout; overrides script the failure shapes. */ -function runResult(stdout: string, overrides?: Partial): BashRunResult { +/** One scripted collect-mode stream, returned by `readFrom(0)` after settlement. */ +interface ScriptedStream { + text: string + lossy?: boolean + spillPath?: string +} + +/** One scripted spawn: exit facts plus the collected streams the tool reads. */ +interface ScriptedRun { + outcome: SubprocessOutcome + stdout: ScriptedStream + stderr: ScriptedStream +} + +/** A successful run over the given stdout; overrides script the failure shapes. */ +function runResult( + stdout: string, + overrides?: Partial & { stdout?: Partial; stderr?: ScriptedStream }, +): ScriptedRun { + const { stdout: stdoutOverrides, stderr: stderrOverrides, ...outcome } = overrides ?? {} return { - exitCode: 0, - signal: null, - timedOut: false, - aborted: false, - timeoutMs: 60_000, - stdout: { text: stdout, truncated: false }, - stderr: { text: '', truncated: false }, - ...overrides, + outcome: { exitCode: 0, signal: null, ...outcome }, + stdout: { text: stdout, ...stdoutOverrides }, + stderr: { text: '', ...stderrOverrides }, + } +} + +/** A fixed-response collect-mode reader: the tools read each stream once, from 0, after settlement. */ +class FakeReader implements SubprocessOutputReader { + constructor(private readonly read: ScriptedStream) {} + + readFrom(_fromByte: number): SubprocessOutputRead { + return { + text: this.read.text, + nextOffset: 0, + lossy: this.read.lossy ?? false, + ...this.read.spillPath !== undefined ? { spillPath: this.read.spillPath } : {}, + } } } /** - * A scriptable fake executor: `resolve()` mirrors the real request→spec - * defaulting (workdir falls back to `/work`), `run()` returns whatever the - * test armed via `handler`, and `start()` throws — the search tools must NEVER - * create a background task. + * A scriptable subprocess handle: `done` resolves with the scripted outcome + * (or rejects with the scripted error), `terminate()` records the call, and + * the spec's abort signal marks the handle terminated — mirroring the seam's + * abort→terminate escalation. */ -class FakeBash extends BashExecutor { - probeRequests: BashExecRequest[] = [] - probeSpecs: BashExecSpec[] = [] - requests: BashExecRequest[] = [] - specs: BashExecSpec[] = [] - startCalls = 0 - forwardSignal = true - probeResult: BashRunResult = runResult('') - probeError?: Error - handler: (spec: BashExecSpec) => BashRunResult = () => runResult('') +class FakeHandle implements SubprocessHandle { + readonly pid = 4242 + readonly stdin = undefined + readonly stdout = undefined + readonly stderr = undefined + readonly collected: SubprocessCollectedOutputs + readonly done: Promise + /** True once `done` settled — the search tools must never leave a spawn running. */ + settled = false + /** True when the handle's termination path ran (abort signal or explicit terminate). */ + terminated = false + /** Scripted handle that drops one requested collect reader (the defensive branch). */ + readonly dropReaders: boolean - override resolve(request: BashExecRequest): BashExecSpec { - if (request.command === RG_PROBE_COMMAND) this.probeRequests.push(request) - else this.requests.push(request) - return { - command: request.command, - workdir: request.workdir ?? '/work', - timeoutMs: request.timeoutMs ?? 60_000, - stdoutMaxBytes: request.stdoutMaxBytes ?? 64_000, - ...this.forwardSignal ? { signal: request.signal } : {}, - sandboxPolicy: request.sandboxPolicy, + constructor(spec: SubprocessSpawnSpec, script: () => ScriptedRun | { reject: Error }, dropReaders = false) { + this.dropReaders = dropReaders + // The abort listener attaches BEFORE the scripted run resolves, mirroring + // a real spawn: the escalation is armed when the process starts. + spec.signal?.addEventListener('abort', () => { this.terminated = true }, { once: true }) + const scripted = script() + if ('reject' in scripted) { + // A spawn failure produces no process output, so no readers exist. + this.collected = {} + this.done = Promise.reject(scripted.reject) + } else { + this.collected = { + ...dropReaders ? {} : { stdout: new FakeReader(scripted.stdout), stderr: new FakeReader(scripted.stderr) }, + } + this.done = Promise.resolve(scripted.outcome) } + this.done.then( + () => { this.settled = true }, + () => { this.settled = true }, + ) } - override async run(spec: BashExecSpec): Promise { - if (spec.command === RG_PROBE_COMMAND) { - this.probeSpecs.push(spec) - if (this.probeError) throw this.probeError - return this.probeResult - } - this.specs.push(spec) - return this.handler(spec) + + terminate(): void { + this.terminated = true } - override start(): BashProcess { - this.startCalls++ - throw new Error('search tools must never start a background task') + + waitForExit(_signal?: AbortSignal): Promise { + return Promise.resolve(true) + } +} + +/** + * A scriptable fake subprocess service: `spawn()` records every spec and + * returns a handle scripted by the armed `handler`. The search tools must + * never spawn outside a single awaited foreground call, so every test can + * assert on the exact spawn specs and settled handles. + */ +class FakeSubprocess extends SubprocessService { + spawns: SubprocessSpawnSpec[] = [] + handles: FakeHandle[] = [] + /** Arms the per-spawn script; a `{ reject }` return scripts a spawn-level failure. */ + handler: (spec: SubprocessSpawnSpec) => ScriptedRun | { reject: Error } = () => runResult('') + /** When true, spawned handles drop their collect readers (the defensive branch). */ + dropReaders = false + + override spawn(spec: SubprocessSpawnSpec): SubprocessHandle { + this.spawns.push(spec) + const handle = new FakeHandle(spec, () => this.handler(spec), this.dropReaders) + this.handles.push(handle) + return handle } } @@ -115,8 +172,6 @@ class FakeSpill extends SpillStore { interface SetupOptions { config?: Partial spill?: boolean - probeError?: Error - probeResult?: BashRunResult } const DEFAULT_CONFIG = { sampleOverCapGlobResults: true } satisfies ToolFsSearch.Config @@ -127,26 +182,12 @@ async function setup(options: SetupOptions = {}) { ctx.logger.warn = ((message: unknown) => { warnings.push(String(message)) }) as typeof ctx.logger.warn await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) - await ctx.plugin(FakeBash) - const bash = ctx.bash as FakeBash - if (options.probeResult) bash.probeResult = options.probeResult - if (options.probeError) bash.probeError = options.probeError + await ctx.plugin(FakeSubprocess) + const subprocess = ctx.subprocess as FakeSubprocess if (options.spill === true) await ctx.plugin(FakeSpill) const fiber = await ctx.plugin(ToolFsSearch, { ...DEFAULT_CONFIG, ...options.config }) const spill = options.spill === true ? ctx.get('spillStore') as FakeSpill : undefined - return { ctx, bash, spill, fiber, warnings } -} - -/** Assert plugin setup rejects without letting Vitest pretty-print a live Context on failure. */ -async function expectSetupRejects(options: SetupOptions, message: RegExp): Promise { - let thrown: string | undefined - try { - const loaded = await setup(options) - await loaded.fiber.dispose() - } catch (error: unknown) { - thrown = error instanceof Error ? error.message : String(error) - } - expect(thrown).toMatch(message) + return { ctx, subprocess, spill, fiber, warnings } } /** A stand-in agent whose session header carries the given cwd (and a stable id). */ @@ -180,11 +221,11 @@ function matchLine(path: string, lineNumber: number, lineText: string): string { } describe('registration', () => { - it('registers glob and grep with their prompt sections', async () => { - const { ctx, bash } = await setup() - expect(bash.probeRequests).toHaveLength(1) - expect(bash.probeRequests[0]?.command).toBe(RG_PROBE_COMMAND) - expect(bash.probeRequests[0]).not.toHaveProperty('workdir') + it('registers glob and grep unconditionally with their prompt sections', async () => { + const { ctx, subprocess } = await setup() + // Registration performs NO load-time probe: the packaged binary is always + // available, so nothing spawns until a tool call. + expect(subprocess.spawns).toHaveLength(0) expect(ctx.tools.schemas().map(s => s.name).sort()).toEqual(['glob', 'grep']) const prompt = renderPrompt(await ctx.systemPrompt.assemble()) expect(prompt).toContain('Use the glob tool') @@ -195,32 +236,11 @@ describe('registration', () => { expect(glob?.description).toContain('sampled across top-level entries') }) - it('does not register glob or grep when the bash executor cannot find rg', async () => { - const { ctx, warnings } = await setup({ probeResult: runResult('', { exitCode: 1 }) }) - expect(ctx.tools.schemas()).toHaveLength(0) - const sections = (await ctx.systemPrompt.assemble()).sections.map(s => s.name) - expect(sections).not.toContain('tool:glob') - expect(sections).not.toContain('tool:grep') - expect(warnings).toEqual([ - 'tool-fs-search: ripgrep (rg) not found on the bash executor PATH; glob/grep tools not registered', - ]) - }) - - it('rejects plugin load when the rg availability probe cannot run', async () => { - await expectSetupRejects({ probeError: new Error('spawn bash ENOENT') }, /spawn bash ENOENT/) - }) - - it('rejects plugin load when the rg availability probe is aborted or killed', async () => { - await expectSetupRejects({ - probeResult: runResult('', { aborted: true, exitCode: null, signal: 'SIGTERM' }), - }, /tool-fs-search: ripgrep availability probe did not complete/) - }) - - it('stays pending until ctx.bash exists (inject)', async () => { + it('stays pending until ctx.subprocess exists (inject)', async () => { const ctx = new Context() await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) - await ctx.plugin(ToolFsSearch, DEFAULT_CONFIG) // no bash executor + await ctx.plugin(ToolFsSearch, DEFAULT_CONFIG) // no subprocess service expect(ctx.tools.schemas()).toHaveLength(0) }) @@ -276,139 +296,176 @@ describe('config validation', () => { const ctx = new Context() await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) - await ctx.plugin(FakeBash) + await ctx.plugin(FakeSubprocess) await expect(ctx.plugin(ToolFsSearch, { ...DEFAULT_CONFIG, ...config })).rejects.toThrow(new RegExp(`tool-fs-search: ${name} must be a positive integer`)) }) }) -describe('command construction (shell-safe)', () => { - it('glob: fixed rg --files template with quoted pattern and paired VCS excludes', () => { - const command = buildGlobCommand({ pattern: '**/*.ts' }) - expect(command).toBe( - "rg --files --glob='**/*.ts' --sort=modified --no-ignore --hidden " - + "--glob='!**/.git' --glob='!**/.git/**' --glob='!**/.svn' --glob='!**/.svn/**' " - + "--glob='!**/.hg' --glob='!**/.hg/**' --glob='!**/.bzr' --glob='!**/.bzr/**' " - + "--glob='!**/.jj' --glob='!**/.jj/**' --glob='!**/.sl' --glob='!**/.sl/**'", - ) +describe('command construction (plain argv)', () => { + it('glob: fixed rg --files argv with the pattern and paired VCS excludes', () => { + expect(buildGlobCommand({ pattern: '**/*.ts' })).toEqual([ + '--files', + '--glob=**/*.ts', + '--sort=modified', + '--no-ignore', + '--hidden', + '--glob=!**/.git', '--glob=!**/.git/**', + '--glob=!**/.svn', '--glob=!**/.svn/**', + '--glob=!**/.hg', '--glob=!**/.hg/**', + '--glob=!**/.bzr', '--glob=!**/.bzr/**', + '--glob=!**/.jj', '--glob=!**/.jj/**', + '--glob=!**/.sl', '--glob=!**/.sl/**', + ]) }) - it('glob: the search root rides behind -- and is quoted', () => { - const command = buildGlobCommand({ pattern: '*.md', path: 'docs dir' }) - expect(command).toContain("-- 'docs dir'") + it('glob: the search root rides behind -- as a plain element', () => { + expect(buildGlobCommand({ pattern: '*.md', path: 'docs dir' })).toEqual(['--files', '--glob=*.md', '--sort=modified', '--no-ignore', '--hidden', + '--glob=!**/.git', '--glob=!**/.git/**', + '--glob=!**/.svn', '--glob=!**/.svn/**', + '--glob=!**/.hg', '--glob=!**/.hg/**', + '--glob=!**/.bzr', '--glob=!**/.bzr/**', + '--glob=!**/.jj', '--glob=!**/.jj/**', + '--glob=!**/.sl', '--glob=!**/.sl/**', + '--', 'docs dir']) }) - it('grep: fixed rg --json template with the pattern in --regexp= form', () => { - expect(buildGrepCommand({ pattern: 'foo.*bar' })).toBe("rg --json --regexp='foo.*bar'") + it('grep: fixed rg --json argv with the pattern in --regexp= form', () => { + expect(buildGrepCommand({ pattern: 'foo.*bar' })).toEqual(['--json', '--regexp=foo.*bar']) }) - it('grep: include and path are quoted, include in --glob= form, path behind --', () => { - const command = buildGrepCommand({ pattern: 'x', path: '-leading-dash', include: '*.{ts,tsx}' }) - expect(command).toBe("rg --json --regexp='x' --glob='*.{ts,tsx}' -- '-leading-dash'") + it('grep: include in --glob= form, path behind --, both plain elements', () => { + expect(buildGrepCommand({ pattern: 'x', path: '-leading-dash', include: '*.{ts,tsx}' })) + .toEqual(['--json', '--regexp=x', '--glob=*.{ts,tsx}', '--', '-leading-dash']) }) it.each([ - ['a command-substitution pattern', '$(rm -rf /)', "'$(rm -rf /)'"], - ['a backtick pattern', '`touch pwned`', "'`touch pwned`'"], - ['a pattern with double quotes and spaces', 'say "hi there"', '\'say "hi there"\''], - ['a pattern with single quotes', "it's", '\'it\'\\\'\'s\''], - ['a pattern with newlines', 'a\nb', "'a\nb'"], - ['a leading-dash pattern', '--flag', "'--flag'"], - ['glob metacharacters', '*?[a-z]{x,y}', "'*?[a-z]{x,y}'"], - ])('quotes %s into one inert shell word', (_label, raw, quoted) => { - expect(buildGrepCommand({ pattern: raw })).toBe(`rg --json --regexp=${quoted}`) + ['a command-substitution pattern', '$(rm -rf /)'], + ['a backtick pattern', '`touch pwned`'], + ['a pattern with double quotes and spaces', 'say "hi there"'], + ['a pattern with single quotes', "it's"], + ['a pattern with newlines', 'a\nb'], + ['a leading-dash pattern', '--flag'], + ['glob metacharacters', '*?[a-z]{x,y}'], + ])('keeps %s as ONE inert argv element (no shell layer to escape)', (_label, raw) => { + // The argv vector is handed to rg verbatim: hostile text cannot break out + // of its argument because there is no shell between the vector and rg. + expect(buildGrepCommand({ pattern: raw })).toEqual(['--json', `--regexp=${raw}`]) + expect(buildGlobCommand({ pattern: raw })[1]).toBe(`--glob=${raw}`) }) }) describe('workdir derivation and signal forwarding', () => { - it('forwards the session cwd as the request workdir', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('a.ts\n') + it('forwards the session cwd as the spawn cwd', async () => { + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('a.ts\n') await call(ctx, 'glob', { pattern: '*' }, { agent: agent('/sessions/s1') }) - expect(bash.requests[0]?.workdir).toBe('/sessions/s1') - expect(bash.specs[0]?.workdir).toBe('/sessions/s1') + expect(subprocess.spawns[0]?.cwd).toBe('/sessions/s1') }) - it('omits the request workdir without a session cwd so resolve() defaults apply', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('a.ts\n') + it('defaults the spawn cwd to process.cwd() without a session cwd', async () => { + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('a.ts\n') await call(ctx, 'glob', { pattern: '*' }, { agent: agent() }) - expect(bash.requests[0]).not.toHaveProperty('workdir') - expect(bash.specs[0]?.workdir).toBe('/work') - // A non-agent caller takes the same default path. + expect(subprocess.spawns[0]?.cwd).toBe(process.cwd()) + // A non-agent caller takes the same default. await call(ctx, 'grep', { pattern: 'x' }) - expect(bash.requests[1]).not.toHaveProperty('workdir') + expect(subprocess.spawns[1]?.cwd).toBe(process.cwd()) }) - it('forwards exec.signal into the bash spec', async () => { - const { ctx, bash } = await setup() + it('spawns the packaged ripgrep binary with the fixed argv and budgeted collect streams', async () => { + const { ctx, subprocess } = await setup({ config: { rawOutputMaxBytes: 1234 } }) + subprocess.handler = () => runResult('', { exitCode: 1 }) + await call(ctx, 'grep', { pattern: 'needle' }) + const spec = subprocess.spawns[0] + expect(spec?.argv[0]).toBe(rgPath) + expect(spec?.argv).toEqual([rgPath, '--json', '--regexp=needle']) + expect(spec?.stdio.stdin).toBe('ignore') + // stdout gets the tool's parse budget; stderr is a diagnostic excerpt. + expect((spec?.stdio.stdout as { maxBytes: number }).maxBytes).toBe(1234) + expect(spec?.graceMs).toBe(3_000) + }) + + it('forwards exec.signal into the spawn spec', async () => { + const { ctx, subprocess } = await setup() const controller = new AbortController() - bash.handler = () => runResult('') + subprocess.handler = () => runResult('') const result = await call(ctx, 'grep', { pattern: 'x' }, { signal: controller.signal }) - expect(bash.specs[0]?.signal).toBe(controller.signal) + expect(subprocess.spawns[0]?.signal).toBe(controller.signal) expect(result.isError).toBe(false) }) - it('reports the bash executor timeout as SEARCH_ABORTED with the budget', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { timedOut: true, timeoutMs: 1234, exitCode: null, signal: 'SIGTERM' }) - const result = await call(ctx, 'glob', { pattern: '*' }) + it('reports an abort fired during the run as SEARCH_ABORTED', async () => { + // The cooperative tool timeout or caller cancellation aborts exec.signal; + // the subprocess seam then kills the process tree. The tool classifies + // the first cause it owns: the abort. + const { ctx, subprocess } = await setup() + const controller = new AbortController() + subprocess.handler = () => { + controller.abort('timeout') + return runResult('', { exitCode: null, signal: 'SIGTERM' }) + } + const result = await call(ctx, 'glob', { pattern: '*' }, { signal: controller.signal }) expect(result.isError).toBe(true) expect(result.error).toMatchObject({ info: { code: 'SEARCH_ABORTED' } }) - expect(text(result)).toContain('timed out after 1234ms') + expect(text(result)).toContain('aborted before completion') + expect(subprocess.handles[0]?.terminated).toBe(true) }) - it('skips a pre-aborted registry call before run()', async () => { - const { ctx, bash } = await setup() + it('skips a pre-aborted registry call before spawn()', async () => { + const { ctx, subprocess } = await setup() const controller = new AbortController() controller.abort() - bash.handler = () => { throw new Error('aborted before spawn') } + subprocess.handler = () => { throw new Error('aborted before spawn') } const result = await call(ctx, 'grep', { pattern: 'x' }, { signal: controller.signal }) expect(result.isError).toBe(true) expect(result.error).toMatchObject({ info: { name: 'AbortError', code: TOOL_ABORTED_BEFORE_DISPATCH } }) - expect(bash.specs).toHaveLength(0) + expect(subprocess.spawns).toHaveLength(0) }) - it('translates a run() rejection after the forwarded signal aborts', async () => { - const { ctx, bash } = await setup() + it('fails a pre-aborted exec.signal before spawn with SEARCH_ABORTED', async () => { + // Direct unit check of runRipgrep's own pre-spawn guard: the registry + // intercepts most pre-aborted calls, but a signal that aborts between the + // registry check and execute reaches this branch. + const { ctx } = await setup() const controller = new AbortController() - bash.handler = () => { + controller.abort() + const exec = { signal: controller.signal, name: 'glob', callId: CallId('direct-pre-abort') } as unknown as ToolExecution + await expect(runRipgrep(ctx, exec, 'glob', ['--files'], 1_000_000)).rejects + .toMatchObject({ name: 'SearchError', code: 'SEARCH_ABORTED' }) + }) + + it('translates a spawn rejection into SEARCH_FAILED even when the signal aborts concurrently', async () => { + // The seam rejects only for infrastructure failures (unusable workdir, + // missing binary); the abort happened after dispatch, so the launch + // failure is the reportable cause with the original error chained. + const { ctx, subprocess } = await setup() + const controller = new AbortController() + subprocess.handler = () => { controller.abort('cancel search') - throw new Error('executor stopped on abort') + return { reject: new Error('spawn ENOENT') } } const result = await call(ctx, 'grep', { pattern: 'x' }, { signal: controller.signal }) expect(result.isError).toBe(true) - expect(result.error).toMatchObject({ info: { name: 'SearchError', code: 'SEARCH_ABORTED' } }) - expect(text(result)).toContain('aborted before completion') + expect(result.error).toMatchObject({ info: { name: 'SearchError', code: 'SEARCH_FAILED' } }) + expect(text(result)).toContain('could not start') }) - it('translates an aborted executor result after dispatch starts', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { aborted: true, exitCode: null }) - - const result = await call(ctx, 'glob', { pattern: '*' }) - - expect(result.isError).toBe(true) - expect(result.error).toMatchObject({ info: { name: 'SearchError', code: 'SEARCH_ABORTED' } }) - expect(text(result)).toContain('aborted before completion') - }) - - it('translates a run() rejection without an abort (unusable workdir) into SEARCH_FAILED', async () => { - const { ctx, bash } = await setup() - bash.forwardSignal = false - bash.handler = () => { throw new Error('spawn bash ENOENT') } + it('rejects when the subprocess implementation drops a requested collect stream', async () => { + const { ctx, subprocess } = await setup() + subprocess.dropReaders = true const result = await call(ctx, 'glob', { pattern: '*' }) expect(result.isError).toBe(true) expect(result.error).toMatchObject({ info: { name: 'SearchError', code: 'SEARCH_FAILED' } }) - expect(text(result)).toContain('could not start') + expect(text(result)).toContain('no collected output streams') }) }) describe('exit semantics and failure classification', () => { it('exit 1 is a successful empty search', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { exitCode: 1 }) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('', { exitCode: 1 }) const glob = await call(ctx, 'glob', { pattern: '*.nope' }) expect(glob.isError).toBe(false) expect(text(glob)).toBe('No files found') @@ -418,108 +475,100 @@ describe('exit semantics and failure classification', () => { }) it('a regex parse error classifies as SEARCH_INVALID_PATTERN', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { exitCode: 2, stderr: { text: 'rg: regex parse error:\n (\nerror: unclosed group', truncated: false } }) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('', { exitCode: 2, stderr: { text: 'rg: regex parse error:\n (\nerror: unclosed group' } }) const result = await call(ctx, 'grep', { pattern: '(' }) expect(result.error).toMatchObject({ info: { code: 'SEARCH_INVALID_PATTERN' } }) expect(text(result)).toContain('regex parse error') }) it('a glob parse error classifies as SEARCH_INVALID_PATTERN', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { exitCode: 2, stderr: { text: 'rg: error parsing glob \'[\': unclosed character class', truncated: false } }) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('', { exitCode: 2, stderr: { text: 'rg: error parsing glob \'[\': unclosed character class' } }) const result = await call(ctx, 'glob', { pattern: '[' }) expect(result.error).toMatchObject({ info: { code: 'SEARCH_INVALID_PATTERN' } }) }) - it('a missing rg binary classifies as SEARCH_FAILED naming ripgrep', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { exitCode: 127, stderr: { text: 'bash: line 1: rg: command not found', truncated: false } }) + it('a failed ripgrep launch classifies as SEARCH_FAILED naming ripgrep', async () => { + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('', { exitCode: 127, stderr: { text: 'sh: rg: command not found' } }) const result = await call(ctx, 'glob', { pattern: '*' }) expect(result.error).toMatchObject({ info: { code: 'SEARCH_FAILED' } }) expect(text(result)).toContain('requires ripgrep (rg)') // The same classification holds from either evidence alone: the 127 exit // with silent stderr, or a shell's command-not-found text on another exit. - bash.handler = () => runResult('', { exitCode: 127 }) + subprocess.handler = () => runResult('', { exitCode: 127 }) expect(text(await call(ctx, 'glob', { pattern: '*' }))).toContain('requires ripgrep (rg)') - bash.handler = () => runResult('', { exitCode: 2, stderr: { text: 'sh: rg: command not found', truncated: false } }) + subprocess.handler = () => runResult('', { exitCode: 2, stderr: { text: 'sh: rg: command not found' } }) expect(text(await call(ctx, 'grep', { pattern: 'x' }))).toContain('requires ripgrep (rg)') }) it('other nonzero exits are SEARCH_FAILED carrying the stderr excerpt', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { exitCode: 2, stderr: { text: 'rg: missing.dir: IO error: no such file or directory', truncated: false } }) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('', { exitCode: 2, stderr: { text: 'rg: missing.dir: IO error: no such file or directory' } }) const result = await call(ctx, 'grep', { pattern: 'x', path: 'missing.dir' }) expect(result.error).toMatchObject({ info: { code: 'SEARCH_FAILED' } }) expect(text(result)).toContain('IO error') }) it('a nonzero exit with EMPTY stderr still reports the exit code', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { exitCode: 3 }) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('', { exitCode: 3 }) const result = await call(ctx, 'glob', { pattern: '*' }) expect(result.error).toMatchObject({ info: { code: 'SEARCH_FAILED' } }) expect(text(result)).toContain('exit 3') }) it('truncated stderr gains a truncation note and stderr.spillPath is never read', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('', { exitCode: 2, - stderr: { text: 'tail of diagnostics', truncated: true, spillPath: '/does/not/exist-and-never-read' }, + stderr: { text: 'tail of diagnostics', lossy: true, spillPath: '/does/not/exist-and-never-read' }, }) const result = await call(ctx, 'grep', { pattern: 'x' }) expect(text(result)).toContain('tail of diagnostics [stderr truncated]') }) it('a signal kill (not timeout, not abort) is SEARCH_FAILED', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { exitCode: null, signal: 'SIGKILL' }) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('', { exitCode: null, signal: 'SIGKILL' }) const result = await call(ctx, 'grep', { pattern: 'x' }) expect(result.error).toMatchObject({ info: { code: 'SEARCH_FAILED' } }) expect(text(result)).toContain('SIGKILL') }) it('a null exit with no signal (defensive) is SEARCH_FAILED', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { exitCode: null, signal: null }) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('', { exitCode: null, signal: null }) const result = await call(ctx, 'glob', { pattern: '*' }) expect(result.error).toMatchObject({ info: { code: 'SEARCH_FAILED' } }) + expect(text(result)).toContain('killed by signal (unknown)') }) }) describe('raw output acquisition', () => { - it('passes rawOutputMaxBytes to bash as the stdout capture budget', async () => { - const { ctx, bash } = await setup({ config: { rawOutputMaxBytes: 1234 } }) - bash.handler = () => runResult('', { exitCode: 1 }) - await call(ctx, 'glob', { pattern: '*.ts' }) - await call(ctx, 'grep', { pattern: 'needle' }) - expect(bash.requests.map(request => request.stdoutMaxBytes)).toEqual([1234, 1234]) - expect(bash.specs.map(spec => spec.stdoutMaxBytes)).toEqual([1234, 1234]) - }) - it('fails with SEARCH_RAW_OUTPUT_OVERFLOW when truncated stdout has a raw spill path', async () => { - const { ctx, bash } = await setup({ config: { rawOutputMaxBytes: 16 } }) - bash.handler = () => runResult('', { stdout: { text: 'x', truncated: true, spillPath: '/does/not/get-read' } }) + const { ctx, subprocess } = await setup({ config: { rawOutputMaxBytes: 16 } }) + subprocess.handler = () => runResult('', { stdout: { text: 'x', lossy: true, spillPath: '/does/not/get-read' } }) const result = await call(ctx, 'glob', { pattern: '*' }) expect(result.error).toMatchObject({ info: { code: 'SEARCH_RAW_OUTPUT_OVERFLOW' } }) expect(text(result)).toContain('narrow pattern, path, or include') }) it('fails with SEARCH_RAW_OUTPUT_OVERFLOW when UNTRUNCATED inline stdout exceeds the cap', async () => { - // An executor retaining more inline than this package's cap (or a - // deployment lowering rawOutputMaxBytes below the bash retention) must not - // smuggle an over-cap parse through the untruncated path. - const { ctx, bash } = await setup({ config: { rawOutputMaxBytes: 16 } }) - bash.handler = () => runResult(`${'x'.repeat(64)}\n`) + // A subprocess implementation retaining more inline than this package's + // cap (or a deployment lowering rawOutputMaxBytes below the retention + // budget) must not smuggle an over-cap parse through the untruncated path. + const { ctx, subprocess } = await setup({ config: { rawOutputMaxBytes: 16 } }) + subprocess.handler = () => runResult(`${'x'.repeat(64)}\n`) const result = await call(ctx, 'grep', { pattern: 'x' }) expect(result.error).toMatchObject({ info: { name: 'SearchError', code: 'SEARCH_RAW_OUTPUT_OVERFLOW' } }) expect(text(result)).toContain('narrow pattern, path, or include') }) it('fails with SEARCH_RAW_OUTPUT_OVERFLOW when truncated stdout has no spill path', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { stdout: { text: 'partial', truncated: true } }) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('', { stdout: { text: 'partial', lossy: true } }) const result = await call(ctx, 'grep', { pattern: 'x' }) expect(result.error).toMatchObject({ info: { code: 'SEARCH_RAW_OUTPUT_OVERFLOW' } }) }) @@ -614,8 +663,8 @@ describe('cross-directory sampling', () => { describe('glob results', () => { it('lists workdir-relative paths (absolute output under the workdir is relativized)', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('/sessions/s1/src/a.ts\n/elsewhere/b.ts\nrel/c.ts\n') + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('/sessions/s1/src/a.ts\n/elsewhere/b.ts\nrel/c.ts\n') const result = await call(ctx, 'glob', { pattern: '*' }, { agent: agent('/sessions/s1') }) if (result.isError) throw new Error('expected glob success') expect(result.value).toEqual({ root: '.', paths: [join('src', 'a.ts'), '/elsewhere/b.ts', 'rel/c.ts'] }) @@ -628,23 +677,30 @@ describe('glob results', () => { expect(text(await call(ctx, 'glob', { pattern: '*', path: ' ' }))).toContain('path must be a non-empty string') }) - it('threads a valid path through to the command as the quoted search root', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('sub/a.ts\n') + it('threads a valid path through to the spawn as the plain search root element', async () => { + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('sub/a.ts\n') const result = await call(ctx, 'glob', { pattern: '*.ts', path: 'sub' }) expect(result.isError).toBe(false) - expect(bash.specs[0]?.command).toContain("-- 'sub'") + expect(subprocess.spawns[0]?.argv).toEqual([rgPath, '--files', '--glob=*.ts', '--sort=modified', '--no-ignore', '--hidden', + '--glob=!**/.git', '--glob=!**/.git/**', + '--glob=!**/.svn', '--glob=!**/.svn/**', + '--glob=!**/.hg', '--glob=!**/.hg/**', + '--glob=!**/.bzr', '--glob=!**/.bzr/**', + '--glob=!**/.jj', '--glob=!**/.jj/**', + '--glob=!**/.sl', '--glob=!**/.sl/**', + '--', 'sub']) }) it('caps at globMaxResults and saves the FULL sorted list through spillStore', async () => { - const { ctx, bash, spill } = await setup({ config: { globMaxResults: 2 }, spill: true }) + const { ctx, subprocess, spill } = await setup({ config: { globMaxResults: 2 }, spill: true }) ctx.on('tools/post-execute', async () => ({ kind: 'accept', additionalContexts: [createUserMessage({ content: [{ type: 'text', text: 'glob context' }], source: { kind: 'plugin', plugin: 'test' }, })], })) - bash.handler = () => runResult('a.ts\nb.ts\nc.ts\nd.ts\n') + subprocess.handler = () => runResult('a.ts\nb.ts\nc.ts\nd.ts\n') const result = await call(ctx, 'glob', { pattern: '*.ts' }, { agent: agent('/w') }) expect(result.isError).toBe(false) if (result.isError) throw new Error('expected glob success') @@ -665,8 +721,8 @@ describe('glob results', () => { // The shipped failure: `*` matches the whole tree, mtime order puts one // freshly-unpacked subtree first, and a head-of-3 reads like the entire // workspace. The sample reaches every top-level entry instead. - const { ctx, bash } = await setup({ config: { globMaxResults: 3 } }) - bash.handler = () => runResult(['vendor/a.ts', 'vendor/b.ts', 'vendor/c.ts', 'src/d.ts', 'guide/e.md', 'top.txt'].join('\n')) + const { ctx, subprocess } = await setup({ config: { globMaxResults: 3 } }) + subprocess.handler = () => runResult(['vendor/a.ts', 'vendor/b.ts', 'vendor/c.ts', 'src/d.ts', 'guide/e.md', 'top.txt'].join('\n')) const result = await call(ctx, 'glob', { pattern: '*' }, { agent: agent('/w') }) expect(text(result)).toBe('vendor/a.ts\nsrc/d.ts\nguide/e.md\n\n' + '(Showing 3 of 6 paths, sampled across 3 of the 4 top-level entries this pattern matched ' @@ -675,18 +731,18 @@ describe('glob results', () => { }) it('keeps the modification-time head when over-cap sampling is disabled', async () => { - const { ctx, bash } = await setup({ + const { ctx, subprocess } = await setup({ config: { globMaxResults: 3, sampleOverCapGlobResults: false }, }) - bash.handler = () => runResult(['vendor/a.ts', 'vendor/b.ts', 'vendor/c.ts', 'src/d.ts', 'guide/e.md'].join('\n')) + subprocess.handler = () => runResult(['vendor/a.ts', 'vendor/b.ts', 'vendor/c.ts', 'src/d.ts', 'guide/e.md'].join('\n')) expect(text(await call(ctx, 'glob', { pattern: '*' }, { agent: agent('/w') }))) .toBe('vendor/a.ts\nvendor/b.ts\nvendor/c.ts\n\n' + '(Showing 3 of 5 paths. The complete result could not be saved; narrow pattern or path to see more.)') }) it('samples relative to the explicit search root instead of its workdir prefix', async () => { - const { ctx, bash } = await setup({ config: { globMaxResults: 3 } }) - bash.handler = () => runResult([ + const { ctx, subprocess } = await setup({ config: { globMaxResults: 3 } }) + subprocess.handler = () => runResult([ 'workspace/vendor/a.ts', 'workspace/vendor/b.ts', 'workspace/source/c.ts', @@ -698,8 +754,8 @@ describe('glob results', () => { }) it('samples relative to an absolute search root after workdir display conversion', async () => { - const { ctx, bash } = await setup({ config: { globMaxResults: 3 } }) - bash.handler = () => runResult([ + const { ctx, subprocess } = await setup({ config: { globMaxResults: 3 } }) + subprocess.handler = () => runResult([ '/w/workspace/vendor/a.ts', '/w/workspace/vendor/b.ts', '/w/workspace/source/c.ts', @@ -711,8 +767,8 @@ describe('glob results', () => { }) it('drops the narrowing hint when the sample reaches every top-level entry', async () => { - const { ctx, bash } = await setup({ config: { globMaxResults: 3 } }) - bash.handler = () => runResult(['vendor/a.ts', 'vendor/b.ts', 'vendor/c.ts', 'src/d.ts'].join('\n')) + const { ctx, subprocess } = await setup({ config: { globMaxResults: 3 } }) + subprocess.handler = () => runResult(['vendor/a.ts', 'vendor/b.ts', 'vendor/c.ts', 'src/d.ts'].join('\n')) expect(text(await call(ctx, 'glob', { pattern: '*' }, { agent: agent('/w') }))) .toBe('vendor/a.ts\nvendor/b.ts\nsrc/d.ts\n\n' + '(Showing 3 of 4 paths, sampled across 2 of the 2 top-level entries this pattern matched ' @@ -721,34 +777,34 @@ describe('glob results', () => { }) it('keeps modification-time order untouched when the whole result fits', async () => { - const { ctx, bash } = await setup({ config: { globMaxResults: 4 } }) - bash.handler = () => runResult('vendor/a.ts\nvendor/b.ts\nsrc/c.ts\n') + const { ctx, subprocess } = await setup({ config: { globMaxResults: 4 } }) + subprocess.handler = () => runResult('vendor/a.ts\nvendor/b.ts\nsrc/c.ts\n') expect(text(await call(ctx, 'glob', { pattern: '*' }, { agent: agent('/w') }))) .toBe('vendor/a.ts\nvendor/b.ts\nsrc/c.ts') }) it('keeps the plain footer for a flat result, where the sample is the modification-time head', async () => { - const { ctx, bash } = await setup({ config: { globMaxResults: 2 } }) - bash.handler = () => runResult('a.ts\nb.ts\nc.ts\n') + const { ctx, subprocess } = await setup({ config: { globMaxResults: 2 } }) + subprocess.handler = () => runResult('a.ts\nb.ts\nc.ts\n') expect(text(await call(ctx, 'glob', { pattern: '*' }, { agent: agent('/w') }))) .toBe('a.ts\nb.ts\n\n(Showing 2 of 3 paths. The complete result could not be saved; narrow pattern or path to see more.)') }) it('does not create a spill file when the result fits inline', async () => { - const { ctx, bash, spill } = await setup({ spill: true }) - bash.handler = () => runResult('a.ts\nb.ts\n') + const { ctx, subprocess, spill } = await setup({ spill: true }) + subprocess.handler = () => runResult('a.ts\nb.ts\n') const result = await call(ctx, 'glob', { pattern: '*' }, { agent: agent('/w') }) expect(text(result)).toBe('a.ts\nb.ts') expect(spill?.saves).toHaveLength(0) }) it('preserves a downstream canonical value replacement instead of spilling the old value', async () => { - const { ctx, bash, spill } = await setup({ config: { globMaxResults: 1 }, spill: true }) + const { ctx, subprocess, spill } = await setup({ config: { globMaxResults: 1 }, spill: true }) ctx.on('tools/post-execute', async () => ({ kind: 'accept' as const, value: { root: '.', paths: ['replacement-a.ts', 'replacement-b.ts'] }, })) - bash.handler = () => runResult('old-a.ts\nold-b.ts\n') + subprocess.handler = () => runResult('old-a.ts\nold-b.ts\n') const result = await call(ctx, 'glob', { pattern: '*.ts' }, { agent: agent('/w') }) @@ -760,8 +816,8 @@ describe('glob results', () => { }) it('keeps the full nested Code value without creating a surface spill', async () => { - const { ctx, bash, spill } = await setup({ config: { globMaxResults: 2 }, spill: true }) - bash.handler = () => runResult('a.ts\nb.ts\nc.ts\nd.ts\n') + const { ctx, subprocess, spill } = await setup({ config: { globMaxResults: 2 }, spill: true }) + subprocess.handler = () => runResult('a.ts\nb.ts\nc.ts\nd.ts\n') const result = await call(ctx, 'glob', { pattern: '*.ts' }, { agent: agent('/w'), parent: Symbol('run_code') as ToolExecutionToken, @@ -777,9 +833,9 @@ describe('glob results', () => { ['saveText fails', { fail: true, spill: true, ownerless: false }], ['no session owner', { fail: false, spill: true, ownerless: true }], ])('keeps the inline page and reports the unsaved remainder when %s', async (_label, mode) => { - const { ctx, bash, spill } = await setup({ config: { globMaxResults: 1 }, spill: mode.spill }) + const { ctx, subprocess, spill } = await setup({ config: { globMaxResults: 1 }, spill: mode.spill }) if (mode.fail && spill) spill.failWith = new Error('disk full') - bash.handler = () => runResult('a.ts\nb.ts\n') + subprocess.handler = () => runResult('a.ts\nb.ts\n') const result = await call(ctx, 'glob', { pattern: '*' }, mode.ownerless ? {} : { agent: agent('/w') }) expect(result.isError).toBe(false) // spill unavailability never fails the search expect(text(result)).toBe('a.ts\n\n(Showing 1 of 2 paths. The complete result could not be saved; narrow pattern or path to see more.)') @@ -788,8 +844,8 @@ describe('glob results', () => { describe('grep results', () => { it('groups matches by file with line numbers', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult([ + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult([ JSON.stringify({ type: 'begin', data: { path: { text: 'a.ts' } } }), matchLine('a.ts', 3, 'const x = 1\n'), matchLine('a.ts', 9, 'const y = 2\n'), @@ -812,23 +868,23 @@ describe('grep results', () => { }) it('reports a single match in the singular', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult(`${matchLine('a.ts', 1, 'hit')}\n`) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult(`${matchLine('a.ts', 1, 'hit')}\n`) expect(text(await call(ctx, 'grep', { pattern: 'hit' }))).toBe('Found 1 match\n\na.ts\nLine 1: hit') }) it('relativizes absolute match paths against the resolved workdir', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult(`${matchLine('/sessions/s1/deep/a.ts', 2, 'hit')}\n`) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult(`${matchLine('/sessions/s1/deep/a.ts', 2, 'hit')}\n`) const result = await call(ctx, 'grep', { pattern: 'hit', path: '/sessions/s1' }, { agent: agent('/sessions/s1') }) expect(text(result)).toContain(`${join('deep', 'a.ts')}\nLine 2: hit`) }) it('previews a long matched line at grepMaxLineBytes preserving UTF-8', async () => { - const { ctx, bash } = await setup({ config: { grepMaxLineBytes: 7 } }) + const { ctx, subprocess } = await setup({ config: { grepMaxLineBytes: 7 } }) // 'héllo wörld' cut at 7 bytes lands mid-'é'? h(1)é(2)l(1)l(1)o(1)=6, space=7 → clean cut at 7. // Use a multibyte straddle instead: 'aé' repeated — cut at 7 bytes: a(1)é(2)a(1)é(2)=6 +a(1)=7 → next é straddles: trimmed. - bash.handler = () => runResult(`${matchLine('a.txt', 1, 'aéaéaéaé')}\n`) + subprocess.handler = () => runResult(`${matchLine('a.txt', 1, 'aéaéaéaé')}\n`) const result = await call(ctx, 'grep', { pattern: 'a' }) if (result.isError) throw new Error('expected grep success') expect(result.value).toEqual({ matches: [{ path: 'a.txt', lineNumber: 1, line: 'aéaéaéaé' }] }) @@ -836,9 +892,9 @@ describe('grep results', () => { }) it('renders a non-UTF-8 line (rg bytes form) as a placeholder instead of failing', async () => { - const { ctx, bash } = await setup() + const { ctx, subprocess } = await setup() const record = JSON.stringify({ type: 'match', data: { path: { text: 'bin.dat' }, lines: { bytes: 'AAECww==' }, line_number: 4 } }) - bash.handler = () => runResult(`${record}\n`) + subprocess.handler = () => runResult(`${record}\n`) expect(text(await call(ctx, 'grep', { pattern: 'x' }))).toContain('Line 4: (line is not valid UTF-8)') }) @@ -848,14 +904,14 @@ describe('grep results', () => { }) it('caps at grepMaxMatches and spills the full formatted match list', async () => { - const { ctx, bash, spill } = await setup({ config: { grepMaxMatches: 2 }, spill: true }) + const { ctx, subprocess, spill } = await setup({ config: { grepMaxMatches: 2 }, spill: true }) ctx.on('tools/post-execute', async () => ({ kind: 'accept', additionalContexts: [createUserMessage({ content: [{ type: 'text', text: 'grep context' }], source: { kind: 'plugin', plugin: 'test' }, })], })) - bash.handler = () => runResult([ + subprocess.handler = () => runResult([ matchLine('a.ts', 1, 'one'), matchLine('a.ts', 2, 'two'), matchLine('b.ts', 3, 'three'), @@ -880,7 +936,7 @@ describe('grep results', () => { }) it('preserves a downstream canonical value replacement instead of spilling the old matches', async () => { - const { ctx, bash, spill } = await setup({ config: { grepMaxMatches: 1 }, spill: true }) + const { ctx, subprocess, spill } = await setup({ config: { grepMaxMatches: 1 }, spill: true }) ctx.on('tools/post-execute', async () => ({ kind: 'accept' as const, value: { @@ -890,7 +946,7 @@ describe('grep results', () => { ], }, })) - bash.handler = () => runResult(`${matchLine('old.ts', 1, 'old')}\n`) + subprocess.handler = () => runResult(`${matchLine('old.ts', 1, 'old')}\n`) const result = await call(ctx, 'grep', { pattern: 'old' }, { agent: agent('/w') }) @@ -907,8 +963,8 @@ describe('grep results', () => { }) it('keeps every nested Code match in the value without creating a surface spill', async () => { - const { ctx, bash, spill } = await setup({ config: { grepMaxMatches: 1 }, spill: true }) - bash.handler = () => runResult(`${matchLine('a.ts', 1, 'one')}\n${matchLine('b.ts', 2, 'two')}\n`) + const { ctx, subprocess, spill } = await setup({ config: { grepMaxMatches: 1 }, spill: true }) + subprocess.handler = () => runResult(`${matchLine('a.ts', 1, 'one')}\n${matchLine('b.ts', 2, 'two')}\n`) const result = await call(ctx, 'grep', { pattern: 'o' }, { agent: agent('/w'), parent: Symbol('run_code') as ToolExecutionToken, @@ -925,8 +981,8 @@ describe('grep results', () => { }) it('reports the unsaved remainder when capped with no spill backend', async () => { - const { ctx, bash } = await setup({ config: { grepMaxMatches: 1 } }) - bash.handler = () => runResult(`${matchLine('a.ts', 1, 'one')}\n${matchLine('a.ts', 2, 'two')}\n`) + const { ctx, subprocess } = await setup({ config: { grepMaxMatches: 1 } }) + subprocess.handler = () => runResult(`${matchLine('a.ts', 1, 'one')}\n${matchLine('a.ts', 2, 'two')}\n`) const result = await call(ctx, 'grep', { pattern: 'o' }, { agent: agent('/w') }) expect(result.isError).toBe(false) expect(text(result)).toBe('Found 1 of 2 matches\n\na.ts\nLine 1: one\n\n(The complete result could not be saved; narrow pattern, path, or include to see more.)') @@ -942,8 +998,8 @@ describe('grep results', () => { }) it('accepts a whitespace-only pattern (a legitimate regex) and brace alternation in include', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('', { exitCode: 1 }) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('', { exitCode: 1 }) const result = await call(ctx, 'grep', { pattern: ' ', include: '*.{ts,tsx}' }) expect(result.isError).toBe(false) }) @@ -960,8 +1016,8 @@ describe('rg --json transport failures (SEARCH_FAILED)', () => { ['a match record with no line content', JSON.stringify({ type: 'match', data: { path: { text: 'a.ts' }, line_number: 1 } })], ['a match record with neither text nor bytes', JSON.stringify({ type: 'match', data: { path: { text: 'a.ts' }, lines: {}, line_number: 1 } })], ])('%s fails the search', async (_label, line) => { - const { ctx, bash } = await setup() - bash.handler = () => runResult(`${line}\n`) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult(`${line}\n`) const result = await call(ctx, 'grep', { pattern: 'x' }) expect(result.isError).toBe(true) expect(result.error).toMatchObject({ info: { name: 'SearchError', code: 'SEARCH_FAILED' } }) @@ -969,13 +1025,16 @@ describe('rg --json transport failures (SEARCH_FAILED)', () => { }) describe('the no-background-task invariant', () => { - it('never calls ctx.bash.start() across successful and failed searches', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult('a.ts\n') + it('settles every spawned search handle across successful and failed searches', async () => { + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('a.ts\n') await call(ctx, 'glob', { pattern: '*' }) - bash.handler = () => runResult('', { exitCode: 2, stderr: { text: 'boom', truncated: false } }) + subprocess.handler = () => runResult('', { exitCode: 2, stderr: { text: 'boom' } }) await call(ctx, 'grep', { pattern: 'x' }) - expect(bash.startCalls).toBe(0) + // One foreground spawn per call, each awaited to settlement before the + // tool returns — the searches never leave a background handle running. + expect(subprocess.spawns).toHaveLength(2) + expect(subprocess.handles.every(handle => handle.settled)).toBe(true) }) }) @@ -991,8 +1050,8 @@ describe('presentation', () => { }) it('grep projects a search card from a real execute, grouped by file with total and truncation', async () => { - const { ctx, bash } = await setup({ config: { grepMaxMatches: 2 } }) - bash.handler = () => runResult([ + const { ctx, subprocess } = await setup({ config: { grepMaxMatches: 2 } }) + subprocess.handler = () => runResult([ matchLine('a.ts', 1, 'one'), matchLine('a.ts', 2, 'two'), matchLine('b.ts', 3, 'three'), @@ -1018,8 +1077,8 @@ describe('presentation', () => { }) it('glob projects a search card from a real execute, a flat path list with total and truncation', async () => { - const { ctx, bash } = await setup({ config: { globMaxResults: 2 } }) - bash.handler = () => runResult('a.ts\nb.ts\nc.ts\n') + const { ctx, subprocess } = await setup({ config: { globMaxResults: 2 } }) + subprocess.handler = () => runResult('a.ts\nb.ts\nc.ts\n') const result = await call(ctx, 'glob', { pattern: '*.ts' }, { agent: agent('/w') }) if (result.isError) throw new Error('expected glob success') expect(result.meta).toEqual({ shape: 'paths', paths: ['a.ts', 'b.ts'], truncated: true, total: 3 }) @@ -1028,8 +1087,8 @@ describe('presentation', () => { }) it('nested Code dispatch computes no meta, so presentResult falls back to the generic card', async () => { - const { ctx, bash } = await setup() - bash.handler = () => runResult(`${matchLine('a.ts', 1, 'one')}\n`) + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult(`${matchLine('a.ts', 1, 'one')}\n`) const result = await call(ctx, 'grep', { pattern: 'o' }, { agent: agent('/w'), parent: Symbol('run_code') as ToolExecutionToken, diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 34eb7b8f87..8b38715d30 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -2931,6 +2931,9 @@ importers: packages/fs/tool-fs-search: dependencies: + '@vscode/ripgrep': + specifier: ^1.18.0 + version: 1.18.0 schemastery: specifier: ^3.18.0 version: 3.18.0 @@ -2938,12 +2941,6 @@ importers: '@deepseek-ai/dsh-agent': specifier: workspace:^ version: link:../../core/agent - '@deepseek-ai/dsh-bash': - specifier: workspace:^ - version: link:../../bash/bash - '@deepseek-ai/dsh-bash-local': - specifier: workspace:^ - version: link:../../bash/bash-local '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../support/invariants @@ -2959,6 +2956,9 @@ importers: '@deepseek-ai/dsh-spill': specifier: workspace:^ version: link:../../spill/spill + '@deepseek-ai/dsh-subprocess': + specifier: workspace:^ + version: link:../../subprocess/subprocess '@deepseek-ai/dsh-subprocess-local': specifier: workspace:^ version: link:../../subprocess/subprocess-local @@ -9010,6 +9010,69 @@ packages: '@vitest/utils@4.1.8': resolution: {integrity: sha512-uOJamYALNhfJ6iolExyQM40yIQwDqYnkKtQ5VCiSe17E33H0aQ/u+1GlRuz4LZBk6Mm3sg90G9hEbmEt37C1Zg==} + '@vscode/ripgrep-darwin-arm64@1.18.0': + resolution: {integrity: sha512-r3ktHSvbFycQNF6sl7sNDPocpsI7J+mEzh1IaZFkY0spm3k2Z9t8hPAeOK7+p0l6p6/swkQC14XWX01low+94Q==} + cpu: [arm64] + os: [darwin] + + '@vscode/ripgrep-darwin-x64@1.18.0': + resolution: {integrity: sha512-25b4gWbL138dGuQU244ebCKKc0q05ULBMoFSz9oAEUHNeqK/lOJViDS7DRvbDazzAzSEdan391Znks/R5mkaTQ==} + cpu: [x64] + os: [darwin] + + '@vscode/ripgrep-linux-arm64@1.18.0': + resolution: {integrity: sha512-lQ/5zTG++U0E3IhVgS4EPTTn/U4okncaRMM5GOFfOYZywS4nuD31GhkHbNYlDk5CuDC68+hYJ0/eQeyCKJDA+g==} + cpu: [arm64] + os: [linux] + + '@vscode/ripgrep-linux-arm@1.18.0': + resolution: {integrity: sha512-GDAvufNDHu8zqLEmXstalQF0Wh6wQvdsBi/Vg3Yi3CK4a8XoFXqqXVEHEZ9xQz3t0NfoSEc9JbvK9DDS6FxyxQ==} + cpu: [arm] + os: [linux] + + '@vscode/ripgrep-linux-ia32@1.18.0': + resolution: {integrity: sha512-YWLkSUtFd4Jh5EepIhA9RJSfv3uMAVMo+2rBIGHPBnvgLrZciIs2cDKei1/p6Wc/aCzUoHyMAg2R6tw4ZCBKGg==} + cpu: [ia32] + os: [linux] + + '@vscode/ripgrep-linux-ppc64@1.18.0': + resolution: {integrity: sha512-quXVY8fwQ8O/lvU1yrSqSl3jlUzysRSb+AfUfCL/tRtphxsKlFvPAejryZ6vg4Bgvn8XL74xb4qMCDmWgYrT5w==} + cpu: [ppc64] + os: [linux] + + '@vscode/ripgrep-linux-riscv64@1.18.0': + resolution: {integrity: sha512-f5kBQBrWfQt8Q7OhSORuNDei5dkYagBj3y4jImSUXGMy8B/Ke7SltSRcUtjPv166FAFfHCAmWuZp3+cWnX2/Vw==} + cpu: [riscv64] + os: [linux] + + '@vscode/ripgrep-linux-s390x@1.18.0': + resolution: {integrity: sha512-rTOcJFGGcl2c07RUOWUo4U1ndnemKhY6A9hnMB18uk7jSgJc0d/QLBGWMWpumdtoJtpizn/wIv5mXIisJukusQ==} + cpu: [s390x] + os: [linux] + + '@vscode/ripgrep-linux-x64@1.18.0': + resolution: {integrity: sha512-mQ3bVrUpnD2vs7QT0vX90Lt0cnUq467uFtEktIdsJJmW296RoSULRGqWgzG1AKxyBpNDD6l4ZO4qKf6SgyC23Q==} + cpu: [x64] + os: [linux] + + '@vscode/ripgrep-win32-arm64@1.18.0': + resolution: {integrity: sha512-vfTIjq1OHnzUjxZcHVQAMbnggp8dpGf+0QKFOZHwWPqFwXxQC8eCWM+5NUdoJ6yrElCeMzoUTXoK/LdZaniB+Q==} + cpu: [arm64] + os: [win32] + + '@vscode/ripgrep-win32-ia32@1.18.0': + resolution: {integrity: sha512-//rfAE+BOw5AC2EMmepmiE36jUuevtQYNQqqlw1s3m9FlRxjxEut97RkRPHAu9BG4mSojatZx+kXZXNdyI9caQ==} + cpu: [ia32] + os: [win32] + + '@vscode/ripgrep-win32-x64@1.18.0': + resolution: {integrity: sha512-KNPvtElldqILHdnAetujPaowkNbpqJy3ssIGGN6F6Kve9Qi+nNLI2DN01O83JjCEVQbCzl8Ov3QZ9Eov3BR8Dg==} + cpu: [x64] + os: [win32] + + '@vscode/ripgrep@1.18.0': + resolution: {integrity: sha512-ns5lWe44tSfbTMbVUsyB+I1819PVSw4AdpgK0RNkzfWfwy6+3IUNSxwSrfTno1/oWaS/hERNz+XLWVyga2aJBQ==} + '@vue/compiler-core@3.5.39': resolution: {integrity: sha512-16KBTEXAJCpDr0mwlw+AZyhu8iyC7R3S2vBwsI7QnWJU6X3WKc9VKeNEZpiMdZ569qWhz9574L3vV55qRL0Vtw==} @@ -14110,6 +14173,57 @@ snapshots: convert-source-map: 2.0.0 tinyrainbow: 3.1.0 + '@vscode/ripgrep-darwin-arm64@1.18.0': + optional: true + + '@vscode/ripgrep-darwin-x64@1.18.0': + optional: true + + '@vscode/ripgrep-linux-arm64@1.18.0': + optional: true + + '@vscode/ripgrep-linux-arm@1.18.0': + optional: true + + '@vscode/ripgrep-linux-ia32@1.18.0': + optional: true + + '@vscode/ripgrep-linux-ppc64@1.18.0': + optional: true + + '@vscode/ripgrep-linux-riscv64@1.18.0': + optional: true + + '@vscode/ripgrep-linux-s390x@1.18.0': + optional: true + + '@vscode/ripgrep-linux-x64@1.18.0': + optional: true + + '@vscode/ripgrep-win32-arm64@1.18.0': + optional: true + + '@vscode/ripgrep-win32-ia32@1.18.0': + optional: true + + '@vscode/ripgrep-win32-x64@1.18.0': + optional: true + + '@vscode/ripgrep@1.18.0': + optionalDependencies: + '@vscode/ripgrep-darwin-arm64': 1.18.0 + '@vscode/ripgrep-darwin-x64': 1.18.0 + '@vscode/ripgrep-linux-arm': 1.18.0 + '@vscode/ripgrep-linux-arm64': 1.18.0 + '@vscode/ripgrep-linux-ia32': 1.18.0 + '@vscode/ripgrep-linux-ppc64': 1.18.0 + '@vscode/ripgrep-linux-riscv64': 1.18.0 + '@vscode/ripgrep-linux-s390x': 1.18.0 + '@vscode/ripgrep-linux-x64': 1.18.0 + '@vscode/ripgrep-win32-arm64': 1.18.0 + '@vscode/ripgrep-win32-ia32': 1.18.0 + '@vscode/ripgrep-win32-x64': 1.18.0 + '@vue/compiler-core@3.5.39': dependencies: '@babel/parser': 7.29.7 diff --git a/scripts/gen-third-party-notices.ts b/scripts/gen-third-party-notices.ts index 5306f9a003..5cb912c9ff 100644 --- a/scripts/gen-third-party-notices.ts +++ b/scripts/gen-third-party-notices.ts @@ -157,6 +157,30 @@ function loadWorkspaceManifests(): { manifests: Map; names: Se return { manifests, names } } +type VirtualManifest = Manifest & { license?: string; repository?: string | { url?: string }; homepage?: string } + +/** + * Resolve one package's manifest inside a pnpm virtual store. The prefix scan + * matches ordinary `@scope+name@version` directory names; pnpm 11 truncates + * long names (a peer-suffixed name past the length limit becomes + * `_`), so a content scan falls back over the whole store when + * the prefix misses. + */ +function virtualManifest(virtual: string, name: string): VirtualManifest | undefined { + const prefix = `${name.replace('/', '+')}@` + const entry = readdirSync(virtual).find(dir => dir.startsWith(prefix)) + if (entry !== undefined) { + return JSON.parse(readFileSync(resolve(virtual, entry, 'node_modules', name, 'package.json'), 'utf8')) as VirtualManifest + } + for (const dir of readdirSync(virtual)) { + const candidate = resolve(virtual, dir, 'node_modules', name, 'package.json') + if (existsSync(candidate)) { + return JSON.parse(readFileSync(candidate, 'utf8')) as VirtualManifest + } + } + return undefined +} + /** License and repository URL for an installed external package, from the pnpm store. */ function installedMetadata(name: string): { license: string; repo: string } { const override = OVERRIDES[name] @@ -171,11 +195,8 @@ function installedMetadata(name: string): { license: string; repo: string } { } const virtual = resolve(root, store, '.pnpm') if (!existsSync(virtual)) continue - const prefix = `${name.replace('/', '+')}@` - const entry = readdirSync(virtual).find(dir => dir.startsWith(prefix)) - if (entry === undefined) continue - manifest = JSON.parse(readFileSync(resolve(virtual, entry, 'node_modules', name, 'package.json'), 'utf8')) as typeof manifest - break + manifest = virtualManifest(virtual, name) + if (manifest !== undefined) break } const license = override?.license ?? manifest?.license const rawRepo = typeof manifest?.repository === 'string' ? manifest.repository : manifest?.repository?.url ?? manifest?.homepage diff --git a/scripts/gen-tool-catalog.ts b/scripts/gen-tool-catalog.ts index a77bdb6d95..3db63b4271 100644 --- a/scripts/gen-tool-catalog.ts +++ b/scripts/gen-tool-catalog.ts @@ -16,8 +16,6 @@ import SessionQuerySqlite from '@deepseek-ai/dsh-session-query-sqlite' import GoalService from '@deepseek-ai/dsh-goal' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry, { type Config as ToolsConfig } from '@deepseek-ai/dsh-tools' -import { BashExecutor } from '@deepseek-ai/dsh-bash' -import type { BashExecRequest, BashExecSpec, BashProcess, BashRunResult } from '@deepseek-ai/dsh-bash' import LocalBashExecutor from '@deepseek-ai/dsh-bash-local' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import LocalFileSystem from '@deepseek-ai/dsh-fs-local' @@ -55,44 +53,6 @@ import * as ToolWorkflow from '@deepseek-ai/dsh-tool-workflow' const root = resolve(import.meta.dirname, '..') const OUT = 'docs/tool-catalog.md' -const CATALOG_RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1' - -/** - * Minimal bash service for harvesting `dsh-tool-fs-search` schemas. The search - * plugin now probes `rg` at registration time, but the generated catalog must - * remain independent of the host PATH and never execute a real search. - */ -class CatalogSearchBashExecutor extends BashExecutor { - override resolve(request: BashExecRequest): BashExecSpec { - return { - command: request.command, - workdir: request.workdir ?? root, - timeoutMs: request.timeoutMs ?? 60_000, - stdoutMaxBytes: request.stdoutMaxBytes ?? 64_000, - signal: request.signal, - sandboxPolicy: request.sandboxPolicy, - } - } - - override run(spec: BashExecSpec): Promise { - if (spec.command !== CATALOG_RG_PROBE_COMMAND) { - throw new Error(`gen-tool-catalog: unexpected search bash command during schema harvest: ${spec.command}`) - } - return Promise.resolve({ - exitCode: 0, - signal: null, - timedOut: false, - aborted: false, - timeoutMs: spec.timeoutMs, - stdout: { text: '', truncated: false }, - stderr: { text: '', truncated: false }, - }) - } - - override start(): BashProcess { - throw new Error('gen-tool-catalog: search schema harvest must not start background processes') - } -} /** * Register the descriptor needed to mount schema-producing consumers. Declares @@ -264,19 +224,19 @@ const TOOL_PACKAGES: ToolPackage[] = [ pkg: '@deepseek-ai/dsh-tool-fs-search', dir: 'tool-fs-search', source: 'packages/fs/tool-fs-search/src/index.ts', - requires: ['ctx.tools', 'ctx.bash', 'ctx.systemPrompt'], + requires: ['ctx.tools', 'ctx.subprocess', 'ctx.systemPrompt'], writes: ['tool/call', 'tool/result'], async mount(ctx) { - // The tools inject `bash` (search executes fixed `rg` commands through - // the executor seam, not ctx.fs). Use a catalog-only executor so the - // registration-time `rg` probe stays deterministic and the generator - // never depends on the host PATH. `ctx.spillStore` is optional (read via - // ctx.get) and does not affect the schemas, so no spill backend is mounted. - await ctx.plugin(CatalogSearchBashExecutor) + // The tools inject `subprocess` (search spawns the packaged ripgrep + // binary through the seam, not ctx.fs); registration itself never + // spawns, so the real local service is inert here. `ctx.spillStore` is + // optional (read via ctx.get) and does not affect the schemas, so no + // spill backend is mounted. + await ctx.plugin(LocalSubprocessService) await ctx.plugin(ToolFsSearch, { sampleOverCapGlobResults: true }) }, note: - 'glob and grep are conditional bash-backed discovery tools: they register only when ctx.bash can find `rg`, then run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). The catalog uses `sampleOverCapGlobResults: true`; deployments must choose that behavior explicitly. Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments.', + 'glob and grep are unconditional discovery tools that spawn the packaged ripgrep binary (`@vscode/ripgrep`) through ctx.subprocess as ordinary foreground calls (never background tasks) — no host `rg` install and no shell layer. The catalog uses `sampleOverCapGlobResults: true`; deployments must choose that behavior explicitly. Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments.', }, { pkg: '@deepseek-ai/dsh-tool-pty',