Files
deepseek-harness/packages/web
Tianyi Cui 45a5175e44 feat(tool-web): replace the regex HTML-to-markdown converter with turndown
Implements the turndown Agent Note from the NIH dependency audit (full
variant, not the minimal entities-only fallback): dsh-tool-web's fetch
rendering now converts HTML through turndown + @joplin/turndown-plugin-gfm
(atx headings, fenced code, dash bullets, GFM tables/strikethrough) over
the real domino DOM, with script/style/noscript removed wholesale. The
hand-rolled ~86-line regex converter html.ts and its entity tables are
deleted; renderBody wraps the conversion in try/catch falling back to
the raw HTML body, because turndown's recursive DOM walk overflows with
a RangeError on pathological nesting (measured: 4k levels on the main
thread, 8k in a worker) where the regex version could never throw.

Closure weight, measured: tool-web IS in the single-exe runtime closure,
and the exe asset globs would pack ~7.9 MB of the three new packages —
but ~6 MB of that is domino's test corpus, with runtime lib/ at ~550 KB
against a ~174 MB artifact (<0.5% either way), so the swap wins.

Per testing policy the previously-missing keyless web_fetch snapshot
ships in the same change: the acp-agent `web-fetch` scenario boots a new
web.cordis.yml overlay (web seam + real dsh-web-fetch-local provider +
tool-web fetch-only + a loopback HTTP fixture server on a fixed port
serving deterministic HTML with entities, a GFM table, and nesting), so
recording and keyless replay both drive the real HTTP fetch and real
conversion end to end; the scenario pins the new `web` header class.

The Agent Note moves proposed -> implemented and is rewritten per the
lifecycle contract (Decision/Consequences/Testing, closure verdict and
alternatives recorded); tool-web and acp-agent READMEs updated in both
languages and pairs re-recorded.
2026-07-27 00:22:13 +08:00
..
2026-07-26 05:06:39 +08:00
2026-07-26 05:06:39 +08:00
2026-07-26 05:06:39 +08:00

web/ - web capability family

English | 中文

The web access capability seam: an abstract web interface, search/fetch provider implementations, and the model-facing web tools. All product packages.

Package Role ctx key
web/ Abstract web seam (search/fetch provider registries + selection + vocabulary + WebError) ctx.web
web-search-exa/ Exa-backed WebSearchProvider (registers on ctx.web)
web-search-perplexity/ Perplexity-backed WebSearchProvider (registers on ctx.web)
web-search-deepseek/ DeepSeek-backed WebSearchProvider using native web_search through the Anthropic-compatible API (registers on ctx.web)
web-fetch-local/ Anonymous public HTTP(S) WebFetchProvider (registers on ctx.web)
tool-web/ Model-facing web_search/web_fetch tool schemas (registers on ctx.tools)

The interface lives at web/web/. Unlike bash/fs, the seam spans two capabilities (search and fetch) with potentially multiple providers each: ctx.web is one web-access middle layer with one provider-selection policy, one abort/error vocabulary, and one product-facing "how this harness reaches the web" config surface. Providers register capabilities, not tools; tool-web is the only owner of model-facing names, schemas, prompt guidance, and presentation. A search provider swap does not change how the model asks for a query, and a fetch implementation swap does not change how the model asks for a URL.

See the web capability seam Agent Note for the design rationale, including why search and fetch are deliberately one seam and why web_fetch's SSRF protection is deferred.