From 800e08e9308ea558e96bf58069e7e2133198627e Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 20 Jun 2026 21:28:02 +0800 Subject: [PATCH] docs: refine package hierarchy RFC --- docs/rfc/README.md | 3 +- .../2026-06-20-classify-packages-by-aspect.md | 46 --------------- .../2026-06-20-discover-package-inventory.md | 8 +-- .../proposed/2026-06-20-package-hierarchy.md | 57 +++++++++++++++++++ ...0-remove-redundant-snapshot-log-goldens.md | 2 +- .../2026-06-20-classify-support-packages.md | 28 --------- scripts/publint-all.ts | 2 +- 7 files changed, 64 insertions(+), 82 deletions(-) delete mode 100644 docs/rfc/proposed/2026-06-20-classify-packages-by-aspect.md create mode 100644 docs/rfc/proposed/2026-06-20-package-hierarchy.md delete mode 100644 docs/rfc/rejected/2026-06-20-classify-support-packages.md diff --git a/docs/rfc/README.md b/docs/rfc/README.md index 6e2fbb9a7e..7098ea216e 100644 --- a/docs/rfc/README.md +++ b/docs/rfc/README.md @@ -41,7 +41,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r | [Extract a generic long-running tool runtime](proposed/2026-06-20-generic-long-running-tool-runtime.md) | 2026-06-20 | | [Make the shared example base providerless](proposed/2026-06-20-providerless-example-base.md) | 2026-06-20 | | [Use `session.jsonl` as the only snapshot session-log artifact](proposed/2026-06-20-remove-redundant-snapshot-log-goldens.md) | 2026-06-20 | -| [Classify packages by aspect metadata](proposed/2026-06-20-classify-packages-by-aspect.md) | 2026-06-20 | +| [Reorganize packages into a modular hierarchy](proposed/2026-06-20-package-hierarchy.md) | 2026-06-20 | | [Discover package inventories instead of maintaining static lists](proposed/2026-06-20-discover-package-inventory.md) | 2026-06-20 | ## Implemented @@ -80,7 +80,6 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r |---|---| | [Deep-readonly public surfaces](rejected/2026-06-11-immutable-public-surfaces.md) | 2026-06-11 | | [Persist assembled assistant messages, not stream chunks](rejected/2026-06-20-assembled-assistant-messages-only.md) | 2026-06-20 | -| [Classify product, integration, and support packages](rejected/2026-06-20-classify-support-packages.md) | 2026-06-20 | | [Drop ACP session/load until resume has a product shape](rejected/2026-06-20-drop-acp-session-load.md) | 2026-06-20 | | [Drop ACP terminal `_meta` rendering](rejected/2026-06-20-drop-acp-terminal-meta.md) | 2026-06-20 | | [Drop bash full-output spill files](rejected/2026-06-20-drop-bash-output-spill-files.md) | 2026-06-20 | diff --git a/docs/rfc/proposed/2026-06-20-classify-packages-by-aspect.md b/docs/rfc/proposed/2026-06-20-classify-packages-by-aspect.md deleted file mode 100644 index e9723a5660..0000000000 --- a/docs/rfc/proposed/2026-06-20-classify-packages-by-aspect.md +++ /dev/null @@ -1,46 +0,0 @@ -# RFC: Classify packages by aspect metadata - -Status: proposed - -## Problem - -The harness package tree is flat, and every package manifest is currently `private: true`. That is fine as a pre-release safety default, but it means neither paths nor npm publish flags tell scripts what role a package plays. [publint-all](../../../scripts/publint-all.ts) needs to know which packages are release-shaped, docs need to describe which packages are core product surface, and future cleanup work needs a way to distinguish support utilities from load-bearing product modules. - -A single hierarchy such as product, integration, support, or testing is too coarse. Packages naturally carry overlapping facts: an LLM adapter is provider-facing and publish-shaped; `tool-bash` is a tool consumer and bash-related; `llm-replay` is an LLM adapter shape and test/snapshot support; ACP is an editor bridge and current product surface. Forcing each package into one bucket would either hide useful facts or recreate static exception lists under different names. - -## Proposal - -Add explicit, repo-owned package aspect metadata to each `packages/*/package.json`, using a manifest-local key such as `dsh.aspects` unless the implementing change finds an already-established repo metadata key. The metadata is a controlled vocabulary, not free-form prose. - -For example: - -```json -{ - "dsh": { - "aspects": ["core", "llm", "publishable"] - } -} -``` - -The initial vocabulary should stay small and useful to scripts. Expected facets include `core`, `implementation`, `consumer`, `llm`, `bash`, `fs`, `persistence`, `agent`, `acp`, `ui`, `example-support`, `test-support`, `replay`, and `publishable`. A package may declare multiple facets; no script should assume exactly one role. - -`publishable` is a repo policy facet, not a mirror of npm's `private` flag. While the harness is unreleased, packages can remain `private: true` and still declare `publishable` so publish-shape gates know which manifests to check. When release policy changes, the aspect continues to describe intent while the npm flag controls whether publication is allowed. - -Scripts should consume the metadata directly. `publint-all` filters on `publishable`, module graph or package inventory docs can group by domain facets, and the adding-a-package cookbook asks authors to choose aspects when creating a new package. Unknown facets should fail loudly so typoed metadata does not silently fork the taxonomy. - -## Acceptance criteria - -- Every `packages/*` manifest declares package aspects from a documented controlled vocabulary. -- The vocabulary explains each facet's meaning and when a new facet is appropriate. -- `publint-all` derives its package list from `publishable` metadata instead of a hard-coded array. -- Package inventory docs and module-graph grouping can read aspects without inferring intent from package names or folder paths. -- Adding a package requires choosing aspects, and CI fails if a package is missing aspect metadata or uses an unknown facet. -- No package path moves are required just to express classification. - -## What we give up - -Aspect metadata is less visually obvious than folders, and a package can be over-tagged if reviewers are careless. The counterweight is that metadata preserves the current package import shape while making policy facts explicit and machine-checkable. If a future package truly needs a new physical boundary, that move can still happen for architectural reasons rather than as a classification workaround. - -## Related - -This supersedes the rejected [product/integration/support package taxonomy](../rejected/2026-06-20-classify-support-packages.md) and supplies the package source of truth expected by [discover package inventories](2026-06-20-discover-package-inventory.md). diff --git a/docs/rfc/proposed/2026-06-20-discover-package-inventory.md b/docs/rfc/proposed/2026-06-20-discover-package-inventory.md index c99412c14b..bb2f3e95ee 100644 --- a/docs/rfc/proposed/2026-06-20-discover-package-inventory.md +++ b/docs/rfc/proposed/2026-06-20-discover-package-inventory.md @@ -6,17 +6,17 @@ Status: proposed Package and gate inventories are repeated by hand. [scripts/publint-all.ts](../../../scripts/publint-all.ts) has a static list of publishable packages. The [package cookbook](../../cookbook/adding-a-package.md) tells authors to update several files. The [package README](../../../packages/README.md) carries a hand-written dependency graph. [CI](../../../.github/workflows/ci.yml) and [development docs](../../development.md) can drift from the actual `doc-sync` subcommands when new gates are added. These lists are small today, but every new package or gate creates another manual synchronization point. -Static lists are appropriate when they encode policy; they are needless friction when they duplicate manifest data that already exists in `package.json`, workspace globs, or package metadata. +Static lists are appropriate when they encode policy; they are needless friction when they duplicate manifest data or layout facts that already exist in `package.json`, workspace globs, or the package hierarchy. ## Proposal -Make package/gate inventories discoverable. Publishability should come from explicit package aspect metadata, not from a static array in a script or the npm `private` flag. Module graph generation should read package manifests. `doc-sync` should be the one command that defines and prints its sub-gates, with docs linking to that command rather than restating a second list. +Make package/gate inventories discoverable. Publishability should come from the deliberate [package hierarchy](2026-06-20-package-hierarchy.md) plus package manifests, not from a static array in a script or the npm `private` flag. Module graph generation should read package manifests. `doc-sync` should be the one command that defines and prints its sub-gates, with docs linking to that command rather than restating a second list. -The metadata should come from [classifying packages by aspect](2026-06-20-classify-packages-by-aspect.md) rather than a single support/product bucket: a package may be core, bash-related, filesystem-related, persistence-related, provider-facing, example-facing, testing-only, and/or publishable. Discovery needs enough explicit facts to drive gates without baking a fragile hierarchy into every script. +The hierarchy does not need to encode every fact about a package, but it should encode the broad maintenance policy: core/product packages, integrations, capability seams, and support/test/example packages should not all require a hand-maintained exception list before scripts can tell them apart. ## Acceptance criteria -- `publint-all` discovers publishable packages from manifests plus explicit aspect metadata. +- `publint-all` discovers publishable packages from the hierarchy plus manifests instead of a hard-coded array. - Adding a package does not require editing a static package list for every gate. - Docs describe the source of truth rather than repeating generated inventories. - CI invokes the aggregate commands and lets those commands own their sub-gate lists. diff --git a/docs/rfc/proposed/2026-06-20-package-hierarchy.md b/docs/rfc/proposed/2026-06-20-package-hierarchy.md new file mode 100644 index 0000000000..bc0c401ca2 --- /dev/null +++ b/docs/rfc/proposed/2026-06-20-package-hierarchy.md @@ -0,0 +1,57 @@ +# RFC: Reorganize packages into a modular hierarchy + +Status: proposed + +## Problem + +`packages/` is flat. Core product packages, provider integrations, capability seams, example UI support, and snapshot-only replay support all sit at the same level and look equally foundational. The [package README](../../../packages/README.md) already has a `FIXME(package-hierarchy)` noting that `ui-stdio` and `llm-replay` were extracted from examples mostly for reuse and coverage. The flat layout makes support packages appear more product-shaped than they are and forces publish/lint/doc scripts to encode intent through comments or static lists. + +This is not just cosmetic. A package's location currently says little about whether it is core API, a swappable capability, an adapter integration, an example harness helper, or test infrastructure. That makes future removal harder because every top-level package looks like part of the same public surface. + +## Proposal + +Move packages into a deliberate hierarchy under `packages/`. The exact layout is deferred to the implementing PR, but it should group packages by modular role rather than keep every package at one flat level. + +One plausible shape: + +```text +packages/ + core/ + llm/ + session/ + system-prompt/ + tools/ + agent/ + agent-loop/ + invariants/ + capabilities/ + bash/ + bash-local/ + tool-bash/ + session-persistence/ + session-persistence-jsonl/ + session-persistence-sqlite/ + integrations/ + llm-deepseek/ + llm-pi-ai/ + acp/ + support/ + ui-stdio/ + llm-replay/ +``` + +The final implementation may choose different names or groupings, but it should keep the same intent: core APIs, capability seams, concrete integrations, and support/test/example packages are distinguishable from the filesystem alone. Npm package names can stay `@deepseek-ai/dsh-*`; the hierarchy is about repo structure and maintenance policy, not public package renaming. + +This proposal does not delete `llm-replay` or `ui-stdio` by itself. It makes their status honest: either they graduate into product packages with documented consumers, or they live under a support/testing/example classification where release and compatibility expectations are lower. + +## Acceptance criteria + +- Packages move from the flat `packages//` layout into a documented modular hierarchy. +- The implementing PR chooses the exact hierarchy and updates workspace globs, TypeScript paths, package docs, generated module graphs, `cordis.yml` package paths, build scripts, and publish/lint scripts in one coordinated move. +- Scripts that publish, lint publishability, or generate package inventories use the hierarchy instead of an ad hoc static list where the hierarchy is enough to express the policy. +- Docs explain which package groups are part of the product API and which groups are support/test/example infrastructure. +- New package guidance tells authors where to place a package and discourages new one-off top-level groups. + +## What we give up + +The restructure churns imports, workspace globs, docs links, and package paths. That churn is acceptable pre-release if it prevents the flat layout from fossilizing support packages as product contracts. diff --git a/docs/rfc/proposed/2026-06-20-remove-redundant-snapshot-log-goldens.md b/docs/rfc/proposed/2026-06-20-remove-redundant-snapshot-log-goldens.md index cb40dbbf9f..15b996cf61 100644 --- a/docs/rfc/proposed/2026-06-20-remove-redundant-snapshot-log-goldens.md +++ b/docs/rfc/proposed/2026-06-20-remove-redundant-snapshot-log-goldens.md @@ -6,7 +6,7 @@ Status: proposed Model-driving ACP snapshot scenarios ship both `session.jsonl` and `session.golden.jsonl`. For normal recorded scenarios, `session.jsonl` is the replay fixture harvested from a real run, and the replay test normalizes the newly persisted log and compares it to `session.golden.jsonl`. In the current fixtures, the normalized recorded log and normalized golden are identical for the ordinary recorded scenarios. -Authored override scenarios (`error-finish`, `cancel`) currently use `replay.override.json` to drive model behavior and keep `session.jsonl` as a minimal dummy fixture, while `session.golden.jsonl` holds the expected persisted log. That split is also unnecessary: when an override sidecar exists, `llm-replay` replaces the derived script and does not need `session.jsonl` for model chunks, so `session.jsonl` can still be the expected session-log artifact for the scenario. +Authored override scenarios (`error-finish`, `cancel`) currently use `replay.override.json` to drive model behavior and keep `session.jsonl` as a minimal dummy fixture, while `session.golden.jsonl` holds the expected persisted log. The override file is a JSON array of `ReplayEntry` objects: `{ "kind": "chunks", "chunks": StreamChunk[] }`, `{ "kind": "throw", "chunks": StreamChunk[], "message": string, "code": string, "status"?: number }`, or `{ "kind": "hang" }`. That split is also unnecessary: when an override sidecar exists, `llm-replay` replaces the derived script and does not need `session.jsonl` for model chunks, so `session.jsonl` can still be the expected session-log artifact for the scenario. ## Proposal diff --git a/docs/rfc/rejected/2026-06-20-classify-support-packages.md b/docs/rfc/rejected/2026-06-20-classify-support-packages.md deleted file mode 100644 index 254ddbdc7c..0000000000 --- a/docs/rfc/rejected/2026-06-20-classify-support-packages.md +++ /dev/null @@ -1,28 +0,0 @@ -# RFC: Classify product, integration, and support packages - -Status: rejected — a single product/support taxonomy is too coarse. If package metadata changes, it should be aspect-oriented (`core`, `bash`, `fs`, `persistence`, `example`, `testing`, `publishable`, and similar facets) instead of forcing every package into one hierarchy. - -## Problem - -`packages/` is flat. Core product packages, provider integrations, tool implementations, example UI support, and snapshot-only replay support all sit at the same level and look equally publishable. The [package README](../../../packages/README.md) already has a `FIXME(package-hierarchy)` noting that `ui-stdio` and `llm-replay` were extracted from examples mostly for reuse and coverage. The flat layout makes support packages appear more foundational than they are and forces publish/lint/doc scripts to special-case intent in prose or static lists. - -This is not just cosmetic. A package's location currently says little about whether it is core API, an integration, an example harness helper, or test infrastructure. That makes future removal harder because every top-level package looks like part of the same public surface. - -## Proposal - -Introduce an explicit package classification and move packages accordingly, for example `packages/core/`, `packages/integrations/`, `packages/tools/`, `packages/testing/`, and `packages/examples/`, or an equivalent structure decided in the implementing PR. The important part is that example/test support packages are not indistinguishable from product core. - -The rejected part is the one-dimensional taxonomy. The useful follow-up is [explicit package aspect metadata](../proposed/2026-06-20-classify-packages-by-aspect.md) that scripts can consume without pretending a package has only one role. - -This proposal does not delete `llm-replay` or `ui-stdio` by itself. It makes their status honest: either they graduate into product packages with documented consumers, or they live under a support/testing/example classification where release and compatibility expectations are lower. - -## Acceptance criteria - -- Each package has an explicit classification visible from path or package metadata. -- Scripts that publish, lint publishability, or generate module graphs use the classification instead of an ad hoc static list. -- Docs explain which package classes are part of the product API. -- YAML loader paths and TypeScript path aliases are updated in one coordinated move. - -## What we give up - -The restructure churns imports, workspace globs, docs links, and package paths. That churn is acceptable pre-release if it prevents the flat layout from fossilizing a support package as a product contract. diff --git a/scripts/publint-all.ts b/scripts/publint-all.ts index 9d439fba98..e0bee4393e 100644 --- a/scripts/publint-all.ts +++ b/scripts/publint-all.ts @@ -3,7 +3,7 @@ import { resolve } from 'node:path' // publint every publishable package (vendor/ is private upstream code and // examples/ are not packages; both are out of scope). -// TODO(package-inventory): derive this from explicit package classification metadata. +// TODO(package-inventory): derive this from the deliberate package hierarchy. const packages = [ 'packages/llm', 'packages/session',