mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
docs: address final model experience review
This commit is contained in:
@@ -8,7 +8,7 @@ The [documentation standard](../../../AGENTS.md) assigns limitations to the pack
|
||||
|
||||
## Decision
|
||||
|
||||
Every package manifest under `packages/<group>/<pkg>/package.json` has a sibling README carrying a canonical `## Known Limitations and Deferred Work` section: a condensed bullet list of consumer-visible gaps (unimplemented features, platform caveats, deliberate MVP cuts) and consciously postponed work (TODO markers, RFC deferrals still open). A `doc-sync` gate, `verify-readme-limitations` ([scripts/verify-readme-limitations.ts](../../../../scripts/verify-readme-limitations.ts)), derives the package set from those manifests, rejects a missing README, and enforces the shape per README: exactly one limitations-like heading, byte-equal to the canonical h2, with at least one top-level bullet. Near-miss headings at any level ("Limitations", "Deferred", "What is NOT here", "Non-goals", …) fail the gate, so variant sections cannot creep back beside — or instead of — the canonical one.
|
||||
Every package manifest under `packages/<group>/<pkg>/package.json` has a sibling README carrying a canonical `## Known Limitations and Deferred Work` section: a condensed bullet list of consumer-visible gaps (unimplemented features, platform caveats, deliberate MVP cuts) and consciously postponed work (TODO markers, RFC deferrals still open). A `doc-sync` gate, `verify-package-readme-limitations` ([scripts/verify-package-readme-limitations.ts](../../../../scripts/verify-package-readme-limitations.ts)), derives the package set from those manifests, rejects a missing README, and enforces the shape per README: exactly one limitations-like heading, byte-equal to the canonical h2, with at least one top-level bullet. Near-miss headings at any level ("Limitations", "Deferred", "What is NOT here", "Non-goals", …) fail the gate, so variant sections cannot creep back beside — or instead of — the canonical one.
|
||||
|
||||
A package with genuinely nothing to declare is whitelisted (`NO_LIMITATIONS` in the script) and must NOT carry the section. The inverted check keeps the whitelist honest in both directions: an empty or boilerplate section cannot satisfy the gate, and giving a whitelisted package real limitations forces the whitelist edit in the same change. Whitelist entries are validated against the scanned package set, so a package rename or removal fails loud instead of silently un-gating a README.
|
||||
|
||||
@@ -22,7 +22,7 @@ The gate checks presence, shape, and the whitelist; the bullets' truthfulness an
|
||||
|
||||
## Consequences
|
||||
|
||||
- A new package cannot ship without either declaring its gaps or explicitly claiming it has none; a missing, drifted, or empty section fails `doc-sync` locally (pre-push) and in CI (`readme-limitations` in the run-gates doc-sync leaf set).
|
||||
- A new package cannot ship without either declaring its gaps or explicitly claiming it has none; a missing, drifted, or empty section fails `doc-sync` locally (pre-push) and in CI (`package-readme-limitations` in the run-gates doc-sync leaf set).
|
||||
- Every package README answers the limitations question through the canonical heading or an explicit no-limitations allowlist entry.
|
||||
- One more fast tsx script in the `doc-sync` chain; no new dependency (plain `node:fs` glob + line scan).
|
||||
- The canonical heading is enforced verbatim, so renaming it later is a mechanical one-script-plus-all-READMEs change guarded by the same gate.
|
||||
|
||||
@@ -8,11 +8,11 @@ A package README can explain APIs and runtime mechanics without answering the qu
|
||||
|
||||
## Decision
|
||||
|
||||
Every workspace package README ends with the canonical [Model Experience section](../../../cookbook/adding-a-package.md#4-write-the-package-readme), immediately before `## Known Limitations and Deferred Work`; a package on the no-limitations allowlist ends with Model Experience itself. Packages with direct, multi-surface, conditional, capped, or lifetime effects use one H3 block per context surface. Each block says what the relevant model literally receives and when under `**What the model sees**:`, then classifies the token effect under `**Token effect**:`. Short stable source literals are quoted verbatim in the first field, with named placeholders only for interpolated values. Long stable prompts or schema descriptions live in a `### Verbatim model-visible text` appendix after the context surfaces, one titled `markdown` fence per literal, and the relevant surface links to them. Summaries are reserved for data-dependent payloads, provider-owned text, or schemas too large to reproduce. The default subject is the conversation model; a package that invokes an auxiliary model, such as a summarizer or search provider, names that request separately. Agent-scoped visibility is stated where it changes which agent receives a contribution. Prompt text and tool schemas are described separately whenever configuration or scoping can hide one without the other.
|
||||
Every workspace package README ends with the canonical [Model Experience section](../../../cookbook/adding-a-package.md#4-write-the-package-readme), immediately before `## Known Limitations and Deferred Work`; a package on the no-limitations allowlist ends with Model Experience itself. Packages with direct, multi-surface, conditional, capped, or lifetime effects use one H3 block per context surface. Each block says what the relevant model literally receives and when under `**What the model sees**:`, then classifies the token effect under `**Token effect**:`; the structured section grounds at least one surface with inline code, a nested `markdown` block, or an anchored catalog link. Every stable system-prompt paragraph, including a one-liner, follows those fields inside the owning H3 as a titled H4 plus `markdown` fence; the H3 title contains `system prompt`. Other short stable source literals remain inline with named placeholders only for interpolated values; other long non-generated literals use the same nested H4 form. Model Experience subsections do not link to each other because physical nesting owns the literal. Tool-schema surfaces use `schema` in their H3 and link the relevant anchored package section of the generated [tool schema catalog](../../../tool-catalog.md) rather than copying default descriptions or JSON Schema, then state only configuration or composition deltas absent from that catalog. A runtime-only definition outside the catalog's stated scope links that scope and explains the exception before reproducing its stable text. Summaries are reserved for data-dependent payloads and provider-owned text. The default subject is the conversation model; a package that invokes an auxiliary model, such as a summarizer or search provider, names that request separately. Agent-scoped visibility is stated where it changes which agent receives a contribution. Prompt text and tool schemas are described separately whenever configuration or scoping can hide one without the other.
|
||||
|
||||
Every package participates. A package with no model-context effect, or one simple effect rendered entirely by another package, uses the verifier's audited sentence allowlist. It carries exactly one sentence beginning `None, as ` or `Indirectly, through ` instead of expanding a negative fact into a structured block. A provider backend whose single context path is formatted and inserted entirely by a named consumer uses the indirect sentence even when it caps or filters data before returning it. Packages that register model input themselves, invoke a model, or own multiple context paths keep the context-surface blocks even when they add zero direct prompt tokens.
|
||||
Every package participates. A package with no model-context effect, or one simple effect rendered entirely by another package, uses the verifier's audited sentence allowlist. It carries exactly one sentence beginning `None, as ` or `Indirectly, through ` instead of expanding a negative fact into a structured block. Pure routing, transport, and keyless test-support packages use the none form when they create no model-bound content. A provider backend whose single context path is formatted and inserted entirely by a named consumer uses the indirect sentence even when it caps or filters data before returning it; wiring bundles do the same when every model effect belongs to named children. The indirect form names that consumer only to locate this package's contribution and does not restate the consumer's implementation. Structured blocks likewise document only package-owned inputs, transformations, and deltas. Packages that own model input, output shaping, multiple context paths, or an auxiliary request keep context-surface blocks even when they add zero direct prompt tokens.
|
||||
|
||||
`verify-package-readme-model-experience` discovers packages from `packages/*/*/package.json`, requires one sibling README and the canonical final-section order, and validates one of two package-classified bodies. An allowlisted package carries exactly one sentence with its assigned prefix; every other package carries at least one H3 context surface with the two exact, non-empty fields and one blank line between each element. Optional long literals follow the structured blocks under the exact appendix heading, with each H4 title paired to one non-empty `markdown` fence and linked from a model-view field. It runs in `doc-sync` and the parallel gate runner. The check owns package classification, structural presence, block and appendix shape, and order; implementation review owns coverage and the truth of the prose.
|
||||
`verify-package-readme-model-experience` discovers packages from `packages/*/*/package.json`, requires one sibling README and the canonical final-section order, and validates one of two package-classified bodies. An allowlisted package carries exactly one sentence with its assigned prefix; every other package carries at least one H3 context surface with the two exact, non-empty fields and one blank line between each element plus at least one inline literal, nested block, or catalog link across the section. Optional verbatim literals follow those fields inside that surface, each as an H4 title paired to one non-empty `markdown` fence. Local subsection links are rejected, every system-prompt surface requires at least one nested block, and every tool-schema surface links an existing H2 section in the generated tool catalog. The check runs in `doc-sync` and the parallel gate runner. It owns package classification, structural presence, concrete-literal evidence, nested-block shape, catalog-link shape, and order; implementation review owns coverage, link relevance, and the truth of the prose.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
Reference in New Issue
Block a user