mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
docs(skills): fix dsh-trim-cot-leakage review findings from #2117
The battery self-exclusion widens from references/ to the whole skill directory (SKILL.md quotes the same leaked shapes), and the invocation rules name the owning note's evidence self-hits; natural-language battery lines gain -i so sentence-initial capitals hit; the actually/ wait false-positive family states that no committed line probes it. The one test now says resolvability clears only this skill's bar — current-state surfaces still reject resolvable change stories; the never-deletion-alone rule is scoped to passages that carry factual clauses. The first few-shot Fixed example links the real web input-machine note path. The citations note's Verification exemption (+zh, pair re-recorded) names the skill's files and the note's own quoted evidence, making the claim mechanically honest.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-09-committed-artifact-citations.md
|
||||
2026-08-09-committed-artifact-citations.md: 181378aab12a4423d9ac5216bac5304f3cfca11c
|
||||
2026-08-09-committed-artifact-citations.zh.md: 36f32bb1fd0d6991919959e1ebe6482bf7b19b32
|
||||
2026-08-09-committed-artifact-citations.md: a578b1e32268af2669ee26c59d19c738bc9f707b
|
||||
2026-08-09-committed-artifact-citations.zh.md: d525dc0f58884bdf808b597213a7e23c1d6a0301
|
||||
|
||||
@@ -28,7 +28,7 @@ One repo-wide purge applied these rules across the prose surfaces, including the
|
||||
|
||||
## Verification
|
||||
|
||||
The audit's grep batteries (English and Chinese, comments and prose, `--hidden` for `.agents/`) return no design-ordinal citations outside recorded fixtures, archived notes, and the trim skill's own calibration examples; `verify-type-equiv`, the `gen-*` freshness checks, and `verify-translation-pairing` pin the regenerated and re-recorded surfaces. Coverage gap: no gate rejects a new ordinal citation — review owns the rule.
|
||||
The audit's grep batteries (English and Chinese, comments and prose, `--hidden` for `.agents/`) return no design-ordinal citations outside recorded fixtures, archived notes, the trim skill's own files, and this note's quoted evidence; `verify-type-equiv`, the `gen-*` freshness checks, and `verify-translation-pairing` pin the regenerated and re-recorded surfaces. Coverage gap: no gate rejects a new ordinal citation — review owns the rule.
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ Status: implemented
|
||||
|
||||
## 验证
|
||||
|
||||
审计的成批 grep 检索(英文与中文、注释与行文、对 `.agents/` 加 `--hidden`)在已录制 fixture、已归档 Agent Note 与该清理技能自带的校准示例之外没有命中任何设计序号引用;`verify-type-equiv`、各 `gen-*` 新鲜度检查与 `verify-translation-pairing` 把重新生成与重新记录的表面固定下来。覆盖缺口:没有门禁会拒绝新的序号引用,这条规则由评审把关。
|
||||
审计的成批 grep 检索(英文与中文、注释与行文、对 `.agents/` 加 `--hidden`)在已录制 fixture、已归档 Agent Note、该清理技能自身文件与本笔记引用的证据之外没有命中任何设计序号引用;`verify-type-equiv`、各 `gen-*` 新鲜度检查与 `verify-translation-pairing` 把重新生成与重新记录的表面固定下来。覆盖缺口:没有门禁会拒绝新的序号引用,这条规则由评审把关。
|
||||
|
||||
## 后果
|
||||
|
||||
|
||||
@@ -5,11 +5,11 @@ description: Use when auditing or fixing prose that reads like a leaked reasonin
|
||||
|
||||
# Trimming Chain-of-Thought Leakage
|
||||
|
||||
Chain-of-thought leakage is prose whose vantage is the authoring session rather than the repository: it cites artifacts only that session could see, narrates the change instead of the state, or argues with a reviewer who has left. The fix is never deletion alone — restate every factual clause so it stands at HEAD, then delete the transcript around it. **REQUIRED BACKGROUND:** [dsh-prose-standard](../dsh-prose-standard/SKILL.md) owns the complete-proposition rule this skill applies; the [committed-artifact-citations note](../../notes/implemented/process/2026-08-09-committed-artifact-citations.md) owns the citation rule's rationale. It is guidance, not a script.
|
||||
Chain-of-thought leakage is prose whose vantage is the authoring session rather than the repository: it cites artifacts only that session could see, narrates the change instead of the state, or argues with a reviewer who has left. The fix is never deletion alone when a passage carries factual clauses — restate each so it stands at HEAD, then delete the transcript around it; a passage carrying none (an audit code, control-flow narration) is deleted outright. **REQUIRED BACKGROUND:** [dsh-prose-standard](../dsh-prose-standard/SKILL.md) owns the complete-proposition rule this skill applies; the [committed-artifact-citations note](../../notes/implemented/process/2026-08-09-committed-artifact-citations.md) owns the citation rule's rationale. It is guidance, not a script.
|
||||
|
||||
## The one test
|
||||
|
||||
For every suspect passage ask: **could a reader at HEAD, with no access to any session transcript, PR thread, or uncommitted draft, resolve every reference and verify every claim?** If yes, it is not leakage, however historical it sounds. If no, restate the surviving facts from the repository's vantage and delete the rest.
|
||||
For every suspect passage ask: **could a reader at HEAD, with no access to any session transcript, PR thread, or uncommitted draft, resolve every reference and verify every claim?** If no, restate the surviving facts from the repository's vantage and delete the rest. If yes, it is not leakage, however historical it sounds — but resolvability only clears this skill's bar: on current-state surfaces (READMEs, docs, JSDoc) a resolvable change story is still change narration, and class 3 routes it to its sanctioned home.
|
||||
|
||||
## Taxonomy
|
||||
|
||||
@@ -42,4 +42,4 @@ The citation boundary is where unaided passes fail in both directions — deleti
|
||||
2. Audit read-only first: run the [recall batteries](references/recall-batteries.md) (with `--hidden` so `.agents/` is searched), then judge every hit semantically. The batteries are probes, not the definition — each review round of the original purge surfaced shapes the batteries missed, so also read the densest prose in scope (module JSDoc, READMEs, Agent Notes) without a pattern in hand.
|
||||
3. Fix owner-first per surface: generated catalogs → fix the source JSDoc or generator template, then regenerate; type-equivalence fences → fix the source JSDoc, then re-paste both bilingual pages (`verify-type-equiv` pins them); bilingual pairs → update the counterpart and re-record per [dsh-translate-docs](../dsh-translate-docs/SKILL.md); model-visible strings → wording is behavior, so flag for a snapshot-backed change instead of silently rewording.
|
||||
4. Before deleting anything, enumerate the passage's propositions (prose-standard) and check the [overcorrection traps](references/examples.md#overcorrection-traps): trims that flip an obligation into an endorsement, promote a hypothetical to a shipped feature, delete a true fact, or drop provenance.
|
||||
5. Verify: re-run the batteries expecting only sanctioned keeps and this skill's own references; confirm every remaining citation resolves at HEAD; run the gates for touched surfaces (`doc-sync` for docs, `verify-type-equiv`, `verify-translation-pairing`).
|
||||
5. Verify: re-run the batteries expecting only sanctioned keeps, this skill's own directory, and the owning note's quoted evidence; confirm every remaining citation resolves at HEAD; run the gates for touched surfaces (`doc-sync` for docs, `verify-type-equiv`, `verify-translation-pairing`).
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Few-shot leakage examples
|
||||
|
||||
Distilled from the 2026-08 repo-wide purge and its review rounds. Use them to identify the governing principle, not as text templates. This file deliberately quotes leaked shapes as calibration material — the [recall batteries](recall-batteries.md) exclude this directory, and its wording is not a license elsewhere.
|
||||
Distilled from the 2026-08 repo-wide purge and its review rounds. Use them to identify the governing principle, not as text templates. This file deliberately quotes leaked shapes as calibration material — the [recall batteries](recall-batteries.md) exclude the skill's directory, and its wording is not a license elsewhere.
|
||||
|
||||
## Dead citations
|
||||
|
||||
@@ -8,9 +8,9 @@ Distilled from the 2026-08 repo-wide purge and its review rounds. Use them to id
|
||||
|
||||
**Leaked:** "Slash input resolves against the visible catalog (decision 21)."
|
||||
|
||||
**Fixed:** "Slash input resolves against the visible catalog — the plain-text-reference decision, owned by the web input-machine note."
|
||||
**Fixed:** "Slash input resolves against the visible catalog — the plain-text-reference decision, owned by [the web input-machine note](../../../notes/implemented/architecture/2026-07-25-web-input-machine-and-slash-pipeline.md)."
|
||||
|
||||
The ordinal resolves nowhere at HEAD; the decision's name and owning note path do. Name the owning note's path at least once per file; later mentions may use the searchable name alone.
|
||||
The ordinal resolves nowhere at HEAD; the decision's name and owning note path do. Name the owning note's path at least once per file — as a link where the surface supports one — and later mentions may use the searchable name alone.
|
||||
|
||||
### Decision ordinal without an owner
|
||||
|
||||
|
||||
@@ -5,18 +5,19 @@ Probes for [the taxonomy](../SKILL.md#taxonomy), tuned during the 2026-08 purge.
|
||||
## Invocation rules
|
||||
|
||||
- Add `--hidden --glob '!.git/**'` so `.agents/` is searched; ripgrep skips dot-directories by default and the purge's biggest miss risk was Agent Notes.
|
||||
- Exclusions go last so a later include cannot re-admit them: `--glob '!vendor/**' --glob '!node_modules/**' --glob '!.agents/notes/archived/**' --glob '!.agents/skills/dsh-trim-cot-leakage/references/**'` (this directory quotes leaked shapes as calibration), plus recorded fixture and snapshot directories in scope.
|
||||
- Exclusions go last so a later include cannot re-admit them: `--glob '!vendor/**' --glob '!node_modules/**' --glob '!.agents/notes/archived/**' --glob '!.agents/skills/dsh-trim-cot-leakage/**'` (the skill's own files quote leaked shapes as calibration), plus recorded fixture and snapshot directories in scope. The [owning note](../../../notes/implemented/process/2026-08-09-committed-artifact-citations.md) also self-hits through its quoted evidence; judge it as evidence, not usage.
|
||||
- Natural-language lines carry `-i` so sentence-initial capitals hit ("This PR adds…", "Probably fine…"); the code-shaped first line stays case-sensitive — `-i` would turn `\bT\d\b` and `\bP-I\b` into noise.
|
||||
- A zero-hit pattern proves nothing until you have seen it match: test it against a known-positive string before trusting the negative.
|
||||
|
||||
## English battery
|
||||
|
||||
```sh
|
||||
rg -n --hidden '\(decision \d|\(audit [A-Z]\d|design §|plan §|design ledger|\(B ruling|\bP-I\b|\bW\d\b|\bT\d\b' ...
|
||||
rg -n --hidden 'this PR|this branch|this stack|later PR|previous commit|this commit' ...
|
||||
rg -n --hidden 'used to |no longer|previously|the old |was renamed|was moved' ...
|
||||
rg -n --hidden '\bv1\b|this cut|\bcut \d|\btoday\b|\bfor now\b|roadmap' ...
|
||||
rg -n --hidden -i 'this PR|this branch|this stack|later PR|previous commit|this commit' ...
|
||||
rg -n --hidden -i 'used to |no longer|previously|the old |was renamed|was moved' ...
|
||||
rg -n --hidden -i '\bv1\b|this cut|\bcut \d|\btoday\b|\bfor now\b|roadmap' ...
|
||||
rg -n --hidden -i 'rejected in review|review round|reviewer|as of v\d' ...
|
||||
rg -n --hidden 'probably |should be enough|should suffice|it simply|is safe —|is safe --' ...
|
||||
rg -n --hidden -i 'probably |should be enough|should suffice|it simply|is safe —|is safe --' ...
|
||||
rg -n --hidden '§\d' ...
|
||||
```
|
||||
|
||||
@@ -36,7 +37,7 @@ Judged and kept during the purge; expect them again:
|
||||
- **"This PR" in process docs** — documentation *about* PR workflow ("the PR body should…", templates, this repo's process notes) legitimately says "PR"; the ban is on a doc adopting one PR's vantage about the code.
|
||||
- **`v1` as protocol or path segment** — `/v1/chat` endpoints and wire-format names are identifiers, not version stamps.
|
||||
- **`§N` with a committed owner** — external standards (RFC 9110 §10.1.5) and committed docs that own their §-numbering stay citable by section.
|
||||
- **Contrastive "actually" and noun "wait"** — ordinary English, not hedging.
|
||||
- **Contrastive "actually" and noun "wait"** — ordinary English, not hedging; no committed line probes them, so they surface only when you extend the battery with broader hedging patterns.
|
||||
- **"Today" in generated timestamps and CLI output samples** — recorded output keeps its voice.
|
||||
- **本版本 in zh prose** — a legitimate rendering of "this release" in versioned-artifact contexts; the banned indexical is 本版 as a bare stamp mirroring "this cut".
|
||||
- **Alternatives-considered sections** — "rejected" inside an Agent Note's genre slot is the sanctioned home, not review choreography.
|
||||
|
||||
Reference in New Issue
Block a user