DeepSeek/deepseek-harness
1
0
Fork 0
mirror of https://github.com/deepseek-ai/deepseek-harness synced 2026-08-15 21:04:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(review): fail closed across skill maintenance

Browse Source 
A saved complete-file candidate could overwrite newer skill guidance, while a provider-wide outage was indistinguishable from a healthy no-op run. Record the source blob and provenance, reject drift during promotion, and make total adapter failure visible.

Align cadence and feedback acquisition with the evidence contract, and narrow the borrowed-state and negative-control checks so they do not contradict the package and testing standards.
This commit is contained in:
Tianyi Cui
2026-07-15 21:39:20 +08:00
parent 3735f8c955
commit dffeac037d
3 changed files with 29 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
.agents/skills/dsh-code-review/SKILL.md
View File

@@ -32,11 +32,11 @@ description: Use when reviewing a pull request in the deepseek-harness repo —
- **Scope, ownership, and necessity:** tie each abstraction, state machine, option, defensive copy, and compatibility path to a current contract or production consumer. Challenge unrelated features, speculative generality, and behavior placed outside its owning plugin or service.
- **Configuration:** deployment-varying timeouts, caps, models, URLs, paths, and retry counts are validated `Config` fields, not literals or `DEFAULT_*` constants.
- **Enforcement boundaries:** hidden schema fields, filtered prompts, facades, wrappers, and listener ordering are not authoritative enforcement when direct or alternate callers can bypass them. Exercise denial paths at the boundary that actually executes the operation.
- **Borrowed and derived state:** snapshot caller-owned values retained after return or crossing queues, logs, processes, workers, and wire boundaries. Commit notifications and derived state only at the documented success boundary, and trace caches, prompts, UI echoes, replay, and query views to one authoritative source.
- **Final artifact bounds:** verify byte, token, item, and time limits against the complete emitted or retained artifact, including wrappers and metadata. Probe tiny limits, threshold edges, oversized single chunks, and multibyte text.
- **Borrowed and derived state:** determine whether retained caller-owned values are borrowed or snapshotted by contract; do not demand copies at typed same-process seams. Materialize mutable values that cross queues, model/tool JSON, durable logs or files, workers, processes, or wire boundaries. Commit notifications and derived state only at the documented success boundary, and trace caches, prompts, UI echoes, replay, and query views to one authoritative source.
- **Bounds cover the final operation:** verify byte, token, item, and time limits at the boundary that owns the complete emitted or retained result, including wrappers and metadata. Probe tiny limits, exact thresholds, oversized single chunks, and multibyte text for byte limits.
- **Real entry path:** tests exercise the shipped Loader, bin, worker, ACP bridge, or subprocess where relevant. A hand-mounted plugin does not catch Loader export-shape failures; a function plugin must named-export its namespace and have no default export.
- **Test strength:** assertions fail on the intended regression and verify external state, logs, events, or disposal rather than restating the implementation or trusting an agent's report. Coverage is necessary but not evidence that the scenario is correct.
- **Changed checks have a negative control:** a new or changed automated check includes a deliberately invalid fixture that fails through the real top-level runner; a green happy path does not prove the check is wired.
- **Changed checks have a negative control:** a new automated check, or a changed acceptance path in one, has a deliberately invalid case that reaches the real top-level runner and fails for the intended rule; a green happy path does not prove the check is wired.
- **Implemented RFCs match shipped reality:** when a PR implements a proposed RFC, move and rewrite it as present-tense shipped state in the same diff, then verify paths, names, and mechanisms against the implementation.
- **Transcript changes:** editor-visible or model-visible changes update snapshots or explain why no snapshot applies. Review golden diffs as behavior changes, not formatting noise.
- **Bilingual changes:** compare meaning and terminology on both sides; a green pairing hash does not prove translation quality.