mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
docs(i18n): record the v1-vs-v2 briefing A/B in the Agent Note
Head-to-head replay of the same ten historical examples, both arms in one time window with identical prompts and pairwise blind judging: prose quality and cost at parity (stylistic margins only); the shipped briefing wins two objective outcomes — code-fence-only examples land byte-identical to the human-reviewed updates with zero model tokens, and the flagged first-occurrence move reproduces the human-reviewed gloss relocation the section-only form leaves as a contract violation. Chinese counterpart brought along via the briefed path and the pair re-recorded.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-26-briefed-minimal-translation-updates.md
|
||||
2026-07-26-briefed-minimal-translation-updates.md: 42baedc8d68557bc0d273c5a476806ac480d4afd
|
||||
2026-07-26-briefed-minimal-translation-updates.zh.md: 18653fe1097f4028a0671b6d15d1982ad137f47a
|
||||
2026-07-26-briefed-minimal-translation-updates.md: 63f25d5c36caecba435e9192534e0b494e1e0105
|
||||
2026-07-26-briefed-minimal-translation-updates.zh.md: 17cf2f895b187fb794e691c2b14d6e0bff78363d
|
||||
|
||||
@@ -26,6 +26,8 @@ The decision followed a controlled replay of ten real pair updates from this rep
|
||||
- On the briefing, a small model performed at parity with the large one, so the update path no longer assumes a frontier translator.
|
||||
- Batching three pairs into one subagent showed no reliable saving over three briefed runs and couples unrelated failures; it was rejected.
|
||||
|
||||
A second head-to-head replay on the same ten examples compared this note's shipped briefing against its earlier section-only form (no unit tier, no computed mechanical path, counterpart-only context, no first-occurrence tracking). Prose quality and cost were at parity — pairwise blind verdicts split with only stylistic margins — and the shipped form won on two objective outcomes: the two code-fence-only examples were completed byte-identical to the human-reviewed historical updates in under a second with no model tokens, and on the example whose edit moved a term's document-wide first occurrence, the shipped briefing's flagged move reproduced the human-reviewed gloss relocation while the section-only form left a 首次出现 violation for review to catch.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep the workflow, just scope the gate** — the gate scan was the smaller cost; the corpus loads and archaeology dominated. Scoping alone would have left the ~3x overhead in place.
|
||||
|
||||
@@ -26,6 +26,8 @@ Status: implemented
|
||||
- 以简报为输入,小模型的表现与大模型持平,因此更新路径不再假定翻译必须由前沿模型完成。
|
||||
- 把三对文档合并给同一个 subagent,相比三次各自带简报的运行没有可靠的节省,还把互不相关的失败耦合在一起;该方案被否决。
|
||||
|
||||
在同样这十个样例上进行的第二次正面对比回放,把本文最终交付的简报与其早前仅按章节的形态(没有单元层级、没有直接算出的机械路径、上下文只含对侧文件、不跟踪首次出现)相对照。行文质量与成本两相持平(两两盲评裁定各有胜负,差距仅在文风),而最终交付的形态在两项客观结果上胜出:两个只涉及围栏代码块的样例在一秒之内完成且不消耗任何模型 token,产出与经人工评审的历史更新逐字节一致;而在那个编辑使某术语在整篇文档中的首次出现发生移位的样例上,最终交付的简报所标记的移位复现了经人工评审的括注迁移,仅按章节的形态则留下一处「首次出现」违例,留待评审去捕捉。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
- **保留原工作流,只让门禁支持按对检查**:门禁扫描本是较小的开销,大头在语料加载与翻查历史。只收窄检查范围,约 3 倍的开销仍会原地保留。
|
||||
|
||||
Reference in New Issue
Block a user