mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
Merge remote-tracking branch 'origin/master' into worktree/pr498-conflict-20260723
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-23-browser-demo-gif-recording.md: 096edf453d6b61c4d9046b284ef67a460edf4e88
|
||||
2026-07-23-browser-demo-gif-recording.zh.md: f5b8eac1c8dd57a59e9c2293ecc71511078a4896
|
||||
@@ -0,0 +1,29 @@
|
||||
# Agent Note: Browser demo GIF recording
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-23-browser-demo-gif-recording.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Browser demonstrations have been assembled with one-off capture and encoding commands. That makes timing and output size inconsistent, encourages continuous recordings that obscure the useful state changes, and can blur the boundary between a genuine server or API flow and a fixture. Combining local recording with attachment upload or pull-request editing also gives a media task unrelated remote-write authority.
|
||||
|
||||
## Decision
|
||||
|
||||
The repository provides the [`record-browser-gif`](../../../skills/record-browser-gif/SKILL.md) skill for local browser-demo artifacts. It uses the available browser-control workflow, establishes whether the requested flow is real, fixture-backed, or otherwise simulated, and captures a small storyboard only after semantically observable UI states. Frames and the output live outside the Git worktree by default.
|
||||
|
||||
The bundled `encode_gif.py` helper orders frames lexically, assigns explicit hold durations, uses an `ffmpeg` palette pipeline, and validates source dimensions plus the encoded frame count, dimensions, duration, and byte limit through `ffprobe`. The workflow stops after returning the verified absolute GIF path; uploading the artifact and mutating a pull request, issue, or document remain separate workflows.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Record continuous video and convert it afterward.** Continuous capture preserves every cursor movement and loading transition but produces larger, noisier artifacts and makes deterministic timing harder. A state storyboard better fits short feature demonstrations where the meaningful evidence is a handful of visible transitions.
|
||||
|
||||
**Keep an inline `ffmpeg` recipe in the skill.** Reconstructing quoting, timing manifests, palette filters, overwrite behavior, and post-encode checks in every run is error-prone. A bundled helper keeps those mechanics executable while the skill owns capture judgment.
|
||||
|
||||
**Include GitHub attachment and description editing.** Upload and remote mutation require separate authentication, confirmation, and recovery rules. Excluding them keeps invocation of a recording skill local and reversible.
|
||||
|
||||
**Use a fixture whenever it is easier to stage.** Fixtures are valid when the requested demonstration is explicitly fixture-backed, but they do not substantiate a real-server or real-API claim. The skill preserves the requested provenance and reports a missing prerequisite instead of silently changing it.
|
||||
|
||||
## Consequences
|
||||
|
||||
Recordings are small, repeatable local artifacts with explicit provenance and a clean repository boundary. The workflow gives up smooth continuous motion, depends on locally available `ffmpeg` and `ffprobe`, and requires the recorder to identify semantic capture points. The helper is exercised against a four-state browser demonstration and invalid duration input; skill shape and repository links are covered by the skill validator and documentation gates.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Agent Note: 浏览器演示 GIF 录制
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-23-browser-demo-gif-recording.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
浏览器演示一直通过一次性的截取与编码命令制作。这会导致播放节奏和输出大小不一致,容易让录制者选择连续录制,反而掩盖有用的状态变化,还可能模糊真实服务器或 API 流程与 fixture(测试前置数据)之间的界限。将本地录制与附件上传或 PR(Pull Request)编辑合并在同一任务中,还会让本应仅处理媒体的任务获得无关的远程写入权限。
|
||||
|
||||
## 决策
|
||||
|
||||
仓库提供 [`record-browser-gif`](../../../skills/record-browser-gif/SKILL.md) skill(技能),用于生成本地浏览器演示产物。该 skill 使用当前可用的浏览器控制工作流,先确认请求的流程是真实流程、由 fixture 支撑,还是采用其他模拟方式,再仅在 UI 达到语义上可观察的状态后截取一组精简的分镜帧。帧文件与输出产物默认存放在 Git worktree 之外。
|
||||
|
||||
随附的 `encode_gif.py` 辅助脚本按词法顺序排列各帧,为每帧设置明确的停留时长,通过 `ffmpeg` 调色板流水线编码,并借助 `ffprobe` 校验源图像尺寸以及编码结果的帧数、尺寸、时长和字节上限。工作流在返回已验证的 GIF 绝对路径后即结束;上传产物以及修改 PR、issue 或文档仍属于独立的工作流。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**连续录制视频后再转换。**连续录制能保留每一次光标移动和加载过渡,但会产生体积更大、干扰更多的产物,也更难保持确定的播放时序。状态分镜更适合简短的功能演示,因为有意义的证据只是少数几个可见的状态变化。
|
||||
|
||||
**在 skill 中保留内联 `ffmpeg` 配方。**每次运行都重新组装引号转义、时序清单、调色板过滤器、覆盖行为和编码后检查,容易出错。随附的辅助脚本使这些机制保持可执行,skill 则负责判断何时截取画面。
|
||||
|
||||
**纳入 GitHub 附件上传与描述编辑。**上传和远程修改需要各自独立的身份认证、确认与恢复规则。将它们排除在外,可以使录制 skill 的调用保持本地且可撤销。
|
||||
|
||||
**每当 fixture 更容易布置时就使用它。**当请求明确要求由 fixture 支撑演示时,使用 fixture 是有效的;但它无法为真实服务器或真实 API 的声明提供证据。该 skill 会保持请求指定的演示来源,并在缺少先决条件时报告问题,不会擅自更改来源。
|
||||
|
||||
## 后果
|
||||
|
||||
录制结果成为体积小、可重复生成的本地产物,明确标注演示来源,并与仓库保持清晰边界。该工作流放弃了流畅的连续动态效果,依赖本机提供的 `ffmpeg` 和 `ffprobe`,并要求录制者识别具有语义意义的截取时点。测试使用四状态浏览器演示与无效时长输入检验辅助脚本;skill 的结构及仓库链接由 skill 校验器和文档门禁覆盖。
|
||||
53
.agents/skills/record-browser-gif/SKILL.md
Normal file
53
.agents/skills/record-browser-gif/SKILL.md
Normal file
@@ -0,0 +1,53 @@
|
||||
---
|
||||
name: record-browser-gif
|
||||
description: Record browser or Web UI interaction demos as optimized local GIFs using the available built-in browser, state-based frame capture, and deterministic encoding. Use when Codex is asked to make, record, or generate a GIF that demonstrates a browser workflow, including real-server or real-API behavior. Stop after returning the verified local artifact; do not upload it or edit a pull request.
|
||||
---
|
||||
|
||||
# Record Browser GIF
|
||||
|
||||
Produce a short, truthful UI demonstration as a local GIF. Use the browser-control skill for interaction and the bundled encoder for repeatable timing, dimensions, and size.
|
||||
|
||||
## Keep the boundary explicit
|
||||
|
||||
- Produce frame images and one local `.gif` artifact only.
|
||||
- Never upload the artifact, post a comment, or change a pull request, issue, or document under this skill. Hand those actions to a separate workflow if the user requests them.
|
||||
- Preserve the requested provenance. A real-server or real-API demo must not use fixture queries, mock transports, synthetic event injection, or test-only hooks. If credentials or the server are unavailable, report that limitation instead of substituting a fixture.
|
||||
- Never read or expose credential values. Use the application's normal configuration path and a benign demonstration prompt.
|
||||
|
||||
## Record the flow
|
||||
|
||||
1. Invoke the available browser-control skill and follow its setup, interaction, and cleanup instructions. Use the user's existing Chrome state only when requested or required.
|
||||
2. Resolve the evidence boundary before recording: identify the exact origin, whether the app is built or in development, the transport, and any fixture or mock mode. Record only claims that the observed setup supports.
|
||||
3. Choose three to six states that tell one story, such as initial, typed, submitted, and completed. Prefer semantic state changes over continuous capture; omit loading churn that does not help the viewer.
|
||||
4. Keep one viewport and crop for every frame. Store frames in an absolute artifact directory outside the Git worktree unless the user requests another location, and name them lexically: `00-initial.png`, `01-typed.png`, and so on.
|
||||
5. Before each screenshot, wait for a concrete UI condition such as a unique label, enabled control, changed document title, or completed response. Do not use a fixed delay as proof that the application reached the state.
|
||||
6. Capture no secrets, personal data, unrelated tabs, or transient notifications. Stop any unnecessarily long real-API run after the demonstrated state is visible.
|
||||
|
||||
Use the browser's own screenshot API. When it returns image bytes, save those bytes directly; the encoder detects image content independently of the filename extension.
|
||||
|
||||
## Encode the GIF
|
||||
|
||||
Require `python3`, `ffmpeg`, and `ffprobe`. If either media binary is missing, report the dependency instead of installing software without authorization.
|
||||
|
||||
Set `GIF_SKILL_DIR` to this skill's absolute directory, then encode the lexically ordered frames:
|
||||
|
||||
```sh
|
||||
python3 "$GIF_SKILL_DIR/scripts/encode_gif.py" \
|
||||
/absolute/path/to/frames \
|
||||
/absolute/path/to/demo.gif \
|
||||
--durations 1.5,1.5,1.5,3.5 \
|
||||
--fps 10 \
|
||||
--max-width 1200 \
|
||||
--colors 128
|
||||
```
|
||||
|
||||
One duration applies to every frame; otherwise provide one comma-separated positive duration per frame. The encoder rejects fewer than two frames, mismatched dimensions or durations, invalid limits, accidental overwrite, unexpected duration, and output above `--max-bytes`.
|
||||
|
||||
For a large artifact, reduce `--max-width` first, then `--colors` or `--fps`; retain readable text and the final state long enough to inspect. Use `--force` only after resolving the exact output path.
|
||||
|
||||
## Verify and deliver
|
||||
|
||||
1. Read the encoder's JSON summary and confirm the output path, source and encoded frame counts, dimensions, duration, and byte size.
|
||||
2. Inspect the first and final source frames and the resulting GIF. Confirm that the transition is legible, the last state is held long enough, and no sensitive content appears.
|
||||
3. If capture occurred near a repository, run `git status --short` and confirm the artifact did not dirty the worktree.
|
||||
4. Return the absolute GIF path, render it when the client supports local media, and state whether the recording used a real API, fixture, or another transport. Stop without uploading it or editing remote content.
|
||||
4
.agents/skills/record-browser-gif/agents/openai.yaml
Normal file
4
.agents/skills/record-browser-gif/agents/openai.yaml
Normal file
@@ -0,0 +1,4 @@
|
||||
interface:
|
||||
display_name: "Record Browser GIF"
|
||||
short_description: "Record and optimize local browser demo GIFs"
|
||||
default_prompt: "Use $record-browser-gif to record this browser flow as a verified local GIF."
|
||||
279
.agents/skills/record-browser-gif/scripts/encode_gif.py
Executable file
279
.agents/skills/record-browser-gif/scripts/encode_gif.py
Executable file
@@ -0,0 +1,279 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Encode lexically ordered browser screenshots into a verified GIF."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import NoReturn
|
||||
|
||||
|
||||
DEFAULT_MAX_BYTES = 5 * 1024 * 1024
|
||||
|
||||
|
||||
def fail(message: str) -> NoReturn:
|
||||
"""Exit with a concise user-correctable error."""
|
||||
raise SystemExit(f"error: {message}")
|
||||
|
||||
|
||||
def positive_float(value: str) -> float:
|
||||
"""Parse one finite positive command-line number."""
|
||||
try:
|
||||
parsed = float(value)
|
||||
except ValueError:
|
||||
fail(f"expected a number, got {value!r}")
|
||||
if not math.isfinite(parsed) or parsed <= 0:
|
||||
fail(f"expected a positive finite number, got {value!r}")
|
||||
return parsed
|
||||
|
||||
|
||||
def positive_int(value: str) -> int:
|
||||
"""Parse one positive command-line integer."""
|
||||
try:
|
||||
parsed = int(value)
|
||||
except ValueError:
|
||||
fail(f"expected an integer, got {value!r}")
|
||||
if parsed <= 0:
|
||||
fail(f"expected a positive integer, got {value!r}")
|
||||
return parsed
|
||||
|
||||
|
||||
def parse_durations(value: str, frame_count: int) -> list[float]:
|
||||
"""Expand one hold duration or validate one duration per source frame."""
|
||||
parts = [part.strip() for part in value.split(",")]
|
||||
if not parts or any(not part for part in parts):
|
||||
fail("--durations must be a number or a comma-separated list of numbers")
|
||||
durations = [positive_float(part) for part in parts]
|
||||
if len(durations) == 1:
|
||||
return durations * frame_count
|
||||
if len(durations) != frame_count:
|
||||
fail(f"--durations supplied {len(durations)} values for {frame_count} frames")
|
||||
return durations
|
||||
|
||||
|
||||
def require_binary(name: str) -> str:
|
||||
"""Resolve a required media binary or fail without attempting installation."""
|
||||
path = shutil.which(name)
|
||||
if path is None:
|
||||
fail(f"required binary {name!r} is not available on PATH")
|
||||
return path
|
||||
|
||||
|
||||
def run_json(command: list[str]) -> dict[str, object]:
|
||||
"""Run a media probe and parse its JSON object."""
|
||||
try:
|
||||
completed = subprocess.run(command, check=True, capture_output=True, text=True)
|
||||
except subprocess.CalledProcessError as error:
|
||||
detail = error.stderr.strip() or error.stdout.strip() or str(error)
|
||||
fail(detail)
|
||||
try:
|
||||
value = json.loads(completed.stdout)
|
||||
except json.JSONDecodeError as error:
|
||||
fail(f"media probe returned invalid JSON: {error}")
|
||||
if not isinstance(value, dict):
|
||||
fail("media probe returned a non-object JSON value")
|
||||
return value
|
||||
|
||||
|
||||
def probe_stream(ffprobe: str, path: Path) -> dict[str, object]:
|
||||
"""Read the first video stream's dimensions and timing metadata."""
|
||||
result = run_json(
|
||||
[
|
||||
ffprobe,
|
||||
"-v",
|
||||
"error",
|
||||
"-select_streams",
|
||||
"v:0",
|
||||
"-show_entries",
|
||||
"stream=width,height,nb_frames,duration,r_frame_rate",
|
||||
"-of",
|
||||
"json",
|
||||
str(path),
|
||||
]
|
||||
)
|
||||
streams = result.get("streams")
|
||||
if not isinstance(streams, list) or len(streams) != 1 or not isinstance(streams[0], dict):
|
||||
fail(f"expected one video stream in {path}")
|
||||
return streams[0]
|
||||
|
||||
|
||||
def stream_int(stream: dict[str, object], key: str, path: Path) -> int:
|
||||
"""Read a positive integer stream field."""
|
||||
try:
|
||||
value = int(stream[key])
|
||||
except (KeyError, TypeError, ValueError):
|
||||
fail(f"missing integer {key!r} in media probe for {path}")
|
||||
if value <= 0:
|
||||
fail(f"non-positive {key!r} in media probe for {path}")
|
||||
return value
|
||||
|
||||
|
||||
def ffconcat_quote(path: Path) -> str:
|
||||
"""Quote an ffconcat path while preserving literal backslashes."""
|
||||
value = str(path)
|
||||
if "\n" in value or "\r" in value:
|
||||
fail(f"frame path contains a newline: {path}")
|
||||
return "'" + value.replace("'", "'\\''") + "'"
|
||||
|
||||
|
||||
def write_concat_manifest(path: Path, frames: list[Path], durations: list[float]) -> None:
|
||||
"""Write an ffconcat manifest that materializes the final frame's hold."""
|
||||
lines = ["ffconcat version 1.0"]
|
||||
for frame, duration in zip(frames, durations):
|
||||
lines.append(f"file {ffconcat_quote(frame)}")
|
||||
lines.append(f"duration {duration:.6f}")
|
||||
lines.append(f"file {ffconcat_quote(frames[-1])}")
|
||||
path.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
"""Build the command-line contract."""
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("frames", type=Path, help="directory containing lexically ordered frames")
|
||||
parser.add_argument("output", type=Path, help="output .gif path")
|
||||
parser.add_argument("--pattern", default="*.png", help="frame glob within the input directory")
|
||||
parser.add_argument(
|
||||
"--durations",
|
||||
default="2",
|
||||
help="one hold duration or one comma-separated value per frame",
|
||||
)
|
||||
parser.add_argument("--fps", type=positive_int, default=10, help="encoded frames per second")
|
||||
parser.add_argument(
|
||||
"--max-width",
|
||||
type=positive_int,
|
||||
default=1200,
|
||||
help="maximum output width",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--colors",
|
||||
type=positive_int,
|
||||
default=128,
|
||||
help="palette colors, from 4 through 256",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-bytes",
|
||||
type=positive_int,
|
||||
default=DEFAULT_MAX_BYTES,
|
||||
help="maximum output size",
|
||||
)
|
||||
parser.add_argument("--force", action="store_true", help="replace an existing output file")
|
||||
return parser
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Validate inputs, encode the GIF, verify it, and print a JSON summary."""
|
||||
args = build_parser().parse_args()
|
||||
frame_dir = args.frames.resolve()
|
||||
output = args.output.resolve()
|
||||
|
||||
if not frame_dir.is_dir():
|
||||
fail(f"frame directory does not exist: {frame_dir}")
|
||||
if output.suffix.lower() != ".gif":
|
||||
fail(f"output must end in .gif: {output}")
|
||||
if output.exists() and not args.force:
|
||||
fail(f"output already exists (pass --force to replace it): {output}")
|
||||
if not 4 <= args.colors <= 256:
|
||||
fail("--colors must be between 4 and 256")
|
||||
if args.fps > 30:
|
||||
fail("--fps must not exceed 30")
|
||||
|
||||
frames = sorted(path.resolve() for path in frame_dir.glob(args.pattern) if path.is_file())
|
||||
if len(frames) < 2:
|
||||
fail(f"expected at least two frames matching {args.pattern!r} in {frame_dir}")
|
||||
if output in frames:
|
||||
fail("output path must not match an input frame")
|
||||
|
||||
durations = parse_durations(args.durations, len(frames))
|
||||
expected_duration = sum(durations)
|
||||
ffmpeg = require_binary("ffmpeg")
|
||||
ffprobe = require_binary("ffprobe")
|
||||
|
||||
dimensions = {
|
||||
(stream_int(stream, "width", frame), stream_int(stream, "height", frame))
|
||||
for frame in frames
|
||||
for stream in [probe_stream(ffprobe, frame)]
|
||||
}
|
||||
if len(dimensions) != 1:
|
||||
fail(f"all frames must have identical dimensions, got {sorted(dimensions)}")
|
||||
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
with tempfile.TemporaryDirectory(prefix="record-browser-gif-") as temporary:
|
||||
manifest = Path(temporary) / "frames.ffconcat"
|
||||
write_concat_manifest(manifest, frames, durations)
|
||||
scale = f"scale='min({args.max_width},iw)':-2:flags=lanczos"
|
||||
palette = f"palettegen=max_colors={args.colors}:stats_mode=full"
|
||||
filters = (
|
||||
f"fps={args.fps},{scale},split[base][palette_input];"
|
||||
f"[palette_input]{palette}[palette];"
|
||||
"[base][palette]paletteuse=dither=bayer:bayer_scale=3:diff_mode=rectangle"
|
||||
)
|
||||
command = [
|
||||
ffmpeg,
|
||||
"-hide_banner",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-f",
|
||||
"concat",
|
||||
"-safe",
|
||||
"0",
|
||||
"-i",
|
||||
str(manifest),
|
||||
"-vf",
|
||||
filters,
|
||||
"-loop",
|
||||
"0",
|
||||
"-t",
|
||||
f"{expected_duration:.6f}",
|
||||
"-y" if args.force else "-n",
|
||||
str(output),
|
||||
]
|
||||
try:
|
||||
subprocess.run(command, check=True)
|
||||
except subprocess.CalledProcessError as error:
|
||||
fail(f"ffmpeg failed with exit code {error.returncode}")
|
||||
|
||||
stream = probe_stream(ffprobe, output)
|
||||
width = stream_int(stream, "width", output)
|
||||
height = stream_int(stream, "height", output)
|
||||
encoded_frames = stream_int(stream, "nb_frames", output)
|
||||
try:
|
||||
actual_duration = float(stream["duration"])
|
||||
except (KeyError, TypeError, ValueError):
|
||||
fail(f"missing duration in media probe for {output}")
|
||||
tolerance = max(0.2, 2 / args.fps)
|
||||
if abs(actual_duration - expected_duration) > tolerance:
|
||||
fail(f"expected about {expected_duration:.3f}s, encoded {actual_duration:.3f}s")
|
||||
if width > args.max_width:
|
||||
fail(f"expected width at most {args.max_width}, encoded {width}")
|
||||
if encoded_frames < 2:
|
||||
fail(f"expected an animated GIF, encoded {encoded_frames} frame")
|
||||
|
||||
byte_size = output.stat().st_size
|
||||
if byte_size > args.max_bytes:
|
||||
fail(f"output is {byte_size} bytes, above --max-bytes {args.max_bytes}")
|
||||
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"output": str(output),
|
||||
"sourceFrames": len(frames),
|
||||
"encodedFrames": encoded_frames,
|
||||
"width": width,
|
||||
"height": height,
|
||||
"durationSeconds": actual_duration,
|
||||
"fps": args.fps,
|
||||
"bytes": byte_size,
|
||||
},
|
||||
indent=2,
|
||||
sort_keys=True,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user