diff --git a/.agents/notes/implemented/process/2026-07-23-browser-demo-gif-recording.i18n.yaml b/.agents/notes/implemented/process/2026-07-23-browser-demo-gif-recording.i18n.yaml new file mode 100644 index 0000000000..1aee1563ad --- /dev/null +++ b/.agents/notes/implemented/process/2026-07-23-browser-demo-gif-recording.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-07-23-browser-demo-gif-recording.md: 096edf453d6b61c4d9046b284ef67a460edf4e88 +2026-07-23-browser-demo-gif-recording.zh.md: f5b8eac1c8dd57a59e9c2293ecc71511078a4896 diff --git a/.agents/notes/implemented/process/2026-07-23-browser-demo-gif-recording.md b/.agents/notes/implemented/process/2026-07-23-browser-demo-gif-recording.md new file mode 100644 index 0000000000..096edf453d --- /dev/null +++ b/.agents/notes/implemented/process/2026-07-23-browser-demo-gif-recording.md @@ -0,0 +1,29 @@ +# Agent Note: Browser demo GIF recording + +Status: implemented + +English | [中文](2026-07-23-browser-demo-gif-recording.zh.md) + +## Problem + +Browser demonstrations have been assembled with one-off capture and encoding commands. That makes timing and output size inconsistent, encourages continuous recordings that obscure the useful state changes, and can blur the boundary between a genuine server or API flow and a fixture. Combining local recording with attachment upload or pull-request editing also gives a media task unrelated remote-write authority. + +## Decision + +The repository provides the [`record-browser-gif`](../../../skills/record-browser-gif/SKILL.md) skill for local browser-demo artifacts. It uses the available browser-control workflow, establishes whether the requested flow is real, fixture-backed, or otherwise simulated, and captures a small storyboard only after semantically observable UI states. Frames and the output live outside the Git worktree by default. + +The bundled `encode_gif.py` helper orders frames lexically, assigns explicit hold durations, uses an `ffmpeg` palette pipeline, and validates source dimensions plus the encoded frame count, dimensions, duration, and byte limit through `ffprobe`. The workflow stops after returning the verified absolute GIF path; uploading the artifact and mutating a pull request, issue, or document remain separate workflows. + +## Alternatives considered + +**Record continuous video and convert it afterward.** Continuous capture preserves every cursor movement and loading transition but produces larger, noisier artifacts and makes deterministic timing harder. A state storyboard better fits short feature demonstrations where the meaningful evidence is a handful of visible transitions. + +**Keep an inline `ffmpeg` recipe in the skill.** Reconstructing quoting, timing manifests, palette filters, overwrite behavior, and post-encode checks in every run is error-prone. A bundled helper keeps those mechanics executable while the skill owns capture judgment. + +**Include GitHub attachment and description editing.** Upload and remote mutation require separate authentication, confirmation, and recovery rules. Excluding them keeps invocation of a recording skill local and reversible. + +**Use a fixture whenever it is easier to stage.** Fixtures are valid when the requested demonstration is explicitly fixture-backed, but they do not substantiate a real-server or real-API claim. The skill preserves the requested provenance and reports a missing prerequisite instead of silently changing it. + +## Consequences + +Recordings are small, repeatable local artifacts with explicit provenance and a clean repository boundary. The workflow gives up smooth continuous motion, depends on locally available `ffmpeg` and `ffprobe`, and requires the recorder to identify semantic capture points. The helper is exercised against a four-state browser demonstration and invalid duration input; skill shape and repository links are covered by the skill validator and documentation gates. diff --git a/.agents/notes/implemented/process/2026-07-23-browser-demo-gif-recording.zh.md b/.agents/notes/implemented/process/2026-07-23-browser-demo-gif-recording.zh.md new file mode 100644 index 0000000000..f5b8eac1c8 --- /dev/null +++ b/.agents/notes/implemented/process/2026-07-23-browser-demo-gif-recording.zh.md @@ -0,0 +1,29 @@ +# Agent Note: 浏览器演示 GIF 录制 + +Status: implemented + +[English](2026-07-23-browser-demo-gif-recording.md) | 中文 + +## 问题 + +浏览器演示一直通过一次性的截取与编码命令制作。这会导致播放节奏和输出大小不一致,容易让录制者选择连续录制,反而掩盖有用的状态变化,还可能模糊真实服务器或 API 流程与 fixture(测试前置数据)之间的界限。将本地录制与附件上传或 PR(Pull Request)编辑合并在同一任务中,还会让本应仅处理媒体的任务获得无关的远程写入权限。 + +## 决策 + +仓库提供 [`record-browser-gif`](../../../skills/record-browser-gif/SKILL.md) skill(技能),用于生成本地浏览器演示产物。该 skill 使用当前可用的浏览器控制工作流,先确认请求的流程是真实流程、由 fixture 支撑,还是采用其他模拟方式,再仅在 UI 达到语义上可观察的状态后截取一组精简的分镜帧。帧文件与输出产物默认存放在 Git worktree 之外。 + +随附的 `encode_gif.py` 辅助脚本按词法顺序排列各帧,为每帧设置明确的停留时长,通过 `ffmpeg` 调色板流水线编码,并借助 `ffprobe` 校验源图像尺寸以及编码结果的帧数、尺寸、时长和字节上限。工作流在返回已验证的 GIF 绝对路径后即结束;上传产物以及修改 PR、issue 或文档仍属于独立的工作流。 + +## 曾考虑的替代方案 + +**连续录制视频后再转换。**连续录制能保留每一次光标移动和加载过渡,但会产生体积更大、干扰更多的产物,也更难保持确定的播放时序。状态分镜更适合简短的功能演示,因为有意义的证据只是少数几个可见的状态变化。 + +**在 skill 中保留内联 `ffmpeg` 配方。**每次运行都重新组装引号转义、时序清单、调色板过滤器、覆盖行为和编码后检查,容易出错。随附的辅助脚本使这些机制保持可执行,skill 则负责判断何时截取画面。 + +**纳入 GitHub 附件上传与描述编辑。**上传和远程修改需要各自独立的身份认证、确认与恢复规则。将它们排除在外,可以使录制 skill 的调用保持本地且可撤销。 + +**每当 fixture 更容易布置时就使用它。**当请求明确要求由 fixture 支撑演示时,使用 fixture 是有效的;但它无法为真实服务器或真实 API 的声明提供证据。该 skill 会保持请求指定的演示来源,并在缺少先决条件时报告问题,不会擅自更改来源。 + +## 后果 + +录制结果成为体积小、可重复生成的本地产物,明确标注演示来源,并与仓库保持清晰边界。该工作流放弃了流畅的连续动态效果,依赖本机提供的 `ffmpeg` 和 `ffprobe`,并要求录制者识别具有语义意义的截取时点。测试使用四状态浏览器演示与无效时长输入检验辅助脚本;skill 的结构及仓库链接由 skill 校验器和文档门禁覆盖。 diff --git a/.agents/skills/record-browser-gif/SKILL.md b/.agents/skills/record-browser-gif/SKILL.md new file mode 100644 index 0000000000..e48e16ca40 --- /dev/null +++ b/.agents/skills/record-browser-gif/SKILL.md @@ -0,0 +1,53 @@ +--- +name: record-browser-gif +description: Record browser or Web UI interaction demos as optimized local GIFs using the available built-in browser, state-based frame capture, and deterministic encoding. Use when Codex is asked to make, record, or generate a GIF that demonstrates a browser workflow, including real-server or real-API behavior. Stop after returning the verified local artifact; do not upload it or edit a pull request. +--- + +# Record Browser GIF + +Produce a short, truthful UI demonstration as a local GIF. Use the browser-control skill for interaction and the bundled encoder for repeatable timing, dimensions, and size. + +## Keep the boundary explicit + +- Produce frame images and one local `.gif` artifact only. +- Never upload the artifact, post a comment, or change a pull request, issue, or document under this skill. Hand those actions to a separate workflow if the user requests them. +- Preserve the requested provenance. A real-server or real-API demo must not use fixture queries, mock transports, synthetic event injection, or test-only hooks. If credentials or the server are unavailable, report that limitation instead of substituting a fixture. +- Never read or expose credential values. Use the application's normal configuration path and a benign demonstration prompt. + +## Record the flow + +1. Invoke the available browser-control skill and follow its setup, interaction, and cleanup instructions. Use the user's existing Chrome state only when requested or required. +2. Resolve the evidence boundary before recording: identify the exact origin, whether the app is built or in development, the transport, and any fixture or mock mode. Record only claims that the observed setup supports. +3. Choose three to six states that tell one story, such as initial, typed, submitted, and completed. Prefer semantic state changes over continuous capture; omit loading churn that does not help the viewer. +4. Keep one viewport and crop for every frame. Store frames in an absolute artifact directory outside the Git worktree unless the user requests another location, and name them lexically: `00-initial.png`, `01-typed.png`, and so on. +5. Before each screenshot, wait for a concrete UI condition such as a unique label, enabled control, changed document title, or completed response. Do not use a fixed delay as proof that the application reached the state. +6. Capture no secrets, personal data, unrelated tabs, or transient notifications. Stop any unnecessarily long real-API run after the demonstrated state is visible. + +Use the browser's own screenshot API. When it returns image bytes, save those bytes directly; the encoder detects image content independently of the filename extension. + +## Encode the GIF + +Require `python3`, `ffmpeg`, and `ffprobe`. If either media binary is missing, report the dependency instead of installing software without authorization. + +Set `GIF_SKILL_DIR` to this skill's absolute directory, then encode the lexically ordered frames: + +```sh +python3 "$GIF_SKILL_DIR/scripts/encode_gif.py" \ + /absolute/path/to/frames \ + /absolute/path/to/demo.gif \ + --durations 1.5,1.5,1.5,3.5 \ + --fps 10 \ + --max-width 1200 \ + --colors 128 +``` + +One duration applies to every frame; otherwise provide one comma-separated positive duration per frame. The encoder rejects fewer than two frames, mismatched dimensions or durations, invalid limits, accidental overwrite, unexpected duration, and output above `--max-bytes`. + +For a large artifact, reduce `--max-width` first, then `--colors` or `--fps`; retain readable text and the final state long enough to inspect. Use `--force` only after resolving the exact output path. + +## Verify and deliver + +1. Read the encoder's JSON summary and confirm the output path, source and encoded frame counts, dimensions, duration, and byte size. +2. Inspect the first and final source frames and the resulting GIF. Confirm that the transition is legible, the last state is held long enough, and no sensitive content appears. +3. If capture occurred near a repository, run `git status --short` and confirm the artifact did not dirty the worktree. +4. Return the absolute GIF path, render it when the client supports local media, and state whether the recording used a real API, fixture, or another transport. Stop without uploading it or editing remote content. diff --git a/.agents/skills/record-browser-gif/agents/openai.yaml b/.agents/skills/record-browser-gif/agents/openai.yaml new file mode 100644 index 0000000000..720f55f7dc --- /dev/null +++ b/.agents/skills/record-browser-gif/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: "Record Browser GIF" + short_description: "Record and optimize local browser demo GIFs" + default_prompt: "Use $record-browser-gif to record this browser flow as a verified local GIF." diff --git a/.agents/skills/record-browser-gif/scripts/encode_gif.py b/.agents/skills/record-browser-gif/scripts/encode_gif.py new file mode 100755 index 0000000000..2a14ae47fd --- /dev/null +++ b/.agents/skills/record-browser-gif/scripts/encode_gif.py @@ -0,0 +1,279 @@ +#!/usr/bin/env python3 +"""Encode lexically ordered browser screenshots into a verified GIF.""" + +from __future__ import annotations + +import argparse +import json +import math +import shutil +import subprocess +import tempfile +from pathlib import Path +from typing import NoReturn + + +DEFAULT_MAX_BYTES = 5 * 1024 * 1024 + + +def fail(message: str) -> NoReturn: + """Exit with a concise user-correctable error.""" + raise SystemExit(f"error: {message}") + + +def positive_float(value: str) -> float: + """Parse one finite positive command-line number.""" + try: + parsed = float(value) + except ValueError: + fail(f"expected a number, got {value!r}") + if not math.isfinite(parsed) or parsed <= 0: + fail(f"expected a positive finite number, got {value!r}") + return parsed + + +def positive_int(value: str) -> int: + """Parse one positive command-line integer.""" + try: + parsed = int(value) + except ValueError: + fail(f"expected an integer, got {value!r}") + if parsed <= 0: + fail(f"expected a positive integer, got {value!r}") + return parsed + + +def parse_durations(value: str, frame_count: int) -> list[float]: + """Expand one hold duration or validate one duration per source frame.""" + parts = [part.strip() for part in value.split(",")] + if not parts or any(not part for part in parts): + fail("--durations must be a number or a comma-separated list of numbers") + durations = [positive_float(part) for part in parts] + if len(durations) == 1: + return durations * frame_count + if len(durations) != frame_count: + fail(f"--durations supplied {len(durations)} values for {frame_count} frames") + return durations + + +def require_binary(name: str) -> str: + """Resolve a required media binary or fail without attempting installation.""" + path = shutil.which(name) + if path is None: + fail(f"required binary {name!r} is not available on PATH") + return path + + +def run_json(command: list[str]) -> dict[str, object]: + """Run a media probe and parse its JSON object.""" + try: + completed = subprocess.run(command, check=True, capture_output=True, text=True) + except subprocess.CalledProcessError as error: + detail = error.stderr.strip() or error.stdout.strip() or str(error) + fail(detail) + try: + value = json.loads(completed.stdout) + except json.JSONDecodeError as error: + fail(f"media probe returned invalid JSON: {error}") + if not isinstance(value, dict): + fail("media probe returned a non-object JSON value") + return value + + +def probe_stream(ffprobe: str, path: Path) -> dict[str, object]: + """Read the first video stream's dimensions and timing metadata.""" + result = run_json( + [ + ffprobe, + "-v", + "error", + "-select_streams", + "v:0", + "-show_entries", + "stream=width,height,nb_frames,duration,r_frame_rate", + "-of", + "json", + str(path), + ] + ) + streams = result.get("streams") + if not isinstance(streams, list) or len(streams) != 1 or not isinstance(streams[0], dict): + fail(f"expected one video stream in {path}") + return streams[0] + + +def stream_int(stream: dict[str, object], key: str, path: Path) -> int: + """Read a positive integer stream field.""" + try: + value = int(stream[key]) + except (KeyError, TypeError, ValueError): + fail(f"missing integer {key!r} in media probe for {path}") + if value <= 0: + fail(f"non-positive {key!r} in media probe for {path}") + return value + + +def ffconcat_quote(path: Path) -> str: + """Quote an ffconcat path while preserving literal backslashes.""" + value = str(path) + if "\n" in value or "\r" in value: + fail(f"frame path contains a newline: {path}") + return "'" + value.replace("'", "'\\''") + "'" + + +def write_concat_manifest(path: Path, frames: list[Path], durations: list[float]) -> None: + """Write an ffconcat manifest that materializes the final frame's hold.""" + lines = ["ffconcat version 1.0"] + for frame, duration in zip(frames, durations): + lines.append(f"file {ffconcat_quote(frame)}") + lines.append(f"duration {duration:.6f}") + lines.append(f"file {ffconcat_quote(frames[-1])}") + path.write_text("\n".join(lines) + "\n", encoding="utf-8") + + +def build_parser() -> argparse.ArgumentParser: + """Build the command-line contract.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("frames", type=Path, help="directory containing lexically ordered frames") + parser.add_argument("output", type=Path, help="output .gif path") + parser.add_argument("--pattern", default="*.png", help="frame glob within the input directory") + parser.add_argument( + "--durations", + default="2", + help="one hold duration or one comma-separated value per frame", + ) + parser.add_argument("--fps", type=positive_int, default=10, help="encoded frames per second") + parser.add_argument( + "--max-width", + type=positive_int, + default=1200, + help="maximum output width", + ) + parser.add_argument( + "--colors", + type=positive_int, + default=128, + help="palette colors, from 4 through 256", + ) + parser.add_argument( + "--max-bytes", + type=positive_int, + default=DEFAULT_MAX_BYTES, + help="maximum output size", + ) + parser.add_argument("--force", action="store_true", help="replace an existing output file") + return parser + + +def main() -> None: + """Validate inputs, encode the GIF, verify it, and print a JSON summary.""" + args = build_parser().parse_args() + frame_dir = args.frames.resolve() + output = args.output.resolve() + + if not frame_dir.is_dir(): + fail(f"frame directory does not exist: {frame_dir}") + if output.suffix.lower() != ".gif": + fail(f"output must end in .gif: {output}") + if output.exists() and not args.force: + fail(f"output already exists (pass --force to replace it): {output}") + if not 4 <= args.colors <= 256: + fail("--colors must be between 4 and 256") + if args.fps > 30: + fail("--fps must not exceed 30") + + frames = sorted(path.resolve() for path in frame_dir.glob(args.pattern) if path.is_file()) + if len(frames) < 2: + fail(f"expected at least two frames matching {args.pattern!r} in {frame_dir}") + if output in frames: + fail("output path must not match an input frame") + + durations = parse_durations(args.durations, len(frames)) + expected_duration = sum(durations) + ffmpeg = require_binary("ffmpeg") + ffprobe = require_binary("ffprobe") + + dimensions = { + (stream_int(stream, "width", frame), stream_int(stream, "height", frame)) + for frame in frames + for stream in [probe_stream(ffprobe, frame)] + } + if len(dimensions) != 1: + fail(f"all frames must have identical dimensions, got {sorted(dimensions)}") + + output.parent.mkdir(parents=True, exist_ok=True) + with tempfile.TemporaryDirectory(prefix="record-browser-gif-") as temporary: + manifest = Path(temporary) / "frames.ffconcat" + write_concat_manifest(manifest, frames, durations) + scale = f"scale='min({args.max_width},iw)':-2:flags=lanczos" + palette = f"palettegen=max_colors={args.colors}:stats_mode=full" + filters = ( + f"fps={args.fps},{scale},split[base][palette_input];" + f"[palette_input]{palette}[palette];" + "[base][palette]paletteuse=dither=bayer:bayer_scale=3:diff_mode=rectangle" + ) + command = [ + ffmpeg, + "-hide_banner", + "-loglevel", + "error", + "-f", + "concat", + "-safe", + "0", + "-i", + str(manifest), + "-vf", + filters, + "-loop", + "0", + "-t", + f"{expected_duration:.6f}", + "-y" if args.force else "-n", + str(output), + ] + try: + subprocess.run(command, check=True) + except subprocess.CalledProcessError as error: + fail(f"ffmpeg failed with exit code {error.returncode}") + + stream = probe_stream(ffprobe, output) + width = stream_int(stream, "width", output) + height = stream_int(stream, "height", output) + encoded_frames = stream_int(stream, "nb_frames", output) + try: + actual_duration = float(stream["duration"]) + except (KeyError, TypeError, ValueError): + fail(f"missing duration in media probe for {output}") + tolerance = max(0.2, 2 / args.fps) + if abs(actual_duration - expected_duration) > tolerance: + fail(f"expected about {expected_duration:.3f}s, encoded {actual_duration:.3f}s") + if width > args.max_width: + fail(f"expected width at most {args.max_width}, encoded {width}") + if encoded_frames < 2: + fail(f"expected an animated GIF, encoded {encoded_frames} frame") + + byte_size = output.stat().st_size + if byte_size > args.max_bytes: + fail(f"output is {byte_size} bytes, above --max-bytes {args.max_bytes}") + + print( + json.dumps( + { + "output": str(output), + "sourceFrames": len(frames), + "encodedFrames": encoded_frames, + "width": width, + "height": height, + "durationSeconds": actual_duration, + "fps": args.fps, + "bytes": byte_size, + }, + indent=2, + sort_keys=True, + ) + ) + + +if __name__ == "__main__": + main()