mirror of
https://github.com/deepseek-ai/deepseek-harness
synced 2026-08-15 21:04:50 +00:00
ci: add real-API e2e workflow against external DeepSeek API
Adds .github/workflows/e2e.yml, which runs `pnpm run test:e2e` against the external DeepSeek API (https://api.deepseek.com) using a DEEPSEEK_API_KEY repo secret. ci.yml stays keyless/forkable; this is a separate, secret-consuming workflow that fills the gap of nothing in CI exercising the with-key suites. - Triggers: workflow_dispatch + push to main/master + nightly schedule + pull_request. A job-level `if:` skips untrusted PRs (forks + Dependabot, both keyless), keying the Dependabot test on the PR author (pull_request.user.login) not github.actor. A job-level skip reports as success, so this is safe as a required check. - Unconditional preflight hard-fails on a missing secret so the self-skipping suite can't report a false green when the key is misconfigured. - Secret scoped to the preflight + e2e steps only; permissions: contents: read; DEEPSEEK_BASE_URL pinned to the external API; single Node 24; timeout 45m; cancel-in-progress only for PR runs. Plan converged with Codex (gpt-5.5:xhigh) over 3 review rounds.
This commit is contained in:
89
.github/workflows/e2e.yml
vendored
Normal file
89
.github/workflows/e2e.yml
vendored
Normal file
@@ -0,0 +1,89 @@
|
||||
name: E2E (real DeepSeek API)
|
||||
|
||||
# Real-API end-to-end suite (`pnpm run test:e2e`). Unlike ci.yml this job
|
||||
# consumes the DEEPSEEK_API_KEY secret and hits the external API at
|
||||
# https://api.deepseek.com (DEEPSEEK_BASE_URL is pinned to it explicitly so a
|
||||
# stray repo-root .env can't redirect the run).
|
||||
#
|
||||
# pull_request IS a trigger, but GitHub withholds repo secrets from BOTH forked
|
||||
# PRs and Dependabot PRs (the latter are same-repo, fork==false, but still
|
||||
# keyless). The job-level `if:` below skips the whole job for those untrusted
|
||||
# PRs. The Dependabot test keys on the PR AUTHOR (pull_request.user.login), not
|
||||
# github.actor (the run trigger) — a maintainer reopening a Dependabot PR would
|
||||
# make github.actor human while the PR is still keyless. When the job runs, the
|
||||
# secret is expected, so the preflight hard-fails if it's missing (a missing
|
||||
# secret would otherwise make the self-skipping suite report a false green). A
|
||||
# job skipped by a job-level `if:` reports as a SUCCESSFUL check, so this
|
||||
# workflow is safe to use as a required status check if desired.
|
||||
#
|
||||
# Note: scheduled triggers are auto-disabled after 60 days of repo inactivity;
|
||||
# push/pull_request/workflow_dispatch act as backstops.
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches: [main, master]
|
||||
pull_request:
|
||||
schedule:
|
||||
# 09:17 UTC nightly — off-peak and off the top-of-hour cron stampede.
|
||||
- cron: '17 9 * * *'
|
||||
|
||||
# Cancel a superseded PR run (it is on a stale commit); never cancel a
|
||||
# push/schedule run — it is already producing the post-merge/nightly signal.
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||
|
||||
# Least privilege: this job only reads the repo to run tests.
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
e2e:
|
||||
runs-on: ubuntu-latest
|
||||
# Run on every trusted event. Skip untrusted PRs (forks + Dependabot) where
|
||||
# the secret is withheld — they would otherwise hard-fail the preflight.
|
||||
if: >-
|
||||
github.event_name != 'pull_request'
|
||||
|| !(github.event.pull_request.head.repo.fork || github.event.pull_request.user.login == 'dependabot[bot]')
|
||||
# Serial files (fileParallelism: false), 120s/test, retry 2. 45m bounds a
|
||||
# wedged run while leaving headroom for retry storms against a slow API.
|
||||
timeout-minutes: 45
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 24
|
||||
|
||||
- name: Enable corepack (pnpm)
|
||||
run: corepack enable
|
||||
|
||||
- name: Install (immutable)
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
# Guard against a false green: the e2e suites self-skip when the key is
|
||||
# absent, so a missing/misconfigured secret would otherwise pass as
|
||||
# "all skipped". This job only runs on trusted events (the `if:` above
|
||||
# excludes keyless PRs), so the secret MUST be present — fail loudly if not.
|
||||
- name: Preflight (require DEEPSEEK_API_KEY)
|
||||
env:
|
||||
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${DEEPSEEK_API_KEY:-}" ]; then
|
||||
echo "::error::DEEPSEEK_API_KEY is not set. The e2e suite would self-skip and"
|
||||
echo "::error::report a false green. Configure the repo secret DEEPSEEK_API_KEY."
|
||||
exit 1
|
||||
fi
|
||||
echo "DEEPSEEK_API_KEY present."
|
||||
|
||||
# Real-API end-to-end tests only. The keyless gates (lint/typecheck/
|
||||
# coverage/snapshot/build/etc.) already run in ci.yml on every push/PR;
|
||||
# no need to repeat them or build first (tests run unbuilt via tsx).
|
||||
# DEEPSEEK_BASE_URL is pinned to the external API; the secret is scoped to
|
||||
# this step (and preflight) only — never exposed to checkout/setup/install.
|
||||
- name: E2E tests (real DeepSeek API)
|
||||
env:
|
||||
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
|
||||
DEEPSEEK_BASE_URL: https://api.deepseek.com
|
||||
run: pnpm run test:e2e
|
||||
Reference in New Issue
Block a user