GitHub Actions Runtime Regression Audit

Compare baseline vs current GitHub Actions run exports to catch workflow/job runtime regressions before CI costs and lead time spike.

Security Scan

The skill's code and instructions are consistent with its stated purpose (comparing GitHub Actions run JSON exports); it reads local run-export files, computes statistics, and emits text/JSON with no network exfiltration or extra credentials requested — but the shipped script was truncated in the provided listing, so full review is incomplete.

Purpose & Capability

Name/description ask to compare baseline vs current GH Actions run exports. The skill only requires bash and python3 and operates on user-supplied JSON run exports (BASELINE_GLOB, CURRENT_GLOB). Those requirements are proportionate and expected for this task.

Instruction Scope

SKILL.md instructs the user to capture GH run JSONs (gh run view) and run the included script with globs; the script parses those files, computes averages/p95 and flags regressions. This is within scope. Note: the script will read any files matched by the supplied globs, so a careless glob could cause it to parse unrelated JSON files — expected behavior but something to be cautious about.

Install Mechanism

No install spec (instruction-only plus a shipped script). No downloads or package installs. Lowest-risk install surface.

Credentials

No required credentials or config paths. The script uses only environment variables that control glob patterns and thresholds (declared in SKILL.md). No secrets are requested or used.

Persistence & Privilege

always:false and no install hooks. The skill does not request persistent/privileged presence or modify other skills or system-wide configs.

Assessment

This skill appears to do exactly what it claims: parse local GitHub Actions run JSON exports and report runtime regressions. Before running: (1) inspect the full runtime-regression-audit.sh file locally (the provided listing was truncated here, so verify the file's tail and final output/exit logic), (2) run it first against the bundled fixtures to confirm behavior, (3) avoid overly-broad globs (they can make the script parse unrelated JSON files), and (4) run with OUTPUT_FORMAT=json and review output before using FAIL_ON_CRITICAL=1 in automation. If you need higher assurance, open the full script to confirm there are no network calls, credential reads, or command execution paths beyond parsing and local reporting.

Use this skill to detect runtime regressions between historical baseline runs and current runs.

What this skill does

• Reads baseline and current GitHub Actions run JSON exports (gh run view --json ...)
• Calculates average and p95 runtime per repository + workflow + job
• Compares current metrics against baseline and ranks largest regressions
• Flags warn/critical regressions by absolute seconds and percent delta
• Emits text summary for humans or JSON for automation

Inputs

Required:

• BASELINE_GLOB (glob for baseline run JSON files)
• CURRENT_GLOB (glob for current run JSON files)

Optional:

• TOP_N (default: 20)
• OUTPUT_FORMAT (text or json, default: text)
• WARN_DELTA_SECONDS (default: 30)
• CRITICAL_DELTA_SECONDS (default: 90)
• WARN_DELTA_PERCENT (default: 15)
• CRITICAL_DELTA_PERCENT (default: 35)
• FAIL_ON_CRITICAL (0 or 1, default: 0)
• WORKFLOW_MATCH (regex, optional)
• WORKFLOW_EXCLUDE (regex, optional)
• JOB_MATCH (regex, optional)
• JOB_EXCLUDE (regex, optional)
• REPO_MATCH (regex, optional)
• REPO_EXCLUDE (regex, optional)

Collect run JSON

gh run view <run-id> --json databaseId,workflowName,headBranch,headSha,url,repository,jobs \
  > artifacts/github-actions/run-<run-id>.json

Capture a stable baseline window (for example previous 2 weeks), then current runs from latest commits.

Run

Text report:

BASELINE_GLOB='artifacts/github-actions/baseline/*.json' \
CURRENT_GLOB='artifacts/github-actions/current/*.json' \
TOP_N=15 \
WARN_DELTA_SECONDS=45 \
CRITICAL_DELTA_SECONDS=120 \
bash skills/github-actions-runtime-regression-audit/scripts/runtime-regression-audit.sh

JSON output with CI gate:

BASELINE_GLOB='artifacts/github-actions/baseline/*.json' \
CURRENT_GLOB='artifacts/github-actions/current/*.json' \
OUTPUT_FORMAT=json \
FAIL_ON_CRITICAL=1 \
bash skills/github-actions-runtime-regression-audit/scripts/runtime-regression-audit.sh

Run with bundled fixtures:

BASELINE_GLOB='skills/github-actions-runtime-regression-audit/fixtures/baseline-*.json' \
CURRENT_GLOB='skills/github-actions-runtime-regression-audit/fixtures/current-*.json' \
bash skills/github-actions-runtime-regression-audit/scripts/runtime-regression-audit.sh

Output contract

• Exit 0 in reporting mode (default)
• Exit 1 when FAIL_ON_CRITICAL=1 and at least one job crosses critical regression thresholds
• text mode prints summary + top regressions + new jobs without baseline
• json mode outputs summary, ranked regressions, and newly observed jobs