GitHub Actions Actor Reliability Audit

Audit GitHub Actions run reliability by actor to surface high-risk contributors and flaky automation owners.

Security Scan

The skill is internally coherent: it analyzes local GitHub Actions run JSON files with bash+python and does not request credentials or install remote code; one minor mismatch: the SKILL.md examples use the 'gh' CLI but 'gh' is not listed in required binaries.

Purpose & Capability

Name/description match the implementation: the script reads exported GitHub Actions run JSON files, groups by actor, computes failure metrics, and emits text/JSON. Required binaries (bash, python3) are appropriate for the provided scripts. Minor inconsistency: the SKILL.md's suggested data-collection command uses the 'gh' CLI (gh run view) to create the JSON exports, but 'gh' is not listed under required binaries; this is a usability/documentation mismatch rather than a functional red flag.

Instruction Scope

Runtime instructions and the script operate on local JSON files matched by RUN_GLOB, apply user-provided regex filters, compute statistics, and output results. The SKILL.md clearly documents how to collect run JSONs (via 'gh run view') and how to run the script. The instructions do not direct the agent to read unrelated system files, access secrets, or send data to arbitrary external endpoints; any GitHub access comes from the user's invocation of the 'gh' CLI to produce the JSON artifacts.

Install Mechanism

No install spec is provided and this is an instruction-only skill with included scripts. Nothing is downloaded or written by an installer; risk from installation is minimal.

Credentials

The skill declares no required environment variables or credentials, which is proportionate to its stated purpose of processing local JSON exports. Note: running 'gh run view' (per SKILL.md) requires the user's GitHub CLI authentication if used, but the skill itself does not request or store any tokens or secrets.

Persistence & Privilege

The skill does not set always:true, does not request persistent system presence, and does not modify other skills or system-wide configuration. It runs ad hoc against files the user provides.

Assessment

This skill looks like what it says: a local analyzer that consumes exported GitHub Actions run JSON files and ranks actors by reliability. Before installing or running: (1) Note the SKILL.md examples use the 'gh' CLI to export run JSONs — make sure you have 'gh' installed and authenticated if you follow those steps (adding 'gh' to the declared required binaries would be sensible). (2) The script only reads files matching RUN_GLOB, so verify the glob points to only the data you intend to analyze. (3) No credentials are requested by the skill itself, but using 'gh' will use your existing GitHub CLI auth when you export runs — that is expected behavior. (4) If you want extra assurance, review the remainder of the script (the file was partially truncated in the provided listing) to confirm it contains only reporting logic and no network calls; run the script on sanitized fixtures first.

Use this skill to rank which actors (humans or bots) are associated with the least reliable GitHub Actions outcomes.

What this skill does

• Reads GitHub Actions run JSON exports
• Groups runs by actor (optionally actor + workflow)
• Measures failure rate, failed-run volume, and latest failure streak per actor
• Scores severity (ok, warn, critical) for triage and CI policy gates
• Emits text or JSON output for automation

Inputs

Optional:

• RUN_GLOB (default: artifacts/github-actions/*.json)
• TOP_N (default: 20)
• OUTPUT_FORMAT (text or json, default: text)
• GROUP_BY (actor or actor-workflow, default: actor)
• FAILURE_CONCLUSIONS (comma-separated, default: failure,cancelled,timed_out,startup_failure)
• MIN_RUNS (minimum runs required, default: 5)
• WARN_FAILURE_RATE (0..1, default: 0.25)
• CRITICAL_FAILURE_RATE (0..1, default: 0.5)
• WARN_FAILED_RUNS (default: 4)
• CRITICAL_FAILED_RUNS (default: 8)
• WARN_FAILURE_STREAK (default: 2)
• CRITICAL_FAILURE_STREAK (default: 4)
• ACTOR_MATCH / ACTOR_EXCLUDE (regex, optional)
• WORKFLOW_MATCH / WORKFLOW_EXCLUDE (regex, optional)
• BRANCH_MATCH / BRANCH_EXCLUDE (regex, optional)
• EVENT_MATCH / EVENT_EXCLUDE (regex, optional)
• REPO_MATCH / REPO_EXCLUDE (regex, optional)
• FAIL_ON_CRITICAL (0 or 1, default: 0)

Collect run JSON

gh run view <run-id> --json databaseId,workflowName,event,headBranch,conclusion,createdAt,updatedAt,url,repository,actor,triggeringActor \
  > artifacts/github-actions/run-<run-id>.json

Run

Text report:

RUN_GLOB='artifacts/github-actions/*.json' \
bash skills/github-actions-actor-reliability-audit/scripts/actor-reliability-audit.sh

JSON output + fail gate:

RUN_GLOB='artifacts/github-actions/*.json' \
OUTPUT_FORMAT=json \
FAIL_ON_CRITICAL=1 \
bash skills/github-actions-actor-reliability-audit/scripts/actor-reliability-audit.sh

Run against bundled fixtures:

RUN_GLOB='skills/github-actions-actor-reliability-audit/fixtures/*.json' \
bash skills/github-actions-actor-reliability-audit/scripts/actor-reliability-audit.sh

Output contract

• Exit 0 in report mode (default)
• Exit 1 when FAIL_ON_CRITICAL=1 and one or more actor groups are critical
• Text mode prints summary + ranked actor groups
• JSON mode prints summary + ranked groups + critical groups