GitHub Actions Conclusion Volatility Audit

Audit GitHub Actions workflow conclusion volatility to surface unstable pipelines before they become chronic failures.

Security Scan

The skill's code, instructions, and required binaries align with its stated purpose of analyzing GitHub Actions run JSON exports for workflow volatility; there are no unexpected network endpoints, secret requests, or installs.

Purpose & Capability

The name/description match the implementation: the script reads GitHub Actions run JSON files, groups runs, computes transition-based volatility, and emits text/JSON reports. Required binaries (bash, python3) are appropriate for the provided shell + embedded Python implementation.

Instruction Scope

SKILL.md and the script consistently instruct the agent to collect run JSON (example uses 'gh run view'), run the audit script against artifacts/* JSON files, and emit results or fail CI on critical groups. The instructions operate only on local JSON artifacts and do not instruct broad file-system reads or transmitting data to unknown endpoints.

Install Mechanism

This is an instruction-only skill with one included script file; there is no install spec, no archives or third-party downloads, and nothing is written to disk beyond the normal execution of the script and the user-supplied artifacts. Risk from the install mechanism is minimal.

Credentials

The skill declares no required environment variables and the script accepts many optional environment switches (RUN_GLOB, thresholds, filters). One thing to note: SKILL.md shows collecting run JSON via the 'gh' CLI, which uses the user's GitHub authentication (stored credentials or token). The skill itself does not request any secrets, which is proportionate, but collectors (gh) will use whatever GitHub auth is configured on the host.

Persistence & Privilege

The skill is not always-enabled and has no install-time persistence. It does not modify other skills or system-wide config. Autonomous invocation is allowed (platform default) but not combined with elevated privileges or secret access.

Assessment

This skill appears to do what it says: analyze GitHub Actions run JSON files and report unstable workflows. Before installing/run: (1) Ensure bash and python3 are available. (2) Prepare artifacts by exporting workflow runs (the SKILL.md suggests using 'gh run view' — that will contact GitHub and use the host's gh authentication), and ensure you are comfortable with that network access and the credentials the gh CLI will use. (3) Run the script in a controlled workspace where the JSON artifacts come from trusted repositories (these files include repo names and run URLs). (4) No extra secrets or external endpoints are required by the skill itself, but if you adapt it to automatically fetch runs you should be aware it will contact GitHub via the gh CLI. If you need confirmation of behavior, inspect the included script (scripts/conclusion-volatility-audit.sh) — it is fully self-contained and readable.

Use this skill to detect unstable workflows that frequently flip between success and failure-like outcomes.

What this skill does

• Reads one or more workflow run JSON exports
• Groups runs by repository + workflow + branch
• Calculates volatility using conclusion transitions across run history
• Flags groups by warn/critical instability thresholds
• Emits text or JSON output for CI reporting and quality gates

Inputs

Optional:

• RUN_GLOB (default: artifacts/github-actions/*.json)
• TOP_N (default: 20)
• OUTPUT_FORMAT (text or json, default: text)
• MIN_RUNS (default: 5) — minimum runs before severity is applied
• WARN_INSTABILITY_PCT (default: 35)
• CRITICAL_INSTABILITY_PCT (default: 60)
• FAIL_ON_CRITICAL (0 or 1, default: 0)
• WORKFLOW_MATCH, WORKFLOW_EXCLUDE (regex, optional)
• BRANCH_MATCH, BRANCH_EXCLUDE (regex, optional)
• REPO_MATCH, REPO_EXCLUDE (regex, optional)

Failure-like conclusions are: failure, cancelled, timed_out, action_required, startup_failure.

Collect run JSON

gh run view <run-id> --json databaseId,workflowName,headBranch,conclusion,createdAt,updatedAt,url,repository \
  > artifacts/github-actions/run-<run-id>.json

Run

Text report:

RUN_GLOB='artifacts/github-actions/*.json' \
WARN_INSTABILITY_PCT=35 \
CRITICAL_INSTABILITY_PCT=60 \
bash skills/github-actions-conclusion-volatility-audit/scripts/conclusion-volatility-audit.sh

JSON output + fail gate:

RUN_GLOB='artifacts/github-actions/*.json' \
OUTPUT_FORMAT=json \
FAIL_ON_CRITICAL=1 \
bash skills/github-actions-conclusion-volatility-audit/scripts/conclusion-volatility-audit.sh

Output contract

• Exit 0 in reporting mode
• Exit 1 when FAIL_ON_CRITICAL=1 and one or more critical groups are found
• Text output includes summary and top unstable workflow groups
• JSON output includes summary, ranked groups, and critical_groups