CI Flake Triage

Detect flaky tests from JUnit XML retries and emit a triage report with top unstable cases.

Security Scan

The skill's code and runtime instructions match its stated purpose (parsing JUnit XML to find flaky vs persistent failures), it requires only bash and python3, and there are no unexplained network calls, credential requests, or installs.

Purpose & Capability

The name/description (CI Flake Triage) aligns with the included script and SKILL.md: both parse JUnit XML, aggregate statuses, classify flaky vs persistent failures, and emit text or JSON reports. Required binaries (bash, python3) are appropriate for the provided shell+Python implementation.

Instruction Scope

SKILL.md instructs the agent to run the bundled script against a JUnit file glob. The script only reads files matched by the glob, parses XML, and prints/returns results. It does not reference unrelated system paths, environment secrets, or external endpoints.

Install Mechanism

There is no install spec (instruction-only plus bundled script). Nothing is downloaded or written to disk beyond running the included script; this is low-risk and proportionate.

Credentials

The skill requests no environment variables or credentials by default. Optional runtime variables (JUNIT_GLOB, TRIAGE_TOP, OUTPUT_FORMAT, FAIL_ON_PERSISTENT, FAIL_ON_FLAKE) are reasonable and scoped to the task.

Persistence & Privilege

The skill does not request persistent presence (always:false) and does not modify other skills or system config. Autonomous invocation is allowed (platform default) but combined with the script's limited scope this is not a concern.

Assessment

This skill appears coherent and safe for use: it only reads JUnit XML files (based on the provided JUNIT_GLOB), parses them with Python, and emits reports. Before running in CI, confirm the JUNIT_GLOB only matches the intended test artifacts (it can read any files the glob matches). No network calls or credential access are present, so there is no obvious data exfiltration risk from the skill itself.

Use this skill to turn noisy JUnit retry artifacts into a focused flaky-test report.

What this skill does

• Reads one or more JUnit XML files (for example: first run + rerun artifacts)
• Aggregates status per test case (passed, failed, skipped, error)
• Flags flaky candidates when a test has both fail-like and pass outcomes
• Separates persistent failures from flaky failures
• Prints top flaky tests to prioritize stabilization work

Inputs

Optional:

• JUNIT_GLOB (default: test-results/**/*.xml)
• TRIAGE_TOP (default: 20)
• OUTPUT_FORMAT (text or json, default: text)
• FAIL_ON_PERSISTENT (0 or 1, default: 0) — exit non-zero when persistent failures exist
• FAIL_ON_FLAKE (0 or 1, default: 0) — exit non-zero when flaky candidates exist

Run

Text report:

JUNIT_GLOB='artifacts/junit/**/*.xml' \
TRIAGE_TOP=15 \
bash skills/ci-flake-triage/scripts/triage-flakes.sh

JSON output for CI ingestion:

JUNIT_GLOB='artifacts/junit/**/*.xml' \
OUTPUT_FORMAT=json \
FAIL_ON_PERSISTENT=1 \
bash skills/ci-flake-triage/scripts/triage-flakes.sh

Run with bundled fixtures:

JUNIT_GLOB='skills/ci-flake-triage/fixtures/*.xml' \
bash skills/ci-flake-triage/scripts/triage-flakes.sh

Output contract

• Exit 0 when no fail gates are enabled (default)
• Exit 1 if FAIL_ON_PERSISTENT=1 and persistent failures are found
• Exit 1 if FAIL_ON_FLAKE=1 and flaky candidates are found
• In text mode, prints summary + top flaky + persistent failures
• In json mode, prints machine-readable summary and testcase details