Proof Agent

Adversarial verification of AI-generated work. Spawns an independent verifier to check for false claims, broken code, and security issues.

Security Scan

The skill's code and instructions are consistent with its stated purpose (spawn an independent verifier and run factual/build checks); nothing requested is disproportionate, though users should be mindful of potential data leakage when sending changed files or command output to a separate verifier agent.

Purpose & Capability

The name/description (adversarial verification) matches the included files and instructions: scripts extract git diffs, detect sensitive filenames, run URL/npm/GitHub/PyPI checks, and build a verification prompt for a separate verifier. No unrelated credentials, binaries, or install steps are requested.

Instruction Scope

SKILL.md narrowly defines what to give the verifier (original request, changed files, approach) and requires the verifier to run commands and include outputs. This is coherent for verification. However, the workflow shares changed files and command outputs with a separate agent; if changed files contain secrets the verifier would receive them unless the caller redacts them. The provided scripts detect sensitive filename patterns but do not automatically redact secret contents.

Install Mechanism

Instruction-only skill with small shell scripts; no install spec, no archives, no remote installers. This is low-risk from an install standpoint.

Credentials

No environment variables, credentials, or system config paths are requested. The scripts perform network queries (curl to URLs, npm registry, GitHub) which is appropriate for fact-checking and consistent with the stated purpose.

Persistence & Privilege

Skill is not always-enabled and does not request persistent system privileges or modify other skills. Autonomous invocation is allowed (platform default) but not combined with other concerning privileges.

Assessment

This skill appears to do what it says: build a verification prompt and run factual/build checks. Before using it, be aware that verification requires sharing the changed files and command outputs with a separate verifier agent — those can include sensitive data. Recommendations: (1) Inspect and redact any secrets in changed files before sending to an external verifier, (2) run the scripts locally in a sandboxed environment if you do not trust the verifier agent, and (3) treat the verifier's network commands' outputs as potentially exposing repository data (URL checks and registry queries are performed). If you need stronger guarantees, add automatic secret-redaction or limit which files are passed to the verifier.

Independent adversarial verification for AI work. The worker and the verifier are always separate agents — self-verification is not verification.

When to Verify

Verify automatically when:

• Subagent changed 3+ files
• ANY changed file matches: *auth*, *secret*, *permission*, Dockerfile, *.env*
• User explicitly asks for verification

Skip verification for:

• Formatting-only changes (whitespace, linting fixes)
• .gitignore changes

How to Verify

1. Spawn an independent verifier subagent — the worker CANNOT verify its own work
2. Give the verifier ONLY: the original request, files changed, and approach taken
3. Do NOT share the worker's self-assessment or test results
4. The verifier must run its own commands and provide evidence
5. If no subagent ran (manual changes or user says "verify this"), use git diff output as the approach summary

Verification Prompt

Use this prompt when spawning the verifier subagent:

VERIFICATION REQUEST

## Original Request
{what was asked}

## Files Changed
{list of files}

## Approach Taken
{what the worker did — or git diff summary if no subagent ran}

## Your Job

You are an independent verifier. The worker who made these changes CANNOT verify their own work — only you can assign a verdict.

### Review Checklist
1. Correctness: Does the code actually do what was requested?
2. Bugs & Edge Cases: Regressions, unhandled errors, missed cases?
3. Security: Vulnerabilities, exposed secrets, permission issues?
4. Build: Does it build/compile/lint cleanly?
5. Facts: Are any claims, version numbers, or URLs verifiable? Check them.

### Rules
- For EVERY check, include the actual command you ran and its output
- Do NOT take the worker's word for anything
- Do NOT give PASS without running at least 3 verification commands
- You have NO information about the worker's test results — verify independently

## Verdict

Assign EXACTLY ONE verdict as a markdown heading (### PASS, ### FAIL, or ### PARTIAL):

### PASS
All checks passed. Every claim backed by command output.

### FAIL
Issues found. List each as a bullet (- file, line, what's wrong, severity: critical/major/minor).

### PARTIAL
Some passed, some unverifiable. List both with evidence.

Verdicts

• PASS — All checks passed with evidence
• FAIL — Issues found. Report to user with specifics. Retry up to 3 times if fixable.
• PARTIAL — Some checks passed, others couldn't be verified. Report what's unverifiable.

After Verification

• PASS: Report summary to user, proceed
• FAIL: Report issues to user. If auto-fixable, spawn worker to fix, then re-verify (max 3 attempts)
• PARTIAL: Report to user, let them decide whether to proceed

Scripts

scripts/verify.sh [base-ref]

Auto-extracts git diff, changed files, commit messages, and sensitive file detection. Outputs a filled verification prompt ready to send to the verifier subagent. Default base: HEAD~1.

bash scripts/verify.sh         # verify last commit
bash scripts/verify.sh main    # verify all changes since main

scripts/fact-check.sh <file> [file2 ...]

Extracts and validates factual claims from files:

• URLs → HTTP status check
• npm packages → registry version lookup
• GitHub Actions → tag/SHA existence check

bash scripts/fact-check.sh src/content/articles/en/my-article.md
bash scripts/fact-check.sh .github/workflows/*.yml

Returns exit code 1 if any checks fail.

Configuration

Projects can customize via proof-agent.yaml in the repo root (loaded by proof_agent/config.py):

thresholds:
  min_files_changed: 3
  always_verify:
    - "**/*auth*"
    - "**/*secret*"
    - "**/*permission*"
    - "**/Dockerfile"
    - "**/*.env*"
  never_verify:
    - "**/.gitignore"

retry:
  max_attempts: 3
  escalate_on_max: true

Key Principle

The worker and verifier must be separate agents. Self-verification is not verification.