Proof Loop Clawhub V030

• Outcome: Review
• Latest audit: Sep 16, 2026 · 8:48 AM UTC
• Version: 0.3.0

Overview

This is a mostly disclosed repo-local proof workflow, but its completion gate can be fooled by incomplete proof artifacts, so it needs review before relying on it.

Install only if you are comfortable with a file-based discipline aid rather than a cryptographically enforced proof system. Review generated .agent/tasks artifacts, keep verifier sessions genuinely separate, and do not treat PROOF_LOOP_PASS or PROOF_LOOP_SCHEMA_PASS as sufficient for CI or release approval without stronger validation of spec criteria and evidence.

A.I.G

Vulnerability Patterns

• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally

Findings (2)

T09 · Insecure Skill Coding Practices

Warning

• Location: scripts/check_task.py:61
• Finding: Completion Gate Accepts Forged or Incomplete Verification Artifacts<![CDATA[ ## Vulnerability Details **File Location**: `scripts/check_task.py:61-79` **Vulnerability Type**: Insufficient validation of security-critical proof artifacts **Risk Level**: Medium ### Vulnerable Code ```python if verdict: if verdict.get("overall") != PASS: failures.append(f"overall is {verdict.get('overall')!r}, expected PASS") criteria = verdict.get("criteria") if not isinstance(criteria, list) or not criteria: failures.append("criteria must be a non-empty list") else: for index, criterion in enumerate(criteria, start=1): if not isinstance(criterion, dict): failures.append(f"criterion #{index} must be an object") continue cid = criterion.get("id", f"#{index}") status = criterion.get("status") if status not in VALID_STATUSES: failures.append(f"{cid} has invalid status {status!r}") elif status != PASS: failures.append(f"{cid} is {status}, expected PASS") if not problems_are_clear(problems_path): failures.append("problems.md is not empty") ``` ### Technical Analysis The completion gate treats the repository-controlled `verdict.json` file as authoritative without establishing that it represents the acceptance criteria frozen in `spec.md`. The implementation only requires: 1. `spec.md` to exist. 2. `verdict.json` to contain `overall: "PASS"`. 3. `criteria` to be a non-empty list whose supplied entries have `status: "PASS"`. 4. `problems.md` to be absent or empty. It does not: - Extract acceptance-criterion identifiers from `spec.md`. - Require an exact one-to-one match between the frozen criteria and verdict entries. - Reject duplicate, omitted, or unexpected criterion identifiers. - Require the verdict to satisfy the bundled verdict schema. - Require `phase` to identify an independent verification phase. - Require evidence for each acceptance criterion. - Authenticate the veri ...[truncated 1608 chars]
• Remediation: <![CDATA[ ## Remediation Suggestions 1. Parse the canonical acceptance-criterion identifiers from `spec.md` using a strict, documented format. 2. Require the verdict to contain exactly one result for every criterion in the frozen specification. 3. Reject missing, duplicate, malformed, and unexpected criterion identifiers. 4. Apply full JSON Schema validation before evaluating completion. 5. Require completion verdicts to have an explicit verification phase, such as `phase: "verify"`. 6. Require structured evidence references for every PASS result and verify that referenced artifacts exist. 7. Bind `task_id` in the verdict to the task directory and specification. 8. If verifier separation must be technically enforceable, use provenance that the builder cannot freely forge, such as separately controlled CI identity, signed attestations, or repository permissions. A plain editable `agent` field is not sufficient. 9. Add regression tests proving that the gate rejects: - Omitted criteria. - Duplicate criteria. - Criteria absent from the specification. - Minimal fabricated verdicts. - Non-verification phases. - Missing evidence. ]]>

T09 · Insecure Skill Coding Practices

Note

• Location: bin/proof-loop:113
• Finding: Schema Validation Checks Only Required Top-Level Keys<![CDATA[ ## Vulnerability Details **File Location**: `bin/proof-loop:113-148` **Vulnerability Type**: Incomplete JSON Schema validation **Risk Level**: Low ### Vulnerable Code ```python def schema_validate_json(data: Any, schema: dict[str, Any], path: str) -> list[str]: errors: list[str] = [] required = schema.get("required", []) if isinstance(data, dict): for key in required: if key not in data: errors.append(f"{path}: missing required key {key}") else: errors.append(f"{path}: expected object") return errors def cmd_validate(args: argparse.Namespace) -> int: path = task_dir(Path(args.root), args.task) checks = [ (path / "verdict.json", ROOT / "schemas" / "verdict.schema.json"), (path / "evidence.json", ROOT / "schemas" / "evidence.schema.json"), ] errors: list[str] = [] for data_path, schema_path in checks: if not data_path.exists(): if data_path.name == "evidence.json" and not args.require_evidence_json: continue errors.append(f"missing {data_path}") continue try: errors.extend(schema_validate_json(load_json(data_path), load_json(schema_path), str(data_path))) except Exception as exc: errors.append(f"{data_path}: {exc}") if errors: print("PROOF_LOOP_SCHEMA_FAIL") for err in errors: print(f"- {err}") return 1 print(f"PROOF_LOOP_SCHEMA_PASS {path}") return 0 ``` ### Technical Analysis Although the command is presented as validation against the bundled JSON schemas, `schema_validate_json` only verifies that the input is an object and that required top-level keys exist. It ignores the remaining JSON Schema constraints, including: - Property types. - Enumerated values. - Array types and item structures. - String formats. - Nested required properties. - Additional-property restrictions, where applicable. - Other ...[truncated 1539 chars]
• Remediation: <![CDATA[ ## Remediation Suggestions 1. Replace the custom required-key check with a conforming JSON Schema Draft 2020-12 validator. 2. If using Python's `jsonschema` package, pin and verify the dependency and use `Draft202012Validator`. 3. Validate bundled schemas themselves during development or startup using the validator's schema-checking facility. 4. Collect and report all validation errors with their JSON paths so users can identify malformed fields. 5. Strengthen the schemas to define complete criterion item structures, required nested fields, identifier patterns, and appropriate `additionalProperties` behavior. 6. Ensure the completion gate invokes the same full validator instead of maintaining a weaker, separate interpretation. 7. Add negative tests for: - Invalid `overall` values. - Incorrect property types. - Non-array `criteria`. - Malformed criterion entries. - Missing nested fields. - Invalid evidence records. ]]>

SkillSpector

Vulnerability Patterns

• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• Rogue Agent: Self-Modification, Session Persistence
• Behavioral AST: exec() Call, eval() Call, Dynamic Import
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• MCP Tool Poisoning: Hidden Instructions, Unicode Deception, Parameter Description Injection

Findings (10)

Tp4

High

• Category: MCP Tool Poisoning
• Confidence: 97% confidence
• Finding: The declared description suggests a higher-level process or orchestration skill for evidence-gated development workflows, including frozen acceptance criteria, separated builder/verifier roles, and durable proof artifacts. The supplied code does none of that. It is simply a tiny verifier script that loads a JSON file and checks whether it exactly equals a hardcoded set of navigation labels. This is a materially different primary purpose, so the description does not accurately represent the code chunk.

Self-Modification

High

• Category: Rogue Agent

Content

def write(path: Path, content: str, force: bool) -> None:
    if path.exists() and not force:
        raise SystemExit(f"Refusing to overwrite existing file: {path}")
    path.write_text(content, encoding="utf-8")

• Confidence: 85% confidence
• Finding: Skill modifies its own code, configuration, or behavior at runtime. Self-modification enables an agent to escalate privileges, disable safety constraints, or install persistent backdoors.

Lp3

Medium

• Category: MCP Least Privilege
• Confidence: 95% confidence
• Finding: The skill instructs the agent to read and write repository files and run local helper scripts, but it does not declare any explicit tool scope such as permissions or allowed tools. That creates an authorization gap where the effective capabilities are broader than the manifest communicates, increasing the chance of unintended file modification or shell execution beyond what reviewers expect.

subprocess module call

Medium

• Category: Dangerous Code Execution

Content

def run(cmd: list[str], cwd: Path) -> subprocess.CompletedProcess[str]:
    print("$ " + " ".join(cmd))
    result = subprocess.run(cmd, cwd=cwd, text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    print(result.stdout.strip())
    print(f"exit={result.returncode}")
    return result

• Confidence: 70% confidence
• Finding: subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Natural-Language Policy Violations

Medium

• Confidence: 89% confidence
• Finding: The example notes require navigation labels to be German and treats English form labels as a failure, which implies a mandatory language constraint. Because this markdown does not indicate user opt-in or document a justified region-specific requirement, it appears to violate the language/locale policy.

Autonomous Decision Making

Medium

• Category: Excessive Agency

Content

| Failure | Prevention |
|---------|-----------|
| Agent claims done without checking | Verifier is separate, required |
| ACs drift during build | Spec is frozen before build starts |
| Later sessions can't tell what was verified | Verdict artifacts stay in repo |
| Builder judges own work | Fresh verifier is a hard rule |

• Confidence: 75% confidence
• Finding: Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

subprocess module call

Medium

• Category: Dangerous Code Execution

Content

def run(*args: object, cwd: Path | None = None) -> subprocess.CompletedProcess[str]:
    return subprocess.run(
        [str(a) for a in args],
        cwd=cwd,
        text=True,

• Confidence: 70% confidence
• Finding: subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Natural-Language Policy Violations

Low

• Confidence: 82% confidence
• Finding: This SVG contains natural-language text stating that the demo check confirms expected English labels and expected German labels. Because the file presents specific language expectations without any visible indication of user-selectable locale or justification, it may reflect a locale-policy constraint embedded in skill artifacts.

Scope Creep

Low

• Category: Excessive Agency

Content

## Hard Boundaries

- Do not broaden scope.
- Do not rewrite unrelated code.
- Do not write final sign-off.

• Confidence: 75% confidence
• Finding: Skill's behavior or capabilities extend beyond its stated purpose. Scope creep allows an agent to perform actions unrelated to its documented functionality, increasing the attack surface.

Missing User Warnings

Low

• Confidence: 83% confidence
• Finding: This markdown file directs the agent to update `verdict.json` and `problems.md`, which are file-write operations affecting the user's workspace. The description provides no explicit warning or disclosure that running the skill will modify files, so a user may not realize it performs persistent changes.

Static analysis

No suspicious patterns detected.