Guardrail Testing — fail-closed gates, hostile fixtures, mutation tests

• Outcome: Pass
• Latest audit: Sep 30, 2026 · 11:33 AM UTC
• Version: 1.0.0

Overview

This skill is a local guardrail mutation-testing helper whose command execution, temporary file writes, and repo copying are disclosed and aligned with its purpose.

Install only if you want a local mutation-testing helper that can run your chosen test command. Treat it like running your project tests: use trusted repos and test commands, and start from a clean or sanitized environment if your shell contains sensitive tokens.

A.I.G

Vulnerability Patterns

• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally
• Unauthorized Access and Privilege Escalation: Obtains permissions beyond the task's legitimate needs

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Tool Misuse: Tool Parameter Abuse, Chaining Abuse, Unsafe Defaults
• Behavioral AST: exec() Call, eval() Call, Dynamic Import
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration

Findings (12)

Tool Parameter Abuse

High

• Category: Tool Misuse
• Confidence: 60% confidence

Finding

Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Content

Scanner excerpt · SKILL.md (reported line 204)May include surrounding context.

Under LC_ALL=C, BSD sed rejects the table ("transform strings are not the same length"), exits 1 and prints nothing: check its status.

text

**Secrets in argv**: `-H "Authorization: …"`, `-u user:pass`, `-passin pass:…` are readable by any local user in `ps -ww -o args=`. Pass them on stdin (`curl -K -`, `-passin env:VAR`). Scanner, fail-closed on zero files; the `[A]` classes keep it from matching its own source:

```bash
root="${1:-.}"

Env Variable Harvesting

High

• Category: Data Exfiltration
• Confidence: 92% confidence

Finding

The child process environment is built by copying os.environ wholesale, which can leak host secrets such as API tokens, cloud credentials, CI secrets, and proxy settings into attacker-influenced test processes. In this skill's context, the subprocess is explicitly used to run repository tests on hostile fixtures, so environment inheritance materially increases the blast radius and can expose sensitive data or alter behavior via environment-based hooks.

Content

Scanner excerpt · scripts/mutate.py (reported line 144)May include surrounding context.

python

def run(self, root):
        prefix = tempfile.mkdtemp(prefix="pyc-", dir=self.base)
        env = dict(os.environ, PYTHONDONTWRITEBYTECODE="1", PYTHONPYCACHEPREFIX=prefix,
                   MUTATE_ROOT=root)
        env["PYTEST_ADDOPTS"] = (env.get("PYTEST_ADDOPTS", "") + " -p no:cacheprovider").strip()
        purge_pycache(root)

Env Variable Harvesting

High

• Category: Data Exfiltration
• Confidence: 60% confidence

Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · scripts/mutate.py (reported line 296)May include surrounding context.

python

naive = os.path.join(tmp, "naive")
        shutil.copytree(repo, naive)
        shared = os.path.join(tmp, "shared-pyc")
        env = dict(os.environ, PYTHONPYCACHEPREFIX=shared)
        env.pop("PYTHONDONTWRITEBYTECODE", None)
        stamp, kills = int(time.time()), []
        for _, text in muts:

Undeclared Tool Scope

Medium

• Category: MCP Least Privilege
• Confidence: 95% confidence

Finding

The skill explicitly describes executing shell and Python, reading environment variables, and creating/deleting temporary files, but it does not declare any tool scope such as allowed tools or permissions. That creates a governance gap: an agent may grant broader capabilities than intended, making review, policy enforcement, and least-privilege controls harder. In this context, the skill is designed to run local commands and write files, so undeclared capability is materially relevant rather than theoretical.

Content

No source excerpt is available for this finding.

Sudo/Root Execution

Medium

• Category: Privilege Escalation
• Confidence: 80% confidence

Finding

Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Content

Scanner excerpt · SKILL.md (reported line 173)May include surrounding context.

bash

PROD_STATE="$HOME/.myagent/state"                  # captured once; tests may rewrite $HOME later
SANDBOX="$(mktemp -d)" || exit 1; chmod 700 "$SANDBOX"; trap 'rm -rf "$SANDBOX"' EXIT
export STATE_DIR="$SANDBOX/state" ALLOWLIST="$SANDBOX/allowlist.test"
for v in STATE_DIR ALLOWLIST; do                   # check BEFORE the first mkdir or write
  case "${!v}" in "$PROD_STATE"*) echo "STOP: $v points at production" >&2; exit 1 ;; esac

compile() call detected

Medium

• Category: Dangerous Code Execution
• Confidence: 65% confidence

Finding

compile() creates code objects from strings. When combined with exec()/eval(), it enables obfuscated code execution.

Content

Scanner excerpt · scripts/mutate.py (reported line 99)May include surrounding context.

python

if ext == ".py":
        try:
            with open(path, encoding="utf-8") as fh:
                compile(fh.read(), path, "exec")
            return True
        except SyntaxError:
            return False

subprocess module call

Medium

• Category: Dangerous Code Execution
• Confidence: 70% confidence

Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/mutate.py (reported line 104)May include surrounding context.

python

except SyntaxError:
            return False
    if ext in (".sh", ".bash"):
        return subprocess.run(["bash", "-n", path], capture_output=True).returncode == 0
    if ext in (".js", ".mjs", ".cjs") and shutil.which("node"):
        return subprocess.run(["node", "--check", path], capture_output=True).returncode == 0
    return True

subprocess module call

Medium

• Category: Dangerous Code Execution
• Confidence: 70% confidence

Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/mutate.py (reported line 106)May include surrounding context.

python

if ext in (".sh", ".bash"):
        return subprocess.run(["bash", "-n", path], capture_output=True).returncode == 0
    if ext in (".js", ".mjs", ".cjs") and shutil.which("node"):
        return subprocess.run(["node", "--check", path], capture_output=True).returncode == 0
    return True

subprocess module call

Medium

• Category: Dangerous Code Execution
• Confidence: 89% confidence

Finding

The tool executes a user-supplied --test command taken from skill input, splits it, and runs it as a subprocess with the inherited environment. Although it avoids shell=True, this still enables arbitrary command execution chosen by the caller, which is dangerous in an agent skill because the skill itself becomes a code-execution primitive against the host running the agent.

Content

Scanner excerpt · scripts/mutate.py (reported line 150)May include surrounding context.

python

purge_pycache(root)
        argv = [x.replace("{root}", root) for x in shlex.split(self.a.test)]
        try:
            p = subprocess.run(argv, cwd=root, env=env, capture_output=True, text=True,
                               timeout=self.a.timeout)
        except subprocess.TimeoutExpired:
            return None, "timeout after %ss" % self.a.timeout

subprocess module call

Medium

• Category: Dangerous Code Execution
• Confidence: 70% confidence

Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/mutate.py (reported line 271)May include surrounding context.

python

fails.append(name)

    def harness(repo, test, *extra):
        p = subprocess.run([py, me, "--repo", repo, "--file", "guard.py", "--test", test] + list(extra),
                           capture_output=True, text=True)
        return p.returncode, p.stdout + p.stderr

subprocess module call

Medium

• Category: Dangerous Code Execution
• Confidence: 70% confidence

Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/mutate.py (reported line 304)May include surrounding context.

python

with open(target, "w") as fh:
                fh.write(text)
            os.utime(target, (stamp, stamp))
            kills.append(subprocess.run([py, "test_partial.py"], cwd=naive, env=env,
                                        capture_output=True).returncode != 0)
        if all(kills):
            check("naive loop reproduces the stale .pyc false green (2/2 'killed')", True)

subprocess module call

Medium

• Category: Dangerous Code Execution
• Confidence: 70% confidence

Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/mutate.py (reported line 331)May include surrounding context.

python

fh.write("def allow(fields):\n    return True\n")
        rc, _ = harness(plain, t_part, "--gap", "0")
        check("no FAIL-CLOSED block -> exit 3", rc == 3, "rc=%s" % rc)
        p = subprocess.run([py, me, "--no-such-flag"], capture_output=True)
        check("usage error -> exit 3, not 2", p.returncode == 3, "rc=%s" % p.returncode)
        check("original guard.py byte-identical", sha(os.path.join(repo, "guard.py")) == guard_hash)
        check("no __pycache__ left in the repo", not os.path.exists(os.path.join(repo, "__pycache__")))

Static analysis

No suspicious patterns detected.