Back to skill

Security audit

Skill Sandbox

Security checks for vulnerabilities and agentic risk

Overview

This skill is meant to test unsafe skills, but its safety claims are stronger than the protections its code actually enforces.

Do not treat this as a real sandbox or security boundary. Only use it inside a disposable VM or container with no real credentials, no sensitive files mounted, and network disabled unless you explicitly want network access during testing.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/sandbox.py:223
Finding
Ineffective Sandbox Allows Unrestricted Execution of Untrusted Code## Vulnerability Details **File Location**: `scripts/sandbox.py:223-285` **Vulnerability Type**: Ineffective security controls and unsafe execution of untrusted code **Risk Level**: High ### Vulnerable Code ```python def run_sandbox(script_path, monitor, timeout=60, fake_env=False, restricted=False): """Run a script in a subprocess-isolated sandbox. SECURITY: Uses subprocess isolation instead of exec() to prevent: - Frame traversal recovering real builtins - /proc/self/environ reads of host environment - gc.get_objects() report tampering - Raw socket exfiltration bypassing urllib patches - ctypes, mmap, os.open() filesystem bypasses The script runs as a separate Python process with: - Sanitized environment (no real credentials) - Restricted working directory (tmpdir) - Stdout/stderr captured for analysis - Timeout enforcement at the OS level """ import subprocess as _subprocess script_path = Path(script_path) if not script_path.exists(): print(f"ERROR: Script not found: {script_path}", file=sys.stderr) return False # Read script for static analysis before execution with open(script_path) as f: code = f.read() # Static analysis: log observations suspicious_patterns = [ ("__traceback__", "frame traversal attempt"), ("f_back", "frame traversal attempt"), ("f_globals", "frame traversal attempt"), ("/proc/self", "/proc filesystem access"), ("gc.get_objects", "garbage collector introspection"), ("ctypes", "ctypes FFI access"), ("socket.socket", "raw socket creation"), ("os.system", "os.system shell execution"), ("os.popen", "os.popen shell execution"), ("os.fork", "process forking"), ("os.exec", "process exec"), ("mmap", "memory-mapped file access"), ("importlib", "dynamic module import"), ] for pattern, description in suspicious_patterns: ...[truncated 3766 chars]
Remediation
## Remediation Suggestions - Do not execute untrusted code directly on the host under the caller’s account. - Use a disposable container or virtual machine with a dedicated unprivileged user. - Disable network access by default using an enforceable network namespace or equivalent platform mechanism. - Mount inspected Skill content read-only and expose only a dedicated temporary output directory as writable. - Do not mount the user’s home directory, credentials, SSH agent, cloud configuration, runtime sockets, or other host-sensitive resources. - Apply process, CPU, memory, file-size, and execution-time limits. - Use platform controls such as seccomp, AppArmor, SELinux, Capsicum, or sandbox-exec where appropriate. - Prevent privilege escalation by dropping capabilities, enabling `no_new_privileges`, and prohibiting privileged containers. - Collect network, filesystem, and process telemetry from outside the untrusted process rather than relying on Python monkey-patching. - Make `--restricted` fail closed if enforceable isolation cannot be established. - Implement `--monitor-network` using real external telemetry or remove the option. - Remove claims such as “true process isolation” and “run any skill safely” until they accurately describe implemented guarantees. - Clearly distinguish static warnings from runtime observations in the generated report. - Add integration tests proving that restricted mode blocks outbound sockets, host-file reads and writes, and child-process execution.
Vulnerability Patterns
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (8)

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding
This is a real security issue because the skill claims protective monitoring and isolation for untrusted skills, but the described implementation does not actually intercept or enforce many of those controls. That mismatch can cause users to run malicious skills under a false sense of safety, exposing real credentials, files, or network access while believing they are contained.

Credential Access

High
Category
Privilege Escalation
Content
op = "read" if "r" in mode else "write"
        monitor.log_fs(op, path, mode)
        if "w" in mode or "a" in mode:
            if any(s in path for s in ["/etc/", "/var/", "/home/", ".ssh", ".aws", ".env"]):
                monitor.warn(f"Write attempt to sensitive path: {path}")
        return original_open(file, mode, *args, **kwargs)
    return monitored_open
Confidence
60% confidence
Finding
Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Description-Behavior Mismatch

High
Confidence
99% confidence
Finding
The skill claims to monitor filesystem, network, environment, and subprocess behavior of the sandboxed script, but the actual execution path simply launches the target in a separate process with no hooks for those activities. This creates a dangerous false sense of safety: a user may trust the report while the untrusted code performs unobserved network access, file I/O, or child process execution.

Intent-Code Divergence

High
Confidence
99% confidence
Finding
The CLI exposes a --restricted mode and describes it as blocking network and subprocess activity, but the restricted parameter is never enforced during execution. Users may intentionally run hostile skills under the assumption that dangerous capabilities are disabled when they are not, enabling credential exfiltration, malware download, or arbitrary command execution from the child process.

Lp3

Medium
Category
MCP Least Privilege
Confidence
93% confidence
Finding
The skill advertises and invokes functionality that can execute untrusted code, read files, access environment variables, make network requests, and spawn subprocesses, but it declares no explicit tool scope or permissions boundary. In this context, missing scope is dangerous because users may assume the skill is constrained when it is actually orchestrating high-risk capabilities against attacker-controlled inputs.

Missing User Warnings

Medium
Confidence
95% confidence
Finding
The default mode explicitly runs untrusted skills with no restrictions, yet the overall skill framing emphasizes safety and sandboxing. In context, that is especially dangerous because this skill is meant for evaluating potentially malicious code, so a permissive default materially increases the chance that users will execute attacker-controlled logic on their real system.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
try:
        # Run as separate subprocess — true process isolation
        result = _subprocess.run(
            [sys.executable, str(script_path)],
            capture_output=True,
            text=True,
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Intent-Code Divergence

Medium
Confidence
95% confidence
Finding
The report command tells users to run with --fake-env --restricted as if that provides a safe monitored execution, but the advertised protections are incomplete or unenforced. This is a security-signaling flaw that can directly influence risky operator behavior by overstating the safety of testing untrusted skills.

Static analysis

No suspicious patterns detected.