Ai Autotester

• Outcome: Review
• Latest audit: Sep 11, 2026 · 9:35 AM UTC
• Version: 1.0.0

Overview

This automated tester is purpose-aligned, but it needs Review because it can modify arbitrary accessible directories, install dependencies, and execute project code without clear containment or user approval.

Only use this skill on trusted projects inside a disposable, isolated environment with no sensitive credentials available. Expect it to modify the target project, install packages, and run code from that project; review requirements.txt first and avoid pointing it at arbitrary filesystem locations.

A.I.G

Vulnerability Patterns

• Unauthorized Access and Privilege Escalation: Obtains permissions beyond the task's legitimate needs
• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (2)

T05 · Unauthorized Access and Privilege Escalation

Error

• Location: run.py:5

Finding

Unrestricted Target Path Allows Unauthorized Filesystem Modification

Content

View full analysis

Vulnerability Details

File Location: run.py, lines 5-9
Vulnerability Type: Unrestricted path resolution and unauthorized file modification
Risk Level: High

Vulnerable Code

python

ws=Path("/home/jason/.openclaw/workspace");t=(ws/a.task).resolve() if not Path(a.task).is_absolute() else Path(a.task).resolve()
t.exists() or (print(json.dumps({"skill":"AI_AutoTester","status":"error","error":f"Target path not found: {t}"},ensure_ascii=False,indent=2)),sys.exit(1))
(td:=t/"tests").mkdir(parents=True,exist_ok=True)
(tf:=td/"test_smoke.py").exists() or tf.write_text('from app.main import app\nfrom fastapi.testclient import TestClient\n\ndef test_root():\n c=TestClient(app)\n r=c.get("/")\n assert r.status_code==200\n',encoding="utf-8")
(t/"requirements-test.txt").write_text("pytest==8.3.2\nhttpx==0.27.2\n",encoding="utf-8")

Technical Analysis

The --task argument is treated as a filesystem path. Absolute paths are accepted directly, while relative paths are resolved against /home/jason/.openclaw/workspace. After resolution, the program does not verify that the resulting path remains inside the intended workspace.

Consequently, a relative path containing traversal components such as ../../target, or an explicitly supplied absolute path, can select any existing directory accessible to the process. The program then:

• Creates a tests directory.
• Creates tests/test_smoke.py if it does not already exist.
• Unconditionally overwrites requirements-test.txt.

Calling Path.resolve() canonicalizes a path but does not enforce a security boundary. The absence of an allowlist or containment check violates least-privilege principles and permits modifications beyond the documented testing workspace.

The --constraints option does not mitigate this issue because it is parsed but never consulted before filesystem changes occur.

Attack Path

1. An attacker or untrusted caller invokes the ...[truncated 1119 chars]

Remediation

View remediation

Remediation Suggestions

• Reject absolute values for --task unless an explicitly authorized administrative mode requires them.

• Resolve both the workspace and requested target, then enforce containment:

  python

  workspace = Path("/home/jason/.openclaw/workspace").resolve()
  target = (workspace / args.task).resolve()
  
  if not target.is_relative_to(workspace):
      raise ValueError("Target must remain within the configured workspace")
  

• On Python versions without Path.is_relative_to(), use a safe containment comparison based on relative_to(), handling ValueError.

• Consider restricting targets to an allowlist of registered project directories.

• Verify that the target is a directory rather than merely checking that it exists.

• Do not overwrite requirements-test.txt unconditionally. Fail safely if it exists, create a uniquely named generated file, or require explicit confirmation.

• Require confirmation before modifying project files and clearly report every planned change.

• Apply defenses against symlink-based boundary bypasses and revalidate paths immediately before writing where concurrent attackers are in scope.

• Honor the supplied constraints before performing any filesystem mutation.

T08 · Insecure Dependencies

Error

• Location: run.py:10

Finding

Untrusted Project Dependencies Are Installed Without Isolation or Verification

Content

View full analysis

Vulnerability Details

File Location: run.py, lines 10-12
Vulnerability Type: Unsafe dependency installation from a user-selected project
Risk Level: High

Vulnerable Code

python

(r1:=t/"requirements.txt").exists() and subprocess.run(["python","-m","pip","install","-q","-r",str(r1)],cwd=str(t),check=False)
(t/"requirements-test.txt").write_text("pytest==8.3.2\nhttpx==0.27.2\n",encoding="utf-8")
subprocess.run(["python","-m","pip","install","-q","-r",str(t/"requirements-test.txt")],cwd=str(t),check=False)

Technical Analysis

The script automatically installs every dependency declared in the selected target's requirements.txt. The target is user-controlled and, due to the unrestricted path handling, may be any accessible project directory.

No controls are applied to the dependency specification. In particular, the implementation does not:

• Require approval before installation.
• Restrict package indexes or direct URL sources.
• Require cryptographic hashes.
• Validate package names or versions against an allowlist.
• Disable source distributions or untrusted build backends.
• Use a disposable virtual environment or sandbox.
• Separate installation privileges from the agent's broader privileges.

A requirements file can refer to attacker-controlled package sources, local projects, version-control repositories, or malicious source distributions. Installing a source distribution may execute attacker-controlled build backend code during package preparation or wheel construction.

Installation is performed through the active python interpreter, so packages can alter or compromise the environment used by the skill and subsequent Python tasks. The use of an argument array avoids shell injection, but it does not make the dependency contents trustworthy.

The program also ignores installation failures through check=False and continues to testing. This can hide partial or failed environmen ...[truncated 1818 chars]

Remediation

View remediation

Remediation Suggestions

• Never install target-controlled dependencies directly into the skill's active interpreter environment.
• Execute dependency installation and tests inside a disposable container, virtual machine, or tightly sandboxed virtual environment.
• Use a fresh environment for each target and delete it after completion.
• Require explicit user confirmation after displaying the complete dependency plan and package sources.
• Permit packages only from approved indexes and reject unexpected direct URLs, VCS references, local paths, and editable installations.
• Use a lock file with exact versions and cryptographic hashes, such as pip's --require-hashes.
• Prefer prebuilt, verified wheels and consider rejecting source distributions with --only-binary=:all: where compatible.
• Run installation and testing with minimal filesystem permissions, no unnecessary credentials, and restricted network access.
• Check subprocess return codes and terminate safely when installation fails instead of continuing in a partially modified environment.
• Maintain dependency scanning and provenance checks for approved packages.
• Keep test-runner dependencies separate from target dependencies and avoid overwriting project dependency files.

SkillSpector

Vulnerability Patterns

• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• Behavioral AST: exec() Call, eval() Call, Dynamic Import
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration

Findings (8)

Context-Inappropriate Capability

High

• Category: Not specified by scanner
• Confidence: 98% confidence

Finding

The skill's core behavior is to install dependencies and execute tests against an arbitrary resolved path, which effectively grants code execution on untrusted project contents. Because the task parameter can point to any accessible directory and the code performs dynamic execution with minimal validation, the skill is especially dangerous in an agent context.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium

• Category: Not specified by scanner
• Confidence: 90% confidence

Finding

The skill creates a tests directory and injects a smoke test into the target project, modifying user-controlled code without warning. This can corrupt repositories, interfere with existing test suites, and create an avenue for unexpected behavior when combined with subsequent execution.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The skill silently creates test and dependency-related files in the target project without notifying the user. In an automation context, unannounced writes are dangerous because they can alter repository state, break builds, or hide malicious persistence among normal-looking test artifacts.

Content

No source excerpt is available for this finding.

subprocess module call

Medium

• Category: Dangerous Code Execution
• Confidence: 96% confidence

Finding

This command installs dependencies from the target project's requirements.txt, which is fully controlled by the selected repository. Installing arbitrary packages can trigger execution of malicious setup/install hooks or pull attacker-chosen code, making this an explicit arbitrary code execution and supply-chain vulnerability.

Content

Scanner excerpt · run.py (reported line 10)May include surrounding context.

python

t.exists() or (print(json.dumps({"skill":"AI_AutoTester","status":"error","error":f"Target path not found: {t}"},ensure_ascii=False,indent=2)),sys.exit(1))
(td:=t/"tests").mkdir(parents=True,exist_ok=True)
(tf:=td/"test_smoke.py").exists() or tf.write_text('from app.main import app\nfrom fastapi.testclient import TestClient\n\ndef test_root():\n c=TestClient(app)\n r=c.get("/")\n assert r.status_code==200\n',encoding="utf-8")
(r1:=t/"requirements.txt").exists() and subprocess.run(["python","-m","pip","install","-q","-r",str(r1)],cwd=str(t),check=False)
(t/"requirements-test.txt").write_text("pytest==8.3.2\nhttpx==0.27.2\n",encoding="utf-8")
subprocess.run(["python","-m","pip","install","-q","-r",str(t/"requirements-test.txt")],cwd=str(t),check=False)
r=subprocess.run(["python","-m","pytest","-q"],cwd=str(t),capture_output=True,text=True)

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 96% confidence

Finding

The skill installs packages via pip without any warning or consent, exposing the environment to untrusted dependency execution and system changes. In this context, package installation is especially risky because it is driven by repository contents and occurs before any trust decision is made.

Content

No source excerpt is available for this finding.

subprocess module call

Medium

• Category: Dangerous Code Execution
• Confidence: 89% confidence

Finding

This subprocess call installs packages into an arbitrary user-selected project directory. Even though shell injection is not present, running pip against project-controlled dependency files can execute untrusted package installation code and alters the environment without confirmation, creating a real code-execution and supply-chain risk.

Content

Scanner excerpt · run.py (reported line 12)May include surrounding context.

python

(tf:=td/"test_smoke.py").exists() or tf.write_text('from app.main import app\nfrom fastapi.testclient import TestClient\n\ndef test_root():\n c=TestClient(app)\n r=c.get("/")\n assert r.status_code==200\n',encoding="utf-8")
(r1:=t/"requirements.txt").exists() and subprocess.run(["python","-m","pip","install","-q","-r",str(r1)],cwd=str(t),check=False)
(t/"requirements-test.txt").write_text("pytest==8.3.2\nhttpx==0.27.2\n",encoding="utf-8")
subprocess.run(["python","-m","pip","install","-q","-r",str(t/"requirements-test.txt")],cwd=str(t),check=False)
r=subprocess.run(["python","-m","pytest","-q"],cwd=str(t),capture_output=True,text=True)
print(json.dumps({"skill":"AI_AutoTester","status":"ok" if r.returncode==0 else "failed","task":a.task,"target":str(t),"returncode":r.returncode,"stdout":r.stdout[-4000:],"stderr":r.stderr[-2000:],"timestamp":datetime.now(UTC).isoformat().replace("+00:00","Z")},ensure_ascii=False,indent=2))
sys.exit(0 if r.returncode==0 else 2)

subprocess module call

Medium

• Category: Dangerous Code Execution
• Confidence: 95% confidence

Finding

Running pytest in an arbitrary target directory executes project code, test hooks, plugins, and import-time side effects under the agent's privileges. In this skill, the path is attacker-controllable via --task, so the subprocess becomes a direct arbitrary code execution mechanism rather than a simple test runner.

Content

Scanner excerpt · run.py (reported line 13)May include surrounding context.

python

(r1:=t/"requirements.txt").exists() and subprocess.run(["python","-m","pip","install","-q","-r",str(r1)],cwd=str(t),check=False)
(t/"requirements-test.txt").write_text("pytest==8.3.2\nhttpx==0.27.2\n",encoding="utf-8")
subprocess.run(["python","-m","pip","install","-q","-r",str(t/"requirements-test.txt")],cwd=str(t),check=False)
r=subprocess.run(["python","-m","pytest","-q"],cwd=str(t),capture_output=True,text=True)
print(json.dumps({"skill":"AI_AutoTester","status":"ok" if r.returncode==0 else "failed","task":a.task,"target":str(t),"returncode":r.returncode,"stdout":r.stdout[-4000:],"stderr":r.stderr[-2000:],"timestamp":datetime.now(UTC).isoformat().replace("+00:00","Z")},ensure_ascii=False,indent=2))
sys.exit(0 if r.returncode==0 else 2)

Natural-Language Policy Violations

Low

• Category: Not specified by scanner
• Confidence: 91% confidence

Finding

The file mixes English headings with Chinese content for the purpose and notes, which effectively constrains core skill information to a specific language. Under the policy, forcing a language without user opt-in or a justified locale constraint is a natural-language policy violation.

Content

No source excerpt is available for this finding.