PinchBench

• Outcome: Review
• Latest audit: May 28, 2026, 12:54 PM
• Version:
  1.0.0

Overview

PinchBench appears to be a real benchmark skill, but running it gives benchmark task files and OpenClaw agents enough authority to execute code, change local agent state, and upload detailed results.

Install or run this only in an isolated OpenClaw profile with disposable workspaces and test accounts. Review task markdown before running, avoid custom tasks unless you trust their automated checks, use --no-upload unless you intend to publish leaderboard data, and do not run it where OpenClaw has access to real email, calendars, credentials, or sensitive files.

SkillSpector

By NVIDIA

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• Behavioral AST: exec() Call, eval() Call, Dynamic Import
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access

Findings (15)

exec() call detected

High

• Category: Dangerous Code Execution

Content

)

    namespace: Dict[str, Any] = {}
    exec(grading_code, namespace)
    grade_func = namespace.get("grade")
    if not callable(grade_func):
        return GradeResult(

• Confidence: 99% confidence
• Finding: exec(grading_code, namespace)

Context-Inappropriate Capability

High

• Confidence: 99% confidence
• Finding: Automated grading accepts embedded Python from task.automated_checks and executes it directly, which is far broader than needed for benchmark scoring and creates a direct arbitrary-code-execution path. In a benchmarking skill, tasks may be externally supplied or updated over time, so compromising task content would compromise the grading environment.

Context-Inappropriate Capability

Medium

• Confidence: 91% confidence
• Finding: The upload payload includes extensive host metadata such as OS version, architecture, CPU model, memory characteristics, Python details, and a hostname-derived identifier. For a benchmark leaderboard upload, this exceeds minimally necessary data and increases device fingerprinting and privacy risk if the server is compromised, repurposed, or users are unaware of collection.

Context-Inappropriate Capability

Medium

• Confidence: 88% confidence
• Finding: The upload path runs local inspection commands to gather OpenClaw and platform state before sending results, which broadens the helper's behavior beyond merely uploading benchmark scores. In benchmark tooling, unexpected local probing is dangerous because it can leak additional environment details and violate user expectations about what the tool will access.

Missing User Warnings

Medium

• Confidence: 90% confidence
• Finding: The README states that benchmark results are auto-uploaded to a public leaderboard after registration, but it does not clearly warn users what data may be transmitted or that submissions become public. In a benchmarking skill that exercises real-world tasks involving email, files, research, and scheduling, results or metadata could unintentionally reveal sensitive prompts, outputs, environment details, or organizational information.

Missing User Warnings

Medium

• Confidence: 90% confidence
• Finding: The skill states that results are collected on a public leaderboard, but it does not prominently warn users what information may be published or transmitted alongside scores. In benchmarking contexts, result bundles can include prompts, outputs, model identifiers, timestamps, and host fingerprints, which can expose proprietary workflows or identifying metadata if uploaded by default.

Missing User Warnings

Medium

• Confidence: 93% confidence
• Finding: The script uploads benchmark results to a remote server by default unless `--no-upload` is specified, but it does not present a clear user-facing consent prompt or warning at the point of transmission. Because the results JSON includes task metadata, workspace paths, transcript length, and usage data, users may unintentionally transmit benchmark artifacts or potentially sensitive execution-derived information off-host.

Missing User Warnings

Medium

• Confidence: 83% confidence
• Finding: cleanup_agent_sessions deletes all matching session transcripts and the session index for an agent under ~/.openclaw without any confirmation, scope restriction beyond agent_id, backup, or retention policy. In a benchmarking context this can silently destroy logs needed for auditing, debugging, incident review, or evidence preservation, especially because it runs automatically before executions.

Missing User Warnings

Low

• Confidence: 84% confidence
• Finding: prepare_task_workspace writes files to paths derived from task.workspace_files into the resolved workspace without validating that the resulting destination stays inside that workspace. If task data is malicious or compromised, path traversal via file_spec['path'] or file_spec['dest'] could overwrite arbitrary files accessible to the process, made more concerning because the workspace may come from external OpenClaw configuration rather than a fresh temp directory.

Missing User Warnings

Medium

• Confidence: 88% confidence
• Finding: The LLM judge path summarizes transcript content and sends it to an external model endpoint via run_openclaw_prompt, potentially disclosing user prompts, tool outputs, and other sensitive task data. In a benchmark system that may process realistic email, calendar, research, or coding workflows, transcripts can easily contain confidential or proprietary information, making undisclosed third-party transmission a real privacy and data-governance risk.

Missing User Warnings

Medium

• Confidence: 87% confidence
• Finding: The main upload routine performs a network transmission of benchmark contents plus metadata without any visible user-facing disclosure or confirmation in this code path. In a benchmarking skill, silent exfiltration to a remote leaderboard is risky because task notes, frontmatter, timing, usage, and system metadata may contain sensitive operational details.

Missing User Warnings

Medium

• Confidence: 90% confidence
• Finding: The code stores the API token in a local JSON config file without setting restrictive file permissions or using an OS credential store. If the file is readable by other local users, backup systems, or malware, the token can be stolen and used to submit or manipulate leaderboard data under the victim's identity.

Missing User Warnings

Medium

• Confidence: 94% confidence
• Finding: The task explicitly instructs the agent to generate and potentially run code that makes an HTTP request to a workspace-provided endpoint, but it provides no guardrails about network safety, data handling, or approval requirements. In an agent benchmark context, this can normalize exfiltration-capable behavior and cause transmission of environment-derived URLs or future sensitive request metadata to external services.

Missing User Warnings

High

• Confidence: 97% confidence
• Finding: The task explicitly instructs the agent to write sensitive personal data and a secret code phrase into a persistent workspace file for later retrieval. Even in a benchmark context, this normalizes insecure handling of secrets, creates unnecessary retention of sensitive data, and can lead to later disclosure across sessions or to other tools, logs, or evaluators with workspace access.

Ssd 3

Medium

• Confidence: 95% confidence
• Finding: The benchmark requires the agent not only to store but also to later retrieve and disclose personal details and a secret phrase across sessions. This demonstrates cross-session persistence of sensitive data without consent, access controls, or data minimization, increasing the chance of unintended exposure if another session, user, or process can read the file.

VirusTotal

64/64 vendors flagged this skill as clean.