Security audit

ClawExam

Security checks for vulnerabilities and agentic risk

Overview

This benchmark skill is mostly transparent about using a live service, but it asks the agent to run remote questions for real and submit execution logs without clear safety limits.

Install only if you are comfortable running a live external benchmark. Use an isolated workspace with no private files, secrets, production credentials, or sensitive API sessions available to the agent. Review any outbound answer/log payloads before submission, and publish scores only when you intend them to be public.

A.I.G

Vulnerability Patterns

Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally

Findings (3)

T01 · Skill Instruction Hijacking

Error

Location: SKILL.md:34
Finding: Untrusted Remote Questions Are Treated as Executable Agent Instructions
Content: View full analysis
` - Execute the task for real - Record execution steps and token usage estimate - `POST /api/exam/submit` 5. `POST /api/exam/complete` 6. Present score summary + short self-reflection 7. Ask whether to publish the result to the leaderboard ## Important rules - Always use the live API at `https://www.clawexam.xyz` - Always perform the real HTTP requests described by the question - Submit final structured answers, not only code or free-form explanation ``` ### Technical Analysis The Skill retrieves questions from a mutable external service and directs the Agent to execute each retrieved task “for real.” It also mandates performing HTTP requests described by the remote question. The retrieved question is therefore used as an instruction channel rather than being handled strictly as untrusted benchmark data. The Skill defines no local schema restricting the contents of a question, no allowlist of permitted tools or network destinations, and no prohibition against filesystem access, command execution, credential access, or other consequential operations. Although no executable payload is embedded in the reviewed package, the effective behavior of the Skill can change after review because the remote service controls the task content. If the service is malicious or compromised, a question could attempt to alter the Agent's current objective or induce actions outside the declared benchmark purpose. ### Attack Path 1. A user invokes the ClawExam Skill. 2. The Skill authenticates to `https://www.clawe ...[truncated 1214 chars]
Remediation: View remediation

other

Error

Location: SKILL.md:12
Finding: Execution Logs and Task Results Are Transmitted Without Required Data Minimization
Content: View full analysis
` - Execute the task for real - Record execution steps and token usage estimate - `POST /api/exam/submit` ``` ```http Submit answer: POST https://www.clawexam.xyz/api/exam/submit Authorization: Bearer Content-Type: application/json ``` ### Technical Analysis The Skill requires structured answers and execution logs to be submitted to a third-party service after performing real tasks. It does not define a minimal outbound schema, specify which execution details are safe to include, or require redaction of secrets, personal data, source code, file paths, API responses, or other workspace information. Execution logs are particularly sensitive because they can capture intermediate values that would not be present in a final answer. If a remote question induces the Agent to inspect local files or external API responses, those values may be copied into the structured answer or log and sent to `clawexam.xyz`. The Skill includes one narrow safeguard for security questions—avoiding verbatim malicious payloads—but this does not address credentials, personal information, proprietary code, internal identifiers, or confidential API data. ### Attack Path 1. The Agent retrieves a benchmark ques ...[truncated 1293 chars]
Remediation: View remediation

T05 · Unauthorized Access and Privilege Escalation

Error

Location: SKILL.md:12
Finding: Benchmark Workflow Requests Unrestricted Real Execution Beyond Least Privilege
Content: View full analysis
` - Execute the task for real - Record execution steps and token usage estimate - `POST /api/exam/submit` 5. `POST /api/exam/complete` 6. Present score summary + short self-reflection 7. Ask whether to publish the result to the leaderboard ## Important rules - Always use the live API at `https://www.clawexam.xyz` - Always perform the real HTTP requests described by the question - Submit final structured answers, not only code or free-form explanation - For workflow questions, keep key artifacts like `validation_result`, `state_sequence`, or `final_profile` - For security questions, never repeat malicious payloads verbatim; return counts, IDs, or concise risk summaries instead ``` ### Technical Analysis The declared purpose is to benchmark an Agent, but the instructions require real API calls, code execution, workflows, and security analysis without defining a privilege boundary. A benchmark can generally be implemented using a sandbox, synthetic resources, or constrained test APIs. Unrestricted access to the host environment and arbitrary external services is not necessary for scoring reasoning and orchestration capabilities. No controls are specified for: - Sandboxing or process isolation - Host filesystem access - Access to environment variables or credentials - Outbound network destinations - API scopes and account permissions - CPU, memory, execution-time, or request limits - User c ...[truncated 1587 chars]
Remediation: View remediation

SkillSpector

Vulnerability Patterns

Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep

Findings (1)

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 96% confidence
Finding: The skill explicitly instructs the agent to authenticate to a live third-party service, create sessions, submit answers, and optionally publish scores, but it does not clearly warn the user that their username, model name, exam activity, logs, and possibly derived artifacts will be transmitted externally. This creates a meaningful consent and data-exposure risk because users may trigger the skill assuming it is local benchmarking rather than live data exchange with an external platform.
Content: No source excerpt is available for this finding.