ClawHub
Back to skill

Security audit

Karpathy Autoresearch

Security checks across malware telemetry and agentic risk

Overview

This is a disclosed self-optimization skill, but it can edit files, run shell evaluation commands, and change git history with broad triggers and limited safeguards.

Install only if you intentionally want an agent to modify files and repository history. Use a disposable clone or clean branch, name the exact mutable file, use only evaluator commands you wrote or trust, keep secrets and production configs out of the workdir, and review diffs before keeping or publishing results.

SkillSpector

By NVIDIA
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (10)

subprocess module call

Medium
Category
Dangerous Code Execution
Content
def run_eval(eval_cmd: str) -> float:
    """Run the evaluation command and parse SCORE from output."""
    try:
        result = subprocess.run(
            eval_cmd,
            shell=True,
            capture_output=True,
Confidence
99% confidence
Finding
`run_eval` executes a user-supplied string with `shell=True`, which permits arbitrary shell command execution in the host environment. In a skill intended to run evaluations repeatedly, this dramatically increases risk because any attacker-controlled or untrusted eval string can run destructive commands, exfiltrate data, or alter the repository.

Lp3

Medium
Category
MCP Least Privilege
Confidence
92% confidence
Finding
The skill instructs the agent to read and write project files, initialize git, commit changes, revert files, and run evals, which are privileged file and shell operations despite no declared permissions. This is dangerous because users and enforcement layers may trust the declared metadata while the actual behavior can modify repositories and execute arbitrary project-defined evaluation commands.

Tp4

High
Category
MCP Tool Poisoning
Confidence
88% confidence
Finding
The skill claims to optimize skills autonomously, but the instructions also introduce undeclared repository mutation and shell-driven eval execution that materially expand risk beyond the stated purpose. That mismatch can mislead users into approving a prompt-optimization workflow that in practice changes git state and may run arbitrary project code supplied by an eval script.

Vague Triggers

Medium
Confidence
94% confidence
Finding
The example trigger phrases are broad enough to match ordinary user requests like 'improve this skill' or 'run evals on', which can cause the agent to invoke an autonomous self-modification workflow unexpectedly. In this skill, that risk is amplified because invocation can lead to repeated experimentation and repository changes, making accidental activation materially impactful.

Missing User Warnings

Medium
Confidence
96% confidence
Finding
The README advertises automatic git commits and reverts as part of the loop but does not prominently warn that the skill will modify repository state. Because this is an autonomous optimization skill, users may invoke it without realizing it can create commit history, alter files repeatedly, and potentially interfere with ongoing work or CI workflows.

Vague Triggers

Medium
Confidence
84% confidence
Finding
The trigger phrases are broad enough to activate on routine requests like improving or benchmarking a skill, increasing the chance the agent invokes this workflow unintentionally. In context, accidental activation is meaningful because the workflow includes repeated file edits, git operations, and evaluation execution, which are state-changing actions.

Vague Triggers

Medium
Confidence
83% confidence
Finding
The trigger section defines vague activation conditions without hard boundaries on what targets, directories, or eval mechanisms are allowed. That ambiguity makes it easier for the skill to run in unsafe contexts or against unintended files, especially since it is designed to loop through mutations and keep changes automatically.

Missing User Warnings

Medium
Confidence
90% confidence
Finding
The skill omits prominent warnings that it will repeatedly modify files and alter git state through init, commit, and checkout operations. This is risky because users may treat it as a harmless analysis/optimization helper when it can create persistent repository history, revert changes, or overwrite work during iterative experiments.

Missing User Warnings

Medium
Confidence
94% confidence
Finding
The script accepts an eval command and executes it through the shell without a strong warning or restriction. In this skill context, users are encouraged to iteratively run arbitrary evaluations, so lack of disclosure and guardrails materially increases the chance of unsafe command execution and misuse.

Missing User Warnings

Medium
Confidence
87% confidence
Finding
The script silently initializes git, stages all files, and creates commits in the selected workdir. In an optimization loop, that can unexpectedly persist sensitive files or alter unrelated repositories, especially if the workdir is broad or user-specified without clear warnings.

VirusTotal

64/64 vendors flagged this skill as clean.

View on VirusTotal

Static analysis

No suspicious patterns detected.