ClawHub
Back to skill

Security audit

SOTA Tracker (Claw)

Security checks across malware telemetry and agentic risk

Overview

This appears to be a legitimate SOTA model tracker, but it needs Review because it encourages persistent automated edits to local agent instruction and MCP config files without enough safeguards.

Install only if you are comfortable with a tool that can persistently change agent instruction files. Run the update scripts manually first, back up agents.md, CLAUDE.md, and .mcp.json, review diffs before enabling cron/systemd timers, and pin or upgrade dependencies before exposing the REST or MCP server beyond localhost.

SkillSpector

By NVIDIA
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Findings (28)

Intent-Code Divergence

Medium
Confidence
90% confidence
Finding
The module docstring describes the script as initializing the database with schema and seed data, but omits the fact that execution also unconditionally deletes any existing database file. That mismatch can cause operators or automation to run the script expecting a non-destructive setup step and instead lose all stored data.

Missing User Warnings

Medium
Confidence
89% confidence
Finding
The README instructs users to set up an automated process that updates `~/.claude/CLAUDE.md`, which is a local agent context file, but it does not clearly warn that this file will be modified on a recurring basis. In an agent-skill context, silently changing persistent prompt/context files can alter future agent behavior and trust boundaries, especially if the upstream data or script later changes.

Missing User Warnings

Medium
Confidence
95% confidence
Finding
The `cat > .mcp.json << 'EOF'` pattern will overwrite any existing `.mcp.json` configuration without prompting, and the README does not warn users about that destructive behavior. In this context, replacing MCP configuration can disable other servers, change tool availability, or introduce unintended execution paths for future agent sessions.

Missing User Warnings

Medium
Confidence
88% confidence
Finding
The README tells users to run `update_agents_md.py`, including via cron/systemd automation, to update `agents.md` files without clearly stating that local agent instruction files will be modified persistently. Because `agents.md` can influence agent behavior, unreviewed automated edits increase the risk of prompt/context tampering or unintended operational changes over time.

Missing User Warnings

Medium
Confidence
91% confidence
Finding
The documentation instructs users to run a script that updates `~/.claude/CLAUDE.md`, a user-scoped agent configuration/context file, but it does not prominently warn that this modifies local persistent files. In an agent-skill context, silently encouraging writes into agent prompt/config surfaces is risky because it can alter future agent behavior and create lasting side effects beyond the current session.

Missing User Warnings

Medium
Confidence
94% confidence
Finding
This section instructs users to schedule recurring execution of `update_agents_md.py`, which will repeatedly modify `agents.md` over time, but it does not clearly warn about persistent automated file changes. In the context of agent tooling, unattended updates to agent instruction files are especially sensitive because they can continuously change how downstream agents behave without fresh user review.

Vague Triggers

Medium
Confidence
94% confidence
Finding
The example skill description and protocol are framed broadly enough that the skill could auto-activate during ordinary conversations about AI models, not only when authoritative SOTA validation is actually needed. That can bias responses toward mandatory tool usage and embedded preferences, increasing the chance of irrelevant or overreaching behavior in benign user interactions.

Vague Triggers

Medium
Confidence
90% confidence
Finding
The document promotes 'automatic activation when discussing AI models' without defining boundaries, which makes invocation ambiguous and prone to over-triggering. In a skill system, unclear activation conditions are risky because they can cause hidden policy injection into unrelated or low-risk conversations.

Natural-Language Policy Violations

High
Confidence
97% confidence
Finding
The skill example hard-codes a preference for 'uncensored models' and repeats that preference in both hardware context and quick-reference guidance, even when the user has not asked for that. This can steer the assistant toward bypassing safety-aligned defaults and recommending less restricted models, creating a meaningful risk of harmful or policy-evading assistance.

Natural-Language Policy Violations

Medium
Confidence
94% confidence
Finding
The dataset includes the model name "AutismMix SDXL," which contains ableist language presented without context, quotation, or moderation metadata. While this is not an exploit-style software vulnerability, it is a genuine safety and content-quality issue because downstream systems may surface, normalize, or recommend harmful terminology to users.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
The dataset contains a model name with a stigmatizing disability-related term ('AutismMix SDXL') presented without contextual explanation, normalization, or moderation metadata. While this is not a code-execution issue, it is a genuine content-safety and quality vulnerability because downstream systems may surface, normalize, or amplify harmful terminology in user-facing outputs.

Missing User Warnings

Medium
Confidence
96% confidence
Finding
The script deletes the SQLite database file whenever it already exists, with no confirmation, backup, or environment guard. If run accidentally, via automation, or in the wrong directory context, it causes immediate and irreversible data loss for that application state.

Missing User Warnings

Medium
Confidence
86% confidence
Finding
The script replaces the user's configured agents.md automatically and moves any existing file to a predictable backup path without confirmation or atomic safety checks. In an agent/automation context, this can silently destroy user-authored configuration, overwrite important instructions, or be abused to redirect writes to unintended files if the target path is attacker-controlled or symlinked.

Session Persistence

Medium
Category
Rogue Agent
Content
cp scripts/update_sota_claude_md.py ~/scripts/

# Enable systemd timer (runs at 6 AM daily)
systemctl --user enable --now sota-update.timer

# Or run manually
python ~/scripts/update_sota_claude_md.py --update
Confidence
84% confidence
Finding
systemctl --user enable

Session Persistence

Medium
Category
Rogue Agent
Content
cp scripts/update_sota_claude_md.py ~/scripts/

# Enable systemd timer (runs at 6 AM daily)
systemctl --user enable --now sota-update.timer

# Or run manually
python ~/scripts/update_sota_claude_md.py --update
Confidence
88% confidence
Finding
systemctl --user enable

Unpinned Dependencies

Low
Category
Supply Chain
Content
fastmcp>=2.0,<3.0
aiohttp>=3.9
huggingface_hub>=0.20
python-dotenv>=1.0
playwright>=1.40
Confidence
95% confidence
Finding
aiohttp>=3.9

Unpinned Dependencies

Low
Category
Supply Chain
Content
fastmcp>=2.0,<3.0
aiohttp>=3.9
huggingface_hub>=0.20
python-dotenv>=1.0
playwright>=1.40
fastapi>=0.100.0
Confidence
93% confidence
Finding
huggingface_hub>=0.20

Unpinned Dependencies

Low
Category
Supply Chain
Content
fastmcp>=2.0,<3.0
aiohttp>=3.9
huggingface_hub>=0.20
python-dotenv>=1.0
playwright>=1.40
fastapi>=0.100.0
uvicorn>=0.23.0
Confidence
93% confidence
Finding
python-dotenv>=1.0

Unpinned Dependencies

Low
Category
Supply Chain
Content
aiohttp>=3.9
huggingface_hub>=0.20
python-dotenv>=1.0
playwright>=1.40
fastapi>=0.100.0
uvicorn>=0.23.0
slowapi>=0.1.9
Confidence
94% confidence
Finding
playwright>=1.40

Unpinned Dependencies

Low
Category
Supply Chain
Content
huggingface_hub>=0.20
python-dotenv>=1.0
playwright>=1.40
fastapi>=0.100.0
uvicorn>=0.23.0
slowapi>=0.1.9
Confidence
95% confidence
Finding
fastapi>=0.100.0

Unpinned Dependencies

Low
Category
Supply Chain
Content
python-dotenv>=1.0
playwright>=1.40
fastapi>=0.100.0
uvicorn>=0.23.0
slowapi>=0.1.9
Confidence
94% confidence
Finding
uvicorn>=0.23.0

Unpinned Dependencies

Low
Category
Supply Chain
Content
playwright>=1.40
fastapi>=0.100.0
uvicorn>=0.23.0
slowapi>=0.1.9
Confidence
93% confidence
Finding
slowapi>=0.1.9

Known Vulnerable Dependency: fastmcp — 10 advisory(ies): CVE-2025-69196 (FastMCP OAuth Proxy token reuse across MCP servers); GHSA-c2jp-c369-7pvx (FastMCP Auth Integration Allows for Confused Deputy Account Takeover); CVE-2025-64340 (FastMCP has a Command Injection vulnerability - Gemini CLI) +7 more

Critical
Category
Supply Chain
Confidence
95% confidence
Finding
fastmcp

Known Vulnerable Dependency: aiohttp — 10 advisory(ies): CVE-2024-52303 (aiohttp has a memory leak when middleware is enabled when requesting a resource ); CVE-2026-54279 (aiohttp: Host-Only Cookies Become Domain Cookies After CookieJar Persistence); CVE-2026-34514 (AIOHTTP has CRLF injection through multipart part content type header constructi) +7 more

High
Category
Supply Chain
Confidence
84% confidence
Finding
aiohttp

Known Vulnerable Dependency: fastmcp==2.0 — 10 advisory(ies): CVE-2025-69196 (FastMCP OAuth Proxy token reuse across MCP servers); GHSA-c2jp-c369-7pvx (FastMCP Auth Integration Allows for Confused Deputy Account Takeover); CVE-2025-64340 (FastMCP has a Command Injection vulnerability - Gemini CLI) +7 more

Critical
Category
Supply Chain
Confidence
98% confidence
Finding
fastmcp==2.0

VirusTotal

63/63 vendors flagged this skill as clean.

View on VirusTotal

Static analysis

No suspicious patterns detected.