SOTA AI Model Tracker
WarnAudited by ClawScan on May 10, 2026.
Overview
The model-tracking purpose is coherent, but the package includes unrelated background-agent orchestration instructions and recommends persistent daily updates to agent instruction files.
The SOTA tracker’s core data lookup purpose appears legitimate, but install it carefully: remove or ignore the unrelated Cyrus automation instructions, do not enable daily edits to Claude/agents files unless you want persistent behavior changes, and verify the GitHub source before running dependency installs or timers.
Findings (4)
Artifact-based informational review of SKILL.md, metadata, install specs, static scan signals, and capability signals. ClawScan does not execute the skill or run runtime probes.
If an agent follows these instructions, it may create or delegate background work through a separate system rather than only answering model-tracking questions.
This file is packaged with the skill and instructs an agent to hand work to an external/background orchestration system, which is unrelated to SOTA model tracking and could cause autonomous activity outside the user’s intended task.
Delegate to Cyrus - execution happens automatically
Remove maintainer-specific Cyrus/Linear orchestration instructions from the distributed skill, or clearly isolate them from user-facing agent instructions and require explicit user approval before any delegation.
Future Claude sessions could rely on automatically refreshed model recommendations or instructions without the user noticing the change each time.
The skill recommends writing externally updated SOTA data into a persistent Claude instruction/context file, so scraped or compromised data could influence future agent sessions.
This embeds a compact SOTA summary directly in your `~/.claude/CLAUDE.md` file.
Only enable persistent agent-file updates if you trust the data source; prefer a clearly marked block, backups, easy rollback, and explicit opt-in before enabling daily timers.
Users may run code or install dependencies from a repository/branch that is not pinned or verified by the registry metadata.
The registry metadata does not declare a source or homepage, while the documentation points users toward manual GitHub clone/install workflows. That is not malicious by itself, but it leaves provenance verification to the user.
Source: unknown; Homepage: none
Verify the GitHub repository owner, review the code and dependency files, and preferably pin to a trusted commit before running setup commands.
Other devices on the network may be able to query the tracker API if firewall rules permit it.
Binding the REST API to 0.0.0.0 is a disclosed, user-directed server command, but it can expose the API beyond localhost if the machine or network allows inbound access.
uvicorn rest_api:app --host 0.0.0.0 --port 8000
Use 127.0.0.1 for local-only use, and add authentication or firewall restrictions before exposing the API on a network.