AI Intelligence Hub - Real-time Model Capability Tracking
v1.0.0Real-time AI model capability tracking via leaderboards (LMSYS Arena, HuggingFace, etc.) for intelligent compute routing and cost optimization
⭐ 0· 343·1 current·1 all-time
MIT-0
Download zip
LicenseMIT-0 · Free to use, modify, and redistribute. No attribution required.
Security Scan
OpenClaw
Suspicious
medium confidencePurpose & Capability
The README/SKILL.md claim 'real-time' pulls from LMSYS, BigCode, HuggingFace and 'No external dependencies'. The bundled scripts (scripts/run.py) currently implement mocked fetch functions and write local JSON files rather than performing actual network scraping/API queries. The code includes BENCHMARK_SOURCES with real-looking URLs (HuggingFace spaces) and imports urllib, but does not actually fetch those endpoints in the provided implementation. This is a clear capability mismatch: the skill advertises live data but ships a simulated/local-only implementation.
Instruction Scope
Runtime instructions center on running the included Python script to fetch, query, recommend, and write local benchmark data, and on integrating results into OpenClaw config or dashboards. The instructions do not direct the agent to read unrelated system files or exfiltrate data. Examples include sending alerts to external endpoints (Slack webhook) and invoking `openclaw config set`, but those are optional example workflows and are within the skill's stated purpose (integration/automation).
Install Mechanism
There is no install spec and no remote download/install step; the skill is instruction-only with bundled Python scripts. Nothing in the manifest writes or executes code fetched from external URLs during installation, which reduces installation-time risk.
Credentials
The skill declares no required environment variables or credentials. However, integration examples reference external webhook variables (e.g., SLACK_WEBHOOK_URL) and CLI commands that rely on an existing OpenClaw installation and its credentials. If you enable the roadmap features (OpenRouter/Anthropic price polling) or modify BENCHMARK_SOURCES to call external APIs, those will likely require API keys—none are declared now. Be aware future/modified versions could ask for unrelated secrets.
Persistence & Privilege
always:false and the skill does not auto-enable itself. Documentation and examples recommend scheduling runs with cron and programmatically changing OpenClaw config (`openclaw config set`). Those are reasonable for the skill's goal but create persistent changes (cron jobs, config updates) under your account if you follow the examples. The skill itself does not request elevated privileges or modify other skills.
What to consider before installing
This skill appears to be an early or local-only implementation: it promises real-time leaderboard scraping but the shipped code returns mocked benchmark and price data and does not call the listed external APIs. Before installing or scheduling it to run automatically: 1) Inspect scripts/run.py fully to confirm whether it will fetch external endpoints or require API keys (and never provide credentials unless you trust the source). 2) If you add cron jobs or use the example scripts, be aware they will regularly write logs and may call `openclaw config set` (so they can change agent config). 3) Only wire up external webhooks (Slack) or API keys you control and trust; the skill does not declare or validate those env vars. 4) If you need real-time external data, either update/verify the fetch implementations yourself or only run the skill in a safe environment until upstream adds proper API integrations and explicit credential handling. If you want higher assurance, ask the publisher for a version that performs actual API calls with documented credential requirements and for a reproducible audit of network behavior.Like a lobster shell, security has layers — review code before you run it.
aivk97epkensag46j4fyf0tyc1jhd822ek8benchmarksvk97epkensag46j4fyf0tyc1jhd822ek8bigcodevk97epkensag46j4fyf0tyc1jhd822ek8cost-optimizationvk97epkensag46j4fyf0tyc1jhd822ek8data-drivenvk97epkensag46j4fyf0tyc1jhd822ek8huggingfacevk97epkensag46j4fyf0tyc1jhd822ek8intelligencevk97epkensag46j4fyf0tyc1jhd822ek8latestvk97epkensag46j4fyf0tyc1jhd822ek8lmsysvk97epkensag46j4fyf0tyc1jhd822ek8modelsvk97epkensag46j4fyf0tyc1jhd822ek8performancevk97epkensag46j4fyf0tyc1jhd822ek8routingvk97epkensag46j4fyf0tyc1jhd822ek8
License
MIT-0
Free to use, modify, and redistribute. No attribution required.