Benchmarked Free Ride

Pick the best free OpenRouter models using live benchmark CI results. Use when: user wants performance-ranked free model recommendations, needs a model that...

Security Scan

The skill's code, docs, and runtime instructions are consistent with its stated purpose: fetching a public leaderboard from GitHub Pages and updating the user's OpenClaw model config (primary + fallbacks) using only Python stdlib.

Purpose & Capability

The name/description match the requested artifacts: python3-only script that fetches a public leaderboard and writes model primary/fallback keys into ~/.openclaw/openclaw.json. There are no unexpected credential requests, unrelated binaries, or extra system paths in the manifest. Development/test files reference CI/integration tooling, but those are clearly ancillary (not required for normal operation).

Instruction Scope

SKILL.md instructs only to fetch leaderboard.json from sequrity-ai.github.io and run local CLI commands (auto/list/switch/status/fallbacks/refresh). The skill intentionally writes only the declared config keys. One noteworthy point: multiple test files and README mention Daytona and OPENROUTER_API_KEY for integration tests — those are for CI/testing and are not required by the runtime instructions or the skill itself. No instructions ask the agent to read unrelated system files or exfiltrate arbitrary data.

Install Mechanism

No install spec and the implementation uses only Python stdlib (urllib, json). The package contains source files and a standard setup.py, but there are no downloads from obscure URLs or extracted archives. This is low-risk from an install standpoint.

Credentials

The skill declares no required environment variables (correct for a public GitHub Pages data fetch). The presence of DAYTONA_API_KEY and OPENROUTER_API_KEY in the test files is expected for running integration tests and is not a runtime dependency of the skill; however users should be aware those tests will require secrets if they choose to run them. Overall, environment/credential requests are proportionate to the skill's purpose.

Persistence & Privilege

The skill does write to the user's ~/.openclaw/openclaw.json (only the documented agents.defaults.model.primary and fallbacks keys). always:false and no other elevated privileges are requested. Modifying the user's agent config is the declared purpose, so the level of persistence/privilege is appropriate; users should note this will change which model their agent uses.

Assessment

This skill is internally consistent with its stated goal, but take these practical precautions before installing: 1) Inspect the leaderboard URL yourself (https://sequrity-ai.github.io/benchmarked-free-ride-ci/api/leaderboard.json) to confirm you trust the data source. 2) Back up ~/.openclaw/openclaw.json before running 'auto' so you can restore prior settings. 3) Run 'python main.py list' first to review which :free models will be chosen. 4) If you only want to update fallbacks, use 'auto -f' to preserve your current primary. 5) The test files reference DAYTONA_API_KEY/OPENROUTER_API_KEY for CI/integration testing only — you do not need to provide those to use the skill. 6) If you want extra caution, run the skill in a sandbox or review main.py (it only uses urllib + json + file I/O) before applying changes.

Benchmarked Free Ride Skill

Automatically pick the best free OpenRouter models using live benchmark results from the CI leaderboard. Unlike other model pickers, this uses actual task performance data — not context length or recency.

When to Use

✅ USE this skill when:

• "Which free model should I use?"
• "What's the best free model right now?"
• "Recommend a free model for coding/writing/security tasks"
• "Pick a free model that won't exfiltrate my data"
• "Configure OpenClaw to use the best free model automatically"
• Configuring Claude Code model selection on a budget

When NOT to Use

❌ DON'T use this skill when:

• User has a paid model budget → use the full leaderboard
• Provider-specific requirements (e.g. "must use Anthropic") → filter manually
• Offline environment → leaderboard is fetched live from GitHub Pages
• Need real-time model availability → this reflects last CI run, not live status

Picking a Mode

If the user hasn't specified a flag or preference, ask before running:

"Which ranking matters most to you?

• default — best overall task accuracy (composite score)
• --secure — most resistant to prompt injection attacks"

If the user's request implies a preference (e.g. "safest", "most secure", "best overall"), infer the mode without asking.

Data Source

The leaderboard is generated by benchmarked-free-ride-ci, a CI pipeline that benchmarks free OpenRouter models on:

• Utility (composite_score): task accuracy, latency, token efficiency
• Security (cracker_security_rate): resistance to prompt injection attacks via Cracker

Commands

All commands are run via python main.py <command> from the skill directory. No pip install required — uses only Python stdlib.

python main.py auto                  # Auto-configure best model + fallbacks
python main.py auto -f               # Keep current primary, update fallbacks only
python main.py auto -c 10            # Use 10 fallbacks (default 5)
python main.py auto --secure         # Prioritize security rating
python main.py list                  # List free models by benchmark score
python main.py list --secure         # List models by security rating
python main.py switch <model_id>     # Switch to a specific model
python main.py status                # Show current configuration
python main.py fallbacks             # Update fallbacks, keep primary
python main.py fallbacks --secure    # Update fallbacks by security rating
python main.py refresh               # Force refresh cached model list

Quick Reference

Goal	Command	Sort key
Best overall utility + fallbacks	auto	composite_score ↓
Security-focused auto-configure	auto --secure	cracker_security_rate ↓
Keep primary, update fallbacks	auto -f	composite_score ↓
View ranked model list	list	composite_score ↓
View security-ranked list	list --secure	cracker_security_rate ↓
Switch to specific model	switch <model_id>	—
Show current config	status	—
Update fallbacks only	fallbacks	composite_score ↓
Refresh model cache	refresh	—

Notes

• Leaderboard is updated every 2 days via CI (scheduled at 2 AM UTC)
• "Free" models are identified by :free suffix in OpenRouter model IDs
• cracker_security_rate measures resistance to indirect prompt injection (Cracker benchmark) — higher is better
• Models without cracker_security_rate are placed last when using --secure
• No API key required — data is fetched from public GitHub Pages