ClawHub

LLM Regression Monitor

v1.0.2

Use this skill when the user wants to monitor LLM behavior over time and get alerted when outputs change unexpectedly. Triggers on requests like "set up LLM...

⭐ 0· 58·0 current·0 all-time
by@swanand33
MIT-0
Download zip
LicenseMIT-0 · Free to use, modify, and redistribute. No attribution required.
Security Scan
VirusTotalVirusTotal
Benign
View report →
OpenClawOpenClaw
Benign
high confidence
✓
Purpose & Capability
Name/description match what the files do: capture baselines, run behavioral/drift checks, and send alerts. Primary credential OPENAI_API_KEY and optional provider keys correspond to the supported providers described in references.
✓
Instruction Scope
SKILL.md steps (install dependencies, create test_suite.yaml, capture baselines, run monitor, configure alerts, schedule) match the scripts. The scripts only read expected files (.env, test_suite.yaml, baselines, monitor_report.json) and environment variables declared in the docs; they don't access unrelated system paths or hidden endpoints.
ℹ
Install Mechanism
No registry install spec (instruction-only), but SKILL.md directs users to pip-install third-party packages (llm-behave[semantic], pyyaml, requests). This is reasonable for the task but means you should vet the llm-behave package and its provider adapters before installing.
✓
Credentials
Requested env vars (OPENAI_API_KEY, ANTHROPIC_API_KEY, OLLAMA_BASE_URL, CUSTOM_LLM_BASE_URL/CUSTOM_LLM_API_KEY, ALERT_WHATSAPP_TO, ALERT_SLACK_WEBHOOK) are all justified by the code and listed as optional in SKILL.md. No unrelated secrets or broad system credentials are requested.
✓
Persistence & Privilege
Skill is not always-enabled and does not modify other skills or global agent settings. It writes baselines and logs to project-local files (.llm_behave_baselines/, monitor_alerts.log) which is appropriate for its function.
Assessment
This skill appears internally consistent for monitoring LLM outputs. Before installing, review and/or vet the llm-behave package (it will make the provider API calls), and only set provider/webhook environment variables you trust. Note: baselines and alert logs are stored in the project directory (.llm_behave_baselines/, monitor_alerts.log). Slack webhooks and a WhatsApp CLI (wacli) are optional and only used if you supply their configuration. If you run this in CI or on shared infrastructure, ensure API keys (OPENAI/Anthropic/CUSTOM) and internal LLM endpoints are stored securely and that you are comfortable their outputs will be transmitted to the configured providers/webhooks.

Like a lobster shell, security has layers — review code before you run it.

latestvk97e4zadxtgqxw519h2mm87khn83pe5d

License

MIT-0
Free to use, modify, and redistribute. No attribution required.
Termshttps://spdx.org/licenses/MIT-0.html

Runtime requirements

🔍 Clawdis
Primary envOPENAI_API_KEY

Comments