Rag Evaluator
v2.0.0Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like a ragaai catalyst, python, agentic-ai.
⭐ 0· 105·0 current·0 all-time
MIT-0
Download zip
LicenseMIT-0 · Free to use, modify, and redistribute. No attribution required.
Security Scan
OpenClaw
Benign
high confidencePurpose & Capability
Name/description (Rag Evaluator) match the behavior: a small CLI that logs evaluation/benchmark/prompt data and provides exports, search, stats, and status. There are no unrelated env vars, binaries, or services requested.
Instruction Scope
Runtime instructions and the included script only read/write files in ~/.local/share/rag-evaluator and run standard Unix utilities. However, the tool logs arbitrary user-provided inputs (prompts, configs, costs, etc.) so users could accidentally store sensitive secrets or API keys. The JSON export implementation is brittle (naive string formatting, no escaping, and uses a literal "\n]" which may produce invalid JSON), so exported data may be malformed.
Install Mechanism
There is no install spec (instruction-only skill) which is low risk. A shell script is included; it is plain Bash and not obfuscated. Because there's no automatic installer, the script won't be placed on PATH by the platform unless the agent/user does so; inspect the script before running or install into an isolated environment if desired.
Credentials
The skill declares no required environment variables or credentials and the script only uses HOME and standard utilities. Requested accesses are proportional to a local logging CLI.
Persistence & Privilege
The script persists logs and exports under the user's home (~/.local/share/rag-evaluator) which is expected for this tool. It does not request elevated privileges, does not modify other skills, and always:false. Consider the privacy implications of storing inputs on disk.
Assessment
This skill appears coherent and local-only: it stores logs and exports under ~/.local/share/rag-evaluator and does not contact external endpoints or request credentials. Before installing or running it, review the included scripts (scripts/script.sh). Be aware that commands like `prompt`, `configure`, or `evaluate` log whatever you pass — avoid entering secrets or API keys into those entries. Exports and logs are stored on disk (they may grow over time), so consider disk space and file permissions. If you want extra safety, run the script in an isolated environment (container or dedicated user account) or modify DATA_DIR to point to a controlled location. Finally, note the JSON export is implemented naively and may produce invalid JSON for values containing quotes or newlines; treat exported files accordingly.Like a lobster shell, security has layers — review code before you run it.
latestvk977gc82nfapnbervbp2nb9enx837zt9
License
MIT-0
Free to use, modify, and redistribute. No attribution required.