Benchmark Store

What to check before installing/using this skill: - Functionality vs CLI: The compare CLI examples assume the script will run by itself, but scripts/benchmark_db.py's compare path expects an evaluator callable (i.e., a function to run tests and return scores). Confirm how you will supply/implement that evaluator when invoking --action compare; otherwise the script deliberately refuses to return mocked scores. - Hidden tests / secrets: Hidden test loading and decryption APIs require a password/key (not declared as an env var). Decide how to store/provide any decryption password (avoid embedding in repo). Ensure you understand where any decrypted test data will be kept in plaintext and who can access it. - Filesystem writes: The skill will create or update benchmarks.db and state/pareto.json and can export reports. Run it in a sandbox or with a controlled working directory to avoid accidental writes to sensitive locations. - Code review: Because this package contains Python code (interfaces/, scripts/), review the full main/entry code paths (the truncated main() in benchmark_db.py and any functions that may import dynamic evaluators or execute code) to ensure there is no dynamic execution of untrusted code (exec/eval/os.system). The static check here did not flag obvious exec/os.system usage, but you should verify the omitted/truncated portions. - No network creds by default: The package does not request API keys or network credentials; if you later integrate external benchmark imports or remote data sources, require explicit review and avoid putting secrets into repo files. - Next steps: If you intend to run comparisons, prepare or inspect an evaluator implementation that conforms to the expected signature, confirm how encrypted hidden tests are unlocked (password handling), and run the skill in a disposable environment first. If you want, I can: (1) point out the exact lines where compare raises on missing evaluator and where HiddenTest decryption is expected, or (2) suggest a safe wrapper/driver to call the compare path with a controlled evaluator implementation.