Singleshot Prompt Testing
v0.1.0Test and optimize prompts for cost, token use, and performance with detailed reports using single shot queries across multiple providers and models.
⭐ 3· 1.5k·1 current·1 all-time
byVincent@vincentzhangz
MIT-0
Download zip
LicenseMIT-0 · Free to use, modify, and redistribute. No attribution required.
Security Scan
OpenClaw
Benign
high confidencePurpose & Capability
The skill is named and documented as a prompt cost/testing/optimization helper and all instructions show usage of a singleshot CLI for generating token/cost reports. The declared requirements (API keys for providers) match the described multi-provider testing capability and nothing extraneous (e.g., cloud admin creds) is requested.
Instruction Scope
SKILL.md instructs the agent to run the singleshot CLI, generate reports, cat/grep/diff report files, and optionally point the CLI at providers via environment variables. These actions are within the stated purpose. One noteworthy point: the documentation allows configuring OPENAI_BASE_URL (a custom endpoint) and other provider endpoints which can redirect model requests to arbitrary servers—this is a normal feature for alternate endpoints but increases risk if you point keys to untrusted endpoints.
Install Mechanism
The published skill is instruction-only and contains no automated install spec. The docs recommend installing a third-party CLI via Homebrew tap (vincentzhangz/singleshot) or cargo. That is consistent with a CLI-based skill; however, installing from a third-party tap or crate is an explicit user action and you should audit the upstream repo before installing.
Credentials
The skill recommends supplying provider API keys (OPENAI_API_KEY, ANTHROPIC_API_KEY, OPENROUTER_API_KEY) which are directly relevant to calling model providers. No unrelated secrets or system credentials are requested. Caution: OPENAI_BASE_URL and similar endpoint variables can be used to route requests (and therefore your keys/data) to nonstandard endpoints—only set them to trusted URLs.
Persistence & Privilege
The skill does not request always:true and is user-invocable only. It is instruction-only and does not install persistent hooks or modify other skills or global agent settings. It therefore requests no elevated persistence or privileges.
Assessment
This skill appears coherent and implements what it says: a wrapper/workflow for a third-party singleshot CLI that measures tokens, costs, and latency. Before installing or running the CLI yourself: 1) Inspect the upstream repository (https://github.com/vincentzhangz/singleshot) and the Homebrew tap to confirm code provenance; 2) Only provide API keys for providers you trust, and avoid setting OPENAI_BASE_URL or other custom endpoints to unknown servers (they could receive your requests and keys); 3) Prefer using local/no-key options (e.g., Ollama) for early testing; 4) Consider using scoped or short-lived keys if supported, and do not paste keys into public files. If you want a deeper review, provide the upstream repo or the installed binary source and I can look for network calls, telemetry, or unexpected behavior.Like a lobster shell, security has layers — review code before you run it.
latestvk975m1y6g5gcdyxqnrkca71jxn80h546
License
MIT-0
Free to use, modify, and redistribute. No attribution required.