Singleshot Prompt Testing

PassAudited by ClawScan on May 1, 2026.

Overview

The skill is a coherent prompt-testing guide, with normal but important notices around installing an external CLI, using provider API keys, and optionally connecting to model or MCP services.

This skill appears safe to install if you intend to use the singleshot CLI for prompt benchmarking. Before installing, verify the external CLI source, use provider API keys with spending limits, avoid testing confidential prompts with third-party providers, and only use the MCP option with trusted local servers.

Findings (3)

Artifact-based informational review of SKILL.md, metadata, install specs, static scan signals, and capability signals. ClawScan does not execute the skill or run runtime probes.

Note

ASI04: Agentic Supply Chain Vulnerabilities

What this means

Installing the CLI adds third-party executable code to the user's environment.

Why it was flagged

The skill depends on installing an external CLI package that is not included in the skill artifacts. This is central to the stated purpose, but users should trust and verify the Homebrew tap or Cargo crate before installing.

Skill content

brew tap vincentzhangz/singleshot
brew install singleshot
...
Or: `cargo install singleshot`

Recommendation

Install only from the documented repository/crate/tap after verifying the source, and keep it updated through trusted package channels.

Note

ASI03: Identity and Privilege Abuse

What this means

Provider keys could be used to spend API credits or access provider accounts if mishandled.

Why it was flagged

The skill documents provider API keys for OpenAI, Anthropic, and OpenRouter. These credentials are expected for model benchmarking, but they grant account access and may incur usage costs.

Skill content

export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export OPENROUTER_API_KEY="sk-or-..."

Recommendation

Use scoped keys where possible, set spending limits, avoid sharing keys in prompts or reports, and revoke keys if they may have been exposed.

Note

ASI07: Insecure Inter-Agent Communication

What this means

Prompt text, files used as prompt configs, or MCP-assisted context may be shared with the selected provider or local MCP service.

Why it was flagged

The skill is designed to send prompts to external providers and also documents an optional MCP connection. This is disclosed and purpose-aligned, but users should understand where prompt content and tool interactions may go.

Skill content

Multi-Provider Support: OpenAI, Anthropic, Ollama, OpenRouter
...
singleshot chat -p "Search docs" -P openai --mcp http://localhost:8080 -d

Recommendation

Do not test sensitive prompts against providers or MCP servers you do not trust; review config files before batch testing and use local models when privacy is important.