Singleshot Prompt Testing

• Outcome: Review
• Latest audit: Sep 10, 2026 · 9:38 AM UTC
• Version: 0.1.0

Overview

This skill appears purpose-built for prompt cost testing, but it needs review because it installs an unpinned third-party CLI and can send prompts, images, and API-key-backed requests to external providers without strong data-handling warnings.

Review before installing. Use non-sensitive or redacted prompts and images, choose providers deliberately, expose only the API key needed for the current test, store generated reports in approved locations, and prefer pinned or verified CLI releases when available.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally

Findings (1)

T08 · Insecure Dependencies

Error

• Location: SKILL.md:9

Finding

Unpinned Third-Party CLI Installation from Mutable Sources

Content

View full analysis

Remediation

View remediation

' --locked ``` 2. For source installation, check out an immutable audited commit rather than the mutable default branch: ```bash git clone https://github.com/vincentzhangz/singleshot cd singleshot git checkout --detach '' cargo install --path . --locked ``` 3. Publish expected SHA-256 checksums or cryptographic signatures for release artifacts and require users to verify them before installation. 4. Prefer a trusted and reproducible distribution channel. If the third-party Homebrew tap remains supported, pin an audited formula revision and document how users can verify the formula and downloaded artifact. 5. Add release provenance controls, such as signed Git tags, signed release artifacts, protected publisher accounts with multi-factor authentication, and build attestations. 6. Document the expected package publisher, repository, exact version, commit identifier, and integrity values so users can detect dependency confusion or publisher substitution. 7. Run the CLI with least privilege and expose only the provider credential needed for the current command. Avoid installing or executing it from an administrator account. 8. Review and update pinned versions deliberately, repeating the security review and integrity verification before changing the documented version or commit. ]]>

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• Trigger Abuse: Overly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access

Findings (16)

External Model or Provider Selection

High

• Category: Excessive Agency
• Confidence: 90% confidence

Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · README.md (reported line 116)May include surrounding context.

md

singleshot chat -p "prompt" -P openai -d -r report.md

# With specific model
singleshot chat -p "prompt" -P openai -m gpt-4o-mini -d -r report.md

# From config file
singleshot chat -l config.md -d -r report.md

External Model or Provider Selection

High

• Category: Excessive Agency
• Confidence: 90% confidence

Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · README.md (reported line 142)May include surrounding context.

bash

prompt="Explain lifetimes in Rust"

singleshot chat -p "$prompt" -P openai -m gpt-4o-mini -d -r openai.md
singleshot chat -p "$prompt" -P anthropic -m claude-sonnet-4-20250514 -d -r anthropic.md

echo "Cost comparison:"

External Model or Provider Selection

High

• Category: Excessive Agency
• Confidence: 90% confidence

Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · README.md (reported line 143)May include surrounding context.

md

prompt="Explain lifetimes in Rust"

singleshot chat -p "$prompt" -P openai -m gpt-4o-mini -d -r openai.md
singleshot chat -p "$prompt" -P anthropic -m claude-sonnet-4-20250514 -d -r anthropic.md

echo "Cost comparison:"
grep "Total Cost" openai.md anthropic.md

External Model or Provider Selection

High

• Category: Excessive Agency
• Confidence: 90% confidence

Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · SKILL.md (reported line 35)May include surrounding context.

md

singleshot chat -l config.md -d -r report.md

# Compare providers
singleshot chat -p "Test" -P openai -m gpt-4o-mini -d -r openai.md
singleshot chat -p "Test" -P anthropic -m claude-sonnet-4-20250514 -d -r anthropic.md

# Batch test variations

External Model or Provider Selection

High

• Category: Excessive Agency
• Confidence: 90% confidence

Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · SKILL.md (reported line 36)May include surrounding context.

md

# Compare providers
singleshot chat -p "Test" -P openai -m gpt-4o-mini -d -r openai.md
singleshot chat -p "Test" -P anthropic -m claude-sonnet-4-20250514 -d -r anthropic.md

# Batch test variations
for config in *.md; do

External Model or Provider Selection

High

• Category: Excessive Agency
• Confidence: 90% confidence

Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · SKILL.md (reported line 98)May include surrounding context.

1. Test with cheaper models first:

   bash

   singleshot chat -p "Test" -P openai -m gpt-4o-mini -d -r report.md
   

2. Reduce tokens:

External Model or Provider Selection

High

• Category: Excessive Agency
• Confidence: 90% confidence

Finding

Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.

Content

Scanner excerpt · SKILL.md (reported line 108)May include surrounding context.

3. Test locally (free):

   bash

   singleshot chat -p "Test" -P ollama -m llama3.2 -d -r report.md
   

Example: Full Optimization

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 92% confidence

Finding

The README repeatedly instructs users to send prompts, optional images, and MCP-connected requests to external providers and to save generated reports locally, but it never warns that sensitive data may leave the local environment or be persisted to disk. In a prompt-testing skill, this omission is material because users may test real production prompts, secrets, customer data, or proprietary images and unknowingly disclose them to third parties or leave them in report files.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 93% confidence

Finding

The skill repeatedly instructs users to write detailed prompt-testing reports to local files, but it never warns that those reports can contain full prompts, model outputs, token/cost metadata, and other potentially sensitive material. In prompt-engineering workflows, those artifacts are often later shared, committed, or retained insecurely, which can lead to unintended disclosure of proprietary or sensitive data.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 96% confidence

Finding

These examples send prompts to external providers such as OpenAI and Anthropic without disclosing that prompt content and associated data leave the local system. Users may test real internal prompts or customer data, not realizing the skill is instructing transmission to third-party services, which creates confidentiality and compliance risk.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 98% confidence

Finding

The vision-test example explicitly uploads a local image to an external model provider but gives no warning that the image contents may be transmitted off-system. Images can contain sensitive visual data, embedded text, personal information, or confidential documents, so this omission materially increases the chance of accidental leakage.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The phrase "test prompt" is highly ambiguous and common in routine prompt-engineering workflows, making accidental invocation likely. In an agent setting, broad triggers can route user data into an unnecessary external tool path or alter system behavior without sufficiently clear user intent.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 92% confidence

Finding

The trigger "optimize prompt" is a generic instruction that overlaps with a wide range of harmless assistant tasks, so it does not reliably signal that this specific skill should run. This ambiguity increases the chance of overbroad activation and unintended use of installed tooling or provider APIs.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 93% confidence

Finding

The trigger phrase "compare models" is broad and likely to match many ordinary AI-assistant conversations unrelated to this specific skill. That can cause unintended activation, exposing user prompts or causing the agent to invoke benchmarking behavior in contexts where the user did not explicitly request this tool.

Content

No source excerpt is available for this finding.

Vague Triggers

Low

• Category: Not specified by scanner
• Confidence: 92% confidence

Finding

The phrase "openclaw skill triggers: prompt testing, optimization, benchmarking" lists broad nouns rather than specific invocation phrases or conditions. Without explicit scope, examples, or exclusions, these terms could overlap with common requests and make it unclear when the skill should activate.

Content

No source excerpt is available for this finding.

Vague Triggers

Low

• Category: Not specified by scanner
• Confidence: 88% confidence

Finding

The trigger set as a whole lacks clear boundaries, which increases the risk that normal conversation about prompts, benchmarking, or model comparison will spuriously activate the skill. While this metadata does not itself execute code, ambiguous routing in an agent ecosystem can still create privacy, reliability, and least-privilege issues.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.