Back to skill

Security audit

sith-choir

Security checks for vulnerabilities and agentic risk

Overview

The skill is mostly transparent about routing prompts through many model API keys, but its custom endpoint support can forward credentials and content to arbitrary configured URLs without enough validation.

Install only if you are comfortable letting this skill use the API keys in its credentials file and send your prompts, answers, and selected images to the configured providers. Avoid secrets or regulated data, use --plan and --no-cache for sensitive work, keep credentials chmod 600, and do not use custom base endpoints unless you fully trust the exact destination.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/choir.py:327
Finding
Unvalidated Custom Provider Endpoint Can Receive API Credentials and User Data<![CDATA[ ## Vulnerability Details **File Location**: `scripts/choir.py:327-332`, with credential transmission at `scripts/choir.py:916-937` and `scripts/choir.py:952-969` **Vulnerability Type**: Arbitrary credential forwarding, insecure endpoint configuration, and server-side request forgery **Risk Level**: Medium ### Complete Code Snippet ```python # scripts/choir.py:327-332 base = cfg.get("base") or DEFAULT_BASE.get(name, "") kind = cfg.get("kind") or ("gemini" if name in ("gemini", "google") else "openai") model = cfg.get("model") or cfg.get("default_model") or DEFAULT_MODEL.get(name, "") tags = set(cfg.get("tags", []) or []) self.providers[name] = { "keys": keys, "base": base, "kind": kind, "model": model, "tags": tags, ``` ```python # scripts/choir.py:916-937 def _call_openai(m, prompt, system, max_tokens, timeout, image_path): base = m["base"] or DEFAULT_BASE.get(m["provider"], "") url = base.rstrip("/") + "/chat/completions" headers = {"Authorization": "Bearer " + m["key"], "Content-Type": "application/json", "User-Agent": f"sith-choir/{VERSION}"} messages = [] if system: messages.append({"role": "system", "content": system}) if image_path: mime, b64 = _image_data_uri(image_path) messages.append({"role": "user", "content": [ {"type": "text", "text": prompt}, {"type": "image_url", "image_url": {"url": f"data:{mime};base64,{b64}"}}]}) else: messages.append({"role": "user", "content": prompt}) body = {"model": m["model"], "messages": messages, "max_tokens": max_tokens, "temperature": 0.4} for attempt in (1, 2): try: t0 = time.monotonic() data = _http_json(url, headers, body, timeout) ``` ```python # scripts/choir.py:952-969 def _call_gemini(m, prompt, system, max_tokens, timeout, image_path): base = m["base"] or DEFAULT_BASE.get(m["provider"], "") mdl = m["model ...[truncated 3778 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Parse custom endpoints with `urllib.parse.urlsplit()` and reject malformed URLs. 2. Require `https` for every credential-bearing request. Permit plaintext HTTP only through an explicit development-only option that refuses real credentials. 3. Enforce an allowlist of approved provider hostnames corresponding to the declared outbound network policy. 4. If custom OpenAI-compatible endpoints are required, place them behind an explicit flag such as `--allow-custom-base` and display the exact destination before sending credentials. 5. Resolve destination hostnames and reject loopback, link-local, multicast, reserved, and private IP ranges unless a separate, clearly documented local-endpoint option is enabled. 6. Reject URLs containing embedded user information, fragments, unexpected ports, or ambiguous hostname encodings. 7. Disable redirects for credential-bearing requests or independently validate every redirect target. Never forward authorization headers across origins. 8. Bind credentials to expected origins in the credentials schema, rather than allowing any key to be combined with any arbitrary `base`. 9. Add offline tests covering HTTP rejection, malicious hosts, internal IP addresses, encoded IP representations, DNS resolution, and cross-origin redirects. 10. Update the documentation to distinguish approved provider endpoints from explicitly trusted custom endpoints and explain that custom endpoints receive keys and all submitted content. ]]>
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Findings (5)

Intent-Code Divergence

Medium
Confidence
96% confidence
Finding
The privacy comments claim that provenance, cache, and related state avoid storing raw content, but the implementation stores full answers in the cache and full user/assistant content in session files. In a skill explicitly designed to relay prompts across many providers and reuse prior conversations, this mismatch can expose sensitive prompts, images-derived content, or model outputs to anyone with access to the local state directory or backups.

Description-Behavior Mismatch

Medium
Confidence
95% confidence
Finding
Session handling wraps prior context in main() and then cmd_ask wraps session context again, causing duplicated history in outbound prompts; main() also appends the exchange after command completion while cmd_ask appends it internally, causing duplicate transcript entries. In this skill, duplicated context increases data exposure to external providers, inflates token costs, and can unintentionally persist more user content than expected.

Missing User Warnings

Medium
Confidence
94% confidence
Finding
The design explicitly promotes orchestration across a large fleet of stored API keys, including per-key health tracking, failover, and cross-provider use, but does not pair that with prominent warnings or safeguards about credential sensitivity, data sharing boundaries, or privacy impact. In this skill’s context, that omission is more dangerous than usual because the whole architecture is built to route user prompts and possibly sensitive context across many third-party providers and many credentials, increasing blast radius if secrets, regulated data, or proprietary inputs are sent without operator awareness.

Missing User Warnings

Medium
Confidence
93% confidence
Finding
Session transcripts persist raw user prompts and model answers to disk, but the operational UX does not provide an explicit warning at the time the user enables --session. In a tool that federates requests across multiple third-party LLM providers, users may unknowingly store sensitive prompts, secrets, or regulated data locally beyond the duration of the run.

Credential Access

High
Category
Privilege Escalation
Content
bins: ["python3"]
    network:
      outbound: ["*.openai.com", "generativelanguage.googleapis.com", "openrouter.ai", "api.groq.com", "api.mistral.ai", "api.cohere.com", "router.huggingface.co", "api.z.ai", "api.llm7.io", "api.cerebras.io", "api.x.ai", "api.moonshot.ai"]
    configPaths: ["~/.config/sith-choir/credentials.json", "~/.cache/sith-choir/"]
---

# 🦞 sith-choir — Cross-Model Intellect Relay (all keys join the choir)
Confidence
93% confidence
Finding
The skill is explicitly designed to read a credentials store containing multiple provider API keys and then route user prompts and images to many third-party endpoints. This creates a real high-risk trust boundary: compromise, misuse, prompt injection through downstream workflows, or accidental use on sensitive data could expose secrets-bearing accounts and send confidential content across numerous external providers.

Static analysis

No suspicious patterns detected.