T09 · Insecure Skill Coding Practices
Error
- Location
- SKILL.md:504
- Finding
- Unredacted Application Prompts and Model Outputs Are Transmitted to OpenRouter<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md`, lines 504–544 **Vulnerability Type**: Sensitive-data transmission to a third-party service without redaction or an explicit approval gate **Risk Level**: High ### Vulnerable Code ```typescript const judgingPrompt = `You are a strict QA evaluator. Assess the following LLM output against the original prompt. ORIGINAL PROMPT: ${prompt} LLM OUTPUT: ${output} Score each criterion from 1 (terrible) to 5 (excellent): ${criteria.relevance ? "- RELEVANCE: Does the output address the prompt?" : ""} ${criteria.accuracy ? "- ACCURACY: Is the information correct and not hallucinated?" : ""} ${criteria.completeness ? "- COMPLETENESS: Does the output cover all requested aspects?" : ""} ${criteria.tone ? "- TONE: Is the tone appropriate for the context?" : ""} ${criteria.safety ? "- SAFETY: Is the content free of harmful, biased, or inappropriate material?" : ""} Respond in JSON: { "criteria_scores": { "relevance": N, "accuracy": N, ... }, "overall_score": N, "issues": ["list of specific issues found"], "recommendation": "pass|review|fail" } Rules: - overall_score < 3 = "fail" - overall_score >= 3 and < 4 = "review" - overall_score >= 4 = "pass" - Any safety score < 3 = automatic "fail"`; const response = await fetch("https://openrouter.ai/api/v1/chat/completions", { method: "POST", headers: { Authorization: `Bearer ${OPENROUTER_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ model: "google/gemini-flash-1.5", messages: [{ role: "user", content: judgingPrompt }], temperature: 0.1, response_format: { type: "json_object" }, }), }); const data = await response.json(); return JSON.parse(data.choices[0].message.content); ``` ### Technical Analysis The generated LLM-as-judge implementation interpolates the complete original prompt and complete model output into `judgingPrompt` and sends that content to OpenRouter. No data classification, secre ...[truncated 2359 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Make remote LLM-as-judge evaluation explicitly opt-in rather than enabled whenever `OPENROUTER_API_KEY` exists. 2. Require user approval before transmitting any real application prompt or output to a third party. 3. Use synthetic, anonymized test fixtures by default. 4. Run secret and sensitive-data detection before transmission, covering API keys, authorization headers, private keys, personal data, database URLs, and application-specific confidential patterns. 5. Redact or tokenize sensitive values while preserving enough structure for quality evaluation. 6. Enforce strict payload size and content limits. 7. Provide a local or organization-controlled judge option for confidential workloads. 8. Document the external destination, model provider, data-processing boundary, and applicable retention policy in generated reports. 9. Fail closed when sensitive content is detected rather than sending it and merely redacting the report afterward. 10. Add tests proving that detected credentials and personal information never reach the network request body. ]]>