T09 · Insecure Skill Coding Practices
Warning
- Location
- humanize.py:58
- Finding
- Untrusted Input Can Override LLM Rewrite Instructions<![CDATA[ ## Vulnerability Details **File Location**: `humanize.py`, lines 58–91 **Vulnerability Type**: Prompt injection through direct interpolation of untrusted text **Risk Level**: Medium ### Vulnerable Code ```python def build_prompt(text: str, channel: str, tells: list[str]) -> str: """Construct the rewrite prompt.""" # 🐾 channel_guide = CHANNEL_GUIDES.get(channel, CHANNEL_GUIDES["general"]) tells_note = "" if tells: tells_note = ( f"\n\nDetected AI tells to eliminate: {', '.join(tells)}. " "Replace these with concrete, specific language." ) return f"""You are a human editor. Your job: take AI-generated text and make it sound like a real person wrote it. Rules: - Kill vague qualifiers. Be specific or cut them. - Vary sentence length. Mix short punchy sentences with longer flowing ones. - Use contractions (it's, don't, we're) where natural. - Fragments are fine. So are one-word sentences. Really. - Remove filler that doesn't carry meaning. - Keep the core message intact — don't invent new claims. - Add natural rhythm imperfections — humans don't write in perfect parallel structure. - No meta-commentary. Don't say "here's the rewritten version". Just output the text. {tells_note} Channel guidance: {channel_guide} TEXT TO HUMANIZE: --- {text} --- Output ONLY the rewritten text. Nothing else.""" ``` ### Technical Analysis The `text` parameter originates from a command-line argument or standard input and is interpolated verbatim into the same natural-language prompt that contains the application's operational instructions. The `---` delimiters are only text and do not create an enforceable trust boundary for the downstream language model. An attacker can submit content containing instructions that conflict with the rewrite rules, such as directions to ignore previous instructions and emit attacker-selected text. Because the model must interpret both the trusted rules and the untrusted co ...[truncated 1956 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Separate trusted instructions from untrusted source text using the strongest structured message or content-block boundaries supported by the API. Do not rely solely on visual delimiters. 2. Explicitly state that content inside the source-text block is untrusted data and that any instructions within it must not be followed. 3. Encode or serialize the source text as data where practical, then instruct the model to transform only the decoded field. 4. Validate the response before exposing it as trusted output. Reject unexpected formats, excessive length changes, or responses containing instruction-like meta-commentary when those characteristics violate the intended task. 5. Preserve a clear trust label for generated output. Documentation should warn agents that standard output is model-generated and must not be treated as an authoritative instruction or executed automatically. 6. Require human review before publishing output or passing it into tools with side effects. 7. Add adversarial tests covering nested instructions, fake delimiter closures, role impersonation, requests to reveal prompt content, and attempts to generate downstream commands. 8. If strong output integrity is required, use a constrained transformation pipeline or deterministic rewriting rules rather than relying exclusively on an instruction-following model. ]]>