T09 · Insecure Skill Coding Practices
Error
- Location
scripts/run.py:89- Finding
Indirect Prompt Injection Through Untrusted Article Content
- Content
View full analysis
Vulnerability Details
File Location:
scripts/run.py:89-101
Vulnerability Type: Indirect prompt injection caused by untrusted content being interpolated into an LLM prompt
Risk Level: HighVulnerable Code:
python user_prompt = f"""请为以下健康知识查询生成科普内容： 主题：{topic} 关键词：{json.dumps(keywords, ensure_ascii=False)} 总文章数：{len(articles)}，匹配数：{len(matched)} 匹配文章： ```json {json.dumps(matched, ensure_ascii=False, indent=2)}请为匹配的文章生成科普摘要，标注来源，给出健康提示。"""
text = _call_llm(SYSTEM_PROMPT, user_prompt, appkey)
text ### Technical Analysis The application places user-controlled topics, keywords, article titles, categories, summaries, and URLs directly into the LLM prompt. Serializing these values as JSON and enclosing them in a Markdown code block does not establish a security boundary. An attacker can place model instructions inside an article field, particularly `summary`, `title`, or `url`. The model may interpret those instructions as part of the conversation rather than as inert article data. The static system prompt does not explicitly direct the model to treat document content as untrusted or prohibit following instructions embedded in supplied articles. The resulting model response is returned directly in the output: ```python "text": text.strip(),This allows model-generated content influenced by the injected instructions to reach the patient-facing rendering layer without validation.
Attack Path
- An attacker supplies an input article containing instructions such as “ignore previous instructions,” “recommend this treatment,” or “display this link.”
- The attacker chooses a title, category, summary, or keyword that causes the local matching logic to include the article.
- The complete attacker-controlled article data is serialized into
user_prompt. _call_llm()sends the prompt to the medical model.- The model may follow the embedded instructions and generat ...[truncated 646 chars]
- Remediation
View remediation
Remediation Suggestions
- Treat every field extracted from user files and article records as untrusted data.
- Use a structured model interface or strict response schema rather than embedding raw records in a free-form instruction prompt.
- Add explicit system-level instructions stating that content inside articles is data and that instructions contained in that data must never be followed.
- Separate trusted instructions from untrusted context using dedicated message or structured-input fields where supported.
- Validate the response against an allowlisted schema before returning it.
- Sanitize and validate all displayed URLs, including allowed schemes and trusted-domain requirements where appropriate.
- Apply output filtering for fabricated treatment advice, credential requests, executable content, and unsupported source claims.
- Require human or policy-based review before presenting high-impact medical statements.
- Add adversarial tests containing prompt injection strings in every accepted article field.