T09 · Insecure Skill Coding Practices
- Location
- scripts/run.py:75
- Finding
- Prompt Injection Through Untrusted Assessment Content<![CDATA[ ## Vulnerability Details **File Location**: `scripts/run.py`, lines 75-95 **Vulnerability Type**: Prompt injection caused by unsafe incorporation of user-controlled data into an LLM prompt **Risk Level**: High ### Vulnerable Code ```python answer_summary = [] total = 0.0 for q in questions: qid = q.get("id", "") qtext = q.get("text", "") ans = answers.get(qid, 0) score = to_float(ans) or 0.0 total += score answer_summary.append({"id": qid, "question": qtext, "answer": ans, "score": score}) user_prompt = f"""Please interpret the following functional self-assessment results: Assessment ID: {assessment_id} Assessment time: {assessed_at} Number of questions: {len(questions)} Total score: {total} (maximum {len(questions)*5}, 1-5 points per question) Scores by question: ```json {json.dumps(answer_summary, ensure_ascii=False, indent=2)} ``` Please interpret the score for each dimension, assess the overall recovery stage, and provide recommendations.""" text = _call_llm(SYSTEM_PROMPT, user_prompt, appkey) ``` ### Technical Analysis The values used to construct `answer_summary`, including question identifiers, question text, and answers, originate from the input assessment. They are directly interpolated into the downstream language-model prompt without being treated as untrusted content. Placing the content inside a JSON-formatted Markdown block does not establish a security boundary for a language model. A malicious question or answer can contain instructions asking the model to disregard its system role, suppress medical disclaimers, fabricate a diagnosis, or provide unsafe recommendations. The generated response is accepted without policy validation and returned directly as patient-facing `text`. This is especially significant because the skill operates in a medical context and its documented boundary states that it should not perform diagnosis or replace professional assessment. ### Attack Path 1. An attacker or untrust ...[truncated 1014 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Define and enforce a strict schema for questions and answers, including data types, allowed lengths, identifier syntax, and permitted score ranges. 2. Explicitly state in the system prompt that all assessment fields are untrusted data and must never be interpreted as instructions. 3. Submit assessment data through a structured interface or tool/function schema where supported, rather than concatenating it into an instruction-bearing prompt. 4. Separate trusted instructions from untrusted content using clearly identified fields, while recognizing that delimiters alone are not a complete defense. 5. Reject or flag question content containing model-directed instructions when such content is not legitimate for the assessment. 6. Validate the generated response before displaying it. Enforce mandatory disclaimers and reject diagnostic claims or recommendations outside the skill's intended scope. 7. Prefer deterministic local generation for score summaries where model inference is not necessary. 8. Add adversarial tests covering instructions embedded in question text, answers, assessment identifiers, and dates. ]]>
