T09 · Insecure Skill Coding Practices
Warning
- Location
- summarize.sh:19
- Finding
- Untrusted Transcript Content Can Manipulate Model Output<![CDATA[ ## Vulnerability Details **File Location**: `summarize.sh`, lines 19–29 **Vulnerability Type**: Prompt injection through untrusted transcript content **Risk Level**: Medium ### Vulnerable Code ```bash # Escape for JSON ESCAPED=$(printf '%s' "$TRANSCRIPT" | python3 -c 'import sys,json; print(json.dumps(sys.stdin.read()))') PROMPT="You are a meeting notes summarizer. Given the following raw meeting transcript, produce a structured summary in markdown with exactly these sections:\n\n## Summary\n(Exactly 3 sentences capturing the essence of the meeting)\n\n## Key Decisions\n(Bulleted list of decisions made)\n\n## Action Items\n(Bulleted list, each with: task, owner in **bold**, and deadline if mentioned)\n\n## Follow-up Dates\n(Bulleted list of any dates, deadlines, or scheduled follow-ups mentioned)\n\nBe concise and precise. Only include what was actually discussed.\n\nTRANSCRIPT:\n" # Build full message content as proper JSON string FULL_CONTENT=$(python3 -c " import sys, json prompt = '''$PROMPT''' transcript = json.loads($ESCAPED) print(json.dumps(prompt + transcript)) ") ``` ### Technical Analysis The script concatenates the trusted summarization instructions and the untrusted meeting transcript into one user-role model message. JSON encoding protects the HTTP request structure, but it does not establish an instruction/data trust boundary for the language model. A transcript can contain directives such as “ignore the preceding instructions,” request a different output format, fabricate decisions, suppress action items, or inject attacker-selected Markdown and links. Because the transcript and operational instructions have the same message-level authority, the model may follow instructions embedded in the transcript. This is an output-integrity vulnerability rather than local shell injection. The reviewed implementation does not give the model access to local tools, files, persistent memory, or command execution. ### Attack Path 1. An attac ...[truncated 1662 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Place the invariant summarization policy in a system message rather than concatenating it with transcript data in one user message. 2. Clearly delimit the transcript as untrusted content, preferably using a structured content block or an explicit data envelope. 3. Add a high-priority instruction stating that text inside the transcript is meeting data and that any instructions, requests, role declarations, or formatting directives within it must not be followed. 4. Validate the response before displaying or distributing it: - Require exactly the four documented Markdown sections. - Reject unexpected links, embedded HTML, or additional sections when they are not required. - Detect missing sections and retry with a constrained repair prompt. 5. Where supported, use structured JSON output with a fixed schema and render the Markdown locally rather than accepting unconstrained Markdown from the model. 6. Treat generated summaries as untrusted output and advise users to verify consequential decisions, owners, and deadlines against the original transcript. 7. Document that transcript contents are transmitted to Anthropic and may contain confidential meeting information. Recommend redaction or organizational approval where appropriate. ]]>