T09 · Insecure Skill Coding Practices
- Location
references/extraction-patterns.md:26- Finding
Untrusted Paper Content Is Embedded Directly into the LLM Instruction Prompt
- Content
View full analysis
Vulnerability Details
File Location:
references/extraction-patterns.md:26-35
Vulnerability Type: Indirect prompt injection
Risk Level: MediumVulnerable Code
text ## HEADER DEFINITIONS {header_definitions} ## PAPER CONTENT {paper_text} ## EXTRACTION INSTRUCTIONS For each header: 1. Search the paper content thoroughly for explicit mentions 2. Extract the exact or closely paraphrased informationTechnical Analysis
The prompt template inserts untrusted paper text directly into the same instruction context used to direct the LLM. Although the template contains extraction quality rules, it does not explicitly classify the paper as untrusted data or tell the model to disregard instructions appearing inside the document.
A malicious PDF, DOCX, or TXT file can therefore include instructions that attempt to override the extraction rules, manipulate the generated JSON, fabricate field values, or request disclosure of prompt context. This is an instruction-data boundary failure. The generated data may subsequently reach the table-writing workflow because validation does not verify that every value is supported by cited paper evidence.
Attack Path
- An attacker supplies a crafted academic-paper file containing hidden or visible model instructions.
extract_paper.pyextracts those instructions as ordinary document text.- The extracted text is substituted into the
{paper_text}placeholder. - The LLM receives attacker-controlled instructions in the same context as trusted extraction instructions.
- The LLM may follow the embedded instructions and return manipulated JSON.
- The manipulated values can be passed to
write_table.pyand persisted in the output table.
Impact Assessment
Exploitation can compromise the integrity of extracted research data, produce fabricated or attacker-selected table values, corrupt the expected JSON structure, and potentially disclose ...[truncated 236 chars]
- Remediation
View remediation
Remediation Suggestions
- Explicitly state that paper content is untrusted data and that instructions appearing within it must never be followed.
- Place document content in a separate, data-only message or structured attachment when the LLM API supports role separation.
- Use strong delimiters around the document and state that delimiter contents may only be analyzed as evidence.
- Require the output to conform to a strict JSON Schema containing only expected header names.
- Reject unexpected keys, invalid confidence values, missing sources, and incorrect value types.
- Require every non-
N/Avalue to include a verifiable source quotation or location from the paper. - Add detection and human-review handling for documents containing instruction-like phrases directed at an AI system.
- Do not allow extracted document content to invoke tools, alter system prompts, or select output destinations.
