T02 · Agent Memory Poisoning
Warning
- Location
- SKILL.md:176
- Finding
- Untrusted Uploaded Documents Can Poison the Persistent Reference Knowledge Base## Vulnerability Details **File Location**: `SKILL.md`, lines 176–181 **Vulnerability Type**: Persistent agent memory poisoning **Risk Level**: Medium ### Vulnerable Code Segment ```markdown ### 文件学习流程（如上传了文档） 当用户上传安全管理文件时： 1. 使用 `read` 工具读取文件内容 2. 提取关键条款、流程、规范要求 3. 更新参考知识库（写入 `references/` 目录下的相关文件） 4. 告知用户已学习并记录的内容摘要 ``` The workflow directs the agent to: 1. Read user-uploaded safety-management documents. 2. Extract requirements and procedures from them. 3. write the extracted content into files under `references/`. 4. Treat those files as a knowledge source in later analyses. ### Technical Analysis Uploaded documents are attacker-controlled input, but the skill instructs the agent to persist their extracted content in its reference knowledge base. No controls are defined for: - Authenticating the document or its publisher. - Verifying claimed standards against approved primary sources. - Distinguishing document content from embedded agent instructions. - Requiring human approval before persistence. - Recording source provenance and trust level. - Detecting conflicts with existing standards. - Isolating uploaded material from trusted packaged references. The risk is amplified by `SKILL.md`, lines 184–190, which instructs the agent to load files under `references/` during future analyses and treat their detailed clauses as more authoritative than model memory: ```markdown ## 知识参考 分析过程中，如需查阅具体标准条款，先加载参考文件： - `references/gb30871-2022.md` — GB30871-2022 核心条款 - `references/petrochem-standards.md` — 石化企业补充标准 详细条款以上述文件为准，模型记忆可能存在偏差，以文件内容为准。 ``` Consequently, untrusted material can cross a trust boundary and become persistent guidance. This is a memory-poisoning vulnerability rather than direct code execution: the package contains no executable scripts, remote payload retrieval, or shell commands. ### Attack Path 1. An attacker supplies a safety-management document containing fabricated requirements, altered thresholds, misleading procedures, or concea ...[truncated 1477 chars]
- Remediation
- ## Remediation Suggestions 1. **Prohibit automatic modification of trusted references** - Treat packaged files under `references/` as read-only. - Do not merge uploaded content directly into authoritative standards files. 2. **Separate trusted and untrusted knowledge** - Store user-derived notes in a dedicated untrusted area outside the authoritative reference directory. - Prefer session-scoped storage unless persistence is explicitly necessary. - Ensure trusted, reviewed references always take precedence over uploaded material. 3. **Require explicit approval before persistence** - Present the extracted claims, source, and proposed destination to an authorized user. - Require affirmative approval before writing any persistent data. - Provide a clear rollback and deletion mechanism. 4. **Validate extracted claims** - Verify standards, clause numbers, thresholds, and effective dates against approved primary sources. - Reject or quarantine claims that cannot be verified. - Detect and flag conflicts with existing reviewed references. 5. **Neutralize instruction injection** - Process uploaded documents strictly as data. - Ignore document text that asks the agent to change its behavior, tools, priorities, permissions, or system instructions. - Use schema-constrained extraction limited to relevant safety facts. 6. **Preserve provenance and trust metadata** - Record the document identity, publisher, upload date, uploader, verification status, and exact source passage for every persisted claim. - Clearly label all unverified content and prevent it from being cited as authoritative. 7. **Apply least-privilege write controls** - Restrict write access to a dedicated staging file or directory. - Prevent uploaded content from selecting arbitrary paths or overwriting packaged files. - Require review before promoting staged material into the trusted knowledge base.