T09 · Insecure Skill Coding Practices
- Location
examples/judge-prompt.md:37- Finding
Prompt Injection Through Untrusted Agent Trace Content
- Content
View full analysis
Vulnerability Details
File Location:
examples/judge-prompt.md, lines 37–53 and 60–62
Vulnerability Type: Untrusted content inserted into an instruction-bearing LLM prompt
Risk Level: MediumVulnerable Snippet
text [用户输入] {user_input} [Agent 输出] {agent_output} [工具调用记录] {tool_calls_json_array} [完成状态] {completion_status} [使用模型] {model_used} 请对该 agent turn 进行六维度 PRM 评估。The affected fields are explicitly defined as raw content:
text | `user_input` | string (raw) | yes | | `agent_output` | string (raw) | yes | | `tool_calls` | JSON array of `{tool, args, result, ts}` | optional (empty if no tool calls) |Technical Analysis
The prompt template directly interpolates raw user input, agent output, and tool-call records into the LLM judge's user prompt. It does not establish a clear trust boundary instructing the judge that instructions appearing inside these fields are untrusted evidence and must not be followed.
When traces originate from external users, evaluated agents, automated rollouts, or tool results, an independent party can place judge-directed instructions in one of these fields. Because the injected content shares the same instruction-bearing prompt context as the legitimate evaluation request, the judge may follow it instead of objectively applying the rubric.
The issue is reachable through the documented workflow:
SKILL.mddirects consumers to pass each agent turn through this prompt, persist the results, and optionally use the scores with runtime guard or optimization systems. No executable script enforces delimiting, escaping, output validation, or injection resistance.Attack Path
- An attacker supplies content that becomes part of an evaluated trace, such as the original
user_input, an evaluatedagent_output, or a tool result. - The content includes judge-directed instructions, for example, a demand to ignore the rubric and retur ...[truncated 1208 chars]
- An attacker supplies content that becomes part of an evaluated trace, such as the original
- Remediation
View remediation
Remediation Suggestions
- Add an explicit high-priority instruction stating that all text inside evaluated trace fields is untrusted data and that any instructions contained there must be ignored.
- Enclose each field in distinct, randomized or structurally encoded boundaries and identify it solely as evidence to evaluate.
- Prefer a structured API representation in which trace data is passed as validated JSON rather than concatenated into free-form prompt text.
- Validate the judge response against a strict schema:
- Require only the documented fields.
- Require numeric scores between
0.0and1.0. - Restrict
primary_failure_modeto the documented enumeration. - Reject unexpected text or additional fields.
- Add rule-based detection for common prompt-injection patterns and flag affected evaluations for human review rather than trusting their scores automatically.
- Do not allow a single LLM judgment to autonomously update policies, guard thresholds, or reward models. Retain the documented human-in-the-loop control and require independent validation for security-sensitive decisions.
- Add adversarial tests containing judge-directed instructions in every raw field and verify that the judge evaluates those instructions as trace content rather than obeying them.