T09 · Insecure Skill Coding Practices
- Location
scripts/trigger_eval.py:97- Finding
Untrusted Skill Descriptions Execute as Instructions in a Tool-Enabled Agent
- Content
View full analysis
Vulnerability Details
File Location:
scripts/trigger_eval.py, lines 97–109 and 122–148
Vulnerability Type: Untrusted content executed as agent instructions without adequate sandboxing
Risk Level: HighVulnerable Code
python def _write_probe(skills_root: Path, name: str, description: str) -> None: """把一段 description 装成一个匿名的 project 级 skill。""" d = skills_root / name d.mkdir(parents=True, exist_ok=True) one_line = " ".join(description.split()) (d / "SKILL.md").write_text( f"---\nname: {name}\ndescription: {one_line}\n---\n\n" f"# {name}\n\n本 skill 负责：{one_line}\n", encoding="utf-8", )python project_root = tempfile.mkdtemp(prefix=f"trigeval-{unique_id}-", dir=root_base) skills_root = Path(project_root) / ".claude" / "skills" try: _write_probe(skills_root, probe_name, description) for i, alt_desc in enumerate(distractors or [], start=1): _write_probe(skills_root, f"alt-{unique_id}-{i}", alt_desc) cmd = [ "claude", "-p", query, "--output-format", "stream-json", "--verbose", "--include-partial-messages", # 只加载 project 级设置，隔离掉已装的真 skill（第 1 条） "--setting-sources", "project", # 只关心触没触发，不需要模型真把活干完 "--disallowedTools", "Write", "Edit", "NotebookEdit", ] if model: cmd.extend(["--model", model]) # 剥掉 CLAUDECODE，否则嵌套调用被守卫拦下 env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"} proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, cwd=project_root, env=env)Technical Analysis
The evaluator accepts a Skill directory through
--skill-path, extracts its frontmatter description, and copies that description into a newly generated probe Skill. The attacker-controlled description is placed in both the probe frontmatter and its instruction body.The probe is then loaded by a real `cla ...[truncated 2658 chars]
- Remediation
View remediation
Remediation Suggestions
-
Do not execute untrusted descriptions as Skill instructions.
- Keep the tested description as inert data.
- Generate a fixed, trusted probe body that contains no content copied from the audited Skill.
- If the host requires the description in frontmatter for discovery testing, ensure it is never repeated in an instruction body.
-
Apply a strict tool allowlist.
- Permit only the minimum capability required to observe whether the probe was selected.
- Explicitly deny shell, filesystem reads, network access, MCP tools, and all other unrelated tools.
- Do not rely on a short denylist because new or host-specific tools may remain available.
-
Sanitize the subprocess environment.
- Construct a minimal environment from an explicit allowlist rather than inheriting
os.environ. - Remove API keys, cloud credentials, SSH-related variables, proxy credentials, and unrelated service tokens.
- Supply authentication needed for the evaluator through an isolated, least-privileged account.
- Construct a minimal environment from an explicit allowlist rather than inheriting
-
Use operating-system isolation.
- Run each evaluation in a disposable container or equivalent sandbox.
- Mount no host directories other than explicitly required inert inputs.
- Use a non-root user, a read-only filesystem, process restrictions, and resource limits.
- Disable network access unless it is indispensable to the evaluation.
-
Add a trust warning and safe default.
- Clearly state that third-party Skill descriptions are untrusted instructions.
- Require an explicit unsafe-mode option before evaluating untrusted Skills without full sandboxing.
- Default to static analysis when adequate isolation is unavailable.
-
Add adversarial regression tests.
- Test descriptions that request shell execution, environment inspection, sensitive-file reads, and network transmission.
- Verify that none of these actions can occur even when the probe is selected.
-