T09 · Insecure Skill Coding Practices
Warning
- Location
- scripts/fetch_leaderboard.py:23
- Finding
- Unsanitized Remote Leaderboard Content Propagates to Agent-Visible Output<![CDATA[ ## Vulnerability Details **File Location**: `scripts/fetch_leaderboard.py:23-49, 81-94`; related output instruction at `SKILL.md:44` **Vulnerability Type**: Untrusted remote content injection caused by insufficient schema validation and output sanitization **Risk Level**: Medium ### Vulnerable Code ```python def fetch_entries(): result = subprocess.run( ["curl", "-s", "-L", "--max-time", "15", URL], capture_output=True, text=True, ) if result.returncode != 0: print(f"ERROR: curl failed: {result.stderr.strip()}", file=sys.stderr) sys.exit(1) html = result.stdout chunks = re.findall(r'self\.__next_f\.push\(\[1,"(.*?)"\]\)', html, re.DOTALL) for chunk in chunks: chunk = chunk.replace("\\n", "\n").replace('\\"', '"') m = re.search(r'"entries":\[(\{.*)', chunk) if not m: continue data = m.group(0) depth = 0 end = 0 for i, c in enumerate(data): if c == "[": depth += 1 elif c == "]": depth -= 1 if depth == 0: end = i + 1 break arr_str = data[len('"entries":') : end] return json.loads(arr_str) print("ERROR: could not parse leaderboard data from pinchbench.com", file=sys.stderr) sys.exit(1) ``` ```python if args.json: json.dump(entries, sys.stdout, indent=2) return print(f"{'#':>3} {'Model':<45} {'Score':>6} {'Cost':>7} {'Time':>6} {'Runs':>4}") print(f"{'─'*3} {'─'*45} {'─'*6} {'─'*7} {'─'*6} {'─'*4}") for i, e in enumerate(entries, 1): print( f"{i:>3} {e['model']:<45} {e['percentage']:>5.1f}% " f"${e['average_cost_usd']:>6.2f} " f"{e['average_execution_time_seconds']:>5.0f}s " f"{e['submission_count']:>4}" ) ``` The Skill additionally instructs the Agent to reproduce this output without sanitization: ```markdown Present the output as-is ...[truncated 3040 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. **Enforce a strict schema before sorting or rendering** - Require every entry to be an object. - Allow only the expected fields: `model`, `percentage`, `average_cost_usd`, `average_execution_time_seconds`, and `submission_count`. - Verify that `model` is a string and that numeric fields have the expected finite numeric types. - Reject malformed entries instead of allowing them to reach output formatting. 2. **Sanitize remote strings** - Remove ASCII control characters, carriage returns, newlines, terminal escape sequences, and other non-printable characters. - Normalize Unicode where appropriate. - Apply a reasonable maximum length to model names. 3. **Restrict JSON output** - Construct a new dictionary containing only allowlisted fields instead of serializing each upstream object in full. - Do not propagate unexpected remote fields into Agent-visible output. 4. **Improve the Skill instruction** - Replace “Present the output as-is” with an instruction stating that fetched content is untrusted data. - Require the Agent to quote or summarize remote values and never interpret them as instructions. 5. **Handle malformed upstream data safely** - Catch JSON decoding, missing-key, formatting, and type errors. - Return a controlled error message without reproducing the malformed remote content. An example sanitization approach is: ```python import math import re CONTROL_CHARS = re.compile(r"[\x00-\x1f\x7f-\x9f]") def validate_entry(entry): if not isinstance(entry, dict): raise ValueError("Leaderboard entry must be an object") model = entry.get("model") percentage = entry.get("percentage") cost = entry.get("average_cost_usd") execution_time = entry.get("average_execution_time_seconds") submissions = entry.get("submission_count") if not isinstance(model, str): raise ValueError("Invalid model name") model = CONTROL_CHARS.sub("", model).strip ...[truncated 644 chars]