T09 · Insecure Skill Coding Practices
Error
- Location
- scripts/crawl.py:45
- Finding
- Unrestricted User-Controlled URL Fetching Enables Server-Side Request Forgery<![CDATA[ ## Vulnerability Details **File Location**: `scripts/crawl.py:45-68, 233-239, 296-300, 336-337, 382-395` **Vulnerability Type**: Server-Side Request Forgery (SSRF) caused by insufficient URL and redirect validation **Risk Level**: High ### Vulnerable Code ```python def fetch(url: str): """Fetch URL, return (html_text, final_url) or (None, None) on failure.""" try: r = httpx.get(url, headers=HEADERS, timeout=TIMEOUT, follow_redirects=True) if r.status_code == 200 and "text/html" in r.headers.get("content-type", ""): return r.text, str(r.url) else: print(f" [skip] {url} → HTTP {r.status_code}", file=sys.stderr) return None, None except Exception as e: print(f" [error] {url} → {e}", file=sys.stderr) return None, None def fetch_text_file(url: str): """Fetch a plain text file (e.g. existing llms.txt).""" try: r = httpx.get(url, headers=HEADERS, timeout=TIMEOUT, follow_redirects=True) if r.status_code == 200: return r.text return None except Exception: return None ``` User-controlled root and extra URLs are passed directly to these functions: ```python level1_urls = [root_url] + extra_urls level1_pages = {} for url in level1_urls: html, final_url = fetch(url) if html: level1_pages[final_url or url] = extract_page(html, final_url or url) print(f" ✓ {url}", file=sys.stderr) time.sleep(0.5) ``` Discovered links are also fetched without validating the resolved destination: ```python for link in level2_targets: print(f" → Crawling: {link['text']} ({link['url']})", file=sys.stderr) html, final_url = fetch(link["url"]) if html: page = extract_page(html, final_url or link["url"]) ``` The command-line inputs are accepted without destination restrictions: ```python args = sys.argv[1:] deep = "--deep" in args args = [a for a in args if a != "--deep"] root_url = ar ...[truncated 3401 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Parse every URL with `urllib.parse.urlsplit` and allow only explicit `http` and `https` schemes. 2. Reject URLs containing usernames or passwords and reject ports outside an approved set, normally 80 and 443. 3. Resolve the hostname before connecting and reject every result belonging to loopback, private, link-local, multicast, reserved, or unspecified address ranges using Python's `ipaddress` module. 4. Apply equivalent checks to IPv4, IPv6, IPv4-mapped IPv6 addresses, and alternate textual IP representations. 5. Disable automatic redirects. Follow redirects manually only after validating each new URL and its resolved addresses. 6. Defend against DNS rebinding by ensuring that the validated address is the address actually used for the connection, or by using a hardened outbound proxy with destination policies. 7. Restrict discovered links and additional sources to the approved registrable domain unless the user explicitly authorizes a different public domain. 8. Consider an explicit allowlist when the Skill runs in environments with access to sensitive internal networks. 9. Set strict response-size limits and stream responses rather than loading arbitrary response bodies into memory. 10. Ensure crawler output is treated as untrusted data and obtain confirmation before transmitting unexpectedly sensitive content to an external LLM provider. 11. Add tests covering loopback addresses, RFC 1918 ranges, link-local addresses, cloud metadata addresses, IPv6 local addresses, redirect chains, and DNS rebinding scenarios. ]]>