T09 · Insecure Skill Coding Practices
Error
- Location
- scripts/verify_links.py:113
- Finding
- Unrestricted URL Fetching Enables Server-Side Request Forgery<![CDATA[ ## Vulnerability Details **File Locations**: - `scripts/verify_links.py:113-130, 221-228` - `scripts/serp_gap_analyzer.py:111-136, 225-232, 310-316` - `scripts/post_publish_check.py:59-63, 75-79` **Vulnerability Type**: Server-Side Request Forgery (SSRF) through unvalidated URLs **Risk Level**: High ### Vulnerable Code `scripts/verify_links.py:113-130`: ```python def curl_head(url: str) -> str: cmd = [ "curl", "-sI", "-o", "/dev/null", "-w", "%{http_code}", "--max-time", "10", "-L", "-A", USER_AGENT, url, ] try: out = subprocess.check_output(cmd, text=True).strip() return out or "000" except Exception: return "000" def fetch(url: str) -> tuple[str, str, str]: req = Request(url, headers={"User-Agent": USER_AGENT}) try: with urlopen(req, timeout=TIMEOUT) as resp: body = resp.read(120000).decode("utf-8", errors="ignore") return str(resp.status), resp.geturl(), body ``` `scripts/verify_links.py:221-228`: ```python def verify_url(url: str) -> LinkResult: domain = normalize_domain(url) http_status = curl_head(url) result = LinkResult(url=url, domain=domain, http_status=http_status) if http_status in {"404", "410"}: result.verdict = "dead" result.evidence = f"HTTP {http_status}" ``` `scripts/serp_gap_analyzer.py:111-136`: ```python def fetch(url: str): req = Request(url, headers={"User-Agent": USER_AGENT}) try: with urlopen(req, timeout=TIMEOUT) as resp: body = resp.read(180000).decode("utf-8", errors="ignore") return str(resp.status), resp.geturl(), body except HTTPError as e: body = e.read(180000).decode("utf-8", errors="ignore") if hasattr(e, "read") else "" return str(e.code), url, body except URLError: return "000", url, "" except Exception: return "000", url, "" def jina_fetch(url: str) -> str: """Fetch a URL via Jina Reader (r.ji ...[truncated 4128 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Permit only `http` and `https` URLs and reject URLs containing user information. 2. Resolve the hostname before each request and reject every address classified as loopback, private, link-local, multicast, unspecified, or reserved. 3. Explicitly block common metadata destinations, including `169.254.169.254`, even if other address checks are present. 4. Disable automatic redirects and validate each redirect target before following it. 5. Re-resolve and revalidate each destination immediately before connecting to reduce DNS-rebinding risk. 6. Restrict destination ports to an approved set such as 80 and 443 unless the user explicitly authorizes another port. 7. Consider an optional domain allowlist for controlled publishing environments. 8. Run network checks in a sandbox with no access to cloud metadata, localhost services, or private application networks. 9. Apply the same validation helper consistently to `curl`, `urlopen`, Jina target URLs, and all direct-fetch fallbacks. 10. Add regression tests for direct private IPs, IPv6 loopback, encoded addresses, DNS rebinding, and public-to-private redirects. ]]>