T09 · Insecure Skill Coding Practices
Error
- Location
scripts/web_relevance_extract.py:117- Finding
Unrestricted URL Fetching Enables Server-Side Request Forgery
- Content
View full analysis
str: parsed = urlparse(source) if parsed.scheme in {"http", "https"}: request = Request( source, headers={ "User-Agent": "Mozilla/5.0 (compatible; web-relevance-extract/1.0)" }, ) try: with urlopen(request, timeout=timeout) as response: charset = response.headers.get_content_charset() or "utf-8" return response.read().decode(charset, errors="replace") except URLError as exc: raise SystemExit(f"Failed to fetch {source}: {exc}") from exc ``` ### Technical Analysis The source validation only checks whether the supplied URL uses the `http` or `https` scheme. It does not validate the destination hostname or resolved IP address. Consequently, the extractor can request: - Loopback services such as `127.0.0.1` or `::1` - RFC1918 private-network services - Link-local addresses - Cloud instance metadata services - Reserved or otherwise non-public IP ranges - Public URLs that redirect to any of the preceding destinations Python's `urlopen` follows HTTP redirects by default. A public attacker-controlled endpoint can therefore redirect the request to an internal destination even if validation of only the original hostname were later added. The fetched response is parsed and relevant portions are emitted through standard output. An attacker can choose or refine the relevance query to target expected words in an internal response. This creates a response-disclosure SSRF path rather than merely a blind request primitive. Fetching public webpages is necessary for the declared functionality. Access to internal and privileged network destinations is not necessary and exceeds ...[truncated 1608 chars]- Remediation
View remediation