T09 · Insecure Skill Coding Practices
Error
- Location
scripts/fetch_external_sources.py:162- Finding
Redirect-Based Server-Side Request Forgery in External Source Fetcher
- Content
View full analysis
Vulnerability Details
File Location:
scripts/fetch_external_sources.py:162-179; related validation inscripts/url_safety.py:19-59
Vulnerability Type: Server-Side Request Forgery through automatically followed redirects
Risk Level: HighVulnerable Code
python def fetch_note(url: str, *, timeout: int) -> dict[str, str]: assert_public_http_url(url) request = Request( url, headers={ "User-Agent": "SopaperEvidenceBot/0.6 (+https://github.com/sheepxux/SoPaper-Evidence)" }, ) try: with urlopen(request, timeout=timeout) as response: content_type = response.headers.get("Content-Type", "") raw = response.read().decode("utf-8", errors="replace") except HTTPError as exc: if exc.code in {301, 302, 303, 307, 308} and exc.headers.get("Location"): redirected = urljoin(url, exc.headers["Location"]) assert_public_http_url(redirected) return fetch_note(redirected, timeout=timeout) raiseThe URL validator itself checks the supplied hostname and its currently resolved addresses:
python def assert_public_http_url(url: str) -> None: parsed = urlparse(url) if parsed.scheme not in {"http", "https"}: raise ValueError("only http and https URLs are allowed") if not parsed.hostname: raise ValueError("URL must include a hostname") if parsed.username or parsed.password: raise ValueError("URLs with embedded credentials are not allowed") host = parsed.hostname.strip().lower().rstrip(".") if host in BLOCKED_HOSTS or host.endswith(".localhost") or host.endswith(".local"): raise ValueError(f"local host is not allowed: {host}") try: if not is_public_ip(host): raise ValueError(f"non-public IP address is not allowed: {host}") return except ValueError a ...[truncated 3123 chars]- Remediation
View remediation
Remediation Suggestions
- Disable the default automatic redirect handler.
- Process redirects manually and call
assert_public_http_urlon every resolved redirect target before connecting. - Enforce a small redirect limit, such as three redirects, and reject redirect loops.
- Reject HTTPS-to-HTTP downgrade redirects.
- Address DNS rebinding by connecting to a validated and pinned public IP while preserving the original hostname for TLS certificate and HTTP host validation, preferably through a mature SSRF-resistant HTTP client.
- Revalidate the peer address after connection where the networking library permits it.
- Apply outbound firewall rules that deny loopback, private, link-local, multicast, and reserved address ranges.
- Add regression tests covering direct private URLs, public-to-private redirects, relative redirects, redirect chains, mixed DNS answers, IPv4-mapped IPv6 addresses, and DNS rebinding scenarios.