T09 · Insecure Skill Coding Practices
- Location
scripts/extract.py:36- Finding
Unrestricted URL Fetching Enables SSRF and Local File Disclosure
- Content
View full analysis
Vulnerability Details
File Location:
scripts/extract.py, lines 36–45 and 77–80
Vulnerability Type: Server-Side Request Forgery (SSRF) and unrestricted URI scheme access
Risk Level: HighVulnerable Code
python def fetch_url(url): """抓取网页内容""" headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36' } req = urllib.request.Request(url, headers=headers) try: with urllib.request.urlopen(req, timeout=30) as response: return response.read().decode('utf-8', errors='ignore') except Exception as e: print(f"Error fetching URL: {e}", file=sys.stderr) return NoneThe untrusted value originates in
main():python url = sys.argv[1] print(f"Fetching: {url}", file=sys.stderr) html = fetch_url(url)Technical Analysis
A URL supplied through the command line is passed directly to
urllib.request.Requestandurllib.request.urlopenwithout validation of its scheme, hostname, resolved IP address, port, or redirect destination.This creates an SSRF primitive because the request executes with the process's network access rather than the caller's access. An attacker can submit URLs targeting loopback interfaces, private networks, link-local addresses, internal services, or cloud instance metadata endpoints. DNS-based checks would also be insufficient unless the resolved address used for the connection is validated against prohibited address ranges.
The implementation does not restrict input to HTTP and HTTPS. Python's
urllibcan handle additional URI schemes, includingfile://, allowing files readable by the process to be opened and returned. HTTP redirects are followed by default, so validating only the initial URL would not prevent a public endpoint from redirecting to a prohibited local or internal destination.The retrieved response is decoded, processed as text, ...[truncated 1493 chars]
- Remediation
View remediation
Remediation Suggestions
- Parse URLs with
urllib.parse.urlsplitand permit only explicithttpandhttpsschemes. - Reject URLs containing embedded credentials, malformed hostnames, fragments where inappropriate, or ports outside an approved set.
- Resolve the hostname before connecting and reject every address that is loopback, private, link-local, multicast, reserved, or unspecified, for both IPv4 and IPv6.
- Explicitly block cloud metadata destinations and equivalent metadata hostnames. Do not rely solely on hostname deny lists.
- Disable automatic redirects or implement a redirect handler that repeats all scheme, hostname, port, DNS, and IP-range checks for every redirect destination.
- Mitigate DNS rebinding by ensuring the validated address is the address used for the connection, or by routing requests through a controlled outbound proxy with equivalent restrictions.
- Apply response-size and redirect-count limits to reduce memory exhaustion and denial-of-service risk.
- Enforce outbound network policy at the operating-system, container, or proxy layer so the process cannot reach private networks or metadata services even if application validation is bypassed.
- Add regression tests covering
file://inputs, loopback and private IPv4/IPv6 addresses, encoded IP representations, metadata endpoints, DNS rebinding scenarios, and redirects to prohibited destinations.
- Parse URLs with