T09 · Insecure Skill Coding Practices
Warning
- Location
- summarize.py:67
- Finding
- Unrestricted URL Fetching Enables Server-Side Request Forgery<![CDATA[ ## Vulnerability Details **File Location**: `summarize.py`, lines 67–75; user-controlled input and invocation occur at lines 129 and 159 **Vulnerability Type**: Server-Side Request Forgery (SSRF) **Risk Level**: Medium ### Vulnerable Code ```python def get_url_content(url): """抓取网页正文内容""" try: headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36' } response = requests.get(url, headers=headers, timeout=30) response.encoding = response.apparent_encoding soup = BeautifulSoup(response.text, 'html.parser') ``` The URL originates from the command-line argument and is passed directly to the vulnerable function: ```python group.add_argument('--url', help="网页URL地址") ``` ```python elif args.url: text = get_url_content(args.url) source = args.url ``` ### Technical Analysis The application passes a user-controlled URL directly to `requests.get()` without validating the URL scheme, hostname, resolved IP address, destination port, or network range. Consequently, the process can be instructed to send HTTP requests to destinations that may be reachable from the host but unavailable to an external attacker, including: - Loopback services such as `127.0.0.1` or `::1` - Private network services - Link-local addresses and cloud instance metadata endpoints - Internal administrative interfaces - Services exposed on nonstandard ports The `requests` library follows redirects by default. Therefore, validating only the initial hostname would remain insufficient: an apparently public URL could redirect the request to an internal destination. DNS rebinding or hostnames resolving to prohibited addresses could similarly bypass hostname-only checks. The returned response body is parsed as HTML and incorporated into generated summaries and keywords. This provides a limited response-disclosure channel rather than a blind ...[truncated 1506 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Parse the URL before making a request and allow only explicitly supported schemes, preferably `https` and, only where necessary, `http`. 2. Reject URLs containing credentials, malformed hosts, unsupported ports, or ambiguous address representations. 3. Resolve the hostname and reject every resolved IPv4 or IPv6 address belonging to loopback, private, link-local, multicast, reserved, unspecified, or other prohibited ranges. 4. Disable automatic redirects with `allow_redirects=False`, or validate every redirect target using the same scheme, hostname, port, and resolved-address controls. 5. Defend against DNS rebinding by ensuring the validated address is the address actually used for the connection. Prefer a hardened outbound proxy or network egress policy where possible. 6. Consider an explicit domain allowlist if the expected use cases involve a limited set of trusted sources. 7. Apply network-level controls preventing the process from reaching cloud metadata endpoints, internal administrative networks, and sensitive local services. 8. Limit accepted response sizes and content types to reduce resource-exhaustion risk and avoid processing unexpected binary content. 9. Call `response.raise_for_status()` and handle failures without incorporating error pages or unintended service responses into output. 10. Add tests covering direct private addresses, IPv6 loopback, encoded IP forms, public-to-private redirects, and hostnames resolving to prohibited ranges. ]]>