T09 · Insecure Skill Coding Practices
Error
- Location
- scripts/xiaohongshu_extract.py:45
- Finding
- Unrestricted URL Fetching Enables Server-Side Request Forgery## Vulnerability Details **File Location**: `scripts/xiaohongshu_extract.py`, lines 45–51 and line 281 **Vulnerability Type**: Server-Side Request Forgery (SSRF) caused by unrestricted user-controlled URL fetching **Risk Level**: High **Vulnerable code:** ```python def fetch_url(url: str, timeout: int) -> Tuple[str, str, int]: resp = requests.get( url, allow_redirects=True, timeout=timeout, headers={"User-Agent": DEFAULT_UA}, ) return resp.url, resp.text, resp.status_code ``` The function is invoked with the unvalidated command-line URL: ```python final_url, html, status_code = fetch_url(args.url, args.timeout) ``` ### Technical Analysis The extractor passes a user-provided URL directly to `requests.get()`. It does not: - Restrict the scheme to HTTPS. - Enforce an allowlist of official Xiaohongshu hostnames. - Resolve and reject loopback, private, link-local, reserved, multicast, or unspecified IP addresses. - Validate redirect destinations. - Limit the downloaded response size. In addition, `allow_redirects=True` permits an initially acceptable public URL to redirect the request to an internal destination. Therefore, validation of only the original URL would not be sufficient. An attacker can cause the host running the Skill to send HTTP requests to destinations reachable from that host, including localhost services, private-network systems, and cloud instance metadata endpoints. Although the application normally expects an XHS page containing `window.__INITIAL_STATE__`, HTTP status codes, redirect URLs, parsing outcomes, blocker hints, and timing differences can still act as a network-discovery oracle. If a targeted response contains content in the expected initial-state format, selected response data may also be returned in the generated JSON. ### Attack Path 1. An attacker supplies a URL such as a loopback, private-network, link-local, or attacke ...[truncated 1400 chars]
- Remediation
- ## Remediation Suggestions 1. Accept only `https` URLs and reject URLs containing credentials, malformed authority components, or unsupported ports. 2. Normalize hostnames and enforce an exact allowlist of official Xiaohongshu domains. Avoid unsafe substring or suffix checks; permit either the exact approved hostname or a dot-delimited subdomain of an approved parent. 3. Resolve all destination addresses before connecting and reject loopback, private, link-local, reserved, multicast, and unspecified IPv4 and IPv6 ranges. 4. Disable automatic redirects. If redirects are required, follow them manually with a small redirect limit and repeat scheme, hostname, DNS, IP-range, and port validation for every destination. 5. Account for DNS rebinding by ensuring the validated address is the address used for the connection, or by applying equivalent network-layer egress controls. 6. Apply outbound firewall or proxy rules so the process cannot reach localhost, private networks, link-local ranges, or cloud metadata endpoints. 7. Stream responses with a strict maximum byte limit rather than loading an unbounded response into memory. 8. Use conservative connection and read timeouts, and avoid exposing unnecessary redirect or internal-network details in user-visible errors. 9. Add regression tests covering direct private addresses, IPv6 loopback, alternative IP representations, user-information URL confusion, DNS responses resolving to private addresses, and public-to-private redirects.