T09 · Insecure Skill Coding Practices
Error
- Location
- smart_research.py:1055
- Finding
- Unrestricted URL Fetching Enables SSRF and Disclosure of Sensitive URLs to Third Parties<![CDATA[ ## Vulnerability Details **File Location**: `smart_research.py:494-500`, `smart_research.py:544-551`, `smart_research.py:589-596`, `smart_research.py:659-681`, `smart_research.py:1055-1058` **Vulnerability Type**: Server-Side Request Forgery and sensitive URL disclosure **Risk Level**: High ### Vulnerable Code The externally supplied URL is accepted without validation and passed directly to the fallback fetch chain: ```python elif action == "fetch": url = input_data.get("url", "") if not url: return {"error": "url 不能为空"} fetch_result = fetch_with_fallback(url) ``` The fallback chain attempts the URL using local browser-capable fetchers and external extraction services: ```python def fetch_with_fallback(url: str) -> FetchResult: """ 多级降级抓取 降级顺序：crawl4ai → jina → markdown_new → defuddle → playwright """ fetchers = [ ("crawl4ai", fetch_crawl4ai, 15), ("jina", fetch_jina, 10), ("markdown_new", fetch_markdown_new, 8), ("defuddle", fetch_defuddle, 8), ("playwright", fetch_playwright, 30), ] last_error = None for name, fn, timeout in fetchers: logger.debug(f"[{name}] 尝试抓取: {url}") result = fn(url, timeout) if result.is_success: logger.info(f"[{name}] 成功抓取: {url} ({result.fetch_time_ms}ms)") return result last_error = result.error logger.debug(f"[{name}] 失败，降级: {last_error}") return FetchResult( url=url, success=False, error=last_error or "所有抓取方式均失败", fetcher_name="none", ) ``` The complete URL is subsequently embedded in requests to external services: ```python resp = requests.get( f"https://r.jina.ai/{url}", headers=headers, timeout=timeout, ) ``` ```python resp = requests.get( f"https://markdown.new/{url}", headers={ "Accept": "text/markdown", "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/53 ...[truncated 3409 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Enforce an explicit URL policy before invoking any fetcher: - Permit only `http` and `https`. - Reject missing or malformed hostnames. - Reject URLs containing embedded usernames or passwords. - Reject nonstandard ports unless explicitly required and allowlisted. - Normalize the hostname before validation. 2. Resolve the hostname and reject every address in non-public ranges, including: - Loopback. - Private-use networks. - Link-local networks. - Multicast. - Reserved and unspecified addresses. - IPv6 unique-local and IPv4-mapped IPv6 forms. 3. Revalidate every redirect destination before following it. Disable automatic redirects where necessary and process each `Location` header through the same policy. 4. Protect against DNS rebinding by ensuring the address used for the connection is the same validated public address, or by enforcing the restriction through an outbound proxy or network sandbox. 5. Disable third-party extraction fallbacks by default. Require explicit, informed user consent before forwarding a URL to Jina Reader, markdown.new, or defuddle. 6. Remove embedded credentials and redact known sensitive query parameters before any external forwarding. Prefer sending only URLs that are already public and contain no secrets. 7. Apply infrastructure-level egress controls that prevent the Skill process from contacting loopback, private, link-local, metadata, and internal network ranges. 8. Update the privacy documentation to identify every external service, what data it receives, and when fallback forwarding occurs. ]]>