T09 · Insecure Skill Coding Practices
Error
- Location
- scripts/extract_page_images.py:15
- Finding
- Unrestricted URL Fetching Enables Server-Side Request Forgery<![CDATA[ ## Vulnerability Details **File Location**: `scripts/extract_page_images.py:15-16`; `scripts/vision_pipeline.py:66`; `scripts/webctx_verify.py:63-79`; `scripts/source_router.py:103-108` **Vulnerability Type**: Server-Side Request Forgery (SSRF) **Risk Level**: High ### Vulnerable Code `scripts/extract_page_images.py:15-16`: ```python r = requests.get(url, headers={"User-Agent": UA, "Accept-Language": "zh-CN,zh;q=0.9"}, timeout=15) ``` `scripts/vision_pipeline.py:66`: ```python r = requests.get(url, headers=headers, timeout=timeout, stream=True) ``` `scripts/webctx_verify.py:63-79`: ```python if not page_url or not page_url.startswith("http"): return None from playwright.sync_api import sync_playwright out_dir = Path(out_dir) out_dir.mkdir(parents=True, exist_ok=True) with sync_playwright() as pw: browser = pw.chromium.launch( executable_path=exe, headless=True, args=["--no-sandbox", "--disable-blink-features=AutomationControlled"]) ctx = browser.new_context(user_agent=se.UA, locale="zh-CN", viewport={"width": viewport_w, "height": viewport_h}) ctx.add_init_script( "Object.defineProperty(navigator, 'webdriver', {get: () => undefined})") pg = ctx.new_page() try: pg.goto(page_url, timeout=40000, wait_until="domcontentloaded") ``` `scripts/source_router.py:103-108`: ```python for d in domains: for scheme in ("https", "http"): base = f"{scheme}://{d}" r = _get(base, timeout=12) if not r: continue ``` ### Technical Analysis The Skill fetches user-controlled or externally derived URLs without validating the destination network address. Relevant input channels include: - Direct URLs passed to `extract_page_images.py`. - Values supplied through `--extra-urls`. - URLs imported through `--extra-file`. - Domains supplied through `source_router.py --domains`. - Source-page URLs stored in candidate metadata and s ...[truncated 2062 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Accept only normalized `https` URLs unless plain HTTP is explicitly required. 2. Parse URLs with `urllib.parse.urlsplit` and reject: - Embedded usernames or passwords. - Missing or malformed hostnames. - Unsupported schemes. - Unexpected ports. 3. Resolve all destination hostnames before connecting. 4. Reject every resolved IPv4 and IPv6 address belonging to: - Loopback ranges. - Private ranges. - Link-local ranges. - Multicast ranges. - Reserved or unspecified ranges. - Known cloud metadata addresses. 5. Disable automatic redirects or validate each redirect target using the same policy. 6. Apply destination allowlists for fixed-purpose integrations such as App Store, Weibo, Baidu, and Bing. 7. Revalidate DNS immediately before connection to reduce DNS-rebinding exposure. 8. Restrict outbound network access at the container or host firewall layer as defense in depth. 9. Treat candidate and selected JSON files as untrusted input and revalidate every URL when consumed. ]]>
