T09 · Insecure Skill Coding Practices
Warning
- Location
- scripts/ebook_to_md.py:230
- Finding
- Unrestricted Fetching of Server-Supplied URLs Enables SSRF<![CDATA[ ## Vulnerability Details **File Location**: `scripts/ebook_to_md.py:230-269` and `scripts/ebook_to_md.py:437-466` **Vulnerability Type**: Server-Side Request Forgery (SSRF) and unrestricted resource retrieval **Risk Level**: Medium ### Vulnerable Code ```python def _download_markdown(markdown_url: str) -> str: resp = requests.get(markdown_url) resp.raise_for_status() return resp.text def _download_parse_result_json(parse_result_url: str) -> dict: resp = requests.get(parse_result_url) resp.raise_for_status() return json.loads(resp.content.decode("utf-8", errors="replace")) def _detect_image_mime(raw: bytes) -> str: if raw[:2] == b"\xff\xd8": return "image/jpeg" if raw[:8] == b"\x89PNG\r\n\x1a\n": return "image/png" if raw[:6] in (b"GIF87a", b"GIF89a"): return "image/gif" if raw[:2] == b"BM": return "image/bmp" if len(raw) > 12 and raw[:4] == b"RIFF" and raw[8:12] == b"WEBP": return "image/webp" return "image/jpeg" def _fetch_image_raw(url: str): resp = requests.get(url) resp.raise_for_status() raw = resp.content mime = _detect_image_mime(raw) ext_map = { "image/jpeg": ".jpg", "image/png": ".png", "image/gif": ".gif", "image/bmp": ".bmp", "image/webp": ".webp", } return raw, ext_map.get(mime, ".jpg") ``` The image-fetching functions are subsequently invoked for URLs extracted from downloaded Markdown: ```python def _inline_images_as_base64(md_content: str) -> str: def repl(m): url = m.group(1).strip() try: return "".format(_fetch_image_as_base64(url)) except Exception as e: return m.group(0) + " <!-- Download failed: {} -->".format(e) return IMG_SRC_PATTERN.sub(repl, md_content) def _inline_images_as_local(md_content: str, output_path: Path) -> str: out_dir = output_path.parent images_dir = output_path.stem + "_image ...[truncated 3929 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. **Allowlist trusted destinations** - Permit only HTTPS URLs on explicitly approved Baidu-owned result and asset hosts. - Do not accept arbitrary hosts merely because the original API request went to Baidu. 2. **Validate every URL** - Parse URLs with `urllib.parse.urlsplit`. - Reject unsupported schemes, embedded credentials, malformed ports, fragments where inappropriate, and non-HTTPS URLs. - Reject loopback, private, link-local, multicast, reserved, and unspecified IP ranges using the `ipaddress` module. 3. **Defend against DNS rebinding** - Resolve the hostname before connecting and validate every returned address. - Ensure the connection is made to a validated address while preserving correct TLS hostname verification. - Revalidate the destination after every redirect. 4. **Restrict redirects** - Disable automatic redirects or process them manually. - Apply the same scheme, hostname, port, and resolved-address validation to every redirect target. - Enforce a low redirect limit. 5. **Apply resource controls** - Set explicit connection and read timeouts. - Stream responses rather than reading them into memory at once. - Enforce maximum response sizes for Markdown, JSON, and images. - Validate `Content-Type` and image signatures before embedding or writing content. 6. **Constrain image sources** - Prefer image URLs explicitly returned as structured fields by the trusted OCR API. - Do not automatically fetch arbitrary `<img src>` values from generated Markdown. - If arbitrary image retrieval is required, expose it as an opt-in mode with a clear security warning and strict network isolation. 7. **Add security tests** - Test rejection of loopback, RFC1918, link-local, IPv6 private, and metadata-service addresses. - Test redirects from an allowed host to a prohibited host. - Test DNS rebinding scenarios and oversized or slow responses. ]]>