T09 · Insecure Skill Coding Practices
Error
- Location
scripts/scrapling_fetch.py:243- Finding
Unrestricted URL Fetching Enables Server-Side Request Forgery
- Content
View full analysis
Vulnerability Details
File Location:
scripts/scrapling_fetch.py, lines 243-266
Vulnerability Type: Server-Side Request Forgery (SSRF)
Risk Level: HighVulnerable Code
python parser.add_argument("url", help="目标网页 URL") parser.add_argument("max_chars", nargs="?", type=int, default=30000, help="最大输出字符数（默认: 30000）") parser.add_argument("--mode", choices=["basic", "stealth", "auto"], default="auto", help="抓取模式: basic（快速）/ stealth（隐身）/ auto（自动检测）") parser.add_argument("--json", action="store_true", help="JSON 格式输出") parser.add_argument("--debug", action="store_true", help="显示调试信息") args = parser.parse_args() # 确定抓取模式 if args.mode == "auto": mode = "stealth" if needs_stealth_mode(args.url) else "basic" if args.debug: print(f"[DEBUG] 自动选择模式: {mode}", file=sys.stderr) else: mode = args.mode try: # 抓取页面 if mode == "stealth": html, selector, is_wechat = fetch_stealth(args.url, args.debug) else: html, selector, is_wechat = fetch_basic(args.url, args.debug)Technical Analysis
The command-line caller has complete control over
args.url, which is passed to eitherfetch_stealthorfetch_basicwithout security validation. The implementation does not:- Restrict URLs to the expected
httpandhttpsschemes. - Reject loopback, private, link-local, multicast, reserved, or unspecified IP addresses.
- Resolve hostnames and validate all returned IP addresses.
- Revalidate redirect destinations.
- Enforce an approved-domain allowlist.
- Prevent DNS rebinding between validation and connection.
Consequently, when the Skill runs in an environment with access to internal networks, local services, or cloud metadata endpoints, an attacker can use it as an SSRF primitive.
Attack Path
- An attacker supplies a URL targeting a non-public resource, such as a loopback service, priv ...[truncated 1483 chars]
- Restrict URLs to the expected
- Remediation
View remediation
Remediation Suggestions
- Parse URLs with a standards-compliant URL parser and permit only
httpandhttps. - Reject URLs containing credentials or ambiguous hostname representations.
- Resolve the destination hostname before connecting and reject every address classified as loopback, private, link-local, multicast, reserved, or unspecified.
- Apply the same checks to IPv4, IPv6, IPv4-mapped IPv6 addresses, and alternative numeric address forms.
- Disable automatic redirects or validate the destination after every redirect before issuing the next request.
- Prefer an explicit allowlist of approved domains when the expected scraping scope is known.
- Mitigate DNS rebinding by connecting to a validated resolved address while preserving the intended hostname for TLS verification and the HTTP
Hostvalue. - Enforce outbound network restrictions at the container, firewall, or proxy layer so the process cannot reach metadata, loopback, or private network services.
- Add automated tests covering loopback addresses, RFC 1918 networks, IPv6 local addresses, cloud metadata addresses, encoded IP representations, redirects, and DNS rebinding scenarios.
- Parse URLs with a standards-compliant URL parser and permit only