T09 · Insecure Skill Coding Practices
- Location
scrapers/aibase.py:67- Finding
Server-Side Request Forgery Through Unvalidated Article URLs
- Content
View full analysis
Vulnerability Details
File Location:
scrapers/aibase.py:67-83,scrapers/base.py:53-58, and request invocation atscrapers/base.py:91
Vulnerability Type: Server-Side Request Forgery (SSRF)
Risk Level: HighVulnerable Code
scrapers/aibase.py:67-83:python url = link_elem.get("href", "") # 保存原始URL用于去重 original_url = url if not url.startswith("http"): # 确保使用中文版URL获取内容 if "/zh" not in url: url = "/zh" + url url = "https://www.aibase.com" + url # 原始URL用于去重 original_url = "https://www.aibase.com" + original_url else: # 如果是完整URL，确保也获取中文版本 if "/zh" not in url: url = url.replace("www.aibase.com/news", "www.aibase.com/zh/news") original_url = urlscrapers/base.py:53-58:python try: resp = self.session.get(url, timeout=timeout) resp.raise_for_status() resp.encoding = resp.apparent_encoding return resp.text except Exception as e:The resulting URL is requested during detail-page processing at
scrapers/base.py:91:python detail_html = self.fetch(item["url"])Technical Analysis
Article URLs are extracted from the remotely supplied listing page. Relative URLs are attached to the expected AIBase origin, but absolute URLs beginning with
httpare accepted without validating their scheme, hostname, port, or resolved IP address.Calling
str.replace()for the expected AIBase path is not an origin validation mechanism. An absolute URL hosted on an unrelated domain remains usable if the target substring is absent. Furthermore,requests.Session.get()follows HTTP redirects by default, and redirect destinations are not validated.Consequently, an attacker who can influence a listing link or an intermediate redirect can cause the scraper to issue requests from the environment in which the Skill runs. Potential targets include loopback services, private network addresses, link-l ...[truncated 1385 chars]
- Remediation
View remediation
Remediation Suggestions
- Parse every URL with
urllib.parse.urlsplit()and permit onlyhttps. - Require the normalized hostname to exactly match an explicit allowlist, such as
www.aibase.com. Do not use substring or suffix checks that can be bypassed by domains such aswww.aibase.com.attacker.example. - Reject URLs containing unexpected credentials, ports, malformed hostnames, or nonstandard schemes.
- Resolve the hostname and reject every result belonging to loopback, private, link-local, multicast, unspecified, or reserved IP ranges. Apply equivalent checks to IPv4 and IPv6.
- Disable automatic redirects with
allow_redirects=False, or validate the scheme, hostname, port, and resolved addresses of every redirect destination before following it. - Repeat validation immediately before each request to reduce DNS rebinding and time-of-check/time-of-use risk.
- Apply strict response-size limits to prevent untrusted endpoints from causing excessive memory consumption.
- Where possible, enforce outbound network restrictions outside the application so the process cannot reach metadata, loopback, or private-network services.
- Parse every URL with