T09 · Insecure Skill Coding Practices
Error
- Location
- lib/config.py:52
- Finding
- Arbitrary Network Access and Server-Side Request Forgery## Vulnerability Details **File Location**: `lib/config.py:52-68` **Related Locations**: `lib/scraper.py:70-72`, `cli.py:259-274` **Vulnerability Type**: Server-Side Request Forgery (SSRF) **Risk Level**: High ### Vulnerable Code ```python # lib/config.py:52-68 # Validate inputs if not name.strip(): raise ValueError("Profile name cannot be empty") if not base_url.strip(): raise ValueError("base_url cannot be empty") if not sitemap_url.strip(): raise ValueError("sitemap_url cannot be empty") if search_method not in ["keyword", "semantic", "hybrid"]: raise ValueError("search_method must be 'keyword', 'semantic', or 'hybrid'") self.configs[name] = { "name": name, "base_url": base_url.rstrip("/"), "sitemap_url": sitemap_url, "search_method": search_method, "cache_ttl_days": cache_ttl_days, } ``` ```python # lib/scraper.py:70-72 try: resp = self.session.get(url, timeout=10) resp.raise_for_status() ``` ```python # cli.py:259-274 # Build full URL if path doesn't start with http if path.startswith("http"): url = path else: url = f"{cfg['base_url']}/{path.lstrip('/')}" # Try cache first cached_page = config_obj.cache_mgr.get_page(url, cfg["cache_ttl_days"]) if cached_page: click.echo(f"Title: {cached_page['title']}\n") click.echo(cached_page['content']) return # Fetch fresh click.echo(f"Fetching {url}...") engine = DiscoveryEngine(cfg['base_url'], cfg['sitemap_url']) page = engine.scrape_page(url) ``` ### Technical Analysis Profile validation only verifies that URL strings are nonempty. It does not parse or constrain the URL scheme, hostname, port, embedded credentials, or resolved IP address. The `fetch` command also treats any value beginning with `http` as a directly fetchable URL. Sitemap entries are subsequently passed to the same unrestricted HTTP client. Consequently, a malicious sitemap ...[truncated 1847 chars]
- Remediation
- ## Remediation Suggestions 1. Parse URLs with `urllib.parse.urlsplit()` and permit only explicitly supported schemes, preferably `https`. 2. Reject URLs containing user information, malformed hosts, fragments where inappropriate, and nonstandard ports unless explicitly approved. 3. Resolve hostnames before every request and reject loopback, private, link-local, multicast, unspecified, and reserved IP ranges for both IPv4 and IPv6. 4. Explicitly block cloud metadata destinations, including `169.254.169.254`, even when reached through DNS aliases. 5. Require all sitemap entries and crawled pages to match the configured documentation origin or a per-profile hostname allowlist. 6. Disable automatic redirects or validate every redirect destination using the same scheme, origin, and resolved-address policy. 7. Replace string-prefix origin checks with normalized scheme/hostname/port comparisons. 8. Apply response-size and content-type limits to prevent memory or disk exhaustion. 9. Update the documentation so security claims accurately reflect the controls implemented.
