T09 · Insecure Skill Coding Practices
Warning
- Location
scripts/make_digest.py:89- Finding
Server-Side Request Forgery Through Unvalidated arXiv URLs
- Content
View full analysis
str | None: """Best-effort: fetch arXiv abs page and extract the first 1–2 sentences of abstract.""" # Keep very lightweight; regex over HTML. req = urllib.request.Request( url, headers={"User-Agent": "ai-builder-signal-digest/1.0"}, method="GET", ) try: with urllib.request.urlopen(req, timeout=timeout_s) as resp: html = resp.read().decode("utf-8", errors="replace") except Exception: return None ``` The request is reached using an unrestricted substring check: ```python if not why and "arxiv.org/abs/" in url: abs_sum = summarize_arxiv_abstract(url) if abs_sum: why = abs_sum ``` JSONL input validation only verifies that the value begins with `http`: ```python url = r.get("canonical_url") title = r.get("title") if not isinstance(url, str) or not url.startswith("http"): continue ``` ### Technical Analysis The script treats any URL containing the text `arxiv.org/abs/` as an arXiv abstract URL. It does not parse the URL and require the hostname to be exactly `arxiv.org` or another explicitly approved arXiv hostname. Consequently, attacker-controlled URLs such as the following satisfy the check even though their actual destination is not arXiv: ```text http://127.0.0.1/admin?source=arxiv.org/abs/test https://attacker.example/redirect?source=arxiv.org/abs/test ``` The URL is passed directly to `urllib.request.urlopen()`. Python's URL opener also follows ordinary HTTP redirects by default, and redirect destinations are not revalidated. An a ...[truncated 2373 chars]- Remediation
View remediation
`. 9. Apply outbound network controls so the process cannot access localhost, private networks, link-local ranges, or cloud metadata services unless explicitly required. 10. Replace the substring test with strict validation before calling the fetch function. For example: ```python import ipaddress import socket import urllib.parse ALLOWED_ARXIV_HOSTS = {"arxiv.org", "www.arxiv.org"} def validate_arxiv_url(url: str) -> bool: try: parsed = urllib.parse.urlsplit(url) host = parsed.hostname if parsed.scheme != "https" or host not in ALLOWED_ARXIV_HOSTS: return False if parsed.username or parsed.password: return False if not parsed.path.startswith("/abs/"): return False for result in socket.getaddrinfo(host, 443, type=socket.SOCK_STREAM): address = ipaddress.ip_address(result[4][0]) if not address.is_global: return False ...[truncated 218 chars]