T09 · Insecure Skill Coding Practices
- Location
scripts/arxiv_section_extractor.py:303- Finding
Unvalidated Upstream PDF URL Allows Arbitrary Network Requests
- Content
View full analysis
Vulnerability Details
File Location:
scripts/arxiv_section_extractor.py, lines 75–78 and 303–309
Vulnerability Type: Unrestricted outbound request / potential SSRF
Risk Level: MediumVulnerable Code:
python pdf_link = soup.find('a', {'class': 'download-pdf'}) if pdf_link and pdf_link.get('href'): href = pdf_link['href'] links['pdf'] = f"https://arxiv.org{href}" if href.startswith('/') else hrefpython try: pdf_resp = requests.get(access_links['pdf'], stream=True, timeout=60) pdf_resp.raise_for_status() with open(pdf_path, 'wb') as f: for chunk in pdf_resp.iter_content(8192): f.write(chunk) except Exception as e: print(f" PDF download failed: {e}")Technical Analysis
The PDF URL is extracted from remotely supplied HTML. Relative links are assigned to
arxiv.org, but absolute links are accepted without validating their scheme, hostname, port, or destination IP address. The subsequentrequests.get()call also follows redirects by default without validating each redirect destination.If the upstream HTML or HTTP redirect chain is compromised or manipulated, the script can be induced to send a request to an attacker-selected host. This includes public attacker infrastructure and, depending on the runner's network configuration, loopback, link-local, or internal network services.
The network activity is necessary for the declared paper-download functionality. However, permitting arbitrary destinations exceeds the minimum network privilege required, because the expected destination should be limited to trusted arXiv hosts.
Attack Path
- An attacker gains the ability to influence the arXiv HTML response, a trusted intermediary, or the redirect chain used for a paper download.
- The response supplies an absolute PDF URL targeting an attacker-controlled or internal endpoint.
- A user or Agent invokes the extractor f ...[truncated 807 chars]
- Remediation
View remediation
Remediation Suggestions
- Resolve relative links with
urllib.parse.urljoin()against a trusted arXiv base URL. - Require HTTPS and allowlist the exact approved arXiv hostnames.
- Reject URLs containing credentials, unexpected ports, fragments, or unsupported schemes.
- Resolve the hostname and reject loopback, private, reserved, multicast, and link-local addresses.
- Disable automatic redirects and validate every redirect target before following it.
- Prefer constructing the expected PDF URL directly from the validated arXiv ID rather than trusting a link extracted from remote HTML.
- Log rejected destinations without exposing sensitive URL components.
- Resolve relative links with
