T09 · Insecure Skill Coding Practices
- Location
scripts/save_paper.py:97- Finding
Insufficient Validation of Remote PDF URLs Enables Arbitrary Network Requests
- Content
View full analysis
Vulnerability Details
File Location:
scripts/save_paper.py, lines 97–125
Vulnerability Type: Improper URL validation and unrestricted remote file retrieval
Risk Level: MediumVulnerable Code
python # 下载并附加 PDF if 'arxiv.org' in args.url: try: import urllib.request import tempfile # 将摘要链接转换为 PDF 链接 pdf_url = args.url.replace('/abs/', '/pdf/') if not pdf_url.endswith('.pdf'): pdf_url += '.pdf' print(f"正在下载 PDF...") # 设置 User-Agent 以支持下载 opener = urllib.request.build_opener() opener.addheaders = [('User-agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) Chrome/120.0.0.0')] urllib.request.install_opener(opener) # 创建安全的文件名 safe_title = "".join(c for c in args.title if c.isalnum() or c in (" ", "-", "_")).strip() safe_title = safe_title[:50] # 限制长度 safe_filename = f"{safe_title}.pdf" # 使用临时目录,但指定文件名 with tempfile.TemporaryDirectory() as td: pdf_path = os.path.join(td, safe_filename) urllib.request.urlretrieve(pdf_url, pdf_path) print(f"正在上传 PDF 附件({safe_filename})...") zot.attachment_simple([pdf_path], item_key)Technical Analysis
The code determines whether a URL is an arXiv URL by checking whether the untrusted string contains the substring
arxiv.org. This does not validate the URL's parsed hostname, scheme, port, or redirect destination.For example, both of the following attacker-controlled URLs pass the check even though their effective hosts are not arXiv:
text https://arxiv.org.attacker.example/payload https://attacker.example/files/arxiv.org/payload...[truncated 1518 chars]
- Remediation
View remediation
Remediation Suggestions
- Parse the URL using
urllib.parse.urlparserather than performing a substring check. - Require the
httpsscheme. - Allow only the exact hostname
arxiv.organd explicitly approved arXiv subdomains. - Reject embedded credentials, unexpected ports, malformed URLs, and non-HTTP schemes.
- Derive the PDF URL from a validated arXiv identifier instead of modifying an arbitrary URL.
- Disable redirects or validate the hostname, scheme, and port after every redirect.
- Stream downloads while enforcing a strict maximum response size.
- Require an expected PDF content type and validate the downloaded file signature before uploading it.
- Apply connection and read timeouts and delete rejected downloads immediately.
- Parse the URL using
