T09 · Insecure Skill Coding Practices
Warning
- Location
- scripts/arxiv_watcher.py:23
- Finding
- ArXiv API Responses Are Retrieved over Unencrypted HTTP<![CDATA[ ## Vulnerability Details **File Location**: `scripts/arxiv_watcher.py:23`, with request sinks at `scripts/arxiv_watcher.py:68-74` and `scripts/arxiv_watcher.py:125-143` **Vulnerability Type**: Unauthenticated plaintext network communication **Risk Level**: Medium ### Vulnerable Code ```python # ArXiv API endpoints ARXIV_API_URL = "http://export.arxiv.org/api/query" ARXIV_NEW_URL = "https://arxiv.org/list/{category}/new" ``` The plaintext endpoint is used to retrieve individual paper details: ```python params = {"id_list": clean_id} url = f"{ARXIV_API_URL}?" + "&".join(f"{k}={v}" for k, v in params.items()) try: req = urllib.request.Request(url, headers={"User-Agent": "ArXiv-Watcher/1.0"}) with urllib.request.urlopen(req, timeout=30) as response: xml_content = response.read().decode("utf-8") except urllib.error.URLError as e: return None ``` It is also used to retrieve category feeds: ```python url = f"{ARXIV_API_URL}?" + "&".join(f"{k}={v}" for k, v in params.items()) try: req = urllib.request.Request(url, headers={"User-Agent": "ArXiv-Watcher/1.0"}) with urllib.request.urlopen(req, timeout=30) as response: xml_content = response.read().decode("utf-8") except urllib.error.URLError as e: print(f"Error fetching papers: {e}", file=sys.stderr) return [] ``` ### Technical Analysis `ARXIV_API_URL` uses HTTP rather than HTTPS. Consequently, the client does not receive transport-layer confidentiality, server authentication, or response integrity. Any actor capable of intercepting network traffic—such as a malicious access point, compromised proxy, or on-path network operator—can inspect and alter the Atom response. The modified XML is parsed without cryptographic verification. Attacker-controlled titles, abstracts, authors, identifiers, categories, and URLs can therefore be accepted as legitimate arXiv metadata. These values are subsequently rendered in Markdown and can also be written to `assets/starred.j ...[truncated 1286 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Replace the endpoint with an HTTPS URL supported by the service: ```python ARXIV_API_URL = "https://export.arxiv.org/api/query" ``` 2. Prevent downgrade attacks by rejecting redirects whose destination does not use HTTPS. A custom redirect handler can enforce an `https` scheme for every redirect target. 3. Retain normal TLS certificate and hostname verification; do not introduce an unverified SSL context. 4. Treat transport or TLS failures as hard failures rather than falling back to plaintext HTTP. 5. Validate response content type, enforce a reasonable maximum response size, and catch XML parsing errors. 6. Sanitize and validate remote metadata independently, because TLS protects transport integrity but does not make paper-submitter content inherently trustworthy. ]]>