T09 · Insecure Skill Coding Practices
Warning
- Location
- scripts/fetch_and_digest.py:35
- Finding
- Untrusted URLs Can Be Misclassified as High-Credibility Sources<![CDATA[ ## Vulnerability Details **File Location**: `scripts/fetch_and_digest.py:35-48` **Vulnerability Type**: Improper URL hostname validation **Risk Level**: Medium ### Vulnerable Code ```python def classify_source_credibility(url): """Classify source credibility level""" if not url: return 'B' high_credibility = ['people.com.cn', 'xinhuanet.com', 'cctv.com', 'gov.cn', 'reuters.com', 'bbc.com', 'apnews.com', 'nytimes.com'] medium_credibility = ['sina.com', '163.com', 'qq.com', 'toutiao.com', 'thepaper.cn', 'huanqiu.com'] for domain in high_credibility: if domain in url: return 'A' # High credibility for domain in medium_credibility: if domain in url: return 'B' # Medium credibility return 'C' # User-generated/social media ``` ### Technical Analysis The function determines source credibility by checking whether a trusted domain appears anywhere in the complete URL string. This does not establish that the URL's actual hostname belongs to the trusted organization. An attacker can place a trusted string in an untrusted URL, including: ```text https://reuters.com.attacker.example/fabricated-report https://attacker.example/article?source=bbc.com https://attacker.example/nytimes.com/fake-news ``` Each of these URLs can receive an `A` credibility rating despite not being controlled by the referenced news organization. The URLs originate in externally supplied Tavily search results. The resulting credibility rating is subsequently displayed in an automatically distributed digest, where an `A` rating is described as an official or highly credible source. There is no independent hostname validation or verification of the article. ### Attack Path 1. An attacker publishes a fabricated article on an attacker-controlled domain. 2. The URL is constructed to contain a trusted-domain string, such as `reuters.com.attacker.exa ...[truncated 1201 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions Parse the URL and validate only its normalized hostname: ```python from urllib.parse import urlparse HIGH_CREDIBILITY = { "people.com.cn", "xinhuanet.com", "cctv.com", "gov.cn", "reuters.com", "bbc.com", "apnews.com", "nytimes.com", } def belongs_to_domain(hostname, trusted_domain): hostname = hostname.rstrip(".").lower() trusted_domain = trusted_domain.rstrip(".").lower() return hostname == trusted_domain or hostname.endswith("." + trusted_domain) def classify_source_credibility(url): if not url: return "C" try: parsed = urlparse(url) if parsed.scheme not in {"https", "http"} or not parsed.hostname: return "C" hostname = parsed.hostname.encode("idna").decode("ascii").lower() if any(belongs_to_domain(hostname, domain) for domain in HIGH_CREDIBILITY): return "A" except (TypeError, ValueError, UnicodeError): return "C" return "C" ``` Additional hardening should include: 1. Prefer HTTPS URLs and downgrade or reject plaintext HTTP sources. 2. Maintain an explicit allowlist of canonical publication hostnames. 3. Account for provider-specific domains rather than trusting arbitrary subdomains indiscriminately. 4. Treat search-engine results as untrusted input. 5. Do not describe automated domain classification as fact-checking. 6. Include the normalized source hostname in each digest entry so recipients can assess provenance. 7. Add tests for deceptive domains, query strings, user-info components, mixed case, trailing dots, and internationalized domain names. ]]>