T09 · Insecure Skill Coding Practices
Error
- Location
scripts/media_downloader.py:25- Finding
Server-Side Request Forgery Through Attacker-Controlled Markdown Media URLs
- Content
View full analysis
Vulnerability Details
File Location:
scripts/media_downloader.py, lines 25-29 and 43-76
Vulnerability Type: Server-Side Request Forgery (SSRF) and unrestricted outbound request
Risk Level: HighVulnerable Code
python # Extract Markdown image format:  markdown_img_pattern = r'!\[.*?\]\((https?://[^\)]+)\)' matches = re.findall(markdown_img_pattern, content, re.IGNORECASE) urls.extend(matches)python def download_media(self, url: str) -> Tuple[Optional[str], str]: """Download a media file and return (local path, original URL).""" try: # Clean URL by removing parameters such as name=orig clean_url = re.sub(r'[?\&]name=[^&\s\)\]]*', '', url) # Parse URL to obtain a filename parsed = urlparse(clean_url) path = parsed.path filename = path.split('/')[-1] # Add an extension if one is missing ext = os.path.splitext(filename)[1] if not ext: ext = '.jpg' filename = filename + ext # Generate a unique filename base_name = os.path.splitext(filename)[0] or 'image' counter = 1 while os.path.exists(self.images_dir / filename): filename = f"{base_name}_{counter}{ext}" counter += 1 filepath = self.images_dir / filename # Download the file response = requests.get(clean_url, stream=True, timeout=30) response.raise_for_status() with open(filepath, 'wb') as f: for chunk in response.iter_content(chunk_size=8192): if chunk: f.write(chunk)Related data flow:
scripts/craw_hot.py, lines 535-558 and 607-627, inserts untrusted post text and media URLs directly into Markdown.scripts/craw_hot.py, lines 1177-1183 and 1260-1274, automatically invokesMediaDownloader.process_markdown()after crawling.
...[truncated 3353 chars]
- Remediation
View remediation
Remediation Suggestions
- Do not extract arbitrary Markdown image URLs from untrusted post text. Download only URLs taken from structured API media fields.
- Enforce an exact hostname allowlist, such as
pbs.twimg.comandvideo.twimg.com, using parsed hostnames rather than suffix or substring checks. - Permit HTTPS only.
- Resolve the hostname before connecting and reject every resolved loopback, private, link-local, multicast, unspecified, or reserved address.
- Disable redirects, or validate the scheme, hostname, port, and resolved addresses again at every redirect hop.
- Restrict destination ports to 443 unless another port is explicitly required.
- Verify that responses have an approved image or video media type before saving them.
- Consider running media retrieval in an isolated process or network namespace that cannot reach localhost, private networks, or metadata endpoints.
- Escape untrusted post text before embedding it in Markdown so that tweet text cannot create active images or links.
- Add tests covering direct private IPs, IPv6 loopback, alternate IP representations, DNS rebinding, and redirect-to-private-address cases.