T09 · Insecure Skill Coding Practices
Error
- Location
skills/deep-research/deep_research.py:72- Finding
Arbitrary URL Fetching Enables Server-Side Request Forgery
- Content
View full analysis
bool: """Check if URL should be processed.""" if not url or url in self.seen_urls: return False for domain in IGNORED_DOMAINS: if domain in url.lower(): return False return True async def fetch_content(self, url: str) -> Optional[str]: """Fetch and extract readable content from URL.""" for attempt in range(MAX_RETRIES): try: headers = { "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) " "AppleWebKit/537.36 (KHTML, like Gecko) " "Chrome/120.0.0.0 Safari/537.36" } async with aiohttp.ClientSession() as session: async with session.get( url, headers=headers, timeout=aiohttp.ClientTimeout(total=REQUEST_TIMEOUT) ) as resp: if resp.status != 200: continue html = await resp.text() soup = BeautifulSoup(html, "html.parser") ``` ```python async def process_result(self, result: Dict) -> Optional[Dict]: """Process a single search result.""" url = result.get("url", "") title = result.get("title", "") snippet = result.get("content", "") if not self.is_valid_url(url): return None self.seen_urls.add(url) # Try to fetch full content content = await self.fetch_content(url) ``` ### Technical Analysis URLs obtained from SearXNG results are requested directly by `aiohttp`. The validation function only checks for duplicate URLs and a small list of ignored domain substrings. It does not: - Restrict requests to HTTP and HTTPS. - Resolve ...[truncated 1667 chars]- Remediation
View remediation