T09 · Insecure Skill Coding Practices
Error
- Location
generator.py:18- Finding
Unrestricted URL Scraping Enables Server-Side Request Forgery
- Content
View full analysis
str: """Scrape a URL and return its text content.""" try: r = subprocess.run( ["python3", SCRAPE_SCRIPT, "web", url], capture_output=True, text=True, timeout=30 ) if r.returncode != 0 or not r.stdout.strip(): return "" d = json.loads(r.stdout) content = d.get("content", []) if isinstance(content, list): return " ".join(c for c in content if c).strip() return str(content).strip() except Exception as e: print(f" [scrape error] {url}: {e}", file=sys.stderr) return "" def scrape_business(url: str) -> str: """Scrape homepage and try to also grab about/contact subpages.""" base = url.rstrip("/") texts = [] # Homepage print(f" Scraping: {base}", file=sys.stderr) home_text = scrape_url(base) if home_text: texts.append(home_text) # Try common subpages for path in ["/about", "/about-us", "/contact", "/services"]: sub_url = base + path print(f" Scraping: {sub_url}", file=sys.stderr) sub_text = scrape_url(sub_url) if sub_text and len(sub_text) > 100: texts.append(sub_text) combined = " ".join(texts) # Trim to ~2000 chars to keep Ollama prompt manageable return combined[:2000] if combined else "" ``` The command-line input reaches this code without destination validation: ```python elif args.name and args.url: process_business(args.name, args.url, args.phone, args.save) ``` ### Technical Analysis The application passes a user-controlled URL, or a URL parsed from the local lead list, directly to an external scraping process. It does not validate: - The URL scheme. - The destination hostname or IP addre ...[truncated 2235 chars]- Remediation
View remediation