T09 · Insecure Skill Coding Practices
Error
- Location
scripts/competitor-scraper.py:118- Finding
Unrestricted User-Controlled URL Fetching Enables Server-Side Request Forgery
- Content
View full analysis
Vulnerability Details
File Location:
scripts/competitor-scraper.py:118-129; reachable throughscripts/competitor-scraper.py:247-252and CLI arguments atscripts/competitor-scraper.py:280-282
Vulnerability Type: Server-Side Request Forgery (SSRF)
Risk Level: HighVulnerable Code
python def scrape_generic_reviews(url): """ Generic web scraper for review pages. Looks for common review HTML patterns. """ reviews = [] try: headers = { 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36' } response = requests.get(url, headers=headers, timeout=10) response.raise_for_status()The vulnerable function is exposed through the following dispatch logic:
python def scrape_competitor(platform, identifier): if platform == "gumroad": return scrape_gumroad_reviews(identifier) elif platform == "producthunt": return scrape_producthunt_reviews(identifier) elif platform == "url": return scrape_generic_reviews(identifier) else: print(f"Unknown platform: {platform}", file=sys.stderr) return []User input reaches the function through these arguments:
python parser.add_argument("--platform", required=True, choices=["gumroad", "producthunt", "url"], help="Platform to scrape") parser.add_argument("--identifier", help="Product identifier (slug or ID)") parser.add_argument("--url", help="Direct URL to scrape")Technical Analysis
When the
urlplatform is selected, the program passes the user-controlled--urlor--identifiervalue directly torequests.get(). It does not validate:- The URL scheme
- The destination hostname
- The resolved IP address
- The destination port
- Whether the address is loopback, private, link-local, or reserved
- Redirect destinations
The
requestsli ...[truncated 2388 chars]- Remediation
View remediation
Remediation Suggestions
- Restrict generic scraping to an explicit allowlist of approved public review domains where feasible.
- Permit only HTTPS URLs and reject URLs containing embedded credentials.
- Reject nonstandard destination ports unless explicitly required.
- Resolve the hostname before connecting and reject every resolved address belonging to loopback, private, link-local, multicast, unspecified, or reserved ranges for both IPv4 and IPv6.
- Disable automatic redirects with
allow_redirects=False, or validate every redirect target using the same scheme, hostname, port, and resolved-address controls. - Protect against DNS rebinding by ensuring the validated address is the address actually used for the connection.
- Apply outbound firewall or proxy rules so the Skill can connect only to approved public destinations.
- Impose response-size limits in addition to the existing timeout.
- Return a clear validation error before making a request when a destination is not approved.
- Add automated tests covering loopback, RFC1918 private ranges, IPv6 local addresses, link-local metadata addresses, encoded IP representations, and redirect-based bypasses.