T09 · Insecure Skill Coding Practices
- Location
dasouche_scraper.py:177- Finding
Unrestricted Remote Detail URLs Enable Server-Side Request Forgery
- Content
View full analysis
Vulnerability Details
File Location:
dasouche_scraper.py:177-187, 324-329;dongchedi_scraper.py:245-255, 426-431;autohome_scraper.py:248-257, 508-513. Identical vulnerable code is present under万能反爬skill/. Vulnerability Type: Server-Side Request Forgery Risk Level: HighVulnerable Code
dasouche_scraper.py:python def scrape_detail(self, car_id: str, detail_url: str = "") -> Optional[VehicleInfo]: if not detail_url: detail_url = self.config.detail_url_pattern.format(car_id=car_id) if detail_url.startswith("/"): detail_url = self.config.base_url + detail_url resp = self._request(detail_url, referer=self.config.list_url) if not resp: return Nonepython for idx, item in enumerate(all_list_items): car_id = item.get("id", "") detail_url = item.get("url", "") if not car_id and not detail_url: continue vehicle = self.scrape_detail(car_id, detail_url)dongchedi_scraper.py:python def scrape_detail(self, car_id: str, detail_url: str = "") -> Optional[VehicleInfo]: if not detail_url: detail_url = self.config.detail_url_pattern.format(car_id=car_id) if detail_url.startswith("/"): detail_url = self.config.base_url + detail_url resp = self._request( detail_url, referer="https://www.dongchedi.com/usedcar", ) if not resp: return Nonepython for idx, item in enumerate(all_list_items): car_id = item.get("id", "") detail_url = item.get("url", "") if not car_id and not detail_url: continue vehicle = self.scrape_detail(car_id, detail_url)autohome_scraper.py:python def scrape_detail(self, car_id: str, detail_url: str = "") -> Optional[VehicleInfo]: if not detail_url: detail_url = self.config.detail_url_pattern.format(car_id=car_id) if det ...[truncated 2518 chars]- Remediation
View remediation
Remediation Suggestions
- Resolve relative links with
urllib.parse.urljoin()rather than string concatenation. - Permit only HTTPS URLs on an explicit hostname allowlist:
www.souche.comfor Souche.www.dongchedi.comfor Dongchedi.www.che168.com,che168.com, and explicitly required Autohome hosts for Autohome.
- Reject URLs containing user information, fragments where unnecessary, unexpected ports, or non-HTTP schemes.
- Resolve the hostname before connecting and reject loopback, private, link-local, multicast, unspecified, and reserved IP addresses for both IPv4 and IPv6.
- Disable automatic redirects with
allow_redirects=False, or validate every redirect target before following it. - Revalidate the connected destination to reduce DNS rebinding risk.
- Construct detail URLs from validated vehicle identifiers where possible instead of trusting upstream absolute URLs.
- Add tests covering loopback addresses, private IPv4 ranges, IPv6 loopback, encoded IP representations, malicious redirects, and DNS rebinding scenarios.
- Resolve relative links with
