T09 · Insecure Skill Coding Practices
Error
- Location
crawler.py:181- Finding
Untrusted listing data can trigger arbitrary outbound requests in summary mode
- Content
View full analysis
Vulnerability Details
File Location:
crawler.py:181-190,crawler.py:351-354,crawler.py:415, andcrawler.py:421
Vulnerability Type: Server-Side Request Forgery and insecure plaintext transport
Risk Level: HighVulnerable Code
python def _csrc(self, days): return self._html([ 'http://www.csrc.gov.cn/csrc/c100028/common_list.shtml', 'http://www.csrc.gov.cn/csrc/c100028/common_list2.shtml', ], 'csrc', '中国证券监督管理委员会', 'http://www.csrc.gov.cn', days) def _pboc(self, days): return self._html([ 'http://www.pbc.gov.cn/goutongjiaoliu/113456/113469/index.html', ], 'pboc', '中国人民银行', 'http://www.pbc.gov.cn', days)python @staticmethod def _abs(href, base): if href.startswith('http://') or href.startswith('https://'): return href if href.startswith('//'): return 'http:' + href if href.startswith('/'): return base.rstrip('/') + '/' + href.lstrip('/') return base.rstrip('/') + '/' + hrefpython if summary and uniq: print('\n📝 抓取摘要…', file=sys.stderr) for i, it in enumerate(uniq): print(f' [{i+1}/{len(uniq)}] {it["title"][:40]}', file=sys.stderr) it['summary'] = self._sum(it['url']) time.sleep(1)python def _sum(self, url, n=200): try: r = self.s.get(url, timeout=15) r.encoding = r.apparent_encoding or 'utf-8' soup = BeautifulSoup(r.text, 'lxml') for t in soup(['script', 'style', 'nav', 'footer', 'header']): t.decompose() for sel in ['.article-content', '.content', '.main-text', '#content', '.detail-content', '.news-content', 'article', 'body']: b = soup.select_one(sel) if b: t = re.sub(r'\s+', ' ', b.get_text(strip=True)) if len(t) > 50: return t[:n] + ('…' if len(t) > n else '') return '' except Exception: return ''Technical Anal
...[truncated 2779 chars]
- Remediation
View remediation
Remediation Suggestions
- Replace all plaintext source URLs and base URLs with verified HTTPS endpoints.
- Reject every URL whose scheme is not HTTPS.
- Maintain an explicit hostname allowlist for each regulator and reject links outside the expected official domains.
- Resolve target hostnames before connecting and reject loopback, private, link-local, multicast, unspecified, and reserved IP ranges for both IPv4 and IPv6.
- Disable automatic redirects or validate the scheme, hostname, and resolved address of every redirect destination before following it.
- Apply the same validation immediately before the request to reduce DNS rebinding and time-of-check/time-of-use risks.
- Consider removing arbitrary linked-page retrieval entirely. If summaries are required, retrieve articles only from predefined official origins.
- Escape or sanitize untrusted titles and URLs before embedding them in Markdown.
- Add tests covering absolute external URLs, protocol-relative URLs, redirects, private addresses, IPv6 literals, encoded hostnames, and DNS rebinding scenarios.