T09 · Insecure Skill Coding Practices
Warning
- Location
- scripts/crawl_douban_self_history.py:300
- Finding
- Unrestricted Request Destinations Combined with Unvalidated Cookie Domains<![CDATA[ ## Vulnerability Details **File Location**: `scripts/crawl_douban_self_history.py:131-137, 261-281, 300-310` **Vulnerability Type**: Unvalidated outbound destinations and cookie scope **Risk Level**: Medium ### Vulnerable Code ```python def parse_next_page_url(html: str, current_url: str) -> str | None: soup = BeautifulSoup(html, "lxml") next_el = soup.select_one("span.next a") or soup.select_one("a.next") if not next_el: return None href = next_el.get("href", "").strip() return urljoin(current_url, href) if href else None ``` ```python def crawl_category(client: httpx.Client, uid: str, category: str, interval: float, page_limit: int | None) -> dict[str, Any]: items: list[Item] = [] page_count = 0 for status in STATUSES: next_url = build_start_url(uid, category, status) while next_url: page_count += 1 resp = client.get(next_url) resp.raise_for_status() html = resp.text if looks_like_login_or_auth_problem(html, str(resp.url)): raise RuntimeError(f"Authentication failed while fetching {resp.url}") source_name = f"{category}-{status}-page-{page_count}.html" items.extend(parse_page(category, html, status, str(resp.url), source_name)) if page_limit and page_count >= page_limit: next_url = None else: next_url = parse_next_page_url(html, str(resp.url)) if next_url: time.sleep(interval) ``` ```python def make_client(cookie_file: Path, timeout: float) -> httpx.Client: raw = load_cookies(cookie_file) client = httpx.Client(timeout=timeout, follow_redirects=True, headers={"User-Agent": "Mozilla/5.0 OpenClaw DoubanSelfTasteSkill"}) for item in raw: name = item.get("name") value = item.get("value") domain = item.get("domain") path = item.get("path", "/") if not name or value ...[truncated 2634 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Restrict imported cookies to an explicit domain allowlist, such as: - `douban.com` - `.douban.com` - the exact required Douban subdomains 2. Reject cookies with missing, malformed, unrelated, or public-suffix domains. 3. Permit only HTTPS requests to an explicit hostname allowlist, for example: - `www.douban.com` - `movie.douban.com` - `book.douban.com` - `music.douban.com` - required Douban authentication hosts, if strictly necessary 4. Disable automatic redirects with `follow_redirects=False`. Validate each `Location` header before following it. 5. Validate every pagination URL after `urljoin` and before `client.get()`: ```python ALLOWED_HOSTS = { "www.douban.com", "movie.douban.com", "book.douban.com", "music.douban.com", } def validate_url(url: str) -> str: parsed = urlparse(url) if parsed.scheme != "https" or parsed.hostname not in ALLOWED_HOSTS: raise RuntimeError(f"Refusing unexpected outbound URL: {url}") return url ``` 6. Prefer a dedicated cookie jar containing only the minimum Douban cookies required for authentication. 7. Document that users must not provide an unrestricted whole-browser cookie export. ]]>
