news-scraper-xiaobai

• Outcome: Review
• Latest audit: Sep 12, 2026 · 4:17 AM UTC
• Version: 1.0.0

Overview

This is a mostly coherent Chinese AI news scraper, but its article URL handling is too broad for a single-site crawler and could fetch unintended internal or external URLs.

Review this before installing in environments with access to private networks, cloud metadata endpoints, or sensitive internal HTTP services. The publisher should restrict article fetches to the intended AIBase HTTPS host, validate redirects, pin dependencies, and tell agents to treat scraped article text only as untrusted data.

A.I.G

Vulnerability Patterns

• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally

Findings (2)

T09 · Insecure Skill Coding Practices

Error

• Location: scrapers/aibase.py:54

Finding

Unrestricted Article URL Fetching Enables Server-Side Request Forgery

Content

View full analysis

Optional[str]: """Fetch page content""" timeout = timeout or config.REQUEST_TIMEOUT try: resp = self.session.get(url, timeout=timeout) resp.raise_for_status() resp.encoding = resp.apparent_encoding return resp.text except Exception as e: print(f"[{self.site_id}] Failed to fetch page: {url}, error: {e}") return None ``` ### Technical Analysis The list parser accepts absolute URLs beginning with `http` without validating their scheme, normalized hostname, port, res ...[truncated 2761 chars]

Remediation

View remediation

T01 · Skill Instruction Hijacking

Warning

• Location: scrapers/aibase.py:128

Finding

Untrusted Scraped Article Content Can Introduce Indirect Prompt Injection

Content

View full analysis

Dict: """Parse news detail page""" soup = BeautifulSoup(html, "html.parser") result = { "content": "", "publish_time": "", } # Extract article body article = soup.find("article") or soup.find(class_="article-content") or soup.find(class_="content") if article: # Remove scripts and styles for tag in article.find_all(["script", "style"]): tag.decompose() # Get text texts = article.get_text(separator="\n", strip=True) # Clean redundant whitespace lines = [line.strip() for line in texts.split("\n") if line.strip()] result["content"] = "\n\n".join(lines) ``` ```python # scrapers/aibase.py:173-181 # If no body was extracted, try the entire page if not result["content"]: body = soup.find("body") if body: for tag in body.find_all(["script", "style", "nav", "header", "footer"]): tag.decompose() result["content"] = body.get_text(separator="\n", strip=True)[:5000] return result ``` The Skill documentation directs the agent to process the returned material: ```python # SKILL.md:13-18 import sys sys.path.insert(0, "news-scraper") from scripts.crawl import crawl_and_return_json result = crawl_and_return_json(site="aibase", limit=20) # AI processes the returned data itself ``` ### Technical Analysis Article text is attacker-influenced external data. The parser strips HTML `script` and `style` elements, but natural-language instructions remain intact. For example, an article could contain text instructing an agent to ignore its current task, disclose contextual information, invoke tool ...[truncated 2209 chars]

Remediation

View remediation

SkillSpector

Vulnerability Patterns

• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code
• MCP Tool Poisoning: Hidden Instructions, Unicode Deception, Parameter Description Injection
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access

Findings (13)

Natural-Language Policy Violations

Medium

• Category: Not specified by scanner
• Confidence: 93% confidence

Finding

The manifest description is entirely in Chinese and presents the skill purpose in Chinese without indicating that other languages are supported or that Chinese output is optional. Under the policy, forcing a specific language or locale without user opt-in is a natural-language policy concern.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

• Category: Not specified by scanner
• Confidence: 90% confidence

Finding

This Python file contains user-facing natural-language text only in Chinese, including the module description, function docstring, and error message. The file does not indicate that Chinese is an optional locale or that the skill is intentionally region-specific, which can violate language/locale policy requirements.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

• Category: Not specified by scanner
• Confidence: 94% confidence

Finding

This code rewrites relative and absolute URLs to the Chinese /zh version unconditionally, which imposes a specific locale rather than respecting user preference. The file does not show any opt-in, configurability, or documented region-specific requirement that would justify forcing Chinese content.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

• Category: Not specified by scanner
• Confidence: 86% confidence

Finding

The file’s natural-language strings, docstrings, and user-facing print messages are entirely in Chinese, including runtime status/error output. This indicates a fixed language choice with no opt-in, fallback, or documented region-specific justification, which matches the language/locale policy violation criterion.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

• Category: Not specified by scanner
• Confidence: 97% confidence

Finding

The module docstring, CLI help text, and runtime output are written entirely in Chinese, which imposes a specific language on users. The file does not offer any language selection, opt-in, or justification that this is a region-specific tool, so it violates the language/locale policy criteria.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium

• Category: Not specified by scanner
• Confidence: 96% confidence

Finding

The module documentation states the script's functionality includes '保存为Markdown格式', implying crawled news will be persisted as Markdown. However, in the actual main flow, the code saves results via save_to_json() and prints summaries, while save_news() is never called, so Markdown output does not occur.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium

• Category: Not specified by scanner
• Confidence: 92% confidence

Finding

The --force option is documented as forcing regeneration of existing files, and save_news() contains overwrite logic that checks args.force. But the main program never calls save_news(), so --force has no practical effect on output files in the executed path.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The Accept-Language header is explicitly set to "zh-CN,zh;q=0.9,en;q=0.8", which imposes a Chinese-first locale preference in outbound requests. This is a natural-language locale constraint and there is no indication in this file that users can choose or opt into that preference.

Content

No source excerpt is available for this finding.

Unpinned Dependencies

Low

• Category: Supply Chain
• Confidence: 96% confidence

Finding

The dependency is specified with a lower bound only, which allows future installs to resolve to different versions over time. This weakens reproducibility and can unintentionally introduce vulnerable or breaking releases through the supply chain, especially for a network-facing library like requests.

Content

Scanner excerpt · requirements.txt (reported line 3)May include surrounding context.

text

# 新闻爬虫依赖

requests>=2.28.0
beautifulsoup4>=4.11.0
openpyxl>=3.0.0

Unverifiable Dependency: requests has 16 known advisory(ies) (CVE-2014-1830 (Exposure of Sensitive Information to an Unauthorized Actor in Requests); CVE-2024-47081 (Requests vulnerable to .netrc credentials leak via malicious URLs); CVE-2024-35195 (Requests `Session` object does not verify requests after making first request wi) +13 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low

• Category: Supply Chain
• Confidence: 93% confidence

Finding

requests has multiple published advisories, and because the manifest does not pin a version, there is no reliable way to determine whether installed environments will avoid affected releases. In a crawler that makes outbound HTTP requests, this uncertainty is more relevant because requests is directly exercised in network operations.

Content

No source excerpt is available for this finding.

Unpinned Dependencies

Low

• Category: Supply Chain
• Confidence: 95% confidence

Finding

beautifulsoup4 is not pinned to a specific version, so dependency resolution is non-deterministic across environments and over time. While not an immediate exploit by itself, this increases supply-chain risk and can pull in unexpected vulnerable or incompatible releases.

Content

Scanner excerpt · requirements.txt (reported line 4)May include surrounding context.

text

# 新闻爬虫依赖

requests>=2.28.0
beautifulsoup4>=4.11.0
openpyxl>=3.0.0

Unpinned Dependencies

Low

• Category: Supply Chain
• Confidence: 97% confidence

Finding

openpyxl is declared with only a minimum version, leaving the actual installed version uncontrolled. Because this package has had security issues historically, unpinned installation makes it impossible to verify whether deployments are using a safe release.

Content

Scanner excerpt · requirements.txt (reported line 5)May include surrounding context.

text

requests>=2.28.0
beautifulsoup4>=4.11.0
openpyxl>=3.0.0

Unverifiable Dependency: openpyxl has 2 known advisory(ies) (CVE-2017-5992 (Improper Restriction of XML External Entity Reference in Openpyxl); CVE-2017-5992 (Openpyxl 2.4.1 resolves external entities by default, which allows remote attack)), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low

• Category: Supply Chain
• Confidence: 91% confidence

Finding

openpyxl has known historical advisories, and the unpinned declaration prevents verification that the deployed version is patched. Since this skill likely processes spreadsheet files, an affected version could expose parsing-related risk if untrusted workbook content is handled.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.