Back to skill

Security audit

Novel Scraper

Security checks for vulnerabilities and agentic risk

Overview

This is a coherent novel-scraping skill, but users should understand that it contacts websites and stores scraped content, cache, progress, and logs locally.

Install only if you are comfortable with a scraper that makes outbound web requests, follows redirects, and saves fetched text, logs, cache, and progress files locally. Use it on public novel-site URLs you trust, review cached output before sharing it, and prefer pinned dependencies if you need reproducible installs.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/scraper_v5.py:74
Finding

Unrestricted HTTP(S) Requests Enable Server-Side Request Forgery

Content
View full analysis
Remediation
View remediation

T08 · Insecure Dependencies

Note
Location
requirements.txt:4
Finding

Unpinned and Redundant Third-Party Dependencies Increase Supply-Chain Risk

Content
View full analysis
=4.12.0 bs4>=0.0.1 ``` ### Technical Analysis Both dependencies use open-ended minimum-version constraints. A future installation can therefore resolve to package versions that were not present or reviewed when this skill was audited. This weakens build reproducibility and permits unexpected upstream changes to enter the runtime environment. The `bs4` package is a compatibility package that depends on `beautifulsoup4`; declaring both is redundant and unnecessarily expands dependency-resolution and supply-chain exposure. The project imports from `bs4`, but that module is provided by `beautifulsoup4`, so the separate `bs4` distribution is not required. This finding does not demonstrate that either currently named package is malicious. The risk arises from mutable, unaudited dependency selection during future installations. ### Attack Path 1. A user follows the installation instructions and runs `pip install -r requirements.txt`. 2. The package index resolves versions satisfying the open-ended `>=` constraints. 3. A future compromised, malicious, or incompatible release may be selected without any change to the skill repository. 4. Package installation logic or imported runtime code executes in the user's Python environment. 5. Such code receives the filesystem and network privileges of the user performing the installation or running the scraper. ### Impact Assessment If an upstream package or release were compromised, code could execute with the privileges of the installing or runtime user. This could affect files, environment data, and network resources accessible to that account. The practical severity is low because the reviewed dependency names correspond to known packages and no evidence of an active malicious depend ...[truncated 172 chars]
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
Findings (27)

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The most concerning mismatch is the reported use of external browser automation, system memory inspection via /proc, and persistent logs/cache despite a much simpler declared curl+BeautifulSoup scraper. These undeclared capabilities expand the attack surface and may expose local system details or leave behind sensitive artifacts without user awareness.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The most concerning mismatch is the reported use of external browser automation, system memory inspection via /proc, and persistent logs/cache despite a much simpler declared curl+BeautifulSoup scraper. These undeclared capabilities expand the attack surface and may expose local system details or leave behind sensitive artifacts without user awareness.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The most concerning mismatch is the reported use of external browser automation, system memory inspection via /proc, and persistent logs/cache despite a much simpler declared curl+BeautifulSoup scraper. These undeclared capabilities expand the attack surface and may expose local system details or leave behind sensitive artifacts without user awareness.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The most concerning mismatch is the reported use of external browser automation, system memory inspection via /proc, and persistent logs/cache despite a much simpler declared curl+BeautifulSoup scraper. These undeclared capabilities expand the attack surface and may expose local system details or leave behind sensitive artifacts without user awareness.

Content

No source excerpt is available for this finding.

Tool Parameter Abuse

High
Category
Tool Misuse
Confidence
90% confidence
Finding

The troubleshooting section instructs users to run a recursive force-delete shell command against a temporary path. Even though the target is narrower than root, destructive shell snippets in end-user docs are risky because path expansion, variable substitution, copy/paste mistakes, or symlink abuse could lead to unintended file deletion.

Content

Scanner excerpt · SKILL.md (reported line 149)May include surrounding context.

md
### 抓取内容为空
1. 检查 URL 格式
2. 清除缓存:`rm -rf /tmp/novel_scraper_cache/*`
3. 检查网站是否可访问

### 章节号不正确

Tool Parameter Abuse

High
Category
Tool Misuse
Confidence
88% confidence
Finding

The troubleshooting section instructs users to run a recursive force-delete shell command against a temporary path. Even though the target is narrower than root, destructive shell snippets in end-user docs are risky because path expansion, variable substitution, copy/paste mistakes, or symlink abuse could lead to unintended file deletion.

Content

Scanner excerpt · SKILL.md (reported line 149)May include surrounding context.

md
### 抓取内容为空
1. 检查 URL 格式
2. 清除缓存:`rm -rf /tmp/novel_scraper_cache/*`
3. 检查网站是否可访问

### 章节号不正确

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

The entire skill-facing document is written in Chinese, including usage guidance, warnings, installation steps, and pointers to documentation, with no indication that users may choose another language. Under the stated policy, forcing a specific language without user opt-in is a natural-language policy violation unless the locale restriction is clearly documented and justified.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
93% confidence
Finding

The skill advertises and instructs use of capabilities that imply shell execution, filesystem access, and network activity, but the manifest does not declare any tool scope or permissions boundary. That makes it harder for a runtime or reviewer to constrain what the skill may do and increases the chance of over-privileged execution.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The manifest description and 'Use when' guidance are presented in Chinese, which imposes a specific language on users without documenting a choice or opt-in. Under the policy, language constraints should either be optional or clearly justified as region-specific.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/fetch_catalog.py (reported line 23)May include surrounding context.

python
"-A", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
        url,
    ]
    result = subprocess.run(cmd, capture_output=True, text=True, timeout=35)
    return result.stdout if result.returncode == 0 else None

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/scraper_v5.py (reported line 91)May include surrounding context.

python
"-A", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
        url,
    ]
    result = subprocess.run(cmd, capture_output=True, text=True, timeout=35)
    return result.stdout if result.returncode == 0 else None

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The module docstring and subsequent CLI/help/log messages are written entirely in Chinese, which imposes a specific language on users without any opt-in or localization mechanism. This matches the language/locale policy violation category because the skill does not appear region-specific and provides no alternative language selection.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

A novel scraper is expected to fetch web content and write TXT output, but spawning an external command-line tool introduces a broader execution capability than the manifest describes. The manifest mentions curl and BeautifulSoup, not subprocess-based browser control, so this capability is not clearly justified by the declared scope.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The manifest explicitly describes the skill as using 'curl+BeautifulSoup' to scrape novel sites, implying a direct HTTP-and-parser implementation. In practice, the code shells out to 'openclaw browser' and drives a browser session, which is a materially different behavior and execution model than the one advertised.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/scraper.py (reported line 318)May include surrounding context.

python
cmd.extend([f"--{key}", str(value)])
        
        logger.debug(f"执行:{' '.join(cmd)}")
        result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, 
                               universal_newlines=True, timeout=60)
        
        if result.returncode != 0:

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The script creates workspace and cache directories, writes logs to a file, caches fetched chapter text under /tmp, and saves merged chapter content under the user's workspace. Although there are informational logs at save time, there is no upfront disclosure in the program description or argument help that scraped content and logs will be persisted locally across multiple locations.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

This code fetches remote pages by invoking curl against user-supplied URLs, which necessarily transmits network metadata such as the requested URL and User-Agent to external servers. While the code logs retry status and success, it does not include any user-facing warning, prompt, or explanatory comment/docstring disclosing that external network requests will be made to third-party sites.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The manifest describes a scraper that fetches novel chapters with curl and BeautifulSoup and saves TXT output. Here, the code invokes another Python script through subprocess to recover catalog mappings after failures, adding arbitrary local code execution capability that is not justified by the stated scraping-and-formatting purpose.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · scripts/scraper_v5.py (reported line 410)May include surrounding context.

python
catalog_script = Path(__file__).parent / "fetch_catalog.py"
                    logger.info(f"📚 运行目录页脚本:{catalog_script}")
                    
                    result = subprocess.run(
                        ["python3", str(catalog_script)],
                        capture_output=True,
                        text=True,

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
87% confidence
Finding

This markdown file includes release notes in Chinese within an otherwise English changelog, but it does not state that the skill or its documentation is bilingual or limited to a Chinese-speaking audience. That can violate language/locale policy expectations when users are not given an explicit language choice or opt-in.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
84% confidence
Finding

This JSON includes natural-language values such as the site names and notes only in Chinese, which can imply a fixed language/locale expectation without any documented opt-in or scope limitation. Under the policy rule for language/locale constraints, this is a potential violation because the file does not indicate that the skill is intentionally region-specific or user-selectable.

Content

No source excerpt is available for this finding.

Unpinned Dependencies

Low
Category
Supply Chain
Confidence
93% confidence
Finding

The dependency is specified with a lower bound only, so installs may resolve to any newer release, including future versions with breaking changes or compromised upstream packages. While this file does not itself execute code, dependency drift weakens build reproducibility and increases supply-chain risk over time.

Content

Scanner excerpt · requirements.txt (reported line 4)May include surrounding context.

text
# Novel Scraper - Python Dependencies
# Install: pip3 install -r requirements.txt --user

beautifulsoup4>=4.12.0
bs4>=0.0.1

Unpinned Dependencies

Low
Category
Supply Chain
Confidence
94% confidence
Finding

This package is also unpinned, allowing resolution to arbitrary newer versions at install time. In a scraping skill that relies on external parsing libraries, unpinned dependencies create unnecessary supply-chain exposure and can lead to unexpected or unsafe behavior if a malicious or flawed release is published.

Content

Scanner excerpt · requirements.txt (reported line 5)May include surrounding context.

text
# Install: pip3 install -r requirements.txt --user

beautifulsoup4>=4.12.0
bs4>=0.0.1

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
97% confidence
Finding

This Python file contains all user-facing documentation and runtime messages in Chinese, including the module docstring and error/help output. Under the policy rule for language/locale, this is a natural-language constraint with no visible user opt-in or explanation that the tool is intentionally region-specific.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
90% confidence
Finding

This file contains user-facing natural language in the module docstring and function docstrings/comments exclusively in Chinese. Under the language/locale policy, forcing a specific language without opt-in can be a policy concern unless the locale restriction is explicitly justified.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.