Novel Scraper SPA

支持 JavaScript 加载的小说抓取工具，使用 Playwright 渲染 SPA 网站。自动检测网站类型，静态网站使用 curl，SPA 网站使用浏览器渲染。

Security Scan

The code implements an SPA-capable novel scraper as claimed, but the package is sloppy about dependencies and install steps (Playwright and browser binaries are required but not declared), which is an incoherence you should address before running it.

Purpose & Capability

The skill's stated purpose (scraping SPA and static novel sites) matches the included scripts: requests/BeautifulSoup for static pages and Playwright-based rendering for SPA. However, the skill does not declare required Python packages (playwright, requests, bs4) or the browser binaries Playwright needs. That omission is disproportionate to the task and may surprise users at runtime.

Instruction Scope

SKILL.md instructs running the provided CLI (python3 scripts/scraper.py ...) and describes parameters and output location (~/.openclaw/workspace/novels/). The instructions do not ask the agent to read unrelated files, credentials, or to transmit scraped data to external endpoints. The doc contains leftover template TODO text, indicating the author didn't finish editing the README.

Install Mechanism

There is no install spec. That reduces automated risk (nothing is auto-downloaded by the skill itself) but is a practical problem because the code requires external runtime dependencies (Playwright and its browser engines). Playwright typically requires a manual 'playwright install' (which downloads browser binaries) or package install; the skill provides no guidance or declared dependency list.

Credentials

The skill requests no environment variables, credentials, or config paths. That is proportionate to its stated purpose: it scrapes arbitrary URLs provided by the user and saves output locally. There are no signs of hidden credential access or exfiltration in the included files.

Persistence & Privilege

The skill does not request 'always: true' or elevated persistence. It is user-invocable and can run autonomously per platform defaults, which is expected. It does not modify other skills or system configuration beyond writing output to the user's ~/.openclaw workspace.

What to consider before installing

Before installing or running this skill: - Expect to install Python dependencies manually: at minimum 'playwright', 'requests', and 'beautifulsoup4'. Also run 'playwright install' (or equivalent) to download browser engines — the skill does not declare or perform this. - Because Playwright downloads browser binaries, running the skill will cause additional network downloads and write large files to disk; consider running in an isolated environment or virtualenv. - The SKILL.md contains leftover template/TODO text and no dependency declarations — treat this as low-quality packaging rather than proof of malicious intent, but exercise caution. - The scripts only fetch the target URL(s) and save scraped text locally (~/.openclaw/workspace/novels/). They do not exfiltrate data to other endpoints, but the tool will fetch arbitrary URLs you provide — avoid pointing it at internal/private endpoints or URLs requiring secrets unless you understand the risks. - If you plan to use it, inspect and run the code in a sandboxed environment first, verify dependencies and that browser downloads are acceptable, and consider adding a requirements.txt and explicit install instructions (and lock Playwright/browser behavior) before using in production.

功能

支持抓取 JavaScript 加载的小说网站（SPA 架构），使用 Playwright 浏览器渲染。

使用场景

• 抓取笔趣阁等 SPA 架构小说网站
• 需要 JavaScript 渲染才能获取内容的网站
• 普通静态网站（自动使用 curl，更快速）

命令

python3 scripts/scraper.py --url <URL> --book <书名> --chapter <章节号>

参数

• --url: 目标 URL
• --book: 书名（用于输出文件名）
• --chapter: 章节号
• --force-spa: 强制使用浏览器渲染
• -v: 详细日志

输出

保存到 ~/.openclaw/workspace/novels/ 目录

2. Task-Based (best for tool collections)

• Works well when the skill offers different operations/capabilities
• Example: PDF skill with "Quick Start" → "Merge PDFs" → "Split PDFs" → "Extract Text"
• Structure: ## Overview → ## Quick Start → ## Task Category 1 → ## Task Category 2...

3. Reference/Guidelines (best for standards or specifications)

• Works well for brand guidelines, coding standards, or requirements
• Example: Brand styling with "Brand Guidelines" → "Colors" → "Typography" → "Features"
• Structure: ## Overview → ## Guidelines → ## Specifications → ## Usage...

4. Capabilities-Based (best for integrated systems)

• Works well when the skill provides multiple interrelated features
• Example: Product Management with "Core Capabilities" → numbered capability list
• Structure: ## Overview → ## Core Capabilities → ### 1. Feature → ### 2. Feature...

Patterns can be mixed and matched as needed. Most skills combine patterns (e.g., start with task-based, add workflow for complex operations).

Delete this entire "Structuring This Skill" section when done - it's just guidance.]

[TODO: Replace with the first main section based on chosen structure]

[TODO: Add content here. See examples in existing skills:

• Code samples for technical skills
• Decision trees for complex workflows
• Concrete examples with realistic user requests
• References to scripts/templates/references as needed]

Resources

This skill includes example resource directories that demonstrate how to organize different types of bundled resources:

scripts/

Executable code (Python/Bash/etc.) that can be run directly to perform specific operations.

Examples from other skills:

• PDF skill: fill_fillable_fields.py, extract_form_field_info.py - utilities for PDF manipulation
• DOCX skill: document.py, utilities.py - Python modules for document processing

Appropriate for: Python scripts, shell scripts, or any executable code that performs automation, data processing, or specific operations.

Note: Scripts may be executed without loading into context, but can still be read by Claude for patching or environment adjustments.

references/

Documentation and reference material intended to be loaded into context to inform Claude's process and thinking.

Examples from other skills:

• Product management: communication.md, context_building.md - detailed workflow guides
• BigQuery: API reference documentation and query examples
• Finance: Schema documentation, company policies

Appropriate for: In-depth documentation, API references, database schemas, comprehensive guides, or any detailed information that Claude should reference while working.

assets/

Files not intended to be loaded into context, but rather used within the output Claude produces.

Examples from other skills:

• Brand styling: PowerPoint template files (.pptx), logo files
• Frontend builder: HTML/React boilerplate project directories
• Typography: Font files (.ttf, .woff2)

Appropriate for: Templates, boilerplate code, document templates, images, icons, fonts, or any files meant to be copied or used in the final output.

---

Any unneeded directories can be deleted. Not every skill requires all three types of resources.