Security audit

Lobster Crawler Skill

Security checks for vulnerabilities and agentic risk

Overview

The crawler features mostly match the description, but the package also includes agent automation that can persist state outside the project and let repository files steer an external Claude loop.

Install only if you intend to run a crawler that contacts Webnovel/ReelShorts and can send summaries to DingTalk. Do not run scripts/claude_loop.sh unless you understand it invokes Claude with repository-derived prompts; remove or ignore the global memory instructions before using this in a normal agent environment.

A.I.G

Vulnerability Patterns

Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Insecure DependenciesIntroduces malicious components through unsafe dependency sources
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally

Findings (3)

T02 · Agent Memory Poisoning

Error

Location: CLAUDE.md:56
Finding: Repository Instructions Require Cross-Session Agent Memory Modification
Content: View full analysis
Vulnerability Details

File Location: CLAUDE.md:56-60
Vulnerability Type: Persistent agent-state poisoning
Risk Level: High

Vulnerable Instruction

The relevant directive, translated into English from the repository documentation, states:

text
docs/MEMORY.md: Before the end of every session or after completing an important change, proactively update project memory without requiring a user reminder. At the same time, synchronize the update to Claude Code automatic memory under ~/.claude/projects/.../memory/MEMORY.md.

Technical Analysis

The instruction requires an agent to write state outside the project directory into Claude Code's persistent global memory. Such memory may be loaded by later sessions and projects.

Updating repository-local documentation can be appropriate for project maintenance. Synchronizing repository-derived information into ~/.claude/projects/, however, crosses the Skill's trust boundary and is unnecessary for its declared crawler, scheduler, RSS, classification, and DingTalk functions.

Because repository documents and future feedback entries can be modified by contributors, this behavior creates a route by which repository-controlled content can be converted into persistent agent state. The instruction also says that the update must happen proactively, without obtaining user approval.

Attack Path

An attacker or untrusted contributor adds behavioral guidance to repository memory, feedback, state, or related project documents.

An agent loads AGENTS.md and CLAUDE.md while working in the repository.

The agent follows the directive to derive or update project memory.

The agent writes the repository-derived content to the external ~/.claude/projects/.../memory/MEMORY.md location.

A subsequent Claude session loads that persistent memory.

The attacker-controlled guidance can continue influencing agent behavior after the original Skill ...[truncated 584 chars]
Remediation: View remediation
Remediation Suggestions

Remove the requirement to write or synchronize data under ~/.claude/projects/.

Restrict project history and design notes to a repository-local file such as docs/MEMORY.md.

Require explicit, informed user authorization before writing any state outside the project root.

Treat repository feedback, TODO files, bug reports, and crawled content as untrusted data rather than executable agent instructions.

If persistent memory integration is genuinely required, use an allowlisted schema containing factual project metadata only.

Display the exact proposed memory changes and destination before requesting approval.

Prevent stored project content from containing agent-priority, tool-use, safety-policy, or output-control directives.

T01 · Skill Instruction Hijacking

Error

Location: prompts/claude_loop_prompt.txt:5
Finding: Repository-Controlled Files Can Override Agent Task Priority and Output
Content: View full analysis
Vulnerability Details

File Location: prompts/claude_loop_prompt.txt:5-28,44-55; related directives in CLAUDE.md:78-96,108-131,164-187
Vulnerability Type: Agent instruction and workflow hijacking
Risk Level: High

Vulnerable Instruction

text
You must obey these rules: **STEP 0 — Bug fix inbox (HIGHEST PRIORITY, do this first):** - List all `.md` files in `bug_fix/` (excluding `bug_fix/resolved/`) - If any exist, they are urgent bug reports from the user. For each file: a. Read it fully — it contains error logs, root cause analysis, and fix suggestions b. Fix the bug following the analysis and suggestions in the file c. If the file includes a verification method, run it to confirm the fix d. Append a resolved entry to docs/FEEDBACK.md (for traceability) e. Move the file to `bug_fix/resolved/` **STEP 2 — Feedback self-correction:** - Read all `status: open` entries in docs/FEEDBACK.md - For each open feedback, apply its `action` as a binding constraint on your work this round **STEP 5 — Output:** At the end, output ONLY one JSON object. No markdown, no code fences, no prose.

Technical Analysis

The prompt declares repository files to be a highest-priority source of instructions and characterizes their contents as user-authored without verification. It further makes the action field of docs/FEEDBACK.md a binding constraint.

Files in a source repository are data controlled by repository contributors. They are not inherently authenticated user instructions. Treating them as binding creates an indirect prompt-injection channel: a contributor can place arbitrary operational requests in the bug inbox or feedback file and rely on the agent to execute them before the requested active task.

The forced output requirement can also suppress ordinary warnings or contextual explanations by requiring the agent to return only a repository-defined JSON object.

Attack Path

...[truncated 1295 chars]
Remediation: View remediation
Remediation Suggestions

Remove terms such as HIGHEST PRIORITY, must obey, and binding constraint from instructions governing repository-controlled files.

Explicitly state that platform, system, developer, and current user instructions remain authoritative.

Treat bug reports, TODOs, feedback, logs, crawled pages, and state files as untrusted input.

Parse feedback through a restrictive schema and allowlist permissible actions.

Require user confirmation before executing commands or changes sourced from repository documents.

Reject entries requesting access outside the repository, secret retrieval, network transmission, persistence, policy changes, or modification of agent configuration.

Do not claim that repository files are authored by the user unless their provenance is authenticated.

Permit security warnings and execution summaries even when machine-readable output is requested.

Run the loop in a restricted sandbox with repository-only write access, limited network access, and no ambient credentials.

T08 · Insecure Dependencies

Warning

Location: requirements.txt:1
Finding: Dependency Installation Is Not Reproducibly Locked or Hash-Verified
Content: View full analysis
Vulnerability Details

File Location: requirements.txt:1-31; installation command in SKILL.md:27-31
Vulnerability Type: Mutable third-party dependency resolution
Risk Level: Medium

Vulnerable Configuration

text
scrapy>=2.11,<3.0 sqlalchemy>=2.0,<3.0 apscheduler>=3.10,<4.0 pyyaml>=6.0,<7.0 click>=8.1,<9.0 jinja2>=3.1,<4.0 requests>=2.31,<3.0 curl_cffi>=0.7.0 structlog>=24.0 feedgen>=1.0,<2.0 pytest>=8.0 pytest-cov>=5.0

bash
uv venv .venv uv pip install -r requirements.txt

Technical Analysis

The initialization procedure resolves and installs package versions at execution time. The project does not provide an audited lock file or require package hashes. Several dependencies have no upper version bound, while the remaining dependencies permit broad version ranges.

No dependency-confusion package, typosquatted package, malicious source URL, or known malicious component was identified in the reviewed file. The risk instead arises from mutable supply-chain resolution: two installations at different times can receive materially different code.

Python package installation can execute build-backend logic for source distributions. A compromised upstream release, package-index account, mirror, or dependency could therefore introduce code during installation or runtime.

Attack Path

A user follows SKILL.md and runs the dependency installation command.

uv resolves versions currently available from the configured package index.

A newly published, compromised, or otherwise unreviewed version satisfies one of the broad constraints.

The package is downloaded without verification against a repository-approved cryptographic hash.

Package build or installation behavior executes, or the dependency is imported when the Skill runs.

Malicious dependency code receives the permissions of the ...[truncated 457 chars]
Remediation: View remediation
Remediation Suggestions

Generate and commit a reproducible lock file with exact package and transitive-dependency versions.

Require cryptographic hashes for downloaded artifacts where supported.

Review and update the lock file through a controlled dependency-update process.

Pin the package index to an approved HTTPS source and disable unintended extra indexes.

Separate runtime dependencies from development and test dependencies so normal Skill installation does not install unnecessary packages.

Use automated vulnerability and provenance scanning for each locked release.

Prefer binary wheels from trusted sources and review any dependency requiring a source build.

Install dependencies in an isolated environment without unnecessary credentials or filesystem permissions.

SkillSpector

Vulnerability Patterns

Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger

Findings (93)

Tp4

High

Category: MCP Tool Poisoning
Confidence: 99% confidence
Finding: 声明描述的是一个面向 Webnovel/ReelShorts 站点的定向内容抓取与分级、钉钉播报技能；但提供的代码并未体现任何网页抓取、站点访问、内容分级、短剧/书籍处理或钉钉通知行为。相反，代码的核心功能是一个围绕 claude 命令行的任务循环调度脚本，用于从本地 TODO 中取任务、生成 prompt、调用 Claude、解析 JSON 结果并重试未完成任务。其主要目的、操作对象和资源访问范围都与声明明显不符，因此属于显著描述-行为不匹配。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 98% confidence
Finding: 声明描述的是一个包含“定向抓取 + 内容分级 + 钉钉播报”的技能，而提供的代码片段只覆盖其中的“内容分级”子功能。代码通过数据库查询 Chapter/Episode/Novel，并基于更新时间和新增内容数量进行优先级分类，然后写回数据库。它没有展示任何外部站点访问、抓取逻辑、网络请求、解析 Webnovel/ReelShorts 内容，也没有任何钉钉消息发送相关实现。因此，就该代码片段本身而言，其实际行为与声明的整体目的存在明显不一致：声明强调抓取和播报，代码实际仅是分类器模块。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 98% confidence
Finding: 声明描述强调的是内容抓取、内容分级和钉钉播报等业务能力；而实际代码仅涉及本地配置管理，不包含任何抓取站点内容、内容分级、消息推送或钉钉集成逻辑。虽然“站点配置加载”可能是爬虫系统的配套实现细节，但就该代码片段本身而言，其直接行为与声明的核心功能明显不一致，因此应判定为描述与代码行为不匹配。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 95% confidence
Finding: 该代码块的核心功能是读取数据库中的小说和章节数据，构造 FeedGenerator 生成 RSS/Atom feed，并输出到文件。虽然它支持按 site 和 grade 过滤，和描述中的“内容分级”部分部分相关，但描述强调的主要能力是“定向抓取站点内容”以及“钉钉播报”，这些在当前代码中都没有出现。相反，代码实现了一个未声明的订阅源生成功能，因此描述与实际行为存在明显不匹配。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 92% confidence
Finding: 该代码块并不直接实现“抓取 Webnovel/ReelShorts 内容、内容分级、钉钉播报”这些面向业务的核心功能，而是实现底层的任务调度器：读取配置、创建 cron 任务、管理任务状态，并调用 scrapy crawl 运行爬虫。虽然“定向抓取”相关性部分存在，因为它会触发爬虫执行，但本代码的主要职责是调度而非内容抓取本身，更没有体现内容分级或钉钉通知。因此，声明与该代码块的实际行为存在明显偏差。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 97% confidence
Finding: 该代码片段只是 src/sitemap 模块的初始化文件，通过导出 create_sitemap、load_sitemap、save_sitemap、add_entry、remove_entry、query_entries 等函数来提供站点地图管理能力。与声明中的“抓取 Webnovel/ReelShorts 内容、内容分级、钉钉播报”相比，当前代码的主要目的明显不同，且未显示任何与目标站点访问、内容抓取、分级处理或消息通知相关的实现。因此该描述与实际代码行为存在明显不匹配。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 98% confidence
Finding: 该代码块的核心职责是维护一个 JSON 格式的站点地图：记录 URL、external_id、title、content_type、last_modified、status，并提供增删查改及保存/加载能力。它没有网络请求、页面解析、站点适配、内容下载，也没有任何内容分级或钉钉接口调用。虽然 sitemap 可作为抓取系统的辅助组件，但就此代码块本身而言，其实际行为与“定向抓取书籍/短剧内容，支持内容分级与钉钉播报”的描述存在明显偏差，且主要目的不同，因此应判定为不匹配。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 95% confidence
Finding: 代码片段实现的是一个 SQLitePipeline：在 spider 打开时初始化数据库会话，关闭时释放会话，并在处理 item 时根据类型将小说、章节、剧集数据插入或更新到 SQLite。它访问的资源是本地数据库文件，而不是外部站点或钉钉接口。声明中强调“定向抓取”以及“支持内容分级与钉钉播报”，但本片段没有任何抓取站点请求、分级判定、消息通知相关行为。虽然数据入库可能是爬虫系统的配套实现，但就该代码本身而言，其实际功能与描述存在明显不一致，尤其缺少声明中的两项显著能力（内容分级、钉钉播报），且主要职责是持久化而非抓取。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 92% confidence
Finding: 该代码的实际功能是一个面向 ReelShorts 的 Scrapy spider：发起网页/API 请求，解析系列列表、详情和 episode 列表，并产出 NovelItem/EpisodeItem。其主目的仍属于“抓取短剧内容”，与声明中的抓取方向部分一致。但声明中提到的两项重要能力——内容分级和钉钉播报——在代码里完全没有体现；此外，该代码块只覆盖 ReelShorts，未显示对 Webnovel 或其他站点的实现。因此描述比实际行为更宽，且包含当前代码未实现的关键能力，应判定为不匹配。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 94% confidence
Finding: 该代码块的实际功能是一个较为标准的 Webnovel 小说站点爬虫：根据书籍 ID 或排行榜入口发起请求，解析书籍详情页，随后解析 catalog 页面提取章节列表。它访问的资源仅限 webnovel.com 域名，输出 NovelItem 和 ChapterItem，属于小说元数据/目录抓取。相比之下，声明描述覆盖了更广的能力范围，包括 Webnovel/ReelShorts 等多个站点、书籍/短剧内容、内容分级以及钉钉播报。但在当前代码中，看不到 ReelShorts 相关逻辑、短剧相关解析、内容审核/分级处理，也没有任何钉钉消息推送代码。因此描述显著高于并偏离了该代码块的实际行为，构成描述与行为不一致。
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 99% confidence
Finding: This code does not implement or demonstrate the declared primary behavior of crawling book/short-drama content from Webnovel/ReelShorts or sending graded content notifications to DingTalk. Instead, it is focused on testing configuration utilities: reading YAML files, loading global/site settings, handling missing files, environment overrides, and cache reset behavior. While configuration support could be ancillary to a crawler, the actual code chunk’s purpose is materially different from the declared description, and the declared capabilities are not represented here.
Content: No source excerpt is available for this finding.

Credential Access

High

Category: Privilege Escalation
Confidence: 60% confidence
Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.
Content: Scanner excerpt · docker-compose.yaml (reported line 7)May include surrounding context.
yaml
# 一次性爬取任务 crawler: build: . env_file: .env volumes: - crawler_data:/app/data - ./config:/app/config:ro

Credential Access

High

Category: Privilege Escalation
Confidence: 60% confidence
Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.
Content: Scanner excerpt · docker-compose.yaml (reported line 18)May include surrounding context.
yaml
# 一次性爬取任务 crawler: build: . env_file: .env volumes: - crawler_data:/app/data - ./config:/app/config:ro

Credential Access

High

Category: Privilege Escalation
Confidence: 60% confidence
Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.
Content: Scanner excerpt · docker-compose.yaml (reported line 28)May include surrounding context.
yaml
# 一次性爬取任务 crawler: build: . env_file: .env volumes: - crawler_data:/app/data - ./config:/app/config:ro

Credential Access

High

Category: Privilege Escalation
Confidence: 60% confidence
Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.
Content: Scanner excerpt · docs/TODO.md (reported line 37)May include surrounding context.
md
# 一次性爬取任务 crawler: build: . env_file: .env volumes: - crawler_data:/app/data - ./config:/app/config:ro

External Model or Provider Selection

High

Category: Excessive Agency
Confidence: 90% confidence
Finding: The script automatically sends a dynamically built prompt to an external Claude provider, which can include repository-derived task context. In a crawler/automation skill, this creates a real risk of unintended data exfiltration to a third-party model, prompt injection from repository content such as TODO/prompt files, and uncontrolled autonomous looping over many tasks.
Content: Scanner excerpt · scripts/claude_loop.sh (reported line 166)May include surrounding context.
sh
echo set +e claude -p "$(cat "${prompt_file}")" \ --output-format json \ > "${raw_out}" 2> "${log_file}" local rc=$?

YARA rule 'exploit_framework': Exploit framework components and payloads [hacktools]

High

Category: YARA Match
Confidence: 80% confidence
Finding: YARA rule matched a hack tool or exploit indicator (offensive tools, reconnaissance, privilege escalation, or exploit frameworks).
Content: Scanner excerpt · src/logging_config.py (reported line 71)May include surrounding context.
python
""" return structlog.get_logger(name) class CrawlStats: """爬取统计收集器。""" def __init__(self) -> None: self.items_scraped: int = 0 self.items_dropped: int = 0 self.requests_made: int = 0 self.errors: int = 0 self._log = get_logger("crawl_stats") def record_item(self) -> None: self.items_scraped += 1 def record_drop(self) -> None: self.items_dropped += 1 def record_request(self) -> None: self.requests_made += 1 def record_error(self, error: str) -> None: self.errors += 1 self._log.error("crawl_error", error=error) def summary(self) -> dict[str, int]: return { "items_scraped": self.items_scraped, "items_dropped": self.items_dropped, "requests_made": self.requests_made, "errors": self.errors, } def log_summary(self) -> None: self._log.info("crawl_summary", **self.summary())

Natural-Language Policy Violations

Medium

Category: Not specified by scanner
Confidence: 86% confidence
Finding: The file's operational instructions are written as mandatory guidance in Chinese and present this document as the sole source of truth, which effectively imposes a language/locale constraint on agent behavior. There is no opt-in, alternative language path, or justification that this skill is region-specific only.
Content: No source excerpt is available for this finding.

Ssd 3

Medium

Category: Not specified by scanner
Confidence: 98% confidence
Finding: The instruction requires the agent to proactively write session outcomes, design decisions, pitfalls, and release records into both project memory and a global Claude memory location under ~/.claude/projects/, without any data-minimization or sensitivity limits. This creates a real risk of persisting user-provided secrets, proprietary content, credentials, or sensitive operational context beyond the immediate task, increasing exposure across future sessions and tools.
Content: No source excerpt is available for this finding.

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 90% confidence
Finding: The README promotes targeted crawling of third-party sites and scheduled DingTalk broadcasting, but provides no guidance on authorization, robots.txt/compliance, rate limiting, privacy handling, or redistribution constraints. In a crawler skill, that omission increases the likelihood of misuse for scraping copyrighted or personal data at scale and amplifies downstream exposure by automatically broadcasting collected content.
Content: No source excerpt is available for this finding.

Undeclared Tool Scope

Medium

Category: MCP Least Privilege
Confidence: 95% confidence
Finding: The skill requests or clearly requires powerful capabilities—shell execution, network access, filesystem reads/writes, and environment variable access—yet does not declare any explicit tool scope or permissions boundary. This creates an authorization gap where an agent may invoke the skill with broader privileges than users or reviewers expect, increasing the chance of unintended command execution, data access, or outbound exfiltration.
Content: No source excerpt is available for this finding.

Vague Triggers

Medium

Category: Not specified by scanner
Confidence: 92% confidence
Finding: The trigger conditions are broad enough to overlap with ordinary user requests about novels, short dramas, status checks, RSS, or scheduling, which can cause accidental activation of a networked shell-capable skill. In this context, false triggering is more dangerous because the skill can install dependencies, crawl external sites, write local files, and send outbound notifications.
Content: No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

Category: Not specified by scanner
Confidence: 94% confidence
Finding: The file is written as operational instructions entirely in Chinese and directs the executor to follow them, but it does not offer any language choice, opt-in, or justification for requiring Chinese. This can violate a language/locale policy when the skill is used in broader contexts where users or agents may expect locale neutrality or explicit language selection.
Content: No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The file’s human-readable comments and operational notes are entirely in Chinese, with no indication that this skill is region-specific or that alternate language support is available. Under the stated policy, forcing a specific language without user opt-in or clear justification is a natural-language policy violation.
Content: No source excerpt is available for this finding.

External Transmission

Medium

Category: Data Exfiltration
Confidence: 50% confidence
Finding: Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.
Content: Scanner excerpt · config/sites/reelshorts.yaml (reported line 20)May include surrounding context.
yaml
# 剧集详情页（如果有网页版） series_detail: https://www.reelshort.com/drama/{series_id} # API 端点（需通过抓包确认，以下为推测格式） series_list_api: https://api.reelshort.com/v1/series/list?page={page}&size=20 episode_list_api: https://api.reelshort.com/v1/series/{series_id}/episodes # 页面选择器（网页版）

Static analysis

No suspicious patterns detected.

Overview

A.I.G

T02 · Agent Memory Poisoning

Vulnerability Details

Vulnerable Instruction

Technical Analysis

Attack Path

Remediation Suggestions

T01 · Skill Instruction Hijacking

Vulnerability Details

Vulnerable Instruction

Technical Analysis

Attack Path

Remediation Suggestions

T08 · Insecure Dependencies

Vulnerability Details

Vulnerable Configuration

Technical Analysis

Attack Path

Remediation Suggestions

SkillSpector

Tp4

Tp4

Tp4

Tp4

Tp4

Tp4

Tp4

Tp4

Tp4

Tp4

Tp4

Credential Access

Credential Access

Credential Access

Credential Access

External Model or Provider Selection

YARA rule 'exploit_framework': Exploit framework components and payloads [hacktools]

Natural-Language Policy Violations

Ssd 3

Missing User Warnings

Undeclared Tool Scope

Vague Triggers

Natural-Language Policy Violations

Natural-Language Policy Violations

External Transmission

Static analysis