Smart Crawler
v1.0.0智能爬虫工具 - 企业级数据采集与反爬虫处理 | Smart Web Crawler - Enterprise data collection with anti-detection
⭐ 0· 526·2 current·2 all-time
byLv Lancer@kaiyuelv
MIT-0
Download zip
LicenseMIT-0 · Free to use, modify, and redistribute. No attribution required.
Security Scan
OpenClaw
Benign
high confidencePurpose & Capability
Name/description (enterprise web crawler with anti-detection) aligns with included code, examples, and requirements. The code implements request-based and Playwright-based crawling, proxy support, extraction, batching and dynamic rendering as claimed.
Instruction Scope
SKILL.md and README instruct installing dependencies (pip and playwright) and running tests; runtime instructions and code only perform crawling, extraction, downloading, and optional proxy use. There are no instructions to read unrelated host files, environment variables, or to send data to unexpected endpoints.
Install Mechanism
There is no automated install spec in the registry entry (instruction-only), but SKILL.md tells users to run pip install -r requirements.txt and playwright install. Dependencies are standard public packages; Playwright will download browser binaries when installed — this is expected but worth noting.
Credentials
The skill declares no required env vars or credentials. Code supports optional proxy pools and mentions databases (in README), but no credentials are requested or embedded. Any credential use would come from how a user configures proxies or DB connectors, not from the skill itself.
Persistence & Privilege
Skill is not always-enabled and does not request elevated platform privileges. It does not modify other skills or system-wide agent settings.
Assessment
This package is a normal web crawler and appears coherent with its description, but take usual precautions: install and run in a controlled environment (Playwright will download browser binaries); review any proxy or database connection strings you provide (they can contain credentials); be aware that the tool makes network requests to target URLs you supply (so don't point it at internal or sensitive endpoints without authorization); and audit any config you create for storage backends (MongoDB/MySQL) before entering secrets. If you plan to run it in an automated agent, consider sandboxing network access and limiting where results can be uploaded.Like a lobster shell, security has layers — review code before you run it.
latestvk9754rf6rf5fc9wd6p4tkp7vr5834db7
License
MIT-0
Free to use, modify, and redistribute. No attribution required.