Web Crawl
v1.0.0Advanced web crawling and content extraction tool with multiple extraction modes
⭐ 0· 130·0 current·0 all-time
by不白@nowhitestar
MIT-0
Download zip
LicenseMIT-0 · Free to use, modify, and redistribute. No attribution required.
Security Scan
OpenClaw
Benign
medium confidencePurpose & Capability
The name/description (web crawling and extraction) match the included code (web_crawl.py, research.py), README, and examples. No unrelated credentials, binaries, or config paths are requested.
Instruction Scope
SKILL.md and examples instruct the agent to run searches, crawl URLs, and synthesize results — this is expected. However, the skill enables fetching arbitrary URLs, which can reach internal or otherwise sensitive endpoints if the agent has network access (SSRF-like risk). EXAMPLES.md also shows an exec-style local python invocation that, if executed, runs code from the skill workspace; the agent should not run arbitrary shell execs without user consent.
Install Mechanism
No install spec is provided (instruction-only installation). The package contains Python source files and documents pip dependencies (requests, beautifulsoup4). No remote downloads or unusual install steps were found.
Credentials
The skill requests no environment variables, credentials, or config paths. Its network access is inherent to crawling and is proportional to its stated purpose.
Persistence & Privilege
No elevated persistence or 'always' flag is requested. The skill is user-invocable and allows autonomous invocation by default (normal for skills) but does not appear to modify other skills or system-wide settings.
Assessment
This skill appears to do what it says (crawl and extract web content), but take the following precautions before installing or allowing it to run autonomously:
- Review the full web_crawl.py file in your environment (the provided manifest shows the file content truncated), because hidden code or unexpected behavior could be present in the omitted portion.
- Be aware that crawling arbitrary URLs can access internal services (metadata endpoints, intranet, admin consoles) if the agent has network access — avoid allowing this skill to scan sensitive hosts or provide a restricted allowlist of target domains.
- Examples include an exec-style command that runs Python from the skill workspace. Do not run shell execs or workspace-local scripts without explicit review and user consent.
- Ensure required Python dependencies (requests, beautifulsoup4) are installed from trusted sources.
If you want higher assurance, ask the skill author for a full unobfuscated source, or run it in a sandboxed environment and limit its outbound network access and allowed target domains.Like a lobster shell, security has layers — review code before you run it.
latestvk97e0qjy1ws4mgqygv7yxeeykx836jx5
License
MIT-0
Free to use, modify, and redistribute. No attribution required.