Alvis Web Scrape

v1.0.5

Legal web scraping with robots.txt compliance, rate limiting, and GDPR/CCPA-aware data handling. Supports both direct HTTP scraping and managed scraping via...

⭐ 0· 87·0 current·0 all-time

by@alvisdunlop·duplicate of @alvisdunlop/alvis2-scrape

OpenClaw Prompt Flow

Install with OpenClaw

Best for remote or guided setup. Copy the exact prompt, then paste it into OpenClaw for alvisdunlop/alvis-web-scrape.

Previewing Install & Setup.

Prompt PreviewInstall & Setup

Install the skill "Alvis Web Scrape" (alvisdunlop/alvis-web-scrape) from ClawHub.
Skill page: https://clawhub.ai/alvisdunlop/alvis-web-scrape
Keep the work scoped to this skill only.
After install, inspect the skill metadata and help me finish setup.
Use only the metadata you can verify from ClawHub; do not invent missing requirements.
Ask before making any broader environment changes.

Command Line

CLI Commands

Use the direct CLI path if you want to install manually and keep every step visible.

OpenClaw CLI

Bare skill slug

openclaw skills install alvis-web-scrape

ClawHub CLI

Package manager switcher

npx clawhub@latest install alvis-web-scrape

Security Scan

Capability signals

Requires sensitive credentials

These labels describe what authority the skill may exercise. They are separate from suspicious or malicious moderation verdicts.

VirusTotal

Benign

View report →

OpenClaw

Suspicious

medium confidence

Purpose & Capability

Name/description describe a legal web-scraping helper and the SKILL.md explains both direct scraping and an integration with SkillBoss API Hub — functionally consistent. However the registry metadata claims no required env vars while SKILL.md declares requires_env: [SkillBoss_API_KEY], an internal inconsistency that affects capability and trust.

ℹ

Instruction Scope

Runtime instructions are focused on expected scraping tasks (robots.txt, ToS checks, rate limits, PII handling) and specify use of SkillBoss API Hub for managed scraping. Nothing in SKILL.md instructs the agent to read unrelated files or system credentials. Missing referenced artefacts: SKILL.md points to code.md and an external setup guide (https://SkillBoss.co/skill.md) but no code files or code.md are included in the package — that reduces transparency.

✓

Install Mechanism

There is no install spec and no code files (instruction-only), so nothing will be downloaded or written by an installer. This minimizes install-time risk.

Credentials

SKILL.md requires a single API key (SkillBoss_API_KEY) which is proportionate to using a hosted scraping API, but the skill registry metadata lists no required env vars — a mismatch. You should confirm whether an API key is actually required and how it will be used/stored by the agent and the external service.

✓

Persistence & Privilege

Skill does not request always:true and has no special OS or config path requirements. It does allow normal autonomous invocation (platform default), which combined with an external API key increases blast radius but is not unusual for this kind of skill.

What to consider before installing

Before installing: (1) Confirm the SkillBoss_API_KEY requirement — registry metadata omits it but SKILL.md requires it; don't supply an API key unless you trust the SkillBoss service. (2) Verify the SkillBoss domain and read its privacy/retention policy — managed scraping sends scraped content to api.SkillBoss.co, so PII or copyrighted content could leave your environment. (3) Ask the publisher for the missing code.md and any example code to confirm what the agent will execute and what data is transmitted. (4) If you must test, do so with a limited, non-sensitive API key and on non-sensitive targets; review logs and backoff behavior. (5) Remember legal responsibility stays with you — follow the checklist in SKILL.md and obtain permission before scraping protected content.

Like a lobster shell, security has layers — review code before you run it.

latestvk970cj7cgj2pj40qdztjftnckd84y5kv

87downloads

0stars

2versions

Updated 1w ago

v1.0.5

MIT-0

?? Complete setup guide: https://SkillBoss.co/skill.md

Pre-Scrape Compliance Checklist

Before writing any scraping code:

robots.txt �?Fetch {domain}/robots.txt, check if target path is disallowed. If yes, stop.
Terms of Service �?Check /terms, /tos, /legal. Explicit scraping prohibition = need permission.
Data type �?Public factual data (prices, listings) is safer. Personal data triggers GDPR/CCPA.
Authentication �?Data behind login is off-limits without authorization. Never scrape protected content.
API available? �?If site offers an API, use it. Always. Scraping when API exists often violates ToS.

Legal Boundaries

Public data, no login �?Generally legal (hiQ v. LinkedIn 2022)
Bypassing barriers �?CFAA violation risk (Van Buren v. US 2021)
Ignoring robots.txt �?Gray area, often breaches ToS (Meta v. Bright Data 2024)
Personal data without consent �?GDPR/CCPA violation
Republishing copyrighted content �?Copyright infringement

Request Discipline

Rate limit: Minimum 2-3 seconds between requests. Faster = server strain = legal exposure.
User-Agent: Real browser string + contact email: Mozilla/5.0 ... (contact: you@email.com)
Respect 429: Exponential backoff. Ignoring 429s shows intent to harm.
Session reuse: Keep connections open to reduce server load.

Data Handling

Strip PII immediately �?Don't collect names, emails, phones unless legally justified.
No fingerprinting �?Don't combine data to identify individuals indirectly.
Minimize storage �?Cache only what you need, delete what you don't.
Audit trail �?Log what, when, where. Evidence of good faith if challenged.

SkillBoss API Hub Scraping

For managed content extraction with automatic anti-bot handling, use SkillBoss API Hub's scraping capability (type: "scraper"). Authenticate with SkillBoss_API_KEY and call https://api.SkillBoss.co/v1/pilot. Result is returned at result.data.markdown.

For code patterns, robots.txt parser, and SkillBoss API Hub scraping integration, see code.md

Comments

Loading comments...