ClawHub
Back to skill

Security audit

scrape

Security checks for vulnerabilities and agentic risk

Overview

This scraping skill is mostly coherent, but its examples undercut its legal-safety claims and can expose URLs, page content, and contact details to third parties.

Install only if you are comfortable with a scraping helper that may use SkillBoss as a third-party processor. Use a dedicated API key and contact alias, avoid URLs containing secrets, sanitize logs, require explicit approval before proceeding when robots.txt cannot be verified, and treat LLM analysis of scraped pages as untrusted unless additional safeguards are added.

A.I.G

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (4)

T09 · Insecure Skill Coding Practices

Warning
Location
code.md:25
Finding

Mandatory Contact Email Disclosure to Scraped Hosts

Content
View full analysis
requests.Session: session = requests.Session() session.headers.update({ "User-Agent": f"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 (contact: {contact_email})", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Accept-Language": "en-US,en;q=0.9", "Accept-Encoding": "gzip, deflate, br", "Connection": "keep-alive", }) return session ``` ### Technical Analysis The Skill instructs users to place a real contact email address in the `User-Agent` header. The session applies this header to every subsequent request, disclosing the address to target websites and potentially to redirected hosts, reverse proxies, content-delivery networks, analytics systems, and server-log processors. A personal email address is not technically required to fetch public content, enforce rate limits, or check `robots.txt`. Requiring it therefore exceeds the minimum data disclosure necessary for the declared scraping functionality. Reusing the same address across unrelated sites also provides a stable cross-site correlation identifier. ### Attack Path 1. A user supplies a personal or organizational email address to `create_session`. 2. The function embeds that address in the session-wide `User-Agent`. 3. The scraper requests a target URL or follows a redirect. 4. Each receiving infrastructure component can record the complete header. 5. A malicious site operator extracts the address and uses it for tracking, profiling, spam, or targe ...[truncated 420 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
code.md:7
Finding

Fail-Open robots.txt Compliance Check

Content
View full analysis
bool: parsed = urlparse(url) robots_url = f"{parsed.scheme}://{parsed.netloc}/robots.txt" rp = RobotFileParser() rp.set_url(robots_url) try: rp.read() except Exception: return True # No robots.txt = allowed return rp.can_fetch(user_agent, url) ``` ### Technical Analysis The function catches every exception raised while retrieving or parsing `robots.txt` and treats the failure as authorization to scrape. This conflates a confirmed absence of policy with DNS failures, TLS errors, connection timeouts, malformed files, authentication failures, and other unknown states. The behavior contradicts the Skill's declared guarantee that `robots.txt` will be fetched and checked before scraping. Because the check fails open, an attacker or transient network failure can bypass the intended compliance control without requiring the caller to acknowledge the uncertainty. ### Attack Path 1. The scraper calls `can_scrape` for a target URL. 2. Retrieval or parsing of `robots.txt` fails due to a network error, malformed response, TLS issue, or deliberate interference. 3. The broad `except Exception` handler returns `True`. 4. The caller interprets the result as affirmative permission. 5. Scraping proceeds without successfully evaluating the target's published policy. A target operator could also serve malformed or unstable `robots.txt` content to produce an exception, although the exact parser behavior depends on the runtime and response. ### Impact Assessment The issue does not grant an attacker local execution or additional operating-system privileges. It bypasses the Skill's own policy gate and expands the scraper's effective scope to URLs whose rules could not be verified. Consequences ...[truncated 140 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
code.md:44
Finding

Sensitive Query Data May Be Exposed Through Full-URL Logging

Content
View full analysis
Remediation
View remediation

T01 · Skill Instruction Hijacking

Warning
Location
code.md:144
Finding

Untrusted Scraped Content Is Passed Directly to an LLM Prompt

Content
View full analysis
Remediation
View remediation

SkillSpector

Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (7)

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · SKILL.md (reported line 22)May include surrounding context.

md
- **Public data, no login** — Generally legal (hiQ v. LinkedIn 2022)
- **Bypassing barriers** — CFAA violation risk (Van Buren v. US 2021)
- **Ignoring robots.txt** — Gray area, often breaches ToS (Meta v. Bright Data 2024)
- **Personal data without consent** — GDPR/CCPA violation
- **Republishing copyrighted content** — Copyright infringement

## Request Discipline

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 41)May include surrounding context.

md
## SkillBoss API Hub Scraping

For managed content extraction with automatic anti-bot handling, use SkillBoss API Hub's scraping capability (`type: "scraper"`). Authenticate with `SKILLBOSS_API_KEY` and call `https://api.skillbossai.com/v1/pilot`. Result is returned at `result.data.markdown`.

For code patterns, robots.txt parser, and SkillBoss API Hub scraping integration, see `code.md`

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · code.md (reported line 122)May include surrounding context.

md
## SkillBoss API Hub Scraping

For managed content extraction with automatic anti-bot handling, use SkillBoss API Hub's scraping capability (`type: "scraper"`). Authenticate with `SKILLBOSS_API_KEY` and call `https://api.skillbossai.com/v1/pilot`. Result is returned at `result.data.markdown`.

For code patterns, robots.txt parser, and SkillBoss API Hub scraping integration, see `code.md`

External Transmission

Medium
Category
Data Exfiltration
Confidence
91% confidence
Finding

This code sends request bodies to an external SkillBoss API endpoint, which means scraped URLs, page-derived content, or subsequent analysis inputs may leave the local environment. In a scraping skill, that is security-relevant because users may assume local/direct scraping, while managed mode introduces third-party data exposure and dependency on an external service.

Content

Scanner excerpt · code.md (reported line 125)May include surrounding context.

md
API_BASE = "https://api.skillbossai.com/v1"

def pilot(body: dict) -> dict:
    r = requests.post(
        f"{API_BASE}/pilot",
        headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
        json=body,

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill is presented as a scraping capability, but this section expands it into arbitrary LLM analysis of scraped content. That broadens the trust boundary and data flow: content gathered from external sites is transmitted to a chat endpoint for secondary processing, which can expose scraped data externally and enable prompt-injection risks from untrusted page content.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

This code introduces a general chat/LLM invocation path unrelated to the core scraping function. Because the LLM is fed raw page content from untrusted websites, the skill can be used to process attacker-controlled text, expanding the attack surface and creating a channel for unintended data disclosure or misuse beyond scraping.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

The session setup hard-codes the HTTP Accept-Language header to "en-US,en;q=0.9", which imposes an English/US locale on outbound requests. This is a natural-language locale choice without user opt-in, and the document does not explain why that locale is required.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.