Back to skill

Security audit

AI Data Scraper

Security checks for vulnerabilities and agentic risk

Overview

This is a basic scraping skill, but it needs Review because its script accepts unvalidated URLs that could reach local files or internal services and save the results.

Review this before installing or invoking it with untrusted input. Use it only for URLs you are authorized to access, avoid sensitive or internal targets, and consider adding HTTPS-only URL validation, private-network blocking, redirect controls, rate limits, and clearer documentation before production use.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
main.sh:5
Finding

Unrestricted URL Handling Enables Local File Access and Server-Side Request Forgery

Content
View full analysis

Vulnerability Details

File Location: main.sh, lines 5–7, 30, and 52
Vulnerability Type: Unrestricted URL scheme and destination handling
Risk Level: Medium

Vulnerable Code

bash
URL="${1:-}"
API_URL="${2:-}"
FORMAT="${3:-json}"
bash
local data=$(curl -sL "$URL" --compressed)
bash
local data=$(curl -sL "$API_URL" --compressed)

Technical Analysis

The script passes caller-controlled values directly to curl without validating the URL scheme, hostname, resolved IP address, port, or redirect destinations. Quoting prevents shell argument injection, but it does not constrain the network or local resources that curl may access.

Depending on the installed curl build, non-web schemes such as file:// can read locally accessible files. HTTP or HTTPS URLs can also target loopback, private-network, link-local, or cloud metadata addresses. The -L option follows redirects, so validating only an initial destination would remain insufficient because an allowed public endpoint could redirect to a prohibited internal address.

The downloaded response is subsequently written beneath ./output, and the generated path is printed. This can make data from otherwise inaccessible local or internal resources available to a caller who can invoke the skill and read its output files.

Attack Path

  1. An attacker or untrusted caller supplies a crafted URL argument.
  2. The URL references a local file using a supported scheme, such as file:///etc/passwd, or targets a loopback, private-network, link-local, or cloud metadata endpoint.
  3. curl accesses the destination because the script applies no scheme or address restrictions.
  4. If the initial endpoint redirects, curl -L follows the redirect without checking whether the new destination is internal or otherwise prohibited.
  5. The response content is stored in a timestamped file under ./output.
  6. A caller with access to t ...[truncated 1150 chars]
Remediation
View remediation

Remediation Suggestions

  1. Parse and validate each URL before invoking curl; do not rely on string-prefix checks.
  2. Allow only explicitly required schemes, preferably HTTPS:
    bash
    curl --proto '=https' --proto-redir '=https' ...
    
    If HTTP is operationally necessary, allow it explicitly and document the risk.
  3. Reject URLs containing user-information fields, ambiguous host representations, unsupported ports, or malformed hostnames.
  4. Resolve the destination hostname and reject loopback, private, link-local, multicast, unspecified, reserved, and cloud metadata ranges for both IPv4 and IPv6.
  5. Apply the same validation after every redirect. Consider disabling redirects unless required, or process each redirect manually and revalidate its destination.
  6. Protect against DNS rebinding by ensuring the validated address is the address used for the connection and by enforcing network-level egress controls.
  7. Run the scraper with a dedicated, unprivileged account and restrict its filesystem access and outbound network reachability.
  8. Store output in an access-controlled directory and avoid exposing retrieved content to callers who are not authorized to access the source resource.
  9. Add automated tests covering file://, loopback hosts, private IPv4 ranges, IPv6 loopback and unique-local ranges, link-local addresses, metadata endpoints, encoded IP representations, and redirects to blocked destinations.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Findings (7)

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill promotes web scraping, API extraction, proxy use, and scheduled monitoring but provides no warnings about privacy, terms-of-service compliance, authorization, rate limits, or impact on target systems. This can encourage misuse or non-compliant collection practices, increasing the risk of scraping personal data, violating site policies, or causing service disruption.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 22)May include surrounding context.

md
openclaw run scraper --url "https://example.com" --format "json"

# 抓取 API
openclaw run scraper --api "https://api.example.com/data" --output "data.json"

# 定时抓取
openclaw run scraper --cron "0 */6 * * *" --target "stocks"

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The script fetches arbitrary URL content with curl and writes the retrieved data to a local output file, but the operation is only accompanied by internal status logging rather than a clear user warning about network access and local data storage. Because this script handles external content and persists it to disk, it should explicitly disclose that behavior to the user.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The API path performs an outbound HTTP request and saves the response to a timestamped file, but there is no explicit warning to the user about transmitting requests to external services and storing returned data locally. The existing log messages indicate progress, but they do not function as a meaningful safety warning about privacy or system-impacting behavior.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
98% confidence
Finding

The inline usage documentation says the script should be called with --url or --api flags, but the code assigns URL from $1, API_URL from $2, and FORMAT from $3 without any flag parsing. This is an active contradiction between the documented interface and the implemented behavior, which can mislead users about what the script does and how inputs are interpreted.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
89% confidence
Finding

The operational description and headings are presented in Chinese, while there is no indication that the skill is region-specific or that users may choose their preferred language. Under the stated policy, forcing a specific language without opt-in is a natural-language policy issue.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
92% confidence
Finding

The package description includes Chinese text ("自动化数据抓取和清洗") alongside English, which imposes a language expectation without any stated user opt-in or explanation. Under the language/locale policy rule, natural-language content should either offer choice or clearly justify locale-specific wording.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.