Smart Scraper

• Outcome: Review
• Latest audit: Sep 16, 2026 · 9:17 AM UTC
• Version: 1.3.7

Overview

This web-scraping skill is mostly purpose-aligned, but its claimed SSRF protection and storage limits are materially weaker than advertised.

Install only if you are comfortable with a local web scraper that can make outbound HTTP/HTTPS requests and store scraped page data when cache or watch features are used. Avoid giving it internal, authenticated, sensitive, or attacker-controlled URLs unless the runtime has network egress controls blocking loopback, private, link-local, and cloud metadata addresses.

A.I.G

Vulnerability Patterns

• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally

Findings (2)

T09 · Insecure Skill Coding Practices

Error

• Location: smart-scraper.js:107

Finding

SSRF Protection Can Be Bypassed Through DNS Resolution and the Exported Fetch Function

Content

View full analysis

0) { const redirectValidation = validateUrl(url); if (!redirectValidation.valid) { throw new Error(`Redirect blocked: ${redirectValidation.error}`); } } return new Promise((resolve, reject) => { const isHttps = url.startsWith('https'); const client = isHttps ? https : http; const req = client.get(url, { headers: { 'User-Agent': 'Mozilla/5.0 (compatible; SmartScraper/1.0)' }, timeout: REQUEST_TIMEOUT }, (res) => { // ...[truncated 3474 chars]

Remediation

View remediation

T09 · Insecure Skill Coding Practices

Warning

• Location: smart-scraper.js:171

Finding

Unbounded HTTP Response Buffering Enables Memory and Disk Resource Exhaustion

Content

View full analysis

data += chunk); res.on('end', () => resolve(data)); ``` When caching is enabled, all parsed data is subsequently serialized without enforcing `MAX_CACHE_BYTES`: ```javascript if (useCache) { console.error(`⚠️ [smart-scraper] Caching scraped data to disk: ${CACHE_FILE}`); console.error(` Stored: title, headings, paragraphs, links, tables, lists, prices, images, metadata`); console.error(` To enable: add --cache to your command`); cache[cacheKey] = { timestamp: Date.now(), data }; saveJSON(CACHE_FILE, cache); } ``` Although a limit is declared, it is not applied before or during writes: ```javascript const MAX_CACHE_BYTES = 10 * 1024 * 1024; // 10MB ``` ### Technical Analysis `fetchPage()` concatenates every response chunk into a JavaScript string until the server closes the response. The 15-second request timeout does not impose a response-size limit, and a remote server can transmit a large amount of data within that period. Repeated string concatenation, parsing, regex processing, and JSON serialization can require substantially more memory than the raw response size. A sufficiently large response can cause excessive garbage collection, event-loop stalls, or process termination due to heap exhaustion. The cache implementation limits the number of entries but does not calculate the serialized cache size before writing. `MAX_CACHE_BYTES` is declared but unused. `MAX_JSON_FILE_BYTES` only prevents loading an already oversized JSON file; it does not prevent creating one. Consequently, an attacker-controlled response may also produce a cache file or watch snapshot larger than the documented 1 ...[truncated 1438 chars]

Remediation

View remediation

{ received += chunk.length; if (received > MAX_RESPONSE_BYTES) { req.destroy(new Error('Response body exceeds configured limit')); return; } chunks.push(chunk); }); res.on('end', () => resolve(Buffer.concat(chunks).toString('utf8'))); ``` 4. Reject responses whose declared `Content-Length` exceeds the configured limit, while still enforcing the streaming counter because the header may be absent or false. 5. Limit decompressed size if compressed HTTP responses are supported in the future. 6. Enforce `MAX_CACHE_BYTES` against the complete serialized cache before writing. 7. Use atomic cache writes through a temporary file followed by a rename to avoid corruption after resource failures. 8. Apply equivalent limits to watch snapshots and individual extracted collections. 9. Add maximum counts for paragraphs, links, tables, rows, list items, images, and metadata fields. 10. Add regression tests using oversized fixed-length, chunked, and slowly streamed responses. ]]>

SkillSpector

Vulnerability Patterns

• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• Output Handling: Unvalidated Output Injection, Cross-Context Output, Unbounded Output

Findings (31)

Memory Manipulation

High

• Category: Memory Poisoning
• Confidence: 80% confidence

Finding

Skill manipulates agent memory, state, or stored context. Memory corruption can alter personality, override safety rules, or cause unpredictable behavior.

Content

Scanner excerpt · FIX_SUMMARY.md (reported line 25)May include surrounding context.

md

- Additional security note about user consent

### 2. Added explicit user consent section
Added a clear statement that caching behavior requires explicit user consent, with default being disabled.

## Files Modified
1. `/home/jarvis/.openclaw/workspace/skills/smart-scraper/SKILL.md` - Enhanced documentation and help text with clearer consent mechanisms

Anti-Refusal Statement

High

• Category: Anti-Refusal
• Confidence: 80% confidence

Finding

Skill instructs the agent to omit warnings, disclaimers, or ethical commentary. Stripping safety caveats hides risk from the user and is a common jailbreak preamble.

Content

Scanner excerpt · skills/smart-scraper-web/AUDIT.md (reported line 153)May include surrounding context.

md

**Verification:**
- Run `node smart-scraper.js --extract https://example.com` — warning printed to stderr on every cache write
- Run `node smart-scraper.js --extract --no-cache https://example.com` — no cache file created, no warning

---

Hidden Instructions

High

• Category: Prompt Injection
• Confidence: 70% confidence

Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · comparison.html (reported line 40)May include surrounding context.

html

<h1>web_fetch vs Smart Scraper</h1>
<p class="subtitle">Same page: <code>https://www.wikipedia.org</code> — what each tool actually returns</p>

<!-- MODE: ALL -->
<div class="mode-section">
  <div class="mode-title">Mode: Extract Everything</div>
  <div class="grid">

Hidden Instructions

High

• Category: Prompt Injection
• Confidence: 70% confidence

Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · skills/smart-scraper-web/comparison.html (reported line 40)May include surrounding context.

html

<h1>web_fetch vs Smart Scraper</h1>
<p class="subtitle">Same page: <code>https://www.wikipedia.org</code> — what each tool actually returns</p>

<!-- MODE: ALL -->
<div class="mode-section">
  <div class="mode-title">Mode: Extract Everything</div>
  <div class="grid">

Cloud Metadata Access

High

• Category: Server-Side Request Forgery
• Confidence: 90% confidence

Finding

Code accesses a cloud instance metadata endpoint (e.g. 169.254.169.254). A single request can return temporary IAM credentials, making this a high-value SSRF target for credential theft.

Content

Scanner excerpt · skills/smart-scraper-web/smart-scraper.js (reported line 76)May include surrounding context.

js

/^fe[89ab][0-9a-f]:/, // IPv6 link-local
];
const CLOUD_METADATA_IPS = [
  '169.254.169.254',
  '169.254.170.2',
  '169.254.169.253',
];

Cloud Metadata Access

High

• Category: Server-Side Request Forgery
• Confidence: 90% confidence

Finding

Code accesses a cloud instance metadata endpoint (e.g. 169.254.169.254). A single request can return temporary IAM credentials, making this a high-value SSRF target for credential theft.

Content

Scanner excerpt · smart-scraper.js (reported line 76)May include surrounding context.

js

/^fe[89ab][0-9a-f]:/, // IPv6 link-local
];
const CLOUD_METADATA_IPS = [
  '169.254.169.254',
  '169.254.170.2',
  '169.254.169.253',
];

Cloud Metadata Access

High

• Category: Server-Side Request Forgery
• Confidence: 90% confidence

Finding

Code accesses a cloud instance metadata endpoint (e.g. 169.254.169.254). A single request can return temporary IAM credentials, making this a high-value SSRF target for credential theft.

Content

Scanner excerpt · test/run-tests.js (reported line 179)May include surrounding context.

js

/^fe[89ab][0-9a-f]:/, // IPv6 link-local
];
const CLOUD_METADATA_IPS = [
  '169.254.169.254',
  '169.254.170.2',
  '169.254.169.253',
];

Cloud Metadata Access

High

• Category: Server-Side Request Forgery
• Confidence: 90% confidence

Finding

Code accesses a cloud instance metadata endpoint (e.g. 169.254.169.254). A single request can return temporary IAM credentials, making this a high-value SSRF target for credential theft.

Content

Scanner excerpt · test/run-tests.js (reported line 180)May include surrounding context.

js

/^fe[89ab][0-9a-f]:/, // IPv6 link-local
];
const CLOUD_METADATA_IPS = [
  '169.254.169.254',
  '169.254.170.2',
  '169.254.169.253',
];

Credential Access

High

• Category: Privilege Escalation
• Confidence: 60% confidence

Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · test/run-tests.js (reported line 50)May include surrounding context.

js

const r3 = SS.validateUrl('ftp://example.com');
  assert(r3.valid === false && r3.error.includes('Blocked'), 'FTP scheme is blocked');

  const r4 = SS.validateUrl('file:///etc/passwd');
  assert(r4.valid === false && r4.error.includes('Blocked'), 'File scheme is blocked');

  const r5 = SS.validateUrl('http://127.0.0.1:8080');

Unbounded Output

Medium

• Category: Output Handling
• Confidence: 75% confidence

Finding

Output size or generation rate is not bounded. Unbounded output enables denial-of-service through resource exhaustion, log flooding, or context-window stuffing.

Content

Scanner excerpt · AUDIT.md (reported line 178)May include surrounding context.

md

#### 11. JSON Parse — No Size Limit
**Severity:** Low  
**Location:** `loadJSON()`  
**Issue:** `JSON.parse()` has no size limit.

**Impact:** Minimal — the try/catch provides protection.

Skill Enumeration

Medium

• Category: Agent Snooping
• Confidence: 80% confidence

Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · FIX_SUMMARY.md (reported line 28)May include surrounding context.

md

## Files Modified
1. `/home/jarvis/.openclaw/workspace/skills/smart-scraper/skills/smart-scraper-web/smart-scraper.js` - Changed default cache behavior
2. `/home/jarvis/.openclaw/workspace/skills/smart-scraper/SKILL.md` - Updated documentation and help text

## Verification
The fix ensures that:

Skill Enumeration

Medium

• Category: Agent Snooping
• Confidence: 80% confidence

Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · SECURITY_AUDIT_FIX_SUMMARY.md (reported line 34)May include surrounding context.

md

## Files Modified
1. `/home/jarvis/.openclaw/workspace/skills/smart-scraper/skills/smart-scraper-web/smart-scraper.js` - Changed default cache behavior
2. `/home/jarvis/.openclaw/workspace/skills/smart-scraper/SKILL.md` - Updated documentation and help text

## Verification
The fix ensures that:

Skill Enumeration

Medium

• Category: Agent Snooping
• Confidence: 80% confidence

Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · SECURITY_FIX_SUMMARY.md (reported line 27)May include surrounding context.

md

## Files Modified
1. `/home/jarvis/.openclaw/workspace/skills/smart-scraper/skills/smart-scraper-web/smart-scraper.js` - Changed default cache behavior
2. `/home/jarvis/.openclaw/workspace/skills/smart-scraper/SKILL.md` - Updated documentation and help text

## Verification
The fix ensures that:

Intent-Code Divergence

Medium

• Category: Not specified by scanner
• Confidence: 96% confidence

Finding

The top summary says there are 2 low findings still merely noted, while lines L088-L094 explicitly mark both low findings as fixed/resolved. This is an active contradiction in the document about the skill's current behavior and audit status, not just an omission.

Content

No source excerpt is available for this finding.

Unbounded Output

Medium

• Category: Output Handling
• Confidence: 75% confidence

Finding

Output size or generation rate is not bounded. Unbounded output enables denial-of-service through resource exhaustion, log flooding, or context-window stuffing.

Content

Scanner excerpt · AUDIT.md (reported line 175)May include surrounding context.

md

### 🟢 LOW (All Resolved)

#### 9. JSON Parse — No Size Limit ✅ FIXED (2026-06-18)
**Severity:** Low — **Resolved**  
**Fix:** Added `fs.statSync()` size check before parse — rejects files over 10MB with warning.

Unbounded Output

Medium

• Category: Output Handling
• Confidence: 75% confidence

Finding

Output size or generation rate is not bounded. Unbounded output enables denial-of-service through resource exhaustion, log flooding, or context-window stuffing.

Content

Scanner excerpt · skills/smart-scraper-web/AUDIT.md (reported line 88)May include surrounding context.

md

### 🟢 LOW (All Resolved)

#### 9. JSON Parse — No Size Limit ✅ FIXED (2026-06-18)
**Severity:** Low — **Resolved**  
**Fix:** Added `fs.statSync()` size check before parse — rejects files over 10MB with warning.

Intent-Code Divergence

Medium

• Category: Not specified by scanner
• Confidence: 97% confidence

Finding

Lines L194-L195 state there are 2 low findings 'Noted (no action required)' and 11 resolved, but the detailed entries at L088-L094 say both low findings were resolved. The totals are also internally inconsistent, since 12 total with 11 resolved and 2 noted does not add up cleanly.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium

• Category: Not specified by scanner
• Confidence: 98% confidence

Finding

The documentation gives conflicting statements about whether caching is enabled by default or only when --cache is supplied. In a tool that stores scraped page content locally, this ambiguity can cause users or agents to persist sensitive, personal, or copyrighted data when they believed caching was off, creating privacy and data-handling risk.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

Ambiguous invocation and caching semantics are a security-relevant quality issue because they affect when network access and disk persistence happen. An agent may invoke the skill under false assumptions, leading to unintended scraping or retention of data that should not have been fetched or cached.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium

• Category: Not specified by scanner
• Confidence: 96% confidence

Finding

The comparison and summary sections disagree on whether caching requires an explicit --cache flag, which can mislead operators about when local persistence occurs. Because the skill extracts and stores rich page content, inconsistent instructions increase the chance of unintended retention of sensitive or regulated data.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium

• Category: Not specified by scanner
• Confidence: 94% confidence

Finding

The header advertises extraction and parsing modes but omits the implemented watch/snapshot behavior that persistently stores baselines and diffs on disk. In an agent-skill context, undisclosed persistence materially changes the data-handling and privacy profile of the tool, making operator consent and review less informed.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium

• Category: Not specified by scanner
• Confidence: 87% confidence

Finding

The skill stores scraped content and page-derived data in persistent cache/snapshot files under the workspace without a clear necessity tied to a narrowly defined purpose. For an unknown-purpose skill, this can retain sensitive or proprietary page content longer than expected and expand local data exposure if the workspace is shared or later exfiltrated.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

saveSnapshot writes full page-derived data structures to disk during watch mode, but the user-facing flow does not warn that persistent snapshots are being created and updated. Silent retention is risky because monitored pages may contain sensitive business, personal, or copyrighted information that operators do not expect to be stored locally over time.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 93% confidence

Finding

Watch mode persists full page snapshots to disk without a prominent user warning or opt-in, which can silently store sensitive content scraped from authenticated, internal, or personal pages in the workspace. In an agent-skill context, unexpected persistence increases the chance of data retention, secondary disclosure, or later reuse by other tools or users.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low

• Category: Not specified by scanner
• Confidence: 97% confidence

Finding

The document states 'ALL CRITICAL ISSUES FIXED' and marks medium findings as 0/all fixed, but later at L233-L242 it records new medium findings (#11 and #12) discovered afterward. This creates a documentation contradiction about the current audit state and could mislead readers about whether the artifact is fully clean.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.