Apify Scraper

Scrape content from sites that block bots — Twitter/X threads, Reddit, LinkedIn, YouTube comments, Google SERP, and more. Use when standard web_fetch is bloc...

Security Scan

The skill's requests and instructions are coherent with a tool that runs Apify cloud actors for scraping; it asks only for an Apify API key and python, but it references local scripts and secret-store paths that aren't included — review before use.

Purpose & Capability

Name/description (scraping sites that block bots) match the declared requirements: python3 and APIFY_API_KEY. Calling apify.com to run cloud actors is expected for this purpose.

Instruction Scope

SKILL.md instructs running a local script (scripts/apify-run.py) and polling Apify datasets; the skill bundle does not include that script. It also references an op:// secret path and a gateway plist as locations for APIFY_API_KEY. These are within the scope of using Apify but you must ensure the referenced local script and secret-store access exist and are safe before executing.

Install Mechanism

Instruction-only skill; no install spec and no code files are provided. Low installation risk because nothing is downloaded or written by the skill itself.

Credentials

Only APIFY_API_KEY is required (declared as primaryEnv), which is proportional to running Apify actors. The SKILL.md also references an op:// secret path and a gateway plist key for the API key — this is reasonable but keep in mind the agent will need access to that secret store or the APIFY_API_KEY env var.

Persistence & Privilege

always is false and the skill does not request system-wide persistence or modify other skills. It may be invoked autonomously by the agent (default), but that is expected for skills.

Assessment

What to check before installing/using: - Only provide an APIFY_API_KEY you control; the key grants Apify actions and billing under your account. Monitor billing and quotas — actors can incur costs. - The skill references a local script (scripts/apify-run.py) and a secret-store path (op://OpenClaw/Apify API Credentials/credential) but does not include the script. Do not run unknown scripts; obtain and inspect the script content before executing. - Scraped data will be processed on Apify infrastructure under your account. Consider privacy and legal/ToS risks (LinkedIn scraping is explicitly flagged as ToS-sensitive). Avoid scraping content you are not permitted to access. - The SKILL.md includes a dashboard account label (“redditech”) and other documentation-like text; treat these as examples, not defaults. - If you want higher assurance, ask the author for the apify-run.py script and a minimal example of the exact API calls the agent will make, or run sample commands manually from a controlled environment before granting the skill access to secrets.

Apify Scraper Skill

Use this skill when you need to scrape content from sites that block bots — Twitter/X threads, Reddit, LinkedIn, YouTube comments, Google SERP, Amazon, Product Hunt, etc.

When to Use

• A Twitter/X URL is shared and you need the full thread (not just the first tweet)
• You need Reddit thread content without the expensive API
• LinkedIn company/profile data is needed
• YouTube comments or video metadata beyond what the API gives
• Google search results programmatically
• Any site that blocks standard web_fetch

Do NOT use for: sites accessible via normal web_fetch or web_search. Apify costs credits — use it only when needed.

Setup

• API Key: op://OpenClaw/Apify API Credentials/credential (also in gateway plist as APIFY_API_KEY)
• Dashboard: https://console.apify.com (account: redditech)
• Plan: FREE ($5/mo credit)
• Script: python3 scripts/apify-run.py <actor> <input_json>

Running an Actor

python3 /Users/loki/.openclaw/workspace/scripts/apify-run.py \
  "apidojo/tweet-scraper" \
  '{"twitterHandles": ["solanamobile"], "maxItems": 50}'

Key Actors

Twitter/X

• apidojo/tweet-scraper — $0.40/1K tweets. Full thread support via conversationIds. Advanced search syntax.

  {"conversationIds": ["2034675043033375103"], "maxItems": 50}
  

  or by handle:

  {"twitterHandles": ["solanamobile"], "maxItems": 20}
  

Reddit

• trudax/reddit-scraper-lite — Free tier friendly. Fetch threads + comments.

  {"startUrls": [{"url": "https://reddit.com/r/solana/comments/..."}], "maxItems": 100}
  

YouTube

• streamers/youtube-scraper — Comments + metadata.

  {"startUrls": [{"url": "https://youtube.com/watch?v=..."}], "maxComments": 200}
  

Google SERP

• apify/google-search-scraper — Search results as structured data.

  {"queries": "solana mobile grants", "maxPagesPerQuery": 1}
  

LinkedIn

• anchor/linkedin-profile-scraper — ⚠️ ToS risk. Use sparingly for research only.

Pricing Reference

• 1 CU = 1 GB RAM × 1 hour
• Free tier: $5/mo (~16.7 CU)
• Tweet scraping: 0.035–0.04 CU/1K tweets ($0.01/1K on free tier)
• Some actors charge flat per-result: $0.25–$0.40/1K tweets
• Check usage: https://console.apify.com/billing

Notes

• Results are returned as a dataset — the script polls until complete
• Timeout: 5 minutes default (most actors finish in 30–60s)
• If an actor breaks (community-maintained), check Apify Store for alternatives
• MCP integration pending — Apify MCP server exists but openclaw.json doesn't support mcpServers key yet (schema validation rejects it). Use this script approach instead.