CRW Web Scraper

CRW web scraping plugin for OpenClaw — scrape, crawl, and map websites

openclaw plugins install clawhub:openclaw-plugin-crw

openclaw-plugin-crw

CRW web scraping plugin for OpenClaw. Gives your AI agents the ability to scrape, crawl, and map websites.

bash

openclaw plugins install openclaw-plugin-crw

No server to install. Sign up at fastcrw.com and get 500 free credits to start scraping:

json

{
  "plugins": {
    "crw": {
      "apiKey": "crw_live_..."
    }
  }
}

That's it — cloud is the default. Your agents can now scrape, crawl, and map any website.

Single binary, ~15 MB download, ~6 MB idle RAM. No Docker needed.

bash

curl -fsSL https://raw.githubusercontent.com/us/crw/main/install.sh | bash
crw  # starts on http://localhost:3000

json

{
  "plugins": {
    "crw": {
      "apiUrl": "http://localhost:3000"
    }
  }
}

bash

docker run -d -p 3000:3000 ghcr.io/us/crw:latest

json

{
  "plugins": {
    "crw": {
      "apiUrl": "http://localhost:3000"
    }
  }
}

Tool	Description
`crw_scrape`	Scrape a single URL and get clean markdown
`crw_crawl`	BFS crawl a website, collect content from multiple pages
`crw_map`	Discover all URLs on a website via sitemap + link traversal

Once installed and configured, your OpenClaw agents use the CRW tools automatically:

User (via WhatsApp/Telegram/Discord): "Summarize this article: https://example.com/blog/post"

Agent uses crw_scrape → gets clean markdown → summarizes → responds.

User: "Research everything on docs.example.com"

Agent uses crw_crawl → discovers and scrapes all pages → synthesizes findings → responds.

User: "What pages does example.com have?"

Agent uses crw_map → returns all discovered URLs via sitemap + link traversal.

A simple OpenClaw agent that scrapes any URL users send:

With fastcrw.com, no infrastructure needed — just plug in your API key and go.

MIT