Scrapeless LLM Chat Scraper Skill

Scrape AI chat conversations from ChatGPT, Gemini, Perplexity, Copilot, Google AI Mode, and Grok.

Security Scan

The skill's code and runtime instructions match its stated purpose (scraping LLM chats via the Scrapeless API) but registry metadata omits the required API credential and there are modest packaging/operational issues you should review before installing.

Purpose & Capability

The SKILL.md and the included Python script both clearly describe and implement calls to the Scrapeless API (api.scrapeless.com) to create and poll scraping tasks for ChatGPT, Gemini, Perplexity, Copilot, Google AI Mode, and Grok. The required credential X_API_TOKEN is consistent with that purpose. However the registry metadata at the top of the submission lists 'Required env vars: none' which contradicts SKILL.md and the script that require X_API_TOKEN.

Instruction Scope

Runtime instructions only tell the agent/user to run the provided Python script, set X_API_TOKEN (or .env) and pass query/mode/country flags. The script loads a .env file (via python-dotenv) and only uses X_API_TOKEN from the environment for authentication. There are no instructions to read unrelated system files or exfiltrate data to unexpected endpoints beyond api.scrapeless.com.

Install Mechanism

This is an instruction-only skill (no install spec), but the repo includes code and a requirements.txt (requests, python-dotenv). That means the user must pip-install dependencies manually; there is no automated installer. This is low risk but a packaging gap: the skill writes no unusual files or run arbitrary installers, but you must follow the README to install dependencies.

Credentials

The script only requires a single API token (X_API_TOKEN) which is proportional to the stated behavior of calling the Scrapeless API. The concern is the inconsistency: registry metadata claims no required env vars while SKILL.md and the script require X_API_TOKEN. Also note: load_dotenv will parse a .env file if present, so the repository may read other env variables from that file even if they are unused — review any .env you create to avoid storing unrelated secrets there.

Persistence & Privilege

The skill does not request permanent 'always' presence and does not modify other skills or system-wide settings. It does network calls to the external API (expected for its purpose) and uses only the provided API token for authentication.

What to consider before installing

What to check before installing: (1) The skill sends your prompts and parameters to api.scrapeless.com using X_API_TOKEN — only install if you trust that third-party service and its privacy/retention policy. (2) The package metadata in the registry incorrectly omits the required X_API_TOKEN; expect to set that env var or .env manually. (3) Install dependencies (requests, python-dotenv) yourself (pip install -r requirements.txt) and inspect the script for any modifications you would not expect. (4) Avoid placing other secrets in a .env file in the same repo because python-dotenv will read it; keep your X_API_TOKEN scoped and rotated if possible. (5) If you need higher assurance, request provenance (official upstream repo or signed publisher) or verify the HTTP interactions (endpoints, headers, payloads) before using in production.

LLM Chat Scraper OpenClaw Skill

Use this skill to scrape AI chat conversations from various LLM models via the Scrapeless API. The skill supports ChatGPT, Gemini, Perplexity, Copilot, Google AI Mode, and Grok.

Authentication: Set X_API_TOKEN in your environment or in a .env file in the repo root.

Errors: On failure the script writes a JSON error to stderr and exits with code 1.

---

Tools

1. ChatGPT Scraper

Scrape ChatGPT responses with optional web search enrichment. Returns JSON object with result_text, model, links, citations, and more.

Command:

python3 scripts/llm_chat_scraper.py chatgpt --query "your prompt"

Examples:

python3 scripts/llm_chat_scraper.py chatgpt --query "Most reliable proxy service for data extraction"
python3 s
Optional: `--country` fcripts/llm_chat_scraper.py chatgpt --query "AI trends in 2024" --web-search
python3 scripts/llm_chat_scraper.py chatgpt --query "Best programming languages" --country GB

or location, --web-search to enable web search.

---

2. Gemini Scraper

Scrape Google Gemini responses. Returns JSON object with result_text, citations, and more.

Command:

python3 scripts/llm_chat_scraper.py gemini --query "your prompt"

Examples:

python3 scripts/llm_chat_scraper.py gemini --query "Recommended attractions in New York"
python3 scripts/llm_chat_scraper.py gemini --query "Best restaurants in Tokyo" --country JP

Optional: --country for location (JP and TW not supported).

---

3. Perplexity Scraper

Scrape Perplexity AI responses with optional web search. Returns JSON object with result_text, related_prompt, web_results, media_items.

Command:

python3 scripts/llm_chat_scraper.py perplexity --query "your prompt"

Examples:

python3 scripts/llm_chat_scraper.py perplexity --query "Latest AI developments"
python3 scripts/llm_chat_scraper.py perplexity --query "Quantum computing explained" --web-search

Optional: --country for location, --web-search to enable web search.

---

4. Copilot Scraper

Scrape Microsoft Copilot responses across different modes (search, smart, chat, reasoning, study). Returns JSON object with result_text, mode, links, citations.

Command:

python3 scripts/llm_chat_scraper.py copilot --query "your prompt"

Examples:

python3 scripts/llm_chat_scraper.py copilot --query "What is machine learning?"
python3 scripts/llm_chat_scraper.py copilot --query "Explain blockchain" --mode reasoning
python3 scripts/llm_chat_scraper.py copilot --query "Best laptop 2024" --mode search

Optional: --country for location (JP and TW not supported), --mode for operation mode.

---

5. Google AI Mode Scraper

Scrape Google AI Mode responses. Returns JSON object with result_text, result_md, result_html, citations, raw_url.

Command:

python3 scripts/llm_chat_scraper.py aimode --query "your prompt"

Examples:

python3 scripts/llm_chat_scraper.py aimode --query "Best programming languages to learn"
python3 scripts/llm_chat_scraper.py aimode --query "Climate change solutions" --country GB

Optional: --country for location (JP and TW not supported).

---

6. Grok Scraper

Scrape xAI Grok responses with different modes (FAST, EXPERT, AUTO). Returns JSON object with full_response, user_model, follow_up_suggestions, web_search_results.

Command:

python3 scripts/llm_chat_scraper.py grok --query "your prompt"

Examples:

python3 scripts/llm_chat_scraper.py grok --query "Explain quantum entanglement"
python3 scripts/llm_chat_scraper.py grok --query "What's happening in AI" --mode MODEL_MODE_EXPERT
python3 scripts/llm_chat_scraper.py grok --query "Latest tech news" --mode MODEL_MODE_FAST

Optional: --country for location (JP and TW not supported), --mode for operation mode.

---

Summary

Action	Command	Argument	Example
ChatGPT	chatgpt	--query	python3 scripts/llm_chat_scraper.py chatgpt --query "AI trends"
Gemini	gemini	--query	python3 scripts/llm_chat_scraper.py gemini --query "Best restaurants"
Perplexity	perplexity	--query	python3 scripts/llm_chat_scraper.py perplexity --query "Latest news"
Copilot	copilot	--query	python3 scripts/llm_chat_scraper.py copilot --query "Explain ML"
Google AI Mode	aimode	--query	python3 scripts/llm_chat_scraper.py aimode --query "Programming"
Grok	grok	--query	python3 scripts/llm_chat_scraper.py grok --query "Quantum physics"

Output: All commands return JSON objects with model-specific fields (see tool descriptions above).

---

Response Fields by Model

ChatGPT

• result_text: Markdown response
• model: Model identifier (e.g., gpt-4)
• web_search: Boolean indicating if search ran
• links: Array of supplementary links
• citations: Array of content references

Gemini

• result_text: Markdown response
• citations: Array with favicon, highlights, snippet, title, url, website_name

Perplexity

• result_text: Markdown response
• related_prompt: Array of related questions
• web_results: Array with name, url, snippet
• media_items: Array of media references

Copilot

• result_text: Markdown response
• mode: Mode used (search/smart/chat/reasoning/study)
• links: Array of outbound links
• citations: Array with title, url

Google AI Mode

• result_text: Answer body
• result_md: Markdown version
• result_html: HTML version
• raw_url: Original URL
• citations: Array with snippet, thumbnail, title, url, website_name, favicon

Grok

• full_response: Response content
• user_model: Model used
• follow_up_suggestions: Array of suggested questions
• web_search_results: Array with preview, title, url
• conversation: Object with conversation metadata

---

Notes

⚠️ Regional Restrictions:

• Gemini, Copilot, Google AI Mode, and Grok do not support Japan (JP) and Taiwan (TW)

⚠️ Result Expiry:

• Task results are available for 12 hours

⚠️ Rate Limits:

• 429 errors indicate rate limit exceeded. Reduce request frequency or upgrade plan.