X Tweet Fetcher

Fetch tweets, replies, and user timelines from X/Twitter without login or API keys. Also supports Chinese platforms (Weibo, Bilibili, CSDN, WeChat). Includes...

Security Scan

The skill mostly does what it claims (scraping X and some Chinese sites), but the code includes unexpected local file writes, SSH/scp execution paths, and environment/config variables that are not declared in SKILL.md or metadata — these mismatches and the ability to run remote commands make it suspicious.

Purpose & Capability

The skill's primary purpose (fetch tweets and Chinese-platform content) aligns with most included code (fetch_tweet.py, fetch_china.py, camofox_client.py). However, several capabilities in the codebase go beyond the documented/declared requirements: e.g., sogou_wechat supports 'via-router' (writing to router queue/result files) and 'via-ssh' (scp/ssh to a remote host). The metadata and SKILL.md declare no required env vars or config paths, yet the code reads environment variables (SOGOU_SSH_HOST, ROUTER_CMD_QUEUE/RESULT/OUTPUT, TWEET_GROWTH_* envvars) and uses absolute paths such as /root/router-agent/*. Those extras are not justified in the SKILL.md and are disproportionate to a simple tweet fetcher.

Instruction Scope

SKILL.md describes running Python scripts and optionally running Camofox on localhost. It does not document writing to local router queue/result files or using SSH to execute remote code. The code, however, will: write a curl command into a queue file, poll result/output files, spawn subprocesses, create temporary Python scripts, scp them to a remote SSH host and run them, and read/write local cache/data files under home (~/.tweet-growth, ~/.x-tweet-fetcher). Those file and subprocess operations are broader than the instructions suggest and can lead to surprising behavior or privilege use on the host.

Install Mechanism

No install spec in the skill bundle (instruction-only). The SKILL.md recommends installing a 3rd-party 'Camofox' project either via an OpenClaw plugin or git clone (GitHub). That is expected for a scraper that needs a browser render service; the skill itself contains Python code only and does not download arbitrary binary blobs. Still, using/launching an anti-detection browser (Camofox/Camoufox) is a higher-risk external dependency and the SKILL.md points users to third-party repos they should audit.

Credentials

metadata.json and SKILL.md declare no required environment variables, but code reads several env/config values: TWEET_GROWTH_DATA and TWEET_GROWTH_DISCOVER_CACHE (growth_config.py), SOGOU_SSH_HOST (sogou_wechat.py), and ROUTER_CMD_QUEUE / ROUTER_CMD_RESULT / ROUTER_CMD_OUTPUT defaults (sogou_wechat.py) among others. The SSH/router features are optional in code but can be enabled by environment variables; they require SSH access or control of local router agent files — powerful capabilities not disclosed in the skill's declared requirements.

Persistence & Privilege

The skill does not request 'always:true' and appears to be user-invocable only. It stores local caches and data under user home (e.g., ~/.tweet-growth, ~/.x-tweet-fetcher) which is reasonable. But the code also defaults to writing/reading files under absolute paths (/root/router-agent/...) when using the 'via-router' feature and will create temporary scripts and SCP/SSH them to remote hosts when 'via-ssh' is used. Those behaviors can modify system files or interact with remote systems and increase privilege/persistence risk beyond what the SKILL.md documents.

What to consider before installing

This skill largely does what it claims (scrapes X and some Chinese sites), but caution is warranted: - Hidden/undeclared capabilities: The code can read environment variables (e.g., SOGOU_SSH_HOST, TWEET_GROWTH_*), write to local queue/result files, and perform scp/ssh to remote hosts. None of those env vars or config paths are declared in the SKILL.md/metadata. Treat those as surprising capabilities you should double-check before enabling. - SSH/remote execution: If you set SOGOU_SSH_HOST (or use the 'via-ssh' feature), the skill will copy a script to that host and run it via ssh. Only point it at hosts you fully control/trust and understand that your SSH keys will be used for remote execution. - Router queue files: The 'via-router' mode writes commands to and reads results from files (defaults to /root/router-agent/*). Do not enable that unless you know what service/process is on the other end; misconfigured paths could lead to unexpected file writes or interference with system services. - Camofox dependency: The optional Camofox browser is described as an anti-detection tool; installing or running it increases risk and should be audited separately (it's a 3rd-party project outside OpenClaw). The skill assumes a local HTTP API at localhost:9377 — verify that endpoint before connecting it to this skill. - Safe usage recommendations: run this skill in a sandbox/container or non-privileged account, review/grep the code for any operations you find unsafe (file paths, subprocess calls, scp/ssh), and avoid setting env vars like SOGOU_SSH_HOST unless necessary. If you plan to use the router/SSH features, inspect and control the remote host and any router-agent processes first. If you need full assurance, ask the publisher for the canonical upstream source and review it; if you can't verify the Camofox/Camoufox components, avoid enabling the browser-dependent features.

Fetch tweets from X/Twitter without authentication. Supports tweet content, reply threads, user timelines, Chinese platforms, and tweet growth tracking.

Feature Overview

Feature	Command	Dependencies
Single tweet	--url <tweet_url>	None (zero deps)
Reply threads	--url <tweet_url> --replies	Camofox
User timeline	--user <username> --limit 300	Camofox
Chinese platforms	fetch_china.py --url <url>	Camofox (except WeChat)
Google search	camofox_search("query")	Camofox
X-Tracker (growth)	tweet_growth_cli.py --add/--run/--report	None (zero deps)

---

Basic Usage (Zero Dependencies)

Fetch a Single Tweet

# JSON output
python3 scripts/fetch_tweet.py --url "https://x.com/user/status/123456"

# Text only (human readable)
python3 scripts/fetch_tweet.py --url "https://x.com/user/status/123456" --text-only

# Pretty JSON
python3 scripts/fetch_tweet.py --url "https://x.com/user/status/123456" --pretty

What It Fetches

Content Type	Support
Regular tweets	✅ Full text + stats
Long tweets (Twitter Blue)	✅ Full text
X Articles (long-form)	✅ Complete article text
Quoted tweets	✅ Included
Stats (likes/RT/views)	✅ Included
Media URLs	✅ Images + videos

---

Advanced Features (Requires Camofox)

⚠️ The following features require Camofox browser service running on localhost:9377. See Camofox Setup below.

Fetch Reply Threads

# Fetch tweet + all replies (including nested replies)
python3 scripts/fetch_tweet.py --url "https://x.com/user/status/123456" --replies

# Text-only mode with replies
python3 scripts/fetch_tweet.py --url "https://x.com/user/status/123456" --replies --text-only

Fetch User Timeline

# Fetch latest tweets from a user (supports pagination, MAX_PAGES=20)
python3 scripts/fetch_tweet.py --user <username> --limit 300

Fetch Chinese Platform Content

# Auto-detects platform from URL
python3 scripts/fetch_china.py --url "https://weibo.com/..."     # Weibo
python3 scripts/fetch_china.py --url "https://bilibili.com/..."  # Bilibili
python3 scripts/fetch_china.py --url "https://csdn.net/..."      # CSDN
python3 scripts/fetch_china.py --url "https://mp.weixin.qq.com/..." # WeChat (no Camofox needed!)

Platform	Status	Notes
WeChat Articles	✅	Uses web_fetch directly, no Camofox
Weibo	✅	Camofox renders JS
Bilibili	✅	Video info + stats
CSDN	✅	Articles + code blocks
Zhihu / Xiaohongshu	⚠️	Needs cookie import for login

Google Search (Zero API Key)

# Python
from scripts.camofox_client import camofox_search
results = camofox_search("your search query")
# Returns: [{"title": "...", "url": "...", "snippet": "..."}, ...]

# CLI
python3 scripts/camofox_client.py "your search query"

Uses Camofox browser to search Google directly. No Brave API key needed, no cost.

---

Camofox Setup

What is Camofox?

Camofox is an anti-detection browser service based on Camoufox (a Firefox fork with C++ level fingerprint masking). It bypasses:

• Cloudflare bot detection
• Browser fingerprinting
• JavaScript challenges

Installation

Option 1: OpenClaw Plugin

openclaw plugins install @askjo/camofox-browser

Option 2: Manual Install

git clone https://github.com/jo-inc/camofox-browser
cd camofox-browser
npm install && npm start

Verify

curl http://localhost:9377/health
# Should return: {"status":"ok"}

REST API

# Create tab
POST http://localhost:9377/tabs
Body: {"userId":"test", "sessionKey":"test", "url":"https://example.com"}

# Get page snapshot
GET http://localhost:9377/tabs/<TAB_ID>/snapshot?userId=test

# Close tab
DELETE http://localhost:9377/tabs/<TAB_ID>?userId=test

---

From Agent Code

from scripts.fetch_tweet import fetch_tweet

result = fetch_tweet("https://x.com/user/status/123456")
tweet = result["tweet"]

# Regular tweet
print(tweet["text"])
print(f"Likes: {tweet['likes']}, Views: {tweet['views']}")

# X Article (long-form)
if tweet.get("is_article"):
    print(tweet["article"]["title"])
    print(tweet["article"]["full_text"])

# Links found in replies
for reply in result.get("replies", []):
    for link in reply.get("links", []):
        print(link)

Output Format

{
  "url": "https://x.com/user/status/123",
  "username": "user",
  "tweet_id": "123",
  "tweet": {
    "text": "Tweet content...",
    "author": "Display Name",
    "screen_name": "username",
    "likes": 100,
    "retweets": 50,
    "bookmarks": 25,
    "views": 10000,
    "replies_count": 30,
    "created_at": "Mon Jan 01 12:00:00 +0000 2026",
    "is_note_tweet": false,
    "is_article": true,
    "article": {
      "title": "Article Title",
      "full_text": "Complete article content...",
      "word_count": 4847
    }
  },
  "replies": [
    {
      "author": "@someone",
      "text": "Reply text...",
      "likes": 5,
      "links": ["https://github.com/..."],
      "thread_replies": [{"text": "Nested reply..."}]
    }
  ]
}

File Structure

x-tweet-fetcher/
├── SKILL.md                    # This file
├── README.md                   # GitHub page with full docs
├── scripts/
│   ├── fetch_tweet.py          # Main fetcher (tweet + replies + timeline)
│   ├── fetch_china.py          # Chinese platform fetcher
│   ├── camofox_client.py       # Camofox REST API client + camofox_search()
│   └── x-profile-analyzer.py   # User profile analysis (AI-powered)
└── CHANGELOG.md

Requirements

• Basic: Python 3.7+, no external packages, no API keys
• Advanced: Camofox running on localhost:9377
• Profile Analyzer: MiniMax M2.5 API key (for AI analysis)

---

X-Tracker: Tweet Growth Monitoring

Track your tweets' growth and detect viral moments — inspired by semiconductor ETCH Endpoint Detection.

Quick Start

# Add a tweet to track
python3 scripts/tweet_growth_cli.py --add "https://x.com/user/status/123" "my launch tweet"

# List all tracked tweets
python3 scripts/tweet_growth_cli.py --list

# Run sampling (new tweets <48h, every 15min)
python3 scripts/tweet_growth_cli.py --run --fast

# Run sampling (all tweets, hourly)
python3 scripts/tweet_growth_cli.py --run --normal

# Generate analysis report
python3 scripts/tweet_growth_cli.py --report 123456789

# Report with topic cross-analysis
python3 scripts/tweet_growth_cli.py --report 123456789 --cross

Cron Integration

# Dual-frequency sampling
*/15 * * * * python3 tweet_growth_cli.py --run --fast    # New tweets (<48h)
0 * * * *    python3 tweet_growth_cli.py --run --normal  # All tweets (hourly)

Detection Algorithm

Component	Method	Purpose
Derivative detection	dV/dt per hour	Spot sudden acceleration
Sliding window	5-sample moving average	Filter noise
Multi-signal fusion	views×1 + likes×1 + bookmarks×1.5 + RT×3	Weighted composite score
Burst confirmation	3 consecutive windows above threshold	Prevent false positives
Surge override	Single window +100%/h	Catch massive spikes
Saturation	6 samples < 2%/h growth	Detect long tail
Propagation	RT-per-1k-views ratio	Influencer vs algorithm driven

Output Example

🎯 Burst detected at 2026-03-14 08:45
   - Growth rate: 156%/h
   - Composite score: +2847 (views +1200, RT +89, likes +234)
   - Propagation: 12.3 RT/1k views (influencer-driven)

---

How It Works

• Basic tweets: FxTwitter public API
• Replies/timelines: Camofox → Nitter (privacy-respecting X frontend)
• Chinese platforms: Camofox renders JS → extracts content
• Google search: Camofox opens Google → parses results
• X-Tracker: FxTwitter API + derivative detection + burst confirmation