scraping-twitter-profiles

Scrapes Twitter/X profile data for any list of usernames using apidojo's Twitter User scraper on Apify. Triggers when the user asks to: get profile data for Twitter accounts, fetch follower counts for a list of usernames, scrape Twitter bio and stats for multiple users, export Twitter profile metadata, check account details for a set of handles, or bulk-fetch Twitter user information. Returns username, display name, bio, follower count, following count, tweet count, verified status, and profile URL. Ideal for data analysts, list enrichment pipelines, and market researchers.

Bulk profile data export for any set of Twitter/X usernames. Returns account-level metadata: follower counts, bios, join dates, verification status.

Prerequisites

• APIFY_TOKEN environment variable set
• Optional: Apify MCP server installed

Inputs

Parameter	Type	Required	Default	Notes
startUrls	array	Optional	[]	Twitter profile or tweet URLs
twitterHandles	array	Optional	[]	Twitter usernames (without @)
twitterUserIds	array	Optional	[]	Twitter user IDs
getFollowers	boolean	Optional	false	Extract follower lists
getFollowing	boolean	Optional	false	Extract following lists
getRetweeters	boolean	Optional	false	Extract retweeters of a tweet URL
includeUnavailableUsers	boolean	Optional	false	Include unavailable/suspended users
maxItems	number	Optional	Unlimited	Maximum users to return
customMapFunction	string	Optional	—	JavaScript function to transform each output object

Workflow

text

Progress:
- [ ] Step 1: Normalize username list
- [ ] Step 2: Run twitter-user-scraper
- [ ] Step 3: Poll for SUCCEEDED
- [ ] Step 4: Deliver profile dataset

Step 1: Normalize

Remove @ symbols, deduplicate, validate no empty strings.

Step 2: Run the Actor

Recommended — run_actor.js (handles waiting, output, and file saving automatically):

bash

# Quick answer (prints table to chat)
node scripts/run_actor.js \
  --actor "apidojo~twitter-user-scraper" \
  --input '{"param": "value"}'

# Save as CSV
node scripts/run_actor.js \
  --actor "apidojo~twitter-user-scraper" \
  --input '{"param": "value"}' \
  --output YYYY-MM-DD_results.csv --format csv

# Save as JSON
node scripts/run_actor.js \
  --actor "apidojo~twitter-user-scraper" \
  --input '{"param": "value"}' \
  --output YYYY-MM-DD_results.json --format json

APIFY_TOKEN must be set in environment or .env file.

If Apify MCP is available:

text

Tool: apify:run-actor
Actor: "apidojo~twitter-user-scraper"
Input:
{
  "usernames": ["handle1", "handle2", "handle3"]
}

REST API fallback:

bash

curl -X POST \
  "https://api.apify.com/v2/acts/apidojo~twitter-user-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"usernames": ["handle1", "handle2"]}'

Save id as RUN_ID. Poll until status = SUCCEEDED:

bash

curl "https://api.apify.com/v2/actor-runs/$RUN_ID?token=$APIFY_TOKEN" | grep '"status"'

Fetch results:

bash

curl "https://api.apify.com/v2/actor-runs/$RUN_ID/dataset/items?token=$APIFY_TOKEN&format=json"

Step 3: Handle Edge Cases

• Missing accounts in results: Account suspended, deleted, or handle changed. Note missing handles in output.
• Private accounts: Returns profile metadata but no tweet content — flag isPrivate: true rows.
• Deactivated accounts: Returns null or empty object — exclude from final dataset.

Output Format

text

# Twitter Profile Dataset
Accounts requested: N | Accounts returned: N | Missing: N

| Username | Display Name | Followers | Following | Tweets | Verified | Bio (truncated) | Joined |
|----------|-------------|-----------|-----------|--------|----------|-----------------|--------|
| ...      | ...         | ...       | ...       | ...    | ...      | ...             | ...    |

Available fields: username, displayName, followersCount, followingCount, tweetCount,
isVerified, isBlueVerified, description, location, url, profileImageUrl, createdAt, isPrivate

Troubleshooting

Some handles missing from results: Accounts may be suspended or renamed — cross-reference manually. Large lists (> 1000): Break into batches of 500 and run sequentially.