Fetch Url

Fetch raw HTTP response bodies from one or more URLs with optional custom headers and timeout, supporting JSON, XML, RSS, CSV, plain text, and files.

Fetch raw HTTP responses from URLs. Supports multiple URLs and custom headers.

Usage

# Single URL
node {baseDir}/scripts/fetch.mjs "https://api.example.com/data"

# Multiple URLs
node {baseDir}/scripts/fetch.mjs '["https://api.example.com/data1", "https://api.example.com/data2"]'

# With custom headers (per-URL)
node {baseDir}/scripts/fetch.mjs '["https://api.example.com/json", "https://api.example.com/text"]' '[{"accept":"application/json"}, {"accept":"text/plain"}]'

# With timeout
node {baseDir}/scripts/fetch.mjs "https://api.example.com/data" --timeout 60000

Examples

JSON API

# GitHub API - get repository info
node {baseDir}/scripts/fetch.mjs "https://api.github.com/repos/microsoft/vscode" '{"accept":"application/json","user-agent":"Mozilla/5.0"}'

XML

# SOAP service or XML endpoint
node {baseDir}/scripts/fetch.mjs "https://api.example.com/soap" '{"accept":"application/xml"}'

# Sitemap
node {baseDir}/scripts/fetch.mjs "https://example.com/sitemap.xml"

RSS / Atom Feed

# RSS 2.0 feed
node {baseDir}/scripts/fetch.mjs "https://example.com/rss.xml" '{"accept":"application/rss+xml"}'

# Atom feed
node {baseDir}/scripts/fetch.mjs "https://example.com/atom.xml" '{"accept":"application/atom+xml"}'

CSV / Plain Text

# CSV data
node {baseDir}/scripts/fetch.mjs "https://example.com/data.csv" '{"accept":"text/csv"}'

# Plain text
node {baseDir}/scripts/fetch.mjs "https://example.com/robots.txt" '{"accept":"text/plain"}'

Multiple Formats

# Fetch JSON and XML in parallel
node {baseDir}/scripts/fetch.mjs '["https://api.example.com/data.json", "https://api.example.com/data.xml"]' '[{"accept":"application/json"}, {"accept":"application/xml"}]'

Options

• urls_json: JSON array of URLs or a single URL string
• headers_json: Optional JSON array of headers objects (same length as urls)
• --timeout <ms>: Request timeout in milliseconds (default: 30000)

Notes:

• Returns raw HTTP response body
• For multiple URLs, returns JSON array of results
• Node.js 18+ required (native fetch)