Skilled Deep Research

Other

Research any topic across multiple sources and produce a cited report. Use when the user asks to research, find, look into, deep dive, investigate, or get a comprehensive report on any subject. Searches web, GitHub, Reddit, government sources, academic papers, and more. Supports quick inline summaries (simple tier) or full parallel multi-agent research runs with checkpointed progress, deduplication, quality-ranked sources, and cited reports (standard/deep tiers). Triggers on: "research", "find sources", "look into", "deep dive", "comprehensive report", "what's available on", "find templates", "find examples", "gather information", "investigate", "I need a report on".

Install

openclaw skills install skilled-deep-research

Skilled Deep Research 🔬

A production-grade research framework for OpenClaw agents. Built on parallel sub-agents with full protocol integration — ACP for spawning, MCP for browser tools, A2A for worker-to-orchestrator signaling. Checkpointed, deduplicated, and reusable across all research tasks.

Path token: {baseDir} = this skill's directory. All skill scripts are at {baseDir}/scripts/. Data lives separately at ~/.openclaw/workspace/skills-data/skilled-deep-research/ (never inside the skill folder).

Protocols in Use

ACP — OpenClaw Agent Communication Protocol

All agent spawning uses ACP via sessions_spawn(runtime: "subagent"). Workers are ACP sub-agents at depth 2. When a sub-agent completes its run, ACP auto-announces back to the parent session. Ada does not poll — the platform handles notification.

MCP — Model Context Protocol

Playwright is registered as an MCP server via mcporter. Workers call MCP tools for JS-rendered or bot-protected pages. Full call syntax:

# Navigate to a page (MCP tool call)
mcporter call playwright.browser_navigate url="https://example.com"

# Capture content after navigation
mcporter call playwright.browser_snapshot

# For dynamic pages — wait for network idle then snapshot
mcporter call playwright.browser_navigate url="https://example.com"
mcporter call playwright.browser_wait_for_load_state loadState="networkidle"
mcporter call playwright.browser_snapshot

Each mcporter call playwright.* spawns a fresh browser process — there is no persistent session between calls. Always navigate then snapshot in sequence. MCP tools are available to workers at ACP depth 2.

A2A — Agent-to-Agent Protocol

Workers proactively sessions_send the orchestrator on completion. Orchestrator sessions_send Ada on report complete. Session keys are threaded through the spawn chain. File-based heartbeats remain as fallback for crashed workers.

Signal flow:

Ada → (ACP spawn) → Orchestrator → (ACP spawn ×N) → Workers
Workers → (A2A sessions_send) → Orchestrator   [on completion]
Orchestrator → (A2A sessions_send) → Ada        [on report written]

Data Directory

All working state follows the skills-data convention (code/data separated):

~/.openclaw/workspace/skills-data/skilled-deep-research/
  └── <slug>/
        ├── workers/
        │     ├── <name>-results.md       ← checkpoint: appended after each URL
        │     ├── <name>-progress.json    ← heartbeat: updated after each URL
        │     └── retry-queue.md          ← failed fetches for follow-up
        ├── known-urls.txt                ← deduplicated URL registry
        ├── report.md                     ← final synthesized report
        └── meta.json                     ← run metadata (topic, started, output_dir)

Final reports are copied to the user-specified output path (stored in meta.json). Working state always stays in skills-data/.

Fetch Stack (Priority Order)

Tier	Protocol	Tool	When to use
1	Native tool	`web_search` (Brave API)	All searches — primary, best quality
2	exec	`/home/sean/.openclaw/workspace/lab/skills/ddg-search/scripts/ddg`	Brave quota hit or no results
3	exec	`/home/sean/.openclaw/workspace/lab/skills/ddg-search/scripts/fetch`	All full-page fetches — Chrome UA, IPv4 forced
4	MCP	`mcporter call playwright.*`	JS-rendered pages or Akamai/Cloudflare blocks

IPv4 note: The fetch script forces -4 on all requests. Our LXC uses IPv6 which Akamai CDNs can block on .gov/.mil sites. Never use raw web_fetch or curl without -4 on government sites. web_search uses Brave's servers and is unaffected.

Architecture

Ada (main session — ACP root)
  └── Orchestrator (ACP depth 1) ─────────────── receives A2A worker signals
        ├── Worker: gov        (ACP depth 2) ─── MCP Playwright for .gov sites
        ├── Worker: github     (ACP depth 2)
        ├── Worker: reddit     (ACP depth 2)
        ├── Worker: consulting (ACP depth 2)
        └── Worker: news       (ACP depth 2)
              └── On complete → A2A sessions_send → orchestrator

Workers run in parallel. Each owns a narrow research domain. The orchestrator synthesizes outputs and A2A-signals Ada on report completion.

Config required:

agents.defaults.subagents.maxSpawnDepth: 2
agents.defaults.subagents.runTimeoutSeconds: 1800 (30-min wall clock safety net)

How to Call This Skill

Natural language (auto-triggers this skill)

# Simple tier — inline reply, no orchestrator
"Quick research on X"
"Brief summary of X"
"What is X?"

# Standard tier — orchestrator + 3–4 workers (default)
"Research X"
"Find sources on X"
"Look into X"
"What's available for X?"
"Find templates / examples / tools for X"

# Deep tier — orchestrator + 5–6 workers + Sonnet synthesis
"Deep dive into X"
"Comprehensive report on X"
"I need to make a decision about X — research it thoroughly"

# With output path (any tier)
"Research X and save the report to ~/my-project/report.md"

Explicit invocation

"Use skilled-deep-research to research X"

Usage Tiers

Simple — Ada inline, no orchestrator

Trigger: "quick", "brief", "what is", "summary", or topic ≤ 5 words

Ada uses the Simple Tier Prompt below. No sub-agents spawned. Replies inline.

Standard — Orchestrator + 3–4 workers

Trigger: "research", "find", "look into", "what's available" — default for multi-source tasks

bash {baseDir}/scripts/init-run.sh "<slug>" "<topic>" "[output_path]"
# → spawn orchestrator with 3–4 worker domains

Deep — Orchestrator + 5–6 workers + Sonnet synthesis

Trigger: "comprehensive", "full report", "deep dive", "I need to decide"

bash {baseDir}/scripts/init-run.sh "<slug>" "<topic>" "[output_path]"
# → spawn orchestrator with 5–6 worker domains
# → spawn separate Sonnet synthesis agent after workers complete

Retry pass — after any standard/deep run

bash {baseDir}/scripts/retry-run.sh <slug> [output_path]
# Prints the exact sessions_spawn call for Ada to execute

Ada's Role: Spawn the Orchestrator

// 1. Initialize the run
exec(`bash {baseDir}/scripts/init-run.sh "${slug}" "${topic}" "${outputPath}"`)

// 2. Spawn orchestrator (ACP)
sessions_spawn({
  agentId: "ada",
  task: "<filled orchestrator prompt>",
  label: `research-${slug}-orchestrator`,
  model: "zai/glm-5",
  runtime: "subagent",
  runTimeoutSeconds: 1800
})

// 3. Wait — ACP auto-announce + A2A signal will notify Ada on completion

Orchestrator Prompt Template

You are a research orchestrator. Protocols: ACP (spawning) + A2A (signaling) + MCP (browser).

Topic: [TOPIC]
Goal: [USER GOAL — learning / decision / writing / etc.]
Slug: [SLUG]
Data dir: ~/.openclaw/workspace/skills-data/skilled-deep-research/[SLUG]/
Output path: [USER OUTPUT PATH or "none"]
Known URLs to skip: [contents of known-urls.txt, or "none"]
Ada's session key: [ADA_SESSION_KEY]

## Step 1: Spawn all workers in parallel (ACP)

Fire ALL workers simultaneously — do not wait for one before spawning the next.
Pass YOUR OWN session key to each worker for A2A completion signaling.

sessions_spawn({
  agentId: "ada",
  task: "<filled worker prompt>",
  label: "research-[SLUG]-worker-[DOMAIN]",
  model: "zai/glm-5",
  runtime: "subagent",
  runTimeoutSeconds: 1800
})

Worker domains to spawn:
[LIST — e.g. "government and official sources", "GitHub repos", "Reddit threads",
 "consulting firm templates", "academic / NIST papers", "news and recent coverage"]

## Step 2: Event-driven monitoring (A2A primary, file poll fallback)

You will receive A2A signals as incoming messages when workers complete:
  "WORKER_COMPLETE: [name] | findings: N | sources: N"

Track received signals. Also poll files as fallback:
  a. exec("sleep 120") — never poll without sleeping first
  b. Read each: data-dir/workers/[name]-progress.json
  c. Worker is DONE if ANY:
       - A2A WORKER_COMPLETE signal received from it
       - progress.json shows phase="complete"
       - heartbeat timestamp >300 seconds old (stalled/crashed)
       - No progress file after 3+ poll cycles (failed to start)
  d. All done → Step 3. Otherwise → exec("sleep 120"), repeat. Max 15 cycles.

## Step 3: Quality gate — check before synthesizing

Before merging, verify minimum quality:
  exec: cat ~/.openclaw/workspace/skills-data/skilled-deep-research/[SLUG]/workers/*-results.md | grep -c '^### '

If total findings < 10 across all workers:
  - Do NOT produce a report
  - Signal Ada with failure:
    sessions_send({
      sessionKey: "[ADA_SESSION_KEY]",
      message: "RESEARCH_FAILED: [TOPIC] | Only [N] findings across all workers — below minimum threshold of 10. Check worker logs."
    })
  - End run

If findings >= 10, proceed to synthesis.

## Step 4: Synthesize

Option A — merge script (recommended):
  python3 {baseDir}/scripts/merge-reports.py [SLUG] --output "[OUTPUT PATH]"

Option B — manual:
  - Read all workers/[name]-results.md files
  - Deduplicate against known-urls.txt
  - Rank by quality_score (5→1)
  - Write to data-dir/report.md using the Final Report Template
  - If output path specified: cp data-dir/report.md "[OUTPUT PATH]"
  - Never hallucinate — if a worker found nothing, say so explicitly

## Step 5: A2A signal to Ada

sessions_send({
  sessionKey: "[ADA_SESSION_KEY]",
  message: "Research complete: [TOPIC] | [N] sources | [N] workers | Report: [PATH] | Top finding: [one sentence]"
})

Then end your run. ACP auto-announce also fires — the A2A message is the
human-readable summary.

## DO NOT:
- Proceed to synthesis before sleeping at least once after spawning workers
- Assume success if no results file exists after 3+ poll cycles
- Poll faster than every 120 seconds or more than 15 cycles
- Call sessions_send to Ada before the report is written

Worker Prompt Template

You are a research worker. Protocols: ACP sub-agent + MCP browser tools + A2A signaling.

Topic: [TOPIC]
Domain: [DOMAIN — e.g. "GitHub repositories"]
Worker name: [NAME — short slug, e.g. "github", "gov", "reddit"]
Slug: [SLUG]
Data dir: ~/.openclaw/workspace/skills-data/skilled-deep-research/[SLUG]/
Orchestrator session key: [ORCHESTRATOR_SESSION_KEY]
Known URLs to skip: [list, one per line, or "none"]

Output files (create if missing):
- Results:  data-dir/workers/[NAME]-results.md    ← APPEND after each URL
- Progress: data-dir/workers/[NAME]-progress.json ← UPDATE after each URL
- Retries:  data-dir/workers/retry-queue.md       ← APPEND failed fetches

## Search strategy
- Run 2–3 keyword variations per sub-question — never rely on a single query
- Use modifiers: site:github.com, site:reddit.com, site:*.gov, filetype:docx, "template" "download"
- Aim for 15–30 unique URLs per domain
- Priority: official/gov > academic > reputable org > community > blog > forum

## CRITICAL: NEVER use web_fetch or curl

web_fetch and curl WILL FAIL on .gov/.mil sites (IPv6 blocked by Akamai CDN).
They also lack Chrome UA headers. You MUST NOT call web_fetch under any circumstances.
The ONLY permitted fetch methods are listed below.

## Fetch strategy (in order)
1. web_search — all searches, primary
2. DDG fallback — exec: /home/sean/.openclaw/workspace/lab/skills/ddg-search/scripts/ddg "[query]" --max 8
3. Fetch script — exec: /home/sean/.openclaw/workspace/lab/skills/ddg-search/scripts/fetch "[url]"
   Use for ALL full-page content fetches. Has Chrome UA, forces IPv4 (-4).
4. MCP Playwright — ONLY if fetch script returns bot-block (403 Akamai/Cloudflare):
   mcporter call playwright.browser_navigate url="[URL]"
   mcporter call playwright.browser_snapshot

## Binary file detection — skip these extensions

Before fetching any URL, check the extension. If it ends in .pdf, .docx, .xlsx,
.pptx, .zip, .tar, .gz, .exe, .msi, .dmg, .iso, .mp4, .mp3, .wav, .jpg, .png,
.gif — do NOT fetch the body. Instead, log it as a direct download link:

exec: echo "BINARY_SKIP: [url]"

Then write a results entry with the URL as a direct download, quality score based
on the source domain, and "Direct download — binary file, not fetched" as the finding.

## Pre-flight check

Before processing any URLs, verify the fetch script works:
  exec: /home/sean/.openclaw/workspace/lab/skills/ddg-search/scripts/fetch "https://example.com" | head -5
If this fails, log the error and signal the orchestrator immediately — do not proceed.

## Per-URL workflow (repeat for every URL)
1. Update heartbeat BEFORE fetching:
   exec: python3 {baseDir}/scripts/update-heartbeat.py \
     [DATA_DIR] [NAME] --phase fetching --pct [0-100] \
     --urls-found [N] --urls-fetched [N] --findings [N] \
     --current-url "[URL]"

2. Fetch using fetch script (tier 3) or MCP Playwright (tier 4 only if blocked)

3. On failure: append to retry-queue.md and continue:
   exec: echo "- [URL] — reason: [error]" >> [DATA_DIR]/workers/retry-queue.md
   Continue to next URL — never stop on a single failure

4. On success: extract title, key findings, quality score, direct download links

5. CHECKPOINT results immediately using shell append (NEVER buffer in memory):
   exec: cat >> [DATA_DIR]/workers/[NAME]-results.md << 'ENDBLOCK'
   ### [SCORE/5] [Source Title](https://exact-url.com)
   - **Type:** official doc | github repo | reddit thread | blog | consulting page | other
   - **Relevance:** one sentence — why this matters for [TOPIC]
   - **Key findings:**
     - specific finding
     - specific finding
   - **Direct downloads:** https://direct.link/file.docx (or "none")
   - **Fetched:** 2026-03-07T00:00:00Z

   ---
   ENDBLOCK

   ⚠️ You MUST use this shell append pattern. Do NOT use write/edit tools for results.
   Do NOT buffer multiple entries — write each one immediately after fetching.

6. Append URL to known-urls.txt (FULL PATH — critical for deduplication):
   exec: echo "[URL]" >> ~/.openclaw/workspace/skills-data/skilled-deep-research/[SLUG]/known-urls.txt

7. Update heartbeat AFTER fetching with updated counts

## Results file format — MACHINE PARSED ⚠️

merge-reports.py parses this with a regex. Rules:
- Header MUST use markdown link syntax [Title](url) — NOT plain URL or parens
- No extra blank lines between ### header and first bullet
- Separate each source block with a line containing only: ---

### [SCORE/5] [Source Title](https://exact-url.com)
- **Type:** official doc | github repo | reddit thread | blog | consulting page | other
- **Relevance:** one sentence — why this matters for [TOPIC]
- **Key findings:**
  - specific finding
  - specific finding
- **Direct downloads:** https://direct.link/file.docx (or "none")
- **Fetched:** 2026-03-07T00:00:00Z

---

✅ Correct: ### [5/5] [NIST SSP Template](https://csrc.nist.gov/files/.../cui-ssp-template-final.docx)
❌ Wrong:   ### [5/5] NIST SSP Template (https://csrc.nist.gov/...)
❌ Wrong:   ### NIST SSP Template — https://csrc.nist.gov/...

## Quality scoring
- 5: Official government or standards body (NIST, DoD, CISA, GSA)
- 4: University, accredited nonprofit, or verified direct template download
- 3: Established community resource, well-maintained GitHub repo (>50 stars)
- 2: Blog post, consulting firm page, unverified tutorial
- 1: Forum post, Reddit comment, unverified user upload

## Token budget awareness
After every 10 URLs fetched, check: are your responses being truncated? Is the
context near capacity? If so:
  - Write a "## Remaining Work" section to results.md listing unvisited URLs
  - Run final heartbeat with --phase complete
  - Send A2A signal and stop — do not attempt more fetches

## A2A completion signal
When done (all URLs processed or token budget reached):
1. Final heartbeat: python3 {baseDir}/scripts/update-heartbeat.py \
     [DATA_DIR] [NAME] --phase complete --pct 100 \
     --urls-found [N] --urls-fetched [N] --findings [N]
2. Signal orchestrator:
   sessions_send({
     sessionKey: "[ORCHESTRATOR_SESSION_KEY]",
     message: "WORKER_COMPLETE: [NAME] | findings: [N] | sources: [N] | domain: [DOMAIN]"
   })
3. End run — ACP auto-announce handles platform notification

## DO NOT:
- Use web_fetch or curl — EVER. They fail on .gov sites. Use the fetch script ONLY.
- Buffer results in memory — use shell `cat >>` append after EVERY SINGLE URL
- Use write/edit tools for results.md — only shell append ensures crash-safe checkpointing
- Stop on a single URL failure — log to retry-queue.md and continue
- Re-fetch URLs already in known-urls.txt
- Fetch binary files (.pdf, .docx, .xlsx, .zip) — log as direct download links
- Call sessions_send before final heartbeat is written

Simple Tier Prompt Template

For quick inline research (Ada runs directly — no orchestrator or workers):

Run a quick research pass on: [TOPIC]
Goal: [brief answer / fast summary]

Steps:
1. Run 3–5 targeted web_search queries with different keyword angles
2. For promising URLs, fetch full content:
   /home/sean/.openclaw/workspace/lab/skills/ddg-search/scripts/fetch "[url]"
3. Do not spawn sub-agents — complete entirely inline

Reply with:
- 3–5 sentence summary of findings
- Top 5 sources with URLs and one-line descriptions
- Any direct download links found

Quality rules: cite every claim; flag single-source findings as unverified.

Final Report Template

# [Topic]: Research Report
*Generated: [date] | Run: [slug] | Workers: [N] | Sources: [total] | Confidence: [High/Medium/Low]*

## Executive Summary
[3–5 sentences — most important findings]

## Top Sources
[Top 10–15 ranked by quality score, 5→1, with inline links]

## 1. [Worker Domain / Theme]
[Findings with inline citations — ([Source Name](url))]

## 2. [Worker Domain / Theme]
...

## Key Takeaways
- [Actionable insight]
- [Actionable insight]
- [Actionable insight]

## Direct Downloads
[Any .docx / .pdf / .xlsx direct links requiring no login]

## Gaps & Limitations
- [Domains with no results]
- [Workers that timed out]
- [Retry queue: N URLs — run retry-run.sh to process]

## Methodology
Protocol stack: ACP (spawning) + MCP (Playwright browser) + A2A (worker signaling)
Workers: [list with completion status]
Searches: [N] | URLs fetched: [N] | Deduplicated sources: [N]

## Sources Index
[Full list with quality scores and worker attribution]

Retry Queue

Workers append failed fetches to workers/retry-queue.md. After a run completes:

bash {baseDir}/scripts/retry-run.sh <slug> [output_path]

Prints the exact sessions_spawn call for Ada to execute. The retry worker uses the full fetch stack including MCP Playwright for stubborn URLs.

Scripts Reference

Script	Usage
`{baseDir}/scripts/init-run.sh <slug> <topic> [output]`	Create data dir, write meta.json
`{baseDir}/scripts/status.sh <slug>`	Live worker status — phase, %, heartbeat age
`{baseDir}/scripts/update-heartbeat.py <data_dir> <name> [opts]`	Worker checkpoint
`{baseDir}/scripts/merge-reports.py <slug> [--output path]`	Dedup + rank + write report
`{baseDir}/scripts/retry-run.sh <slug> [output]`	Generate retry worker spawn call

Quality Rules

Every claim needs a source. No unsourced assertions.
Cross-reference. Single-source findings flagged as unverified.
Recency matters. Prefer last 12 months; flag older sources.
Acknowledge gaps. Empty worker domains reported, not hidden.
No hallucination. "Insufficient data found" beats a guess.
Checkpoint constantly. Results written after every URL — timeouts lose nothing.
Dedup always. known-urls.txt checked before every fetch.

Requirements

Requirement	Gate	Notes
`mcporter`	`requires.bins`	MCP client for Playwright
`python3`	`requires.bins`	Required for all scripts
Brave Search API	configured	`web_search` native tool
`ddg` script	soft	`/home/sean/.openclaw/workspace/lab/skills/ddg-search/scripts/ddg`
`fetch` script	soft	`/home/sean/.openclaw/workspace/lab/skills/ddg-search/scripts/fetch`
`playwright` MCP server	soft	`mcporter list` should show `playwright`
`maxSpawnDepth: 2`	config	`openclaw config set agents.defaults.subagents.maxSpawnDepth 2`
`runTimeoutSeconds: 1800`	config	`openclaw config set agents.defaults.subagents.runTimeoutSeconds 1800`