Scholar Deep Research

Use when the user asks for a literature review, academic deep dive, research report, state-of-the-art survey, topic scoping, comparative analysis of methods/...

Agents365.ai@agents365-ai

Install

openclaw skills install @agents365-ai/scholar-deep-research

Scholar Deep Research

End-to-end academic research workflow that turns a question into a cited, structured report. Built for depth: multi-source federation, transparent ranking, citation chasing, and a mandatory self-critique pass before the report ships.

When to use

Explicit triggers: "literature review", "research report", "state of the art", "survey the field", "what's known about X", "deep research on Y", "systematic review", "scoping review", "compare papers on Z".

Proactive triggers (use without being asked):

User asks a factual question whose honest answer is "it depends on the literature"
User frames a research plan and needs the background section
User is drafting a paper intro/related-work and hasn't yet scoped prior work
User proposes a method and asks whether it's novel

Do not use when: a single known paper answers the question, the user wants a tutorial (not a survey), or they're debugging code.

Guiding principles

Scripts over vibes. Every search, dedupe, rank, and export step runs through a script in scripts/. The same input should produce the same output. Do not improvise ranking or counting by eye.
Sources are federated, not singular. OpenAlex is the primary backbone (free, 240M+ works, no key). arXiv, Crossref, and PubMed fill gaps. MCP tools (Semantic Scholar / Brave) are enrichment, not dependency — if they time out, research continues.
State is persistent. Everything goes through research_state.json. Queries ran, papers seen, decisions made, phase progress. Research becomes resumable and auditable.
Citations are anchors, not decorations. Every non-trivial claim in the draft carries [^id] where id matches a paper in state. Unanchored claims are treated as hallucinations and fail the gate.
Saturation, not exhaustion, is the stop signal. A phase ends when a new round of search adds <20% novel papers AND no new paper has >100 citations.
Self-critique is a phase, not a checkbox. Phase 6 reads the draft with adversarial intent. Its output goes into the report appendix.

The 8-phase workflow (Phase 0..7)

text

Phase 0: Scope       → decompose question, pick archetype, init state
Phase 1: Discovery   → multi-source search, dedupe
Phase 2: Triage      → rank, select top-N for deep read
Phase 3: Deep read   → extract evidence per paper
Phase 4: Chasing     → citation graph (forward + backward)
Phase 5: Synthesis   → cluster by theme, map tensions
Phase 6: Self-critique → adversarial review, gap finding
Phase 7: Report      → render archetype template, export bibliography

Each phase writes to research_state.json before advancing. If the user pauses or a session crashes, the next run reads the state and picks up from the last completed phase.

Phase 0 — Scope

Step 0 — Check for skill updates (silent, once per day). Before anything else, run:

bash

python scripts/check_update.py

The script self-throttles to one real check per 24 hours (via a .last_update_check timestamp in the skill root); running it every session is cheap. It always exits 0 and never fails the workflow — route on data.action only when you need to tell the user something:

updated → one line: [Skill updated: <from> → <to> (<commits_behind> commits). Continuing with new version.]. If requirements_changed: true, append: Python deps changed — run `pip install -r requirements.txt` before next use.
skipped_dirty → one line: [Skill update skipped — you have local changes in <dirty_count> file(s). Review with `cd <skill_root> && git status` .] so the user knows they're running a stale version on purpose.
Everything else (up_to_date, skipped_throttled, skipped_disabled, not_a_git_repo, check_failed) → continue silently. Don't mention to the user.

Escape hatches: SCHOLAR_SKIP_UPDATE_CHECK=1 pins the version permanently; python scripts/check_update.py --force bypasses the 24h throttle for an immediate check.

Before searching anything, decompose the question.

Restate the question in one sentence. Surface ambiguities.
PICO-style decomposition (or equivalent for non-biomedical fields):
- Population / Problem — what system, species, setting, or phenomenon?
- Intervention / Independent var — what method, factor, or manipulation?
- Comparison — against what baseline or alternative?
- Outcome — what is being measured or claimed?
Pick an archetype that matches user intent (see references/report_templates.md):
- literature_review — what is known about X (default)
- systematic_review — rigorous PRISMA-lite, comparison of many studies on one narrow question
- scoping_review — what has been studied and how (breadth over depth)
- comparative_analysis — X vs Y, head-to-head
- grant_background — narrative background + gap for a proposal
Draft keyword clusters — 3-5 Boolean clusters covering synonyms, acronyms, and variant spellings. Include a "negative" cluster (terms to exclude).

Initialize state:

bash

python scripts/research_state.py --state research_state.json init \
  --question "<restated question>" \
  --archetype literature_review

(--state is top-level and applies to every subcommand; init itself takes --question, --archetype, and optional --force.)

When in doubt about archetype, ask the user. The choice shapes everything downstream.

Phase 1 — Discovery

Run searches across all available sources in parallel. OpenAlex is primary; the others fill gaps.

bash

# Primary (no API key, always available)
python scripts/search_openalex.py --query "<cluster 1>" --limit 50 --state research_state.json
python scripts/search_openalex.py --query "<cluster 2>" --limit 50 --state research_state.json

# Domain-specific (use when relevant)
python scripts/search_arxiv.py  --query "<cluster>" --limit 50 --state research_state.json  # CS/ML/physics
python scripts/search_pubmed.py --query "<cluster>" --limit 50 --state research_state.json  # biomedical
python scripts/search_crossref.py --query "<cluster>" --limit 50 --state research_state.json  # DOI-backed metadata

# Dedupe across sources (DOI-first, title-similarity fallback)
python scripts/dedupe_papers.py --state research_state.json

MCP enrichment (optional, run if available): call mcp__asta__search_papers_by_relevance and mcp__asta__snippet_search and feed results via scripts/research_state.py ingest. If the MCP call errors or times out, do not retry — move on.

Iterate. Read the state file. Are there keyword gaps? Are there authors appearing 3+ times whose other work you haven't pulled? Run another round. Stop when saturation hits — every source, not just the last one queried:

bash

python scripts/research_state.py saturation --state research_state.json
# Returns { "per_source": {...}, "overall_saturated": true/false, ... }

overall_saturated is true only when every queried source has run at least --min-rounds (default 2) rounds AND each is individually below the new-paper percentage and new-citation thresholds. A source that has been queried only once cannot be declared saturated, which rules out the failure mode where a single quiet source falsely ends discovery. Use --source openalex to check one source in isolation.

Phase 2 — Triage

Rank the deduplicated corpus and pick the top-N for deep reading.

bash

python scripts/rank_papers.py \
  --state research_state.json \
  --question "<phase 0 question>" \
  --alpha 0.4 --beta 0.3 --gamma 0.2 --delta 0.1 \
  --top 20

The formula is transparent — the script prints it and writes the components to state so the report can cite its own methodology:

text

score = α·relevance + β·log10(citations+1)/3 + γ·recency_decay(half-life=5yr) + δ·venue_prior

Defaults target a literature review. For a scoping review prefer higher α (relevance) and lower β (citations). For a systematic review of a narrow question, lower α and higher β.

Write the top-N selection to state:

bash

python scripts/research_state.py select --state research_state.json --top 20

Phase 3 — Deep read

For each paper in the top-N: get the best available full text, extract evidence, attach to state.

Preferred order for full text: resolve by DOI (automatic OA chain) → publisher PDF URL from OpenAlex → arXiv PDF → institutional repository → preprint server → abstract only (with a warning attached to the paper record).
Extract text — use --doi when the paper's DOI is known (resolves and extracts in one step), or --input/--url for direct paths:
bash
```
python scripts/extract_pdf.py --doi 10.1038/s41586-020-2649-2 --output paper.txt
python scripts/extract_pdf.py --input paper.pdf --output paper.txt
```
DOI mode uses the paper-fetch skill (5-source OA chain) if installed, otherwise falls back to Unpaywall. The output envelope includes fetch_meta with title, authors, year, and source when using --doi.
Fill the per-paper evidence slot in state (the agent does this; no script). For each paper capture:
- question_or_hypothesis
- method (one sentence)
- key_findings (3-5 bullets, each with a page/section anchor)
- limitations
- relevance_to_question (how this paper moves the answer)

Abstract-only papers are marked depth: shallow in state — they can appear in the report but should not be the only source for any claim.

Phase 4 — Citation chasing

Take the top 5-10 highest-ranked papers and expand the graph.

bash

# Preview the request count first — this is the most expensive command
python scripts/build_citation_graph.py \
  --state research_state.json \
  --seed-top 8 --direction both --depth 1 --dry-run

# Run with an idempotency key so a retry after a network blip is free
python scripts/build_citation_graph.py \
  --state research_state.json \
  --seed-top 8 --direction both --depth 1 \
  --idempotency-key "chase-$(date -u +%Y%m%dT%H%M)"

The script pulls backward references (what did this paper cite?) and forward citations (who cited this paper?), deduplicates against existing state, and writes new candidate papers with discovered_via: citation_chase. Run rank + deep read again on any new high-scoring additions.

Idempotency. When --idempotency-key <k> is set, the first successful run writes {response, signature} to .scholar_cache/<hash>.json. A retried run with the same key replays the cached response without re-hitting OpenAlex or re-mutating state. Reusing the same key with different arguments returns idempotency_key_mismatch rather than silently serving stale data. Cache directory: SCHOLAR_CACHE_DIR env var, default .scholar_cache/.

Special case — a highly cited paper has never been challenged. If rank says a paper is top-3 by citations but no critiques appear in the corpus, search explicitly for "<first author> <year>" critique OR limitations OR reanalysis OR failed replication. This is the confirmation-bias backstop.

Phase 5 — Synthesis

No scripts here — this is where the agent earns its keep. Cluster and structure:

Thematic clustering. Group the top-N into 3-6 themes that map onto the report outline. Themes should be orthogonal: a paper can be primary to one, secondary to at most one other.
Tension map. Where do papers disagree? For each disagreement, note: which papers, on what, and whether the disagreement is empirical (different data), methodological (different tools), or theoretical (different framings).
Timeline. When relevant, a chronological arc: seminal paper → consolidation → refinement → current frontier.
Venn / gap. What has been studied well, partially, and not at all? The gap is the pivot for Phase 7.

Phase 6 — Self-critique

This is not optional. Load assets/prompts/self_critique.md and run the full checklist against your draft (still unpublished). The checklist covers:

Single-source claims (any claim backed by only one paper?)
Citation/recency skew (is the latest-2-years window covered?)
Venue bias (is the corpus dominated by one journal/venue?)
Author bias (does one lab dominate the citations?)
Untested high-citation papers (anyone cite a paper without reading a critique?)
Contradictions buried (any tension in Phase 5 that got glossed over?)
Archetype fit (does the structure match the chosen archetype?)
Unanchored claims (any statement without a [^id] anchor?)

Write findings to research_state.json under self_critique and fix blockers before Phase 7. Findings go into the report appendix verbatim — the reader deserves to see what the research process doubted itself about.

Phase 7 — Report

Render the chosen archetype template from assets/templates/, filling from state:

bash

# Export bibliography in the user's preferred format
python scripts/export_bibtex.py --state research_state.json --format bibtex --output refs.bib
python scripts/export_bibtex.py --state research_state.json --format csl-json --output refs.json

The report body uses [^id] anchors (the paper id from state). The bibliography section at the bottom lists each cited paper with DOI/URL. Any claim missing an anchor must be removed or cited.

Save path convention: reports/<slug>_<YYYYMMDD>.md. The skill does not write outside the working directory unless the user specifies a path.

Report archetype selection

Archetype	When to use	Primary output shape
`literature_review`	User wants to know what's established about a topic	Thematic sections + synthesis + gap
`systematic_review`	Narrow question, many studies, need rigorous comparison	PRISMA-lite flow + extraction table + pooled findings
`scoping_review`	Broad topic, "what has been studied?"	Coverage map + methods inventory + research gap
`comparative_analysis`	"A vs B" — methods, models, approaches	Axes of comparison + per-axis verdict + recommendation
`grant_background`	Narrative for a proposal introduction	Problem significance + what's known + what's missing + why our approach

Templates live in assets/templates/<archetype>.md. Load only the one you need.

Scripts reference

Script	Purpose
`check_update.py`	Phase 0 Step 0 — fast-forward the skill against its origin; never blocks the workflow.
`research_state.py`	Init, read, write, query the state file. Central to every phase.
`search_openalex.py`	Primary search (no key, 240M works, citation counts).
`search_arxiv.py`	arXiv API — preprints and CS/ML/physics.
`search_crossref.py`	Crossref REST — authoritative DOI metadata.
`search_pubmed.py`	NCBI E-utilities — biomedical corpus with MeSH.
`dedupe_papers.py`	DOI normalization + title similarity merging across sources.
`rank_papers.py`	Transparent scoring formula. Prints the formula and per-paper components.
`build_citation_graph.py`	Forward/backward snowballing via OpenAlex.
`extract_pdf.py`	Full-text extraction (pypdf). Accepts `--input`, `--url`, or `--doi`. DOI mode resolves via paper-fetch skill if installed, falls back to Unpaywall. Safe on scanned PDFs (skips, emits warning).
`export_bibtex.py`	BibTeX / CSL-JSON / RIS export from state.

All scripts accept --help, --schema, emit a structured JSON envelope on stdout, and use research_state.json as the single source of truth. Every script is idempotent on the state file (network-layer idempotency is P1 work).

CLI contract

Every script prints exactly one JSON envelope to stdout and exits with a code from the stable vocabulary below. No prose is ever mixed into stdout; diagnostics go to stderr.

Success envelope:

json

{ "ok": true, "data": { ... } }

Failure envelope:

json

{
  "ok": false,
  "error": {
    "code": "snake_case_routing_key",
    "message": "human sentence",
    "retryable": true,
    "...extra context fields...": "..."
  }
}

Exit codes:

Code	Meaning
`0`	success
`1`	runtime error (e.g. malformed upstream response, missing dependency)
`2`	upstream / network error (retryable)
`3`	validation error (bad input)
`4`	state error (missing, corrupt, or schema mismatch)

Schema introspection. Every script supports --schema, which prints its full parameter schema (types, defaults, choices, required flags, subcommands where applicable) as JSON and exits 0. An agent discovering an unfamiliar script should run --schema before --help — it is machine-parseable and covers everything --help does.

bash

python scripts/search_openalex.py --schema
python scripts/research_state.py --schema   # includes every subcommand

Export bibliography exception. export_bibtex.py without --output writes raw BibTeX/RIS/CSL text to stdout for pipe compatibility (export_bibtex.py --format bibtex > refs.bib). Agents that need a structured response should always pass --output — that path returns {"ok": true, "data": {"output": "...", "format": "bibtex", "count": N}}.

Environment variables

Trust-boundary configuration — set once by the human or orchestrator. CLI flags override where present.

Variable	Used by	Purpose
`SCHOLAR_STATE_PATH`	every script that takes `--state`	Default path to `research_state.json`
`SCHOLAR_MAILTO`	`search_openalex.py`, `search_crossref.py`, `build_citation_graph.py`	Polite-pool email for OpenAlex / Crossref — higher rate limits
`NCBI_API_KEY`	`search_pubmed.py`	NCBI E-utilities API key — higher rate limits
`SCHOLAR_CACHE_DIR`	`build_citation_graph.py` (any command that takes `--idempotency-key`)	Cache directory for idempotent-retry responses; default `.scholar_cache/` in cwd
`PAPER_FETCH_SCRIPT`	`extract_pdf.py`	Path to paper-fetch's `fetch.py`. If unset, auto-discovers across all known skill install paths (Claude Code, OpenCode, OpenClaw, Hermes, ~/.agents). If not found, falls back to Unpaywall
`SCHOLAR_SKIP_UPDATE_CHECK`	`check_update.py`	Set to any non-empty value to pin the current version and skip Phase 0 Step 0's auto-update

Agents should never set these themselves. They belong in the shell profile, a systemd unit, or the orchestrator's env injection.

State file schema (abbreviated)

json

{
  "schema_version": 1,
  "question": "...",
  "archetype": "literature_review",
  "phase": 3,
  "created_at": "...",
  "updated_at": "...",
  "queries": [{"source": "openalex", "query": "...", "hits": 42, "new": 30, "round": 1}],
  "papers": {
    "doi:10.1038/nature12373": {
      "id": "doi:10.1038/nature12373",
      "title": "...",
      "authors": ["..."],
      "year": 2013,
      "venue": "Nature",
      "citations": 523,
      "abstract": "...",
      "source": ["openalex", "crossref"],
      "score": 0.81,
      "score_components": {"relevance": 0.9, "citations": 0.8, "recency": 0.6, "venue": 1.0},
      "selected": true,
      "depth": "full",
      "evidence": {"method": "...", "findings": ["..."], "limitations": "..."},
      "discovered_via": "search"
    }
  },
  "themes": [{"name": "...", "paper_ids": ["..."]}],
  "tensions": [{"topic": "...", "sides": [{"position": "...", "paper_ids": ["..."]}]}],
  "self_critique": {"findings": [], "resolved": [], "appendix": "..."},
  "report_path": "reports/slug_20260411.md"
}

See scripts/research_state.py --help for the full schema.

Completion gates

Each phase transition has a gate (G1..G7). Advance ONLY via:

bash

python scripts/research_state.py --state <path> advance          # advance by 1
python scripts/research_state.py --state <path> advance --check-only   # preview only

The gate predicates are enforced in scripts/_gates.py. Direct set --field phase is rejected — the phase field is no longer settable. If the gate fails, the envelope lists the failing checks by name so you know exactly what's missing.

Target	Gate (enforced)
G1 (→ 1)	Question set, archetype valid, state initialized. `≥3 keyword clusters` is host-checked.
G2 (→ 2)	`overall_saturated == true` across all queried sources AND ≥3 distinct sources in `state.queries`.
G3 (→ 3)	`state.ranking` recorded; `selected_ids` non-empty; every selected paper has `score_components`.
G4 (→ 4)	All selected papers have `depth ∈ {full, shallow}` AND ≥80% are `depth=full`.
G5 (→ 5)	≥1 query with `source=openalex_citation_chase` and `hits > 0`.
G6 (→ 6)	`len(themes) ≥ 3` AND (`len(tensions) ≥ 1` OR a critique finding mentioning "no tensions").
G7 (→ 7)	`state.self_critique.appendix` non-empty; `len(resolved) ≥ len(findings)`.

Enrichment with MCP tools

If the session has Semantic Scholar (asta) or Brave Search MCP tools available, use them as enrichment:

mcp__asta__search_papers_by_relevance — good for dense relevance ranking
mcp__asta__get_citations — lighter weight than citation graph for a few seed papers
mcp__asta__snippet_search — grep-like search across abstracts
Brave Search — non-academic sources (blog posts, press releases, pre-print discussion)

Treat MCP tools as unreliable by design — they may timeout or be unavailable. Never place a phase-critical step behind an MCP call. Scripts are the spine; MCP is the skin.

Pitfalls (short list; see `references/pitfalls.md` for detail)

Treating the first page of search results as "the literature" — run multiple keyword clusters and chase citations.
Unanchored claims — every non-trivial statement in the report needs a [^id] pointing to a paper in state.
Confirmation bias — actively search for critiques of top-cited papers; see Phase 4 special case.
Preprint conflation — arXiv/bioRxiv are preprints; tag them as such in the report and weight evidence accordingly.
Venue monoculture — if >60% of top-N come from one journal/venue, broaden sources.
Author monoculture — same for a single lab or author.
Recency collapse — the last 2 years matter for "state of the art" framings; check explicit coverage.
Stale MCP tool names — MCP servers rename tools; always list available tools before assuming names. Script paths are stable; MCP names are not.
Single-shot search — budget for ≥3 search rounds per cluster, not one.
Skipping self-critique — the temptation to ship a clean draft is exactly when Phase 6 catches the most.

Example interaction

text

User: Give me a deep research report on CRISPR base editing for Duchenne muscular dystrophy.

Agent:
[Phase 0] Restating: "What is the current state and evidence for CRISPR base editing
          as a therapeutic approach for Duchenne muscular dystrophy (DMD)?"
          Archetype: literature_review (default; confirmed — broad overview requested)
          Clusters:
            - ("base editing" OR "adenine base editor" OR "cytosine base editor" OR ABE OR CBE)
            - ("Duchenne muscular dystrophy" OR DMD OR dystrophin)
            - (in vivo OR AAV OR "muscle delivery")
          → research_state.json initialized

[Phase 1] Running OpenAlex + PubMed + arXiv + Crossref across 3 clusters...
          Round 1: 187 hits, 142 unique. Round 2: 94 hits, 31 new.
          Saturation check: new=11%, max_new_citations=23 → SATURATED
          143 unique papers in state.

[Phase 2] Ranking with default weights (literature review)...
          Top 20 selected. Score components written to state.

[Phase 3] Fetching full text... 17/20 full, 3 abstract-only (flagged shallow).
          Evidence extraction complete.

[Phase 4] Citation chasing on top 8 seeds, depth 1.
          Added 24 candidates, 6 re-scored into top 20.

[Phase 5] Themes: (a) delivery platforms, (b) editing efficiency, (c) off-target/safety,
          (d) pre-clinical outcomes, (e) clinical translation barriers.
          Tensions: AAV serotype optimality (Theme a) — 3 papers disagree.

[Phase 6] Self-critique flagged 2 single-source claims and a recency gap
          (no 2025 paper in top 20). Re-ran focused search; added 4 papers.

[Phase 7] Rendering literature_review template...
          Report: reports/crispr-base-editing-dmd_20260411.md
          Bibliography: reports/crispr-base-editing-dmd_20260411.bib (84 refs)

References

Modular documentation, loaded only when needed:

references/search_strategies.md — Boolean clusters, PICO, snowballing, saturation math
references/source_selection.md — which database for which question
references/quality_assessment.md — CRAAP, journal tier, retraction check, preprint handling
references/report_templates.md — the 5 archetypes with section-by-section guidance
references/pitfalls.md — long-form version of the pitfalls list with examples

Scholar Deep Research

Install

Scholar Deep Research

When to use

Guiding principles

The 8-phase workflow (Phase 0..7)

Phase 0 — Scope

Phase 1 — Discovery

Phase 2 — Triage

Phase 3 — Deep read

Phase 4 — Citation chasing

Phase 5 — Synthesis

Phase 6 — Self-critique

Phase 7 — Report

Report archetype selection

Scripts reference

CLI contract

Environment variables

State file schema (abbreviated)

Completion gates

Enrichment with MCP tools

Pitfalls (short list; see references/pitfalls.md for detail)

Example interaction

References

Related skills

Pitfalls (short list; see `references/pitfalls.md` for detail)