InfoSeek

Deep web information search and archival skill for comprehensive research on persons, organizations, or products. Uses multiple search engines (Baidu, Tavily, Multi-Search-Engine) plus browser-based content extraction. Features URL-normalized deduplication with SQLite database, structured file storage with complete metadata, ad/content filtering, and customizable output formats (md/json/txt/csv/xlsx/html/docx). Automatically creates organized folders per subject. Enforces strict deletion policy (recycle bin only, user confirmation required). Use when: user needs comprehensive information gathering, background research, due diligence, media monitoring, competitive intelligence, or building information archives about specific subjects. Triggers: infoseek, deep search, information gathering, background check, media monitoring, archive search, research person, research company, competitive analysis, dossier building.

ExpeditionHub@expeditionhub

Install

openclaw skills install @expeditionhub/infoseek

InfoSeek - Deep Web Search & Archival

Overview

InfoSeek performs comprehensive web research on any subject (person, organization, product) across multiple search engines, deduplicates results, extracts clean content, and archives everything with full metadata in organized folders.

Prerequisites

Before executing a search task, verify these skills are installed:

python

import os
from pathlib import Path

workspace = os.environ.get('OPENCLAW_WORKSPACE')
skills_dir = Path(workspace) / 'skills'

required = ['baidu-search', 'tavily', 'Multi-Search-Engine', 'agent-browser-clawdbot-0.1.0']
missing = [s for s in required if not (skills_dir / s).exists()]

If any are missing, instruct the user to install them:

bash

openclaw skills install baidu-search
openclaw skills install tavily-search
openclaw skills install multi-search-engine

Workflow

Phase 0: Task Setup

Confirm the search subject — name, organization, or product
Collect optional context — background info, time range, output format (default: .md), special requirements
Check dependencies — run the prerequisite check above

Create archive folder — run:

bash

python scripts/infoseek_helper.py create-folder "<subject_name>"

Phase 1: Multi-Engine Deep Search

Execute searches across all available engines. Each engine runs independently.

1.1 Baidu Search (100+ pages)

Use the baidu-search skill:

Query: "<subject> <background_context>"
Depth: 100+ pages
Record: URL, title, website name, publish date for each result

1.2 Tavily Search

Use tavily_search tool:

text

query: "<subject> <background_context>"
search_depth: advanced
max_results: 50

1.3 Multi-Search-Engine

Use the multi-search-engine skill across multiple engines simultaneously.

1.4 Browser Deep-Crawl

For discovered URLs, use the browser tool to:

Open each page
Extract body content (filter ads, sidebars, comments)
Extract metadata: title, author, editor, date, website name

Phase 2: Deduplication

Run URL deduplication on all collected results:

bash

python scripts/infoseek_helper.py deduplicate "<temp_results_file>"

The script normalizes URLs (remove www, tracking params, unify http/https, remove trailing slashes) and checks against the SQLite database to skip duplicates.

Phase 3: Content Extraction & Storage

For each unique URL:

Extract content using the browser tool — get title, body, metadata
Filter content — remove ads, sidebars, navigation, comments, related articles, footers

Generate filename:

bash

python scripts/infoseek_helper.py generate-filename \
  --date "<YYYYMMDD>" --title "<title>" --website "<site>" --format "<ext>"

Format: YYYYMMDD-title-website.ext

Save the file:

bash

python scripts/infoseek_helper.py save-content \
  --folder "<archive_path>" --filename "<name>" --url "<url>" \
  --website "<site>" --source "<source>" --date "<date>" \
  --title "<title>" --author "<author>" --editor "<editor>" \
  --content "<body>" --task "<subject>"

Record in database:

bash

python scripts/infoseek_helper.py add-url \
  --url "<normalized_url>" --task "<subject>" --filename "<name>"

Phase 4: Task Report

Output a summary when complete:

text

InfoSeek Task Report
====================
Subject: {query}
Engines used: {engines}
Total found: {total} | Duplicates skipped: {dupes} | New archived: {new}
Files saved: {count}
Location: {path}
Database records: {db_total}

File Naming

Format: YYYYMMDD-title-website.ext

Date: 8 digits (YYYYMMDD) from page metadata
Title: page title (strip special chars <>:"/\|?*)
Website: domain or media name
Extension: md (default), json, txt, csv, xlsx, html, docx

If filename exists, append 8-char hash to prevent overwrites.

Output Formats

All formats include full metadata (URL, website, source, date, title, author, editor) plus body content.

.md — Markdown with metadata table
.json — Structured JSON with metadata object and content field
.txt — Plain text with header metadata
.csv — One row per article, all metadata as columns
.xlsx — Excel spreadsheet with metadata columns
.html — Styled HTML page with metadata table
.docx — Word document with metadata paragraph

Storage Structure

text

{workspace}/
├── infoseek-archives/
│   ├── <subject_1>/
│   │   ├── 20260404-title-website.md
│   │   └── ...
│   └── <subject_2>/
└── infoseek/
    ├── infoseek.db          # SQLite dedup database
    ├── infoseek.log         # Operation log
    └── backups/

Deletion Policy

Strict data retention — no permanent deletes without confirmation.

Operation	Confirmation	Method
Bulk folder delete	Required	Move to recycle bin
Single file delete	Required	Move to recycle bin
Dedup skip	Automatic	Skip only, no delete
Database cleanup	Required	Mark as deleted

Process:

List files to delete (name, URL, date)
Ask user: "Confirm deletion? Files go to recycle bin and can be recovered."
On confirmation, move to recycle bin (Windows: PowerShell, Mac/Linux: system trash)
Update database, log the deletion, confirm to user

Never:

Delete without user consent
Permanently delete (bypass recycle bin)
Delete without logging
Delete without updating database

Configuration

Override defaults in task instructions:

Search depth: default 100 pages, specify e.g. "150 pages"
Time range: default unlimited, specify e.g. "2020-01-01 to 2026-04-07"
Output format: default md, specify e.g. "xlsx"
Storage path: default {workspace}/infoseek-archives/, specify custom path

Troubleshooting

Problem	Solution
Missing search skill	`openclaw skills install <name>`
Date extraction fails	Check page metadata; use `00000000` for unknown
Encoding errors	Ensure UTF-8; on Windows enable Unicode UTF-8 in region settings
Database corruption	`python scripts/infoseek_helper.py restore-backup`

Security & Privacy

All searches use public channels only
No personal data stored — only search results
SQLite database is local, never uploaded
Deletions use system recycle bin (recoverable)
All operations logged and auditable
No telemetry, no external data transmission

Version History

Version	Date	Notes
2.0.0	2026-04-07	Full rewrite: SQLite dedup, URL normalization, HTML parsing, multi-engine integration
1.0.0	2026-04-06	Initial version (deprecated)

InfoSeek

Install

InfoSeek - Deep Web Search & Archival

Overview

Prerequisites

Workflow

Phase 0: Task Setup

Phase 1: Multi-Engine Deep Search

1.1 Baidu Search (100+ pages)

1.2 Tavily Search

1.3 Multi-Search-Engine

1.4 Browser Deep-Crawl

Phase 2: Deduplication

Phase 3: Content Extraction & Storage

Phase 4: Task Report

File Naming

Output Formats

Storage Structure

Deletion Policy

Configuration

Troubleshooting

Security & Privacy

Version History

Related skills