Openclaw Web Search Mcp

Provides AI agents with web search, page and PDF extraction, YouTube transcripts, summarization, semantic search, and automated research workflows.

Security Scan

The skill mostly matches its advertised browsing/research purpose but contains several incoherent or risky implementation choices (hardcoded external binary path, execSync usage, mismatched search provider, and undeclared runtime requirements) that warrant caution before installing.

Purpose & Capability

Overall functionality (search, page/PDF extraction, transcripts, summarization, embeddings, research) matches the description. However there are mismatches: SKILL.md claims Google search but the code uses DuckDuckGo; mcp.json advertises a 'crawl_site' tool that has no implementation in index.js; package.json lists a 'youtube-transcript' dependency but the code calls an external yt-dlp executable instead. These inconsistencies suggest sloppy packaging or unmet assumptions about the host environment.

Instruction Scope

SKILL.md gives a straightforward CLI usage, but the implementation runs a shell command (child_process.execSync) to invoke a local yt-dlp binary and reads/writes subtitle files in the package directory. The runtime instructions do not declare that yt-dlp is required or that the skill will execute shell commands and touch files, which expands the agent's scope beyond what's documented.

Install Mechanism

There is no packaged install spec beyond 'npm install' (package.json and package-lock.json are provided). Using npm is normal, and dependencies are standard. No remote/external archive downloads are performed by the MCP itself. Still, the code depends on an external non-npm tool (yt-dlp) invoked at runtime, which is not installed via the provided instructions.

Credentials

The skill declares no required environment variables or binaries, yet extraction/youtubeTranscript.js executes a hardcoded absolute path (/home/nishu/.local/bin/yt-dlp). Requiring an external binary but not declaring it is incoherent. The hardcoded path embeds a specific user home (exposing author environment details) and will likely fail or invoke an unexpected binary on other hosts.

Persistence & Privilege

The skill does not request elevated persistence: always:false, no reported changes to other skills or system-wide configuration. It writes a temporary subtitle file inside its directory during transcript extraction, which is limited scope.

What to consider before installing

This skill appears to implement the advertised browsing and research features but has several red flags you should resolve before installing: - The YouTube transcript code runs a shell command (yt-dlp) via execSync and calls a hardcoded path (/home/nishu/.local/bin/yt-dlp). Ask the author to (a) remove the hardcoded path, (b) document that yt-dlp is a required external binary (and how to install it), or (c) use a Node library instead. Running unverified shell commands is a risk. - SKILL.md says 'Google search' but the code scrapes DuckDuckGo. Confirm which search provider you expect and whether scraping is acceptable for your use case. - mcp.json lists a tool ('crawl_site') that is not implemented; verify the tool list and behavior to ensure there is no hidden functionality. - Because the code uses execSync and will write temporary files, run this skill in a sandboxed environment (or on an isolated agent) until you validate it. Review/modify the youtubeTranscript code to avoid arbitrary shell execution if you cannot fully trust the package. If you are not comfortable auditing or modifying the code, do not install this skill in a production or high-privilege agent. Asking the author for corrected packaging and explicit runtime requirements would materially reduce the risk.

Web Search and Research MCP

This MCP provides comprehensive web browsing, search, and research capabilities for AI agents, including PDF extraction, YouTube transcripts, summarization, and semantic search.

Overview

This MCP implements a full suite of tools for AI agents to interact with the web:

• Google search integration
• Web page content extraction
• Link extraction from pages
• PDF text extraction
• YouTube video transcript retrieval
• Text summarization
• Text embedding and semantic search
• Automated research workflows

Features

• Search: Perform Google searches and get structured results
• Page Extraction: Extract clean text content from web pages
• Link Extraction: Get all links from a webpage
• PDF Processing: Extract text from PDF documents
• YouTube Transcripts: Retrieve transcripts from YouTube videos
• Summarization: Generate concise summaries of text content
• Embeddings: Store and search text using semantic embeddings
• Research: Automated multi-step research workflows

Tools

• search - Google search with results
• open_page - Extract text from web pages
• extract_links - Get links from pages
• extract_pdf - Extract text from PDFs
• youtube_transcript - Get YouTube video transcripts
• summarize - Summarize text content
• embed - Store text embeddings
• semantic_search - Search stored embeddings
• research - Conduct automated research

Installation

npm install

Usage

Run the MCP server:

node index.js <tool_name> <json_input>

Example:

node index.js search '{"query":"artificial intelligence"}'

Dependencies

• axios: HTTP requests
• cheerio: HTML parsing
• natural: Text processing
• pdf-parse: PDF text extraction
• youtube-transcript: YouTube transcript API