wos-crawler

WOS 文献一站式工具：Web of Science 检索爬取 + PDF 批量下载，全部在同一个 Web 界面完成。 基于 Selenium 爬取 WOS 文献列表，导出 Excel 后自动调用多渠道 PDF 下载器（Sci-Hub/CORE/S2 OA/OpenAlex/Unpaywall/Publisher...

Security Scan

The skill's description asks the agent to run local Selenium-based scripts, persist cookies, and download PDFs from multiple external services, but the package contains only an instruction document and declares no code, installs, or config — the pieces don't line up.

Purpose & Capability

The declared purpose (WOS crawling + multi-source PDF download) is plausible for the described tools (Selenium, Sci‑Hub, Unpaywall, etc.), but the skill manifest declares no code, no install, no required binaries, and no config paths while the SKILL.md explicitly instructs running python {SKILL_DIR}/scripts/web_ui.py and using Edge + Selenium. That mismatch (instructions requiring scripts and runtime components that are not present or declared) is internally inconsistent.

Instruction Scope

SKILL.md tells the agent/user to run a local web UI that will: open/automate a browser to log into Web of Science, persist cookies, export Excel, and use multiple external download channels (including Sci‑Hub). Those actions involve reading/writing local files, storing cookies, and making network requests to third parties; yet the package provides no implementation and gives broad autonomy to perform credentialed access and file I/O without specifying where data or cookies are stored. Instructions therefore grant broad scope without corresponding code or safeguards.

Install Mechanism

There is no install specification (instruction-only), which is low risk in itself — but SKILL.md assumes existence of scripts and Python dependencies (selenium, flask, etc.) and instructs running a local script. The absence of any shipped code or install steps makes the skill unusable as packaged and raises questions about where the referenced scripts come from.

Credentials

The SKILL.md requires a WOS institution account and an Edge browser and implies persistent cookies and local storage of downloaded PDFs, yet requires.env, primary credential, and config path fields are all empty. Sensitive items (WOS login session/cookies) are indirectly required but not declared or scoped, which is disproportionate and opaque.

Persistence & Privilege

The skill states it will persist cookies and perform breakpoint-resume of downloads and write PDFs to an output directory, indicating it will create and maintain local state. However, the package does not declare where that state is stored nor provide code. While always:false and autonomous invocation are normal, undocumented persistence of cookies and credential-bearing sessions increases risk and should be clarified.

What to consider before installing

Do not run or grant credentials to this skill as-is. The SKILL.md expects local scripts (scripts/web_ui.py), Python packages, browser automation, and persistent cookie storage, but the package contains no code or install instructions — it appears incomplete or mismatched. Before installing or running anything, ask the publisher for: (1) the source repository or packaged code; (2) exact install steps and a manifest of files the skill will write (where cookies and PDFs are stored); (3) a list of external endpoints/domains the skill contacts (e.g., Sci‑Hub, publishers); (4) whether any credentials are uploaded or transmitted externally. Be especially cautious about giving your WOS institutional credentials to unknown software and about automated access to Sci‑Hub or publisher sites — those can have legal and policy implications. If the author cannot provide verifiable code or a trustworthy source repo, treat the skill as unsafe to run.

WOS Literature Toolkit

WOS 文献一站式解决方案：从 Web of Science 检索爬取到 PDF 批量下载，全部在同一个 Web 界面完成。

功能概览

Phase 1 - WOS Crawl:

• 自然语言检索词自动转换为 WOS 高级检索式
• 支持关键词/作者/标题/DOI/期刊/年份/文献类型多维度检索
• 期刊过滤：内置 40+ 预设期刊，支持精确/模糊匹配
• 自动分页爬取、结果去重
• 自动导出 Excel（标题、作者、期刊、DOI、被引频次、摘要等）

Phase 2 - PDF Download:

• 自动使用 Phase 1 爬取的 Excel，一键启动 PDF 下载
• 也支持手动上传任意 Excel 文件
• 6 个免费下载渠道按优先级依次尝试
• 实时进度 + 下载统计 + 渠道分布 + 日志
• 自定义输出目录

下载渠道优先级: Sci-Hub (CDN + 爬取) -> Semantic Scholar OA -> CORE -> Unpaywall -> OpenAlex -> Publisher Direct

使用方式

python {SKILL_DIR}/scripts/web_ui.py

启动后自动打开 http://localhost:5678

工作流程

1. 打开 Web UI，在 WOS Crawl 标签页填写检索条件
2. 点击 Start WOS Crawl，浏览器自动打开 WOS 并开始爬取
3. 爬取完成后自动导出 Excel，点击 Download PDFs 按钮切换到下载标签
4. 在 PDF Download 标签页确认设置，点击开始下载
5. 下载完成后点击 Open PDF Folder 查看所有 PDF 文件

也可以跳过爬取直接下载

如果已有 WOS 导出的 Excel 文件：

1. 切换到 PDF Download 标签页
2. 在 Option B 区域上传 Excel 文件
3. 选择输出目录，点击开始下载

前置条件

• Python 3.7+
• 依赖: pip install selenium pandas openpyxl flask requests
• Edge 浏览器（WOS 爬取需要）
• WOS 机构订阅或登录账号

Agent 使用指南

当用户需要从 WOS 检索并下载文献 PDF 时：

1. 启动 Web UI: python {SKILL_DIR}/scripts/web_ui.py
2. 引导用户在浏览器中填写检索条件
3. WOS 爬取需要用户登录机构账号（首次运行时浏览器会自动打开 WOS 页面）
4. 爬取完成后引导用户点击 "Download PDFs" 按钮
5. 下载完成后帮助用户打开输出文件夹查看结果

关键注意事项

• WOS 爬取需要机构订阅或已登录 WOS 账号
• Sci-Hub 渠道需要网络能访问（部分网络环境可能不可用）
• Cookie 持久化：首次登录 WOS 后会自动保存 Cookie，后续无需重复登录
• 已下载的 PDF 会被自动跳过（断点续传）