Back to skill

Security audit

arXiv Master Search

Security checks for vulnerabilities and agentic risk

Overview

The arXiv research skill is mostly coherent, but crafted input files can make it write JSON files outside the chosen output directory.

Review this skill before installing if you will process JSON or JSONL files from other people. Run it in a scratch output directory, avoid --no-skip-existing unless replacement is intended, prefer trusted metadata/query files only, and pin dependencies in your own environment.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (3)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/batch_search.py:141
Finding
Arbitrary File Write Through Unsanitized Batch Query Names<![CDATA[ ## Vulnerability Details **File Location**: `scripts/batch_search.py`, lines 141 and 188–189 **Vulnerability Type**: Path traversal and arbitrary file overwrite **Risk Level**: Medium ### Vulnerable Code ```python query_name = query_spec.get("name", f"query_{index:03d}") ``` ```python if save_individual: output_file = self.output_dir / f"{result['query_name']}.json" save_json(result["results"], output_file) ``` The called `save_json` function creates parent directories and opens the destination in write mode: ```python filepath = Path(filepath) filepath.parent.mkdir(parents=True, exist_ok=True) with open(filepath, "w", encoding="utf-8") as f: json.dump(data, f, indent=indent, ensure_ascii=ensure_ascii, default=str) ``` ### Technical Analysis The batch query `name` is read directly from an attacker-controlled JSONL object. It is later interpolated into an output path without rejecting absolute paths, directory separators, or `..` traversal components. Python's `pathlib` discards the left operand when the right operand is absolute. Consequently, a name such as `/tmp/target` produces `/tmp/target.json`. A relative name such as `../../target` can similarly escape the configured output directory. Because `save_json` creates missing parent directories and opens the destination in write mode, the issue provides a constrained arbitrary file-write primitive. The destination must end in `.json`, and the written content consists of search results. ### Attack Path 1. An attacker supplies or causes the user to process a crafted JSONL file. 2. The file contains a valid query and a malicious name, for example: ```json {"query":"security","max_results":1,"name":"../../attacker-controlled"} ``` 3. `run_query` accepts the `name` without validation. 4. The arXiv query succeeds. 5. `run_batch` constructs a path outside `output_dir`. 6. `save_json` creates parent directories where possible and overwrites the selected writable `.json` fi ...[truncated 377 chars]
Remediation
<![CDATA[ ## Remediation Suggestions - Reject absolute paths and query names containing `/`, `\`, `..`, or platform-specific separators. - Convert query names to safe basenames using a strict allowlist such as `[A-Za-z0-9._-]`. - Resolve both the output directory and destination, then enforce containment before writing: ```python safe_name = re.sub(r"[^A-Za-z0-9._-]", "_", query_name) base = self.output_dir.resolve() destination = (base / f"{safe_name}.json").resolve() if not destination.is_relative_to(base): raise ValueError("Output path escapes the configured directory") ``` - Use exclusive creation where overwriting is unnecessary. - Consider assigning server-generated filenames instead of accepting filenames from input data. - Add tests covering absolute paths, nested traversal, mixed separators, and symbolic-link escape scenarios. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/summarize.py:397
Finding
Arbitrary JSON File Write Through Unvalidated Metadata arXiv IDs<![CDATA[ ## Vulnerability Details **File Location**: `scripts/summarize.py`, lines 397–401 **Vulnerability Type**: Path traversal and arbitrary file overwrite **Risk Level**: Medium ### Vulnerable Code ```python for paper in papers: summary = summarizer.summarize_paper(paper) summaries.append(summary) arxiv_id = summary["arxiv_id"] paper_file = output_dir / f"{arxiv_id}_summary.json" save_json(summary, paper_file) ``` ### Technical Analysis When processing a metadata file containing multiple papers, the program uses each metadata object's `arxiv_id` as part of an output filename. Metadata loaded through `--metadata` is not schema-validated, and this code does not sanitize the identifier or verify that the resolved destination remains inside `output_dir`. An identifier containing traversal components, such as `../../target`, creates a destination outside the summary directory. An identifier beginning with an absolute path causes `pathlib` to ignore `output_dir` entirely. The downstream `save_json` function creates parent directories and overwrites existing files. Although `parse_arxiv_id` exists elsewhere, it does not enforce a strict arXiv ID grammar and is not invoked on metadata IDs in this path. ### Attack Path 1. An attacker provides a crafted metadata JSON file containing at least two papers so that the batch-summary branch executes. 2. One paper contains a malicious identifier, for example: ```json { "papers": [ {"arxiv_id":"2301.00001","title":"Paper One","abstract":""}, {"arxiv_id":"../../target","title":"Paper Two","abstract":""} ] } ``` 3. The user runs: ```bash python scripts/summarize.py --metadata malicious.json ``` 4. The malicious identifier is copied into the generated summary. 5. The identifier is interpolated into the output path. 6. `save_json` writes `../../target_summary.json`, escaping the configured summary directory and overwriting the destination if permitte ...[truncated 391 chars]
Remediation
<![CDATA[ ## Remediation Suggestions - Validate every metadata object against a schema before processing it. - Strictly accept only recognized modern and legacy arXiv ID formats. - Never use raw metadata fields as filesystem paths. - Generate a safe filename from a validated identifier: ```python if not VALID_ARXIV_ID.fullmatch(arxiv_id): raise ValueError("Invalid arXiv ID") safe_id = arxiv_id.replace("/", "_") base = output_dir.resolve() paper_file = (base / f"{safe_id}_summary.json").resolve() if not paper_file.is_relative_to(base): raise ValueError("Output path escapes the configured directory") ``` - Consider generating opaque filenames and storing the original arXiv ID only inside the JSON document. - Protect against symbolic-link escapes by checking resolved paths immediately before writing and by operating in a trusted output directory. ]]>

T09 · Insecure Skill Coding Practices

Note
Location
scripts/download.py:117
Finding
Unrestricted Output Path in the Paper Downloader API<![CDATA[ ## Vulnerability Details **File Location**: `scripts/download.py`, lines 117–169 **Vulnerability Type**: Arbitrary output path and file overwrite **Risk Level**: Low ### Vulnerable Code ```python def download_by_id( self, arxiv_id: str, filename: str = None, paper_metadata: Dict[str, Any] = None, ) -> Optional[Path]: """ 通过 arXiv ID 下载论文 Args: arxiv_id: arXiv ID filename: 输出文件名,如 None 则自动生成 paper_metadata: 论文元数据(用于生成文件名) Returns: 下载的文件路径,失败返回 None """ normalized_id = parse_arxiv_id(arxiv_id) pdf_url = self._get_pdf_url(normalized_id) # 确定文件名 if not filename: if paper_metadata: filename = self._generate_filename(paper_metadata) else: filename = f"{normalized_id}.pdf" filepath = self.output_dir / filename ``` The resulting path is subsequently opened in truncating write mode: ```python with open(filepath, "wb") as f: ``` ### Technical Analysis The public `download_by_id` API permits callers to supply `filename`, but that value is not passed through `sanitize_filename` and is not checked for absolute paths or traversal components. A path such as `../../target` can escape `output_dir`. An absolute filename causes `pathlib` to discard `output_dir`. If `skip_existing` is disabled, an existing writable destination is truncated and replaced with the response body downloaded from the fixed arXiv endpoint. The standard CLI does not expose the `filename` parameter and normally generates sanitized names. Therefore, exploitation primarily affects applications that import this class and pass untrusted filename values, which lowers the practical severity. ### Attack Path 1. An application imports `PaperDownloader` and permits an untrusted party to influence the `filename` argument. 2. The attacker supplies an absolute path or traversal path. 3. The application invokes `download_by_id` with `skip_existing=False`, or selects a ...[truncated 891 chars]
Remediation
<![CDATA[ ## Remediation Suggestions - Treat the `filename` API argument as an untrusted basename rather than a path. - Reject absolute paths, parent-directory components, and directory separators. - Sanitize caller-provided values before joining them to `output_dir`: ```python candidate = Path(filename) if candidate.is_absolute() or len(candidate.parts) != 1: raise ValueError("filename must be a basename") filename = sanitize_filename(candidate.name) ``` - Resolve the final destination and verify that it remains inside the resolved output directory. - Write downloads to a temporary file inside the output directory, validate the response, and atomically rename it into place. - Validate the response content type and impose a maximum download size. - Avoid overwriting existing files by default and require explicit authorization for replacement. ]]>
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (36)

Natural-Language Policy Violations

Medium
Confidence
72% confidence
Finding
All user-facing documentation in this file is presented in Chinese, while no opt-in, alternative language, or justification for a Chinese-only experience is provided. The policy requires flagging language or locale constraints when the skill effectively forces a specific language without user choice.

Missing User Warnings

Medium
Confidence
89% confidence
Finding
This markdown file applies to SQP-2, and it describes a destructive option: `--no-skip-existing` will overwrite existing files. Although the option is shown, the documentation does not include any explicit warning that existing PDFs may be replaced, which could affect user data.

Natural-Language Policy Violations

Medium
Confidence
93% confidence
Finding
This file hard-codes user-facing descriptions and help text in Chinese, including the module docstring and argparse help strings. Under the policy, forcing a specific language without user opt-in is a natural-language policy violation unless the locale restriction is explicitly justified.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
# 执行模块
    import subprocess

    return subprocess.call([sys.executable] + new_argv)


if __name__ == "__main__":
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Natural-Language Policy Violations

Medium
Confidence
93% confidence
Finding
SQP-3 applies to all file types and covers language or locale policy violations. This file presents its natural-language interface, help text, and operational messages in Chinese only, with no opt-in, alternative locale, or justification that the tool is region-specific.

Natural-Language Policy Violations

Medium
Confidence
97% confidence
Finding
This file presents its module description, CLI help text, and operational messages entirely in Chinese. Under the policy, forcing a specific language without offering a user choice or documenting a justified locale constraint is a natural-language policy violation.

Natural-Language Policy Violations

Medium
Confidence
91% confidence
Finding
This file's user-facing natural-language interface and help text are presented only in Chinese, including the module description and CLI argument documentation. Under the stated policy, forcing a specific language without user opt-in is a natural-language policy violation.

Natural-Language Policy Violations

Medium
Confidence
90% confidence
Finding
SQP-3 applies to all file types and covers language or locale policy violations in natural-language strings. The module docstring, CLI help text, and runtime messages are presented only in Chinese, with no opt-in or alternative language support, which can violate an organizational policy requiring language choice.

Missing User Warnings

Low
Confidence
81% confidence
Finding
This markdown file documents operations that download papers and save outputs to local files and directories, including batch downloads and generated metadata/summaries. Under the markdown-specific warning rule, the description should disclose that the skill writes data to disk and may create or populate output directories so users understand the impact on their filesystem.

Natural-Language Policy Violations

Low
Confidence
96% confidence
Finding
SQP-3 applies to all file types and includes language or locale policy violations. This file presents all instructional content in Chinese and does not indicate that users may choose another language or that the skill is intentionally limited to a Chinese-speaking context.

Natural-Language Policy Violations

Low
Confidence
92% confidence
Finding
The natural-language comments are entirely in Chinese ('示例 arXiv ID 列表', '每行一个 ID,可以包含版本号或完整 URL') with no indication that language choice is optional or region-specific. This can violate language/locale policy when a skill or resource imposes a specific language without user opt-in.

Natural-Language Policy Violations

Low
Confidence
92% confidence
Finding
This requirements file uses Chinese-only natural-language comments such as '核心 arXiv API', 'HTTP 请求', and similar labels throughout. For a general skill file, this imposes a specific language on maintainers or reviewers without offering a language choice or documenting a justified locale constraint.

Unpinned Dependencies

Low
Category
Supply Chain
Content
# arXiv Search Skill Dependencies

# 核心 arXiv API
arxiv>=1.4.8

# HTTP 请求
requests>=2.28.0
Confidence
92% confidence
Finding
Using a lower-bounded but unpinned dependency for arxiv allows future installs to resolve to newer, unreviewed versions with different behavior or newly introduced vulnerabilities. This weakens build reproducibility and increases supply-chain risk, especially for agent skills that may be deployed repeatedly in different environments.

Unpinned Dependencies

Low
Category
Supply Chain
Content
arxiv>=1.4.8

# HTTP 请求
requests>=2.28.0
urllib3>=1.26.0

# 数据处理
Confidence
97% confidence
Finding
An unpinned requests dependency permits installation of arbitrary newer releases at deployment time, making the runtime version non-deterministic. Because this library handles outbound HTTP and authentication-related features, version drift can expose the skill to known or future security issues and behavior changes.

Unverifiable Dependency: requests has 16 known advisory(ies) (CVE-2014-1830 (Exposure of Sensitive Information to an Unauthorized Actor in Requests); CVE-2024-47081 (Requests vulnerable to .netrc credentials leak via malicious URLs); CVE-2024-35195 (Requests `Session` object does not verify requests after making first request wi) +13 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
97% confidence
Finding
The manifest does not pin requests, so it is impossible to verify whether deployed environments will get a patched or vulnerable version despite known advisories affecting some releases. Because requests is used for HTTP interactions, unresolved version ambiguity can expose the skill to credential leakage, TLS-validation issues, or other network-security defects depending on the resolved version.

Unpinned Dependencies

Low
Category
Supply Chain
Content
# HTTP 请求
requests>=2.28.0
urllib3>=1.26.0

# 数据处理
pandas>=1.5.0
Confidence
97% confidence
Finding
Using urllib3 with only a minimum version allows dependency resolution to pull in later releases without review, reducing reproducibility and complicating vulnerability management. Since urllib3 is a core HTTP transport component, weaknesses in it can affect TLS handling, redirects, proxy behavior, and response processing.

Unverifiable Dependency: urllib3 has 16 known advisory(ies) (CVE-2025-66471 (urllib3 streaming API improperly handles highly compressed data); CVE-2024-37891 (urllib3's Proxy-Authorization request header isn't stripped during cross-origin ); CVE-2026-21441 (Decompression-bomb safeguards bypassed when following HTTP redirects (streaming ) +13 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
97% confidence
Finding
urllib3 has known advisories, but the requirement is not pinned, so the actual security posture of any deployment cannot be verified from this manifest. Since urllib3 underpins HTTP transport, vulnerable versions can affect redirect handling, proxy behavior, decompression, or TLS-related logic in ways that increase attack surface.

Unpinned Dependencies

Low
Category
Supply Chain
Content
urllib3>=1.26.0

# 数据处理
pandas>=1.5.0
numpy>=1.23.0

# YAML 配置
Confidence
88% confidence
Finding
An unpinned pandas dependency creates non-reproducible builds and may bring in future versions with security or compatibility regressions. While pandas is less security-sensitive than networking libraries in this context, dependency drift still increases supply-chain uncertainty.

Unverifiable Dependency: pandas has 1 known advisory(ies) (CVE-2020-13091 (** DISPUTED ** pandas through 1.0.3 can unserialize and execute commands from an)), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
86% confidence
Finding
Because pandas is not pinned, the manifest cannot prove that installations avoid advisory-affected versions. In this skill, pandas is likely used for local data handling, so the direct security impact is lower, but the unverifiable version still represents supply-chain uncertainty.

Unpinned Dependencies

Low
Category
Supply Chain
Content
# 数据处理
pandas>=1.5.0
numpy>=1.23.0

# YAML 配置
PyYAML>=6.0
Confidence
88% confidence
Finding
Specifying numpy with only a lower bound means the installed version can vary across environments and over time. This is primarily a supply-chain and reproducibility concern, with lower direct security impact here than for network-facing libraries.

Unverifiable Dependency: numpy has 16 known advisory(ies) (CVE-2014-1859 (Numpy arbitrary file write via symlink attack); CVE-2021-41495 (NumPy NULL Pointer Dereference); CVE-2021-33430 (NumPy Buffer Overflow (Disputed)) +13 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
86% confidence
Finding
The unpinned numpy requirement means a deployment may install a version with known issues, and that cannot be ruled out from this file alone. Although numpy is not the most exposed component here, unverifiable dependency versions still undermine secure and reproducible builds.

Unpinned Dependencies

Low
Category
Supply Chain
Content
numpy>=1.23.0

# YAML 配置
PyYAML>=6.0

# 并行处理
tqdm>=4.64.0
Confidence
96% confidence
Finding
An unpinned PyYAML dependency is risky because YAML parsers have a history of unsafe-deserialization issues and parser-related vulnerabilities. Without a fixed reviewed version, deployments may resolve to versions with different security properties or expose the skill to known vulnerable releases indirectly.

Unverifiable Dependency: PyYAML has 8 known advisory(ies) (CVE-2019-20477 (Deserialization of Untrusted Data in PyYAML); CVE-2020-1747 (Improper Input Validation in PyYAML); CVE-2020-14343 (Improper Input Validation in PyYAML) +5 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
97% confidence
Finding
PyYAML has a history of serious deserialization and input-validation issues, and the lack of a pinned version makes it impossible to know whether deployments are safe. In an agent skill context where configuration may be loaded dynamically, that ambiguity makes this more concerning than a generic utility package.

Unpinned Dependencies

Low
Category
Supply Chain
Content
PyYAML>=6.0

# 并行处理
tqdm>=4.64.0

# PDF 处理
PyPDF2>=3.0.0
Confidence
90% confidence
Finding
Leaving tqdm unpinned allows version drift and reduces reproducibility, which can matter if CLI-related vulnerabilities or argument-handling bugs exist in some releases. In this file, the security impact is limited but still a valid supply-chain hygiene issue.

Unverifiable Dependency: tqdm has 4 known advisory(ies) (CVE-2024-34062 (tqdm CLI arguments injection attack); CVE-2016-10075 (TDQM Arbitrary Code Execution); CVE-2016-10075 (The tqdm._version module in tqdm versions 4.4.1 and 4.10 allows local users to e) +1 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
88% confidence
Finding
tqdm has known advisories in some versions, and because the requirement is unpinned, the manifest cannot establish whether a deployment will be affected. The practical impact in this skill is limited unless tqdm CLI functionality is exposed, but it remains a valid supply-chain risk.

Static analysis

No suspicious patterns detected.