Back to skill

Security audit

follow-arxiv

Security checks for vulnerabilities and agentic risk

Overview

The skill is mostly aligned with arXiv paper search and analysis, but it has unsafe persistent prompt editing and unvalidated external document handling that users should review before installing.

Install only if you are comfortable with the skill making outbound arXiv/PDF requests, caching PDFs locally, and storing editable prompt templates under your home directory. Review or patch prompt-name validation and PDF URL/size checks before using it in an agent environment with broad file or network access.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (4)

T09 · Insecure Skill Coding Practices

Error
Location
src/utils.py:75
Finding

Path Traversal Enables Arbitrary Markdown File Read, Write, and Deletion

Content
View full analysis
str: user_prompt_path = os.path.join( get_prompts_dir(), f"{prompt_name}.md" ) if os.path.exists(user_prompt_path): with open(user_prompt_path, 'r', encoding='utf-8') as f: return f.read() default_prompt_path = os.path.join( "prompts", f"{prompt_name}.md" ) if os.path.exists(default_prompt_path): with open(default_prompt_path, 'r', encoding='utf-8') as f: return f.read() raise FileNotFoundError( f"Prompt template {prompt_name} does not exist" ) def save_prompt(prompt_name: str, content: str) -> None: prompts_dir = get_prompts_dir() prompt_path = os.path.join(prompts_dir, f"{prompt_name}.md") with open(prompt_path, 'w', encoding='utf-8') as f: f.write(content) ``` The corresponding deletion operation is: ```python user_prompts_dir = get_prompts_dir() user_prompt_path = os.path.join(user_prompts_dir, f"{name}.md") if os.path.exists(user_prompt_path): os.remove(user_prompt_path) ``` ### Technical Analysis The prompt name is supplied directly through the `get-prompt`, `set-prompt`, and `reset-prompt` CLI commands. It is concatenated with the `.md` suffix and passed to `os.path.join()` without validation. A value containing `../` components can escape the intended `~/.arxiv-search/prompts/` directory. An absolute path is also dangerous because `os.path.join()` discards its preceding directory when a later component is absolute. The `.md` suffix limits direct access to paths ending in that extension, but does not prevent access to sensitive Markdown files elsewhere on the filesystem. Existing symbolic links may also broaden the overwrite impact. ### Attack Pat ...[truncated 1270 chars]
Remediation
View remediation

T01 · Skill Instruction Hijacking

Error
Location
src/analyzer.py:40
Finding

Indirect Prompt Injection Through Untrusted arXiv Metadata and PDF Text

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
src/pdf_processor.py:20
Finding

Unvalidated PDF URL Allows Server-Side Request Forgery in the Public Processing API

Content
View full analysis
str: pdf_path = os.path.join(self.pdf_dir, f"{arxiv_id}.pdf") if os.path.exists(pdf_path): return pdf_path for attempt in range(self.max_retries): try: response = requests.get(pdf_url, timeout=30) response.raise_for_status() with open(pdf_path, 'wb') as f: f.write(response.content) return pdf_path except Exception as e: if attempt == self.max_retries - 1: raise Exception(f"PDF download failed: {e}") continue raise Exception("PDF download failed") ``` The public helper accepts caller-provided metadata: ```python def process_pdf( paper_metadata: Dict[str, Any], config: Optional[Dict[str, Any]] = None ) -> Dict[str, Any]: if config is None: from utils import load_config config = load_config() processor = PDFProcessor(config) return processor.process_paper(paper_metadata) ``` Within `process_paper()`: ```python arxiv_id = paper_metadata['arxiv_id'] pdf_url = paper_metadata['pdf_url'] pdf_path = self.download_pdf(pdf_url, arxiv_id) ``` ### Technical Analysis `requests.get()` is called with an unvalidated URL. There is no restriction on: - URL scheme. - Destination hostname. - Resolved IP address. - Loopback, private, or link-local networks. - Redirect targets. - Whether the destination belongs to arXiv. The normal CLI path obtains `pdf_url` from the arXiv client, which lowers exposure for standard usage. However, `process_pdf()` is documented and exported as a programmatic API accepting an arbitrary metadata dictionary. Any caller able to control that dictionary can select the request destination ...[truncated 1331 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
src/pdf_processor.py:34
Finding

Unbounded PDF Download and Parsing Enables Resource Exhaustion

Content
View full analysis
Remediation
View remediation
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • System Prompt LeakageDirect Leakage, Indirect Extraction, Tool-Based Exfiltration
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Findings (36)

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

代码片段只包含一个 setup_encoding() 函数,并在导入时自动执行,用于修复 Windows 控制台的 UTF-8 输出问题。这属于通用环境初始化逻辑,而不是 arXiv 论文处理能力。根据给定代码,既没有网络访问、时间范围过滤、论文检索、下载,也没有任何文本分析或摘要处理流程。因此该代码实际行为与声明的核心用途明显不一致,应判定为描述与行为不匹配。

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
94% confidence
Finding

该代码块的核心功能是“为后续 Agent 生成分析报告或日报摘要准备上下文和提示词”,属于论文分析流水线中的一个子模块。它确实与‘深度分析特定论文’和‘最新论文概要’部分声明部分相关,但不包含声明中更关键、更上游的 arXiv 搜索、时间窗口检索、下载或读取论文的逻辑。因此,实际代码行为只覆盖了声明的一部分,且缺失了声明中重要的主要能力,构成描述与实现不一致。

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
94% confidence
Finding

声明描述的是一个较完整的 arXiv 论文处理技能:按时间搜索、下载、阅读并深度分析论文。而实际代码块的主功能仅是调用 arxiv API 进行检索,按发布时间过滤结果,并提取元数据返回。虽然返回结果中包含 pdf_url 和 summary,可作为后续下载或阅读的基础,但当前代码本身并未执行这些能力。因此代码行为只覆盖了声明中的“搜索过去一段时间内的 arXiv 论文并返回概要信息”这一部分,未覆盖“下载、阅读、深度分析特定论文”等重要能力,属于描述与实际行为不一致。

Content

No source excerpt is available for this finding.

Ssd 1

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The flow instructs the model to directly use a prompt field read from a temporary file as authoritative instructions. Because that file is derived from external data and intermediate processing, this creates a prompt-injection sink where untrusted content can steer the model into unintended actions or policy bypass attempts.

Content

No source excerpt is available for this finding.

Ssd 1

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The deep-analysis workflow tells the model to obey a prompt field populated from downloaded paper content. Since papers are attacker-controlled text from the network, this is a classic indirect prompt-injection path that can manipulate analysis behavior, exfiltration attempts, or future persistent changes if combined with other writable features.

Content

No source excerpt is available for this finding.

Direct Prompt Extraction

High
Category
System Prompt Leakage
Confidence
85% confidence
Finding

Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Content

Scanner excerpt · src/utils.py (reported line 69)May include surrounding context.

python
"""获取用户提示词目录"""
    prompts_dir = os.path.join(get_config_dir(), "prompts")
    os.makedirs(prompts_dir, exist_ok=True)
    return prompts_dir


def load_prompt(prompt_name: str) -> str:

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
90% confidence
Finding

The skill describes capabilities that require file reads/writes and network access, but it does not declare any explicit tool scope or allowed-tools boundary. That makes the effective privilege surface ambiguous and increases the chance the skill is invoked with broader capabilities than users or policy expect.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The manifest frames the skill as paper search and analysis, but the instructions also allow persistent modification of user configuration and prompt templates. That materially expands the skill's authority and creates persistence mechanisms not clearly justified or disclosed by the core purpose.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The activation criteria are broad enough to match ordinary requests about papers or summaries, increasing the chance the skill triggers in contexts where the user did not intend network access, file writes, or persistent config changes. Overbroad triggering can turn otherwise acceptable capabilities into surprise side effects.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill tells the model and user that it runs entirely locally, yet it also instructs querying arXiv and downloading PDFs over the network. This is a deceptive or misleading disclosure that can undermine user consent and cause unexpected outbound data flows.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The skill permits arbitrary editing of prompt templates via CLI commands and shell piping, which can persistently alter future model behavior. Because this capability is only loosely related to following arXiv papers, it creates a durable prompt-injection and behavior-tampering surface with potentially broad downstream effects.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The onboarding function and CLI argument parser default the language to "zh", which imposes a specific locale without user opt-in. This is a natural-language policy concern because the tool silently selects a language preference instead of asking the user to choose or clearly requiring consent.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The CLI exposes prompt-template management commands (get/set/reset/list) that are outside the skill’s stated arXiv search, download, and analysis purpose. In an agent context, this expands the attack surface by allowing an operator or upstream prompt flow to modify analysis prompts, which can enable prompt injection persistence, behavior tampering, or data exfiltration through altered templates.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The reset_prompt command performs a file deletion with os.remove on the user's custom prompt file. Although the docstring mentions reset behavior, there is no confirmation prompt or explicit user-facing warning at the point of deletion, and this is a destructive operation on local data.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The complete-onboarding command sets --language to a default of "zh", which means the CLI applies a specific language policy even when the user provides no preference. This does not offer an opt-in or choice-based language selection.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The config sets the language to "zh" by default, which imposes a specific locale without indicating user opt-in or a region-specific requirement. This matches the policy category for language or locale constraints that are enforced without offering a choice.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The prompt is written as an instruction set in Chinese and frames the assistant role and output requirements entirely in Chinese, which effectively constrains the skill to a specific language. The policy for this audit flags language or locale constraints unless the skill offers a user choice or clearly documents a justified regional requirement.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
86% confidence
Finding

The module docstring and inline natural-language documentation are exclusively in Chinese, which imposes a specific language/locale context without indicating user choice or a documented regional requirement. Under the stated policy, forcing a language without opt-in is a locale-policy concern.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The download logic implements a general-purpose remote fetch using requests.get(pdf_url) with no domain, scheme, content-type, or size restrictions. Because this skill is supposed to handle arXiv papers, the unrestricted network access expands capability beyond its stated purpose and can be leveraged to retrieve arbitrary attacker-controlled content or probe reachable services.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The code performs a network request to fetch a PDF and then writes the response content to a local file. Although the docstrings describe downloading, there is no user-facing prompt, log message, or explicit warning that external content will be retrieved and stored locally.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The processor trusts paper_metadata['pdf_url'] and passes it directly into requests.get without validating that the URL is actually an arXiv PDF endpoint. In an agent context, this creates an arbitrary outbound fetch capability that can be abused for SSRF-style access to internal resources or downloading unexpected content while masquerading as an arXiv workflow.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

The manifest describes a skill for searching recent arXiv papers, downloading them, reading them, and analyzing them. This file creates a persistent user config directory and stores configuration and prompt data under ~/.arxiv-search, which is additional local-state behavior not mentioned in the skill description and is not an obvious requirement of the stated paper-processing workflow.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The default configuration sets the language to "zh", which establishes a specific language preference automatically. Per policy, forcing a language or locale without offering user choice or opt-in is a natural-language policy violation.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The utility can load and save arbitrary prompt templates from persistent local storage, creating a hidden prompt-injection surface that is not described in the skill metadata. If other parts of the skill consume these prompts as trusted instructions, a local attacker or prior malicious modification could persistently alter model behavior, data handling, or downstream actions.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
91% confidence
Finding

The markdown instructs users to manually delete ~/.arxiv-search/config.json and files under ~/.arxiv-search/prompts/ to reset configuration or restore defaults. While this is functionally relevant, the documentation does not explicitly warn that these actions permanently remove user configuration and custom prompt data.

Content

No source excerpt is available for this finding.