Back to skill

Security audit

Arxiv Search

Security checks for vulnerabilities and agentic risk

Overview

The core arXiv metadata task is legitimate, but the package has under-scoped write behavior and bundled workflow tooling beyond that narrow purpose.

Review before installing. Use this only in a disposable or version-controlled workspace, avoid passing custom --outputs values, and do not let untrusted UNITS.csv or pipeline files control execution until output paths are constrained to the intended workspace files.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/run.py:47
Finding

Workspace Escape and Arbitrary File Overwrite via Unvalidated Output Path

Content
View full analysis
None: ...[truncated 3213 chars]
Remediation
View remediation
Path: if not relative: raise ValueError("Output path must not be empty") supplied = Path(relative) if supplied.is_absolute(): raise ValueError("Absolute output paths are not allowed") root = workspace.resolve() destination = (root / supplied).resolve() try: destination.relative_to(root) except ValueError as exc: raise ValueError( f"Output path escapes the workspace: {relative!r}" ) from exc return destination ``` 2. Use this function instead of directly joining the supplied value: ```python outputs = parse_semicolon_list(args.outputs) or ["papers/papers_raw.jsonl"] out_path = workspace_path(workspace, outputs[0]) ``` 3. Prefer a strict allowlist because this Skill declares a fixed output contract: ```python ALLOWED_OUTPUTS = {"papers/papers_raw.jsonl"} if outputs[0] not in ALLOWED_OUTPUTS: raise ValueError(f"Unsupported output path: {outputs[0]!r}") ``` 4. Repeat validation in `tooling/executor.py`. Entry-point validation remains mandatory, but executor-side validation provides defense in depth and rejects malicious `UNITS.csv` content earlier. 5. Validate the derived CSV path separately and ensure it remains in the workspace. 6. Consider symlink-aware hardening where workspaces may be attacker-controlled. Resolve the destination's parent and verify it remains below the resolved workspace immediately before writing. 7. Add regression tests covering: - `../../outside.jsonl` - Absolute paths - Nested traversal paths - Symlinked output directories - The valid `papers/papers_raw.jsonl` destination - Validation of the derived CSV destination ]]>
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (43)

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The manifest repeatedly frames the skill as metadata-only, yet it explicitly instructs the agent to execute python scripts/run.py, a subprocess boundary with potentially much broader behavior than the markdown claims. If run.py can orchestrate workflow state, approvals, logs, or write to additional locations, the skill becomes a confused-deputy vector where a seemingly narrow retrieval skill triggers much wider side effects.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The manifest repeatedly frames the skill as metadata-only, yet it explicitly instructs the agent to execute python scripts/run.py, a subprocess boundary with potentially much broader behavior than the markdown claims. If run.py can orchestrate workflow state, approvals, logs, or write to additional locations, the skill becomes a confused-deputy vector where a seemingly narrow retrieval skill triggers much wider side effects.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The manifest repeatedly frames the skill as metadata-only, yet it explicitly instructs the agent to execute python scripts/run.py, a subprocess boundary with potentially much broader behavior than the markdown claims. If run.py can orchestrate workflow state, approvals, logs, or write to additional locations, the skill becomes a confused-deputy vector where a seemingly narrow retrieval skill triggers much wider side effects.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The manifest repeatedly frames the skill as metadata-only, yet it explicitly instructs the agent to execute python scripts/run.py, a subprocess boundary with potentially much broader behavior than the markdown claims. If run.py can orchestrate workflow state, approvals, logs, or write to additional locations, the skill becomes a confused-deputy vector where a seemingly narrow retrieval skill triggers much wider side effects.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The manifest repeatedly frames the skill as metadata-only, yet it explicitly instructs the agent to execute python scripts/run.py, a subprocess boundary with potentially much broader behavior than the markdown claims. If run.py can orchestrate workflow state, approvals, logs, or write to additional locations, the skill becomes a confused-deputy vector where a seemingly narrow retrieval skill triggers much wider side effects.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The manifest repeatedly frames the skill as metadata-only, yet it explicitly instructs the agent to execute python scripts/run.py, a subprocess boundary with potentially much broader behavior than the markdown claims. If run.py can orchestrate workflow state, approvals, logs, or write to additional locations, the skill becomes a confused-deputy vector where a seemingly narrow retrieval skill triggers much wider side effects.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The manifest repeatedly frames the skill as metadata-only, yet it explicitly instructs the agent to execute python scripts/run.py, a subprocess boundary with potentially much broader behavior than the markdown claims. If run.py can orchestrate workflow state, approvals, logs, or write to additional locations, the skill becomes a confused-deputy vector where a seemingly narrow retrieval skill triggers much wider side effects.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The manifest repeatedly frames the skill as metadata-only, yet it explicitly instructs the agent to execute python scripts/run.py, a subprocess boundary with potentially much broader behavior than the markdown claims. If run.py can orchestrate workflow state, approvals, logs, or write to additional locations, the skill becomes a confused-deputy vector where a seemingly narrow retrieval skill triggers much wider side effects.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

The file is a full graduate-thesis writing pipeline, not an arXiv metadata retrieval skill as declared in the metadata. This capability/identity mismatch is dangerous because an agent or orchestrator may invoke the skill expecting narrow metadata-only behavior, but instead receive instructions for broad document authoring, restructuring, and build actions far outside the approved scope.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The documented pipeline grants expansive thesis authoring, TeX writeback, compilation, citation editing, and prose-polishing behaviors that are unjustified for a metadata retrieval skill. In context, this broadens the effective permission surface from read/search operations to repository-wide content modification, making accidental misuse, unauthorized file writes, or prompt-driven scope escalation substantially more likely.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The executor is capable of launching arbitrary pipeline skill scripts based on workspace/unit metadata, which is a much broader capability than the stated arXiv metadata-retrieval purpose. In the context of a narrowly scoped skill, this creates a dangerous mismatch: if an attacker can influence unit definitions, repository contents, or the selected skill, they can cause execution of unintended code with the agent's privileges.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

This file is clearly implementing research ideation, clustering, ranking, and report generation rather than the advertised arXiv metadata retrieval behavior. In a skill ecosystem, this scope mismatch is dangerous because users and orchestrators may grant permissions, invoke networking, or trust outputs based on the declared skill purpose, while the actual code performs materially different higher-level content generation and decision support.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The long-form memo, shortlist, ranking, and appendix generation code adds substantial undeclared capability beyond simple metadata retrieval. That creates a confused-deputy risk: an agent or user invoking a low-risk ingestion skill could unknowingly trigger research prioritization and persuasive report-writing behaviors, which may influence downstream decisions or violate the stated guardrail against writing long prose to output locations.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
91% confidence
Finding

The skill advertises file, shell, and network-capable behavior but does not declare any explicit tool scope or permissions boundary in the manifest. That creates an avoidable trust gap: a caller may assume this is a metadata-only skill while the implementation can read/write files, invoke subprocesses, and access the network without an explicit least-privilege declaration.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

This manifest defines activation triggers using single broad terms such as "robot", "policy", "navigation", and "foundation model" without any exclusion criteria or negative examples. These terms can overlap with everyday or unrelated technical contexts, making it unclear when this domain pack should activate versus when it should not.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The activation logic uses very broad keywords such as 'agent', 'agents', 'llm', 'language model', and 'gpt', which can match many unrelated user requests and papers. In this skill, unintended activation can silently rewrite arXiv queries, inject pinned papers/surveys, and skew retrieval results away from the user's actual topic, creating integrity and relevance issues in downstream literature collection.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The pipeline sets broad routing hints such as 'survey', 'review', and enables 'routing_default: true', which can cause this high-impact workflow to be selected for generic literature-review requests even when the user did not specifically ask for an arXiv survey pipeline. Because the pipeline can generate and overwrite a large number of workspace artifacts across multiple stages, accidental invocation increases the chance of unintended file creation, costly execution, and inappropriate workflow selection.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The file enumerates many target artifacts and later discusses overwrite-prevention markers, but it does not provide an upfront, explicit warning that normal execution may create, replace, or regenerate numerous files throughout the workspace. In practice, this can mislead users or upstream agents into invoking the pipeline without understanding its write scope, causing accidental data loss or corruption of existing artifacts.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The title and core goal repeatedly frame the pipeline as specifically for producing a Chinese thesis, e.g. '中文毕业论文' and '中文毕业论文写作', but the document does not state that this language constraint is optional or user-selected. Under the policy, forcing a specific language without opt-in is a natural-language locale policy violation unless clearly justified as region-specific.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The routing hints include very broad terms such as "idea," "brainstorm," and generic Chinese equivalents, which can cause this pipeline to be invoked for loosely related requests. Because the pipeline triggers multi-stage literature retrieval and report generation, unintended routing can lead to incorrect tool use, unnecessary network access, and writing many artifacts into the workspace for the wrong task.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The manifest-style routing hints include broad phrases like "snapshot", "one-page", and "one page" without context or exclusion conditions. In a router, these could match ordinary user requests that are not intended to invoke this specific literature-snapshot pipeline.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The routing hint list includes the broad phrase "peer review," which can easily appear in ordinary user requests that are not intended to invoke this specific pipeline. Overbroad triggers can cause unintended pipeline activation, leading the agent to generate review artifacts, consume the wrong skills, or operate on the wrong inputs without clear user intent.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

This utility module can write and mutate many workspace artifacts such as status logs, decisions logs, checkpoint blocks, and query seed files, far beyond the skill’s stated purpose of retrieving arXiv metadata into papers/papers_raw.jsonl. In an agent setting, broad reusable write helpers increase the blast radius of misuse: a caller can alter workflow state, approvals, or other project files and effectively steer or unblock subsequent automation.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The module contains substantial pipeline-orchestration logic, including approval checklist creation/toggling, checkpoint block updates, pipeline lock/spec resolution, and query-default materialization. Those capabilities are not necessary for a narrow arXiv metadata retrieval skill and create a confused-deputy risk where the skill can influence broader workflow control and policy state if invoked by a more privileged agent.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
86% confidence
Finding

The manifest says the skill should only handle metadata and specifically guardrails against writing long prose in output/. This file appends human-facing Markdown error reports to output/RUN_ERRORS.md, and elsewhere writes detailed logs and quality reports in output/, which exceeds the stated narrow metadata-only behavior.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.