Security audit

ars-academic-reviewer

Security checks for vulnerabilities and agentic risk

Overview

This is a complex but coherent academic peer-review skill that reads manuscripts and can optionally use external models with consent; no malicious behavior was found.

Install only if you are comfortable giving the skill access to unpublished manuscripts and review materials. Do not enable cross-model verification or provide API keys unless you explicitly want manuscript content sent to the named external provider; keep it disabled for confidential or embargoed work unless you have authorization.

SkillSpector

Vulnerability Patterns

Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
YARA SignaturesMalware Match, Webshell Match, Cryptominer Match
MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection

Findings (23)

Description-Behavior Mismatch

Medium

Confidence: 93% confidence
Finding: The file declares the agent as single-phase and Phase-1-only, but later embeds full Phase 2, re-review, and cross-model operating instructions. This creates authority ambiguity that can cause the agent or orchestrator to follow the wrong contract, expand scope unexpectedly, or bypass phase-isolation assumptions relied on by the surrounding skill.

Intent-Code Divergence

High

Confidence: 89% confidence
Finding: The example output format conflicts with the enforced Phase 2 grammar, giving the model multiple incompatible schemas to obey. In prompt-driven agents, contradictory format authority is dangerous because it increases failure rates, parser breakage, and exploitable confusion where downstream systems may accept malformed or unintended outputs.

Intent-Code Divergence

Medium

Confidence: 94% confidence
Finding: The skill’s output template conflicts with earlier rules that prohibit inferring or substituting a target venue when no author-confirmed target exists. That contradiction can drive the agent to invent journal tiers and recommendations from model priors, increasing hallucination risk and causing misleading review framing or unauthorized steering of the user’s submission strategy.

Intent-Code Divergence

Low

Confidence: 83% confidence
Finding: The examples normalize naming specific journals for reviewer configuration even though the protocol elsewhere says not to select or substitute a target journal absent author-confirmed metadata. In practice, examples are strong behavioral priors for LLMs, so this inconsistency can cause the model to ignore the safer rule and fabricate venue-specific fit judgments.

Vague Triggers

Medium

Confidence: 91% confidence
Finding: The README advertises broad natural-language triggers such as 'review paper', 'critique paper', and multilingual equivalents, with no clear scoping or confirmation step. In agent gateways that auto-match skills from user phrasing, this can cause unintended invocation on ordinary conversation or sensitive documents, increasing the chance of accidental data exposure or inappropriate routing.

Missing User Warnings

Medium

Confidence: 89% confidence
Finding: The skill states it performs environment self-checks and can adapt based on capabilities including file reading and OCR, but the README does not clearly disclose what user content may be accessed, inspected, or retained during those checks. This creates a privacy and transparency risk because users may provide unpublished manuscripts or reviewer materials without understanding that the skill may read files or probe runtime capabilities.

Vague Triggers

Medium

Confidence: 78% confidence
Finding: The trigger list is extremely broad and includes common phrases like 'review paper,' 'critique paper,' and 'simulate review,' which can plausibly match ordinary requests for feedback rather than an explicit invocation of this high-complexity skill. Over-broad activation increases the chance of unintended routing, causing the agent to enter a heavyweight multi-agent workflow and potentially process user content under assumptions the user did not intend.

Vague Triggers

Low

Confidence: 74% confidence
Finding: The quick-start example, 'Review this paper,' is generic enough to overlap with many normal user interactions requesting informal feedback. In a routing system that relies on trigger phrases, such a generic example can contribute to accidental activation and unnecessary exposure of user content to a more complex orchestration path.

Vague Triggers

Medium

Confidence: 80% confidence
Finding: The manifest description is broad enough to trigger on loosely related requests, which can cause the skill to activate outside the user’s intended scope and apply heavy review-specific instructions to general academic conversations. Overbroad invocation increases unintended data exposure to the skill and raises the chance of irrelevant or policy-conflicting behavior.

Natural-Language Policy Violations

Medium

Confidence: 88% confidence
Finding: The skill imposes review-language behavior based on detected paper language, including a default toward English for some languages, without confirming user preference. That can override user intent, mishandle multilingual or sensitive content, and create accessibility, privacy, or accuracy issues if the user expected the review in another language.

External Transmission

Medium

Category: Data Exfiltration
Content: endpoint="${ARS_OPENAI_COMPAT_BASE_URL%/}/chat/completions" GUARD=the cross-model verification module resp="$(curl -sS -w '\n%{http_code}' "$endpoint" \ -H "Authorization: Bearer $ARS_OPENAI_COMPAT_API_KEY" \ -H "Content-Type: application/json" \ -d "$(jq -n --arg model "$ARS_CROSS_MODEL" --arg prompt "$PROMPT" '{
Confidence: 90% confidence
Finding: This skill explicitly sends model prompts and possibly manuscript/review content to external provider endpoints via curl. Although the document includes a consent boundary, the feature still creates a real data-exfiltration surface if downstream agents mis-handle consent checks, send more content than intended, or users do not appreciate the sensitivity of unpublished academic material.

Hidden Instructions

High

Category: Prompt Injection
Content: ## v3.6.2 Sprint Contract Protocol  You operate in two phases when invoked under a sprint contract. The orchestrator controls which phase via the system prompt you receive.
Confidence: 91% confidence
Finding: The file embeds hidden instructions in HTML comments and also contains extensive prompt-authority language telling the model how to interpret later inputs. Hidden or non-obvious instruction channels are dangerous because they can evade review, create split authority, and increase the chance that the model follows concealed guidance rather than visible policy.

Hidden Instructions

High

Category: Prompt Injection
Content: ## v3.6.2 Sprint Contract Protocol  You operate in two phases when invoked under a sprint contract. The orchestrator controls which phase via the system prompt you receive.
Confidence: 88% confidence
Finding: The skill embeds operative instructions inside HTML comments, which are hidden from normal readers but still available in the prompt text. Hidden instructions reduce auditability and can conceal behavior-shaping rules from reviewers, increasing the risk that unsafe or policy-conflicting logic is overlooked.

Hidden Instructions

High

Category: Prompt Injection
Content: - Quick scan of title, abstract, conclusion - Assessment: Is this topic timely? Does it fit the journal scope? - No numeric score is recorded at this step — the first-impression scan produces no numeric output; scoring happens downstream per the active mode's output contract
Confidence: 85% confidence
Finding: This HTML comment contains prompt-relevant behavioral guidance about scoring semantics. Even if not overtly malicious, embedding such instructions in comments creates a covert control channel that complicates security review and may allow future unsafe directives to be hidden the same way.

Hidden Instructions

High

Category: Prompt Injection
Content: orchestrator that runs the deterministic calculator between Phase 1 and Phase 2; a dispatcher that does not is unaffected.  You will receive: - A sprint contract (JSON) under `## Contract`.
Confidence: 88% confidence
Finding: The file embeds large operational prompt fragments inside HTML comments as canonical instruction bodies. Even if intended for synchronization and templating, hidden instruction channels increase the chance that downstream tooling or models treat non-visible content as authoritative, creating prompt-injection ambiguity and audit gaps.

Hidden Instructions

High

Category: Prompt Injection
Content:   You will receive: - The same sprint contract.
Confidence: 88% confidence
Finding: This hidden commented block contains substantial reviewer instructions for Phase 2. Storing executable prompt content in comment-delimited regions is dangerous because some renderers, preprocessors, or LLM ingestion paths may still expose or prioritize it, enabling hidden-instruction behavior that is difficult to audit.

Hidden Instructions

High

Category: Prompt Injection
Content: ---
Confidence: 87% confidence
Finding: The methodology receipt instructions are hidden inside comment markers, continuing the same hidden-control-channel pattern. This creates a security-relevant ambiguity between human-visible and model-visible instructions and can be exploited if an attacker can influence which sections are extracted or rendered.

Hidden Instructions

High

Category: Prompt Injection
Content:   You will receive: - The same sprint contract.
Confidence: 88% confidence
Finding: The DA Phase 2 prompt is embedded as hidden comment content. Because this skill is fundamentally prompt-driven, hidden prompt regions materially increase the risk of covert instruction injection, parser inconsistencies, and reviewer behavior that differs from what a human auditor sees.

Hidden Instructions

High

Category: Prompt Injection
Content: ---
Confidence: 88% confidence
Finding: The synthesizer instructions are likewise hidden in comment blocks, which preserves a covert instruction surface across the entire review pipeline. Hidden orchestration logic in prompt-bearing assets undermines transparency and makes malicious modifications harder to detect during review.

Hidden Instructions

High

Category: Prompt Injection
Content: ## v3.6.2 Sprint Contract Protocol  You operate in two phases when invoked under a sprint contract. The orchestrator controls which phase via the system prompt you receive.
Confidence: 76% confidence
Finding: The HTML comment embeds hidden prompt material references inside the skill file, which creates a covert instruction channel that may not be visible in normal rendering or review workflows. Hidden instructions are dangerous because they can evade human inspection and complicate assurance that the executed behavior matches the reviewed prompt.

Hidden Instructions

High

Category: Prompt Injection
Content: ## v3.6.2 Sprint Contract Protocol  You operate in two phases when invoked under a sprint contract. The orchestrator controls which phase via the system prompt you receive.
Confidence: 87% confidence
Finding: The HTML comment embeds prompt-relevant metadata that is invisible in normal rendered Markdown but still present in the raw prompt supplied to a model. Hidden instructions or hidden control-plane annotations increase the risk of reviewers and tooling missing behavior-affecting content, and this same mechanism can be abused for stealth prompt injection in future edits.

YARA rule 'agent_skill_prompt_injection_hidden_instructions': Prompt injection or hidden instructions embedded in AI agent skill text [agent_skills]

High

Category: YARA Match
Content: You operate in two phases when invoked under a sprint contract. The orchestrator controls which phase via the system prompt you receive. ### Phase 1 — Paper-content-blind pre-commitment You will receive: - A sprint contract (JSON) under `## Contract`.
Confidence: 95% confidence
Finding: The prompt contains classic prompt-injection-sensitive language about system prompts, treating embedded content as instructions/data, and handling 'ignore prior instructions' strings. Even though some of this is defensive, the density of instruction-authority meta-language materially increases prompt complexity and the risk of ambiguous precedence, hidden control flow, or exploitable parser/orchestrator confusion.

YARA rule 'agent_skill_mcp_tool_poisoning_metadata': MCP/tool metadata poisoning indicators in tool schemas or skill manifests [agent_skills]

High

Category: YARA Match
Content: "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://github.com/Imbad0202/academic-research-skills/shared/contracts/evidence/evidence_row_v1_2.schema.json", "title": "ARS Shared Evidence Row v1.2 — cross-document consistency", "description": "Closed observation-level evidence-row/1.2 carrier for the cross-document consistency advisory. One row integrity-binds the complete ordered bilateral or preregistration three-witness evidence set. Evidence and checked-scope states are caller-supplied semantic observations whose bytes are replayed; they are not agreement, completeness, readiness, acceptance, or authorization findings.", "type": "object", "additionalProperties": false, "required": [
Confidence: 93% confidence
Finding: The schema contains bidirectional/zero-width Unicode control characters in a regex meant to block unsafe path content. These invisible characters can poison metadata review, mislead human auditors, or create parser/display discrepancies where malicious filenames or schema text appear different from what is actually validated. In a security-sensitive agent skill, hidden Unicode in validation rules increases the risk of confused-deputy and review-evasion issues.

Static analysis

Detected: suspicious.prompt_injection_instructions

Prompt-injection style instruction pattern detected.

Warn

Code: suspicious.prompt_injection_instructions
Location: references/devils_advocate_reviewer_agent.md:111

Prompt-injection style instruction pattern detected.

Warn

Code: suspicious.prompt_injection_instructions
Location: references/domain_reviewer_agent.md:110

Prompt-injection style instruction pattern detected.

Warn

Code: suspicious.prompt_injection_instructions
Location: references/eic_agent.md:109

Prompt-injection style instruction pattern detected.

Warn

Code: suspicious.prompt_injection_instructions
Location: references/guides/reviewer_sprint_prompt_source.md:117

Prompt-injection style instruction pattern detected.

Warn

Code: suspicious.prompt_injection_instructions
Location: references/guides/sprint_contract_protocol.md:26

Prompt-injection style instruction pattern detected.

Warn

Code: suspicious.prompt_injection_instructions
Location: references/methodology_reviewer_agent.md:104

Prompt-injection style instruction pattern detected.

Warn

Code: suspicious.prompt_injection_instructions
Location: references/perspective_reviewer_agent.md:110