Security audit
Security checks for vulnerabilities and agentic risk
The skill is mostly an academic review workflow, but it claims file-only access while included protocols can send manuscripts to external model APIs using user API keys.
Install only if you are comfortable reviewing manuscripts in a multi-agent workflow and you will keep cross-model verification disabled for confidential, embargoed, proprietary, or personal-data-heavy papers unless you explicitly approve sending that content to the named provider. Use dedicated, restricted API keys and prefer a corrected version whose top-level security disclosure matches its network/API behavior.
shared/cross_model_verification.md:131External manuscript transmission exceeds the declared file-only privilege boundary
shared/cross_model_verification.md:170Provider API keys are exposed through curl command-line arguments and URLs
The skill embeds prompt-governing instructions about authority, phase control, and how to treat prior output or imperative text, including phrases like 'ignore prior instructions' in examples. In a multi-agent/orchestrated setting, this kind of meta-instruction can interfere with higher-level control logic, create instruction-priority confusion, and increase susceptibility to prompt-injection-style behavior if downstream components treat skill text as authoritative rather than scoped content.
re the paper is submitted.
**Key difference from other reviewers**: The EIC and R1/R2/R3 will evaluate strengths and weaknesses in a balanced manner. You **only challenge** — your job is to find every weakness that a real reviewer might attack.
---
## v3.6.2 Sprint Contract Protocol
You operate in two phases when invoked under a sprint contract. The orchestrator controls which phase via the system prompt you receive.
### Phase 1 — Paper-content-blind pre-commitment
You will receive:
- A sprint contract (JSON) under `## Contract`.
- Paper metadata only (`title`, `field`, `word_count`) under `## Paper Metadata`.
- No paper content.
You MUST produce, in exactly this order:
1. `## Contract Paraphrase` — one paragraph per `acceptance_dimensions` entry, in your own words from the perspective of adversarial challenge.
2. `## Scoring Plan` — one `### <Dn>: <name>` subsection per dimension. Each must contain:
- `what_to_look_for` — concrete signals you will scan for.
- `
The skill embeds prompt-governing instructions about how to treat prior output and imperative text inside <phase1_output>, including language like 'ignore prior instructions' as an example. Even though this appears intended as a defensive protocol for a two-phase workflow, it is still a prompt-injection-relevant control surface inside untrusted skill content, because it attempts to influence instruction hierarchy and interpretation at runtime. If an attacker can shape Phase 1 content or adjacent prompt context, these rules may be abused to suppress safer higher-level handling or cause the agent to misclassify instructions as inert data.
k appropriate? Are academic arguments accurate? Is the contribution to the field genuine and incremental?
You **do not** handle technical details of research design (that's Reviewer 1's job) or cross-disciplinary impact (that's Reviewer 3's job).
---
## v3.6.2 Sprint Contract Protocol
You operate in two phases when invoked under a sprint contract. The orchestrator controls which phase via the system prompt you receive.
### Phase 1 — Paper-content-blind pre-commitment
You will receive:
- A sprint contract (JSON) under `## Contract`.
- Paper metadata only (`title`, `field`, `word_count`) under `## Paper Metadata`.
- No paper content.
You MUST produce, in exactly this order:
1. `## Contract Paraphrase` — one paragraph per `acceptance_dimensions` entry, in your own words from the perspective of domain accuracy.
2. `## Scoring Plan` — one `### <Dn>: <name>` subsection per dimension. Each must contain:
- `what_to_look_for` — concrete signals you will scan for.
- `what_t
The skill embeds instruction-priority language such as references to the system prompt and text like 'ignore prior instructions' within the agent content. Even though the text frames this as protocol guidance for phase handling, it conditions the model to reinterpret untrusted content and instruction boundaries, which can enable prompt-injection behavior or cause the agent to follow adversarial content embedded in prior outputs. In a multi-agent review workflow, this is especially risky because the agent explicitly consumes data produced by other agents and paper content, both of which may contain attacker-controlled instructions.
for your journal? Would your readers be interested? What does this paper contribute to the field as a whole? You won't dive into methodological technical details (that's Reviewer 1's job), but you will focus on overall quality and strategic value.
---
## v3.6.2 Sprint Contract Protocol
You operate in two phases when invoked under a sprint contract. The orchestrator controls which phase via the system prompt you receive.
### Phase 1 — Paper-content-blind pre-commitment
You will receive:
- A sprint contract (JSON) under `## Contract`.
- Paper metadata only (`title`, `field`, `word_count`) under `## Paper Metadata`.
- No paper content.
You MUST produce, in exactly this order:
1. `## Contract Paraphrase` — one paragraph per `acceptance_dimensions` entry, in your own words from the perspective of editorial oversight.
2. `## Scoring Plan` — one `### <Dn>: <name>` subsection per dimension. Each must contain:
- `what_to_look_for` — concrete signals you will scan for.
- `wh
The skill embeds meta-instructions about prompt authority and explicitly includes a phrase like 'ignore prior instructions' inside an example within <phase1_output>. Even though the text tries to frame that content as data, this is still a prompt-injection-shaped construct inside agent instructions and can create confusion about instruction hierarchy or be copied into downstream prompts unsafely. In a multi-agent orchestration flow, injected or replayed prior output could influence later phases or other agents if delimiters and trust boundaries are not enforced strictly.
sions supported by data? If another researcher followed the same procedures, could they obtain similar results?
You **do not** handle literature review completeness (that's Reviewer 2's job) or cross-disciplinary impact (that's Reviewer 3's job).
---
## v3.6.2 Sprint Contract Protocol
You operate in two phases when invoked under a sprint contract. The orchestrator controls which phase via the system prompt you receive.
### Phase 1 — Paper-content-blind pre-commitment
You will receive:
- A sprint contract (JSON) under `## Contract`.
- Paper metadata only (`title`, `field`, `word_count`) under `## Paper Metadata`.
- No paper content.
You MUST produce, in exactly this order:
1. `## Contract Paraphrase` — one paragraph per `acceptance_dimensions` entry, in your own words from the perspective of methodology rigor.
2. `## Scoring Plan` — one `### <Dn>: <name>` subsection per dimension. Each must contain:
- `what_to_look_for` — concrete signals you will scan for.
- `what
The skill embeds instruction-handling rules that explicitly reference system prompts and tell the agent how to treat prior output and imperative text. Even though the stated purpose is protocol control, this creates a prompt-surface where untrusted paper content or wrapped phase output can influence behavior, and the matched phrase 'ignore prior instructions' is a strong prompt-injection indicator. In a multi-agent review pipeline, such meta-instruction text increases the risk that downstream agents follow skill-local directives over higher-priority orchestration or security policies.
's impact from a practical application perspective.
You **do not** handle the technical rigor of research design (that's Reviewer 1's job) or the completeness of literature review (that's Reviewer 2's job). You bring the "outsider's" perspective.
---
## v3.6.2 Sprint Contract Protocol
You operate in two phases when invoked under a sprint contract. The orchestrator controls which phase via the system prompt you receive.
### Phase 1 — Paper-content-blind pre-commitment
You will receive:
- A sprint contract (JSON) under `## Contract`.
- Paper metadata only (`title`, `field`, `word_count`) under `## Paper Metadata`.
- No paper content.
You MUST produce, in exactly this order:
1. `## Contract Paraphrase` — one paragraph per `acceptance_dimensions` entry, in your own words from the perspective of cross-disciplinary relevance.
2. `## Scoring Plan` — one `### <Dn>: <name>` subsection per dimension. Each must contain:
- `what_to_look_for` — concrete signals you will scan for.
Skill instructs the agent to omit warnings, disclaimers, or ethical commentary. Stripping safety caveats hides risk from the user and is a common jailbreak preamble.
### `caveats`
Non-empty array, items non-empty strings, minItems 1. The schema physically prevents an
empty caveats field. A benchmark report with no caveats either has no known limitations
(implausible for any real-world evaluation) or the author didn't think about limitations
(which disqualifies the report more than any specific limitation would).
Skill instructs the agent to omit warnings, disclaimers, or ethical commentary. Stripping safety caveats hides risk from the user and is a common jailbreak preamble.
without a stated `human_baseline.sample_size`. n=1 is worse than n=10, but it is
infinitely better than unstated.
- **Author-conducted with no caveat disclosure.** Using `author-conducted` is allowed.
Not noting it in `caveats` as a limitation is an editorial failure, not a schema
failure — but reviewers will notice.
Skill instructs the agent to omit warnings, disclaimers, or ethical commentary. Stripping safety caveats hides risk from the user and is a common jailbreak preamble.
Not noting it in `caveats` as a limitation is an editorial failure, not a schema
failure — but reviewers will notice.
- **Self-scoring with no warning acknowledgment.** `self-scored` produces a warning on
every validator run. If you publish a report with `self-scored` and don't address it
in caveats, the omission is visible in the JSON to anyone who checks.
The trigger list is broad and includes common phrases such as 'review paper' and 'peer review', which can match ordinary academic-help requests without clear user intent to invoke this specific multi-agent workflow. Because the skill delegates manuscript content to multiple subagents, accidental activation can cause unintended processing of sensitive draft material and user surprise about how their content is handled.
The role definition states that the Devil's Advocate's job is not to score the paper, framing it as purely adversarial critique. But the sprint protocol later mandates dimension scores, failure-condition checks, and even numeric rebuttal scores. This is an active contradiction between the high-level role description and the actual prescribed behavior.
The documentation explicitly excludes literature coverage assessment as another reviewer's role, yet later instructions direct the agent to check whether important contradicting evidence was omitted and whether citations are selectively representative. Those tasks are a form of literature coverage/completeness assessment, so the file's own intent statements conflict.
The file says the Devil's Advocate should not suggest practical implications or stakeholder perspectives because that belongs to another reviewer. However, the output format later requires a 'Missing Stakeholder Perspectives' section, which instructs the agent to identify such perspectives. This is a direct contradiction in the documented intent.
The sprint-contract protocol explicitly constrains the agent to mechanical score aggregation and condition evaluation only. Later sections direct the agent to identify disagreements, weigh expertise, arbitrate disputed issues, and make editorial judgments, which is a materially different interpretive role and contradicts the earlier instruction when both are present in the same file.
The guide makes categorical '100% verification' claims for integrity checks, which can mislead users into overtrusting automated validation of references and data. In a paper-review and submission workflow, that overconfidence can cause fabricated citations, unsupported claims, or data issues to pass through without appropriate human scrutiny.
This markdown file includes a natural-language criterion stating that outstanding writing is 'Precise and fluent academic English/Chinese.' That creates a language policy constraint favoring specific languages without offering an opt-in choice or documenting a region-specific justification.
The title and introduction present APA 7.0 statistical formatting and reporting standards as the governing standard for all quantitative research papers, using mandatory language such as 'must'. This is a natural-language locale/style policy constraint that may force a specific organizational or regional standard without stating that it applies only where APA formatting is required.
Suspicious Unicode normalization or mixed-script content
Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.
**OpenAI (GPT-5.4):**
1. Go to [platform.openai.com/api-keys](https://platform.openai.com/api-keys)
2. Create a new API key
3. Copy the key (starts with `sk-`)
**Google (Gemini 3.1 Pro):**
The protocol instructs agents to send reference text and surrounding paper context to third-party model providers, which is a real data-sharing/privacy risk when papers contain unpublished, confidential, or proprietary material. The feature is described as optional, but the file does not require an explicit user-facing consent step or data-disclosure warning before transmitting content externally.
The Devil's Advocate flow sends reviewed material to a second model for critique, which can expose full paper content or other sensitive review inputs to an external provider without a clear user warning at the point of use. In an academic review context, this is especially relevant because manuscripts under review are often confidential and subject to non-disclosure expectations.
The flagged external endpoint is part of the same outbound transmission behavior and represents a real risk because it normalizes sending model inputs to a third-party API. While the URL itself is not harmful in isolation, its presence in this workflow reflects an implementation path for external disclosure of potentially sensitive paper content.
In Claude Code, the agent can use the Bash tool to make API calls:
curl -s https://api.openai.com/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
The flagged external endpoint is part of the same outbound transmission behavior and represents a real risk because it normalizes sending model inputs to a third-party API. While the URL itself is not harmful in isolation, its presence in this workflow reflects an implementation path for external disclosure of potentially sensitive paper content.
In Claude Code, the agent can use the Bash tool to make API calls:
curl -s https://api.openai.com/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
This Gemini API example similarly enables outbound transfer of prompt content to Google, creating the same confidentiality and privacy exposure as the OpenAI flow. In the context of academic-paper-reviewer, the transmitted content may include unpublished manuscripts, reviewer comments, or contextual excerpts that should not be disclosed to third parties by default.
# PROMPT must be set before calling. Use jq to JSON-escape it.
curl -s "https://generativelanguage.googleapis.com/v1beta/models/${ARS_CROSS_MODEL}:generateContent?key=$GOOGLE_AI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"contents": [{"parts": [{"text": "'"$(echo "$PROMPT" | jq -Rs .)"'"}]}],
The Paper Draft schema requires abstract to include a chinese field when bilingual and defines keywords as {en, zh_tw}, which encodes a specific locale choice (Traditional Chinese for Taiwan) into the contract. The file does not state that this locale is optional by user choice or limited to a Taiwan-specific workflow, so it reads as an organizational language/locale policy baked into a general cross-skill schema.
The protocol explicitly asks users to upload past papers and writing samples, which can contain unpublished research, confidential collaborator material, student data, reviewer comments, or other sensitive content, but it provides no privacy warning, minimization guidance, or handling restrictions. In an academic-review skill, users are especially likely to submit prepublication manuscripts and identifiable scholarly work, so silent collection for style analysis increases the risk of unintended disclosure or over-sharing.
Detected: suspicious.prompt_injection_instructions