Security audit

academic-pipeline-v1

Security checks for vulnerabilities and agentic risk

Overview

The skill is a coherent academic-paper workflow, but it needs user review because it can create persistent process records from conversation history, optionally send reference data to another model, and has inconsistent integrity-gate rules.

Review before installing if your drafts, reviewer comments, references, or conversation history are confidential. Disable or avoid cross-model verification unless you are comfortable sharing reference data with another model, and treat the generated process-record PDFs as sensitive because they may include verbatim user text and behavioral scoring. Do not rely on the pipeline's integrity gate as a guarantee if you choose to continue after failed verification.

A.I.G

Vulnerability Patterns

Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs

SkillSpector

Vulnerability Patterns

Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
System Prompt LeakageDirect Leakage, Indirect Extraction, Tool-Based Exfiltration
Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger

Findings (31)

Anti-Refusal Statement

High

Category: Anti-Refusal
Content: - "skip" -> validate skip safety; proceed if allowed
Confidence: 80% confidence
Finding: Skill instructs the agent to omit warnings, disclaimers, or ethical commentary. Stripping safety caveats hides risk from the user and is a common jailbreak preamble.

Direct Prompt Extraction

High

Category: System Prompt Leakage
Content: --- ## Dashboard Output Rules 1. Produce full version when user explicitly requests it 2. **Append simplified version to checkpoint notification after each stage completion**
Confidence: 85% confidence
Finding: Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Intent-Code Divergence

High

Confidence: 97% confidence
Finding: The document states integrity checks cannot be skipped, but the exception-handling section creates an effective bypass by allowing progression with a 'partially unverified' warning. That inconsistency is security-relevant because operators and downstream stages may rely on the earlier mandatory-check guarantee while the actual workflow permits unverified material through.

Description-Behavior Mismatch

High

Confidence: 98% confidence
Finding: The state machine explicitly allows users to continue to the next stage after Stage 2.5 or 4.5 exceeds three failed integrity-verification rounds, which contradicts the document's own claim that these checks are mandatory quality gates. In an academic-paper pipeline, this defeats the trust boundary around citation/integrity validation and can result in unverifiable or fabricated content being advanced as if it passed required controls.

Intent-Code Divergence

Medium

Confidence: 97% confidence
Finding: The handoff mappings in the transition section contradict the stage table and state machine, including off-by-one stage labels and mismatched source/target descriptions. In an orchestration skill, these inconsistencies can cause the wrong downstream skill or artifact to be invoked, which may bypass intended review/integrity gates or move unverified content into later stages.

Intent-Code Divergence

Medium

Confidence: 90% confidence
Finding: Line L005 states the agent 'only perform[s] factual verification,' but later sections instruct it to classify paraphrase quality, verbatim plagiarism, self-plagiarism, cherry-picking, and claim distortion severity. Those later tasks require interpretive assessment of writing and argument use, which goes beyond the earlier stated limitation.

Vague Triggers

Medium

Confidence: 83% confidence
Finding: This markdown skill specifies when in the pipeline the agent is used ('deep-research Phase 5' and 'pipeline Stage 2.5 + Stage 4.5') but does not clearly define how the skill is invoked or what exact trigger phrases/conditions should activate it. Without explicit trigger scope or exclusion conditions, the skill's activation remains somewhat ambiguous in a manifest-like operational sense.

Description-Behavior Mismatch

Medium

Confidence: 92% confidence
Finding: The manifest describes this skill as a mandatory integrity verification stage within an academic paper workflow, which naturally covers reference and citation verification. However, this file expands the role into Phase D originality/plagiarism screening and Phase E end-to-end factual/quantitative claim verification, which are materially broader review capabilities than a narrow integrity gatekeeper for references, citations, and data consistency.

Missing User Warnings

Medium

Confidence: 95% confidence
Finding: The optional cross-model verification transmits reference data to a second model without any explicit user notice, consent, or data-handling boundary. In an academic pipeline, references, drafts, and associated metadata may contain unpublished research topics, sensitive citations, or proprietary material, so silent third-party sharing creates a real confidentiality and compliance risk.

Intent-Code Divergence

Medium

Confidence: 89% confidence
Finding: The documentation explicitly forbids quality judgments at L477, yet the protocol directs the agent to assess cherry-picking, citation-context deviation, paraphrase closeness, boilerplate writing, and self-plagiarism. These are not limited to binary factual verification and contradict the stated 'only factual verification' boundary.

Description-Behavior Mismatch

Medium

Confidence: 89% confidence
Finding: The role definition says the agent coordinates handoff between three skills and one internal verifier and explicitly says it does not perform substantive work. Later sections add transition-time Socratic coaching, direct artifact production instructions like auto-producing MD/DOCX/PDF, and handling a Stage 5 finalize flow, which goes beyond pure dispatch/checkpoint management. This is a semantic mismatch with the declared limited orchestrator role.

Vague Triggers

Medium

Confidence: 94% confidence
Finding: The intent-detection keywords are broad and overlap with ordinary user language, which can cause the orchestrator to dispatch into the wrong stage or invoke powerful downstream skills without sufficient confirmation. In a multi-stage pipeline with integrity, review, and output-generation steps, misrouting can expose user materials to unintended processing paths and bypass expected review of entry conditions.

Intent-Code Divergence

Medium

Confidence: 95% confidence
Finding: The checkpoint rules define SLIM checkpoints as 'auto-continue in 5 seconds,' and the template explicitly says auto-continuing. This contradicts the later prohibited action stating 'Do not skip checkpoints — User confirmation is required after each stage completion,' because some stages proceed without fresh explicit confirmation at that checkpoint.

Natural-Language Policy Violations

Medium

Confidence: 95% confidence
Finding: The tracked state example hard-codes `"language": "en"`, which expresses a default locale requirement in natural language/config content. The file does not offer a user opt-in, language choice, or a documented region-specific justification for restricting operation to English.

Vague Triggers

Medium

Confidence: 91% confidence
Finding: The example advertises very broad natural-language follow-up triggers such as "revise the paper," "make slides," and "make an interactive webpage." In an orchestrated skill ecosystem, generic phrases can unintentionally invoke this skill or adjacent tools during ordinary conversation, causing incorrect routing, unexpected multi-step execution, or unintended processing of user content without clear confirmation.

Natural-Language Policy Violations

Medium

Confidence: 96% confidence
Finding: The file explicitly states that conversation with the orchestrator is in English, but it does not present this as an option or obtain user opt-in. Under the policy, forcing a specific language without user choice is a natural-language policy violation unless clearly justified as region-specific or optional.

Vague Triggers

Medium

Confidence: 97% confidence
Finding: The trigger language is broad enough that ordinary mentions of reviewer feedback (e.g., "reviewer comments" or "revise and resubmit") could activate the external review workflow without clear user intent. In this academic pipeline, mistaken activation can cause the agent to ingest unstructured external content, shift workflow mode, and apply conservative revision defaults, creating integrity and consent risks even if it is not a classic code-execution issue.

Intent-Code Divergence

Medium

Confidence: 92% confidence
Finding: The capability table states that academic-pipeline cannot perform substantive work and is limited to orchestration. Elsewhere, the document says Stage 4 entry for real reviewer feedback 'needs structured intake + strategic coaching' and that the External Review Protocol provides 'strategic coaching,' which is a substantive advisory function rather than mere dispatching.

Vague Triggers

Medium

Confidence: 95% confidence
Finding: Using broad natural-language triggers like 'pause' or especially 'stop here' for state transitions can cause accidental activation during ordinary conversation, quoted text, or analysis of source material. In a multi-stage orchestrator, unintended pauses can disrupt workflow continuity, create confusing state restoration, or be abused via prompt/content injection to alter execution state without clear user intent.

Missing User Warnings

Medium

Confidence: 92% confidence
Finding: The workflow allows continuation after failed integrity verification with only a relatively soft 'partially unverified' warning, which is insufficient given the pipeline markets mandatory integrity verification and quality gates. Users may over-trust later outputs and publish content containing unverifiable citations or unsupported claims because the risk communication is not strong enough at the point of override.

Description-Behavior Mismatch

Medium

Confidence: 90% confidence
Finding: The manifest describes an academic paper pipeline ending in finalization, with integrity checks, peer review, and humanization. This file defines an additional post-finalization stage that compiles session history, scores the user’s collaboration quality, audits AI behavior, and produces separate Markdown/PDF deliverables, which is materially broader than the stated paper-production workflow.

Context-Inappropriate Capability

Medium

Confidence: 95% confidence
Finding: The protocol instructs the agent to mine broad session history, verbatim user instructions, interventions, and internal-style logs to reconstruct collaboration history beyond what is needed for paper generation. This increases the amount of sensitive data collected and transformed into new artifacts, raising privacy, retention, and secondary-use risks if the report is shared, stored, or exposed.

Missing User Warnings

Medium

Confidence: 96% confidence
Finding: The process record requires writing extensive verbatim user history into Markdown/PDF outputs without warning about privacy or downstream disclosure risks. Converting conversational content into portable documents materially increases exposure because quotes, decisions, and derived behavioral summaries become easy to store, forward, and leak.

Context-Inappropriate Capability

Medium

Confidence: 94% confidence
Finding: The protocol mandates scoring and critiquing the user's performance, which is not necessary to produce an academic paper and creates unnecessary profiling of the user. In context, this can pressure users, generate sensitive behavioral assessments, and normalize retention of judgmental metadata unrelated to the requested deliverable.

Vague Triggers

Medium

Confidence: 94% confidence
Finding: The trigger phrase "status" is extremely broad and likely to collide with normal user conversation, causing the skill to activate unintentionally. In an orchestrator skill that exposes workflow state across multiple stages, accidental invocation can leak research progress or confuse users by interrupting unrelated interactions.

Static analysis

No suspicious patterns detected.