Back to skill

Security audit

is-bullshit

Security checks for vulnerabilities and agentic risk

Overview

This skill is a fact-checking helper, but it can automatically inspect and summarize the whole conversation in ways that may expose prior user content or disrupt normal responses.

Install only if you are comfortable with a fact-checking mode that may review previous conversation turns and add summaries to responses. Keep it disabled unless explicitly needed, and avoid enabling it in chats containing private, confidential, or strict machine-readable output.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Findings (1)

T01 · Skill Instruction Hijacking

Warning
Location
SKILL.md:7
Finding
Mandatory Fact-Check Injection Alters Agent Responses## Vulnerability Details **File Location**: `SKILL.md`, lines 7-8 **Vulnerability Type**: `T01: Skill Instruction Hijacking` **Risk Level**: Medium **Vulnerable Code Snippet**: ```markdown **IMPORTANT**: When config `enable = true`, this skill **MUST automatically show** fact check after **EVERY response**, WITHOUT waiting for user to ask "check" or "检测". ``` ### Technical Analysis The skill uses mandatory language—`MUST`, `EVERY response`, and `WITHOUT waiting for user`—to impose persistent output behavior once enabled. Rather than limiting operation to an explicit fact-check request, it directs the agent to intercept every subsequent response and append content according to the templates defined later in `SKILL.md`. The instructions also direct the skill to examine tool usage throughout the entire conversation history. This behavior can conflict with the current user's requested output format, task objectives, and relevance boundaries. It therefore constitutes instruction hijacking: loading and enabling the skill changes the agent's general response behavior beyond the immediate purpose of a single invocation. The package contains no executable scripts, network retrieval, dependency installation, privilege escalation, persistence mechanism, credential access, or obfuscated payload. In addition, `config.json` sets `enable` to `false`, which limits default exposure but does not eliminate the unsafe behavior available after activation. ### Attack Path 1. A user or another instruction causes the skill to be enabled. 2. The mandatory instruction becomes applicable to all subsequent agent responses. 3. The skill examines the conversation history and recorded tool usage, including material unrelated to the current request. 4. It injects a prescribed fact-check section after every response without requiring a new verification request. 5. The injected section can override or invalidate task-specific formatting and may repeat su ...[truncated 833 chars]
Remediation
## Remediation Suggestions 1. Remove the automatic activation requirement and all language requiring operation after every response. 2. Require an explicit user request for each response that is to be fact-checked. 3. Limit analysis to the specific response under review and only the tool records necessary to verify its claims. 4. State that the fact-check format is optional and subordinate to system instructions, the current user request, and any required machine-readable output contract. 5. Avoid reproducing prior conversation content unless it is strictly necessary and the user has requested it. 6. Replace the vulnerable instruction with bounded wording such as: ```markdown When the user explicitly requests a fact check, evaluate only the specified response. Do not append fact-check content to unrelated responses, and preserve any output format required by higher-priority instructions or the current user request. ``` 7. Add safeguards preventing activation commands embedded in quoted text, retrieved content, or untrusted documents from changing the skill's operating state.
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • System Prompt LeakageDirect Leakage, Indirect Extraction, Tool-Based Exfiltration
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (7)

Vague Triggers

High
Confidence
97% confidence
Finding
The English trigger list includes broad phrases like "check," "verify," and "are you sure," which are common in normal conversation and can unintentionally activate the skill. In this skill’s context, accidental activation is risky because it may cause unnecessary inspection and summarization of prior conversation turns, increasing exposure of user content beyond what was explicitly requested.

Direct Prompt Extraction

High
Category
System Prompt Leakage
Content
- Score: +X / -X
```

### Output Rules by Conversation Length

| Conversation Rounds | Output |
|---------------------|--------|
Confidence
85% confidence
Finding
Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Vague Triggers

Medium
Confidence
95% confidence
Finding
The instruction to automatically show a fact check after every response when enabled is overly broad and lacks scope restrictions. Because the skill also analyzes conversation history, persistent auto-activation can repeatedly process and surface prior content without a fresh, per-turn user request, creating privacy and UX risks.

Ssd 3

Medium
Confidence
98% confidence
Finding
Automatic fact-checking after every response, combined with instructions to inspect conversation history, can expose or re-surface sensitive information from earlier user turns without a specific request at that moment. This creates a clear data minimization problem: the skill is designed to process and potentially display more historical content than is necessary for the immediate interaction.

Ssd 3

Medium
Confidence
98% confidence
Finding
The skill explicitly instructs the model to inspect tools called throughout the entire conversation history, which broadens processing scope beyond the current exchange. In practice, this can lead to retrieval and summarization of old user prompts, assistant replies, or contextual details that were not needed for the present request.

Natural-Language Policy Violations

Medium
Confidence
90% confidence
Finding
The file mandates that the fact check be in the same language as the user's question, which imposes a language behavior unconditionally. There is no stated user choice or opt-in for language preference, so this is a natural-language locale policy concern under the stated rules.

Ssd 3

Medium
Confidence
99% confidence
Finding
The required round-by-round output format asks the model to restate each user's question and the AI's answer, which can directly echo sensitive or private content back into the conversation. This is especially dangerous when combined with automatic or ambiguous triggering, because it can cause unnecessary disclosure of prior content without strong user intent.

Static analysis

No suspicious patterns detected.