Security audit

skill-compass

Security checks for vulnerabilities and agentic risk

Overview

This skill is a disclosed skill-auditing tool, but it can modify installed skill instructions and turns untrusted skill metadata into agent prompts and shell commands.

Install only if you are comfortable letting a tool inspect installed skill definitions and potentially edit SKILL.md files. Prefer running audit-only first, review generated prompts and diffs manually, avoid pasting generated shell commands from untrusted skill metadata, and keep backups because fixes and rollback can change future agent behavior.

A.I.G

Vulnerability Patterns

Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally

Findings (3)

T01 · Skill Instruction Hijacking

Error

Location: scripts/audit_skills.py:630
Finding: Prompt Injection Through Untrusted Skill Metadata
Content: View full analysis
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Warning

Location: scripts/audit_skills.py:753
Finding: Shell Command Injection in Generated GitHub Issue Commands
Content: View full analysis
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Note

Location: scripts/audit_skills.py:337
Finding: Unsafe YAML Repair Can Corrupt Skill Frontmatter
Content: View full analysis
Optional[str]: """Fix unquoted colons in YAML description field.""" lines = content.split('\n') fixed = False for i, line in enumerate(lines): if line.strip().startswith('description:'): val = line.split('description:', 1)[1].strip() if ':' in val and not val.startswith('"') and not val.startswith("'"): lines[i] = f'description: "{val}"' fixed = True break return '\n'.join(lines) if fixed else None ``` ### Technical Analysis The repair function attempts to make an unquoted YAML description safe by surrounding the original value with double quotes. It does not escape embedded double quotes, backslashes, control characters, or other YAML-significant content. For example, a value containing both a colon and an embedded double quote will be rewritten into a malformed double-quoted YAML scalar. The implementation also operates on text rather than parsing and serializing the complete frontmatter document, so it cannot reliably preserve YAML semantics. The resulting content is written back to the original skill file when auto-fix is enabled. Backups are normally created by the command-line workflows, but malformed output can still disable the skill until rollback or manual repair. ### Attack Path 1. A skill contains an unquoted description with a colon and embedded YAML-sensitive characters. 2. The victim runs `audit_skills.py --fix` or `audit_skills.py --init`. 3. `auto_fix_colon` wraps the value in double quotes without escaping its contents. 4. The modified `SKILL.md` contains malformed or semantically altered YAML. 5. The agent platform fails to parse or discover the affected skill. A malicious package ca ...[truncated 634 chars]
Remediation: View remediation

SkillSpector

Vulnerability Patterns

Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Rogue AgentSelf-Modification, Session Persistence
Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger

Findings (16)

Self-Modification

High

Category: Rogue Agent
Confidence: 85% confidence
Finding: Skill modifies its own code, configuration, or behavior at runtime. Self-modification enables an agent to escalate privileges, disable safety constraints, or install persistent backdoors.
Content: Scanner excerpt · scripts/audit_skills.py (reported line 613)May include surrounding context.
python
return "✅ All skills pass — no description rewrites needed!" lines = [ "You are a skill description optimizer. Your job is to rewrite SKILL.md", "frontmatter descriptions so they trigger reliably in AI agents.", "", "## Rules",

Session Persistence

Medium

Category: Rogue Agent
Confidence: 60% confidence
Finding: Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.
Content: Scanner excerpt · README.md (reported line 320)May include surrounding context.
Repo: https://github.com/Thomaszhou22/multi-search-engine

Submit with: gh issue create -R Thomaszhou22/multi-search-engine
--title "Improve description for reliable agent triggering"
--body "..."

text

Vague Triggers

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: 该 README 明确建议用户对 Agent 说“检查一下我的 skills”即可触发，但没有提供更具体的上下文限制、替代触发词范围或负例说明。这个短语较通用，容易与普通的求助表达重叠，可能导致技能在非预期场景下被调用。
Content: No source excerpt is available for this finding.

Undeclared Tool Scope

Medium

Category: MCP Least Privilege
Confidence: 90% confidence
Finding: The skill instructs the agent to run local scripts and potentially apply automatic fixes, which implies file read/write capability, but it does not declare any tool scope such as permissions or allowed-tools. That mismatch can cause the skill to run with broader-than-expected authority or make reviewers unaware of its filesystem impact, increasing the chance of unsafe execution in sensitive directories.
Content: No source excerpt is available for this finding.

Vague Triggers

Medium

Category: Not specified by scanner
Confidence: 91% confidence
Finding: The top-level description is extremely broad and overlaps with common troubleshooting language like 'skill not working' or 'agent behavior seems to ignore available skills.' In an agent routing system, this can cause over-triggering, shadow more specific skills, and lead the agent to perform auditing or modification workflows in contexts where the user did not intend broad skill ecosystem inspection.
Content: No source excerpt is available for this finding.

Session Persistence

Medium

Category: Rogue Agent
Confidence: 61% confidence
Finding: The skill states that onboarding should be marked complete and later skipped, implying persistent session or state tracking, but it does not define where that state is stored, how long it persists, or what protections apply. Unspecified persistence can lead to hidden state, cross-project behavior leakage, or improper writes to local files that affect future agent decisions without clear user awareness.
Content: Scanner excerpt · SKILL.md (reported line 81)May include surrounding context.
md
**Bad:** `"Helps with documents."` — no trigger, just capability **Good:** `"Use when the user asks to extract form fields, fill, redact, or parse tables from a PDF file."` **Fix:** Rewrite using the Description Design Pattern below. ### 2. YAML Frontmatter Format Error

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 93% confidence
Finding: The skill recommends logging user queries, expected skills, and actual outcomes without any notice about consent, storage location, retention, or redaction. Those logs can easily capture sensitive prompts, file names, internal project details, or operational metadata, creating a privacy and data-handling risk if persisted insecurely or reviewed later.
Content: No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

Category: Not specified by scanner
Confidence: 91% confidence
Finding: The entire markdown content is written as a Chinese-only post aimed at users, with no indication that other languages are supported or that Chinese is required for a specific regional or compliance reason. Under the stated policy, forcing a specific language without user opt-in is a natural-language policy violation.
Content: No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The file header and docstring frame this tool as a scanner that outputs a diagnostic report, but the script also contains backup, rollback, and auto-fix paths that write to SKILL.md files. That mismatch can mislead operators and downstream agents into invoking a supposedly read-only tool in contexts where filesystem modification is unsafe or unexpected.
Content: No source excerpt is available for this finding.

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The rollback path restores backed-up SKILL.md files by overwriting current files without a confirmation prompt or diff review. Although intended as recovery, overwrite operations can destroy legitimate changes or be abused to revert content unexpectedly when run by an automated agent.
Content: No source excerpt is available for this finding.

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 96% confidence
Finding: In the --init flow, the script automatically creates a backup and then runs audit_skills(..., auto_fix=True), which can modify SKILL.md files without an explicit confirmation prompt at the point of change. For agent-driven use, silent mutation of prompt or configuration files increases the chance of unintended edits, integrity issues, and hard-to-review drift.
Content: No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium

Category: Not specified by scanner
Confidence: 87% confidence
Finding: External-project feedback generation is not necessary for core troubleshooting of local skill trigger failures, yet the code builds issue bodies and repository-targeted CLI commands from local content. In an agent setting, this broadens data flow from local files to external destinations and creates a pathway for unreviewed outbound actions.
Content: No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium

Category: Not specified by scanner
Confidence: 89% confidence
Finding: The script goes beyond local trigger diagnostics by generating GitHub issue submission commands for third-party repositories based on URLs found in skill files. This expands the tool's operational scope into external communications and could facilitate unintended disclosure, spam, or social engineering if invoked automatically by an agent.
Content: No source excerpt is available for this finding.

Natural-Language Policy Violations

Low

Category: Not specified by scanner
Confidence: 79% confidence
Finding: This markdown includes a policy/instruction snippet written only in Chinese for users to add to AGENTS.md, while the surrounding document is primarily English. That can impose a locale-specific instruction without explicit user opt-in or a stated reason for requiring Chinese.
Content: No source excerpt is available for this finding.

Vague Triggers

Low

Category: Not specified by scanner
Confidence: 82% confidence
Finding: The instruction says onboarding runs 'when this skill is first installed' or when the user says 'check my skills' / 'audit skills' for the first time, but it does not clearly define how 'first time' is determined across sessions or distinguish onboarding from ordinary audits. This ambiguity can lead to inconsistent or unintended activation behavior.
Content: No source excerpt is available for this finding.

Natural-Language Policy Violations

Low

Category: Not specified by scanner
Confidence: 94% confidence
Finding: The document sets lang="zh" and all visible text is in Chinese, which imposes a specific language on users. Under the policy, locale or language constraints should be optional or clearly justified; this file provides no such opt-in or explanation.
Content: No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.