skill-usefulness-audit

• Outcome: Pass
• Latest audit: Sep 16, 2026 · 6:52 AM UTC
• Version: 0.3.24

Overview

This is a local installed-skill audit tool with disclosed file scanning and report generation, and I found no evidence of deletion, exfiltration, persistence, or hidden execution.

Install only if you are comfortable with a local tool reading installed skill folders and any evidence files you provide. Prefer passing --skills-root to limit the audit scope, and be careful with usage or history files because they may contain conversation details, local paths, project names, or customer data.

A.I.G

Vulnerability Patterns

• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally
• Unauthorized Access and Privilege Escalation: Obtains permissions beyond the task's legitimate needs

SkillSpector

Vulnerability Patterns

• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• MCP Tool Poisoning: Hidden Instructions, Unicode Deception, Parameter Description Injection
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands

Findings (14)

Tp4

High

• Category: MCP Tool Poisoning
• Confidence: 93% confidence
• Finding: The declared description says this skill helps review installed skills to see what you actually use, overlaps, and what may no longer be worth keeping. The code shown does not measure real usage or overlap between skills. Instead, its primary behavior is a static bundle auditor: it discovers skill files on disk, parses skill metadata, scans code and reference files for risky patterns, install/execution surfaces, private content, broken links, bloat, syntax/import problems, and produces risk/quality metrics. While this could support a broader usefulness audit, the supplied code chunk is materially focused on security and quality assessment of skill contents, not on determining actual usage or redundancy. That is a significant description-behavior mismatch.

Tp4

High

• Category: MCP Tool Poisoning
• Confidence: 94% confidence
• Finding: The declared purpose is a usefulness audit of installed skills—examining usage, overlap, and retention value. The supplied code does not inspect installed skills, usage frequency, overlap, or inventory. Instead, it imports and combines multiple security-oriented risk rule sets into a single RISK_RULES collection. That behavior aligns with risk detection or policy scanning support, not with evaluating whether skills are worth keeping. This is a materially different primary purpose, so it should be flagged as a mismatch.

Tp4

High

• Category: MCP Tool Poisoning
• Confidence: 95% confidence
• Finding: The declared purpose is inventory/usefulness analysis of installed skills, but the supplied code implements security-oriented pattern rules for detecting base64-encoded payload behavior. That is a materially different primary purpose from reviewing which skills are used or redundant. While this could conceivably be part of a broader security audit system, based on the provided description it is not a supporting detail of usefulness auditing and represents undeclared scanning functionality.

Ae1

High

• Category: analysis-evasion

Content

3. Inspect each `SKILL.md` and its script/reference/asset metrics.

• Confidence: 100% confidence
• Finding: Referenced artifact was not completely inspected

Credential Access

High

• Category: Privilege Escalation

Content

GENERIC_REFERENCE_STEMS = {"guide", "guides", "reference", "references", "doc", "docs", "notes", "workflow", "workflows"}
PRIVATE_ROOT_CONTENT_SUFFIXES = {".cfg", ".conf", ".env", ".ini", ".json", ".properties", ".toml", ".yaml", ".yml"}


def reference_is_directly_disclosed(body_lower: str, root: Path, path: Path) -> bool:

• Confidence: 60% confidence
• Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Credential Access

High

• Category: Privilege Escalation

Content

GENERIC_REFERENCE_STEMS = {"guide", "guides", "reference", "references", "doc", "docs", "notes", "workflow", "workflows"}
PRIVATE_ROOT_CONTENT_SUFFIXES = {".cfg", ".conf", ".env", ".ini", ".json", ".properties", ".toml", ".yaml", ".yml"}


def reference_is_directly_disclosed(body_lower: str, root: Path, path: Path) -> bool:

• Confidence: 60% confidence
• Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Credential Access

High

• Category: Privilege Escalation

Content

GENERIC_REFERENCE_STEMS = {"guide", "guides", "reference", "references", "doc", "docs", "notes", "workflow", "workflows"}
PRIVATE_ROOT_CONTENT_SUFFIXES = {".cfg", ".conf", ".env", ".ini", ".json", ".properties", ".toml", ".yaml", ".yml"}


def reference_is_directly_disclosed(body_lower: str, root: Path, path: Path) -> bool:

• Confidence: 60% confidence
• Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Lp3

Medium

• Category: MCP Least Privilege
• Confidence: 86% confidence
• Finding: The skill metadata declares no explicit tool scope or allowed-tools boundary, yet the skill instructions direct execution of a Python script that reads local files and writes reports, and the static analyzer also observed broader capabilities including shell/network/environment access. This creates a permission-transparency gap: users and hosts cannot easily tell from the manifest what powers the skill expects, which increases the chance of over-privileged execution or accidental misuse.

Autonomous Decision Making

Medium

• Category: Excessive Agency

Content

## Output Contract

Use one run for both output layers; do not ask the user to choose a quick or full mode.
Standard output is a short natural-language report. Its opening paragraph states the audited skill count and the total characters plus approximate tokens of loaded entry descriptions. Lead with actual usage, not static risk or bundle health, and keep scores, internal codes, risk flags, and tables out of this layer.
When `--markdown-out` is provided, write the detailed evidence—with scores, action codes, missing evidence, burden, and risk notes—in the same run.
Match the user's language: clean Chinese for `zh-CN` and clean English for `en`, except for skill names and unavoidable paths or commands.

• Confidence: 80% confidence
• Finding: Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Natural-Language Policy Violations

Medium

• Confidence: 92% confidence
• Finding: When `chinese_only_report_profile_enabled()` is true, the parser restricts `--report-language` to `zh-CN` and states the report language is fixed by the package profile. This is a natural-language locale policy constraint that removes user choice and is not justified in this file as a region-specific compliance requirement.

Natural-Language Policy Violations

Medium

• Confidence: 86% confidence
• Finding: The constant `REPORT_PROFILE_ZH_CN_ONLY = "zh-CN-only"` indicates a specific locale restriction embedded in the skill code. In this file there is no nearby natural-language justification, opt-in mechanism, or indication that the tool is intentionally region-specific, so this may violate language/locale policy requirements.

Natural-Language Policy Violations

Medium

• Confidence: 95% confidence
• Finding: This code can unconditionally switch reporting to `zh-CN` either when a Chinese-only profile is enabled or when locale/environment variables indicate Chinese, with English as the only fallback. That creates a natural-language locale policy concern because the skill may present output in a specific language without explicitly offering the user a choice at the point of use.

Context-Inappropriate Capability

Medium

• Confidence: 92% confidence
• Finding: The function enumerates skill directories under both the current working directory and the user's home directory, including hidden locations like ~/.openclaw/skills and ~/.agents/skills. For a usefulness audit this is broader-than-necessary data access, because it inspects potentially unrelated locally installed skills and their contents, which may expose private metadata, references, or bundled artifacts during analysis.

Scope Creep

Low

• Category: Excessive Agency

Content

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

• Confidence: 70% confidence
• Finding: Skill's behavior or capabilities extend beyond its stated purpose. Scope creep allows an agent to perform actions unrelated to its documented functionality, increasing the attack surface.

Static analysis

No suspicious patterns detected.