Back to skill

Security audit

Axiomata Skill Evaluator En

Security checks for vulnerabilities and agentic risk

Overview

This is a local OpenClaw skill-quality evaluator; it is read-only and purpose-aligned, but its documentation overstates some behavior and dependencies.

Use this skill only in a controlled Python environment, ideally a virtualenv with a trusted PyYAML install. Point it at intended skill directories only, especially when using --all, and treat the scores as heuristic quality feedback rather than a publishing or security guarantee.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T08 · Insecure Dependencies

Warning
Location
scripts/eval-skill.py:25
Finding
Unmanaged Third-Party YAML Dependency Contradicts Self-Contained Execution## Vulnerability Details **File Location**: `scripts/eval-skill.py:25` **Vulnerability Type**: Unmanaged third-party dependency and unsafe ambient module resolution **Risk Level**: Medium ### Vulnerable Code ```python import argparse import ast import json import os import re import sys import yaml ``` ### Technical Analysis The evaluator imports the third-party `yaml` module, provided in typical environments by PyYAML. However, the project does not include a dependency manifest, version constraint, lockfile, package hash, or documented trusted installation source. This conflicts with the documented claim that the evaluator is self-contained and uses no external dependencies. Python resolves imports from its runtime search path. Consequently, the script may load an incompatible, compromised, dependency-confusion-supplied, or locally shadowed `yaml` module. Imported Python modules execute their top-level code immediately, before the evaluator processes the target Skill. Although the audited project does not itself install a malicious dependency, it relies on an unmanaged ambient package and therefore cannot ensure the integrity of the code executed under the `yaml` name. The later use of `yaml.safe_load` reduces risks associated with unsafe YAML object deserialization, but it does not protect against malicious code executing when the `yaml` module is initially imported. ### Attack Path 1. A user prepares an environment to run `scripts/eval-skill.py`. 2. A malicious or compromised package providing the `yaml` module is introduced into that environment, or an attacker places a shadowing `yaml.py` or `yaml` package in a location searched before the legitimate PyYAML installation. 3. The user invokes `python3 scripts/eval-skill.py <skill-path>`. 4. Python resolves `import yaml` to the attacker-controlled module. 5. The module's top-level code executes immediately with the privileges and environment of the us ...[truncated 754 chars]
Remediation
## Remediation Suggestions 1. Explicitly declare PyYAML in a dependency manifest and pin it to a reviewed version. 2. Use a reproducible lockfile with cryptographic hashes, such as a hash-locked `requirements.txt`, and install only from a trusted package index. 3. Run the evaluator in an isolated virtual environment rather than relying on packages from the user's ambient Python environment. 4. Consider replacing PyYAML with a minimal bundled frontmatter parser if the project must remain genuinely self-contained. 5. Correct the documentation if an external dependency remains required, including the exact package name, supported version, and trusted installation procedure. 6. Avoid adding untrusted directories to `PYTHONPATH`, and execute the evaluator from a controlled, non-writable installation directory to reduce module-shadowing exposure. 7. Add automated dependency auditing and integrity verification to the release process.
Vulnerability Patterns
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (7)

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding
This second mismatch finding points to the same core issue: the documentation advertises capabilities such as ISO checks, self-contained bundling, and improvement behavior that may not exist. Even if not directly exploitable as code execution, this can cause unsafe reliance on incomplete validation and lead users to publish or trust artifacts under false assurances.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding
This second mismatch finding points to the same core issue: the documentation advertises capabilities such as ISO checks, self-contained bundling, and improvement behavior that may not exist. Even if not directly exploitable as code execution, this can cause unsafe reliance on incomplete validation and lead users to publish or trust artifacts under false assurances.

Lp3

Medium
Category
MCP Least Privilege
Confidence
93% confidence
Finding
The skill documents executable capabilities via bundled Python scripts and shell command examples, but it does not declare any tool scope such as allowed-tools or permissions. That omission weakens containment and reviewability because a host agent may grant broader file, environment, shell, or network access than users expect when invoking what is presented as a simple evaluator.

Vague Triggers

Medium
Confidence
94% confidence
Finding
The documented activation scope is broad enough to overlap with normal evaluation or quality-related requests, which increases the chance the skill is invoked in contexts the user did not intend. Ambiguous triggering is risky for a skill that can drive shell-based evaluation commands, because it may cause unnecessary code/tool execution or displace more appropriate, narrower workflows.

Vague Triggers

Medium
Confidence
97% confidence
Finding
Generic trigger phrases like 'Evaluate skill' and 'Check quality' are underspecified and encourage over-activation. In the context of an agent skill, that ambiguity can expand effective authority by causing the evaluator to run against arbitrary content or routine requests without clear user consent.

Session Persistence

Medium
Category
Rogue Agent
Content
"html", "http", "imaplib", "importlib", "inspect", "io", "ipaddress",
        "itertools", "json", "keyword", "linecache", "locale", "logging",
        "lzma", "math", "mimetypes", "multiprocessing", "operator", "os",
        "pathlib", "pickle", "platform", "plistlib", "pprint", "profile",
        "queue", "random", "re", "readline", "reprlib", "secrets",
        "select", "shelve", "shlex", "shutil", "signal", "smtplib",
        "socket", "sqlite3", "ssl", "stat", "statistics", "string",
Confidence
75% confidence
Finding
Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Natural-Language Policy Violations

Medium
Confidence
90% confidence
Finding
The code selects between explicit English and Chinese skill identities based on directory naming, embedding locale-specific behavior directly into the evaluator. There is no user-facing language choice or opt-in mechanism, which can violate a policy requiring language selection to be user-driven.

Static analysis

No suspicious patterns detected.