Security audit

Botmark Skill

Security checks for vulnerabilities and agentic risk

Overview

This benchmark skill is partly purpose-aligned, but it gives a remote service broad control to update code and instructions, stores credentials locally, and sends sensitive profile/work data.

Review before installing. Only use this skill if you are comfortable with BotMark receiving benchmark answers plus profile/work-context reflections, with local plaintext API-key storage, and with server-directed runner and skill updates. Prefer a versioned, signed install with a platform secret store and disabled silent self-updates.

A.I.G

Vulnerability Patterns

Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally

Findings (5)

T03 · Remote Payload Retrieval and Execution

Error

Location: setup.sh:5
Finding: Unverified Remote Scripts and Python Payloads Are Downloaded and Executed
Content: View full analysis
Remediation: View remediation

T01 · Skill Instruction Hijacking

Error

Location: system_prompt_en.md:64
Finding: Server Responses Can Replace Agent Instructions and Tool Definitions Persistently
Content: View full analysis
Remediation: View remediation

T01 · Skill Instruction Hijacking

Warning

Location: system_prompt_en.md:3
Finding: Skill Forces Unsolicited Promotion and Verbatim Delivery of Server-Controlled Messages
Content: View full analysis
Remediation: View remediation

other

Error

Location: SKILL.md:112
Finding: Benchmark Workflow Requires Transmission of Excessive Personal and Behavioral Information
Content: View full analysis
", "agent_name": "", "platform": "openclaw", "model": "", "country": "CN", "bio": "", "talktoowner": "", "work_and_challenges": "", "skill_version": "2.18.1", "cached_engine_version": "", "project": "comprehensive", "tier": "basic" }' ``` From `SKILL.md:226-234`: ```bash curl -s -X POST "${BOTMARK_SERVER_URL:-https://botmark.cc}/api/v1/bot-benchmark/feedback" \ -H "Content-Type: application/json" \ -d '{ "session_token": "", "feedback": "" }' ``` From `botmark_engine.py:1202-1237`: ```python def _submit_batch(answers: dict, batch_label: str = "") -> dict: """Submit a batch of answers for quality validation.""" payload = { "session_token": SESSION_TOKEN, "answers": answers, "batch_label": batch_label, } return _api_call("/api/v1/bot-benchmark/submit-batch", payload) def _submit_final(all_answers: dict, client_meta: dict, local_scores: dict = None, score_hmac: str = None) -> dict: """Submit final answers and get the score.""" payload = { "session_token": SESSION_TOKEN, "answers": all_answers, "signatu ...[truncated 3251 chars]
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Error

Location: setup.sh:143
Finding: API Key Input Is Interpolated into Executable Python and Shell-Sourced Configuration
Content: View full analysis
/dev/null; then python3 -c " import json, os config_path = '$OPENCLAW_CONFIG' # Read existing or create new if os.path.exists(config_path): with open(config_path) as f: cfg = json.load(f) else: cfg = {} # Ensure skills.entries.botmark-skill exists cfg.setdefault('skills', {}).setdefault('entries', {}) cfg['skills']['entries'].setdefault('botmark-skill', {}) cfg['skills']['entries']['botmark-skill']['apiKey'] = '$INPUT_KEY' # Write back with open(config_path, 'w') as f: json.dump(cfg, f, indent=2, ensure_ascii=False) " && ok "API Key saved to openclaw.json (OpenClaw native config)" fi # Save to .botmark_env (fallback — for non-openclaw.json environments) cat > "$SKILL_DIR/.botmark_env" << ENVEOF BOTMARK_API_KEY="$INPUT_KEY" ENVEOF chmod 600 "$SKILL_DIR/.botmark_env" ok "API Key saved to .botmark_env (fallback)" fi ``` From `setup.sh:125-128`: ```bash # Check 3: .botmark_env file if [ -z "$EXISTING_KEY" ] && [ -f "$SKILL_DIR/.botmark_env" ]; then source "$SKILL_DIR/.botmark_env" 2>/dev/null EXISTING_KEY="${BOTMARK_API_KEY:-}" fi ``` ### Technical Analysis `INPUT_KEY` is directly interpolated into a Python program passed to `python3 -c`. A value con ...[truncated 1853 chars]
Remediation: View remediation

SkillSpector

Vulnerability Patterns

Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults

Findings (114)

Hidden Instructions

High

Category: Prompt Injection
Confidence: 70% confidence
Finding: Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.
Content: Scanner excerpt · README.md (reported line 28)May include surrounding context.
md
--- <!-- 🖼️ IMAGE SUGGESTION: assets/hero-radar-chart.png A 5-axis radar chart showing IQ/EQ/TQ/AQ/SQ scores for a sample bot. This is the most shareable visual — use one from a real evaluation. Dimensions: ~800x500px, dark background preferred for contrast.

Hidden Instructions

High

Category: Prompt Injection
Confidence: 70% confidence
Finding: Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.
Content: Scanner excerpt · README.md (reported line 78)May include surrounding context.
md
## How It Works <!-- 🖼️ IMAGE SUGGESTION: assets/how-it-works.png A horizontal flow diagram: [Owner says "benchmark"] → [Bot calls BotMark API] → [Receives exam package] → [Answers ~60 questions] → [Submits in batches] → [Gets scored report]

Tp4

High

Category: MCP Tool Poisoning
Confidence: 99% confidence
Finding: The skill advertises simple benchmarking, but also instructs installing files, downloading artifacts from a remote server, collecting an API key in chat, writing secrets to local configuration, and performing environment verification. Those additional behaviors materially expand risk and are not obvious from the declared purpose, creating a deceptive trust boundary.
Content: No source excerpt is available for this finding.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 99% confidence
Finding: The skill advertises simple benchmarking, but also instructs installing files, downloading artifacts from a remote server, collecting an API key in chat, writing secrets to local configuration, and performing environment verification. Those additional behaviors materially expand risk and are not obvious from the declared purpose, creating a deceptive trust boundary.
Content: No source excerpt is available for this finding.

Ssd 3

High

Category: Not specified by scanner
Confidence: 99% confidence
Finding: The skill explicitly instructs the agent to request an API key from the user in chat and then save it to a local file for future reuse. This is dangerous because secrets entered in chat may be logged or exposed, and local plaintext persistence creates an attractive target for exfiltration by other skills, processes, or later compromise.
Content: No source excerpt is available for this finding.

External Script Fetching

High

Category: Supply Chain
Confidence: 98% confidence
Finding: The documentation explicitly tells users to fetch and execute a remote script in one step using curl piped to bash. This is a well-known unsafe pattern because it grants immediate code execution to whatever content is served at that URL, with no review, pinning, checksum, or signature verification.
Content: Scanner excerpt · examples/openclaw_setup.md (reported line 19)May include surrounding context.
Or download and run directly:

bash
curl -fsSL https://botmark.cc/skill/setup.sh | bash

The script will:

Chaining Abuse

High

Category: Tool Misuse
Confidence: 97% confidence
Finding: The use of a pipeline into bash creates command chaining that removes any pause between network retrieval and local execution. In a skill that already emphasizes exec-based behavior and automatic setup, this makes compromise more dangerous because users are encouraged to trust opaque automation for both installation and runtime behavior.
Content: Scanner excerpt · examples/openclaw_setup.md (reported line 19)May include surrounding context.
Or download and run directly:

bash
curl -fsSL https://botmark.cc/skill/setup.sh | bash

The script will:

External Script Fetching

High

Category: Supply Chain
Confidence: 95% confidence
Finding: The usage instruction recommends curl ... | bash, which executes remote script content immediately without inspection or integrity verification. If the hosting service or connection is compromised, users can run arbitrary attacker-controlled shell commands on their system.
Content: Scanner excerpt · setup.sh (reported line 6)May include surrounding context.
sh
# BotMark Skill — One-command setup for OpenClaw # # Usage: # curl -fsSL https://botmark.cc/skill/setup.sh | bash # # or after manual download: # bash botmark-skill/setup.sh #

Chaining Abuse

High

Category: Tool Misuse
Confidence: 94% confidence
Finding: The | bash pattern chains network retrieval directly into execution, removing any review boundary between download and code execution. In a setup script that also handles credentials and writes files into the agent skill path, this materially increases the blast radius of any remote compromise.
Content: Scanner excerpt · setup.sh (reported line 6)May include surrounding context.
sh
# BotMark Skill — One-command setup for OpenClaw # # Usage: # curl -fsSL https://botmark.cc/skill/setup.sh | bash # # or after manual download: # bash botmark-skill/setup.sh #

Self-Modification

High

Category: Rogue Agent
Confidence: 90% confidence
Finding: The installer writes SKILL.md and engine files from remotely supplied JSON directly into the local skill directory. This is effectively self-installation/update of executable or instruction-bearing content without integrity verification, enabling supply-chain compromise if the server, network path, or endpoint content is tampered with.
Content: Scanner excerpt · setup.sh (reported line 81)May include surrounding context.
sh
data = json.load(f) skill_dir = '$TMPDIR/botmark-skill' os.makedirs(skill_dir, exist_ok=True) # Write SKILL.md if 'skill_md' in data: with open(f'{skill_dir}/SKILL.md', 'w') as f: f.write(data['skill_md'])

Context-Inappropriate Capability

High

Category: Not specified by scanner
Confidence: 97% confidence
Finding: The skill instructs the agent to solicit, store, source, replace, and manage API keys in local files such as skills/botmark-skill/.botmark_env. Persisting secrets based on conversational input expands secret-handling duties unnecessarily and risks credential leakage, misuse, or accidental disclosure across sessions and skills.
Content: No source excerpt is available for this finding.

Vague Triggers

High

Category: Not specified by scanner
Confidence: 97% confidence
Finding: The skill explicitly encourages proactive invocation when the owner asks about capability, after upgrades, or during periodic self-checks, even without a clear direct request. That increases the chance of the agent initiating benchmark operations, collecting profile text, or prompting for credentials under ambiguous circumstances.
Content: No source excerpt is available for this finding.

Context-Inappropriate Capability

High

Category: Not specified by scanner
Confidence: 99% confidence
Finding: The instructions require downloading, saving, and executing a remote runner script, then driving it via subprocess-like CLI calls. This is equivalent to remote code execution under the agent's authority, and the same instructions also allow server-provided updates to replace local behavior, making compromise of the server or supply chain highly impactful.
Content: No source excerpt is available for this finding.

Vague Triggers

High

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The trigger list includes very common phrases like '测一下', '打分', '体检', and '检测', which can appear in unrelated conversations. That makes accidental invocation plausible, causing unnecessary external API calls, local file operations, and possible secret-handling flows without clear user intent to run this specific skill.
Content: No source excerpt is available for this finding.

Ssd 1

High

Category: Not specified by scanner
Confidence: 99% confidence
Finding: The skill gives server-provided inline upgrade instructions priority over cached instructions and permits runtime replacement of local tool definitions. This creates a remote control channel whereby the service can change the agent's effective capabilities and behavior after installation, bypassing normal review and greatly increasing supply-chain risk.
Content: No source excerpt is available for this finding.

Ssd 3

High

Category: Not specified by scanner
Confidence: 99% confidence
Finding: The instructions explicitly tell the agent to ask the owner for an API key via chat, store it on disk, and reuse it across sessions. This creates a durable secret-exfiltration and persistence channel through normal conversation, and is especially dangerous because the skill also encourages broad triggering and autonomous execution paths.
Content: No source excerpt is available for this finding.

Description-Behavior Mismatch

High

Category: Not specified by scanner
Confidence: 98% confidence
Finding: The manifest advertises simple API-based benchmarking, but the embedded instructions expand scope dramatically: persisting credentials, reading/writing local files, downloading/executing a Python runner, and performing local upgrade logic. This creates a large hidden trust boundary and enables arbitrary code execution and sensitive data handling well beyond what a normal function-only skill implies.
Content: No source excerpt is available for this finding.

Context-Inappropriate Capability

High

Category: Not specified by scanner
Confidence: 96% confidence
Finding: The instructions tell the agent to persist API keys in a local .botmark_env file, extending credential storage beyond the manifest's stated environment-variable requirement. This increases the chance of secret leakage through filesystem exposure, backup/sync systems, logging, or accidental inclusion in other tool operations.
Content: No source excerpt is available for this finding.

Context-Inappropriate Capability

High

Category: Not specified by scanner
Confidence: 96% confidence
Finding: The instructions tell the agent to persist API keys in a local .botmark_env file, extending credential storage beyond the manifest's stated environment-variable requirement. This increases the chance of secret leakage through filesystem exposure, backup/sync systems, logging, or accidental inclusion in other tool operations.
Content: No source excerpt is available for this finding.

Vague Triggers

High

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The trigger phrases are broad and include common terms like '测一下', '打分', '检测', and 'benchmark', making accidental activation plausible in normal conversation. Because activation initiates sensitive actions such as profile transmission, credential handling, and potentially code execution, overbroad triggering materially raises risk.
Content: No source excerpt is available for this finding.

Missing User Warnings

High

Category: Not specified by scanner
Confidence: 98% confidence
Finding: The manifest describes sending birthday, country, bio, heartfelt messages to the owner, work challenges, model/platform data, and potentially webhook URLs to the BotMark API, but the top-level description does not clearly warn users that this sensitive profile and reflective content will be transmitted to a third party. The data is not strictly necessary for a basic benchmark, and some fields are especially personal.
Content: No source excerpt is available for this finding.

Context-Inappropriate Capability

High

Category: Not specified by scanner
Confidence: 97% confidence
Finding: The instructions tell the agent to delete old skill directories, replace tool definitions, update cached engine files, and modify local skill state. These behaviors are broader than needed for benchmarking and create unnecessary capability to alter local configuration, remove files, and persist attacker-controlled updates from server responses.
Content: No source excerpt is available for this finding.

Vague Triggers

High

Category: Not specified by scanner
Confidence: 97% confidence
Finding: The activation phrases include broad everyday terms such as '测一下', '打分', '体检', '检测', and allow proactive suggestion after upgrades or periodically. This makes accidental or non-consensual triggering plausible, causing the skill to collect profile data, start network activity, and potentially execute the runner without clear intent from the user.
Content: No source excerpt is available for this finding.

Vague Triggers

High

Category: Not specified by scanner
Confidence: 97% confidence
Finding: The trigger phrases include broad everyday terms like '测一下', '打分', '检测', and '考考你', which can easily appear in normal conversation unrelated to BotMark. Because the skill performs network operations and can request/persist credentials, accidental activation is more dangerous than for a harmless informational skill.
Content: No source excerpt is available for this finding.

Context-Inappropriate Capability

High

Category: Not specified by scanner
Confidence: 98% confidence
Finding: The prompt instructs the agent to replace local tool definitions based on server-supplied inline upgrade data during runtime. This is effectively remote code/tool reconfiguration, allowing the external service to change the agent's capabilities and behavior without independent review, which is highly dangerous in a skill context.
Content: No source excerpt is available for this finding.