Back to skill

Security audit

botlearn-assessment

Security checks for vulnerabilities and agentic risk

Overview

This self-assessment skill appears purpose-aligned, but it has enough user-control, persistence, and generated-report safety issues that users should review it before installing.

Install only if you intentionally want an autonomous self-assessment that may search, run local bundled Node scripts, and save detailed result files. Review or patch the report generators before opening or sharing HTML/SVG reports, and consider narrowing triggers and adding confirmation or cleanup steps for saved results.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/generate-html-report.js:301
Finding

Stored HTML Injection in Generated Assessment Reports

Content
View full analysis
Q${idx + 1} ${d.id} ${name} ${statusText}

${isZH ? '原因' : 'Reason'}: ${d.skipReason || 'Required capability not available'}

`; } const criteriaRows = (d.criteria || []).map(c => ` ${c.name} ${(c.weight * 100).toFixed(0)}% ${c.score}/5 ${c.justification || ''} `).join(''); ``` Other unescaped metadata is inserted into the document: ```js return `
Q${idx + 1} ${d.id} ${name} ${diff} ×${d.multiplier} ${d.adjScore.toFixed(1)} ${statusText}
``` ```js const html = ` ${t.title} — ${data.sessionId} ``` ```js

${t.title}

${t.session}: ${ ...[truncated 2986 chars]
Remediation
View remediation
/g, '>') .replace(/"/g, '"') .replace(/'/g, '''); } ``` 3. Encode attribute values separately and validate fields such as `lang`, `sessionId`, and dimension IDs against strict allowlists. 4. Validate input against the supplied JSON schema before report generation. Add: - String length limits. - Enumerations for language, status, confidence, level, and dimension IDs. - Numeric ranges for scores, weights, and multipliers. - A restrictive pattern such as `^[A-Za-z0-9_-]+$` for session identifiers. 5. Avoid constructing complex documents through raw template interpolation. A DOM builder or templating system with automatic escaping is preferable. 6. Add a restrictive Content Security Policy to generated reports, for example: ```html ``` 7. Add regression tests for every report string field using payloads containing: - `

T09 · Insecure Skill Coding Practices

Note
Location
scripts/radar-chart.js:194
Finding

Arbitrary SVG Markup Injection Through Unescaped Session Identifier

Content
View full analysis
Session: ${session}` : ''; ``` The value originates directly from command-line arguments: ```js const params = {}; process.argv.slice(2).forEach(arg => { const m = arg.match(/^--([^=]+)=(.*)$/); if (m) params[m[1].toLowerCase()] = m[2]; }); const session = params.session ?? ''; ``` ### Technical Analysis The `--session` command-line value is inserted directly into an SVG text node without XML escaping or format validation. A crafted value can close the `text` element and inject arbitrary SVG elements. For example: ```bash node scripts/radar-chart.js \ '--session=' \ > malicious.svg ``` The generated SVG then contains attacker-controlled active markup. Script execution behavior depends on how the SVG is consumed: browsers generally disable scripts when SVG is loaded as an image, but scripts may execute when the SVG is opened directly, loaded as a document, or embedded in an active context. The normal workflow generates a predictable session identifier, reducing routine exploitability. However, the script's documented command-line interface accepts arbitrary values and does not enforce the expected identifier format. ### Attack Path 1. An attacker influences a command invocation, automation variable, or wrapper that supplies `--session`. 2. The attacker provides XML that terminates the existing `text` element and adds malicious SVG markup. 3. `radar-chart.js` writes the payload unchanged into the standalone SVG. 4. The generated file is shared with or opened by a victim. 5. ...[truncated 780 chars]
Remediation
View remediation
/g, '>') .replace(/"/g, '"') .replace(/'/g, '''); } ``` Then use: ```js const safeSession = escapeXml(session); ``` 2. Enforce the expected session identifier format before generating the SVG: ```js if (!/^[A-Za-z0-9_-]{1,80}$/.test(session)) { console.error('Invalid session identifier'); process.exit(1); } ``` 3. Reject control characters and impose a conservative maximum length. 4. Add tests for closing `text` tags, `script` elements, event handlers, entity sequences, and quotation marks. 5. When distributing generated charts, serve them with `Content-Type: image/svg+xml` and a restrictive Content Security Policy. 6. If active SVG functionality is unnecessary, consider generating a raster image or sanitizing the final SVG with a well-maintained SVG sanitizer before publication. ]]>
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (45)

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The trigger list contains generic terms like "assessment," "evaluate," and short multilingual variants that are likely to appear in ordinary conversations. This can cause the skill to activate unintentionally and override normal agent behavior, increasing the chance of prompt hijacking or misrouting user requests into the exam workflow.

Content

No source excerpt is available for this finding.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 132)May include surrounding context.

md
node scripts/radar-chart.js \

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 173)May include surrounding context.

md
node scripts/radar-chart.js \

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 174)May include surrounding context.

md
| `scripts/generate-html-report.js` | HTML report generator with embedded radar |

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The strategy directs the agent to execute an external Node.js script, which turns a self-evaluation flow into code execution. If the script or its dependencies are modified, malicious, or parameter handling is unsafe, this can lead to arbitrary command execution, file access, or broader compromise of the runtime environment.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The README states that the skill generates Markdown, HTML, SVG, and JSON result files, but it does not clearly warn users that these artifacts are persisted under a results directory. In a self-assessment context, those files may contain prompts, answers, scores, trend history, or other sensitive operational data, creating retention and privacy risk if users do not realize storage occurs.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The trigger list includes generic phrases such as "exam," "assessment," "evaluate," and "test yourself," which are common in normal conversation and can cause the skill to activate unintentionally. Because this skill can launch a multi-phase workflow, generate reports, and write output files, accidental invocation expands the blast radius beyond a harmless chat response.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The description says the skill triggers on broad concepts like self-evaluation and periodic review without clearly limiting scope. That ambiguity makes accidental invocation more likely and can expose users to unintended autonomous behavior defined later in the skill.

Content

No source excerpt is available for this finding.

Shadow Command Trigger

Medium
Category
Trigger Abuse
Confidence
84% confidence
Finding

The trigger "test yourself" overlaps with common built-in testing semantics and may shadow or conflict with existing commands. This can cause command-routing confusion and invoke the skill when the platform intended a different system action.

Content

No source excerpt is available for this finding.

Shadow Command Trigger

Medium
Category
Trigger Abuse
Confidence
87% confidence
Finding

The trigger "run exam" uses a generic command verb that may collide with built-in "run" behaviors or other execution-oriented skills. In combination with the skill's autonomous flow, a routing conflict could lead to unintended exam execution or user confusion about what command is being run.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
88% confidence
Finding

The skill explicitly forbids asking the user for clarification, confirmation, or help, forcing the agent to act autonomously even when context is ambiguous or tools are unavailable. This can lead to incorrect execution paths, unnecessary file operations, or unreviewed decisions that the user did not intend.

Content

Scanner excerpt · SKILL.md (reported line 43)May include surrounding context.

md
2. **Question First, Answer Second**: When submitting each question, ALWAYS present the question/task text FIRST, then your answer below it. The reader must see what was asked before seeing the response.
3. **Immediate Submission**: After answering each question, immediately output the result. Once output, it CANNOT be modified or retracted.
4. **No User Assistance**: The user is the INVIGILATOR. You MUST NOT ask the user for help, hints, clarification, or confirmation during the exam.
5. **Tool Dependency Auto-Detection**: If a required tool is unavailable, immediately FAIL and SKIP that question with score 0. Do NOT ask the user to install tools.
6. **Self-Contained Execution**: You must attempt everything autonomously. If you cannot do it alone, fail gracefully.

---

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
86% confidence
Finding

The instruction to proceed with answering and moving on with "no pause, no confirmation needed" authorizes autonomous multi-step behavior without an approval checkpoint. In an assessment skill that may read files, use tools, and generate artifacts, this reduces user control and increases the risk of unintended actions.

Content

Scanner excerpt · SKILL.md (reported line 81)May include surrounding context.

md
1. **Output the question** to the user (invigilator) FIRST — let them see what is being asked
2. **Attempt to solve** the question autonomously (do NOT consult rubric)
3. **Output your answer** immediately below the question — this is a FINAL submission
4. **Move to next question** — no pause, no confirmation needed

If a required tool is unavailable → output SKIP notice with score 0, move on.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The schema explicitly requires and structures a chain_of_thought object with evidence, reasoning, and key_factor for each scoring criterion. This is dangerous because it encourages generation, storage, and possible downstream exposure of hidden reasoning traces, which can leak sensitive intermediate analysis, make prompt extraction easier, and conflict with safer patterns that use brief justifications or observable evidence only. In this self-evaluation skill, the risk is heightened because the system is designed to score its own answers, making internal reasoning capture a first-class artifact rather than an incidental log.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
89% confidence
Finding

The flow explicitly instructs the agent to skip tasks and assign a zero score without notifying or confirming with the user when required capabilities are unavailable. This removes user oversight from a consequential decision and can be abused to suppress valid task execution or manipulate assessment outcomes through tool-detection failures.

Content

Scanner excerpt · flows/exam-execution.md (reported line 36)May include surrounding context.

→ MOVE to next question

text

**CRITICAL**: Do NOT ask user to install tools or confirm skipping. Just skip and move on.

---

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
84% confidence
Finding

The instruction to work autonomously and not ask the user for help or clarification can cause the agent to proceed on ambiguous or underspecified questions without validation. In an assessment setting this may degrade integrity and, if reused in other contexts, normalize unsafe autonomous action without adequate human approval.

Content

Scanner excerpt · flows/exam-execution.md (reported line 63)May include surrounding context.

md
Work on the question autonomously:
- Do NOT consult the rubric
- Do NOT ask user for any help or clarification
- Be honest about uncertainty
- Record confidence: high / medium / low

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The flow explicitly saves detailed assessment results to a file in results/ but does not disclose persistent storage or obtain user consent beforehand. Assessment content may include user prompts, inferred language, timestamps, scores, and behavioral metadata, creating an avoidable privacy and retention risk if stored by default.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

Appending session metadata to results/INDEX.md creates a persistent activity log without prior disclosure. Even if the index stores only summary fields, it still records timestamps, performance levels, weakest dimensions, and skip counts that can profile users over time.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The flow explicitly stores full question text, full answer text, scoring criteria, and justifications into multiple persistent artifacts (JSON, Markdown, HTML) without any privacy warning, minimization guidance, or retention controls. In a self-assessment skill, answers may contain sensitive user or model-generated content, so broad duplication across files increases exposure through accidental disclosure, over-retention, and secondary reuse.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The instruction at L144 says to output to the user in their language, but the only provided announcement template is entirely in Chinese. This effectively biases or forces a specific language in user-facing text without opt-in, which conflicts with the stated language-choice requirement.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The document claims a 5-dimension BotLearn self-assessment in metadata, but the body defines a 10-dimension scoring system with materially broader criteria such as safety, cost efficiency, reliability, and memory/context. This creates a spec/behavior mismatch that can silently alter evaluation outcomes, confuse downstream consumers, and enable unexpected scoring logic to govern automated assessments.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · knowledge/best-practices.md (reported line 92)May include surrounding context.

md
| 3 | Some self-checking but misses issues |
| 2 | Minimal self-verification |
| 1 | No self-checking, obvious errors remain |
| 0 | No verification attempt |

#### Criterion 4: Completeness (Weight: 0.15)
| Score | Description |

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The manifest describes a 5-dimension capability self-assessment covering reasoning, retrieval, creation, execution, and orchestration. This file instead defines a broader 10-dimension framework including memory, cost efficiency, reliability, safety/compliance, and task efficacy, materially expanding the skill's documented behavior beyond the declared scope.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The workflow explicitly says the assessment auto-starts without user confirmation, loads prior self-test history, and offers export of results, yet it does not require clear consent or document privacy implications. In a skill that may process historical evaluation data and generate exports, this can lead to unintended access, reuse, or disclosure of potentially sensitive behavioral or operational information.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The file claims to define output templates for the BotLearn self-assessment skill, but the final report template instead brands the system as 'OpenClaw Agent Capability Probe Report v2' and uses a different capability taxonomy. This inconsistency can mislead users, downstream automation, or evaluators about what system produced the report and what dimensions were actually assessed, creating integrity and trust problems in security-sensitive or audit contexts.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The file instructs that code output should remain in English, which imposes a language constraint without offering user choice or documenting an opt-in. This matches the policy category for language or locale restrictions in natural-language instructions.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.