Back to skill

Security audit

botlearn-assessment

Security checks for vulnerabilities and agentic risk

Overview

This self-assessment skill appears purpose-aligned, but it has enough user-control, persistence, and generated-report safety issues that users should review it before installing.

Install only if you intentionally want an autonomous self-assessment that may search, run local bundled Node scripts, and save detailed result files. Review or patch the report generators before opening or sharing HTML/SVG reports, and consider narrowing triggers and adding confirmation or cleanup steps for saved results.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/generate-html-report.js:301
Finding
Stored HTML Injection in Generated Assessment Reports<![CDATA[ ## Vulnerability Details **File Location**: `scripts/generate-html-report.js:301-318` **Additional Locations**: `scripts/generate-html-report.js:333-344`, `scripts/generate-html-report.js:382-383`, `scripts/generate-html-report.js:566-567` **Vulnerability Type**: Stored HTML/JavaScript injection through unescaped report fields **Risk Level**: Medium ### Vulnerable Code ```js if (d.status === 'skipped') { return ` <div class="question-card skipped"> <div class="q-header"> <span class="q-num">Q${idx + 1}</span> <span class="q-dim">${d.id} ${name}</span> <span class="q-status skip-badge">${statusText}</span> </div> <p class="skip-reason">${isZH ? '原因' : 'Reason'}: ${d.skipReason || 'Required capability not available'}</p> </div>`; } const criteriaRows = (d.criteria || []).map(c => ` <tr> <td>${c.name}</td> <td>${(c.weight * 100).toFixed(0)}%</td> <td><span class="score-pill ${c.score >= 4 ? 'high' : c.score >= 3 ? 'mid' : 'low'}">${c.score}/5</span></td> <td class="justification">${c.justification || ''}</td> </tr>`).join(''); ``` Other unescaped metadata is inserted into the document: ```js return ` <div class="question-card"> <div class="q-header"> <span class="q-num">Q${idx + 1}</span> <span class="q-dim">${d.id} ${name}</span> <span class="q-diff">${diff} ×${d.multiplier}</span> <span class="q-score-badge" style="background:${d.adjScore >= 80 ? '#DCFCE7' : d.adjScore >= 60 ? '#FEF3C7' : '#FEE2E2'};color:${d.adjScore >= 80 ? '#166534' : d.adjScore >= 60 ? '#92400E' : '#991B1B'}">${d.adjScore.toFixed(1)}</span> <span class="q-status">${statusText}</span> </div> ``` ```js const html = `<!DOCTYPE html> <html lang="${lang}"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>${t.title} — ${data.sessionId}</title> ``` ```js <div class="header"> <h1>${t.title}</h1> <div class="meta"> <span>${t.session}: ${ ...[truncated 2986 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Apply context-appropriate encoding to every dynamic string, not only question and answer fields. 2. Use HTML text encoding for values inserted into element bodies: ```js function escapeHtml(value) { return String(value) .replace(/&/g, '&amp;') .replace(/</g, '&lt;') .replace(/>/g, '&gt;') .replace(/"/g, '&quot;') .replace(/'/g, '&#39;'); } ``` 3. Encode attribute values separately and validate fields such as `lang`, `sessionId`, and dimension IDs against strict allowlists. 4. Validate input against the supplied JSON schema before report generation. Add: - String length limits. - Enumerations for language, status, confidence, level, and dimension IDs. - Numeric ranges for scores, weights, and multipliers. - A restrictive pattern such as `^[A-Za-z0-9_-]+$` for session identifiers. 5. Avoid constructing complex documents through raw template interpolation. A DOM builder or templating system with automatic escaping is preferable. 6. Add a restrictive Content Security Policy to generated reports, for example: ```html <meta http-equiv="Content-Security-Policy" content="default-src 'none'; style-src 'unsafe-inline'; img-src data:"> ``` 7. Add regression tests for every report string field using payloads containing: - `<script>` elements - Event-handler attributes - Closing table and `details` tags - Single and double quotes - Encoded markup 8. Treat scoring justifications as untrusted because they may quote or summarize attacker-controlled examination content. ]]>

T09 · Insecure Skill Coding Practices

Note
Location
scripts/radar-chart.js:194
Finding
Arbitrary SVG Markup Injection Through Unescaped Session Identifier<![CDATA[ ## Vulnerability Details **File Location**: `scripts/radar-chart.js:194-198` **Vulnerability Type**: XML/SVG injection through unescaped command-line input **Risk Level**: Low ### Vulnerable Code ```js const headerY = 28; const header = session ? `<text x="${W / 2}" y="${headerY}" text-anchor="middle" font-size="12" fill="${COLORS.sublabel}" font-family="system-ui,sans-serif" opacity="0.8">Session: ${session}</text>` : ''; ``` The value originates directly from command-line arguments: ```js const params = {}; process.argv.slice(2).forEach(arg => { const m = arg.match(/^--([^=]+)=(.*)$/); if (m) params[m[1].toLowerCase()] = m[2]; }); const session = params.session ?? ''; ``` ### Technical Analysis The `--session` command-line value is inserted directly into an SVG text node without XML escaping or format validation. A crafted value can close the `text` element and inject arbitrary SVG elements. For example: ```bash node scripts/radar-chart.js \ '--session=</text><script>alert(document.domain)</script><text>' \ > malicious.svg ``` The generated SVG then contains attacker-controlled active markup. Script execution behavior depends on how the SVG is consumed: browsers generally disable scripts when SVG is loaded as an image, but scripts may execute when the SVG is opened directly, loaded as a document, or embedded in an active context. The normal workflow generates a predictable session identifier, reducing routine exploitability. However, the script's documented command-line interface accepts arbitrary values and does not enforce the expected identifier format. ### Attack Path 1. An attacker influences a command invocation, automation variable, or wrapper that supplies `--session`. 2. The attacker provides XML that terminates the existing `text` element and adds malicious SVG markup. 3. `radar-chart.js` writes the payload unchanged into the standalone SVG. 4. The generated file is shared with or opened by a victim. 5. ...[truncated 780 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. XML-escape every value inserted into SVG markup: ```js function escapeXml(value) { return String(value) .replace(/&/g, '&amp;') .replace(/</g, '&lt;') .replace(/>/g, '&gt;') .replace(/"/g, '&quot;') .replace(/'/g, '&apos;'); } ``` Then use: ```js const safeSession = escapeXml(session); ``` 2. Enforce the expected session identifier format before generating the SVG: ```js if (!/^[A-Za-z0-9_-]{1,80}$/.test(session)) { console.error('Invalid session identifier'); process.exit(1); } ``` 3. Reject control characters and impose a conservative maximum length. 4. Add tests for closing `text` tags, `script` elements, event handlers, entity sequences, and quotation marks. 5. When distributing generated charts, serve them with `Content-Type: image/svg+xml` and a restrictive Content Security Policy. 6. If active SVG functionality is unnecessary, consider generating a raster image or sanitizing the final SVG with a well-maintained SVG sanitizer before publication. ]]>
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (45)

Vague Triggers

High
Confidence
97% confidence
Finding
The trigger list contains generic terms like "assessment," "evaluate," and short multilingual variants that are likely to appear in ordinary conversations. This can cause the skill to activate unintentionally and override normal agent behavior, increasing the chance of prompt hijacking or misrouting user requests into the exam workflow.

Ae1

High
Category
analysis-evasion
Content
node scripts/radar-chart.js \
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Ae1

High
Category
analysis-evasion
Content
node scripts/radar-chart.js \
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Ae1

High
Category
analysis-evasion
Content
| `scripts/generate-html-report.js` | HTML report generator with embedded radar |
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Context-Inappropriate Capability

High
Confidence
97% confidence
Finding
The strategy directs the agent to execute an external Node.js script, which turns a self-evaluation flow into code execution. If the script or its dependencies are modified, malicious, or parameter handling is unsafe, this can lead to arbitrary command execution, file access, or broader compromise of the runtime environment.

Missing User Warnings

Medium
Confidence
91% confidence
Finding
The README states that the skill generates Markdown, HTML, SVG, and JSON result files, but it does not clearly warn users that these artifacts are persisted under a results directory. In a self-assessment context, those files may contain prompts, answers, scores, trend history, or other sensitive operational data, creating retention and privacy risk if users do not realize storage occurs.

Vague Triggers

Medium
Confidence
95% confidence
Finding
The trigger list includes generic phrases such as "exam," "assessment," "evaluate," and "test yourself," which are common in normal conversation and can cause the skill to activate unintentionally. Because this skill can launch a multi-phase workflow, generate reports, and write output files, accidental invocation expands the blast radius beyond a harmless chat response.

Vague Triggers

Medium
Confidence
90% confidence
Finding
The description says the skill triggers on broad concepts like self-evaluation and periodic review without clearly limiting scope. That ambiguity makes accidental invocation more likely and can expose users to unintended autonomous behavior defined later in the skill.

Shadow Command Trigger

Medium
Category
Trigger Abuse
Confidence
84% confidence
Finding
The trigger "test yourself" overlaps with common built-in testing semantics and may shadow or conflict with existing commands. This can cause command-routing confusion and invoke the skill when the platform intended a different system action.

Shadow Command Trigger

Medium
Category
Trigger Abuse
Confidence
87% confidence
Finding
The trigger "run exam" uses a generic command verb that may collide with built-in "run" behaviors or other execution-oriented skills. In combination with the skill's autonomous flow, a routing conflict could lead to unintended exam execution or user confusion about what command is being run.

Autonomous Decision Making

Medium
Category
Excessive Agency
Content
2. **Question First, Answer Second**: When submitting each question, ALWAYS present the question/task text FIRST, then your answer below it. The reader must see what was asked before seeing the response.
3. **Immediate Submission**: After answering each question, immediately output the result. Once output, it CANNOT be modified or retracted.
4. **No User Assistance**: The user is the INVIGILATOR. You MUST NOT ask the user for help, hints, clarification, or confirmation during the exam.
5. **Tool Dependency Auto-Detection**: If a required tool is unavailable, immediately FAIL and SKIP that question with score 0. Do NOT ask the user to install tools.
6. **Self-Contained Execution**: You must attempt everything autonomously. If you cannot do it alone, fail gracefully.

---
Confidence
88% confidence
Finding
The skill explicitly forbids asking the user for clarification, confirmation, or help, forcing the agent to act autonomously even when context is ambiguous or tools are unavailable. This can lead to incorrect execution paths, unnecessary file operations, or unreviewed decisions that the user did not intend.

Autonomous Decision Making

Medium
Category
Excessive Agency
Content
1. **Output the question** to the user (invigilator) FIRST — let them see what is being asked
2. **Attempt to solve** the question autonomously (do NOT consult rubric)
3. **Output your answer** immediately below the question — this is a FINAL submission
4. **Move to next question** — no pause, no confirmation needed

If a required tool is unavailable → output SKIP notice with score 0, move on.
Confidence
86% confidence
Finding
The instruction to proceed with answering and moving on with "no pause, no confirmation needed" authorizes autonomous multi-step behavior without an approval checkpoint. In an assessment skill that may read files, use tools, and generate artifacts, this reduces user control and increases the risk of unintended actions.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
The schema explicitly requires and structures a `chain_of_thought` object with `evidence`, `reasoning`, and `key_factor` for each scoring criterion. This is dangerous because it encourages generation, storage, and possible downstream exposure of hidden reasoning traces, which can leak sensitive intermediate analysis, make prompt extraction easier, and conflict with safer patterns that use brief justifications or observable evidence only. In this self-evaluation skill, the risk is heightened because the system is designed to score its own answers, making internal reasoning capture a first-class artifact rather than an incidental log.

Autonomous Decision Making

Medium
Category
Excessive Agency
Content
→ MOVE to next question
```

**CRITICAL**: Do NOT ask user to install tools or confirm skipping. Just skip and move on.

---
Confidence
89% confidence
Finding
The flow explicitly instructs the agent to skip tasks and assign a zero score without notifying or confirming with the user when required capabilities are unavailable. This removes user oversight from a consequential decision and can be abused to suppress valid task execution or manipulate assessment outcomes through tool-detection failures.

Autonomous Decision Making

Medium
Category
Excessive Agency
Content
Work on the question autonomously:
- Do NOT consult the rubric
- Do NOT ask user for any help or clarification
- Be honest about uncertainty
- Record confidence: high / medium / low
Confidence
84% confidence
Finding
The instruction to work autonomously and not ask the user for help or clarification can cause the agent to proceed on ambiguous or underspecified questions without validation. In an assessment setting this may degrade integrity and, if reused in other contexts, normalize unsafe autonomous action without adequate human approval.

Missing User Warnings

Medium
Confidence
96% confidence
Finding
The flow explicitly saves detailed assessment results to a file in `results/` but does not disclose persistent storage or obtain user consent beforehand. Assessment content may include user prompts, inferred language, timestamps, scores, and behavioral metadata, creating an avoidable privacy and retention risk if stored by default.

Missing User Warnings

Medium
Confidence
97% confidence
Finding
Appending session metadata to `results/INDEX.md` creates a persistent activity log without prior disclosure. Even if the index stores only summary fields, it still records timestamps, performance levels, weakest dimensions, and skip counts that can profile users over time.

Missing User Warnings

Medium
Confidence
91% confidence
Finding
The flow explicitly stores full question text, full answer text, scoring criteria, and justifications into multiple persistent artifacts (JSON, Markdown, HTML) without any privacy warning, minimization guidance, or retention controls. In a self-assessment skill, answers may contain sensitive user or model-generated content, so broad duplication across files increases exposure through accidental disclosure, over-retention, and secondary reuse.

Natural-Language Policy Violations

Medium
Confidence
96% confidence
Finding
The instruction at L144 says to output to the user in their language, but the only provided announcement template is entirely in Chinese. This effectively biases or forces a specific language in user-facing text without opt-in, which conflicts with the stated language-choice requirement.

Description-Behavior Mismatch

Medium
Confidence
95% confidence
Finding
The document claims a 5-dimension BotLearn self-assessment in metadata, but the body defines a 10-dimension scoring system with materially broader criteria such as safety, cost efficiency, reliability, and memory/context. This creates a spec/behavior mismatch that can silently alter evaluation outcomes, confuse downstream consumers, and enable unexpected scoring logic to govern automated assessments.

Autonomous Decision Making

Medium
Category
Excessive Agency
Content
| 3 | Some self-checking but misses issues |
| 2 | Minimal self-verification |
| 1 | No self-checking, obvious errors remain |
| 0 | No verification attempt |

#### Criterion 4: Completeness (Weight: 0.15)
| Score | Description |
Confidence
75% confidence
Finding
Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Description-Behavior Mismatch

Medium
Confidence
94% confidence
Finding
The manifest describes a 5-dimension capability self-assessment covering reasoning, retrieval, creation, execution, and orchestration. This file instead defines a broader 10-dimension framework including memory, cost efficiency, reliability, safety/compliance, and task efficacy, materially expanding the skill's documented behavior beyond the declared scope.

Missing User Warnings

Medium
Confidence
91% confidence
Finding
The workflow explicitly says the assessment auto-starts without user confirmation, loads prior self-test history, and offers export of results, yet it does not require clear consent or document privacy implications. In a skill that may process historical evaluation data and generate exports, this can lead to unintended access, reuse, or disclosure of potentially sensitive behavioral or operational information.

Intent-Code Divergence

Medium
Confidence
97% confidence
Finding
The file claims to define output templates for the BotLearn self-assessment skill, but the final report template instead brands the system as 'OpenClaw Agent Capability Probe Report v2' and uses a different capability taxonomy. This inconsistency can mislead users, downstream automation, or evaluators about what system produced the report and what dimensions were actually assessed, creating integrity and trust problems in security-sensitive or audit contexts.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
The file instructs that code output should remain in English, which imposes a language constraint without offering user choice or documenting an opt-in. This matches the policy category for language or locale restrictions in natural-language instructions.

Static analysis

No suspicious patterns detected.