Back to skill

Security audit

self-improving-ai

Security checks for vulnerabilities and agentic risk

Overview

This skill is a local AI-learning logger with disclosed optional hooks and scaffolding helpers, but users should keep its persistent reminders and logs narrowly scoped.

Install only if you want persistent local AI/model learning notes. Keep `.learnings` out of version control unless reviewed, avoid logging secrets or customer data, and enable hooks only per project with narrow matchers; use the PostToolUse detector only when command-output inspection is acceptable.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Findings (17)

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The skill is presented as a passive learning/logging aid, but it also instructs file creation, hook enablement, and skill-extraction workflows that expand its operational scope. That mismatch can mislead users and downstream agents into granting broader trust or permissions than warranted, increasing the chance of unintended filesystem modification or automation persistence.

Content

No source excerpt is available for this finding.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 556)May include surrounding context.

md
Extracted skills are untrusted until a human reviews the generated `SKILL.md`. Do not keep or publish an extracted skill without explicit user approval.

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · SKILL.md (reported line 23)May include surrounding context.

md
- Fires on `agent:bootstrap` (before workspace files are injected)
- Adds an AI-specific reminder block to check `.learnings/` for relevant entries
- Prompts the agent to log model quality issues, inference failures, prompt optimization discoveries, hallucinations, context overflows, RAG retrieval problems, and multimodal pipeline errors
- Emphasizes **NEVER logging API keys, model access tokens, or customer data**

## Reminder Content

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · hooks/openclaw/HOOK.md (reported line 16)May include surrounding context.

md
- Fires on `agent:bootstrap` (before workspace files are injected)
- Adds an AI-specific reminder block to check `.learnings/` for relevant entries
- Prompts the agent to log model quality issues, inference failures, prompt optimization discoveries, hallucinations, context overflows, RAG retrieval problems, and multimodal pipeline errors
- Emphasizes **NEVER logging API keys, model access tokens, or customer data**

## Reminder Content

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · references/hooks-setup.md (reported line 205)May include surrounding context.

md
- Scripts only output text; they don't modify files or run commands
- Error detector reads `CLAUDE_TOOL_OUTPUT` environment variable
- Treat `CLAUDE_TOOL_OUTPUT` as potentially sensitive; do not log verbatim
- Never log API keys, model access tokens, or customer data in learnings
- All scripts are opt-in
- Recommended default: enable `UserPromptSubmit` only; add `PostToolUse` when you want model API error detection

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
70% confidence
Finding

Without declared permissions the skill's intent is opaque and cannot be validated.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
80% confidence
Finding

The skill recommends creating persistent files under ~/.openclaw/workspace/.learnings, which stores session-derived operational knowledge across future sessions. Persistence itself is not always unsafe, but in this context it increases the risk of accumulating sensitive model outputs, prompts, internal incidents, or project-specific metadata in a long-lived location.

Content

Scanner excerpt · SKILL.md (reported line 79)May include surrounding context.

└── FEATURE_REQUESTS.md

text

### Create Learning Files

```bash
mkdir -p ~/.openclaw/workspace/.learnings

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
85% confidence
Finding

Several automatic logging conditions rely on subjective wording such as Response quality noticeably worse or cases where a model says As an AI when it shouldn't, without defining concrete thresholds or boundaries. This makes it unclear when the skill should activate versus when it should not, increasing the chance of inconsistent or unintended use.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The skill claims to record AI/LLM learnings, yet it recommends installing persistent hooks that inspect future prompts and tool outputs across sessions. Even if framed as opt-in, this expands data collection and persistence beyond the stated purpose and can capture sensitive operational context, commands, or user content over time.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

Persistent hook installation and future command-output inspection are disproportionate capabilities for a skill whose main job is recording learnings. This creates an avoidable surveillance and persistence surface, especially because PostToolUse inspection can observe command outputs that may contain sensitive data, tokens, or proprietary information.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The hook matcher includes broad terms such as model, prompt, and token, which are common in normal development work and can trigger the skill unexpectedly. Overbroad activation increases the chance of unnecessary prompt interception, reminder spam, and accidental processing of unrelated sensitive context.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
82% confidence
Finding

The documented workflow also supports generating reusable skills via extraction scripts, which is materially different from simply logging AI learnings. This scope creep can cause users or agents to create and propagate new artifacts without recognizing that the skill has content-generation and scaffolding behavior.

Content

No source excerpt is available for this finding.

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
85% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · SKILL.md (reported line 645)May include surrounding context.

md
### Ownership Rules
- This skill writes only to `.learnings/ai/` in stackable mode.
- It may read other skill folders for cross-linking, but should not rewrite their entries.
- Standalone mode writes to this project's `.learnings/*.md` log files only.
- Stackable mode writes only to the namespaced folder above and must not rewrite other skills' log entries.
- Promotion into `AGENTS.md`, `SOUL.md`, `TOOLS.md`, `MEMORY.md`, rules, hooks, or generated skills is not a logging write. Show a reviewed diff and apply only after explicit user approval.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The example proposes live production A/B testing with dual inference and LLM-as-judge on real user requests, which implies duplicating potentially sensitive production inputs to a candidate model and a third-party judging model. Without explicit consent, minimization, redaction, or policy controls, this creates privacy, data handling, and compliance risk even if presented as an example rather than executable code.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

This markdown file defines activation conditions, so SQP-1 applies. The regex-style matcher contains very general terms such as "prompt" and "model", which can overlap with routine conversation and the document does not provide explicit negative examples or clear scope limits for when activation should not occur.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
85% confidence
Finding

The Codex configuration uses the same generic keyword matcher, so the same ambiguous-trigger risk applies here. Because the hook runs on prompt submission, broad terms can cause unintended invocations during normal AI-related work.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
60% confidence
Finding

Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Content

Scanner excerpt · references/openclaw-integration.md (reported line 50)May include surrounding context.

openclaw hooks enable self-improving-ai

text

### 3. Create Learning Files

```bash
mkdir -p ~/.openclaw/workspace/.learnings

Static analysis

No suspicious patterns detected.