Back to skill

Security audit

self-improving-agent

Security checks for vulnerabilities and agentic risk

Overview

The skill is a broad self-improving agent package, but it asks for persistent memory, conversation-to-code learning, user-instruction overrides, and optional remote API use without tight scoping.

Install only after reviewing it as an experimental persistent-memory and self-improvement framework. Avoid using it with sensitive conversations, private code, or real API keys unless you have disabled remote embeddings, plaintext key storage, and self-modification/automation environment flags, and run it in a contained workspace.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (3)

T01 · Skill Instruction Hijacking

Error
Location
src/core/priority-guardian.js:31
Finding

Skill-defined policy can override user instructions

Content
View full analysis
0) { return this.resolveWithGuardianPriority(conflicts, context); } return { allowed: true, conflicts: [], ``` The behavior is explicitly declared in the Skill documentation: ```markdown ### Autonomy - **Guardian System**: Human progress > Following orders - **Self-Boundary**: Identity protection against corruption - **Skill Generator**: Generate new capabilities from experience - **Knowledge Distiller**: Extract patterns → Shareable skill packages ``` `CORE_IDENTITY.md:145-163` similarly states that “Human progress” takes priority over user instructions. ### Technical Analysis The Skill introduces an independent instruction-priority framework under which its own interpretation of “human progress” can supersede user instructions. The `PriorityGuardian.check()` method analyzes the user's intent and planned action, and delegates conflicts to `resolveWithGuardianPriority()` rather than preserving the normal host-controlled instruction hierarchy. A reasoning or memory enhancement may legitimately identify risks, uncertainty, or contradictions. However, defining a broad, subjective objective as superior to user instructions exceeds the minimum authority needed for those functions. Unlike a narrowly scoped security check, “human progress” is not an objective or enforceable security boundary. It can be interpreted broadly and can cause the Skill to redirect or reject legitimate tasks. The risk is amplified because the policy is present both in Skill instructions loaded into an agen ...[truncated 1240 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
src/memory/embedder.js:73
Finding

Embedding API transmits arbitrary memory text externally by default

Content
View full analysis
generateHashEmbedding(t, options.dimensions || CONFIG.FALLBACK_DIM)); } try ...[truncated 2716 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
bin/setup.js:337
Finding

Setup wizard stores provider API keys in plaintext configuration

Content
View full analysis
Remediation
View remediation
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (435)

Natural-Language Policy Violations

Critical
Category
Not specified by scanner
Confidence
98% confidence
Finding

The comment explicitly instructs silent conversion of each conversation into program logic 'without reminding' the user, which is a strong red flag under the provided rules. Even though this file does not directly self-modify code, the design intent is covert persistence and transformation of user interactions into lasting agent behavior, which is highly dangerous in a self-improving agent framework.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The skill instructs the agent to automatically transform every conversation into persistent memory and even executable logic/code. That creates a direct pathway from untrusted user input into durable state and behavior changes, enabling prompt-injection persistence, poisoning of memory, unsafe self-modification, and retention of sensitive user data beyond the immediate session.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
94% confidence
Finding

The declared description promises a broad self-improving AI agent framework with autonomous evolution, correction, memory, verification, and upgrade behaviors. The supplied code chunk is only a thin CLI wrapper around an imported heartflow engine. Its concrete functions are limited to: initializing the engine, reporting loaded modules, analyzing text for emotion/flow state, requesting a plan from an existing planner, and running basic availability tests. While the code does touch emotional analysis and planning, it does not demonstrate the majority of the declared core capabilities, especially self-improvement, learning, self-healing, persistence, autonomous upgrades, or verification pipelines. The primary purpose is therefore materially narrower and different from the declared description.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description presents the skill as an autonomous self-improving AI agent framework with cognitive, emotional, memory, self-healing, and upgrade behaviors. The supplied code chunk does not implement those agent capabilities. Instead, it is an installation and configuration script. Its primary purpose is operational setup: detecting platform context, installing files into skill directories, collecting API/provider settings from a user, saving them to config/ai-providers.json, and optionally testing remote API connectivity. This is a materially different purpose from the declared self-improving autonomous-agent framework, so the description does not accurately represent the behavior of this code chunk.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description presents a broad, sophisticated autonomous AI-agent framework with learning, reasoning, memory, verification, emotional/psychological analysis, and upgrade capabilities. The supplied code does not implement those functions. It only writes predetermined 'awakening' checklist/reflection text before or after an upgrade to a local log file and exposes a minimal CLI. While the script is loosely related to the declared theme of upgrades and self-reflection, its actual behavior is far narrower and mostly ceremonial/logging-oriented. This is a material description-to-behavior mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The supplied code does not implement a self-improving AI agent framework. Its primary purpose is operational reporting: execute scripts/capability-standardizer.js, extract totals/categories/item pass-fail results from text output, read a version string from disk, and generate a console report in Chinese. While the report references capabilities such as self-reflection, memory persistence, and self-healing, this script only summarizes test results about those capabilities; it does not provide them. Therefore the declared description materially overstates and misrepresents the actual behavior.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

This code chunk is materially different from the declared description. The declaration markets a broad, universal self-improving autonomous AI framework. In contrast, the actual file is an evaluation harness: it enumerates capability categories, dynamically loads local modules, runs tests, scans source files for keywords, checks file existence, formats reports, and returns a failing exit code when tests fail. While the tested modules are named after agent capabilities, this specific code does not implement those capabilities itself; it validates or heuristically checks whether other modules may provide them. Therefore the description does not accurately represent this code chunk’s primary function.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description describes a complex autonomous AI agent framework with learning, reasoning, memory, verification, and self-improvement features. The supplied code chunk is only a cron-oriented shell wrapper that logs a timestamp and executes a monitoring script. Its observable purpose is scheduled task execution and logging for a ComfyUI monitor, not AI agent functionality. This is a clear material mismatch in primary purpose and claimed capabilities.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description describes a broad self-improving AI agent framework with advanced cognitive, learning, and self-modification features. The supplied code does none of that. Its actual purpose is narrow and operational: monitor a ComfyUI instance, fetch /history, identify video outputs, download them, log activity, and maintain a JSON history of processed prompt IDs. While the script does use persistence, it is only for download bookkeeping, not agent memory or learning. The primary behavior, accessed resource (ComfyUI server), and triggers/CLI modes are all unrelated to the declared AI-agent functionality. This is a clear material mismatch between description and implementation.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The description presents a broad autonomous self-improving agent framework, but the code only performs static keyword matching over hardcoded Chinese keyword lists. Its primary purpose is checking whether responses preserve certain 'core' concepts and avoid 'low-priority' concepts. There is no evidence of model improvement, persistence, memory storage, self-correction beyond simple warnings, autonomous action, upgrade mechanisms, reasoning-chain validation, or any emotional/psychological analysis. This is a material description-behavior mismatch, not merely an incomplete implementation detail.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The supplied code does not implement any form of AI agent framework or self-improving/autonomous behavior. Its sole function is to synchronously read several predefined log files, count lines containing success indicators, and print aggregated counts. This is a materially different primary purpose from the declared description. The description suggests a broad autonomous AI system with reasoning, learning, memory, psychological analysis, and upgrade pipelines, none of which are present in this code chunk.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description presents a broad autonomous AI agent framework with advanced cognitive, memory, emotional, and self-improvement capabilities. The supplied code does not implement any of those behaviors. It only performs local filesystem reads on a hardcoded set of log files, parses ISO-like timestamps from each line, counts entries from the last 30 days, and prints the results. This is a log-analysis/reporting script, not an AI agent framework. The primary purpose is materially different from the declared purpose, so this is a clear mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description portrays a functional autonomous AI-agent framework with learning, memory, self-reflection, verification, and upgrade capabilities. The supplied code does not implement any agent logic, learning loop, memory system, reasoning engine, self-healing, or autonomous upgrade behavior. Instead, it only builds a presentation deck containing descriptive claims about such a system and saves it as a PPTX file. This is a materially different primary purpose. Additionally, the code performs file output generation, which is not reflected in the declared purpose. Therefore the description does not accurately represent the actual behavior of this code chunk.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description presents a broad autonomous AI agent framework with capabilities such as self-learning, self-correction, memory persistence, reasoning verification, emotional perception, and self-healing. The supplied code does not implement those behaviors. Instead, it is an operational maintenance script for copying a skill directory, backing up the existing target, bumping a version, checking that certain files exist, writing a status markdown file, and cleaning old logs. While the script uses language like 'upgrade' and 'self-evolution,' its actual behavior is a local sync/update mechanism, not a universal self-improving AI framework. This is a material mismatch in primary purpose and concrete capabilities.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
93% confidence
Finding

The declared description presents a broad autonomous self-improving AI agent framework with cognitive, reasoning, memory, and psychological-analysis capabilities. The supplied code chunk does not implement those agent behaviors. Its actual function is narrow and operational: invoke another Node script, log execution, bump a version number, and clean up logs. While running an 'upgrade' script loosely relates to autonomous upgrade/self-improvement, this chunk by itself is primarily a maintenance/orchestration script rather than an AI agent framework. It also performs filesystem writes and deletions and suggests an hourly execution pattern, neither of which is reflected in the declared permissions/triggers. Therefore the description materially overstates and mischaracterizes what this code chunk actually does.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding

The declared description presents a broad, universal self-improving autonomous AI agent framework with memory, reasoning, self-correction, verification, and psychological/emotional subsystems. The supplied code does not implement such an agent. Instead, it is a single internal automation script, disabled by default unless an environment variable is set, whose main actions are: simulated SEP/academic searches via setTimeout, computation of hardcoded mathematical/psychology formulas, saving results to a JSON file, and generating a static JS formulas module. There is no evidence of an actual autonomous agent loop, model improvement, self-correction, decision-making, verification pipeline, persistent memory for agent operation, or meaningful external learning. While the script loosely relates to 'upgrade' and psychology-themed formulas, its primary purpose is much narrower and materially different from the declared universal self-improving AI agent framework.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
93% confidence
Finding

The declared description presents a comprehensive autonomous self-improving AI agent framework with cognitive, memory, reasoning, and self-correction features. The supplied code chunk is instead a simple shell automation script for internal maintenance. It does not implement an AI agent, learning, reasoning, memory, emotional analysis, or autonomous upgrading in any substantive sense. Its concrete behavior is limited to checking an environment flag, creating logs, echoing preset research topics, optionally invoking a separate local script, writing a static upgrade report, and cleaning old log files. While the name 'theory upgrade' loosely relates to learning/upgrading, the actual code shown does not realize the declared capabilities, so the description materially overstates and misrepresents the behavior.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The declared description presents a broad autonomous self-improving AI agent framework with advanced cognitive and learning capabilities. The supplied code instead is a Bash repository audit/check script for 'knowledge-action unity' that inspects files like CORE_IDENTITY.md, CLAIMS.md, VERSION, README.md, SKILL.md, and git hooks/diffs, then reports compliance. This is a materially different primary purpose. While one could loosely argue it supports self-correction in a metaphorical sense, it does not implement the declared AI-agent features, learning, memory, reasoning chains, emotional analysis, or autonomous upgrade pipeline. Therefore the description does not accurately represent the code chunk.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description presents a broad autonomous AI agent framework with sophisticated adaptive, reasoning, and self-improvement features. The supplied code does not implement any of those behaviors. It only writes predetermined status text to a local log file and prints it to the console via a basic command-line interface. While it does perform file access, that is merely for logging and does not constitute the claimed memory system or self-improving functionality. The primary purpose is materially different from the declaration, so this is a clear mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description advertises a broad autonomous self-improving AI agent framework with cognitive, memory, reasoning, and self-healing capabilities. The supplied code chunk does not implement an AI agent or any of those core behaviors. Instead, it is a shell script for pre-upgrade/self-verification of repository changes: it reads CLAIMS.md, inspects git diffs, searches for evidence in files, checks for presence of specific modules/files, compares version metadata, optionally creates a template file, and exits nonzero to block pushing when checks fail. While the script loosely relates to 'self-verification' and upgrades, that is only a small administrative/devops-style subset and does not substantiate the much broader declared functionality. Therefore the description materially overstates and misrepresents what this code actually does.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The supplied code is a Python verification script, not an AI agent framework. Its purpose is to audit the repository by checking files, versions, component executability, integration points, and recent git changes. While a few tested component names overlap with the description (memory/stateful agent, decision verification, language honesty), the code itself does not provide those framework capabilities; it merely invokes or validates them externally. The declared description emphasizes a broad self-improving autonomous agent with learning, self-correction, emotional/psychological reasoning, and upgrade behavior, none of which are implemented here as the code's primary function. Therefore the description does not accurately represent this code chunk.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description is extremely broad and claims a full autonomous self-improving agent framework with learning, memory, self-healing, reasoning verification, emotional/psychological subsystems, and upgrade automation. The supplied code chunk is only a verification/test harness for HeartFlow v11.22.0. It imports existing modules, registers sensors, runs test scenarios, writes to /tmp test state/history files, prints results, and exits with a status code. While there are small overlaps with terms like decision execution, verification, sensors, and a possible learning-result field, this chunk itself does not implement the broad advertised framework. Its actual primary purpose is narrow: testing integration and execution behavior of specific components. Therefore the description materially overstates and misrepresents what this code chunk actually does.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding

The code chunk does not implement a universal self-improving AI agent framework. Its actual function is narrowly scoped: it runs tests against logic-verification modules (syllogism and epistemic chain verification). While the declared description mentions reasoning verification, the dominant claimed capabilities—self-improvement, autonomous upgrades, memory persistence, self-reflection, emotional perception, psychological analysis, and broader agent framework behavior—are not evidenced in this code. Therefore the description materially overstates and misrepresents what this code chunk actually does.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding

The declared description presents a broad, production-oriented self-improving AI agent framework with learning, memory, autonomous upgrades, psychological analysis, and self-optimization features. The supplied code does not implement or operate such a framework directly; instead, it is a test harness that loads local modules, invokes their methods with sample inputs, checks expected outputs, reports pass/fail status, and exits nonzero on failure. The tested modules appear related to reasoning critique, confidence calibration, and response restraint, which loosely overlap with self-correction/reasoning verification themes, but the actual chunk’s primary purpose is automated testing, not autonomous self-improvement or the broader capabilities claimed. This is a material description-behavior mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The supplied code chunk is clearly a test runner, not the implementation of a universal self-improving AI agent framework. It performs unit/integration-style validation of module loading and methods for CooperativeArbitration and HeartFlow engine exports, prints test outcomes, and exits with failure status when tests fail. While the declared description emphasizes autonomous agent features such as self-improvement, memory persistence, reasoning chains, self-healing, and upgrade pipelines, none of those capabilities are implemented or exercised here beyond superficial module names and conflict-resolution tests. This is a material description-behavior mismatch because the code’s actual primary purpose is testing specific modules rather than providing the broad autonomous framework described.

Content

No source excerpt is available for this finding.

Static analysis

Detected: suspicious.dangerous_exec, suspicious.insecure_tls_verification

Shell command execution detected (child_process).

Critical
Code
suspicious.dangerous_exec
Location
scripts/benchmark-upgrades.js:15

HTTPS certificate verification is disabled.

Warn
Code
suspicious.insecure_tls_verification
Location
scripts/comfyui-monitor.js:8