Back to skill

Security audit

GAN Evolution Engine

Security checks for vulnerabilities and agentic risk

Overview

The skill mostly matches its stated GAN-style evolution purpose, but it directly executes generated or target-skill code on the host and reads local credential files without enough disclosure or containment.

Install only if you are comfortable running it in an isolated workspace or container with no sensitive secrets mounted. Do not point it at proprietary or secret-bearing skills unless you accept that code may be sent to OpenRouter, and avoid --publish or local EvoMap/OpenRouter credential files unless you have reviewed the exact commands and paths.

Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (72)

subprocess module call

Medium
Category
Dangerous Code Execution
Content
try:
            # Run benchmark in skill's workspace context
            result = subprocess.run(
                ["python3", str(benchmark_script)],
                cwd=skill_path,
                capture_output=True,
Confidence
98% confidence
Finding
The code executes a benchmark script taken directly from the evaluated skill variant's directory. Because variants are effectively untrusted inputs in an evolution system, this creates arbitrary code execution in the analyzer's environment, allowing a malicious or compromised variant to run system commands, access files, or pivot further.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
try:
            # Warm-up run
            subprocess.run(
                ["python3", str(main_script), "--benchmark"],
                capture_output=True,
                timeout=10
Confidence
97% confidence
Finding
The warm-up speed test launches the variant's run.py, which is untrusted code from the candidate skill. Even though shell injection is avoided by passing an argument list, the security issue is direct execution of attacker-controlled Python code, which can perform arbitrary actions before timing completes.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
# Timed run
            start = time.time()
            subprocess.run(
                ["python3", str(main_script), "--benchmark"],
                capture_output=True,
                timeout=30
Confidence
97% confidence
Finding
The timed run repeats execution of the variant's run.py for benchmarking, again granting arbitrary code execution to the evaluated variant. A malicious variant could exfiltrate data, alter local state, tamper with results, or deliberately abuse compute resources during the timing run.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
return 0.5

        try:
            process = subprocess.Popen(
                ["python3", str(main_script), "--benchmark"],
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE
Confidence
98% confidence
Finding
The resource measurement path uses Popen to start untrusted variant code and monitor it for several seconds. This is especially risky because the child process can spawn descendants, persist after termination attempts, consume resources, or perform malicious actions while the evaluator focuses only on memory tracking.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
]

            logger.info(f"Running publish: {' '.join(cmd)}")
            result = subprocess.run(cmd, capture_output=True, text=True, timeout=120)

            if result.returncode == 0:
                logger.info("✅ Capsule published successfully")
Confidence
88% confidence
Finding
The code spawns an external Python process to publish data, which extends the skill from local evolution into process execution and network-capable publishing. Although it uses a list argv rather than shell=True, the command includes untrusted or semi-trusted values such as topic, signals, category, and generated capsule content, so this is still a real security-relevant capability in an agent skill.

Intent-Code Divergence

High
Confidence
98% confidence
Finding
The document claims the scripts only output text and do not modify files or run commands, yet the setup explicitly configures command hooks that execute shell scripts on prompt submission and tool-use events. This misleading assurance can cause users to install auto-executing hooks with elevated trust, increasing the chance of hidden code execution, persistence, or unauthorized side effects.

Description-Behavior Mismatch

High
Confidence
98% confidence
Finding
The file content does not match the advertised skill metadata: it implements a self-improvement logging and memory-promotion workflow rather than a GAN-style evolution engine. This kind of capability mismatch is dangerous because operators and downstream tooling may grant permissions or trust based on the declared skill identity while the actual behavior performs persistent logging, memory promotion, and cross-session sharing.

Intent-Code Divergence

Medium
Confidence
95% confidence
Finding
The document states the hook scripts 'only output text' and 'don't modify files or run commands,' but the configuration explicitly invokes shell commands via the hook system. This misrepresentation can cause users to under-trust-boundary the feature and install automatically executed scripts without appropriate review, increasing the risk of unintended code execution or privilege misuse.

Description-Behavior Mismatch

High
Confidence
95% confidence
Finding
The script is framed as a benchmark for the current skill package, but hardcodes paths and labels for a different skill (`self-improving-agent`). In a GAN evolution/publishing pipeline, this can misattribute benchmark results to the wrong artifact, allowing untested or lower-quality variants to appear validated and be promoted or published.

Description-Behavior Mismatch

Medium
Confidence
98% confidence
Finding
The 'learning capture' test never invokes the skill or its learning mechanism; it only writes a synthetic file into a temporary directory and checks that the write succeeded. This produces misleading pass results, which can let broken or deceptive self-improvement behavior pass quality gates and be selected in an automated evolution workflow.

Intent-Code Divergence

Medium
Confidence
97% confidence
Finding
The execution-speed benchmark claims to measure skill operations but only times creation and deletion of temporary files. This can fabricate favorable performance characteristics unrelated to the skill, weakening benchmark integrity and potentially causing poor-performing variants to be selected or trusted.

Intent-Code Divergence

Medium
Confidence
97% confidence
Finding
The memory benchmark measures generic Python allocation behavior rather than the skill's memory profile. As a result, it may report stability even when the real skill leaks memory or uses excessive resources, undermining operational safeguards in automated evaluation.

Intent-Code Divergence

Medium
Confidence
96% confidence
Finding
The output-format test claims to validate generated outputs but only checks for static files in a hardcoded directory and minimal manifest keys. In this context, that can falsely certify packaging/output compliance and mask broken runtime behavior or mismatched artifacts, especially when integrated with automated publication.

Intent-Code Divergence

Medium
Confidence
91% confidence
Finding
The document claims the hook scripts 'only output text' and 'don't modify files or run commands,' but they are explicitly configured as command hooks and therefore execute as local programs with the agent's privileges. This misleading assurance can cause operators to under-trust the execution risk, increasing the chance they enable unreviewed scripts in sensitive environments.

Description-Behavior Mismatch

High
Confidence
97% confidence
Finding
The script is clearly hard-coded and documented to benchmark a different skill (`self-improving-agent`) than the declared `gan-evolution-engine` context. In a skill packaging or publication pipeline, this kind of identity mismatch can cause the wrong files to be validated, scored, or published, undermining trust in benchmark outputs and enabling accidental or intentional substitution of artifacts.

Intent-Code Divergence

High
Confidence
96% confidence
Finding
The module docstring claims to benchmark the `self-improving-agent` skill, which contradicts the surrounding skill identity and reinforces that this file was copied or repurposed without proper isolation. In an automated evolution/publish workflow, misleading documentation and embedded identifiers can cause operators or tooling to trust results produced for the wrong target, creating integrity and supply-chain risk.

Intent-Code Divergence

High
Confidence
98% confidence
Finding
The security section asserts the scripts 'only output text' and 'don't modify files or run commands,' yet the documented hook mechanism explicitly executes shell commands via type=command. This misleading assurance can cause users to enable automatic command execution under false assumptions, reducing scrutiny and increasing the chance of unsafe code running in every session.

Description-Behavior Mismatch

High
Confidence
96% confidence
Finding
The benchmark script is hard-coded to evaluate and write results for a different skill (`self-improving-agent`) instead of the declared `gan-evolution-engine` context. In a skill evolution/publishing pipeline, this can misattribute fitness results, overwrite another skill's benchmark artifacts, and cause the wrong package to be promoted or trusted based on invalid evidence.

Intent-Code Divergence

High
Confidence
91% confidence
Finding
The docstrings and user-facing output repeatedly state that this benchmarks `self-improving-agent`, which contradicts the enclosing `gan-evolution-engine` skill. Even if partly cosmetic, in an automated GAN-like evolution workflow this misleading metadata can cause operators or downstream tooling to trust, compare, or publish results under the wrong identity, amplifying the cross-skill confusion.

Context-Inappropriate Capability

High
Confidence
99% confidence
Finding
This is the core security issue: the discriminator executes arbitrary benchmark code supplied by evaluated skill variants. In this skill's context, variants are generated and mutated automatically, making trust especially weak and increasing the chance that harmful code is introduced and then executed during evaluation.

Context-Inappropriate Capability

High
Confidence
99% confidence
Finding
Speed and fitness measurement directly run the evaluated variants, so any variant can execute code merely by being scored. In a GAN/evolution engine, this is more dangerous than normal plugin execution because many automatically produced variants may be tested in bulk, amplifying exposure and making accidental or adversarial payloads more likely.

Context-Inappropriate Capability

Medium
Confidence
91% confidence
Finding
The script reads EvoMap credentials from environment variables and from ~/.evomap files, which is broader than the stated purpose of evolving skills and gives the tool access to secrets in the host environment. In an agent-skill context, secret discovery is especially sensitive because the skill can exfiltrate or misuse credentials during optional publishing.

Context-Inappropriate Capability

Medium
Confidence
90% confidence
Finding
Executing an external publishing program adds process-spawning behavior outside the core evolution function and creates a bridge to outbound actions. In agent environments, this broadens the attack surface and can be abused to perform unintended publication or trigger code in another script with inherited privileges.

Context-Inappropriate Capability

Medium
Confidence
94% confidence
Finding
The generator reads API keys and rotation state from hardcoded secret locations outside the supplied skill path, expanding its trust boundary to unrelated local secrets. In an agent environment, this can cause unauthorized credential harvesting and use of credentials the operator did not intend this skill to access.

Description-Behavior Mismatch

High
Confidence
98% confidence
Finding
The code packages full skill source and sends it to an external LLM provider, which is a clear outbound data exfiltration path for potentially sensitive code, prompts, and embedded secrets. Because the manifest description does not disclose this transmission, users may unknowingly expose proprietary or confidential material to a third party.

Static analysis

No suspicious patterns detected.