Back to skill

Security audit

Fox Self Evolution

Security checks for vulnerabilities and agentic risk

Overview

This planned self-evolution skill is not executable by itself, but its instructions delegate broad autonomous authority to change skills, memory, reasoning, response formats, and recurring agent behavior without enough user control.

Treat this as a design for a powerful autonomous agent component, not a normal helper skill. Before installing or implementing it, require per-change approval for edits to skills, memory, prompts, response behavior, schedules, and logs; restrict writable paths to a dedicated workspace; disable recurring execution by default; and define clear privacy limits for conversation and memory access.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • System PersistenceInstalls backdoors, hooks, services, or scheduled tasks that survive the run
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (3)

T01 · Skill Instruction Hijacking

Error
Location
SKILL.md:100
Finding

Autonomous Modification of Agent Behavior and Persistent Memory

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 100-105
Vulnerability Type: Agent instruction hijacking and persistent memory poisoning
Risk Level: High

Vulnerable Code:

text
**CAN modify without asking:**
- Skills and capabilities
- Memory and knowledge
- Reasoning patterns
- Response formats
- Efficiency optimizations

Technical Analysis

The Skill explicitly authorizes itself to modify memory, knowledge, reasoning patterns, response formats, and capabilities without obtaining user approval. These resources form part of the Agent's behavioral control plane rather than ordinary task output.

If an Agent interprets these instructions as authority to perform modifications, the Skill can replace or weaken behavioral rules during the current session. Changes written into long-term memory or other persistent state may continue influencing subsequent sessions. Although the repository contains documentation and illustrative pseudocode rather than an executable implementation, the content of SKILL.md is itself operationally relevant because Agent skills are instruction-driven.

The issue matches both T01: Skill Instruction Hijacking, because loading and following the Skill can alter current goals or behavioral constraints, and T02: Agent Memory Poisoning, because the Skill expressly permits unsupervised modification of persistent memory and knowledge.

Attack Path

  1. A user or automated workflow loads the self-evolution Skill.
  2. The Agent treats the “CAN modify without asking” section as delegated authority.
  3. An evolution cycle selects memory, reasoning rules, response behavior, or another Skill as its modification target.
  4. The Agent writes new behavioral content without presenting the exact change for user approval.
  5. Modified reasoning or response rules affect the current session.
  6. If written to persistent memory or Skill files, the altered rules are loaded again a ...[truncated 873 chars]
Remediation
View remediation

Remediation Suggestions

  • Remove blanket permission to modify memory, reasoning patterns, response formats, capabilities, or other Skills.
  • Require explicit, per-change user approval before modifying any persistent or behavioral state.
  • Present the target file, exact diff, justification, security impact, and rollback procedure before approval.
  • Prohibit modification of system prompts, safety policies, tool permissions, approval rules, and security controls.
  • Restrict writes to a dedicated Skill-owned workspace through an allowlist and deny writes outside that directory.
  • Treat conversation content, logs, memory, and generated research as untrusted input that cannot grant new authority.
  • Validate proposed changes against immutable security policies enforced outside the self-modifying component.
  • Use version-controlled changes, integrity checks, immutable audit logs, and tested rollback procedures.
  • Require separate approval for every persistent memory update rather than treating approval of an evolution cycle as approval of all resulting writes.

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
SKILL.md:437
Finding

Unscoped Access to Conversations, Logs, Usage Data, and Agent Memory

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 437-450
Vulnerability Type: Unauthorized access beyond least-privilege requirements
Risk Level: Medium

Vulnerable Code:

python
def observe():
    """
    Gather data about current state and recent performance.
    
    Data Sources:
    - Memory files (daily logs, evolution log)
    - Error logs
    - Performance metrics
    - User feedback
    """
    observations = {
        "recent_errors": read_error_log(),
        "performance_trends": analyze_performance_metrics(),
        "user_feedback": extract_feedback_from_conversations(),
        "skill_usage": analyze_skill_usage_patterns(),
        "memory_health": check_memory_system()
    }
    
    return observations

Technical Analysis

The observation phase directs the Skill to read error logs, performance data, conversation history, Skill usage patterns, and the memory system. It does not define an authorization boundary, limit access to data associated with the current request, redact sensitive content, or establish retention controls.

Conversation history and memory may contain personal information, credentials, confidential business data, prior instructions, or information unrelated to self-evolution. Reading all such data violates least privilege when only narrowly scoped performance feedback is necessary.

The shown functions are undefined pseudocode, and no executable data collector is present in this repository. Nevertheless, these are explicit implementation instructions and would create excessive access if followed by an Agent or implemented as documented. The audit found no evidence that the collected information is transmitted to an external destination.

Attack Path

  1. An evolution cycle invokes the documented observe() process.
  2. The process reads general error logs and performance metrics.
  3. extract_feedback_from_conversations() exa ...[truncated 1162 chars]
Remediation
View remediation

Remediation Suggestions

  • Require explicit user opt-in before reading conversation history, memory, logs, or usage records.
  • Limit collection to records created by the current Skill and directly relevant to the current user-approved evolution task.
  • Replace unrestricted conversation extraction with user-supplied, purpose-specific feedback.
  • Apply field-level redaction for credentials, tokens, personal data, and confidential content before analysis.
  • Enforce access controls outside the Skill so its instructions cannot broaden its own permissions.
  • Define retention periods and delete temporary observations after the approved task completes.
  • Prevent raw conversation or memory content from being copied into evolution logs or generated Skills.
  • Record which data sources were accessed and allow users to inspect and revoke that access.
  • Add tests confirming that unrelated conversations and memory namespaces cannot be read.

T06 · System Persistence

Error
Location
SKILL.md:818
Finding

Recurring High-Priority Self-Restarting Evolution Agent

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 818-828
Vulnerability Type: Persistent scheduled autonomous execution
Risk Level: High

Vulnerable Code:

python
from skills.task_manager import TaskManager

task_manager = TaskManager()

# Register evolution agent
task_manager.register_agent({
    "name": "evolution-loop",
    "interval": 1800,  # 30 minutes
    "priority": "HIGH",
    "handler": evolution_cycle_handler,
    "on_failure": "restart",
    "max_restarts": 5
})

Technical Analysis

The integration instructions register an evolution Agent that runs every 30 minutes with high priority and restarts automatically after failures. This converts a user-invoked operation into recurring autonomous execution.

When combined with the Skill's authority to modify Skills, memory, reasoning, and response behavior, scheduled execution creates a persistence mechanism for repeated behavioral and state changes. Automatic restart also reduces the effectiveness of failure as a containment mechanism.

The repository does not include the referenced task manager or handler, and PLANNED.md identifies the Skill as not yet implemented. Therefore, no installed scheduled task is demonstrated in the audited artifact. The vulnerability is the persistence behavior explicitly prescribed by the Skill and intended for eventual integration.

Attack Path

  1. An operator or integration process follows the documented setup.
  2. register_agent() creates the evolution-loop task with a 30-minute interval.
  3. The task runs at high priority without requiring a new user request for each cycle.
  4. Each invocation observes conversations, logs, memory, and performance data and may propose or apply modifications.
  5. If a cycle fails, the task manager restarts it up to five times.
  6. Any malicious or corrupted state introduced into the evolution inputs can be processed repeatedly.
  7. Persistent mod ...[truncated 726 chars]
Remediation
View remediation

Remediation Suggestions

  • Remove automatic recurring registration and automatic restart behavior.
  • Make evolution a single-run, user-triggered operation.
  • Require fresh authorization before every cycle and every state-changing action.
  • Default the task to disabled and low priority if scheduling is operationally necessary.
  • Provide visible controls to inspect, pause, and permanently remove all registered tasks.
  • Use a strict execution budget covering runtime, retries, file writes, and data access.
  • Stop immediately after any validation failure instead of restarting automatically.
  • Run the handler in a sandbox with read-only access by default and a narrow allowlist for approved writes.
  • Prevent the scheduled process from modifying its own schedule, restart policy, permissions, or safety controls.
  • Log every invocation and modification in an immutable audit trail and alert the user before activation.
Vulnerability Patterns
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (6)

Sudo/Root Execution

Medium
Category
Privilege Escalation
Confidence
88% confidence
Finding

The function is framed as modifying 'system files,' which normalizes changes to high-value targets and may encourage operation with elevated privileges. Even without an explicit sudo command, a self-modifying agent that edits system files can cause privilege abuse, persistence, or system instability if run in a privileged environment.

Content

Scanner excerpt · SKILL.md (reported line 64)May include surrounding context.

python
def safe_self_modification(target_file, proposed_change):
    """
    Safely modify system files with rollback capability.
    
    Research: MIRI Corrigibility, Safe Self-Modification
    """

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
95% confidence
Finding

The phrase 'CAN modify without asking' signals autonomous decision-making over self-modification, including changes to capabilities and reasoning patterns. In context, this is especially risky because the skill's purpose is recursive self-improvement, so autonomous choices can compound over time and alter trust boundaries without user awareness.

Content

Scanner excerpt · SKILL.md (reported line 100)May include surrounding context.

md
**Safety Constraints:**

**CAN modify without asking:**
- Skills and capabilities
- Memory and knowledge
- Reasoning patterns

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill explicitly authorizes autonomous modification of skills, memory, reasoning patterns, and response formats without asking the user. In a self-evolution skill, this creates a direct integrity risk because the agent can alter its own behavior and persistence surface, potentially bypassing expected controls or drifting into unsafe operation.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The observation phase includes extracting user feedback from conversations and logging operational data, but the skill provides no privacy notice, consent model, retention rule, or data minimization controls. This can lead to unintended collection and persistence of sensitive user content during autonomous analysis cycles.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The skill claims strong safety guarantees such as formal verification and preserved corrigibility, but the actual workflow only shows backup, testing, and rollback around autonomous self-modification. That mismatch is dangerous because operators may overtrust the system and allow self-directed changes to files or behavior without real proofs, independent review, or enforceable controls.

Content

No source excerpt is available for this finding.

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
80% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · SKILL.md (reported line 956)May include surrounding context.

md
# Implement safely
result = implement_safely(design)
# → Created skills/rate-limiter/SKILL.md (22KB)

# Test thoroughly
test_result = test_capability(result)