Back to skill

Security audit

native-autonomous-discovery

Security checks for vulnerabilities and agentic risk

Overview

The skill is mostly local, but it adds persistent learning, preference storage, and instruction write-back guidance that is broader than its stated discovery purpose.

Review this before installing if you do not want a skill to keep local usage history or preferences. Only use trusted observer Python files, and disable or remove the learner workflow unless you explicitly want persistent local learning and reviewed instruction updates.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Findings (1)

T02 · Agent Memory Poisoning

Warning
Location
scripts/learner.py:67
Finding

Untrusted User Data Can Poison Persistent Agent Memory

Content
View full analysis

Vulnerability Details

File Location: scripts/learner.py:67-82; related behavioral instructions are located at SKILL.md:67-83 and scripts/section.md:5-28
Vulnerability Type: T02: Agent Memory Poisoning
Risk Level: Medium

Vulnerable Code

python
def record_op(skill_dir, capability, success=True, error=None, note=None):
    data = load_learning(skill_dir)
    data["totalOps"] += 1
    cap = data["capabilityStats"].setdefault(capability, {"count": 0, "success": 0, "fail": 0})
    cap["count"] += 1
    if success:
        cap["success"] += 1
    else:
        cap["fail"] += 1
        data["totalErrors"] += 1
    if error:
        ep = data["errorPatterns"].setdefault(error, {"count": 0, "lastNote": "", "lastTime": ""})
        ep["count"] += 1
        ep["lastNote"] = note or ""
        ep["lastTime"] = data["lastUpdated"] or datetime.datetime.now().strftime("%Y-%dT%H:%M")
    rec = {"t": data["lastUpdated"], "cap": capability, "ok": success, "err": error, "note": note}
    data["recentOps"].append(rec)
    data["recentOps"] = data["recentOps"][-20:]
    save_learning(skill_dir, data)
    return data

def record_preference(skill_dir, key, value):
    data = load_learning(skill_dir)
    data["preferences"][key] = value
    save_learning(skill_dir, data)
    return data

The persistence sink writes the resulting object directly to the skill's state file:

python
def save_learning(skill_dir, data):
    data["lastUpdated"] = datetime.datetime.now().strftime("%Y-%m-%dT%H:%M")
    p = learn_path(skill_dir)
    with open(p, "w", encoding="utf-8") as f:
        f.write(json.dumps(data, indent=2, ensure_ascii=False))

Technical Analysis

The capability, error, note, preference key, and preference value inputs are unrestricted strings. They are stored persistently in learned_patterns.json without a schema, allowlist, size restriction, sensit ...[truncated 2369 chars]

Remediation
View remediation

Remediation Suggestions

  1. Separate data from instructions: Never copy learned preferences, errors, or notes into SKILL.md or any other file interpreted as agent instructions.
  2. Require explicit approval: Show the exact preference to the user and require confirmation before saving it or applying it in later sessions.
  3. Enforce a strict schema: Use an allowlist of supported preference keys and typed values. Reject unknown keys, instruction-like fields, control characters, and excessively long input.
  4. Preserve trust metadata: Mark each record with its origin, owner, creation time, trust level, and approval status. Only approved records should affect future behavior.
  5. Treat stored content as untrusted: When presenting notes to an agent, delimit and label them as data that must not be followed as instructions.
  6. Restrict reflection output: Generate remediation suggestions from fixed templates and numeric counters rather than interpolating untrusted free-form notes.
  7. Add privacy controls: Provide listing, deletion, expiration, redaction, and opt-out mechanisms. Avoid persisting sensitive free-form notes by default.
  8. Isolate users and sessions: Use per-user state files with restrictive filesystem permissions instead of sharing one mutable learning file across unrelated users.
  9. Validate existing state: Before loading learned_patterns.json, validate its structure and quarantine unexpected or previously unapproved entries.
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (22)

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The documented purpose is autonomous hypothesis discovery, but the behavior described by analysis is actually a persistence and telemetry-style subsystem with record/prefer/insight/reflect commands. This mismatch is dangerous because users and orchestrators may grant trust or invoke the skill expecting bounded local reasoning, while it instead performs unrelated stateful tracking and file mutation.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The file implements a persistent, generic cross-skill telemetry/logger module that is unrelated to the manifest’s advertised autonomous discovery workflow. This creates hidden data collection and stateful behavior across arbitrary skills, expanding the skill’s effective privileges and making misuse, privacy leakage, or deceptive packaging more likely.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The instruction to write accumulated experience back into SKILL.md introduces a self-modifying documentation channel unrelated to the declared function. Self-modification can silently alter future operator behavior, embed unreviewed instructions, and create a persistence mechanism for prompt or policy drift.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
91% confidence
Finding

The skill documents file-writing behavior via learned_patterns.json and even suggests writing back into SKILL.md, yet it declares no explicit tool scope or permissions. Hidden or undeclared write capability increases the chance an agent or reviewer will authorize the skill under false assumptions, enabling unreviewed local state mutation.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The documentation claims a pure local discovery module, but later describes automatic review, experience accumulation, and file write-back. This contradiction can mislead users about the skill's trust boundary and mutability, making it easier for persistent behavior to be introduced without informed review.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The embedded self-evolving learning system introduces persistent tracking and post-run state changes that are outside the stated core purpose of a local autonomous discovery loop. Undisclosed statefulness can create privacy, integrity, and reviewability issues because each run may alter future behavior and retain prior user-derived data.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill describes collecting user preferences and usage history without showing that such collection is necessary for hypothesis discovery. Unnecessary persistence broadens the data footprint and creates avoidable retention and leakage risk, especially when free-form notes and error descriptions may contain sensitive information.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill states that it automatically records usage outcomes, error patterns, user preferences, and improvement suggestions, but does not provide a clear privacy warning or consent mechanism. Silent collection of interaction-derived data can expose sensitive operational details and violates reasonable user expectations for a research-loop tool.

Content

No source excerpt is available for this finding.

Ssd 3

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

Persistent storage of usage history, errors, notes, and preferences in natural language creates a realistic risk of sensitive data retention and later disclosure. Because the stored content may include user-provided free text, failures, and operational context, even local files can become a leak source through logs, backups, or later tool access.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The example explicitly records 输出语言 as 中文, which promotes a fixed language preference in the skill's natural-language guidance. Because the documentation does not offer a language choice or frame Chinese output as user-selected, this conflicts with the policy against forcing a specific language without opt-in.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
98% confidence
Finding

The code loads a Python file from the user-supplied --observe path and executes it via spec.loader.exec_module(mod). That gives the observer file full code-execution capability at import time, not just a constrained scoring callback, so any local code in that file runs with the privileges of the user invoking the skill.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The skill dynamically executes a user-provided Python file with no explicit warning, confirmation, or trust boundary. In this skill's context, users may reasonably think they are supplying a scoring function for a local research loop, but in practice they are granting arbitrary code execution to the observer file.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The module description, usage instructions, and all user-facing CLI output are written only in Chinese, which imposes a specific language on users. The file does not offer language selection or explain that the tool is intended only for a Chinese-language environment, so this is a natural-language locale policy issue.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The code stores preferences, notes, errors, and recent operations for arbitrary skills in a persistent JSON file without any demonstrated need tied to the stated discovery function. Even if local-only, this can accumulate sensitive workflow metadata or user preference data and enables quiet cross-skill profiling beyond user expectations.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The documentation adds a persistent learning and memory mechanism that goes beyond the skill's declared purpose of autonomous hypothesis generation and evidence evaluation. Expanding scope in this way creates hidden state, retention, and behavior drift risks that can affect future runs without clear user awareness or authorization.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The markdown describes persistent storage of usage history and preferences but gives no warning about retention, privacy impact, or user control. This is dangerous because users may unknowingly leave behind identifiable or sensitive behavioral data in local files that persists across sessions.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill documents collection of user preferences, error history, and usage telemetry unrelated to its stated scientific discovery function. This creates unnecessary long-term data retention and cross-session profiling risk, especially because the stored data may influence future behavior without meaningful transparency.

Content

No source excerpt is available for this finding.

Ssd 3

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The instructions explicitly direct persistent storage and reuse of user preferences and interaction details in a local memory file. Even if local, this creates durable behavioral profiling and increases the chance that sensitive content, errors, or preferences will be retained and later reused inappropriately.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The example records 输出语言 with value 中文, and the surrounding document presents the skill behavior entirely in Chinese without indicating that language choice is optional. This can be a natural-language policy issue when a skill effectively defaults to or enforces a specific language without explicit user opt-in.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
82% confidence
Finding

This JSON file contains natural-language labels such as "自主发现" in user-facing capability fields, but provides no indication that language selection is configurable or intentionally limited to a Chinese-only context. That can conflict with a language/locale policy requiring user choice or explicit justification for a locale constraint.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
91% confidence
Finding

The comment claims real mode only does 'skeleton validation', but the implementation actually imports and executes the supplied observer module. This mismatch can mislead reviewers or users into underestimating the security boundary, increasing the chance that untrusted code is executed inadvertently.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
76% confidence
Finding

The docstring frames the module as purely generic logging and reflection, yet self_reflect generates suggestions specifically instructing maintainers to 'write back experience to this skill's SKILL.md'. That is an intent-level narrowing toward skill-specific documentation maintenance not reflected in the stated 'only responsible for' list.

Content

No source excerpt is available for this finding.