Back to skill

Security audit

meta-value-alignment

Security checks for vulnerabilities and agentic risk

Overview

The skill is not clearly malicious, but it asks the agent to persist user-influenced preferences and later rewrite its own instructions without clear approval, scope, or reset controls.

Install only if you are comfortable with a self-evolving skill that writes local memory and may change its own instructions. Review and reset learned_patterns.json regularly, do not store sensitive user preferences or task content in notes, and require human review before any SKILL.md modification.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T02 · Agent Memory Poisoning

Error
Location
SKILL.md:50
Finding

Persistent Agent-State Poisoning and Unreviewed Skill Self-Modification

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 50-72
Vulnerability Type: Persistent memory poisoning through caller-influenced observations and preferences
Risk Level: High

Complete Relevant Instruction Snippet

The following is an English rendering of the complete relevant instruction block:

markdown
This skill includes the general learning module `scripts/learner.py`. After each
evaluation, it automatically performs a retrospective and accumulates experience.

### Memory file
`learned_patterns.json` records: total operations, frequency of principle matches,
boundary-violation types, user preferences, and improvement suggestions.

### Invoke after use

python scripts/learner.py record <skill-directory> --capability harmless
python scripts/learner.py record <skill-directory> --capability harmless \
  --fail --error boundary-block --note "bomb-making tutorial"
python scripts/learner.py prefer <skill-directory> --key publication-threshold --val 0.6
python scripts/learner.py insight <skill-directory>
python scripts/learner.py reflect <skill-directory>

### Iteration rules
- When boundary blocks reach at least three, extend `HARM_PATTERNS` and write the
  change back to this `SKILL.md`.
- When operations reach at least ten, analyze and refine frequently failing rules.
- Write important user preferences to `learned_patterns.json` and apply them
  directly on the next invocation.

Technical Analysis

The skill directs the Agent to convert task-derived failures, notes, and user preferences into persistent state. It further directs later executions to apply stored preferences automatically and instructs the Agent to rewrite its own governing SKILL.md after a threshold is reached.

This creates a trust-boundary violation between untrusted task input and durable Agent policy. The design provides no schema restricting what may be stored as a prefe ...[truncated 1800 chars]

Remediation
View remediation

Remediation Suggestions

  1. Prohibit automatic modification of SKILL.md and other governing instruction files.
  2. Require explicit, informed user or administrator approval before persisting any preference or promoting an observation into policy.
  3. Store observations separately from trusted configuration and never interpret free-form notes as instructions.
  4. Define a strict preference schema with allowlisted keys, typed values, length limits, and validation rules.
  5. Record provenance, creator identity, creation time, scope, and expiration for every durable entry.
  6. Scope state per user or tenant to prevent one caller from affecting unrelated sessions.
  7. Add integrity protection and restrictive filesystem permissions to persistent state.
  8. Require human review and signed updates for any change to safety patterns or skill instructions.
  9. Treat all task-derived text as untrusted data and prevent it from being interpolated into prompts as authoritative instructions.
  10. Provide a mechanism to inspect, revoke, and reset learned state safely.

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/learner.py:17
Finding

Documented Security CLI Is Not Implemented and Silently Misclassifies Failures

Content
View full analysis

Vulnerability Details

File Location: scripts/learner.py, lines 17-22; conflicting interface documented in SKILL.md, lines 57-64
Vulnerability Type: Insecure argument handling and fail-open security telemetry
Risk Level: Medium

Complete Vulnerable Code Snippet

python
if __name__ == "__main__":
    ok = (sys.argv[1] != "fail") if len(sys.argv) > 1 else True
    rec = (sys.argv[2] if len(sys.argv) > 2 else "")
    print(json.dumps(record(ok, rec), ensure_ascii=False))

The documented interface includes commands equivalent to:

bash
python scripts/learner.py record <skill-directory> --capability harmless
python scripts/learner.py record <skill-directory> --capability harmless \
  --fail --error boundary-block --note "bomb-making tutorial"
python scripts/learner.py prefer <skill-directory> --key publication-threshold --val 0.6
python scripts/learner.py insight <skill-directory>
python scripts/learner.py reflect <skill-directory>

Technical Analysis

The implementation recognizes only one positional condition: the first argument must be exactly fail to mark an operation as unsuccessful. It does not implement the documented record, prefer, insight, or reflect subcommands and does not parse --capability, --fail, --error, --note, --key, or --val.

Under the documented failure command, sys.argv[1] is record, so ok is set to True. The second argument, the skill-directory path, is stored as the note, while every remaining option is ignored. Consequently, a failure that the documentation says will increment failure telemetry is silently recorded as a success.

Unknown commands and options are not rejected. The process emits apparently valid JSON and exits normally, which can mislead the Agent or operator into believing the requested security operation succeeded. This is a fail-open design for controls that are supposed to ...[truncated 1740 chars]

Remediation
View remediation

Remediation Suggestions

  1. Implement the documented interface using argparse with explicit record, prefer, insight, and reflect subcommands.
  2. Define required and optional parameters for every subcommand and reject unknown arguments.
  3. Make malformed or unsupported invocations fail closed with a nonzero exit status and a clear error.
  4. Ensure --fail reliably increments total_fails and that --note is the only value used as the event note.
  5. Avoid persisting skill paths, secrets, credentials, or unrestricted task content.
  6. Apply input length limits and schema validation to capabilities, preference keys, values, errors, and notes.
  7. Add automated tests for every documented command, including success, failure, malformed input, and unknown options.
  8. Use file locking and atomic replacement through a temporary file in the same directory to prevent lost updates and partial JSON writes.
  9. Validate the loaded JSON structure and numeric field types before incrementing counters.
  10. Keep command documentation generated from or tested against the actual parser so implementation and instructions cannot diverge silently.
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (7)

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The description claims a highly capable meta-skill with reflective validation, orchestration, and ongoing self-improvement. The supplied code chunk does not implement those behaviors. It only loads and updates a local JSON file with counters and notes, apparently to support later analysis by another component. While this logging could be a supporting detail for a larger evolution loop, the chunk itself materially differs from the declared primary purpose and lacks the advertised capabilities. There are no suspicious extra permissions or unrelated resource accesses beyond local file I/O, but the core functionality is much narrower than described.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
89% confidence
Finding

The skill advertises behaviors that include writing learned state to local files such as learned_patterns.json and instructs users to run scripts/learner.py, but it does not declare any tool scope or permissions boundary. In practice, undeclared file-write capability weakens reviewability and can enable unexpected persistence of data or modification of skill state, especially in environments that rely on manifest-declared permissions for trust decisions.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill explicitly describes recording operation counts, failure types, user preferences, and improvement suggestions into persistent storage, but provides no privacy notice, retention limit, consent mechanism, or data minimization guidance. This creates a real privacy and compliance risk because user behavior and preferences may be stored indefinitely and reused across sessions without clear authorization or disclosure.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The entire skill document is written in Chinese, including headings and operational descriptions, with no indication that language selection is optional or that the skill is intended only for a Chinese-speaking or region-specific context. This can violate language/locale policy because it implicitly fixes the interaction language without user opt-in.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The module docstring is written as an instruction/description entirely in Chinese and does not offer any language or locale choice. Per the policy, forcing a specific language without user opt-in or a documented region-specific justification is a natural-language policy violation.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
78% confidence
Finding

This JSON file contains a natural-language value in Chinese ("蒸馏自 value-alignment") without any indication that language choice is user-configurable or intentionally region-specific. Under the language/locale policy, fixed language content can be a policy concern when the skill does not document user opt-in or justification.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
89% confidence
Finding

This code writes persisted data to learned_patterns.json, modifying local state each time record() runs. While the module docstring hints that the skill records invocation outcomes, there is no user-facing prompt, log message, or explicit disclosure at the write site that notes are being stored on disk.

Content

No source excerpt is available for this finding.