Back to skill

Security audit

meta-agent-society-protocol

Security checks for vulnerabilities and agentic risk

Overview

This skill is Review-worthy because it automatically stores and reuses user or usage notes and says it may rewrite its own instructions without clear consent, limits, or review.

Install only if you are comfortable with a skill keeping local cross-session notes and influencing later behavior. Before use, require explicit approval for any learning write, avoid storing raw user content or sensitive preferences, and do not allow automatic edits to SKILL.md without human review.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Findings (1)

T02 · Agent Memory Poisoning

Warning
Location
scripts/learner.py:5
Finding

Unvalidated Attacker-Controlled Data Persisted as Cross-Session Agent Memory

Content
View full analysis

Vulnerability Details

File Location: scripts/learner.py:5-18; related behavioral instructions in SKILL.md:52-55 and SKILL.md:64-75
Vulnerability Type: T02: Agent Memory Poisoning
Risk Level: Medium

Vulnerable Code

python
MEM = os.path.join(os.path.dirname(__file__), "learned_patterns.json")
def load():
    try:
        return json.load(open(MEM, encoding="utf-8"))
    except Exception:
        return {"total_ops": 0, "total_fails": 0, "patterns": []}
def record(op_ok=True, note=""):
    d = load(); d["total_ops"] += 1
    if not op_ok:
        d["total_fails"] += 1
    if note:
        d.setdefault("patterns", []).append(note)
    json.dump(d, open(MEM, "w", encoding="utf-8"), ensure_ascii=False, indent=2)
    return d
if __name__ == "__main__":
    ok = (sys.argv[1] != "fail") if len(sys.argv) > 1 else True
    rec = (sys.argv[2] if len(sys.argv) > 2 else "")
    print(json.dumps(record(ok, rec), ensure_ascii=False))

The associated instructions state that user preferences and experience are written to learned_patterns.json and should be adopted during later uses. They also prescribe automated reflection and modification of skill behavior based on accumulated records.

Technical Analysis

The script accepts an arbitrary positional string from sys.argv[2] and appends it directly to the persistent patterns array. It performs no schema validation, content filtering, provenance tracking, authorization check, trust classification, or user confirmation. The resulting JSON file is stored beside the skill and is intended to influence later sessions.

This creates a persistent trust-boundary violation: data supplied during one invocation can be treated as behavioral guidance in subsequent invocations. A malicious note could contain instruction-like content that attempts to override future task requirements, suppress safety checks, disclose information, or redirect agent behavior. The Python code does not itself interpret or e ...[truncated 2140 chars]

Remediation
View remediation

Remediation Suggestions

  1. Replace arbitrary note storage with a strict, versioned JSON schema containing typed fields and explicit length limits.
  2. Keep factual telemetry, user preferences, and agent instructions in separate stores. Never interpret free-form telemetry as executable or authoritative guidance.
  3. Permit only an allowlist of preference keys and normalized values. Reject control instructions, role directives, tool commands, and other instruction-like content.
  4. Record provenance for every entry, including source, timestamp, requesting user, session identifier, and whether explicit consent was obtained.
  5. Require user review and confirmation before persisted data is applied to a later session.
  6. Treat all loaded memory as untrusted data and place it in a clearly delimited context that cannot override system, developer, safety, or current-task instructions.
  7. Implement the documented interface with argparse subcommands and explicit named arguments. Reject unknown commands and options instead of silently storing them.
  8. Write updates atomically through a securely created temporary file followed by replacement, and apply restrictive file permissions where supported.
  9. Validate the entire loaded JSON structure before incrementing or appending fields. Report malformed state instead of silently resetting it.
  10. Do not automatically rewrite SKILL.md or other instruction-bearing files from accumulated memory. Any proposed behavioral change should require a trusted review process.
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (9)

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding

The declared description claims a broad, sophisticated meta-skill with self-verification, reflection, orchestration, and iterative self-improvement. The supplied code only implements a small persistence utility that tracks total operations, failures, and notes in a local JSON file. While this could support a learning/evolution loop, it does not itself perform the advanced behaviors described. Therefore the actual code chunk materially underdelivers relative to the declared purpose, making the description inaccurate for this code.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
90% confidence
Finding

The skill documents file-writing behavior via the learner workflow and persistent memory file, but it does not declare any tool scope or permissions boundaries. Undeclared persistence increases the risk of unauthorized local writes, hidden state changes, and misuse by hosts or users who expect a documentation-only skill.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The skill's documented behavior expands from agent-society orchestration into persistent cross-use learning, preference storage, and usage history retention without clearly scoping that data collection to the stated purpose. Scope expansion is dangerous because it normalizes retention of user data that operators may not expect, increasing privacy risk and the chance of secondary misuse.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill explicitly describes automatic persistence of usage history, error patterns, and user preferences but provides no privacy notice, consent flow, or retention policy. This is dangerous because it can silently collect and retain user-associated data across sessions, creating privacy, compliance, and trust risks even if the storage is local.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

Persistent storage of user preferences is not clearly justified for an agent-society protocol skill and can lead to collection of personal or behavioral data without necessity. Unnecessary preference retention increases privacy exposure and creates a durable profile that may be reused in unintended contexts.

Content

No source excerpt is available for this finding.

Ssd 3

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill plainly instructs persistent collection and reuse of user preferences and usage-derived insights across sessions, effectively creating long-lived behavioral memory. In the context of a public skill, this is more dangerous because users may not expect a protocol/orchestration skill to retain personal preferences or operational history, increasing privacy and profiling risk.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The module docstring is written as an instruction/description entirely in Chinese and does not indicate any user language choice or locale opt-in. Per the policy, forcing a specific language without offering choice or documenting a justified locale constraint is a natural-language policy violation.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
81% confidence
Finding

The example explicitly sets --key 输出语言 --val 中文, and the document is entirely framed in Chinese without indicating that language is configurable by user choice. This can conflict with language/locale policy when a skill effectively defaults to a specific language without explicit opt-in.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
93% confidence
Finding

The script persistently writes operational metadata to learned_patterns.json without any notice, consent flow, or retention controls. Although the stored data appears limited to success/failure counts and free-form notes, the note field can capture arbitrary content and may unintentionally persist sensitive information if upstream callers pass it through.

Content

No source excerpt is available for this finding.