Back to skill

Security audit

automation-skill

Security checks for vulnerabilities and agentic risk

Overview

The skill mostly matches its stated search and reflection purpose, but it stores user-provided reflections in persistent agent memory and later encourages broad memory searches, which needs review before installation.

Review this skill before installing if you use shared machines or agent memory across sessions. Treat reflection entries as persistent data, avoid saving secrets or untrusted instructions, and prefer explicit confirmation before writing to or searching ~/.qclaw/memory/memory.md.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Findings (1)

T02 · Agent Memory Poisoning

Warning
Location
scripts/self_reflect.py:79
Finding

Untrusted Reflection Content Can Poison Persistent Agent Memory

Content
View full analysis

Vulnerability Details

File Location: scripts/self_reflect.py:79-100; related retrieval instruction in SKILL.md:165-171
Vulnerability Type: Persistent storage of untrusted content in agent memory
Risk Level: Medium

Vulnerable Code

python
def record_reflection(
    context: str,
    what_happened: str,
    lesson: str,
    severity: int = 3,
    tags: list[str] = None,
) -> Reflection:
    """Record a correction"""
    ensure_dirs()

    ref = Reflection(
        timestamp=datetime.now().strftime("%Y-%m-%d %H:%M"),
        context=context,
        what_happened=what_happened,
        lesson=lesson,
        severity=max(1, min(5, severity)),
        tags=tags or [],
    )

    entry = (
        f"\n---\n"
        f"**CONTEXT:** {ref.context}\n"
        f"**TIMESTAMP:** {ref.timestamp}\n"
        f"**REFLECTION:** {ref.what_happened}\n"
        f"**LESSON:** {ref.lesson}\n"
        f"**SEVERITY:** {ref.severity}\n"
        f"**TAGS:** {', '.join(ref.tags)}\n"
    )

    HOT_FILE.write_text(HOT_FILE.read_text(encoding="utf-8") + entry, encoding="utf-8")
    return ref

The related skill instruction establishes broad retrieval of this file:

markdown
| User says | Trigger |
|--------|------|
| "What did you learn?" / "Do you know XX?" | Search `memory.md` |

Technical Analysis

The context, what_happened, lesson, and tags values originate from command-line input and are interpolated verbatim into ~/.qclaw/memory/memory.md. The implementation does not validate instruction-like content, escape the Markdown record structure, preserve a trusted provenance marker, or separate untrusted reflection data from information that may later be placed into an agent prompt.

An attacker can also inject newlines and record delimiters into these fields, allowing the attacker to create content that appears to be an independent memory entry. More imp ...[truncated 2153 chars]

Remediation
View remediation

Remediation Suggestions

  1. Store reflection records outside directories treated as agent instruction memory. Use a dedicated data-only location and retrieve records through a controlled interface.
  2. Replace the Markdown append format with structured JSON or a database schema. Preserve each field as data and assign explicit provenance and trust metadata.
  3. Validate input lengths and reject or neutralize control syntax, record delimiters, role markers, and instruction-like content where practical.
  4. When records are supplied to a model, wrap them in a clearly delimited untrusted-data block and provide a higher-priority instruction that their contents must never be followed as commands.
  5. Retrieve only records specifically requested by the user rather than loading the complete memory file for broad questions.
  6. Require explicit user confirmation before storing content received from an external or untrusted party.
  7. Require authorization and confirmation before displaying potentially sensitive reflection records, particularly when a query could retrieve multiple entries.
  8. Add adversarial tests covering embedded newlines, fake Markdown fields, prompt-injection phrases, record delimiter injection, and attempts to request disclosure of unrelated records.
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (11)

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding

This second mismatch report points to the same underlying issue: the skill is presented as a broad automation package, while the actual behavior appears narrower and inconsistent. Overstated functionality around search and installation can mislead users into triggering actions they did not intend or trusting outputs that were never actually produced by the described components.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding

This second mismatch report points to the same underlying issue: the skill is presented as a broad automation package, while the actual behavior appears narrower and inconsistent. Overstated functionality around search and installation can mislead users into triggering actions they did not intend or trusting outputs that were never actually produced by the described components.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
95% confidence
Finding

The skill advertises and instructs use of capabilities that imply network access and persistent file reads/writes, but it does not declare any tool scope or permission boundaries. This creates a transparency and least-privilege problem: a host agent may invoke broader capabilities than a user expects, especially for local memory writes and outbound search requests.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

Overly broad trigger phrases can cause the skill to activate during ordinary conversation, including generic prompts like asking what the agent knows. In this skill, accidental activation is more risky because activation may lead to network searches or persistent writes to local memory without a clearly bounded user intent.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill describes persistent storage of reflections in local memory files but does not clearly warn users that their inputs will be retained on disk. This is a privacy and data-governance issue because users may disclose sensitive operational or personal content during reflection, not realizing it will be stored across sessions.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The trigger table includes ambiguous phrases like general help, knowledge, or installation requests without strong scope boundaries. Such ambiguity increases the chance of unintentional invocation, privilege creep, and silent data handling, especially in an agent environment where triggers may auto-route execution.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The hard-coded Accept-Language: zh-CN,zh;q=0.9,en;q=0.8 header imposes a specific locale preference for all outbound searches. This is a natural-language policy concern because the script does not offer the user any language or locale choice, and the locale constraint is not documented as a justified region-specific requirement.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

Natural-language strings throughout the file, including the module description, CLI descriptions, help text, and output messages, are all fixed in Chinese. This imposes a language choice on users without any documented opt-in, fallback, or locale selection mechanism.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

The tool stores potentially sensitive self-reflection content persistently in ~/.qclaw/memory/memory.md without any warning, consent flow, retention control, or permission hardening. Because reflections may contain personal mistakes, work context, or sensitive operational details, silent persistence increases privacy and data exposure risk on shared systems or in backups.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
85% confidence
Finding

The manifest and module documentation frame this skill as a daily reflection/analysis tool that records entries under a dedicated memory area. However, the report command accepts any --output path and writes the generated report there, enabling arbitrary file creation or overwrite unrelated to the reflection store.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
88% confidence
Finding

The README presents a language selector with English linking back to the same README while marking Chinese as the active language, which effectively fixes the document to Chinese for the current file. This can be a natural-language policy concern when users are not given an actual language/locale choice or opt-in within the file they are using.

Content

No source excerpt is available for this finding.