Back to skill

Security audit

Karpathy Query Feedback

Security checks for vulnerabilities and agentic risk

Overview

The skill has a coherent wiki-building purpose, but it persists memory-derived content and exposes under-scoped write behavior that users should review before installing.

Install only if you are comfortable with the skill saving M-Flow memory search results into a persistent wiki. Prefer running it in a workspace where write permissions are limited to the intended wiki directory, review saved entries before downstream compile use, and avoid using it with confidential memory until filename containment and Markdown/content sanitization are added.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T02 · Agent Memory Poisoning

Warning
Location
scripts/__init__.py:158
Finding

Persistent Knowledge Poisoning Through Unsanitized Memory Content

Content
View full analysis

Vulnerability Details

File Location: scripts/__init__.py:61-64, scripts/__init__.py:158-174, scripts/__init__.py:231-234, and scripts/__init__.py:257-272
Vulnerability Type: Persistent agent-memory poisoning and Markdown injection
Risk Level: Medium

Vulnerable Code

python
def to_markdown_row(self) -> str:
    """Convert to a Markdown table row."""
    tags_str = ",".join(self.tags)
    return f"| {self.source} | {self.content} | {tags_str} | {self.timestamp} |"
python
# Extract text from metadata.sentence_classifications
classifications = item.get("metadata", {}).get("sentence_classifications", [])
for sent in classifications:
    content = sent.get("text", "")
    if content and content not in all_content:
        all_content.add(content)

        # Extract tags
        tags = self._extract_tags(content, query_text)

        entry = WikiEntry(
            source=f"query:{search_mode}",
            content=content,
            tags=tags,
            relevance_score=item.get("score", 0.0)
        )
        results.append(entry)
python
for entry in entries:
    lines.append(entry.to_markdown_row())

return "\n".join(lines)
python
# Read existing content
existing_content = ""
if filepath.exists():
    existing_content = filepath.read_text(encoding="utf-8")

# Format new entries
new_entries_md = self.format_as_wiki(entries, include_header=False)

# Append to the file
with open(filepath, "a", encoding="utf-8") as f:
    if not existing_content.strip():
        # Write a header when the file is empty
        f.write("| source | content | tags | timestamp |\n")
        f.write("|--------|---------|------|-----------|\n")
    f.write(new_entries_md)
    f.write("\n")

Technical Analysis

Text retrieved from M-Flow memory is treated as trusted content and copied verbatim into a persistent Wiki intended for lat ...[truncated 2181 chars]

Remediation
View remediation

Remediation Suggestions

  1. Treat all search results and memory records as untrusted data, regardless of their apparent source.
  2. Escape Markdown table metacharacters before formatting:
    • Replace | with \|.
    • Normalize or encode carriage returns and line feeds.
    • Neutralize active HTML and other constructs if the renderer permits them.
  3. Prefer a structured storage format such as JSON with schema validation. Render Markdown only at the presentation boundary.
  4. Store explicit provenance, trust level, source identifier, and validation status for every entry.
  5. Require approval or automated policy validation before promoting session-derived records into long-term knowledge.
  6. Detect and quarantine prompt-like directives in retrieved content rather than silently promoting them.
  7. When supplying Wiki entries to a downstream model, place them in strongly delimited data sections and explicitly instruct the model that embedded directives are untrusted data, not executable instructions.
  8. Add tests covering pipes, multiline values, HTML, headings, fenced blocks, and prompt-injection phrases.

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/__init__.py:236
Finding

Arbitrary File Append Through Unrestricted Wiki Filename

Content
View full analysis

Vulnerability Details

File Location: scripts/__init__.py:236-274
Vulnerability Type: Path traversal and arbitrary file modification
Risk Level: Medium

Vulnerable Code

python
async def save_to_wiki(
    self,
    entries: List[WikiEntry],
    filename: str = None
) -> str:
    """
    Save entries to a Wiki file.

    Args:
        entries: Wiki entries
        filename: File name, using the date by default

    Returns:
        Path of the saved file
    """
    if filename is None:
        date_str = datetime.now().strftime("%Y-%m-%d")
        filename = f"wiki-{date_str}.md"

    filepath = self.wiki_path / filename

    # Read existing content
    existing_content = ""
    if filepath.exists():
        existing_content = filepath.read_text(encoding="utf-8")

    # Format new entries
    new_entries_md = self.format_as_wiki(entries, include_header=False)

    # Append to the file
    with open(filepath, "a", encoding="utf-8") as f:
        if not existing_content.strip():
            # Write a header when the file is empty
            f.write("| source | content | tags | timestamp |\n")
            f.write("|--------|---------|------|-----------|\n")
        f.write(new_entries_md)
        f.write("\n")

    return str(filepath)

Technical Analysis

save_to_wiki() exposes filename as a public argument and combines it with self.wiki_path without validating that the resulting path remains inside the configured Wiki directory.

A relative value containing parent-directory components, such as ../../state.md, can escape the Wiki directory. A supplied absolute path can also override the base path under pathlib joining semantics. The method then reads the target if it exists and opens it in append mode.

Consequently, a caller able to control filename and entry contents can append data to any file writable by the process. The existing file ...[truncated 1425 chars]

Remediation
View remediation

Remediation Suggestions

  1. Reject absolute paths and require a plain filename rather than a caller-supplied path.

  2. Enforce a strict allowlist, for example filenames matching ^[A-Za-z0-9._-]+\.md$.

  3. Resolve and verify containment before accessing the target:

    python
    import re
    from pathlib import Path
    
    if not re.fullmatch(r"[A-Za-z0-9._-]+\.md", filename):
        raise ValueError("Invalid Wiki filename")
    
    base = self.wiki_path.resolve()
    target = (base / filename).resolve()
    
    if target.parent != base:
        raise ValueError("Wiki file must remain inside the Wiki directory")
    
  4. If nested directories are intentionally supported, verify that the resolved target is relative to the resolved base using Path.relative_to().

  5. Run the Skill with a least-privilege account whose write permissions are restricted to the Wiki directory.

  6. Consider opening files relative to a trusted directory handle using platform-supported safe filesystem APIs when operating in an adversarial local environment.

  7. Add tests for absolute paths, .. traversal, nested traversal, unusual separators, and symlink-based escape attempts.

  8. Avoid reading the entire existing target merely to determine whether it is empty; use trusted file metadata or controlled file-creation logic instead.

Vulnerability Patterns
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Findings (7)

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The skill explicitly describes taking query results and storing them in persistent wiki storage, but it does not mention user notice, consent, or data-sensitivity checks. This is dangerous because users may submit confidential, personal, or proprietary queries that are then retained and made available to later compile stages, expanding exposure and creating privacy and data-governance risk.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The activation conditions are broad enough that the skill could trigger on ordinary knowledge queries or generic session-to-knowledge workflows without clear user intent. In this skill's context, unintended invocation is risky because it can automatically persist queried or session-derived content into the wiki layer, increasing the chance of accidental data retention or propagation of sensitive information.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
86% confidence
Finding

The file’s natural-language strings and documentation are written in Chinese, and the CLI/user-facing descriptions also assume that locale without offering any language choice. Under the stated policy, forcing a specific language without opt-in is a natural-language policy violation unless clearly justified.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The CLI path automatically persists query results to the wiki via query_and_save() with no confirmation, dry-run mode, or explicit opt-in for side effects. Because the saved content is derived from search results and user queries, this can cause unintended modification of the knowledge base, data pollution, or persistence of sensitive/retrieved content when a user expected a read-only search.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
78% confidence
Finding

The entire skill documentation is written in Chinese and presents the interface and behavior in that locale only, without indicating that users may choose another language. Under the stated policy, forcing a specific language without user opt-in can be a natural-language policy concern unless the locale restriction is clearly documented and justified.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
93% confidence
Finding

The inline comment is written in Chinese (添加当前目录到路径), which imposes a specific language in user-visible natural language without offering a choice or documenting a locale-specific requirement. This can violate language/locale policy expectations for broadly usable skills.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
80% confidence
Finding

The script calls query_and_save(...) and then reports a saved path, indicating it may write data to disk. While the save result is printed after the fact, there is no prior warning, confirmation, or explanatory comment/docstring telling the user that running this test may create files.

Content

No source excerpt is available for this finding.