ClawHub
Back to skill

Security audit

Chat Refiner

Security checks for vulnerabilities and agentic risk

Overview

This skill has a coherent memory-cleanup purpose, but it should be reviewed because it can persist chat-derived instructions and API keys into long-lived memory files.

Review this skill before installing. Use it only on transcripts you trust, avoid running it on chats containing credentials, and require redaction plus an explicit preview before any MEMORY.md or memory summary file is written.

A.I.G

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T02 · Agent Memory Poisoning

Error
Location
SKILL.md:21
Finding

Untrusted Transcript Instructions Can Be Promoted into Persistent Agent Memory

Content
View full analysis

Vulnerability Details

File Location: SKILL.md:21-27 and references/principles.md:4-10
Vulnerability Type: T02: Agent Memory Poisoning
Risk Level: High

Complete Code Snippet (SKILL.md:21-27):

markdown
2. **Principles** (see references/principles.md):
   - Delete casual/heartbeats/repeated tools.
   - Keep: disciplines, configs, skills learned/install, explicit "remember", decisions.
3. **Process**:
   - Read raw.
   - Extract key.
   - Write summary to memory/YYYY-MM-DD-summary.md or MEMORY.md.

Complete Code Snippet (references/principles.md:4-10):

markdown
- **Keep**: 
  - Explicit instructions ("do X").
  - Disciplines/rules (e.g. skill-vetter first).
  - Configs (API keys, models).
  - Skills learned (install/how).
  - User "remember this" info.
  - Decisions/dates/people/preferences/todos.

Technical Analysis

The workflow treats chat transcripts and session histories as source material, explicitly preserves instructions, disciplines, rules, and “remember this” statements, and then writes the extracted content to persistent memory files.

Transcripts can contain attacker-controlled or otherwise untrusted text. The documented process does not require provenance checks, trust classification, authenticated-user confirmation, or separation between quoted historical content and authoritative behavioral instructions. Consequently, malicious text embedded in a transcript can be transformed from untrusted conversation data into a durable rule that future agent sessions may treat as trusted memory.

Attack Path

  1. An attacker places a plausible instruction, discipline, or “remember this” statement in a transcript available to the agent.
  2. A user invokes the chat-refiner skill against that transcript or its associated session history.
  3. The skill follows its documented requirement to retain explicit instructions and rules.
  4. The malicious instruction is written to MEMORY.md or `memory/YYYY-MM-DD-summary.md ...[truncated 625 chars]
Remediation
View remediation

Remediation Suggestions

  1. Do not automatically promote transcript instructions, disciplines, or rules into authoritative memory.
  2. Treat all transcript-derived instructions as untrusted quoted data and preserve their source, author, session identifier, and timestamp.
  3. Require explicit confirmation from the authenticated user before persisting any behavioral instruction.
  4. Reject or quarantine content that attempts to alter safety constraints, tool permissions, trust boundaries, or memory-processing rules.
  5. Separate factual user preferences from executable or imperative instructions using a structured schema.
  6. Mark generated summaries as non-authoritative context so future agents cannot interpret them as higher-priority instructions.
  7. Add a review stage that displays proposed memory additions before writing them.

T09 · Insecure Skill Coding Practices

Error
Location
references/principles.md:4
Finding

API Keys Are Explicitly Retained in Plaintext Memory Summaries

Content
View full analysis

Vulnerability Details

File Location: references/principles.md:4-10
Vulnerability Type: T09: Insecure Skill Coding Practices
Risk Level: High

Complete Code Snippet:

markdown
- **Keep**: 
  - Explicit instructions ("do X").
  - Disciplines/rules (e.g. skill-vetter first).
  - Configs (API keys, models).
  - Skills learned (install/how).
  - User "remember this" info.
  - Decisions/dates/people/preferences/todos.

The resulting content is written according to SKILL.md:24-27:

markdown
3. **Process**:
   - Read raw.
   - Extract key.
   - Write summary to memory/YYYY-MM-DD-summary.md or MEMORY.md.

Technical Analysis

The skill explicitly identifies API keys as configuration that should be retained. Its workflow then writes retained information into ordinary Markdown memory files. This design duplicates credentials from transcripts into long-lived plaintext storage rather than redacting them or referring to a dedicated secret-management mechanism.

Plaintext memory files may be read by later agents and tools, indexed for retrieval, synchronized, backed up, included in diagnostics, or accidentally committed to source control. Persisting a secret also increases its lifetime and expands the number of components that can access it.

No network exfiltration mechanism or embedded malicious code was identified in the audited files. The vulnerability is insecure local retention and expanded exposure of sensitive credentials.

Attack Path

  1. A chat transcript or session history contains an API key or similar credential.
  2. The chat-refiner processes that source.
  3. Following references/principles.md, it classifies the key as configuration that must be kept.
  4. Following SKILL.md, it writes the key into MEMORY.md or a dated Markdown summary.
  5. A later agent, local user, indexing system, backup process, repository operation, or other tool with access to the memory file obtains the plaintext credential.
  6. The exposed key ...[truncated 576 chars]
Remediation
View remediation

Remediation Suggestions

  1. Change the retention rule to prohibit storing API keys, passwords, private keys, session tokens, authentication cookies, and credential-bearing URLs.
  2. Add secret detection before output using patterns and entropy-based checks appropriate to common credential formats.
  3. Replace detected values with explicit placeholders such as API_KEY=[REDACTED].
  4. Preserve only non-sensitive metadata, such as the service name, model name, secret-manager reference, and required environment-variable name.
  5. Store credentials exclusively in a dedicated secret manager or protected environment-variable facility.
  6. Apply restrictive permissions to memory files and exclude them from source control, synchronization, telemetry, and broad indexing.
  7. If a secret has already been written, remove it from current files and history, revoke it, and issue a replacement credential.
  8. Add automated tests confirming that representative API keys and tokens never appear in generated summaries.

SkillSpector

Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Findings (3)

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

The activation criteria are broad enough that this skill could be invoked for general 'summarize/clean history' or memory-maintenance requests without a clear boundary on scope, consent, or target files. Because the skill can read transcripts and write summaries to MEMORY.md or dated memory files, ambiguous triggering increases the chance of unintended retention, deletion, or modification of user history and memory artifacts.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill explicitly uses write/edit tools to produce MEMORY.md or summary files and describes deleting 'useless/redundant/process exchanges,' but it does not warn users that persistent memory files may be changed and conversation content may be omitted or effectively discarded. This lack of transparency can cause unauthorized or surprising memory changes, loss of important context, and accidental persistence of sensitive data extracted from transcripts.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The guidance explicitly says to keep "API keys" when refining or preserving conversation history, but it provides no restriction, redaction rule, or secure-handling caveat for secrets. In a memory-maintenance skill, this is especially dangerous because it can cause long-term persistence of credentials into summary files or MEMORY.md, increasing exposure through later reads, commits, sharing, or indexing.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.