Back to skill

Security audit

RAG

Security checks for vulnerabilities and agentic risk

Overview

The skill is coherent RAG guidance, but it asks for unconfirmed persistent writes and deletions in shared infrastructure, project, and finance records.

Install only if you are comfortable with the agent maintaining local RAG memory and shared Clawic records. Before use, require confirmation for any write or deletion outside ~/Clawic/data/rag/, especially servers.md, projects, and finances/subscriptions.md, and avoid treating matching service or server names as proof that a row belongs to this skill.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
SKILL.md:38
Finding

Automatic Persistent and Shared-State Modification Without Prior Authorization

Content
View full analysis

Vulnerability Details

File Location: SKILL.md:38; supporting policy in memory-template.md:25
Vulnerability Type: Persistent shared-state modification beyond least privilege
Risk Level: Medium

Vulnerable Skill Instructions

SKILL.md:38:

markdown
**Write before the session ends** whenever it produced something durable: an index built, migrated, or retired; a source added to or dropped from the corpus; a chunking, embedding, retrieval or reranking parameter that was changed and measured; an eval run and its scores; a failure and the cause it turned out to be; a monthly or per-query cost; a self-hosted store or embedding server; or something the user will read again — an ingestion recipe that finally parsed a hostile format, a prompt template that held up, an architecture decision, a golden set. `memory-template.md` holds every destination, format and threshold, and is the only file you open in order to write.

memory-template.md:25:

markdown
No permission needed; every write is announced in one line that names the file. Writes and deletions stay inside the paths declared in this skill's `configPaths`. A deletion is named in that same line, and in a shared box only rows this skill itself wrote are ever updated or removed.

Technical Analysis

The Skill explicitly authorizes the Agent to perform persistent writes and deletions without obtaining prior user approval. These operations are not limited to a private, Skill-owned workspace. The declared paths include shared server, project, and finance records under:

  • ~/Clawic/data/servers/
  • ~/Clawic/data/projects/
  • ~/Clawic/data/finances/

Automatically announcing a mutation does not provide the same protection as obtaining informed authorization before it occurs. Persistent records can affect later sessions and other Skills, while shared finance and infrastructure records may be operationally significant.

Some persistence is r ...[truncated 1670 chars]

Remediation
View remediation

Remediation Suggestions

  1. Default the Skill to read-only operation unless the user explicitly requests persistence.
  2. Require confirmation before every deletion and before modifying any shared server, project, or finance file.
  3. Present the target path and an exact proposed diff before requesting approval.
  4. Permit automatic writes only within a dedicated Skill-owned directory such as ~/Clawic/data/rag/, and only for low-impact append-only records.
  5. Treat changes inferred from retrieved documents, generated text, or other untrusted content as unverified until confirmed by the user or an authoritative system.
  6. Add atomic writes, backups, and rollback support for persistent state.
  7. Replace “No permission needed” with a least-privilege policy that distinguishes private append-only notes from shared or destructive mutations.

T09 · Insecure Skill Coding Practices

Warning
Location
memory-template.md:250
Finding

Ambiguous Record Ownership Allows Shared Inventory Overwrites

Content
View full analysis

Vulnerability Details

File Location: memory-template.md:250 and memory-template.md:294
Vulnerability Type: Insecure shared-record ownership and authorization logic
Risk Level: Medium

Vulnerable Skill Instructions

memory-template.md:250:

markdown
- **Identity is `Name` + `Provider`.** Read the file before adding. If that pair is already there, update the row in place — it is yours. Never touch a row whose `Provider` is not one you wrote.

memory-template.md:294:

markdown
- **Identity is `Service`.** Read the file before adding; if the service is already listed, update its row in place instead of adding a second one.

Technical Analysis

The Skill determines ownership using mutable, non-unique business identifiers:

  • Server records: Name + Provider
  • Subscription records: Service

A matching identifier proves record identity, but it does not prove that this Skill created or owns the record. The server rule nevertheless states that a matching row “is yours,” while the subscription rule permits any matching service row to be updated.

This conflicts with the broader policy that rows created by other Skills must not be modified. Because the records do not carry immutable provenance, the Agent cannot reliably enforce that restriction.

An attacker can intentionally reuse an existing server or service identifier. Accidental collisions are also plausible, especially for common service names such as a hosted embedding provider or vector database. The resulting read-modify-write operation can silently overwrite records belonging to a user or another Skill.

Attack Path

  1. A shared servers.md or subscriptions.md file already contains a legitimate record created by a user or another Skill.
  2. An attacker supplies a server name and provider, or a service name, matching the existing record.
  3. The Skill searches the shared file and finds the matching business identifie ...[truncated 1100 chars]
Remediation
View remediation

Remediation Suggestions

  1. Add immutable provenance fields to every shared record, including:
    • record_id
    • created_by
    • owner_skill
    • created_at
    • last_modified_by
  2. Permit updates or deletion only when owner_skill explicitly identifies this Skill.
  3. Treat Name + Provider and Service solely as deduplication hints, not authorization evidence.
  4. If a matching foreign record exists, request user confirmation or create a separately identified record instead of modifying it.
  5. Use compare-and-swap or version checks to prevent concurrent Skills from overwriting one another.
  6. Keep an append-only audit trail for shared-record changes, including before and after values.
  7. Require explicit confirmation before changing costs, access references, ownership, retirement status, or subscription cancellation state.
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Memory PoisoningPersistent Context Injection, Context Window Stuffing, Memory Manipulation
Findings (12)

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · memory-template.md (reported line 246)May include surrounding context.

NameProviderAccount / ProjectRegionTypeRoleMonthlyAccess reference
qdrant-1hetznerrag-prodfsn1CCX33vector store62 EURkeychain:qdrant-prod
embed-gpu-1runpodrag-prodeu-roA4000embedding + rerank118 USDenv:RUNPOD_KEY
text

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · memory-template.md (reported line 290)May include surrounding context.

NameProviderAccount / ProjectRegionTypeRoleMonthlyAccess reference
qdrant-1hetznerrag-prodfsn1CCX33vector store62 EURkeychain:qdrant-prod
embed-gpu-1runpodrag-prodeu-roA4000embedding + rerank118 USDenv:RUNPOD_KEY
text

Ae4

Medium
Category
analysis-evasion
Confidence
80% confidence
Finding

Suspicious Unicode normalization or mixed-script content

Content

No source excerpt is available for this finding.

Context Window Stuffing

Medium
Category
Memory Poisoning
Confidence
85% confidence
Finding

Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.

Content

Scanner excerpt · conversation.md (reported line 59)May include surrounding context.

md
## Carrying Retrieved Context Forward

- Keep the `chunk_id` list from the previous turn and merge it with the new retrieval, deduplicated, before reranking. It makes "and what about the second point" resolvable without a lucky re-retrieval.
- Cap the carry-forward at one or two turns. Chunks accumulated across ten turns fill the context with material the conversation moved past.
- When the topic clearly changes, drop the carry-forward entirely. The rewriter's output is a usable signal: a rewrite that shares no entities with the previous one is a topic switch.

## Filters That Persist

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The guidance explicitly recommends logging full queries, rewritten queries, answers, and citation identifiers, which can capture user secrets, personal data, regulated content, or sensitive internal information in cleartext. In a RAG system, these fields often contain the most sensitive runtime data, so broad logging without minimization, redaction, retention limits, or access controls creates a realistic privacy and data-exposure risk.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The markdown tells the agent to update persistent memory and artifact files after ingestion without warning the user that this modifies local state. In an agent setting, silent persistence is dangerous because it can overwrite prior data, create durable side effects across sessions, and normalize writing to local paths based solely on document instructions.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
80% confidence
Finding

The template directs the agent to persist broad operational memory and artifacts across sessions, including system state, index fingerprints, corpus metadata, failures, due dates, and shared infrastructure/subscription records. Even though it forbids raw secrets and confidential chunk text, this still creates durable cross-session retention of potentially sensitive business metadata and user-provided content, which can increase privacy, tenant-isolation, and prompt-injection exposure if later sessions or users can influence or read that memory.

Content

Scanner excerpt · memory-template.md (reported line 46)May include surrounding context.

md
Everything except golden sets, eval logs, artifacts and the shared boxes begins inside `memory.md`. Splitting is a procedure, not a suggestion:

1. Before appending to a section, count its entries.
2. If the append would take it past **~15 entries or ~40 lines of real content** — scaffolding, headings and comments do not count — then, in the same turn: create the new file in `~/Clawic/data/rag/`, move the whole section into it, **delete the section from `memory.md`**, add its line to `## Boxes`, and append the new entry to the new file.
3. Keep the headings identical on both sides of the move, so the split is a copy-paste and never a rewrite.
4. Never leave a copy behind. If the same data ever appears in both places, the extracted file wins and the `memory.md` copy is deleted.

Ae4

Medium
Category
analysis-evasion
Confidence
80% confidence
Finding

Suspicious Unicode normalization or mixed-script content

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The logging guidance explicitly recommends storing raw query text, rewritten queries, active filters, candidate IDs, scores, and context selections, but it does not warn about redaction, minimization, retention, or access controls. In a RAG system, queries and filters can contain personal data, secrets, tenant identifiers, or sensitive business intent, so indiscriminate logging can create a secondary data exposure channel and enlarge breach impact.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
84% confidence
Finding

This markdown file contains operational instructions to "delete the row on cancellation" in a shared finance tracking file. Because that is a user-data modification step in documentation, the file should include some warning or caution about preserving history or confirming the cancellation state before deletion.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Low
Category
Not specified by scanner
Confidence
92% confidence
Finding

The document explicitly instructs the agent to write to persistent local files under ~/Clawic/data/rag/, including memory and artifact records. Even though this appears operational rather than malicious, it expands behavior beyond pure RAG analysis/troubleshooting into filesystem mutation, which can create unintended state changes, stale records, or unauthorized local data persistence if followed automatically.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
83% confidence
Finding

This markdown file describes a hosted rerank API where 'every query leaves the perimeter,' which is a privacy- and data-boundary-relevant behavior. Although the line notes the tradeoff, it does not read as an explicit warning or caution to the user about sending query/passages to an external provider.

Content

No source excerpt is available for this finding.

Static analysis

Detected: suspicious.exposed_secret_literal, suspicious.prompt_injection_instructions

File appears to expose a hardcoded API secret or token.

Critical
Code
suspicious.exposed_secret_literal
Location
memory-template.md:58

Prompt-injection style instruction pattern detected.

Warn
Code
suspicious.prompt_injection_instructions
Location
generation.md:20