Back to skill

Security audit

Session Feedback Analyzer

Security checks for vulnerabilities and agentic risk

Overview

The skill appears purpose-built for feedback analysis, but it scans broad Claude session history and stores snippets of user messages by default, so it needs review before installation.

Install only if you intentionally want local analysis of Claude Code session history. Prefer running it with --no-snippets, a narrow --session-dir or --skill-filter, and an output path outside shared repositories; review and delete feedback-store archives because they may contain copied user-message text.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/analyze.py:423
Finding

Session Message Snippets Are Persisted in Plaintext and Retained Indefinitely in Archives

Content
View full analysis

Vulnerability Details

File Location: scripts/analyze.py:90-96, 118-125, 335-365, 423-430, 437-474
Vulnerability Type: Plaintext storage and excessive retention of potentially sensitive session data
Risk Level: Medium

Vulnerable Code

The analyzer recursively scans the user's Claude project sessions by default:

python
parser.add_argument(
    "--session-dir",
    default=str(Path.home() / ".claude" / "projects"),
    help="Root directory for session JSONL files",
)
python
def iter_session_files(session_dir: Path) -> Iterator[Path]:
    """Yield session JSONL files, skipping test/tmp directories."""
    for path in session_dir.rglob("*.jsonl"):
        path_str = str(path)
        if "pytest" in path_str or "/tmp/" in path_str:
            continue
        if "/subagents/" in path_str:
            continue
        yield path

User text is copied into a feedback event without secret detection or redaction:

python
def _build_event(
    invocation: SkillInvocation,
    outcome: str,
    confidence: float,
    correction_type: str | None,
    user_text: str,
    turns_to_feedback: int,
    ai_tools: list[str],
) -> FeedbackEvent:
    event_id = hashlib.sha256(
        f"{invocation.invocation_id}:{invocation.skill_id}".encode()
    ).hexdigest()[:16]
    snippet = " ".join(user_text.split())[:200]
    dimension = attribute_dimension(snippet)
    return FeedbackEvent(
        event_id=event_id,
        timestamp=invocation.timestamp,
        session_id="",  # filled by caller
        skill_id=invocation.skill_id,
        invocation_uuid=invocation.invocation_id,
        outcome=outcome,
        confidence=confidence,
        correction_type=correction_type,
        user_message_snippet=snippet,
        turns_to_feedback=turns_to_feedback,
        ai_tools_used=ai_tools,
        dimension_hint=dimension,
    )

Snippets are written in plaintext unless the user explicitly selects --no-snippets:

python
with output_path.
...[truncated 4645 chars]
Remediation
View remediation

Remediation Suggestions

  1. Make privacy-preserving behavior the default:

    • Do not persist message snippets unless the user explicitly supplies an opt-in option such as --include-snippets.
    • Retain only the outcome, confidence, dimension, timestamp, and pseudonymous identifiers by default.
  2. Add secret and personal-data redaction before persistence:

    • Detect common API keys, bearer tokens, private-key blocks, credentials, email addresses, and high-entropy strings.
    • Replace detected values with stable placeholders rather than storing raw text.
    • Apply redaction before dimension attribution and serialization so sensitive values do not enter intermediate event objects.
  3. Enforce restrictive storage permissions:

    • Create output directories with owner-only permissions where supported.
    • Open new feedback and archive files with mode 0600.
    • Verify existing output files are not group-readable or world-readable before appending.
  4. Implement an actual retention policy:

    • Delete or aggregate records after the configured retention period instead of moving raw snippets into permanent archives.
    • Make retention duration configurable and document it clearly.
    • If historical metrics are needed, archive only aggregated, snippet-free statistics.
  5. Reduce collection scope:

    • Require explicit project selection or provide a preview of session files before recursively scanning all projects.
    • Support allowlists and exclusions resolved by path components rather than substring matching.
  6. Protect existing data:

    • Provide a migration or cleanup command that strips user_message_snippet from active and archived JSONL files.
    • Warn users that older versions may already have created plaintext copies.
    • Avoid placing the default output inside paths likely to be committed; add feedback-store/ to recommended ignore rules.
  7. Add security tests:

    • Verify snippets are absent under default settings.
    • Verify representative ...[truncated 160 chars]
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Findings (12)

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The declared purpose emphasizes an extraction/inference pipeline from raw session JSONL: identify skill invocations, inspect nearby user turns, classify outcomes, and compute per-skill metrics. The supplied code does not do that. Instead, it assumes feedback events already exist in a normalized JSONL format with fields like skill_id, outcome, timestamp, and dimension_hint, then computes downstream analytics from those events. While correction_rate computation is mentioned in the description, the primary behavior here is post-processing feedback.jsonl rather than parsing sessions or deriving implicit feedback. This is a material description-behavior mismatch.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The README explicitly instructs users to parse ~/.claude/projects/**/*.jsonl session logs and emit derived feedback events, but it does not prominently warn that those logs may contain sensitive prompts, code, secrets, or personal data. Because this skill processes broad local session history and writes a secondary dataset, users may unknowingly create a privacy exposure or retain sensitive information longer than intended, especially when --no-snippets is optional rather than the default.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
88% confidence
Finding

The skill describes reading session logs and writing feedback artifacts, but the manifest does not declare any explicit tool scope such as read-only paths or allowed tools. That creates an overbroad execution surface where an agent may access or modify files beyond what is necessary, especially because the documented workflow touches user session data and output storage.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

Overly broad trigger patterns can cause the skill to activate for unrelated prompts, increasing the chance that session logs are parsed or feedback artifacts are generated when the user did not intend that action. In this skill, accidental invocation is more sensitive because the documented behavior processes historical session data and may emit snippets derived from user content.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

Several trigger phrases are generic enough to overlap with ordinary discussions about quality, corrections, or regressions, which can misroute unrelated requests into this log-mining skill. Because the skill is designed to inspect stored sessions and generate persisted outputs, misinvocation raises privacy and unnecessary data-processing risk beyond a simple UX issue.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
82% confidence
Finding

The skill persists analyzed session-derived data to default output locations and archives older events, creating durable records from potentially sensitive interaction history. Persistent storage is particularly risky here because the source material is user session content and the outputs may include snippets and metadata that enable later disclosure or secondary use.

Content

Scanner excerpt · SKILL.md (reported line 148)May include surrounding context.

CLI

bash
# Basic: analyze all sessions, write to default output
python3 scripts/analyze.py

# Custom session directory and output

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill documents collection of user message snippets from session data and includes them in output by default, while privacy protection is optional via --no-snippets. This creates a substantial risk of exposing sensitive user content, credentials, proprietary text, or personal data into derived artifacts and downstream pipelines without sufficiently prominent consent or warning.

Content

No source excerpt is available for this finding.

Ssd 3

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill explicitly instructs retention of plain-language user message snippets from session logs in feedback.jsonl, which can preserve sensitive content in a searchable, shareable artifact. This expands the blast radius of any private data originally present in sessions by duplicating it into downstream files, archives, and other improvement-pipeline consumers.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The script persists conversation-derived feedback, including session IDs, skill IDs, timestamps, tool usage, and by default a snippet of user text, into a local JSONL store without any explicit runtime notice or affirmative consent at collection time. Because the input source is the user's Claude session history under ~/.claude/projects, this can create a privacy and data-retention issue: sensitive content from user conversations may be copied into a secondary file and retained beyond the original session context.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

Core explanatory sections and some rule descriptions are written partly in Chinese, which effectively imposes a language requirement on readers who may expect a single consistent language. The file does not indicate that the skill is region-specific or offer an English-only alternative, so this is a natural-language policy concern.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
85% confidence
Finding

The analyzer only recognizes acceptance and correction cues from Chinese and English keyword lists. This creates a language-policy issue because users interacting in other languages are not given any opt-in choice, fallback, or documented limitation in the file's natural-language behavior.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
81% confidence
Finding

This is a code file, so SQP-2 applies to safety-relevant operations such as subprocess or shell execution. The test embeds a Bash tool invocation using git checkout -- src/main.py, which is a potentially destructive file-reverting shell command, but the file provides no nearby comment, docstring, or user-facing disclosure explaining that such operations are being exercised.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.