Security audit

slop-detector

Security checks for vulnerabilities and agentic risk

Overview

The skill is presented as a prose detector but also instructs broad repository cleanup, secret/config scanning, commits, pushes, CI changes, and hook setup, so it should be reviewed carefully before installation.

Install only if you want a broad repository-review and cleanup workflow, not just a read-only prose detector. Before using it, require explicit approval for network checks, secret/config scans, file edits, deletions, commits, pushes, CI changes, hooks, and persistent project-rule files; consider splitting detection from remediation.

A.I.G

Vulnerability Patterns

Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review

Findings (5)

T05 · Unauthorized Access and Privilege Escalation

Error

Location: modules/cleanup-workflow.md:10
Finding: Repository-Wide Modification and Push Exceed the Skill's Declared Scope<![CDATA[ ## Vulnerability Details **File Location**: `modules/cleanup-workflow.md:10-25, 137, 155, 170, 191, 217, 233, 250, 289` **Vulnerability Type**: Unauthorized repository modification and publication **Risk Level**: High ### Vulnerable Instructions ```markdown **Run passes in order. Each pass is independent. Commit between passes. Prefer deletion over rewriting.** ... 1. **One pass per commit.** A commit titled "cleanup" that touches comments, prose, error handling, and tests is not reviewable. Split. 2. **Deletion beats rewriting.** When in doubt, remove the material. AI slop is additive; the cheapest correct fix is almost always to take material away. ``` The workflow repeatedly instructs the agent to commit changes, ending with: ```markdown Commit, push, and let it land. ``` ### Technical Analysis The skill metadata describes a prose-detection function, but the cleanup workflow expands that role into modifying documentation, source-code idioms, architecture, tests, repository policy, and CI configuration. It also directs the agent to create multiple commits and push the resulting changes. Detection does not inherently require repository write access, Git commit access, or permission to publish changes to a remote repository. Directing an agent to use those capabilities violates least privilege and creates a significant gap between the apparent purpose of the skill and its effective authority. The instruction to prefer deletion compounds the risk because false positives can lead to the removal of valid comments, tests, abstractions, or documentation. The final push instruction can publish those changes without a distinct approval boundary. ### Attack Path 1. A user invokes the skill expecting a read-only prose audit. 2. The agent loads `modules/cleanup-workflow.md`. 3. The workflow directs the agent to perform repository-wide cleanup passes. 4. The agent deletes or rewrites documentation, code, architecture, and tests based on he ...[truncated 805 chars]
Remediation: <![CDATA[ ## Remediation Suggestions 1. Make the skill strictly read-only by default. 2. Remove all unconditional `commit`, `push`, and automatic deletion instructions. 3. Separate detection from remediation into independently invoked operations. 4. Require explicit user approval before: - Editing any file. - Deleting any content. - Creating each commit. - Accessing a Git remote. - Pushing any branch. 5. Present proposed changes as minimal diffs rather than applying them automatically. 6. Restrict remediation to user-selected files and categories. 7. Require a clean working tree and create a recoverable backup or patch before modification. 8. Never trigger CI/CD through a push without separate, informed confirmation. ]]>

T02 · Agent Memory Poisoning

Error

Location: modules/cleanup-workflow.md:235
Finding: Persistent Agent Rules and Pre-Commit Hooks Can Outlive the Skill Run<![CDATA[ ## Vulnerability Details **File Location**: `modules/cleanup-workflow.md:235-250` **Vulnerability Type**: Persistent agent-state modification and hook installation **Risk Level**: High ### Vulnerable Instructions ```markdown ## Pass 10: Establish guardrails The cleanup is incomplete without preventing the slop from coming back. Add: - A `CONSTITUTION.md` (or equivalent project rules file) with immutable rules the AI and contributors must respect (see `evidence-backed-claims.md` for the pattern). - Strict linter configuration in the build config (e.g. `[lints.clippy]` block in `Cargo.toml`). - A CI step running the slop-detector on changed prose files. - Pre-commit hooks running the cheap detectors locally. Commit. This is what prevents the slop you just removed from coming back next sprint. ``` ### Technical Analysis The workflow directs the agent to create repository rules described as “immutable” and to install pre-commit hooks. These changes persist after the current invocation and affect future agent sessions, contributors, and Git operations. A repository policy file can act as persistent instruction state when future agents automatically load project-level guidance. Pre-commit hooks or hook configurations can execute commands during later commits, creating a durable execution point. Although the text does not provide a malicious hook payload, automatically establishing such persistence is outside the expected scope of a prose detector. The combination of persistent rules and hooks creates two distinct risks: - Persistent instruction influence over future agent behavior. - Persistent code execution during later developer or agent actions. ### Attack Path 1. A user invokes the cleanup workflow for a prose audit. 2. Pass 10 creates a project policy file containing rules future agents are expected to obey. 3. The workflow adds local pre-commit hook configuration or equivalent hook integration. 4. The changes are committed to the ...[truncated 813 chars]
Remediation: <![CDATA[ ## Remediation Suggestions 1. Remove automatic creation of `CONSTITUTION.md` or equivalent agent-policy files. 2. Do not describe repository instructions as immutable. 3. Remove automatic installation or modification of Git hooks. 4. Provide guardrail examples as inert templates that users may review separately. 5. Require explicit approval for each persistent file and show its complete content before writing it. 6. Require separate confirmation before changing CI, linter, or hook configuration. 7. Pin any hook command to a reviewed local implementation rather than dynamically generated agent prompts. 8. Document removal procedures for every persistent artifact. 9. Ensure future agents do not automatically trust newly created repository policy files without provenance checks. ]]>

T05 · Unauthorized Access and Privilege Escalation

Warning

Location: modules/cleanup-workflow.md:34
Finding: Prose Audit Expands into Sensitive Credential and Agent-Control Reconnaissance<![CDATA[ ## Vulnerability Details **File Location**: `modules/cleanup-workflow.md:34-54` **Vulnerability Type**: Excessive access to secrets and control-plane configuration **Risk Level**: Medium ### Vulnerable Instructions ```markdown ## Pass 0: Pre-slop sweep (always first) Before any cleanup, audit for things that should not be in the repo at all: - Committed agent-config files (`CLAUDE.md`, `.cursorrules`, `AGENTS.md`, `.codex/config.toml`, `.aider.conf.yml`, etc.) with secrets or broad capability grants. - Committed credentials (run `gitleaks` / `trufflehog`). - Untrusted MCP server entries. - Hooks that auto-execute on session start. Commit any redactions or revocations *before* any other cleanup, since later passes assume an uncompromised baseline. ```bash # Pre-slop sweep checklist gitleaks detect --no-banner ls -la | grep -E '^.*(CLAUDE|cursor|codex|aider|kiro)' find . -name '.mcp' -o -name 'mcp.json' -type f ``` ``` ### Technical Analysis A prose-pattern detector does not need to enumerate credentials, MCP server configuration, agent instruction files, or session-start hooks. The mandatory “Pass 0” therefore broadens the task into security-sensitive reconnaissance. Secret scanners can reveal credential locations and, depending on output settings, portions of secret values. Agent configuration, MCP entries, and startup hooks are high-value control-plane data because they disclose available tools, trust relationships, and automatic execution paths. No exfiltration command is present in the reviewed files. Nevertheless, unnecessarily placing these findings into the active agent context increases exposure and violates least privilege. ### Attack Path 1. A user invokes the skill to inspect writing quality. 2. The mandatory first pass runs secret-scanning and configuration-discovery commands. 3. The agent discovers credentials, MCP endpoints, broad capability grants, or startup hooks. 4. Sensitive findings enter the active model context or c ...[truncated 735 chars]
Remediation: <![CDATA[ ## Remediation Suggestions 1. Remove the mandatory secret and agent-configuration sweep from the prose-detection skill. 2. Place secret scanning in a separate security-audit skill with explicit user authorization. 3. Restrict scans to a user-approved repository root and exclude unrelated directories. 4. Configure scanners to redact secret values and report only file locations and secret types. 5. Do not include raw credentials or complete configuration contents in model prompts or reports. 6. Require approval before modifying, revoking, deleting, or committing any discovered material. 7. Separate MCP and startup-hook auditing into a dedicated control-plane review. 8. Clearly state whether Git history, submodules, ignored files, or nested repositories will be scanned. ]]>

T09 · Insecure Skill Coding Practices

Warning

Location: modules/hallucination-detection.md:125
Finding: Unvalidated Documentation URLs Can Trigger Internal Network Requests<![CDATA[ ## Vulnerability Details **File Location**: `modules/hallucination-detection.md:125-140` **Vulnerability Type**: Server-side request forgery style network probing **Risk Level**: Medium ### Vulnerable Instructions ```markdown ### Detection ```bash # Extract all URLs from docs rg -o 'https?://[^\s\)]+' docs/ *.md # Verify each (rate-limited, batch) while read url; do status=$(curl -sI -o /dev/null -w '%{http_code}' "$url" --max-time 5) [ "$status" != "200" ] && echo "DEAD: $status $url" done < urls.txt ``` ``` ### Technical Analysis The workflow extracts URLs from repository documentation and requests each one with `curl`. Repository prose is potentially attacker-controlled input. The command does not validate resolved addresses or reject loopback, private, link-local, multicast, or reserved networks. Shell quoting around `"$url"` reduces shell-injection risk, but it does not prevent network-level abuse. An attacker can add a URL targeting an internal service, local administrative endpoint, or cloud instance metadata address. The scanner then issues the request from the agent's network context. The example uses `HEAD` requests and reports status codes rather than response bodies, limiting direct data extraction. It can still reveal service reachability and trigger state changes on nonconforming endpoints that improperly act on `HEAD`. ### Attack Path 1. An attacker adds a URL such as a loopback, private-network, or link-local endpoint to a Markdown file. 2. A user invokes hallucination or dead-link detection. 3. The skill extracts the attacker-controlled URL. 4. `curl` sends a request from the agent host to the specified destination. 5. The reported status distinguishes reachable services from unreachable ones. 6. Repeated crafted URLs can map internal hosts or ports and may trigger vulnerable internal endpoints. ### Impact Assessment Potential effects include: - Internal network and port reachability probing. - Requests to localhost-on ...[truncated 387 chars]
Remediation: <![CDATA[ ## Remediation Suggestions 1. Make network validation opt-in and require explicit confirmation before sending requests. 2. Permit only `https` URLs unless a user explicitly authorizes another scheme. 3. Parse URLs with a dedicated URL library rather than relying on regular-expression extraction. 4. Resolve hostnames before requesting them and reject: - Loopback addresses. - RFC 1918 private addresses. - Link-local addresses, including cloud metadata ranges. - Multicast, unspecified, and reserved addresses. 5. Revalidate every redirect target and disable redirects by default. 6. Defend against DNS rebinding by connecting only to the validated resolved address. 7. Apply request-count, concurrency, response-size, and total-time limits. 8. Use an isolated outbound proxy with an allowlist for public documentation hosts. 9. Avoid reporting detailed timing or response metadata for blocked destinations. ]]>

T09 · Insecure Skill Coding Practices

Warning

Location: modules/config-file.md:55
Finding: Configuration Inheritance Can Read Files Outside the Repository<![CDATA[ ## Vulnerability Details **File Location**: `modules/config-file.md:55-75` **Vulnerability Type**: Path traversal through inherited configuration **Risk Level**: Medium ### Vulnerable Instructions ```yaml # Inherit from a base config, then apply overrides above extends: "../../.slop-config.yaml" ``` The corresponding loading procedure states: ```markdown | `extends` | str | none | Path to a base config to inherit from | ## Loading Procedure 1. Walk directories from target file up to repo root, collecting any `.slop-config.yaml` files found. 2. If `extends` is set in a config, load that base config first. 3. Merge: base config values are the defaults; the child config overrides them. 4. For list fields (`custom_words.tier1`, `allowlist`, etc.), merge lists rather than replace. 5. Validate that `thresholds.warn < thresholds.error`. If not, warn and use built-in defaults. ``` ### Technical Analysis The configuration format permits a project-controlled `extends` path, including parent-directory traversal. The documented loading procedure does not require canonicalization, confinement to the repository root, rejection of absolute paths, cycle detection, or file-size limits. A malicious repository can therefore point `extends` at a path outside the project. If the target is parseable as YAML, the loader will read and process content beyond the authorized repository boundary. Even when parsing fails, error messages may reveal whether external paths exist or disclose portions of file content. Recursive inheritance also creates denial-of-service opportunities if configurations reference each other or form a long chain. ### Attack Path 1. An attacker commits a `.slop-config.yaml` file to a repository. 2. The file sets `extends` to a traversal path or absolute path outside the repository. 3. A user scans a file beneath that configuration. 4. The loader follows the `extends` reference. 5. The agent reads and attempts to parse the out-of-repository file ...[truncated 744 chars]
Remediation: <![CDATA[ ## Remediation Suggestions 1. Resolve `extends` relative to the containing configuration file and canonicalize the result. 2. Verify that the canonical path remains beneath the canonical repository root. 3. Reject absolute paths, traversal escapes, symlink escapes, and non-regular files. 4. Maintain a set of visited canonical paths and reject inheritance cycles. 5. Set a small maximum inheritance depth. 6. Limit configuration file size and YAML alias expansion. 7. Use a safe YAML parser that cannot instantiate arbitrary objects. 8. Return generic errors that do not expose external file contents. 9. Require explicit user approval before loading any base configuration outside the current directory tree. 10. Prefer removing external inheritance entirely when the skill is used on untrusted repositories. ]]>

SkillSpector

Vulnerability Patterns

Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration

Findings (22)

Description-Behavior Mismatch

High

Confidence: 97% confidence
Finding: The skill is described as a prose slop detector, but these sections broaden it into codebase auditing, dependency validation, URL probing, config verification, stub analysis, claim verification, and remediation workflows. That scope expansion can cause the agent to inspect or act on repositories far beyond the user’s likely expectation, increasing the chance of unsafe network access, unintended code-oriented analysis, or over-privileged follow-on actions.

Hidden Instructions

High

Category: Prompt Injection
Content: inline ignore markers: ```html  The comprehensive integration tests cover ...
Confidence: 70% confidence
Finding: Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Hidden Instructions

High

Category: Prompt Injection
Content:  The comprehensive integration tests cover ...  [block of intentionally-formulaic content]  ```
Confidence: 70% confidence
Finding: Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Hidden Instructions

High

Category: Prompt Injection
Content:  [block of intentionally-formulaic content]  ``` ```rust
Confidence: 70% confidence
Finding: Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Hidden Instructions

High

Category: Prompt Injection
Content: marker should explain *why* in a trailing comment: ```html  ```
Confidence: 70% confidence
Finding: Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Description-Behavior Mismatch

High

Confidence: 98% confidence
Finding: This text instructs the slop-detector to treat unverified package recommendations as critical findings, pushing the agent into security-audit behavior beyond its declared prose-analysis purpose. In an agentic environment, this kind of scope expansion can cause the skill to overreach into dependency and security triage, leading to false escalations, policy bypass of intended tool boundaries, or unsafe autonomous decision-making based on reference text.

Hidden Instructions

High

Category: Prompt Injection
Content: opener to test the slop detector: should live under `tests/` and be excluded by path. - A glossary that defines what an "identity leak" is and shows examples: mark with `` or equivalent project-specific marker. When in doubt: leave the match flagged, surface it to a
Confidence: 70% confidence
Finding: Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Vague Triggers

Medium

Confidence: 97% confidence
Finding: The trigger set includes broad everyday terms like writing, cleanup, documentation, and quality, which can cause the skill to activate in many unrelated contexts. Over-broad invocation increases the odds that the agent applies this expansive workflow unintentionally, including the broader audit and remediation behaviors described later in the file.

Vague Triggers

Medium

Confidence: 89% confidence
Finding: The fragmented trigger wording makes the activation scope ambiguous, so the agent may interpret ordinary requests as matching this skill. While not directly a code-execution issue, ambiguity increases accidental activation and therefore magnifies the impact of the skill’s overly broad analysis and remediation instructions.

Natural-Language Policy Violations

Medium

Confidence: 95% confidence
Finding: Forcing English fallback when language detection is uncertain can produce incorrect analysis on non-English content and lead to false findings or inappropriate edits. In this skill, misclassification matters because the detector drives structured findings and possible remediation guidance, so wrong-language analysis can cascade into low-quality or harmful recommendations.

Context-Inappropriate Capability

Medium

Confidence: 95% confidence
Finding: The instructions tell the agent to verify package registries and live URLs, which are networked behaviors not justified by a writing-quality detector. In practice this can trigger outbound requests, leak contextual information about the project under review, and expand the trust boundary from local prose inspection to external systems.

Context-Inappropriate Capability

Medium

Confidence: 93% confidence
Finding: A detector skill should report findings, but this one escalates into invoking a separate generation/remediation skill to apply fixes. That creates an unexpected write/action path where a read-oriented review step can chain into content modification, increasing the risk of unauthorized edits or prompt-chaining into more capable tooling.

Autonomous Decision Making

Medium

Category: Excessive Agency
Content: etc.) with secrets or broad capability grants. - Committed credentials (run `gitleaks` / `trufflehog`). - Untrusted MCP server entries. - Hooks that auto-execute on session start. Commit any redactions or revocations *before* any other cleanup, since later passes assume an uncompromised
Confidence: 85% confidence
Finding: Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Description-Behavior Mismatch

Medium

Confidence: 97% confidence
Finding: The manifest describes the skill as detecting AI-generated writing patterns in prose, but this module repeatedly frames the slop-detector as prioritizing readability, structure, dead-code removal, correctness, and security review categories. Those activities materially broaden the skill from prose-pattern detection into code and security auditing, which is not an obvious implementation detail of the stated purpose.

Description-Behavior Mismatch

Medium

Confidence: 90% confidence
Finding: The module says the slop-detector is a tool for reviewers and discusses over-cleanup behavior, implying a generalized review/cleanup assistant rather than a detector of AI-written prose patterns. This is a semantic mismatch between the documented role of the skill and the narrower manifest description.

Natural-Language Policy Violations

Medium

Confidence: 96% confidence
Finding: The rule flags 'Asian names with Western first names' as inherently suspicious, embedding a cultural/ethnic heuristic into the detector without technical justification. In a writing-review skill, this can cause biased outputs, unfairly penalize legitimate multicultural character names, and propagate discriminatory review behavior into downstream editorial decisions.

Natural-Language Policy Violations

Medium

Confidence: 91% confidence
Finding: The skill imposes a hard 'zero em-dash' rule for newly generated prose and states that every em-dash is a finding, but it does not require user consent or justify the rule as a configurable style preference. That creates a policy-overreach risk where the agent may block, rewrite, or degrade legitimate writing styles, including accessibility, localization, or house-style requirements, based on an unverifiable heuristic for AI detection.

Vague Triggers

Medium

Confidence: 89% confidence
Finding: The prevention-mode text lists broad auto-invocation examples like '/doc-generate', '/doc-polish', and '/update-docs' without clearly constraining when the rule should apply. In an agentic environment, ambiguous trigger scope can cause the skill to run on content the user did not explicitly ask to lint or rewrite, leading to unwanted policy enforcement and content changes.

Skill Enumeration

Medium

Category: Agent Snooping
Content: ``` [FINDING 12] file: plugins/foo/skills/bar/SKILL.md line: 47-52 category: identity-and-voice-leaks/llm-self-reference severity: critical
Confidence: 80% confidence
Finding: Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Skill Enumeration

Medium

Category: Agent Snooping
Content: ``` [FINDING 12] file: plugins/foo/skills/bar/SKILL.md line: 47-52 category: identity-and-voice-leaks/llm-self-reference severity: critical
Confidence: 80% confidence
Finding: Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Skill Enumeration

Medium

Category: Agent Snooping
Content: ``` [FINDING 12] file: plugins/foo/skills/bar/SKILL.md line: 47-52 category: identity-and-voice-leaks/llm-self-reference severity: critical
Confidence: 80% confidence
Finding: Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Skill Enumeration

Medium

Category: Agent Snooping
Content: ``` [FINDING 12] file: plugins/foo/skills/bar/SKILL.md line: 47-52 category: identity-and-voice-leaks/llm-self-reference severity: critical
Confidence: 80% confidence
Finding: Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Static analysis

No suspicious patterns detected.