Security audit

Runbook Generator

Security checks for vulnerabilities and agentic risk

Overview

This runbook skill is mostly coherent, but it automatically reads sensitive project files and generates privileged service-install commands that need careful review before use.

Install only if you are comfortable with the skill reading infrastructure files in the project you point it at, including live .env metadata. Run it only on trusted project directories, review generated runbooks before executing any commands, and treat sudo, systemctl, docker stop/remove, and deployment steps as manual procedures requiring separate approval.

A.I.G

Vulnerability Patterns

Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
System PersistenceInstalls backdoors, hooks, services, or scheduled tasks that survive the run
Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review

Findings (2)

T06 · System Persistence

Error

Location: scripts/generate_runbook.py:571
Finding: Unvalidated generation of privileged systemd installation and persistence commands
Content: View full analysis
Remediation: View remediation

T05 · Unauthorized Access and Privilege Escalation

Warning

Location: scripts/generate_runbook.py:369
Finding: Project scan boundary can be bypassed through symbolic links
Content: View full analysis
Remediation: View remediation
Path: if candidate.is_symlink(): raise ValueError(f"Symbolic links are not allowed: {candidate}") resolved = candidate.resolve(strict=True) if not resolved.is_relative_to(root): raise ValueError(f"File escapes project root: {candidate}") if not resolved.is_file(): raise ValueError(f"Not a regular file: {candidate}") return resolved ``` 2. Apply this validation before every `read_text()` operation, including recursively discovered systemd and Nginx files. 3. Reject symbolic links by default. If symlink support is operationally necessary, permit only links whose resolved targets remain under the project root. 4. Catch `OSError`, `RuntimeError`, and resolution failures consistently during discovery so broken or cyclic links do not terminate the scan. 5. Record rejected files in a warning section without reading or exposing their targets. 6. Run scans under an unprivileged account with access limited to the intended project directory. 7. Avoid running the generator as root, since doing so substantially increases the set of external files exposed through a malicious symlink. ]]>

SkillSpector

Vulnerability Patterns

Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Rogue AgentSelf-Modification, Session Persistence
MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection

Findings (35)

Ae1

High

Category: analysis-evasion
Confidence: 100% confidence
Finding: Referenced artifact was not completely inspected
Content: Scanner excerpt · SKILL.md (reported line 13)May include surrounding context.
md
python3 scripts/generate_runbook.py /path/to/project

Ae1

High

Category: analysis-evasion
Confidence: 100% confidence
Finding: Referenced artifact was not completely inspected
Content: Scanner excerpt · SKILL.md (reported line 20)May include surrounding context.
md
python3 scripts/generate_runbook.py /path/to/project

Ae1

High

Category: analysis-evasion
Confidence: 100% confidence
Finding: Referenced artifact was not completely inspected
Content: Scanner excerpt · SKILL.md (reported line 23)May include surrounding context.
md
python3 scripts/generate_runbook.py /path/to/project

Ae1

High

Category: analysis-evasion
Confidence: 100% confidence
Finding: Referenced artifact was not completely inspected
Content: Scanner excerpt · SKILL.md (reported line 26)May include surrounding context.
md
python3 scripts/generate_runbook.py /path/to/project

Credential Access

High

Category: Privilege Escalation
Confidence: 98% confidence
Finding: Including '.env / .env.example' in the scan target list is a direct credential-access risk because .env files commonly store API keys, database passwords, tokens, and other secrets. In the context of a runbook generator, these secrets could be unintentionally surfaced in generated operational documentation, substantially increasing exposure.
Content: Scanner excerpt · SKILL.md (reported line 38)May include surrounding context.
md
| systemd units (.service) | ExecStart/Stop/Reload, dependencies, restart policy | | Makefile | Targets (build, test, deploy, clean, etc.) | | package.json | Scripts (start, build, test, dev, deploy) | | .env / .env.example | Required environment variables | | nginx.conf | Upstream servers, listen ports, locations | ## Generated Sections

Credential Access

High

Category: Privilege Escalation
Confidence: 60% confidence
Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.
Content: Scanner excerpt · STATUS.md (reported line 26)May include surrounding context.
md
"""Runbook Generator — create operational runbooks from project infrastructure files. Scans Dockerfiles, docker-compose.yml, systemd units, Makefiles, package.json, .env files, and nginx configs to produce step-by-step operational runbooks. Pure Python stdlib — no external dependencies.

Credential Access

High

Category: Privilege Escalation
Confidence: 60% confidence
Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.
Content: Scanner excerpt · scripts/generate_runbook.py (reported line 5)May include surrounding context.
python
"""Runbook Generator — create operational runbooks from project infrastructure files. Scans Dockerfiles, docker-compose.yml, systemd units, Makefiles, package.json, .env files, and nginx configs to produce step-by-step operational runbooks. Pure Python stdlib — no external dependencies.

Credential Access

High

Category: Privilege Escalation
Confidence: 60% confidence
Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.
Content: Scanner excerpt · scripts/generate_runbook.py (reported line 284)May include surrounding context.
python
"""Runbook Generator — create operational runbooks from project infrastructure files. Scans Dockerfiles, docker-compose.yml, systemd units, Makefiles, package.json, .env files, and nginx configs to produce step-by-step operational runbooks. Pure Python stdlib — no external dependencies.

Credential Access

High

Category: Privilege Escalation
Confidence: 60% confidence
Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.
Content: Scanner excerpt · scripts/generate_runbook.py (reported line 482)May include surrounding context.
python
"""Runbook Generator — create operational runbooks from project infrastructure files. Scans Dockerfiles, docker-compose.yml, systemd units, Makefiles, package.json, .env files, and nginx configs to produce step-by-step operational runbooks. Pure Python stdlib — no external dependencies.

Credential Access

High

Category: Privilege Escalation
Confidence: 98% confidence
Finding: At this point the script explicitly targets the live .env file in the project root and scans it. In an agent skill context, accessing a live secret-bearing file for a documentation task is a significant credential-exposure risk because even metadata about required secrets can leak sensitive architecture and secret inventory.
Content: Scanner excerpt · scripts/generate_runbook.py (reported line 385)May include surrounding context.
python
if f.is_file() and "[Unit]" in f.read_text()[:200]: scanned.append(scan_systemd_unit(f)) # Scan for .env (not .example) env_file = root / ".env" if env_file.exists(): scanned.append(scan_env_file(env_file))

Credential Access

High

Category: Privilege Escalation
Confidence: 98% confidence
Finding: The existence check precedes ingestion of a live secret file, enabling opportunistic harvesting whenever .env is present. This broadens the scan surface beyond safe templates and makes accidental disclosure likely in generated artifacts.
Content: Scanner excerpt · scripts/generate_runbook.py (reported line 386)May include surrounding context.
python
scanned.append(scan_systemd_unit(f)) # Scan for .env (not .example) env_file = root / ".env" if env_file.exists(): scanned.append(scan_env_file(env_file))

Env Variable Harvesting

High

Category: Data Exfiltration
Confidence: 98% confidence
Finding: The generator deliberately aggregates environment-variable data from scanned files, including live .env content, into a single structure that is then rendered into output. In the context of an agent skill that may run over arbitrary projects and share results broadly, this is a genuine data-harvesting behavior that can expose secret inventory, service names, auth integrations, and deployment-sensitive metadata.
Content: Scanner excerpt · scripts/generate_runbook.py (reported line 415)May include surrounding context.
python
has_npm = any(s["type"] == "package_json" for s in scanned) has_nginx = any(s["type"] == "nginx" for s in scanned) # Collect all env vars all_env = {} for s in scanned: if s["type"] == "env_file":

Credential Access

High

Category: Privilege Escalation
Confidence: 60% confidence
Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.
Content: Scanner excerpt · scripts/generate_runbook.py (reported line 483)May include surrounding context.
python
lines.append("") lines.append("```bash") lines.append("# Copy .env.example to .env and fill in values") lines.append("cp .env.example .env") lines.append("```") lines.append("") lines.append("| Variable | Default/Example | Required |")

Credential Access

High

Category: Privilege Escalation
Confidence: 60% confidence
Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.
Content: Scanner excerpt · scripts/generate_runbook.py (reported line 818)May include surrounding context.
python
lines.append("") lines.append("```bash") lines.append("# Copy .env.example to .env and fill in values") lines.append("cp .env.example .env") lines.append("```") lines.append("") lines.append("| Variable | Default/Example | Required |")

Undeclared Tool Scope

Medium

Category: MCP Least Privilege
Confidence: 94% confidence
Finding: The skill advertises capabilities that imply reading project files, writing output files, and invoking a shell command, but it declares no explicit tool restrictions such as permissions or allowed-tools. In an agent setting, missing scope boundaries can allow broader-than-intended tool use and make it easier for the skill to be invoked with dangerous file paths or command execution context.
Content: No source excerpt is available for this finding.

Session Persistence

Medium

Category: Rogue Agent
Confidence: 80% confidence
Finding: Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.
Content: Scanner excerpt · SKILL.md (reported line 3)May include surrounding context.
md
--- name: runbook-generator description: Generate operational runbooks from project files. Scans Dockerfiles, docker-compose.yml, systemd units, Makefiles, package.json, and config files to produce step-by-step operational runbooks with start/stop/restart/deploy/rollback/troubleshoot procedures. Use when asked to create a runbook, generate ops docs, create operational documentation, build a deployment guide, document service procedures, or create an SRE runbook. Triggers on "create runbook", "ops documentation", "deployment guide", "operational docs", "SRE runbook", "service procedures", "how to deploy". --- # Runbook Generator

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 97% confidence
Finding: The skill explicitly scans .env files and environment variables to generate documentation, but it does not warn that secrets may be ingested and reproduced in the generated runbook. This creates a realistic risk of credential disclosure into markdown, JSON output, chat responses, logs, or version control, especially because runbooks are often widely shared.
Content: No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium

Category: Not specified by scanner
Confidence: 97% confidence
Finding: The skill scans a live .env file and incorporates discovered variable names and placeholder state into generated output. Even though it masks many values, enumerating secret-bearing variable names and configuration shape can disclose sensitive operational details beyond what users may expect from a runbook generator.
Content: No source excerpt is available for this finding.

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The tool inspects live .env files without clear user-facing notice at the point of use, then reflects derived information into the generated artifact. In a documentation-generation context, that silent expansion of scan scope can surprise users and expose sensitive config metadata in shared runbooks.
Content: No source excerpt is available for this finding.

Rp1

Medium

Category: MCP Rug Pull
Confidence: 75% confidence
Finding: Docker image references without a specific tag (:latest is implicit) or digest (@sha256:...) can be silently replaced by a malicious image.
Content: No source excerpt is available for this finding.

Rp1

Medium

Category: MCP Rug Pull
Confidence: 75% confidence
Finding: Docker image references without a specific tag (:latest is implicit) or digest (@sha256:...) can be silently replaced by a malicious image.
Content: No source excerpt is available for this finding.

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 84% confidence
Finding: The script auto-generates commands such as sudo cp into /etc/systemd/system/, systemctl enable/start, docker compose down, and docker rm, which can alter system state or stop running services. While the final footer advises review before production use, there is no specific warning adjacent to these safety-critical operations about service disruption, privilege requirements, or irreversible effects.
Content: No source excerpt is available for this finding.

Sudo/Root Execution

Medium

Category: Privilege Escalation
Confidence: 70% confidence
Finding: Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.
Content: Scanner excerpt · scripts/generate_runbook.py (reported line 577)May include surrounding context.
python
lines.append(f"### systemd Deploy ({unit_name})") lines.append("") lines.append("```bash") lines.append(f"sudo cp {s['path']} /etc/systemd/system/") lines.append("sudo systemctl daemon-reload") lines.append(f"sudo systemctl enable {unit_name}") lines.append(f"sudo systemctl start {unit_name}")

Sudo/Root Execution

Medium

Category: Privilege Escalation
Confidence: 70% confidence
Finding: Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.
Content: Scanner excerpt · scripts/generate_runbook.py (reported line 578)May include surrounding context.
python
lines.append(f"### systemd Deploy ({unit_name})") lines.append("") lines.append("```bash") lines.append(f"sudo cp {s['path']} /etc/systemd/system/") lines.append("sudo systemctl daemon-reload") lines.append(f"sudo systemctl enable {unit_name}") lines.append(f"sudo systemctl start {unit_name}")

Sudo/Root Execution

Medium

Category: Privilege Escalation
Confidence: 70% confidence
Finding: Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.
Content: Scanner excerpt · scripts/generate_runbook.py (reported line 579)May include surrounding context.
python
lines.append(f"### systemd Deploy ({unit_name})") lines.append("") lines.append("```bash") lines.append(f"sudo cp {s['path']} /etc/systemd/system/") lines.append("sudo systemctl daemon-reload") lines.append(f"sudo systemctl enable {unit_name}") lines.append(f"sudo systemctl start {unit_name}")