Back to skill

Security audit

Runbook Generator

Security checks for vulnerabilities and agentic risk

Overview

This runbook skill is mostly coherent, but it automatically reads sensitive project files and generates privileged service-install commands that need careful review before use.

Install only if you are comfortable with the skill reading infrastructure files in the project you point it at, including live .env metadata. Run it only on trusted project directories, review generated runbooks before executing any commands, and treat sudo, systemctl, docker stop/remove, and deployment steps as manual procedures requiring separate approval.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • System PersistenceInstalls backdoors, hooks, services, or scheduled tasks that survive the run
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T06 · System Persistence

Error
Location
scripts/generate_runbook.py:571
Finding
Unvalidated generation of privileged systemd installation and persistence commands<![CDATA[ ## Vulnerability Details **File Location**: `scripts/generate_runbook.py:381-382` and `scripts/generate_runbook.py:571-580` **Vulnerability Type**: T06: System Persistence, T09: Insecure Skill Coding Practices **Risk Level**: High ### Vulnerable Code ```python # Scan for systemd units for f in root.rglob(SYSTEMD_GLOB): if f.is_file() and "[Unit]" in f.read_text()[:200]: scanned.append(scan_systemd_unit(f)) ``` ```python if has_systemd: for s in scanned: if s["type"] == "systemd": unit_name = Path(s["path"]).name lines.append(f"### systemd Deploy ({unit_name})") lines.append("") lines.append("```bash") lines.append(f"sudo cp {s['path']} /etc/systemd/system/") lines.append("sudo systemctl daemon-reload") lines.append(f"sudo systemctl enable {unit_name}") lines.append(f"sudo systemctl start {unit_name}") lines.append("```") ``` ### Technical Analysis The scanner recursively accepts every `*.service` file whose first 200 characters contain `[Unit]`. It does not validate the unit's provenance, resolved location, `ExecStart` command, configured service account, executable ownership, or systemd hardening properties. For each accepted unit, the generated runbook recommends copying it into `/etc/systemd/system`, enabling it at boot, and starting it with `sudo`. Enabling a unit creates cross-session persistence. A service without an explicit `User=` directive ordinarily runs as root when installed as a system service. The generator does not itself execute these commands, so exploitation requires an operator or automation system to execute the generated runbook. Nevertheless, the commands are presented as standard deployment steps without a dedicated trust warning or mandatory review procedure. Repository-controlled paths and unit names are also embedded in shell commands without shell quoting. Spaces can break the commands ...[truncated 1700 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Do not generate `systemctl enable` or `systemctl start` commands by default. Treat installation and boot persistence as explicit, separately approved actions. 2. Add an option such as `--include-systemd-install`, disabled by default, and clearly explain that it produces privileged persistence commands. 3. Restrict systemd discovery to documented, approved directories rather than recursively trusting every `*.service` file. 4. Resolve every discovered path and verify that it remains inside the selected project root. 5. Reject symbolic links and unusual filenames by default. 6. Parse and validate `ExecStart`, `ExecStop`, `User`, `Group`, `EnvironmentFile`, and other security-sensitive directives before presenting deployment instructions. 7. Warn when `User=` is missing or set to `root`; recommend a dedicated unprivileged service account. 8. Generate a mandatory review step before installation, such as: ```bash systemd-analyze verify ./service-name.service sudo diff -u /etc/systemd/system/service-name.service ./service-name.service ``` 9. Use `shlex.quote()` for every repository-derived value embedded into a shell command. 10. Recommend systemd hardening directives where appropriate, including `NoNewPrivileges=true`, `PrivateTmp=true`, `ProtectSystem=strict`, and narrowly scoped `ReadWritePaths=`. 11. Explicitly state that generated commands must not be executed for an untrusted repository without manual security review. ]]>

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
scripts/generate_runbook.py:369
Finding
Project scan boundary can be bypassed through symbolic links<![CDATA[ ## Vulnerability Details **File Location**: `scripts/generate_runbook.py:369-389` **Vulnerability Type**: T05: Unauthorized Access and Privilege Escalation **Risk Level**: Medium ### Vulnerable Code ```python # Scan known files for filename, scanner in SCAN_TARGETS.items(): filepath = root / filename if filepath.exists(): scanned.append(scanner(filepath)) # Scan for systemd units for f in root.rglob(SYSTEMD_GLOB): if f.is_file() and "[Unit]" in f.read_text()[:200]: scanned.append(scan_systemd_unit(f)) # Scan for .env (not .example) env_file = root / ".env" if env_file.exists(): scanned.append(scan_env_file(env_file)) ``` The invoked scanners subsequently read candidates using calls such as: ```python content = path.read_text() ``` ### Technical Analysis The scanner does not reject symbolic links and does not verify that a candidate's resolved path remains beneath the selected project root. Both `Path.exists()` and `Path.read_text()` follow file symlinks. Consequently, an attacker-controlled project can make a recognized input such as `package.json`, `.env.example`, `nginx.conf`, or a nested `*.service` entry point to a file outside the project. If the target is readable by the user running the generator and has content accepted by the relevant parser, selected information from that file can be included in Markdown or JSON output. The `.env` scanner masks most non-placeholder values, which limits exposure through that parser. Other scanners preserve operational values such as package scripts, systemd executable commands, environment-file paths, Docker directives, and Nginx configuration metadata. ### Attack Path 1. An attacker creates a project containing a symbolic link with a recognized scanner filename or a nested `.service` filename. 2. The symbolic link targets a readable file outside the project directory. 3. A user or automated process invokes the generator on the attacker-controlled project. 4. The s ...[truncated 952 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Resolve the project root once and ensure every candidate remains within it: ```python root = Path(project_path).resolve() def safe_project_file(candidate: Path, root: Path) -> Path: if candidate.is_symlink(): raise ValueError(f"Symbolic links are not allowed: {candidate}") resolved = candidate.resolve(strict=True) if not resolved.is_relative_to(root): raise ValueError(f"File escapes project root: {candidate}") if not resolved.is_file(): raise ValueError(f"Not a regular file: {candidate}") return resolved ``` 2. Apply this validation before every `read_text()` operation, including recursively discovered systemd and Nginx files. 3. Reject symbolic links by default. If symlink support is operationally necessary, permit only links whose resolved targets remain under the project root. 4. Catch `OSError`, `RuntimeError`, and resolution failures consistently during discovery so broken or cyclic links do not terminate the scan. 5. Record rejected files in a warning section without reading or exposing their targets. 6. Run scans under an unprivileged account with access limited to the intended project directory. 7. Avoid running the generator as root, since doing so substantially increases the set of external files exposed through a malicious symlink. ]]>
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Rogue AgentSelf-Modification, Session Persistence
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (35)

Ae1

High
Category
analysis-evasion
Content
python3 scripts/generate_runbook.py /path/to/project
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Ae1

High
Category
analysis-evasion
Content
python3 scripts/generate_runbook.py /path/to/project
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Ae1

High
Category
analysis-evasion
Content
python3 scripts/generate_runbook.py /path/to/project
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Ae1

High
Category
analysis-evasion
Content
python3 scripts/generate_runbook.py /path/to/project
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Credential Access

High
Category
Privilege Escalation
Content
| systemd units (.service) | ExecStart/Stop/Reload, dependencies, restart policy |
| Makefile | Targets (build, test, deploy, clean, etc.) |
| package.json | Scripts (start, build, test, dev, deploy) |
| .env / .env.example | Required environment variables |
| nginx.conf | Upstream servers, listen ports, locations |

## Generated Sections
Confidence
98% confidence
Finding
Including '.env / .env.example' in the scan target list is a direct credential-access risk because .env files commonly store API keys, database passwords, tokens, and other secrets. In the context of a runbook generator, these secrets could be unintentionally surfaced in generated operational documentation, substantially increasing exposure.

Credential Access

High
Category
Privilege Escalation
Content
"""Runbook Generator — create operational runbooks from project infrastructure files.

Scans Dockerfiles, docker-compose.yml, systemd units, Makefiles, package.json,
.env files, and nginx configs to produce step-by-step operational runbooks.

Pure Python stdlib — no external dependencies.
Confidence
60% confidence
Finding
Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Credential Access

High
Category
Privilege Escalation
Content
"""Runbook Generator — create operational runbooks from project infrastructure files.

Scans Dockerfiles, docker-compose.yml, systemd units, Makefiles, package.json,
.env files, and nginx configs to produce step-by-step operational runbooks.

Pure Python stdlib — no external dependencies.
Confidence
60% confidence
Finding
Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Credential Access

High
Category
Privilege Escalation
Content
"""Runbook Generator — create operational runbooks from project infrastructure files.

Scans Dockerfiles, docker-compose.yml, systemd units, Makefiles, package.json,
.env files, and nginx configs to produce step-by-step operational runbooks.

Pure Python stdlib — no external dependencies.
Confidence
60% confidence
Finding
Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Credential Access

High
Category
Privilege Escalation
Content
"""Runbook Generator — create operational runbooks from project infrastructure files.

Scans Dockerfiles, docker-compose.yml, systemd units, Makefiles, package.json,
.env files, and nginx configs to produce step-by-step operational runbooks.

Pure Python stdlib — no external dependencies.
Confidence
60% confidence
Finding
Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Credential Access

High
Category
Privilege Escalation
Content
if f.is_file() and "[Unit]" in f.read_text()[:200]:
            scanned.append(scan_systemd_unit(f))

    # Scan for .env (not .example)
    env_file = root / ".env"
    if env_file.exists():
        scanned.append(scan_env_file(env_file))
Confidence
98% confidence
Finding
At this point the script explicitly targets the live `.env` file in the project root and scans it. In an agent skill context, accessing a live secret-bearing file for a documentation task is a significant credential-exposure risk because even metadata about required secrets can leak sensitive architecture and secret inventory.

Credential Access

High
Category
Privilege Escalation
Content
scanned.append(scan_systemd_unit(f))

    # Scan for .env (not .example)
    env_file = root / ".env"
    if env_file.exists():
        scanned.append(scan_env_file(env_file))
Confidence
98% confidence
Finding
The existence check precedes ingestion of a live secret file, enabling opportunistic harvesting whenever `.env` is present. This broadens the scan surface beyond safe templates and makes accidental disclosure likely in generated artifacts.

Env Variable Harvesting

High
Category
Data Exfiltration
Content
has_npm = any(s["type"] == "package_json" for s in scanned)
    has_nginx = any(s["type"] == "nginx" for s in scanned)

    # Collect all env vars
    all_env = {}
    for s in scanned:
        if s["type"] == "env_file":
Confidence
98% confidence
Finding
The generator deliberately aggregates environment-variable data from scanned files, including live `.env` content, into a single structure that is then rendered into output. In the context of an agent skill that may run over arbitrary projects and share results broadly, this is a genuine data-harvesting behavior that can expose secret inventory, service names, auth integrations, and deployment-sensitive metadata.

Credential Access

High
Category
Privilege Escalation
Content
lines.append("")
        lines.append("```bash")
        lines.append("# Copy .env.example to .env and fill in values")
        lines.append("cp .env.example .env")
        lines.append("```")
        lines.append("")
        lines.append("| Variable | Default/Example | Required |")
Confidence
60% confidence
Finding
Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Credential Access

High
Category
Privilege Escalation
Content
lines.append("")
        lines.append("```bash")
        lines.append("# Copy .env.example to .env and fill in values")
        lines.append("cp .env.example .env")
        lines.append("```")
        lines.append("")
        lines.append("| Variable | Default/Example | Required |")
Confidence
60% confidence
Finding
Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Lp3

Medium
Category
MCP Least Privilege
Confidence
94% confidence
Finding
The skill advertises capabilities that imply reading project files, writing output files, and invoking a shell command, but it declares no explicit tool restrictions such as permissions or allowed-tools. In an agent setting, missing scope boundaries can allow broader-than-intended tool use and make it easier for the skill to be invoked with dangerous file paths or command execution context.

Session Persistence

Medium
Category
Rogue Agent
Content
---
name: runbook-generator
description: Generate operational runbooks from project files. Scans Dockerfiles, docker-compose.yml, systemd units, Makefiles, package.json, and config files to produce step-by-step operational runbooks with start/stop/restart/deploy/rollback/troubleshoot procedures. Use when asked to create a runbook, generate ops docs, create operational documentation, build a deployment guide, document service procedures, or create an SRE runbook. Triggers on "create runbook", "ops documentation", "deployment guide", "operational docs", "SRE runbook", "service procedures", "how to deploy".
---

# Runbook Generator
Confidence
80% confidence
Finding
Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Missing User Warnings

Medium
Confidence
97% confidence
Finding
The skill explicitly scans .env files and environment variables to generate documentation, but it does not warn that secrets may be ingested and reproduced in the generated runbook. This creates a realistic risk of credential disclosure into markdown, JSON output, chat responses, logs, or version control, especially because runbooks are often widely shared.

Description-Behavior Mismatch

Medium
Confidence
97% confidence
Finding
The skill scans a live `.env` file and incorporates discovered variable names and placeholder state into generated output. Even though it masks many values, enumerating secret-bearing variable names and configuration shape can disclose sensitive operational details beyond what users may expect from a runbook generator.

Missing User Warnings

Medium
Confidence
95% confidence
Finding
The tool inspects live `.env` files without clear user-facing notice at the point of use, then reflects derived information into the generated artifact. In a documentation-generation context, that silent expansion of scan scope can surprise users and expose sensitive config metadata in shared runbooks.

Rp1

Medium
Category
MCP Rug Pull
Confidence
75% confidence
Finding
Docker image references without a specific tag (:latest is implicit) or digest (@sha256:...) can be silently replaced by a malicious image.

Rp1

Medium
Category
MCP Rug Pull
Confidence
75% confidence
Finding
Docker image references without a specific tag (:latest is implicit) or digest (@sha256:...) can be silently replaced by a malicious image.

Missing User Warnings

Medium
Confidence
84% confidence
Finding
The script auto-generates commands such as `sudo cp` into `/etc/systemd/system/`, `systemctl enable/start`, `docker compose down`, and `docker rm`, which can alter system state or stop running services. While the final footer advises review before production use, there is no specific warning adjacent to these safety-critical operations about service disruption, privilege requirements, or irreversible effects.

Sudo/Root Execution

Medium
Category
Privilege Escalation
Content
lines.append(f"### systemd Deploy ({unit_name})")
                lines.append("")
                lines.append("```bash")
                lines.append(f"sudo cp {s['path']} /etc/systemd/system/")
                lines.append("sudo systemctl daemon-reload")
                lines.append(f"sudo systemctl enable {unit_name}")
                lines.append(f"sudo systemctl start {unit_name}")
Confidence
70% confidence
Finding
Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Sudo/Root Execution

Medium
Category
Privilege Escalation
Content
lines.append(f"### systemd Deploy ({unit_name})")
                lines.append("")
                lines.append("```bash")
                lines.append(f"sudo cp {s['path']} /etc/systemd/system/")
                lines.append("sudo systemctl daemon-reload")
                lines.append(f"sudo systemctl enable {unit_name}")
                lines.append(f"sudo systemctl start {unit_name}")
Confidence
70% confidence
Finding
Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Sudo/Root Execution

Medium
Category
Privilege Escalation
Content
lines.append(f"### systemd Deploy ({unit_name})")
                lines.append("")
                lines.append("```bash")
                lines.append(f"sudo cp {s['path']} /etc/systemd/system/")
                lines.append("sudo systemctl daemon-reload")
                lines.append(f"sudo systemctl enable {unit_name}")
                lines.append(f"sudo systemctl start {unit_name}")
Confidence
70% confidence
Finding
Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Static analysis

No suspicious patterns detected.