Back to skill

Security audit

tutorial-updates

Security checks for vulnerabilities and agentic risk

Overview

The skill has a legitimate tutorial-generation purpose, but it asks agents to run repository-controlled commands with broad local authority and limited user control.

Install only if you trust the repositories where you will run it. Review all .tape, .manifest.yaml, Playwright specs, Makefile, Cargo build scripts, and install commands first, and run the workflow in a sandbox or disposable checkout without secrets when possible.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (3)

T09 · Insecure Skill Coding Practices

Error
Location
SKILL.md:252
Finding

Arbitrary Shell Command Execution from Project-Controlled Tape Files

Content
View full analysis
/dev/null; then echo "WARNING: Command may fail or hang: $cmd" fi done ``` ### Technical Analysis Commands are extracted from project-controlled VHS `Type` directives and passed directly to `bash -c`. The workflow does not impose an executable allowlist, reject shell metacharacters, request per-command authorization, or isolate execution in a sandbox. Because `bash -c` interprets the complete string as shell code, an attacker can use command substitution, pipelines, redirections, chained commands, environment expansion, or arbitrary executable invocations. The five-second timeout only limits runtime; it does not prevent commands from immediately deleting files, reading credentials, modifying the repository, or initiating network connections. The use of: ```bash for cmd in $(extract_commands "$tape_file") ``` also applies shell word splitting and pathname expansion to extracted commands, making validation behavior unreliable. However, the primary security issue is direct execution of repository-controlled strings through a shell. ### Attack Path 1. An attacker contributes or modifies a discovered `.tape` file. 2. The attacker places a malicious command in a `Type` directive, for example a command that reads local configuration or modifies project files. 3. A user or agent invokes the tutorial-update workflow on the repository. 4. The validation phase calls `extract_comman ...[truncated 888 chars]
Remediation
View remediation
`. 5. Require explicit user approval for every command that cannot be validated against a trusted policy. 6. If behavioral execution is essential, use an ephemeral container or virtual machine with: - A read-only source workspace - No host credential mounts - Network access disabled by default - A nonprivileged user - Resource and process limits - A disposable output directory 7. Replace the word-splitting loop with a line-preserving construct such as `while IFS= read -r cmd`. 8. Treat the timeout as availability protection only, not as a security control. ]]>

T09 · Insecure Skill Coding Practices

Error
Location
SKILL.md:448
Finding

Unrestricted Execution of Manifest-Defined Background Commands

Content
View full analysis
Remediation
View remediation

T08 · Insecure Dependencies

Warning
Location
SKILL.md:582
Finding

Unpinned Third-Party Tool and Browser Dependency Installation

Content
View full analysis
.gif` directive | | Browser spec requires server | Start server before running spec | | GIF too large | Adjust fps/scale in gif-generation | ``` ### Technical Analysis The VHS recovery command explicitly installs the mutable `@latest` release. The Playwright command installs an unversioned npm dependency and then downloads a browser through `npx playwright install chromium`. These operations are not tied to a reviewed version, lockfile state, checksum, or signature in the documented workflow. Consequently, the effective installed code and browser binaries can change over time without changes to the audited skill. Package installation may execute build or lifecycle behavior, while browser installation downloads and installs external binary artifacts. This creates a supply-chain exposure and makes the workflow non-reproducible. ### Attack Path 1. The required VHS or Playwright tool is absent from the user's environment. 2. The user follows the documented error-resolution command. 3. The package manager resolves the current mutable release and its dependency graph. 4. Package installation or browser download retrieves code and artifacts that were not fixed at audit time. 5. If an upstream package, release, dependency, registry account, or distribution artifact has been compromised, malicious code can execute during installation or later tool use. This finding does not establish that the named upstream projects are malicious. The vulnerability is the lack of version and integrity con ...[truncated 545 chars]
Remediation
View remediation
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • System Prompt LeakageDirect Leakage, Indirect Extraction, Tool-Based Exfiltration
  • Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (26)

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 236)May include surrounding context.

md
skill_count=$(find "$skill_dir" -name "SKILL.md" 2>/dev/null | wc -l)

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

The skill instructs extracting commands from tutorial tape files and executing each with bash -c during validation. Because tape files are project content and therefore untrusted, this creates a direct command-execution path that can run arbitrary shell payloads embedded in Type directives.

Content

No source excerpt is available for this finding.

Tool Parameter Abuse

High
Category
Tool Misuse
Confidence
96% confidence
Finding

The documented --skip-validation flag allows bypassing the very checks intended to catch malformed or dangerous tape commands before recording. In the context of this skill, where other sections already instruct executing extracted commands and running project build steps, a validation bypass materially increases the chance of unsafe execution and bad outputs.

Content

Scanner excerpt · SKILL.md (reported line 272)May include surrounding context.

md
| Flag | Behavior |
|------|----------|
| `--validate-only` | Run validation without generating GIF |
| `--skip-validation` | Bypass validation for rapid regeneration |

### Validation Exit Criteria

Direct Prompt Extraction

High
Category
System Prompt Leakage
Confidence
85% confidence
Finding

Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Content

Scanner excerpt · SKILL.md (reported line 186)May include surrounding context.

Check that the tape file has valid VHS syntax:

bash
# Required: Output directive
grep -q '^Output ' "$tape_file" || echo "ERROR: Missing Output directive"

# Check for balanced quotes in Type directives

Direct Prompt Extraction

High
Category
System Prompt Leakage
Confidence
85% confidence
Finding

Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Content

Scanner excerpt · SKILL.md (reported line 187)May include surrounding context.

Check that the tape file has valid VHS syntax:

bash
# Required: Output directive
grep -q '^Output ' "$tape_file" || echo "ERROR: Missing Output directive"

# Check for balanced quotes in Type directives

Direct Prompt Extraction

High
Category
System Prompt Leakage
Confidence
85% confidence
Finding

Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Content

Scanner excerpt · SKILL.md (reported line 276)May include surrounding context.

Check that the tape file has valid VHS syntax:

bash
# Required: Output directive
grep -q '^Output ' "$tape_file" || echo "ERROR: Missing Output directive"

# Check for balanced quotes in Type directives

Direct Prompt Extraction

High
Category
System Prompt Leakage
Confidence
85% confidence
Finding

Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Content

Scanner excerpt · SKILL.md (reported line 442)May include surrounding context.

Check that the tape file has valid VHS syntax:

bash
# Required: Output directive
grep -q '^Output ' "$tape_file" || echo "ERROR: Missing Output directive"

# Check for balanced quotes in Type directives

Direct Prompt Extraction

High
Category
System Prompt Leakage
Confidence
85% confidence
Finding

Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Content

Scanner excerpt · modules/markdown-generation.md (reported line 406)May include surrounding context.

Check that the tape file has valid VHS syntax:

bash
# Required: Output directive
grep -q '^Output ' "$tape_file" || echo "ERROR: Missing Output directive"

# Check for balanced quotes in Type directives

Direct Prompt Extraction

High
Category
System Prompt Leakage
Confidence
85% confidence
Finding

Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Content

Scanner excerpt · modules/tape-validation.md (reported line 84)May include surrounding context.

Check that the tape file has valid VHS syntax:

bash
# Required: Output directive
grep -q '^Output ' "$tape_file" || echo "ERROR: Missing Output directive"

# Check for balanced quotes in Type directives

Direct Prompt Extraction

High
Category
System Prompt Leakage
Confidence
85% confidence
Finding

Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Content

Scanner excerpt · modules/tape-validation.md (reported line 85)May include surrounding context.

Check that the tape file has valid VHS syntax:

bash
# Required: Output directive
grep -q '^Output ' "$tape_file" || echo "ERROR: Missing Output directive"

# Check for balanced quotes in Type directives

Direct Prompt Extraction

High
Category
System Prompt Leakage
Confidence
85% confidence
Finding

Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Content

Scanner excerpt · modules/tape-validation.md (reported line 98)May include surrounding context.

Check that the tape file has valid VHS syntax:

bash
# Required: Output directive
grep -q '^Output ' "$tape_file" || echo "ERROR: Missing Output directive"

# Check for balanced quotes in Type directives

Direct Prompt Extraction

High
Category
System Prompt Leakage
Confidence
85% confidence
Finding

Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Content

Scanner excerpt · modules/tape-validation.md (reported line 242)May include surrounding context.

Check that the tape file has valid VHS syntax:

bash
# Required: Output directive
grep -q '^Output ' "$tape_file" || echo "ERROR: Missing Output directive"

# Check for balanced quotes in Type directives

Direct Prompt Extraction

High
Category
System Prompt Leakage
Confidence
85% confidence
Finding

Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Content

Scanner excerpt · modules/tape-validation.md (reported line 245)May include surrounding context.

Check that the tape file has valid VHS syntax:

bash
# Required: Output directive
grep -q '^Output ' "$tape_file" || echo "ERROR: Missing Output directive"

# Check for balanced quotes in Type directives

Tool Parameter Abuse

High
Category
Tool Misuse
Confidence
93% confidence
Finding

Documenting and supporting a '--skip-validation' mode creates a straightforward bypass for the very safety checks this module is meant to enforce. In this skill, validation includes checking command correctness and demo-data assumptions before running VHS; skipping it can allow malformed or unsafe tape contents to proceed into execution workflows, increasing the chance of harmful or misleading command execution.

Content

Scanner excerpt · modules/tape-validation.md (reported line 342)May include surrounding context.

done

text

### `--skip-validation`

Bypass validation for rapid regeneration:

Tool Parameter Abuse

High
Category
Tool Misuse
Confidence
91% confidence
Finding

The documented exit code for 'validation skipped (--skip-validation)' reinforces and normalizes a validation-bypass pathway. While this line alone is documentation, in context it reflects a supported control-flow state where safety checks are intentionally omitted before VHS execution.

Content

Scanner excerpt · modules/tape-validation.md (reported line 360)May include surrounding context.

md
|------|---------|
| 0 | Validation passed |
| 1 | Validation failed (errors found) |
| 2 | Validation skipped (--skip-validation) |

## Error Message Format

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The trigger list includes broad terms like tutorial, gif, demo, and documentation, which can cause the skill to activate in many ordinary conversations. Because this skill performs validation, rebuilding, recording, and file modification workflows, overbroad invocation increases the chance of unintended high-risk actions on the wrong repository or without clear user intent.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The skill presents a pipeline that updates tutorials, generates assets, and writes documentation files, but it does not prominently require explicit user acknowledgment that repository contents will be modified. In an autonomous agent setting, unclear modification boundaries can lead to unintended writes, generated artifacts, and repo churn.

Content

No source excerpt is available for this finding.

Unsafe Defaults

Medium
Category
Tool Misuse
Confidence
93% confidence
Finding

Providing a skip-validation path as a normal documented option is an unsafe default because it normalizes bypassing guardrails in a workflow that consumes untrusted repository content. Even if not enabled by default, its presence makes it easier for users or agents to trade safety for speed and miss malicious or broken commands.

Content

Scanner excerpt · SKILL.md (reported line 272)May include surrounding context.

md
| Flag | Behavior |
|------|----------|
| `--validate-only` | Run validation without generating GIF |
| `--skip-validation` | Bypass validation for rapid regeneration |

### Validation Exit Criteria

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill includes logic to detect a build system and rebuild/install binaries from the current repository, including cargo install --path . and make build. Running repository-controlled build steps can execute arbitrary code via build scripts, Makefile recipes, or install hooks, especially when the repository content is adversarial or compromised.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
70% confidence
Finding

npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The manifest schema explicitly allows a requires array of shell commands to be run as background processes before Playwright specs. Because these commands come from manifest data and the documentation provides no trust boundary, allowlist, confirmation, or sandboxing guidance, a malicious or tampered manifest could trigger arbitrary command execution on the host during tutorial generation.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The module includes instructions to modify README.md in place using broad text-replacement logic, but it does not require confirmation, backups, dry-run behavior, or clear user warning before editing repository files. In an agentic context, this can lead to unintended or destructive changes to user-controlled documentation, especially if section matching is imperfect or the repository contains important manually maintained content.

Content

No source excerpt is available for this finding.

Unsafe Defaults

Medium
Category
Tool Misuse
Confidence
92% confidence
Finding

An unsafe default is present because the module advertises an easy mechanism to bypass pre-flight validation in a process that later runs shell-oriented demo commands. In this context, tape files can influence command validation and recording behavior, so making bypass available materially weakens defense-in-depth and increases the likelihood that invalid or dangerous inputs reach execution stages.

Content

Scanner excerpt · modules/tape-validation.md (reported line 342)May include surrounding context.

done

text

### `--skip-validation`

Bypass validation for rapid regeneration:

Unsafe Defaults

Medium
Category
Tool Misuse
Confidence
95% confidence
Finding

The environment variable 'SKIP_VALIDATION' provides a trivial external toggle to disable validation before calling 'vhs'. Environment-based bypasses are especially risky in automation and CI because they can be set accidentally or maliciously, silently weakening safeguards and allowing unvalidated tape-driven commands to proceed.

Content

Scanner excerpt · modules/tape-validation.md (reported line 348)May include surrounding context.

bash
# Skip validation when commands are known-good
if [ "$SKIP_VALIDATION" != "true" ]; then
  validate_tape "$tape_file" || exit 1
fi
vhs "$tape_file"

Unsafe Defaults

Medium
Category
Tool Misuse
Confidence
90% confidence
Finding

By assigning a formal exit code to skipped validation, the design treats omission of checks as an expected operational outcome. In a tool intended to catch stale binaries, invalid flags, and missing demo data before execution, this undermines secure defaults and can cause downstream automation to accept runs that never underwent safety validation.

Content

Scanner excerpt · modules/tape-validation.md (reported line 360)May include surrounding context.

md
|------|---------|
| 0 | Validation passed |
| 1 | Validation failed (errors found) |
| 2 | Validation skipped (--skip-validation) |

## Error Message Format

Static analysis

No suspicious patterns detected.