Back to skill

Security audit

Auto Test Generator

Security checks for vulnerabilities and agentic risk

Overview

This skill has a legitimate test-generation purpose, but its implementation can overwrite files and execute commands through an unvalidated skill name.

Install only if you will run it on trusted skills with simple names. Before general use, it should validate skill names, enforce that resolved paths stay inside the skills directory, avoid shell-based execSync, avoid interpolating raw input into generated JavaScript, and refuse to overwrite existing test.js without explicit consent.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T09 · Insecure Skill Coding Practices

Error
Location
index.js:14
Finding
Path Traversal Enables File Overwrite Outside the Skills Root## Vulnerability Details **File Location**: `index.js`, lines 7 and 14–16, with the write operation at line 63 **Vulnerability Type**: Path traversal and unsafe file overwrite **Risk Level**: High ### Vulnerable Code ```js const SKILLS_ROOT = path.resolve(__dirname, '..'); const TARGET_SKILL = process.argv[2]; ``` ```js const skillPath = path.join(SKILLS_ROOT, TARGET_SKILL); const indexJsPath = path.join(skillPath, 'index.js'); const testJsPath = path.join(skillPath, 'test.js'); if (!fs.existsSync(indexJsPath)) { console.error(`Skill ${TARGET_SKILL} not found or index.js missing.`); process.exit(1); } ``` ```js fs.writeFileSync(testJsPath, testContent); ``` ### Technical Analysis `TARGET_SKILL` is accepted directly from the command line and passed to `path.join` without validation or a containment check. Values containing traversal components such as `..` can cause `skillPath` to reference a directory outside `SKILLS_ROOT`. The existence check only confirms that an `index.js` file exists at the selected location. It does not establish that the target is an authorized skill directory or that its canonical path remains beneath `SKILLS_ROOT`. After this check, the program unconditionally writes `test.js`, overwriting an existing file with that name. Symbolic links can also undermine simple lexical path assumptions unless canonical paths are checked. Consequently, a caller who controls the CLI argument can select another accessible directory containing `index.js` and overwrite its `test.js`. ### Attack Path 1. The attacker identifies or creates a writable directory outside the intended skills root that contains an `index.js` file. 2. The attacker invokes the generator with a traversal value such as `../../attacker-controlled-directory`, adjusted for the actual directory layout. 3. `path.join(SKILLS_ROOT, TARGET_SKILL)` resolves the traversal components and selects the external directory. 4. The ...[truncated 968 chars]
Remediation
## Remediation Suggestions 1. Restrict skill identifiers to a conservative allowlist: ```js if (!/^[A-Za-z0-9_-]+$/.test(TARGET_SKILL)) { throw new Error('Invalid skill name'); } ``` 2. Resolve and verify the target path before accessing it: ```js const skillPath = path.resolve(SKILLS_ROOT, TARGET_SKILL); const relative = path.relative(SKILLS_ROOT, skillPath); if ( relative === '' || relative.startsWith(`..${path.sep}`) || relative === '..' || path.isAbsolute(relative) ) { throw new Error('Target must be a child of the skills root'); } ``` 3. If symbolic links are permitted in the directory tree, compare canonical paths obtained with `fs.realpathSync` and ensure the canonical target remains inside the canonical skills root. 4. Refuse to replace an existing `test.js` by default. Use an explicit overwrite option or an exclusive write flag such as `{ flag: 'wx' }`. 5. Consider writing generated tests to a safely created temporary directory instead of modifying the target project. 6. Verify that the target is a direct child of the skills root if nested paths are not required.

T09 · Insecure Skill Coding Practices

Error
Location
index.js:70
Finding
Untrusted Skill Name Reaches Shell and Generated JavaScript Execution Sinks## Vulnerability Details **File Location**: `index.js`, lines 7, 51–55, 63, and 70 **Vulnerability Type**: OS command injection and generated JavaScript injection **Risk Level**: Critical ### Vulnerable Code ```js const TARGET_SKILL = process.argv[2]; ``` ```js const testContent = ` const assert = require('assert'); const skill = require('./index.js'); console.log('Testing ${TARGET_SKILL}...'); try { assert.ok(skill, 'Module should export something'); console.log('✅ Export check passed'); // specific logic would go here } catch (e) { console.error('❌ Test failed:', e); process.exit(1); } `; fs.writeFileSync(testJsPath, testContent); ``` ```js try { execSync(`node ${testJsPath}`, { stdio: 'inherit' }); console.log('✅ Test run successful.'); } catch (e) { console.error('❌ Test run failed.'); process.exit(1); } ``` ### Technical Analysis The untrusted CLI argument influences two code-execution sinks. First, `testJsPath`, which is derived from `TARGET_SKILL`, is concatenated into a string passed to `execSync`. Node.js executes string-form `execSync` commands through a shell. The path is not quoted or escaped, so shell metacharacters and command-substitution syntax in a crafted directory name may be interpreted by the shell rather than treated as literal path characters. Second, when the target `index.js` appears to export a value, the raw `TARGET_SKILL` string is inserted inside a single-quoted JavaScript string in the generated `test.js`. A target name containing a quote and valid JavaScript syntax can terminate that string and inject statements into the generated file. The program writes and immediately executes that file. Independently of injection, generated tests deliberately execute target-controlled code. The exports branch evaluates `require('./index.js')`, while the no-exports branch executes `node index.js --help`. Therefore, the tool should onl ...[truncated 2094 chars]
Remediation
## Remediation Suggestions 1. Eliminate shell interpretation by replacing string-form `execSync` with an argument-array API: ```js const { execFileSync } = require('child_process'); execFileSync(process.execPath, [testJsPath], { stdio: 'inherit', shell: false }); ``` 2. Apply a strict skill-name allowlist such as `/^[A-Za-z0-9_-]+$/` and reject quotes, whitespace, separators, shell metacharacters, and traversal components. 3. Never interpolate raw input into generated source code. Serialize data as a JavaScript literal: ```js const safeSkillName = JSON.stringify(TARGET_SKILL); const testContent = ` const assert = require('assert'); const skill = require('./index.js'); console.log('Testing ' + ${safeSkillName} + '...'); `; ``` 4. Prefer passing descriptive values through environment variables or a data file rather than generating source code containing user input. 5. Apply the path-containment controls described in the path-traversal finding before reading, writing, or executing any target file. 6. Clearly warn users that analyzing a skill executes its `index.js`. For untrusted targets, perform analysis without `require` or direct execution, or run the target in a sandbox with restricted filesystem, network, environment, and process permissions. 7. Add regression tests covering skill names with quotes, spaces, command substitutions, shell metacharacters, path separators, and traversal sequences.
Vulnerability Patterns
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Findings (2)

Missing User Warnings

Medium
Confidence
94% confidence
Finding
The skill explicitly states that it creates a new `test.js` file and then runs that generated file immediately, but the description does not prominently warn the user about these side effects. This is dangerous because it combines filesystem modification with automatic execution, which can surprise users and expand risk if the generated test content is incorrect, unsafe, or targets an unexpected skill directory.

Missing User Warnings

Medium
Confidence
97% confidence
Finding
The script builds a shell command with a path derived from the user-controlled skill name and executes it via execSync using shell interpolation. If the skill name contains shell metacharacters or produces a crafted path, an attacker can trigger arbitrary command execution when the generated test is run, especially because there is no validation, quoting, or warning before execution.