Auto Test Generator

• Outcome: Review
• Latest audit: Sep 10, 2026 · 8:11 PM UTC
• Version: 1.0.0

Overview

This skill has a legitimate test-generation purpose, but its implementation can overwrite files and execute commands through an unvalidated skill name.

Install only if you will run it on trusted skills with simple names. Before general use, it should validate skill names, enforce that resolved paths stay inside the skills directory, avoid shell-based execSync, avoid interpolating raw input into generated JavaScript, and refuse to overwrite existing test.js without explicit consent.

A.I.G

Vulnerability Patterns

• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally

Findings (2)

T09 · Insecure Skill Coding Practices

Error

• Location: index.js:14

Finding

Path Traversal Enables File Overwrite Outside the Skills Root

Content

View full analysis

Vulnerability Details

File Location: index.js, lines 7 and 14–16, with the write operation at line 63
Vulnerability Type: Path traversal and unsafe file overwrite
Risk Level: High

Vulnerable Code

js

const SKILLS_ROOT = path.resolve(__dirname, '..');
const TARGET_SKILL = process.argv[2];

js

const skillPath = path.join(SKILLS_ROOT, TARGET_SKILL);
const indexJsPath = path.join(skillPath, 'index.js');
const testJsPath = path.join(skillPath, 'test.js');

if (!fs.existsSync(indexJsPath)) {
  console.error(`Skill ${TARGET_SKILL} not found or index.js missing.`);
  process.exit(1);
}

js

fs.writeFileSync(testJsPath, testContent);

Technical Analysis

TARGET_SKILL is accepted directly from the command line and passed to path.join without validation or a containment check. Values containing traversal components such as .. can cause skillPath to reference a directory outside SKILLS_ROOT.

The existence check only confirms that an index.js file exists at the selected location. It does not establish that the target is an authorized skill directory or that its canonical path remains beneath SKILLS_ROOT. After this check, the program unconditionally writes test.js, overwriting an existing file with that name.

Symbolic links can also undermine simple lexical path assumptions unless canonical paths are checked. Consequently, a caller who controls the CLI argument can select another accessible directory containing index.js and overwrite its test.js.

Attack Path

1. The attacker identifies or creates a writable directory outside the intended skills root that contains an index.js file.
2. The attacker invokes the generator with a traversal value such as ../../attacker-controlled-directory, adjusted for the actual directory layout.
3. path.join(SKILLS_ROOT, TARGET_SKILL) resolves the traversal components and selects the external directory.
4. The ...[truncated 968 chars]

Remediation

View remediation

Remediation Suggestions

1. Restrict skill identifiers to a conservative allowlist:

   js

   if (!/^[A-Za-z0-9_-]+$/.test(TARGET_SKILL)) {
     throw new Error('Invalid skill name');
   }
   

2. Resolve and verify the target path before accessing it:

   js

   const skillPath = path.resolve(SKILLS_ROOT, TARGET_SKILL);
   const relative = path.relative(SKILLS_ROOT, skillPath);
   
   if (
     relative === '' ||
     relative.startsWith(`..${path.sep}`) ||
     relative === '..' ||
     path.isAbsolute(relative)
   ) {
     throw new Error('Target must be a child of the skills root');
   }
   

3. If symbolic links are permitted in the directory tree, compare canonical paths obtained with fs.realpathSync and ensure the canonical target remains inside the canonical skills root.

4. Refuse to replace an existing test.js by default. Use an explicit overwrite option or an exclusive write flag such as { flag: 'wx' }.

5. Consider writing generated tests to a safely created temporary directory instead of modifying the target project.

6. Verify that the target is a direct child of the skills root if nested paths are not required.

T09 · Insecure Skill Coding Practices

Error

• Location: index.js:70

Finding

Untrusted Skill Name Reaches Shell and Generated JavaScript Execution Sinks

Content

View full analysis

Vulnerability Details

File Location: index.js, lines 7, 51–55, 63, and 70
Vulnerability Type: OS command injection and generated JavaScript injection
Risk Level: Critical

Vulnerable Code

js

const TARGET_SKILL = process.argv[2];

js

const testContent = `
const assert = require('assert');
const skill = require('./index.js');

console.log('Testing ${TARGET_SKILL}...');

try {
  assert.ok(skill, 'Module should export something');
  console.log('✅ Export check passed');
  
  // specific logic would go here
  
} catch (e) {
  console.error('❌ Test failed:', e);
  process.exit(1);
}
`;
fs.writeFileSync(testJsPath, testContent);

js

try {
  execSync(`node ${testJsPath}`, { stdio: 'inherit' });
  console.log('✅ Test run successful.');
} catch (e) {
  console.error('❌ Test run failed.');
  process.exit(1);
}

Technical Analysis

The untrusted CLI argument influences two code-execution sinks.

First, testJsPath, which is derived from TARGET_SKILL, is concatenated into a string passed to execSync. Node.js executes string-form execSync commands through a shell. The path is not quoted or escaped, so shell metacharacters and command-substitution syntax in a crafted directory name may be interpreted by the shell rather than treated as literal path characters.

Second, when the target index.js appears to export a value, the raw TARGET_SKILL string is inserted inside a single-quoted JavaScript string in the generated test.js. A target name containing a quote and valid JavaScript syntax can terminate that string and inject statements into the generated file. The program writes and immediately executes that file.

Independently of injection, generated tests deliberately execute target-controlled code. The exports branch evaluates require('./index.js'), while the no-exports branch executes node index.js --help. Therefore, the tool should onl ...[truncated 2094 chars]

Remediation

View remediation

Remediation Suggestions

1. Eliminate shell interpretation by replacing string-form execSync with an argument-array API:

   js

   const { execFileSync } = require('child_process');
   
   execFileSync(process.execPath, [testJsPath], {
     stdio: 'inherit',
     shell: false
   });
   

2. Apply a strict skill-name allowlist such as /^[A-Za-z0-9_-]+$/ and reject quotes, whitespace, separators, shell metacharacters, and traversal components.

3. Never interpolate raw input into generated source code. Serialize data as a JavaScript literal:

   js

   const safeSkillName = JSON.stringify(TARGET_SKILL);
   const testContent = `
   const assert = require('assert');
   const skill = require('./index.js');
   console.log('Testing ' + ${safeSkillName} + '...');
   `;
   

4. Prefer passing descriptive values through environment variables or a data file rather than generating source code containing user input.

5. Apply the path-containment controls described in the path-traversal finding before reading, writing, or executing any target file.

6. Clearly warn users that analyzing a skill executes its index.js. For untrusted targets, perform analysis without require or direct execution, or run the target in a sandbox with restricted filesystem, network, environment, and process permissions.

7. Add regression tests covering skill names with quotes, spaces, command substitutions, shell metacharacters, path separators, and traversal sequences.

SkillSpector

Vulnerability Patterns

• Trigger Abuse: Overly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code

Findings (2)

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 94% confidence

Finding

The skill explicitly states that it creates a new test.js file and then runs that generated file immediately, but the description does not prominently warn the user about these side effects. This is dangerous because it combines filesystem modification with automatic execution, which can surprise users and expand risk if the generated test content is incorrect, unsafe, or targets an unexpected skill directory.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 97% confidence

Finding

The script builds a shell command with a path derived from the user-controlled skill name and executes it via execSync using shell interpolation. If the skill name contains shell metacharacters or produces a crafted path, an attacker can trigger arbitrary command execution when the generated test is run, especially because there is no validation, quoting, or warning before execution.

Content

No source excerpt is available for this finding.