Back to skill

Security audit

Phy Test Data Factory

Security checks for vulnerabilities and agentic risk

Overview

This is a local code-generation skill, but it needs review because its examples generate unguarded database deletion code for test teardown.

Install only if you are comfortable reviewing generated factory code before running it. Use this with an isolated disposable test database, least-privilege test credentials, pinned dependencies, and added guards such as NODE_ENV=test plus a test database allowlist before enabling any clearTestData or afterAll cleanup code.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T09 · Insecure Skill Coding Practices

Error
Location
SKILL.md:450
Finding
Unrestricted bulk database deletion in generated test teardown## Vulnerability Details **File Location**: `SKILL.md:349`, `SKILL.md:450-454`, `SKILL.md:579`, and `SKILL.md:615` **Vulnerability Type**: Unguarded destructive database operation **Risk Level**: High ### Vulnerable Code ```typescript const prisma = new PrismaClient(); ``` ```typescript export async function clearTestData() { // Delete in reverse dependency order (children before parents) await prisma.order.deleteMany(); await prisma.post.deleteMany(); await prisma.user.deleteMany(); } ``` The generated report also recommends automatic invocation during test teardown: ```typescript afterAll(clearTestData); ``` ### Technical Analysis The generated `PrismaClient` uses the application's ambient Prisma configuration, while `clearTestData()` performs unfiltered `deleteMany()` operations. Each operation deletes every accessible row in the corresponding table. The cleanup function does not verify that: - The process is running in a test environment. - The configured database is a dedicated test database. - The database host or name belongs to an approved test allowlist. - The records were created by the current test run. - The user explicitly authorized destructive cleanup. The skill also recommends connecting the function to `afterAll`, causing it to run automatically. If a developer accidentally supplies production or shared-environment database credentials, normal test execution can therefore trigger destructive deletion without an additional confirmation step. ### Attack Path 1. A user follows the skill and generates Prisma test factories. 2. The generated setup constructs `PrismaClient` from the current environment configuration. 3. The user or CI system runs the test suite with a production, staging, or shared database URL due to configuration error or environment-variable manipulation. 4. The test framework invokes `afterAll(clearTestData)`. 5. `deleteMany()` executes without ...[truncated 815 chars]
Remediation
## Remediation Suggestions 1. Make cleanup explicitly opt-in rather than automatically registering it with `afterAll`. 2. Fail closed unless the process is demonstrably running in a test environment: ```typescript function assertSafeTestDatabase() { if (process.env.NODE_ENV !== 'test') { throw new Error('Database cleanup is restricted to NODE_ENV=test'); } const databaseUrl = process.env.DATABASE_URL; if (!databaseUrl) { throw new Error('DATABASE_URL is required'); } const parsed = new URL(databaseUrl); const approvedDatabases = new Set(['app_test', 'app_test_ci']); if (!approvedDatabases.has(parsed.pathname.replace(/^\//, ''))) { throw new Error('Refusing to clean a non-approved database'); } } ``` 3. Invoke the guard immediately before every destructive operation. 4. Tag generated records with a unique test-run identifier and delete only records carrying that identifier. 5. Prefer transaction-based isolation followed by rollback rather than table-wide deletion. 6. Use a dedicated database account restricted to a dedicated test database. 7. Require an additional explicit environment variable, such as `ALLOW_TEST_DATA_DELETION=true`, for destructive cleanup. 8. Prevent test and CI credentials from accessing production databases through network and database-level access controls.

T08 · Insecure Dependencies

Warning
Location
SKILL.md:667
Finding
Unpinned third-party dependency installation instructions## Vulnerability Details **File Location**: `SKILL.md:344`, `SKILL.md:462`, and `SKILL.md:667-673` **Vulnerability Type**: Unconstrained package installation and supply-chain exposure **Risk Level**: Medium ### Vulnerable Code ```bash # TypeScript / JavaScript npm install -D @faker-js/faker # Python (Django) pip install factory_boy faker # Python (SQLAlchemy) pip install factory_boy faker sqlalchemy # Verify installation node -e "const { faker } = require('@faker-js/faker'); console.log(faker.person.fullName())" python3 -c "import factory; print('factory_boy ready')" ``` Related generated-code comments repeat the same unpinned installation guidance: ```typescript // Install: npm install -D @faker-js/faker ``` ```python # Install: pip install factory_boy faker ``` ### Technical Analysis The installation commands do not specify reviewed versions, integrity hashes, or a required lockfile. Package managers will therefore resolve whichever package versions satisfy their default behavior at installation time. This creates a mutable dependency boundary: two users following the same skill at different times may install different code. Package installation and subsequent imports can execute third-party package code with the privileges of the developer or CI account. The reviewed package names are not visibly obfuscated or typosquatted, and the skill does not specify an explicitly malicious package source. The risk arises from unconstrained future resolution, compromised upstream releases, registry compromise, dependency-chain compromise, or unexpectedly incompatible versions. ### Attack Path 1. A developer or CI job follows the dependency installation instructions. 2. `npm` or `pip` queries its configured package registry and resolves the latest available package versions. 3. An upstream package, transitive dependency, registry account, or configured package source has been compromised. 4. The pa ...[truncated 967 chars]
Remediation
## Remediation Suggestions 1. Pin every direct dependency to a specifically reviewed version. 2. Commit and enforce ecosystem lockfiles, such as `package-lock.json` or an equivalent npm lockfile. 3. For Python, use a locked requirements file with exact versions and hashes: ```bash pip install --require-hashes -r requirements-test.txt ``` 4. Generate Python lock data with a reviewed dependency-locking workflow and include hashes for transitive dependencies. 5. Configure approved registries explicitly and prevent dependency resolution from untrusted package indexes. 6. Run installation in an isolated, least-privileged environment without production credentials. 7. Disable npm lifecycle scripts where compatible with the dependencies and workflow, or review required scripts before allowing them. 8. Use automated dependency vulnerability and provenance checks before updating pinned versions. 9. Review lockfile changes as security-sensitive code changes.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (5)

Description-Behavior Mismatch

Medium
Confidence
95% confidence
Finding
The manifest promises 'pure local file analysis + code generation' and 'Zero external API', which implies a low-risk, non-destructive skill. However, the documented generated TypeScript output instantiates PrismaClient and includes create/delete database operations, so the skill can lead users to run code that performs live writes and bulk deletions against a connected database. This mismatch is dangerous because it weakens informed consent and can cause accidental modification of non-test environments.

Vague Triggers

Medium
Confidence
96% confidence
Finding
The top-level trigger list contains broad phrases like 'generate test data', 'seed database', and 'test fixtures' that are common in ordinary developer conversation. That increases the chance the skill activates unintentionally in contexts where the user only wanted advice, causing unsolicited code generation or guidance for database seeding. In this skill, accidental activation is more concerning because later outputs include live database write and delete patterns.

Vague Triggers

Medium
Confidence
97% confidence
Finding
The dedicated trigger section repeats ambiguous, high-collision phrases without constraints, confirmation steps, or examples of when not to activate. Because the skill is capable of producing destructive cleanup helpers and DB-writing factories, accidental invocation could steer users toward risky outputs they did not request. The surrounding context makes this more dangerous than a harmless text-formatting skill.

Missing User Warnings

Medium
Confidence
96% confidence
Finding
The skill generates a clearTestData function that calls deleteMany on multiple tables in dependency order, but the documentation does not warn that this is destructive or emphasize that it must only target isolated test databases. If copied into a misconfigured environment, it could wipe substantial application data quickly. The danger is elevated because the skill frames itself as routine test setup, which may reduce user caution.

Missing User Warnings

Medium
Confidence
97% confidence
Finding
The quick-start example recommends automatic teardown via afterAll(clearTestData) with no warning about data-loss risk. This normalizes destructive deletion as boilerplate and makes it easy for users to paste into projects without validating environment isolation. If test configuration points at staging or production, the resulting impact could be severe data loss.

Static analysis

Detected: suspicious.exposed_secret_literal

File appears to expose a hardcoded API secret or token.

Critical
Code
suspicious.exposed_secret_literal
Location
SKILL.md:367