Back to skill

Security audit

drift-testing

Security checks for vulnerabilities and agentic risk

Overview

This API testing skill is mostly purpose-aligned, but it includes unsafe installation and CI examples that can run unverified code with broad local or CI privileges.

Review before installing. Use this skill only against mock, local, or dedicated test environments unless you explicitly intend to mutate the target API. Avoid the global and latest install examples; pin package and binary versions, verify checksums, install project-locally, remove shell: true from copied subprocess wrappers, and use least-privilege test tokens that are not printed in logs.

Vulnerability Patterns
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Findings (3)

T08 · Insecure Dependencies

Warning
Location
scripts/start_mock.sh:43
Finding

Automatic Installation of an Unpinned Global npm Package

Content
View full analysis
/dev/null; then echo "Prism not found — installing @stoplight/prism-cli globally..." npm install -g @stoplight/prism-cli fi ``` ### Technical Analysis The script automatically installs the latest available version of `@stoplight/prism-cli` from the configured npm registry. It does not specify an exact version, use a lockfile, verify package integrity, or request confirmation immediately before changing the global npm environment. An npm installation can execute package lifecycle scripts. Consequently, compromise of the upstream package, one of its transitive dependencies, the registry configuration, or the package maintainer account could result in arbitrary code execution. Global installation also changes shared user or system state beyond the project directory. Although Prism is legitimately required for the declared mock-server functionality, automatic global installation exceeds the minimum privilege needed. A project-local, version-pinned dependency would provide the same functionality with less risk. ### Attack Path 1. An attacker compromises the Prism package, one of its transitive dependencies, or the npm registry configured in the execution environment. 2. The attacker publishes malicious code under a version selected by the unpinned package request. 3. A user runs `scripts/start_mock.sh` on a system where `prism` is not already available. 4. The script invokes `npm install -g @stoplight/prism-cli`. 5. Malicious lifecycle or runtime code executes with the privileges of the user running the script. 6. If the script is run through an elevated shell, the malicious package may obtain system-wide privileges. ### Impact Assessment Successful exploitation can execute arbitra ...[truncated 625 chars]
Remediation
View remediation
``` 2. Commit `package.json` and `package-lock.json`, then use: ```bash npm ci --ignore-scripts ``` If lifecycle scripts are required, review them before enabling them. 3. Execute the project-local binary: ```bash npx --no-install prism mock "$SPEC" --port "$PORT" ``` 4. Do not perform automatic global installation inside the launcher. If the dependency is missing, exit with installation instructions or require explicit user confirmation. 5. Use npm integrity metadata, dependency scanning, and a trusted registry configuration. 6. Run the mock server as an unprivileged user and avoid invoking the script with `sudo`. ]]>

T03 · Remote Payload Retrieval and Execution

Error
Location
references/pactflow-and-cicd.md:78
Finding

Mutable Remote Binary Downloaded and Extracted Directly into an Executable Path

Content
View full analysis
Remediation
View remediation
/linux-x86_64.tgz" ``` 3. Verify a publisher-provided SHA-256 checksum or cryptographic signature: ```bash echo " drift.tgz" | sha256sum --check - ``` 4. Inspect archive paths before extraction and reject absolute paths or `..` traversal entries: ```bash tar tzf drift.tgz ``` 5. Extract into a project-local or dedicated tool-cache directory rather than `/usr/local/bin`. 6. Run the downloaded binary as an unprivileged user. 7. Cache only artifacts that have passed integrity verification. 8. Apply the same changes to both the GitHub Actions and GitLab CI examples. ]]>

T09 · Insecure Skill Coding Practices

Error
Location
references/pactflow-and-cicd.md:243
Finding

Command Injection Through Unnecessary Shell Invocation in Jest Wrapper

Content
View full analysis
{ const { testFile = './drift.yaml', serverUrl = 'http://localhost:8080', outputDir = './output', logLevel = 'info' } = options; return new Promise((resolve, reject) => { const child = spawn('drift', [ 'verify', '--test-files', testFile, '--server-url', serverUrl, '--log-level', logLevel, '--output-dir', outputDir ], { stdio: 'inherit', shell: true }); child.on('error', reject); ``` ### Technical Analysis The wrapper correctly constructs an argument array but then enables `shell: true`. This causes Node.js to invoke the command through a platform shell, reintroducing shell parsing of values that should otherwise be passed directly to the Drift executable. The values `testFile`, `serverUrl`, `outputDir`, and `logLevel` are supplied by the caller. If any value comes from an untrusted repository configuration, CI parameter, test fixture, environment-derived input, or user request, shell metacharacters may be interpreted as command separators, substitutions, or redirections. For example, a malicious `serverUrl` containing a shell separator followed by another command could cause that command to execute rather than being treated solely as a URL. Exact payload syntax depends on the operating system and shell. ### Attack Path 1. An attacker obtains control over one of the wrapper options, such as `serverUrl`, `testFile`, `outputDir`, or `logLevel`. 2. The attacker supplies a value containing shell control characters and an additional command. 3. `runDrift()` passes the value to `spawn()` while setting `shell: true`. 4. The operating-system shell parses the malicious value. 5. The injected command executes with the privileges and environment of the No ...[truncated 875 chars]
Remediation
View remediation
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults
Findings (31)

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The skill is described as Drift-based verification, but it also instructs use of Prism mock-server tooling and related workflows that are not clearly declared in the purpose statement. This expands operational behavior beyond the advertised scope, which can surprise downstream orchestrators or users and lead to execution of undeclared tools and network services.

Content

No source excerpt is available for this finding.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 27)May include surrounding context.

md
- [`references/test-cases.md`](references/test-cases.md) — Full test case YAML schema, all patterns, datasets, expressions

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 122)May include surrounding context.

md
- [`references/test-cases.md`](references/test-cases.md) — Full test case YAML schema, all patterns, datasets, expressions

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 275)May include surrounding context.

md
- [`references/test-cases.md`](references/test-cases.md) — Full test case YAML schema, all patterns, datasets, expressions

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 277)May include surrounding context.

md
- [`references/test-cases.md`](references/test-cases.md) — Full test case YAML schema, all patterns, datasets, expressions

Tool Parameter Abuse

High
Category
Tool Misuse
Confidence
90% confidence
Finding

The skill explicitly encourages coverage of destructive operations such as DELETE and elsewhere includes automation to keep rerunning until all tests pass. In an agent setting, broad invocation plus destructive test guidance can cause harmful state changes against live APIs, especially when the skill is triggered by generic 'make all tests pass' requests.

Content

Scanner excerpt · SKILL.md (reported line 216)May include surrounding context.

text
GET /products          → 200, 401, 404
POST /products         → 201, 400, 401
DELETE /products/{id}  → 204, 401, 403, 404

Critical: Any parameter without a spec-level example causes Value for query parameter X is missing. Supply an explicit value in parameters.query/path/headers for each.

Tool Parameter Abuse

High
Category
Tool Misuse
Confidence
80% confidence
Finding

Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Content

Scanner excerpt · evals/02-stateful-delete-endpoint.json (reported line 3)May include surrounding context.

json
{
  "skills": ["drift-testing"],
  "query": "Generate Drift tests for the DELETE /products/{id} endpoint in `evals/fixtures/simple-products-api.yaml`. The API has an empty database, so the product must be created before the test runs.",
  "files": ["evals/fixtures/simple-products-api.yaml"],
  "expected_behavior": [
    "Triggers the drift skill to handle test generation for a stateful endpoint",

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
89% confidence
Finding

The skill documents file-reading scripts and operational helpers but declares no explicit permissions or allowed-tools boundary. In an agentic environment, missing scope declarations can let the skill be invoked with broader tool access than intended, increasing the chance of unintended file access or execution flow beyond simple documentation use.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The activation text includes broad outcome-driven triggers like wanting full endpoint coverage or asking to keep running until everything passes. Over-broad routing can cause this skill to activate in contexts where destructive test logic, network calls, or repetitive loops are inappropriate, increasing the risk of accidental misuse.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
93% confidence
Finding

Using npx @pactflow/drift without pinning a version introduces a supply-chain risk because execution will resolve the latest package at runtime. A compromised upstream package, typo-squatted dependency, or breaking release could cause unreviewed code to execute in the agent environment.

Content

No source excerpt is available for this finding.

Whitespace Padding

Medium
Category
Prompt Injection
Confidence
70% confidence
Finding

Large whitespace padding was detected (a block of blank lines or a long run of spaces). This can push injected instructions below or to the right of the visible area so a human reviewer never sees them while the agent still reads them. Manual review of the hidden content is recommended.

Content

Scanner excerpt · SKILL.md (reported line 311)May include surrounding context.

md
### Step 4 — Diagnose and fix each failure

| Symptom                                  | Likely cause                                        | Fix                                                                                 |
| ---------------------------------------- | --------------------------------------------------- | ----------------------------------------------------------------------------------- |
| Got 404, expected 200                    | Test data doesn't exist                             | Add `operation:started` hook to seed the resource                                   |
| Got 200, expected 404                    | ID happens to exist                                 | Use `${notIn(...)}` or nil UUID `00000000-0000-0000-0000-000000000000`              |

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The documentation’s dynamic token example posts credentials to http://localhost:8080/auth/token, which uses plaintext HTTP and shows a password-bearing request without any warning about transport security. Even though this is presented as a localhost/testing example, users often adapt examples directly; if reused against a non-local or forwarded endpoint, credentials or tokens could be exposed to interception or logging.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
60% confidence
Finding

Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Content

Scanner excerpt · references/cli-reference.md (reported line 136)May include surrounding context.

md
## Configuration file

Create `drift.config.yaml` (or `.toml`, `.json`, `.ini`, `.json5`, `.ron`) in:

- `~/.config/drift/` (Linux)
- `~/Library/Application Support/drift/` (macOS)

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The debugging example prints the full event data structure, which may include request headers, auth tokens, payloads, or other sensitive test data. In CI or shared test environments, such logs can leak secrets and internal API details to build logs, consoles, or log aggregation systems.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The examples demonstrate reading credentials/tokens and transmitting them in requests without any warning about safe secret handling. Even in contract testing, this can normalize insecure patterns such as embedding credential flows in examples, reusing powerful tokens, or copying snippets into CI where secrets may be broadly accessible.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The reference explicitly documents retrieving secrets from environment variables and using them to obtain or return auth tokens within Drift Lua scripts. In a testing skill, this is not inherently malicious, but it expands the skill from passive API verification into credential handling and authenticated request generation, which increases the chance of secret misuse, accidental disclosure, or use against unintended targets.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 131)May include surrounding context.

md
# Step 1: Run tests and generate verification bundle
drift verify \
  --test-files drift.yaml \
  --server-url https://api.example.com/v1 \
  --generate-result \
  --output-dir ./drift-results

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 134)May include surrounding context.

md
# Step 1: Run tests and generate verification bundle
drift verify \
  --test-files drift.yaml \
  --server-url https://api.example.com/v1 \
  --generate-result \
  --output-dir ./drift-results

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 137)May include surrounding context.

md
# Step 1: Run tests and generate verification bundle
drift verify \
  --test-files drift.yaml \
  --server-url https://api.example.com/v1 \
  --generate-result \
  --output-dir ./drift-results

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 140)May include surrounding context.

md
# Step 1: Run tests and generate verification bundle
drift verify \
  --test-files drift.yaml \
  --server-url https://api.example.com/v1 \
  --generate-result \
  --output-dir ./drift-results

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 141)May include surrounding context.

md
# Step 1: Run tests and generate verification bundle
drift verify \
  --test-files drift.yaml \
  --server-url https://api.example.com/v1 \
  --generate-result \
  --output-dir ./drift-results

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 289)May include surrounding context.

md
# Step 1: Run tests and generate verification bundle
drift verify \
  --test-files drift.yaml \
  --server-url https://api.example.com/v1 \
  --generate-result \
  --output-dir ./drift-results

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 295)May include surrounding context.

md
# Step 1: Run tests and generate verification bundle
drift verify \
  --test-files drift.yaml \
  --server-url https://api.example.com/v1 \
  --generate-result \
  --output-dir ./drift-results

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 298)May include surrounding context.

md
# Step 1: Run tests and generate verification bundle
drift verify \
  --test-files drift.yaml \
  --server-url https://api.example.com/v1 \
  --generate-result \
  --output-dir ./drift-results

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 351)May include surrounding context.

md
# Step 1: Run tests and generate verification bundle
drift verify \
  --test-files drift.yaml \
  --server-url https://api.example.com/v1 \
  --generate-result \
  --output-dir ./drift-results