Back to skill

Security audit

Heimdall

Security checks for vulnerabilities and agentic risk

Overview

Heimdall is a legitimate browser/API testing skill, but its test plans can trigger high-impact network actions with weak built-in limits, so it should be reviewed carefully before installation.

Install only if you trust the publisher and will review every plan before running it. Avoid running third-party or agent-generated plans against production accounts or paid/destructive systems unless the targets, request volume, credentials, and rollback path are clear. Treat storageState files and heimdall-runs output as sensitive, and prefer pinned commits or reviewed releases over the mutable install commands.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (4)

T09 · Insecure Skill Coding Practices

Error
Location
src/runner.ts:284
Finding

Plan-Level Hooks Bypass the Risk Authorization Gate

Content
View full analysis
0; if (plan.setup && plan.setup.length > 0 && hadRunnable) { log.info(c.cyan("▶ plan setup") + c.dim(` — ${plan.setup.length} step(s)`)); const setupError = await runPlanHook(plan.setup, ctx, "plan setup"); if (setupError) { const reason = redactString(`plan setup failed: ${setupError}`); log.err(reason); for (const p of runnable) { results.push(blocked(p.tc, p.driver, reason)); log.warn(`${p.tc.id}: ${reason}`); } runnable = []; } } ``` ```ts if (plan.teardown && plan.teardown.length > 0 && hadRunnable) { log.info(c.cyan("▶ plan teardown") + c.dim(` — ${plan.teardown.length} step(s)`)); const teardownError = await runPlanHook(plan.teardown, ctx, "plan teardown"); if (teardownError) log.warn(redactString(`plan teardown (best-effort) did not fully complete: ${teardownError}`)); } ``` The corresponding schema explicitly exposes the same unrestricted step language to hooks: ```ts setup: z .array(Step) .optional() .describe( "steps run ONCE before any case (in a throwaway context) to seed shared state; if any fails, every runnable case is marked blocked. TRUSTED, author-controlled fixtures: plan hooks run UNGATED — they are NOT subject to the per-case risk/--allow-risk gate, so keep destructive operations out of the ...[truncated 2194 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
src/schema.ts:165
Finding

Unbounded Load Steps Can Generate Unauthorized Denial-of-Service Traffic

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
src/execute.ts:313
Finding

Plan-Controlled Screenshot Baseline Path Allows Arbitrary File Creation

Content
View full analysis
Remediation
View remediation

T08 · Insecure Dependencies

Warning
Location
docker/Dockerfile:10
Finding

Container and Skill Installation Use Mutable Supply-Chain Inputs

Content
View full analysis
Remediation
View remediation
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (66)

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The declared description presents a broad live browser/API testing skill with planning and execution semantics across multiple lanes. The supplied code chunk instead implements a specific accessibility-scanning helper for browser pages using axe-core. While this is loosely related to browser testing, its primary purpose is materially narrower and different: injecting axe into a page and processing accessibility violations. It does not implement the broader declared capabilities such as API exercising, test-plan authoring, execution orchestration, blocked/non-run semantics, Docker/browser-lane management, or human-controlled login handling. Therefore the description does not accurately represent what this code chunk actually does.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

This code chunk does not implement browser or API test-plan authoring/execution. Its primary purpose is a local diagnostic command that verifies prerequisites for different drivers/lane support. While these checks may support the broader testing system, the actual behavior here is materially different from the declared purpose because it performs environment validation rather than exercising a web app or API. Therefore the description does not accurately represent this specific code chunk.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
94% confidence
Finding

The declared description emphasizes live test execution across browser/API environments. This code chunk instead performs plan ingestion, validation, filtering, and manifest generation for an "extension" driver, plus parsing external results from disk. Those are supporting/build-time utilities, not the promised test-running capability. There is no Playwright/CDP control, Docker browser orchestration, human-browser lane interaction, or API exercising in this snippet. Therefore the actual behavior is materially narrower and different from the declared purpose.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description says this skill is for creating and executing browser/API test plans. The supplied code does not author tests, run tests, drive browsers, access APIs, or manage human/browser execution lanes. Instead, it performs local cache housekeeping for Playwright browser binaries by enumerating directories in the Playwright cache, calculating their sizes, and deleting older revisions when confirmed. This is a materially different primary purpose and includes filesystem cleanup capabilities not reflected in the description.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

This code chunk's primary behavior is schema export: it converts planJsonSchema() to formatted JSON and outputs it to stdout or a file. That is materially different from the declared purpose of authoring and running live browser/API test plans. The filesystem write capability is also undeclared in the description. While schema generation could be tangentially related to a testing tool, this specific chunk does not perform the described testing actions or access the described resources, so it is a clear description-behavior mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
94% confidence
Finding

The supplied code chunk is focused entirely on plan validation UX: reading a JSON file, parsing it, validating it against Plan and TestCase schemas, collecting and formatting all schema errors, and setting exit codes. It explicitly states that validate is a dry-run that executes nothing. That is materially narrower and different from the declared purpose, which centers on authoring and running live browser/API test plans through Playwright/CDP, Docker, or human-controlled browser lanes. While validation could be a supporting part of a larger testing skill, this specific chunk does not implement the declared primary behavior and instead implements only schema validation functionality.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description presents a test execution/orchestration skill for browser and API plans. The actual code chunk does not author or run tests, control browsers, interact with APIs, or manage execution semantics. Instead, it performs output sanitization/redaction for secrets in reports and HAR artifacts. This is a materially different primary purpose from the declared skill. While redaction could be a supporting component of a larger testing system, the supplied chunk itself is specifically about secret scrubbing, an undeclared capability not represented in the description.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description centers on executing live browser/API tests in various environments. The supplied code does not run tests, interact with browsers, call APIs, or manage execution lanes. Instead, it is a pure reporting utility that compares two existing run reports and produces a regression summary, with optional filesystem write output. That is a materially different primary purpose from the declared skill behavior, so this is a clear mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The supplied code is a reporting component (src/reporters/html.ts). It reads screenshot files, formats result/evidence data into HTML, and writes an HTML report to disk. While it does represent blocked/skipped semantics in the report, its primary purpose is report rendering, not test planning or execution. Therefore, this chunk does not accurately match the declared purpose of a skill for authoring and running browser/API test plans.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

This code chunk is narrowly focused on reporting. It escapes XML, derives JUnit testcase class names, renders pass/fail/error/skipped cases into JUnit XML, and writes the XML report to a specified file path. The declared purpose describes a much broader and materially different capability: authoring and running live browser/API test plans via Playwright/CDP, Docker browsers, or human-controlled sessions. There is no evidence here of browser automation, API calls, test-plan execution, session control, or live app interaction. Because the actual code's primary purpose is CI report generation and file writing, not test execution, the description does not accurately represent this code chunk.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description centers on live browser and API testing capabilities. The supplied code chunk does not implement any such behavior; it merely reads package.json from the filesystem to obtain a version string. While version reporting can be a benign support utility, this chunk by itself is materially unrelated to the declared primary purpose and accesses a local resource not mentioned in the description. Therefore, this code chunk does not accurately represent the described skill behavior.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description presents a broad browser/API testing skill that can run live tests through browser automation or API channels. The supplied code does not implement any of that. Instead, it is a self-contained image comparison helper operating only on provided PNG buffers in memory. While visual diffing could be a supporting component within a larger testing system, this code chunk by itself does not match the declared primary purpose and exposes a materially different actual behavior.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding

The description claims a broad testing skill for running browser and API test plans in live environments using several execution lanes. The code shown is only a unit test file for accessibility-related utility functions. While it tangentially relates to browser testing by validating an axe execution bridge, its actual behavior is much narrower and does not deliver the declared primary capability set. This is a material scope mismatch rather than a minor implementation detail difference.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description centers on executing live browser/API test plans. The actual code chunk is narrowly focused on unit tests for configuration handling (loadConfig) and numeric option resolution (resolveRunNumber). It interacts only with local temp files and environment variables in the test process. There is no evidence of browser control, API calls, evidence collection, blocked/non-run semantics, or any live test execution capability. This is a material purpose mismatch, not just a supporting implementation detail.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding

The declared description is for a higher-level test-execution skill that authors and runs browser/API test plans against live systems. The supplied code chunk does not perform browser automation, API test orchestration, blocked/non-run semantics, or live app interaction. Instead, it is a dependency-free local fixture server used to support integration tests by simulating many web/API behaviors. While this is test-related, its primary purpose is materially different from the declared purpose: it provides mock endpoints and deterministic fixtures rather than running test plans. The auth/secret echo and PII-shaped response behavior are also undeclared specialized capabilities. Therefore this is a substantive description-behavior mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

There is a clear mismatch between the declared purpose and the observed code. The description promises a skill for executing browser/API test plans against real targets, but this code only contains local unit tests for helper functions related to load metrics and concurrency. These tests validate pure logic and formatting behavior in isolation; they do not access external resources, run browser sessions, hit APIs, or manage blocked/non-run semantics. While load-testing helpers could be supporting pieces of a broader testing system, this specific chunk’s actual behavior is materially narrower and different from the declared primary purpose.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared purpose centers on executing live browser/API tests through browser automation or related lanes. The actual code chunk is narrowly focused on testing data-redaction helpers: registering secrets, scrubbing strings/objects/HAR data, and ensuring env-resolved secrets are redacted in reports. This is a materially different primary purpose from browser/API test plan authoring or execution. While redaction could be a supporting feature in a larger testing skill, this code chunk itself does not implement or invoke the declared behavior and instead exercises undeclared redaction functionality.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description emphasizes a runtime testing skill for exercising live web apps/APIs through browser automation or related lanes. The actual code only contains unit tests for plan validation utilities, focused on detecting malformed test-plan cases and formatting error messages. That is a materially different primary purpose from running evidence-backed browser/API tests. While validation could be a supporting part of such a system, this chunk itself does not implement or demonstrate the declared core behavior, so the description does not accurately represent the supplied code chunk.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description describes a broad browser/API testing skill for exercising live applications through browser automation or API calls. The actual code chunk does something materially narrower and different: it is a Vitest unit test for an image comparison helper operating entirely on generated in-memory PNG buffers. It does not interact with browsers, APIs, Docker, CDP, authentication, or run/blocked semantics. This is a clear description-behavior mismatch because the primary purpose and capabilities shown in the code are unrelated to the declared skill purpose.

Content

No source excerpt is available for this finding.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 64)May include surrounding context.

md
`src/schema.ts`.

Tool Parameter Abuse

High
Category
Tool Misuse
Confidence
80% confidence
Finding

Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Content

Scanner excerpt · test/fixtures/server.ts (reported line 20)May include surrounding context.

ts
*                       drives the "zero events, time out" sse path (status 0)
 *   POST   /resource     -> 201 {"id":"res-N"}; creates an in-memory resource
 *   GET    /resource?id= -> 200 {"exists":true} if it exists, else 404
 *   DELETE /resource?id= -> 200 {"deleted":true}; removes it
 *   GET    /resources    -> 200 {"count":N,"ids":[...]} — the setup/teardown fixture
 *   DELETE /resources    -> 200 {"cleared":N}; empties the store (teardown cleanup)
 *   GET    /echo-auth    -> 200 {"authorization":"<the request's Authorization header>"}

Tool Parameter Abuse

High
Category
Tool Misuse
Confidence
80% confidence
Finding

Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Content

Scanner excerpt · test/fixtures/server.ts (reported line 22)May include surrounding context.

ts
*   GET    /resource?id= -> 200 {"exists":true} if it exists, else 404
 *   DELETE /resource?id= -> 200 {"deleted":true}; removes it
 *   GET    /resources    -> 200 {"count":N,"ids":[...]} — the setup/teardown fixture
 *   DELETE /resources    -> 200 {"cleared":N}; empties the store (teardown cleanup)
 *   GET    /echo-auth    -> 200 {"authorization":"<the request's Authorization header>"}
 *                          — reflects an injected ${env.X} secret back (config fixture)
 *   GET    /a11y-bad     -> 200 HTML with an alt-less <img>, an unlabeled control and a

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
80% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · MAP.md (reported line 133)May include surrounding context.

md
| `heimdall.schema.json` | Generated JSON Schema of a Plan (emitted from `src/schema.ts`) |
| `examples/plan.example.json` | Example plan mirroring `SAMPLE_PLAN` |
| `MAP.md` / `README.md` / `ROADMAP.md` | Engineering map / agent playbook / roadmap (feeds `improve-heimdall`) |
| `.claude/skills/heimdall/SKILL.md` | Heimdall skill — author/run plans through the three drivers |
| `.claude/skills/live-test/{SKILL,roster}.md` | Live-test skill + expert persona roster (composes agent-orchestra + visual-qa + Heimdall) |
| `.claude/workflows/improve-heimdall.js` | Orchestration workflow — 2 implementers → arbiter → adversary → applyable diff |
| `LICENSE` / `.gitignore` | MIT license / ignore rules (auth storageState, heimdall-runs evidence) |

Rp1

Medium
Category
MCP Rug Pull
Confidence
70% confidence
Finding

npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
88% confidence
Finding

The skill advertises commands that use shell execution, network access, and environment-derived secrets, but it declares no explicit tool scope or permission boundaries. In an agent setting, missing scope metadata can cause over-broad execution authority and makes it harder for orchestrators to sandbox or require approval for risky actions.

Content

No source excerpt is available for this finding.

Static analysis

Detected: suspicious.env_credential_access

Environment variable access combined with network send.

Critical
Code
suspicious.env_credential_access
Location
src/execute.ts:57