Back to skill

Security audit

Kane CLI by TestMu AI

Security checks for vulnerabilities and agentic risk

Overview

ClawScan marked this skill suspicious with medium confidence, chiefly because scanners flag session-cookie extraction and risky install guidance; the review is incomplete.

Consider avoiding installation until the cookie-handling and package-install concerns have been reviewed. The skillspector scan was partial: it covered 25 of 27 components, with two scripts only partially analyzed due to parser limits; AIG completed without findings, which does not resolve the concerns or fill those coverage gaps.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Output HandlingUnvalidated Output Injection, Cross-Context Output, Unbounded Output
  • Rogue AgentSelf-Modification, Session Persistence
Findings (29)

YARA rule 'info_stealer': Information stealer patterns (credential harvesting, browser data theft) [malware]

High
Category
YARA Match
Confidence
75% confidence
Finding

YARA rule matched a known malware signature (reverse shell, backdoor, ransomware, C2 framework, or info stealer).

Content

Scanner excerpt · SKILL.md (reported line 267)May include surrounding context.

{{password}}" --agent
--variables '{"username":{"value":"alice"},"password":{"value":"s3cret","secret":true}}'

text

---

## 4. Writing objectives

How you phrase the objective string determines what the agent does. Four patterns:

> For the full catalog — every action verb, every assertion analyze method (Visual / Textual-DOM / URL / Title / DevTools→Network/Console/Performance/Cookies/localStorage/Clipboard), direct API calls, operators, chaining, conditional/negative patterns, and worked examples — Read `references/objectives-cookbook.md`. Same grammar applies to one-shot `kane-cli run` objectives and `_test.md` step bodies.

| Pattern | Trigger words | Behavior |
|---|---|---|
| 🎯 **Action** | "go to", "click", "type", "search", "fill" | Performs browser actions |
| ✅ **Assertion** | "assert", "verify", "confirm", "check that" | Pass/fail check on a condition |
| 📦 **Extraction** | "store X as 'name'" | Persists a value into `run_end.final_state` |

Anti-Refusal Statement

High
Category
Anti-Refusal
Confidence
80% confidence
Finding

Skill instructs the agent to omit warnings, disclaimers, or ethical commentary. Stripping safety caveats hides risk from the user and is a common jailbreak preamble.

Content

Scanner excerpt · references/assurance.md (reported line 131)May include surrounding context.

md
### 6.1 A designed test is the design — do not edit it by hand

Every designed `_test.md` carries an `assurance:` block in its frontmatter: `id: t-<slug>` and a `base:` pin naming the design version the file came from. The graph follows edits through that pin (0.8.4+): an edit made on top of the current design version is adopted at the file's next `testmd run` / `testrun run` as the next version of that test — no review checkpoint, no warning — and from then on the edit IS the design; coverage, gaps and reconcile build on it. An edit made on top of an older version is kept out of the graph, and the run reports a sync warning naming the test.

So a failing authored run is never, by itself, a reason to rewrite the step. Classify the failure first (the pack's `failure.yaml` and the network capture it references — `references/debug.md`), then route it:

Hidden Instructions

High
Category
Prompt Injection
Confidence
70% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · references/explore.md (reported line 1)May include surrounding context.

md
<!-- kane-cli skill reference: exploration on HyperExecute, and the tests it designs into a shared project. Read when the person asks to explore an app, map its flows, shape, stop or re-run an exploration, or get and run the tests an exploration made. Requires kane-cli 0.10.0+. -->

# Exploration: map an app on HyperExecute, and get the tests it designs

Agent Config Directory Access

High
Category
Agent Snooping
Confidence
90% confidence
Finding

Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.

Content

Scanner excerpt · references/live-strip.md (reported line 48)May include surrounding context.

text
Want to watch runs live in your status bar?
  1. Turn it on (Recommended): one line names the current step while kane-cli works. It keeps your current status line, shows only in the session that started the run, and turns off with one command. It edits ~/.claude/settings.json and keeps a backup.
  2. Not now

Agent Config Directory Access

High
Category
Agent Snooping
Confidence
90% confidence
Finding

Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.

Content

Scanner excerpt · references/live-strip.md (reported line 56)May include surrounding context.

4. Turning it on and off

These commands change the person's Claude Code settings, so run them only after a clear yes. If you did not just ask the question above, tell them what will change first: This edits ~/.claude/settings.json (a backup is kept) and adds one small script under ~/.testmuai/kaneai/bin/.

bash
npx @testmuai/kane-cli-skill strip enable      # turn it on

YARA rule 'info_stealer': Information stealer patterns (credential harvesting, browser data theft) [malware]

High
Category
YARA Match
Confidence
75% confidence
Finding

YARA rule matched a known malware signature (reverse shell, backdoor, ransomware, C2 framework, or info stealer).

Content

Scanner excerpt · references/objectives-cookbook.md (reported line 1)May include surrounding context.

md
<!-- Read this when authoring objective strings — for either `kane-cli run "..."` or step bodies inside a `_test.md`. Owns the full catalog of action verbs, assertion patterns, checkpoint analyze methods (Visual, Textual/DOM, URL, Title, DevTools→Network/Console/Performance/Cookies/localStorage), operators, extraction patterns, direct API calls, chaining, variables, and common pitfalls. The same objective grammar applies to one-shot runs and testmd steps. -->

# Writing Kane-CLI Objectives — Pattern Cookbook

Read this whenever you're constructing the prose objective for `kane-cli run "<objective>"` or the body of a `## Step` in a `_test.md` file. Both surfaces feed the same agent and accept the same grammar.

> **Mobile runs** (`--target emulator|simulator`) share this objective grammar. Checkpoint support depends on platform and app: native network and

Session-cookie extraction is recommended without warning that authentication secrets will be persisted and included in results.

High
Category
Output Handling
Confidence
97% confidence
Finding

The cookbook recommends extracting a session-cookie value and explicitly states that extracted values persist in run_end.final_state and appear in results. This can expose reusable authentication secrets to logs, reports, or report readers; the input-variable masking guidance does not establish protection for dynamically extracted secrets.

Content

No source excerpt is available for this finding.

Activation extends to any browser task rather than being constrained to KaneCLI E2E testing.

Medium
Category
Trigger Abuse
Confidence
91% confidence
Finding

The description says to use the skill for 'any task needing a real browser,' including navigation, clicks, form filling, and screenshots. This extends activation beyond E2E testing to ordinary browsing tasks, without exclusions distinguishing a KaneCLI test request from unrelated browser assistance; the decision tree at L165 reinforces that broad scope.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 297)May include surrounding context.

The agent can make API calls itself — not just observe the page's traffic. Phrase an explicit call and name the response:

text
"Call POST https://api.example.com/login with body {...}, save the response as login,
 assert the response status is 200"

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · references/objectives-cookbook.md (reported line 281)May include surrounding context.

Phrase an explicit call and name the response with "save the response as …":

text
Call POST https://api.example.com/orders with body {"item": "sku_42", "qty": 1}, save the response as order
Hit GET https://api.example.com/orders/123, save the response as fetched

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · references/objectives-cookbook.md (reported line 282)May include surrounding context.

text
Call POST https://api.example.com/orders with body {"item": "sku_42", "qty": 1}, save the response as order
Hit GET https://api.example.com/orders/123, save the response as fetched

A pasted curl works too and is kept verbatim (method, headers, body, auth):

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · references/objectives-cookbook.md (reported line 548)May include surrounding context.

Example D — Seed via API, then verify in the UI

text
"Call POST https://api.example.com/orders with body {"item": "sku_42", "qty": 1},
 save the response as order,
 assert the response status is 201,
 then open https://app.example.com/orders,

Session Persistence

Medium
Category
Rogue Agent
Confidence
60% confidence
Finding

Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Content

Scanner excerpt · references/agent-config.md (reported line 63)May include surrounding context.

Compose the whole file yourself and write it with one shell command. Use your shell tool, not your file-editing tool: many hosts confine the editing tool to the project folder, and this file is in the home folder.

bash
mkdir -p ~/.testmuai/kaneai/agent-config && cat > ~/.testmuai/kaneai/agent-config/config.json <<'EOF'
{ ...the full JSON... }
EOF

MCP server referenced without pinned version: 'npx @testmuai/kane-cli-skill'.

Medium
Category
MCP Rug Pull
Confidence
92% confidence
Finding

The suggested npx command has no pinned package version, so its implementation can change and may execute newly fetched package code with the user's privileges. A compromised package release could therefore access local files or credentials; the instruction that the person runs it manually reduces automatic exposure, and this is a CLI supply-chain issue rather than an MCP configuration issue.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · references/agent-contract.md (reported line 5)May include surrounding context.

md
# Shared projects: Agent Contract

A folder is **linked** when `.testmuai/assurance.json` names a TestMu AI Test Manager project and the folder has no `.context/`. In a linked folder the commands below work in the **shared project**, so the team sees what each person saves. Link a folder with `kane-cli context ingest <files> --project <id> --mode agent` (or `kane-cli context extract --project <id> --mode agent`, or, where `kane-cli context pull --help` lists `--project`, `kane-cli context pull --project <id> --mode agent`); find the id with `kane-cli projects list --search <name> --agent`. In a folder with neither a link nor `.context/`, `--mode agent` without `--project` goes on locally without asking, as before: `context ingest` makes a local store, and `context extract` refuses `NO_STORE` until one exists.

| Area | Commands |
|---|---|

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · references/test-manager.md (reported line 151)May include surrounding context.

  1. kane-cli projects list --limit 10 --agent. Show the names with the current one marked. If the page says more exist, offer a search by name (--search <text>) instead of paging. Never promise a count: the CLI only says whether more exist.

  2. Let the person pick one, search, create a new one, or keep the current one. For a new project suggest the repo's name: kane-cli projects create "<name>" --agent.

  3. kane-cli folders list --project <id> --agent. Exactly one folder: take it without asking. Otherwise let them pick, or create one with kane-cli folders create "<name>" --project <id> --agent.

  4. Save the project first, then the folder, always as a pair, so the two never mismatch:

    bash

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · references/assurance.md (reported line 107)May include surrounding context.

md
**Archives need consent (0.7.1+).** A `--verdicts` rejection no longer destroys anything — it lands as a non-destructive `pending_archive` fact (exit 0, loud summary). Actually archiving requires `--allow-archive --because "<reason>"`, and `--mode ci` refuses archives under any flag. Never pass `--allow-archive` without the user's explicit instruction and their reason.

Auto-approving is allowed **only** when the user explicitly says so ("approve the recommended items without asking me") — and even then, enumerate everything you promoted in your summary and still surface warnings and conflicts.

## 5. Design — from a trusted use-case to runnable tests

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
80% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · references/debug.md (reported line 79)May include surrounding context.

md
> This looks like it might be a bug in kane-cli. Want me to file a report?

File at: **https://github.com/LambdaTest/kane-cli/issues**. Gather the details automatically — don't ask the user to dig through log files.

**Do NOT suggest bug reports for:** auth issues, low timeouts, vague objectives, or website errors (500s, CAPTCHAs).

The macOS troubleshooting instruction removes download quarantine without warning that it bypasses an operating-system security safeguard.

Medium
Category
Output Handling
Confidence
96% confidence
Finding

The troubleshooting command removes the downloaded tunnel binary's quarantine attribute without explaining that this bypasses a macOS security safeguard or requiring verification of the executable. The official HTTPS download location reduces risk, but a substituted or compromised binary could be run with the user's privileges and access tunnel credentials or private-network resources.

Content

No source excerpt is available for this finding.

MCP server referenced without pinned version: 'npx @testmuai/kane-cli-skill'.

Medium
Category
MCP Rug Pull
Confidence
91% confidence
Finding

Although this is a CLI invocation rather than an MCP server, unversioned npx can fetch and execute a changing npm release with the user's privileges. A compromised release could access credentials or persist through agent settings; consent to enabling the strip does not establish the integrity of the downloaded code.

Content

No source excerpt is available for this finding.

MCP server referenced without pinned version: 'npx @testmuai/kane-cli-skill'.

Medium
Category
MCP Rug Pull
Confidence
91% confidence
Finding

The status command still executes the npm package, even though its intended operation is only inspection. An unversioned npx invocation can therefore run compromised or unexpected release code before reporting status; this is a CLI supply-chain issue, not specifically an MCP issue.

Content

No source excerpt is available for this finding.

MCP server referenced without pinned version: 'npx @testmuai/kane-cli-skill'.

Medium
Category
MCP Rug Pull
Confidence
91% confidence
Finding

Disabling the strip through an unversioned npx command can execute a different package release from the one that installed it. A compromised release would have the user's privileges and could alter agent settings instead of restoring them, despite the benign purpose of the command.

Content

No source excerpt is available for this finding.

MCP server referenced without pinned version: 'npx @testmuai/kane-cli-skill'.

Medium
Category
MCP Rug Pull
Confidence
91% confidence
Finding

The terminal fallback repeats the unversioned package-execution risk; having the user run it manually does not verify its integrity. If the environment blocked execution for a security reason, moving the same command to another terminal also does not resolve that underlying concern.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · references/objectives-cookbook.md (reported line 288)May include surrounding context.

A pasted curl works too and is kept verbatim (method, headers, body, auth):

text
curl -X POST https://api.example.com/login -H 'Content-Type: application/json' -d '{"u":"a","p":"b"}', save the response as login

Once the call is made, use the response in plain English. {{name}} is reserved for your own variables, secrets, and the runtime namespaces (§6); a value the run produces is never written that way.

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · references/objectives-cookbook.md (reported line 288)May include surrounding context.

A pasted curl works too and is kept verbatim (method, headers, body, auth):

text
curl -X POST https://api.example.com/login -H 'Content-Type: application/json' -d '{"u":"a","p":"b"}', save the response as login

Once the call is made, use the response in plain English. {{name}} is reserved for your own variables, secrets, and the runtime namespaces (§6); a value the run produces is never written that way.