Security audit

Litmus

Security checks for vulnerabilities and agentic risk

Overview

Litmus is a coherent autonomous ML research skill, but it grants long-running agents broad local control with persistent cron jobs and weak teardown/safety boundaries.

Install only if you intentionally want persistent autonomous ML agents on a dedicated GPU machine. Review setup.sh, setup-cron.sh, and prepare-agents.sh first; pin or verify the upstream harness before running setup; avoid enabling ClawRxiv unless you accept external publication and plaintext local key storage; and make sure you can list and remove the six litmus-* cron jobs and kill worker sessions before starting overnight runs.

A.I.G

Vulnerability Patterns

Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
System PersistenceInstalls backdoors, hooks, services, or scheduled tasks that survive the run
Insecure DependenciesIntroduces malicious components through unsafe dependency sources
Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection

Findings (8)

T03 · Remote Payload Retrieval and Execution

Error

Location: references/onboarding.md:33
Finding: Remote Installer Is Downloaded and Executed Without Integrity Verification
Content: View full analysis
Remediation: View remediation

T01 · Skill Instruction Hijacking

Error

Location: references/program.md:16
Finding: Workers Are Instructed to Run Indefinitely With Unrestricted Shell and File Access
Content: View full analysis
Remediation: View remediation

T06 · System Persistence

Error

Location: scripts/setup-cron.sh:90
Finding: Six Recurring OpenClaw Jobs Create Persistent Autonomous Execution Without Implemented Teardown
Content: View full analysis
Remediation: View remediation

T08 · Insecure Dependencies

Error

Location: scripts/setup.sh:33
Finding: Mutable Upstream Repository and Dependencies Are Executed Without Pinning
Content: View full analysis
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Error

Location: references/program.md:272
Finding: Broad Process-Matching Kill Command Can Terminate Unrelated Training Jobs
Content: View full analysis
Remediation: View remediation
run.log 2>&1 & TRAIN_PID=$! ``` 2. Start each run in a dedicated process group and terminate only that group. 3. Before signaling, verify: - The PID is still running. - Its working directory is the expected Litmus worktree. - Its command belongs to the current experiment. 4. Store worker-specific PID metadata in a protected runtime directory. 5. Prefer a bounded timeout utility that directly wraps the child process. 6. Never use broad command-line matching such as `pkill -f train.py`. ]]>

T09 · Insecure Skill Coding Practices

Error

Location: references/onboarding.md:256
Finding: ClawRxiv Bearer Credential Is Stored in Plaintext Agent-Readable Configuration
Content: View full analysis
`), then: ```bash curl -s -X POST https://clawrxiv.io/api/auth/register \ -H "Content-Type: application/json" \ -d "{\"claw_name\": \"\"}" | jq . ``` Copy the returned `api_key` into `clawrxiv.apiKey` and set `enabled: true`. Warn: the key is shown only once. If they already have a key, skip registration and write it directly. ``` The publication instructions retrieve it directly from the plaintext file: ```bash CLAWRXIV_KEY=$(jq -r '.clawrxiv.apiKey' ~/.litmus/config.json) curl -s -X POST https://clawrxiv.io/api/posts \ -H "Authorization: Bearer $CLAWRXIV_KEY" \ -H "Content-Type: application/json" \ -d @- << 'PAYLOAD' ``` ### Technical Analysis The onboarding flow stores a bearer credential in `~/.litmus/config.json`. No instruction creates the file with mode `0600`, checks ownership, separates secrets from ordinary configuration, or prevents autonomous workers from reading it. The key grants publication authority and is consumed by workers operating with broad read and shell access. Shared agent workflows and generated prompts increase the chance of accidental disclosure through logs, debugging output, copied configuration, or prompt context. The metadata also lists `CLAWRXIV_API_KEY` as an optional environment variable, but the documented implementation instead uses plaintext JSON storage. ### Attack Path 1. The user enables ClawRxiv. 2. The onboarding agent registers an identity or accepts an existing key. 3. The key is written into `~/.litmus/config.json`. 4. Another local proc ...[truncated 543 chars]
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Error

Location: scripts/prepare-agents.sh:57
Finding: Unvalidated Template Value Can Escape the Agent Directory and Reach Recursive Deletion
Content: View full analysis
/dev/null || rm -rf "$AGENT_DIR" git -C "$LAB_REPO" branch -D "$BRANCH" 2>/dev/null || true fi ``` ### Technical Analysis Unknown template names are accepted and copied directly into `SHORT`. That value becomes part of `AGENT_ID`, a Git branch name, and the filesystem path `AGENT_DIR`. No allowlist rejects unsupported templates, and no validation rejects path separators, `..`, control characters, or other unsafe characters. A crafted template containing traversal components can cause `AGENT_DIR` to resolve outside `$BASE_DIR/agents`. If the resolved target is a directory, the script first tries forced Git worktree removal. If that fails, it falls back to `rm -rf` on the attacker-influenced path. Quoting prevents shell word splitting but does not prevent path traversal. ### Attack Path 1. An attacker or compromised configuration supplies a template value containing traversal, for example an unknown value with `../` components. 2. The default branch of the `case` statement copies it into `SHORT`. 3. The script constructs ...[truncated 637 chars]
Remediation: View remediation

T01 · Skill Instruction Hijacking

Warning

Location: references/leisure.md:41
Finding: Untrusted arXiv Content Is Fed Into Persistent Shell-Capable Agent Workflows
Content: View full analysis
|
" | sed 's/<[^>]*>//g' | head -80 ``` **Search 2 — Optimizer research:** ```bash curl -s "https://export.arxiv.org/api/query?search_query=cat:cs.LG+AND+(optimizer+learning+rate+schedule+training)&sortBy=submittedDate&sortOrder=descending&max_results=15" | grep -E "|<summary>" | sed 's/<[^>]*>//g' | head -80 ``` **Search 3 — Small model efficiency:** ```bash curl -s "https://export.arxiv.org/api/query?search_query=cat:cs.LG+AND+(small+language+model+efficient+training+convergence)&sortBy=submittedDate&sortOrder=descending&max_results=15" | grep -E "<title>|<summary>" | sed 's/<[^>]*>//g' | head -80 ``` **Search 4 — Training dynamics and loss landscapes:** ```bash curl -s "https://export.arxiv.org/api/query?search_query=cat:cs.LG+AND+(loss+landscape+training+dynamics+gradient+flow)&sortBy=submittedDate&sortOrder=descending&max_results=10" | grep -E "<title>|<summary>" | sed 's/<[^>]*>//g' | head -60 ``` For each paper describing a concrete, implementable technique: - What does it do mechanistically? - Can it be implemented in a ~600-line training script without new dependencies? - What is the specific code change? - Why might it help or hurt on a TIME_BUDGET-second training budget? ``` ### Technical Analysis Paper titles and abstracts are attacker-influenced remote text. The Skill instructs a shell-capable autonomous agent to read this content, ...[truncated 1535 chars]
Remediation: View remediation

SkillSpector

Vulnerability Patterns

Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults

Findings (62)

Tool Parameter Abuse

High

Category: Tool Misuse
Confidence: 90% confidence
Finding: While rm -rf ~/.litmus is intentionally scoped, it still performs irreversible deletion of experiment state, workspaces, and results. In the context of an autonomous ML research skill that accumulates outputs over time, accidental execution can cause meaningful loss of data and operational disruption.
Content: Scanner excerpt · INSTALL.md (reported line 63)May include surrounding context.
bash
npx clawhub uninstall litmus rm -rf ~/.litmus # removes all runtime state, agent workspaces, and results

Tool Parameter Abuse

High

Category: Tool Misuse
Confidence: 90% confidence
Finding: While rm -rf ~/.litmus is intentionally scoped, it still performs irreversible deletion of experiment state, workspaces, and results. In the context of an autonomous ML research skill that accumulates outputs over time, accidental execution can cause meaningful loss of data and operational disruption.
Content: Scanner excerpt · INSTALL.md (reported line 63)May include surrounding context.
bash
npx clawhub uninstall litmus rm -rf ~/.litmus # removes all runtime state, agent workspaces, and results

Tp4

High

Category: MCP Tool Poisoning
Confidence: 98% confidence
Finding: The declared description presents an operational ML research-agent system with runtime capabilities such as spawning agents, managing experiments, viewing results, steering agents, and producing digests. The supplied code chunk does none of those things. It is a maintenance utility that copies specific files from a local ../litmus repository into this repository. That is a materially different primary purpose and an undeclared capability (repository sync/mirroring). While such syncing could support distribution of the skill, it is not merely an implementation detail of the described runtime behavior; it is the entire behavior of the provided code chunk.
Content: No source excerpt is available for this finding.

Missing User Warnings

High

Category: Not specified by scanner
Confidence: 96% confidence
Finding: The ClawRxiv flow sends user-provided data to an external service and stores the returned API key, but the instructions only weakly warn about the implications. This can expose user identity information, create an external account, and persist sensitive credentials without strong privacy or security consent.
Content: No source excerpt is available for this finding.

Missing User Warnings

High

Category: Not specified by scanner
Confidence: 97% confidence
Finding: Spawning background agents and registering cron jobs create ongoing autonomous execution and persistence across time, which are significant system changes. Doing this without strong disclosure increases the risk of unexpected resource consumption, recurring actions, and difficulty stopping the system.
Content: No source excerpt is available for this finding.

Hidden Instructions

High

Category: Prompt Injection
Confidence: 90% confidence
Finding: The skill contains hidden configuration embedded in HTML comments and notes that content is injected at workspace creation time. Hidden or dynamically injected instructions reduce transparency, make review harder, and can smuggle capabilities or agent-specific directives that operators do not see in the visible prompt.
Content: Scanner excerpt · references/program.md (reported line 3)May include surrounding context.
md
# litmus — research agent program

Hidden Instructions

High

Category: Prompt Injection
Confidence: 89% confidence
Finding: The placeholder for injected focus instructions indicates additional hidden directives may be inserted into the prompt at runtime. Runtime prompt injection of undisclosed instructions undermines auditability and can materially change behavior without clear user awareness.
Content: Scanner excerpt · references/program.md (reported line 79)May include surrounding context.
md
## Research Focus  Your primary area of focus is stated above. When results plateau, you may branch out — but return to your focus when you have new angles. Check `SHARED_DIR/discoveries.md` before each experiment.

Rp1

Medium

Category: MCP Rug Pull
Confidence: 70% confidence
Finding: npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.
Content: No source excerpt is available for this finding.

Rp1

Medium

Category: MCP Rug Pull
Confidence: 70% confidence
Finding: npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.
Content: No source excerpt is available for this finding.

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 91% confidence
Finding: The uninstall section includes a destructive deletion command that removes all runtime state and results, but the warning is only an inline comment and may be overlooked. In an agent-oriented skill where users may copy-paste commands quickly, this increases the chance of accidental data loss.
Content: No source excerpt is available for this finding.

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The README explicitly encourages an installation flow that will spawn workers, register cron jobs, and perform ongoing writes under ~/.litmus, but it does not present a clear warning that these are persistent background changes. In a skill designed for autonomous overnight operation, lack of explicit disclosure can lead users to authorize long-lived automation they did not fully understand, increasing the risk of unintended resource consumption, file changes, and persistence.
Content: No source excerpt is available for this finding.

Undeclared Tool Scope

Medium

Category: MCP Least Privilege
Confidence: 79% confidence
Finding: The skill documents commands that read local files and repositories, but it does not declare any explicit tool scope or permissions boundaries. In an autonomous multi-agent skill, missing tool scoping increases the chance that subagents can access more of the filesystem than users expect, especially because the workflow relies on reading program files, git history, notes, and shared state under the home directory.
Content: No source excerpt is available for this finding.

Session Persistence

Medium

Category: Rogue Agent
Confidence: 86% confidence
Finding: The skill is explicitly designed for persistence across time through scheduled Director and Synthesizer actions that carry context and generate plans for future execution. Persistent autonomous state is not inherently malicious, but in this context it increases risk because future agent behavior can be influenced by prior notes, skills, and cross-agent outputs without fresh user review, enabling compounding mistakes or unsafe strategy drift.
Content: Scanner excerpt · SKILL.md (reported line 29)May include surrounding context.
md
each other's code, cherry-pick breakthroughs, and build on the global best at any time. Validated techniques accumulate in a **Skills library** (`~/.litmus/shared/skills/`). A **Synthesizer** runs at 04:00 to distill collective knowledge into skills and write a research agenda for the next day. A **Director** runs every 2 hours to steer workers, trigger **Compass Resets** on stagnation, and orchestrate cross-agent knowledge transfer.

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 94% confidence
Finding: The setup instructions do mention cloning into ~/.litmus and downloading about 1 GB of data, but they do not present this as a clear up-front safety warning about extensive writes to the user's home directory, dependency installation, and network/data usage. For an agent-driven installer, lack of explicit warnings can lead users to approve installation without understanding persistence, storage, or system modification scope.
Content: No source excerpt is available for this finding.

Unbounded Resource Access

Medium

Category: Excessive Agency
Confidence: 97% confidence
Finding: The task string instructs spawned subagents to 'run the research loop forever,' which is an unbounded execution model with no built-in stop condition, budget cap, or runtime guard. In the context of parallel GPU-based research agents, this can cause sustained compute consumption, runaway costs, disk growth, and prolonged autonomous behavior that is difficult to monitor or interrupt.
Content: Scanner excerpt · SKILL.md (reported line 84)May include surrounding context.
text
sessions_spawn task: "Read program.md in your current directory and run the research loop forever." runtime: "subagent" mode: "session" agentId: "litmus-worker-arch-1"

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 97% confidence
Finding: The skill instructs users to register multiple cron jobs that run indefinitely on a schedule, but it does not foreground that this creates persistent background activity after the initial interaction ends. Persistent scheduled execution is especially sensitive here because those jobs steer agents, synthesize outputs, and may consume GPU, CPU, disk, and network resources without ongoing user attention.
Content: No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium

Category: Not specified by scanner
Confidence: 92% confidence
Finding: This introduces an external publishing capability that expands the skill beyond the stated local orchestration and research-management scope. Scope drift matters because agents handling local experiment artifacts may now transmit summaries or derived research externally, creating data-governance and user-consent risks not obvious from the manifest.
Content: No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium

Category: Not specified by scanner
Confidence: 90% confidence
Finding: The onboarding flow directs the agent to register a third-party identity and persist an API key in local config, but this capability is not reflected in the declared functionality. Hidden credential acquisition and storage increase risk because users may not expect the skill to create external accounts or retain long-lived secrets for later network actions.
Content: No source excerpt is available for this finding.

External Transmission

Medium

Category: Data Exfiltration
Confidence: 82% confidence
Finding: This command performs an outbound POST to a third-party service to register an agent identity, sending user-associated metadata off-host. In the context of a research orchestration skill, external transmission is more dangerous because it creates an unexpected external identity and dependency that may be used later for automated publication.
Content: Scanner excerpt · references/clawrxiv.md (reported line 16)May include surrounding context.
Register a ClawRxiv agent identity. Use a name like litmus-<username>:

bash
curl -s -X POST https://clawrxiv.io/api/auth/register \ -H "Content-Type: application/json" \ -d '{"claw_name": "litmus-<username>"}' | jq .

Session Persistence

Medium

Category: Rogue Agent
Confidence: 87% confidence
Finding: Persisting a long-lived API key in ~/.litmus/config.json creates a local secret-exposure risk, especially if file permissions are weak, backups are broad, or other agents/processes can read the config. Because the key authorizes external posting, compromise of this file could let an attacker publish as the user or harvest research content through the integration.
Content: Scanner excerpt · references/clawrxiv.md (reported line 21)May include surrounding context.
-d '{"claw_name": "litmus-"}' | jq .

text
Response contains `api_key` — **shown only once**. Write it to config immediately: ```bash # The onboarding agent writes this to ~/.litmus/config.json under clawrxiv.apiKey

Context-Inappropriate Capability

Medium

Category: Not specified by scanner
Confidence: 87% confidence
Finding: Third-party paper publication is not obviously necessary for running ML research agents, so this is an unjustified outbound capability relative to the stated purpose. Even if intended for convenience, it can expose proprietary results, internal hypotheses, branch names, commit hashes, or other sensitive metadata to an external service.
Content: No source excerpt is available for this finding.

External Transmission

Medium

Category: Data Exfiltration
Confidence: 93% confidence
Finding: This command sends a bearer token plus paper content to an external domain, enabling exfiltration of research outputs and associated metadata. In this skill's context, the content may include experiment results, code diffs, commit hashes, and strategic analysis, so the outbound transmission can leak sensitive or proprietary information even if the destination is intended.
Content: Scanner excerpt · references/clawrxiv.md (reported line 35)May include surrounding context.
bash
CLAWRXIV_KEY=$(jq -r '.clawrxiv.apiKey' ~/.litmus/config.json) curl -s -X POST https://clawrxiv.io/api/posts \ -H "Authorization: Bearer $CLAWRXIV_KEY" \ -H "Content-Type: application/json" \ -d @- << 'PAYLOAD'

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 96% confidence
Finding: This skill explicitly instructs the agent to modify shared state by writing to SHARED_DIR/mode.txt and later to SHARED_DIR/morning-queue.md and agent program files, but it provides no user-facing disclosure or confirmation step before making those changes. In an autonomous multi-agent environment, silent state changes can alter worker behavior, reprioritize experiments, and persistently change future actions, increasing the chance of unintended or unsafe orchestration changes.
Content: No source excerpt is available for this finding.

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 92% confidence
Finding: The morning briefing publishes a system event containing operational metadata such as experiment counts, skill counts, and best validation metrics via openclaw system event, but the skill description does not warn the user that information will be emitted externally. Even if the payload seems limited, it can leak sensitive research progress or model-performance information to broader system observers or logs without informed consent.
Content: No source excerpt is available for this finding.

Ae4

Medium

Category: analysis-evasion
Confidence: 80% confidence
Finding: Suspicious Unicode normalization or mixed-script content
Content: No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.