ClawHub
Back to skill

Security audit

experiment-lifecycle-governance

Security checks for vulnerabilities and agentic risk

Overview

This skill is mostly coherent MLOps guidance, but it tells automated agents to bypass its own PIN safeguard for destructive experiment actions.

Install only if you are comfortable giving agents or automation access to manage experiment runs and local expflow state. Before use, require explicit human approval or a scoped automation credential for cancel, stop, delete, or PIN-clear actions, and avoid blanket use of --force on PIN-protected commands.

A.I.G

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
SKILL.md:251
Finding

Automation guidance bypasses PIN authorization for destructive operations

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 88–96 and 251–256
Vulnerability Type: Destructive-operation authorization bypass
Risk Level: Medium

Vulnerable instructions:

bash
expflow pin clear [--force]    # Remove PIN
expflow run cancel <id>            # Interactive PIN prompt
expflow run cancel <id> --force    # Skip PIN
markdown
### 3. `--force` Flag for Script Calls
Always provide `--force` / `-f` on PIN-guarded commands for CI/automation.

### 4. Interactive `getpass` vs Non-Interactive
`getpass.getpass()` works in terminals but fails in piped commands, CI, or subagent calls. Always provide `--pin` or `--force` as alternative paths.

Technical Analysis

The Skill presents its four-digit PIN as protection for destructive operations such as canceling experiments and clearing PIN protection. However, it documents a --force option that skips the PIN and then explicitly instructs automated callers to always use that bypass.

The relevant security boundary is the human/PIN authorization gate. In an interactive workflow, knowledge of the PIN is expected to authorize a destructive action. Under the documented automation workflow, the agent can cross that boundary merely by adding a public command-line flag. No equivalent authorization credential, mandatory confirmation parameter, dry-run requirement, or narrowly scoped automation policy is documented.

This is high-risk behavior rather than evidence of malicious intent: the bypass appears intended to support non-interactive execution, but it makes the advertised PIN control ineffective precisely in agent, CI, and subagent environments.

Attack Path

  1. An agent or automated workflow receives a request that results in a PIN-protected destructive command, such as canceling an experiment or clearing PIN protection.
  2. The workflow follows the Skill instruction to “always provide” --force or -f.
  3. The C ...[truncated 1135 chars]
Remediation
View remediation

Remediation Suggestions

  • Remove the instruction to automatically append --force to PIN-protected commands.
  • Require explicit, operation-specific user confirmation before an agent invokes any destructive command.
  • Replace the unauthenticated bypass with a separately provisioned automation credential that is narrowly scoped to approved actions, resources, and environments.
  • Make destructive automation fail closed when neither a valid PIN nor an approved automation credential is supplied.
  • Do not permit PIN removal through --force alone; require current-PIN verification or a secure administrative recovery process.
  • Provide a non-destructive dry-run mode so automation can display the exact target and planned action before authorization.
  • Record the authenticated principal, target, action, and authorization method in an audit log.

SkillSpector

Vulnerability Patterns
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Findings (1)

Credential Access

High
Category
Privilege Escalation
Confidence
60% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · SKILL.md (reported line 247)May include surrounding context.

md
### 1. YAML/Env vs File Storage for PIN
PIN hash must NOT go into `config.yaml` (risk of git commit). Use `~/.expflow/pin.hash`.
Precedence: `pin.hash` file > `.env EXPFLOW_PIN_HASH` > `config.yaml pin.hash`.

### 2. `get_last_scalar_metrics()` clearml API
Returns nested dict: `{"Score": {"seg_total": {"last": 57.09, ...}}, ...}`. Flatten to `{"seg_total": 57.09}` for compare_scores.