Back to skill

Security audit

Huo15 Autoresearch Loop

Security checks for vulnerabilities and agentic risk

Overview

The skill is an autonomous code-iteration tool, but it can run arbitrary shell commands and broadly commit, reset, or delete repository changes without tight scoping or confirmation.

Review carefully before installing. Use only in a disposable or clean Git worktree, inspect any CLAUDE_TASK and verification command before running, and avoid repositories with uncommitted or sensitive files because the skill can commit unrelated changes or delete untracked work.

Vulnerability Patterns
  • Tool Hijacking and SpoofingModifies or replaces tools so legitimate-looking calls execute attacker logic
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (5)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/loop.sh:66
Finding

Arbitrary Command Execution Through Shell Evaluation

Content
View full analysis
/dev/null || echo "") ``` ### Technical Analysis The script passes the contents of the `CLAUDE_TASK` environment variable directly to Bash's `eval` builtin. `eval` parses its input as shell source code, so shell operators, command substitutions, redirections, pipelines, and additional commands embedded in the variable are executed with the privileges of the process running the skill. No allowlist, executable validation, argument separation, or sandbox boundary is applied. Although `CLAUDE_TASK` is intended to represent an agent modification task, the script does not establish that its value originates from a trusted component. ### Attack Path 1. An attacker, compromised child agent, or unsafe integration influences `CLAUDE_TASK`. 2. The variable is populated with shell syntax, such as a legitimate-looking task followed by an additional command. 3. The user starts the autonomous loop. 4. `eval "$CLAUDE_TASK"` reparses the entire value as shell code. 5. The injected commands execute under the account running the skill. ### Impact Assessment Successful exploitation provides arbitrary command execution with the current user's privileges. The attacker could read or modify accessible files, inspect environment variables, alter source repositories, access credentials available to the process, initiate network connections, or execute additional local programs. The scope is not restricted to the target repository. ]]>
Remediation
View remediation

T07 · Tool Hijacking and Spoofing

Error
Location
scripts/loop.sh:82
Finding

PATH-Based Verification Tool Hijacking

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
scripts/loop.sh:87
Finding

Repository-Wide Commit and Destructive Rollback Ignore the Configured Scope

Content
View full analysis
/dev/null || true fi state update "$iteration" "success" "验证通过" "$changes" else log_error "❌ 验证失败" # git revert if [[ "$revert_on_fail" == "true" ]] && [[ -n "$pre_commit" ]]; then log_info "↩ 回滚更改..." git reset --hard HEAD > /dev/null 2>&1 || true git clean -fd > /dev/null 2>&1 || true fi ``` ### Technical Analysis The skill accepts and records `scope_globs`, but never enforces that scope when staging or reverting files. On success, `git add -A` stages all repository changes, including unrelated modifications and newly created files. On failure, `git reset --hard HEAD` discards all tracked working-tree and index changes, while `git clean -fd` recursively deletes untracked files and directories. The script records `initial_commit` and `pre_commit`, but neither value is used to restore an iteration-specific snapshot. Merely checking that `pre_commit` is nonempty does not protect changes that existed before the iteration began. ### Attack Path 1. A user starts the skill in a repository containing unrelated uncommitted work or untracked files. 2. The modification task makes a change and verification fails. 3. The failure branch executes `git reset --hard HEAD`. 4. All tracked uncommitted changes are discarded, including changes that predated the iteration. 5. `git clean -fd` then removes all untracked files and directories. An alternative success path is: 1. The repository contains unrelated or sensitive uncommitted files. 2. Verification succeeds. 3. `git add -A` stages every repository change. 4. The skill commits unrelated content under the automated iteration commit. ...[truncated 343 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/state.sh:20
Finding

Unescaped User Input Corrupts the JSON State File

Content
View full analysis
"$STATE_FILE" <
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/state.sh:174
Finding

Sourced State Module Executes Its Command-Line Router and Terminates the Main Script

Content
View full analysis
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Findings (19)

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding

The skill claims to implement a full Modify→Verify→Keep/Discard loop, but analysis indicates it actually runs arbitrary shell commands via bash and may rely on config-provided defaults rather than a transparent, bounded workflow. When a skill's documented purpose obscures direct shell execution, users and reviewers cannot accurately assess risk, increasing the chance of unsafe invocation.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding

The skill claims to implement a full Modify→Verify→Keep/Discard loop, but analysis indicates it actually runs arbitrary shell commands via bash and may rely on config-provided defaults rather than a transparent, bounded workflow. When a skill's documented purpose obscures direct shell execution, users and reviewers cannot accurately assess risk, increasing the chance of unsafe invocation.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill documentation does not adequately warn users that it will automatically modify files, execute verification commands, and commit or revert repository changes. This omission is especially risky because the workflow includes destructive rollback behavior and repeated iterations, so users may consent without understanding the operational and data-loss implications.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

Arbitrary shell execution sourced from CLAUDE_TASK occurs with no user warning, approval gate, or command preview. Because the command is agent-supplied and executed verbatim, the lack of transparency and confirmation greatly increases the chance of silent harmful actions such as credential theft, remote downloads, or destructive system commands.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

The script executes the contents of CLAUDE_TASK with eval, giving arbitrary shell execution to whatever populates that environment variable. In an autonomous loop, this effectively grants full command execution in the current user context, enabling data exfiltration, filesystem damage, persistence, or lateral actions far beyond a narrowly scoped code-modification task.

Content

No source excerpt is available for this finding.

Tool Parameter Abuse

High
Category
Tool Misuse
Confidence
95% confidence
Finding

Using git reset --hard together with git clean -fd gives the skill a destructive primitive that can discard tracked modifications and delete untracked files. In an autonomous workflow, this is dangerous because tool parameters are broad and repository-wide, making accidental or malicious data loss easy if triggered under the wrong conditions.

Content

Scanner excerpt · scripts/loop.sh (reported line 99)May include surrounding context.

sh
# git revert
            if [[ "$revert_on_fail" == "true" ]] && [[ -n "$pre_commit" ]]; then
                log_info "↩ 回滚更改..."
                git reset --hard HEAD > /dev/null 2>&1 || true
                git clean -fd > /dev/null 2>&1 || true
            fi

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
90% confidence
Finding

The skill describes file modification, local state persistence, and repository operations but declares no explicit tool scope or permissions. In an agent environment, missing scope boundaries can allow broader-than-expected file read/write access and makes it harder for reviewers or policy engines to constrain execution safely.

Content

No source excerpt is available for this finding.

Unbounded Resource Access

Medium
Category
Excessive Agency
Confidence
84% confidence
Finding

Advertising a workflow as 'Repeat forever' signals potentially unbounded execution. Even if practical stop conditions exist elsewhere, presenting the loop this way encourages long-running or uncontrolled agent activity that can consume resources, repeatedly mutate a repository, or keep executing verification commands beyond user intent.

Content

Scanner excerpt · SKILL.md (reported line 8)May include surrounding context.

md
- 火一五自动迭代
  - 火一五自动循环
  - 自动迭代循环
description: 基于 Karpathy 自主研究循环的 OpenClaw Skill,Modify → Verify → Keep/Discard → Repeat forever
---

# huo15-autoresearch-loop

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

Broad trigger phrases such as everyday expressions can cause accidental activation of a skill that modifies files, executes commands, and performs git operations. In this context, mis-triggering is more dangerous than usual because the skill is not read-only and its looped behavior can amplify the effects of an unintended invocation.

Content

No source excerpt is available for this finding.

Unbounded Resource Access

Medium
Category
Excessive Agency
Confidence
85% confidence
Finding

The functionality section explicitly states a Repeat-forever autonomous loop, which creates risk of unbounded CPU, disk, process, and repository activity. In a skill that also executes commands and performs commits/reverts, unbounded looping can magnify failures, fill logs/history, and repeatedly apply unsafe changes.

Content

Scanner excerpt · SKILL.md (reported line 24)May include surrounding context.

md
## 功能

实现 Karpathy 的 **Modify → Verify → Keep/Discard → Repeat forever** 自主研究循环。

## 核心流程

Unbounded Resource Access

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill allows unbounded resource consumption (API calls, storage, compute). Without rate limits or quotas, a compromised or misbehaving agent can cause denial-of-service or cost overruns.

Content

Scanner excerpt · config.json (reported line 4)May include surrounding context.

json
{
  "name": "huois-autoresearch-loop",
  "version": "1.0.0",
  "description": "Karpathy 自主研究循环 - Modify → Verify → Keep/Discard → Repeat forever",
  "max_iterations": 50,
  "verify_command": "",
  "scope_globs": ["**/*.py", "**/*.js", "**/*.ts"],

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The description on L04 is written entirely in Chinese with no indication that users may choose another language or locale. This can violate language/locale policy when a skill implicitly requires a specific language without documenting opt-in or justification.

Content

No source excerpt is available for this finding.

Unbounded Resource Access

Medium
Category
Excessive Agency
Confidence
78% confidence
Finding

The loop is designed to repeat indefinitely and only stops via external state checks or configured limits, which can lead to excessive API/tool consumption, runaway edits, or prolonged execution if stop conditions fail or are bypassed. In an autonomous agent context, unbounded iteration increases both cost exposure and the window for harmful actions.

Content

Scanner excerpt · scripts/loop.sh (reported line 3)May include surrounding context.

sh
#!/bin/bash
# loop.sh - 自主研究循环主脚本
# Modify → Verify → Keep/Discard → Repeat forever

set -e

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The loop performs automatic git commits and destructive rollback/cleanup operations, including hard reset and untracked-file deletion, without those capabilities being clearly surfaced in the high-level skill description. This mismatch creates a dangerous trust gap: users may invoke a 'research loop' expecting iterative edits, but instead lose local work or have repository history modified automatically.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The script automatically commits changes and performs rollback/cleanup operations that alter repository state, but it does not present a clear warning or obtain confirmation before these actions occur. This is especially risky in developer worktrees where uncommitted or untracked files may be valuable and can be silently deleted by git clean -fd.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

This shell script contains natural-language comments and user-facing output entirely in Chinese, including status and usage-related text, with no indication that the skill is region-specific or that users can opt into another language. That can violate language/locale policy because it imposes a specific language on users by default.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The script title and multiple user-facing status/error messages are written in Chinese, which imposes a specific language on users without opt-in or documented justification. The policy requires avoiding fixed language/locale behavior unless the user is given a choice or the regional constraint is clearly justified.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The quick verification path executes an arbitrary shell command via bash -c "$verify_cmd" with no validation, sandboxing, or user-facing warning. Because the command may come from user or configuration input and this skill is designed for an autonomous research loop, this creates a real command-injection/unsafe-command-execution risk that can run destructive or exfiltrating commands silently due to output redirection to /dev/null.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
91% confidence
Finding

The script's natural-language comments, status messages, and usage examples are written in Chinese throughout, with no indication that users may choose another language. This creates a locale/language constraint in user-facing behavior without documented opt-in or justification.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.