Back to skill

Security audit

huawei-cloud-skill-tester

Security checks for vulnerabilities and agentic risk

Overview

ClawScan rates this skill suspicious with high confidence because scanners report unsafe command execution and installation cleanup, weak authorization and cleanup safeguards, and misleading claims about what testing actually verifies.

Consider avoiding installation, especially in an environment with cloud credentials or valuable existing skill installations. The scan reports credible risks to command execution, cloud and local state, and the reliability of test results; coverage was partial, so some behavior remains unevaluated. The suspicious verdict is a review warning and does not establish malicious intent.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/tier1/phase-4-execute-tests.sh:136
Finding

Untrusted target commands bypass the default read-only execution gate ## Remediation Suggestions 1. Replace shell command strings with structured executable and argument fields, and execute validated argument arrays without `bash -c`. 2. Enforce an explicit allowlist of read-only cloud operations. Treat unknown operations as requiring authorization rather than assuming they are safe. 3. Reject shell operators, substitutions, and redirections in automatically executed CLI test definitions. 4. Treat target-provided scripts and SDK snippets as untrusted executable code. Require separate authorization and run them in an isolated environment. 5. Provide subprocesses only the credentials and environment variables required for the approved operation. Prefer a cloud identity whose permissions independently enforce read-only access. 6. Apply the same execution policy to Phase 4 and single-Skill Phase 6, and add regression tests covering compound commands and command substitution.

Content
View full analysis

Vulnerability Details

File Location:

  • scripts/tier1/phase-1-skill-analysis.sh:317–327
  • scripts/tier1/phase-3-gen-testcases.sh:172–190
  • scripts/tier1/phase-4-execute-tests.sh:136–172
  • Additional affected path: scripts/tier2/phase-6-full-flow.sh:175–209

Vulnerability Type: T09: Insecure Skill Coding Practices
Risk Level: High
Validation: Confirmed by static analysis; no exploitation was executed.

Technical Analysis

Phase 1 extracts command lines from the target Skill's documentation. For lines beginning with hcloud, it determines whether the entire command is a write operation using only four substring checks. It does not parse or reject shell composition.

python
elif cl.startswith('hcloud '):
    cmd_id += 1
    is_write = any(kw in cl.lower() for kw in ['create', 'delete', 'update', 'destroy'])
    clean_cmd = re.sub(r'<[^>]+>', '', cl).strip()
    commands.append({
        'id': 'CMD-%02d' % cmd_id,
        'source': 'SKILL.md-bash-block',
        'description': clean_cmd[:80],
        'command': clean_cmd,
        'executor': 'cli',
        'is_write': is_write
    })

Phase 3 inherits the classification rather than independently validating execution safety:

python
is_write = cmd.get('is_write', False)

The generated test case retains the command and classification:

python
'command': cmd_text,
python
'is_write': is_write,

Phase 4 evaluates ALLOW_WRITES only for cases classified as writes:

python
if is_write:
python
allow_writes = os.environ.get('ALLOW_WRITES', '0') == '1'
if not allow_writes:
    entry['status'] = 'skip'
    entry['output_snippet'] = '写操作已跳过 (ALLOW_WRITES=0)'
    skip_count += 1
    print(f"    ⏭️ SKIP-写操作 (ALLOW_WRITES=0)")
    exec_results.append(entry)
    continue

The CLI execution branch then interprets the retained command as shell code:

python
r = subprocess.run(
    ['bash', '-c', cmd_text],
    capture_output=True, text=True, timeout=int(o
...[truncated 2157 chars]

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/lib/utils.sh:498
Finding

Installation verification and exit cleanup delete pre-existing Skill installations ## Remediation Suggestions 1. Run installation, uninstallation, and reinstallation tests in an isolated temporary runtime directory rather than the user's active Skill installation directory. 2. Refuse destination collisions by default. Do not interpret an existing directory as a test-owned installation. 3. Maintain a per-run ownership record of paths successfully created by the test. Permit automatic cleanup only for those exact paths. 4. Ensure exit and signal handlers cannot delete installations that existed before the run, including runs started from a later phase. 5. If testing an existing installation is explicitly requested, require confirmation and create a recoverable backup before modification. 6. Add regression tests for distinct source/destination paths with matching basenames, skipped or failed installation, resumed execution, and interrupted runs.

Content
View full analysis

Vulnerability Details

File Location:

  • scripts/tier1/phase-0-install-check.sh:92–130,154–165
  • scripts/lib/utils.sh:498–510
  • scripts/run-test-pipeline.sh:165–172

Vulnerability Type: T09: Insecure Skill Coding Practices
Risk Level: Medium
Validation: Confirmed by static analysis; no destructive operation was executed.

Technical Analysis

When an installation destination already exists, Phase 0 skips the initial installation:

bash
elif [ -d "$target_dir" ]; then
bash
install_status="skipped"

However, its subsequent uninstall branch deletes that directory when the source and destination are different physical paths:

bash
elif [ -d "$target_dir" ] && [ -n "$target_dir" ] && [ "$target_dir" != "/" ] && [ "$target_dir" != "$HOME" ]; then
  local uninst_start; uninst_start=$(date +%s%N)
  rm -rf "$target_dir" && uninstall_status="pass" || uninstall_status="fail"
  local uninst_end; uninst_end=$(date +%s%N)
  uninstall_duration=$(( (uninst_end - uninst_start) / 1000000 ))

The reinstall branch subsequently copies the selected workspace source into the destination:

bash
cp -r "$skill_dir" "$target_dir" && reinstall_status="pass" || reinstall_status="fail"

The shared exit cleanup independently constructs a destination from the source directory's basename. Its safeguards check physical path equality and directory existence, but do not verify whether the current test run created the installation:

bash
local sn; sn=$(basename "$sp")
local install_target="$SKILL_INSTALL_DIR/$sn"
bash
rp_sp=$(realpath "$sp" 2>/dev/null || echo "$sp")
rp_it=$(realpath "$install_target" 2>/dev/null || echo "$install_target")
if [ "$rp_sp" = "$rp_it" ]; then
bash
elif [ -d "$install_target" ]; then
bash
rm -rf "$install_target"

The main pipeline registers cleanup regardless of whether Phase 0 created a test installation:

bash
cleanup_on_exit() {
  local rc=$?
  echo ""
  cleanup_after_test "${S
...[truncated 1764 chars]
Vulnerability Patterns
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (37)

Description-Behavior Mismatch

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The source makes conflicting claims about real execution and repeatedly promises per-operation confirmation, while also exposing a global write-enable flag. Misrepresenting unexecuted checks, cleanup, or authorization as successful undermines the integrity of a live-cloud safety gate; the detailed implementation allegations cannot all be independently verified from this document.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
MCP Tool Poisoning
Confidence
87% confidence
Finding

The supplied chunk is a quality-CLI bootstrapper, not a testing implementation. Dependency setup can be a legitimate supporting detail, but the declaration does not identify this quality-reporting dependency or disclose its automatic remote-code execution and persistent executable installation. Skill installation validation is distinct from installing an additional reporting tool. The hash check and OBS URL restrictions provide safeguards but do not establish alignment with the declared purpose. This chunk does not itself upload test results or demonstrate data exfiltration.

Content

No source excerpt is available for this finding.

Incomplete referenced artifact analysis

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 107)May include surrounding context.

md
> Full per-phase implementation specs (steps, pass criteria, JSON fields) are in `references/phase-details.md`.

Incomplete referenced artifact analysis

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 124)May include surrounding context.

md
> Detailed steps, chain-verification rules, and JSON schemas: `references/phase-details.md` and `references/output-schema-spec.md`.

Incomplete referenced artifact analysis

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 241)May include surrounding context.

md
- `references/phase-details.md` — Full per-phase implementation specs (steps, pass criteria, JSON fields)

Incomplete referenced artifact analysis

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 124)May include surrounding context.

md
> Detailed steps, chain-verification rules, and JSON schemas: `references/phase-details.md` and `references/output-schema-spec.md`.

Incomplete referenced artifact analysis

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 237)May include surrounding context.

md
- `references/output-schema-spec.md` — Complete JSON field specification for each phase (including Phase 7 final report schema)

Incomplete referenced artifact analysis

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 284)May include surrounding context.

md
See `references/output-schema-spec.md` for the JSON schema. Phase 5 and 6 additionally output scenario execution logs with real CLI/SDK responses for auditabili

Incomplete referenced artifact analysis

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 190)May include surrounding context.

md
bash scripts/tier1/phase-1-skill-analysis.sh --skill "huawei-cloud-bss-voucher-manage"

Incomplete referenced artifact analysis

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 191)May include surrounding context.

md
bash scripts/tier2/phase-5-orchestration.sh --skills "skill-a, skill-b"

Incomplete referenced artifact analysis

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 199)May include surrounding context.

md
bash scripts/tier2/phase-5-orchestration.sh --skills "ecs-manage, vpc-manage, eip-manage"

Incomplete referenced artifact analysis

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 202)May include surrounding context.

md
bash scripts/tier2/phase-6-full-flow.sh --skill "huawei-cloud-rds-intelligent-service"

Incomplete referenced artifact analysis

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 238)May include surrounding context.

md
- `references/phase-transition-rules.md` — Phase transition / fallback / skip rules

External Script Fetching

High
Category
Supply Chain
Confidence
90% confidence
Finding

Remote code is downloaded and executed. This bypasses code review and could introduce malicious code.

Content

Scanner excerpt · scripts/tier1/phase-1-skill-analysis.sh (reported line 357)May include surrounding context.

sh
if ' ' not in clean_cmd and re.match(r'^[\w./{}\[\]$~-]+$', clean_cmd):
                continue
            if not (clean_cmd.startswith('hcloud ') or clean_cmd.startswith('python3 ')
                   or clean_cmd.startswith('bash ') or clean_cmd.startswith('curl ')
                   or clean_cmd.startswith('from ') or clean_cmd.startswith('sh ')):
                continue
            exe = 'cli'

The documented ALLOW_WRITES boundary is not enforced for cases with missing or incorrect write metadata.

High
Category
Excessive Agency
Confidence
99% confidence
Finding

The write restriction trusts each test case's is_write field, which defaults to false, rather than independently restricting executable operations. A malicious or misclassified case can therefore execute destructive shell commands or SDK operations with inherited cloud credentials even when ALLOW_WRITES=0.

Content

No source excerpt is available for this finding.

Resource lifecycle records contain fabricated identifiers and reuse the tested operation as the cleanup command.

High
Category
Excessive Agency
Confidence
99% confidence
Finding

Successful write cases record fabricated resource IDs rather than identifiers returned by the cloud service, and the cleanup command repeats a truncated version of the tested operation rather than its inverse. This prevents reliable resource lifecycle management and could leave billable or exposed resources behind; a downstream cleanup executor could also repeat a destructive operation.

Content

No source excerpt is available for this finding.

The documented user-confirmation gate is absent, but reports claim confirmation occurred.

High
Category
Excessive Agency
Confidence
98% confidence
Finding

The script executes an automatically derived single-skill scenario without obtaining scenario-specific confirmation, yet records user_confirmed=true in both scenario and execution metadata. ALLOW_WRITES provides a general mutation opt-in, but it does not establish approval of the particular generated actions, making the claimed confirmation unreliable and permitting unintended changes when that variable is already enabled.

Content

No source excerpt is available for this finding.

Single-skill execution declares cleanup successful without implementing resource cleanup verification.

High
Category
Excessive Agency
Confidence
99% confidence
Finding

Single-skill execution can perform real cloud mutations, but resource_changes remains empty and cleanup is unconditionally reported as successful with no verification. Ordinary delete steps are not a reliable substitute for tracked cleanup, so partial failures or mismatched commands can leave billable or publicly accessible resources while concealing the need for intervention.

Content

No source excerpt is available for this finding.

Multi-skill planning is reported as successful end-to-end execution without running tests.

High
Category
Excessive Agency
Confidence
100% confidence
Finding

The multi-skill branch assigns pass to derived steps without executing them, then unconditionally reports successful state consistency and cleanup. Even ALLOW_REAL_E2E=1 only adds a credential check, so consumers can mistake an unexecuted plan for verified cloud behavior and approve unsafe workflows.

Content

No source excerpt is available for this finding.

Skill declares no tool scope ('permissions' or 'allowed-tools') but code capabilities were detected: env, file_read, file_write, network, shell.

Medium
Category
MCP Least Privilege
Confidence
70% confidence
Finding

Without declared permissions the skill's intent is opaque and cannot be validated.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
80% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · SKILL.md (reported line 80)May include surrounding context.

md
1. **hcloud CLI** installed and authenticated (for Tier 2 CLI mode testing) — Reference: https://support.huaweicloud.com/qs-hcli/hcli_02_003.html
2. **Python 3.8+** + `huaweicloudsdk` packages (for SDK mode testing) — SDK Reference: https://console.huaweicloud.com/apiexplorer/
3. **Huawei Cloud AK/SK** — auto-scans all environment variables prefixed `HUAWEI` / `HW` / `HWC` (key-value pairs whose keys carry AK/SK markers). **If missing, the framework emits the env-var setup template to stderr and exits 77. NEVER ask the user to type AK/SK in chat; the user must set env vars in their shell profile out-of-band and re-run.** (full protocol: `references/agent-protocol.md`).
4. **Target Skill** must be under `$SKILL_INSTALL_DIR/` (auto-detected: `~/.agents/skills/` → `~/.hermes/skills/` → default) or a user-specified path
5. **jq** command (all JSON processing depends on it)
6. **API Reference**: https://console.huaweicloud.com/apiexplorer/

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
80% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · SKILL.md (reported line 299)May include surrounding context.

md
Three-track eight-phase strictly follows sequential order; chain verification prevents skipping
- API endpoints are strictly prohibited from being inferred; only obtain from SDK `_http_info` or API Explorer
- Credentials are read from environment variables; hardcoding is prohibited
- **If AK/SK is missing, the framework emits the env-var setup template to stderr and exits 77. The Agent MUST output that template to the user verbatim and instruct them to set env vars out-of-band (in their shell / PowerShell $PROFILE). The Agent MUST NEVER ask the user to type or paste AK/SK in chat. Strictly prohibited from silently skipping any step that requires credentials.** (full protocol: `references/agent-protocol.md`)
- Resources created during testing must be tracked; if any are left behind, output manual cleanup instructions
- Orchestration scenarios are auto-derived; user should review and confirm before execution
- Write operations in orchestration scenarios require per-step user confirmation

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
80% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · SKILL.md (reported line 342)May include surrounding context.

md
- **Real-Environment First** — All orchestrations and E2E flows execute against real Huawei Cloud; no mocks
- **Degrade Without Losing Value** — Single skill does not run empty orchestration phases; degrades to meaningful single-skill lifecycle tests
- **Resource Safety** — Resources created during testing must be tracked; if any remain, output clear manual cleanup instructions
- **Credentials Mandatory** — If AK/SK is missing, the framework emits the env-var setup template to stderr and exits 77. The Agent MUST output that template to the user and instruct them to set env vars out-of-band. The Agent MUST NEVER ask the user to type or paste AK/SK in chat. Strictly prohibited from silently skipping any step that requires credentials.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
80% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · scripts/lib/utils.sh (reported line 440)May include surrounding context.

sh
# === 3. No credentials — emit env-var template + exit 77 ===
  # The agent (or human caller) MUST:
  #   - Output the template below to the user (verbatim)
  #   - NEVER ask the user to type or paste AK/SK in chat
  #   - Wait for the user to set env vars out-of-band, then re-run
  _emit_cred_request_sentinel
  return 77

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
85% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · SKILL.md (reported line 318)May include surrounding context.

md
| User unsatisfied with derived orchestration scenarios | Manually edit the derived scenario or skip it; Phase 5 derivation is metadata-only, not executed |
| Phase 4 write op with no `ALLOW_WRITES=1` | Skipped with `status=skip`, no resource_changes recorded |
| Phase 4 hits missing business params (e.g. coupon_id) | Marked `status=warn`, surfaces as `manual_test_items` in the report; user must supply real data and retry |
| Cross-skill data flow mismatch | Logged in Phase 5 as `data_flow_tests` candidate; not auto-executed |
| Orphaned resources detected after E2E flow | Listed in `phase-6-summary.json` under `cleanup.manual_required` with concrete cleanup commands |

## Agent Protocol — Credential Request