T03 · Remote Payload Retrieval and Execution
Error
- Location
scripts/common.py:1187- Finding
Automatic Execution of an Unverified Remote Installation Script
- Content
View full analysis
- Remediation
View remediation
Security audit
alibabacloud-migration-lhm-migrate-hive-to-paimon
Security checks for vulnerabilities and agentic risk
Overview
This appears to be a real Hive-to-Paimon migration skill, but it needs Review because it can change production data and includes unsafe command execution, dependency installation, and credential-handling paths.
Install only in a controlled migration environment. Preinstall rclone from a trusted, pinned source; do not rely on automatic installation. Run dry-runs first, inspect generated SQL and sync_commands.sh, use dedicated least-privilege temporary credentials, avoid passing secrets through CLI flags or background mode, and require an explicit human approval process before any DROP, INSERT OVERWRITE, or rclone write.
Vulnerability Patterns
- Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
- Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
- Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
- Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
- Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (5)
T09 · Insecure Skill Coding Practices
Error
- Location
scripts/incremental_migrate.py:86- Finding
Arbitrary Shell Command Injection Through Imported sync_commands.sh Files
- Content
View full analysis
Tuple[List[str], List[str]]: """ Parse sync_commands.sh and separate rclone configuration and copy commands. """ config_cmds = [] copy_cmds = [] with open(sh_path, 'r', encoding='utf-8') as f: for line in f: line = line.strip() if not line or line.startswith('#') or line.startswith('#!/'): continue if line.startswith('rclone config'): config_cmds.append(line) elif line.startswith('rclone copy') or line.startswith('rclone sync'): copy_cmds.append(line) return config_cmds, copy_cmds ``` Accepted copy commands are executed by a shell: ```python process = subprocess.Popen( cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True ) ``` Accepted configuration commands are also executed by a shell: ```python for cmd in config_cmds: _safe_print(f" 执行: {cmd[:100]}{'...' if len(cmd) > 100 else ''}", master_log_f) result = subprocess.run(cmd, shell=True, capture_output=True, text=True) if result.returncode != 0: _safe_print(f" 错误: {result.stderr[:200]}", master_log_f) ``` ### Technical Analysis The incremental migration workflow trusts `sync_commands.sh` from a user-supplied exploration directory. The parser does not parse or validate command arguments; it merely verifies that a stripped line begins with an expected string. Because the entire line is then passed to `/bin/sh` through `shell=True`, shell control operators and expansions remain active. F ...[truncated 1827 chars]- Remediation
View remediation
T09 · Insecure Skill Coding Practices
Error
- Location
scripts/step4_rclone_sync.py:125- Finding
Shell Command Injection in Full-Migration rclone Command Construction
- Content
View full analysis
- Remediation
View remediation
T09 · Insecure Skill Coding Practices
Error
- Location
scripts/step4_rclone_sync.py:185- Finding
Cloud Credentials Exposed in Process Arguments and Background Logs
- Content
View full analysis
- Remediation
View remediation
T09 · Insecure Skill Coding Practices
Warning
- Location
scripts/step3_execute_ddl.py:162- Finding
Destructive Migration Operations Do Not Enforce the Documented Confirmation Policy
- Content
View full analysis
- Remediation
View remediation
` value. 5. Require a separate acknowledgement for `--force`, including the exact target table name. 6. Refuse destructive execution in a noninteractive session unless a validated approval artifact is supplied. 7. Check target existence and, where possible, verify that a backup or snapshot exists before dropping or overwriting. 8. Use separate credentials for preview and execution, with write privileges granted only to the execution identity. 9. Preserve audit records of the approved plan without storing secrets. ]]>
Vulnerability Patterns
- Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
- Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
- Output HandlingUnvalidated Output Injection, Cross-Context Output, Unbounded Output
- Memory PoisoningPersistent Context Injection, Context Window Stuffing, Memory Manipulation
- Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults
Findings (57)
Tp4
High
- Category
- MCP Tool Poisoning
- Confidence
- 90% confidence
- Finding
The supplied code matches part of the description well: it clearly implements the incremental migration three-phase pipeline (DDL → rclone → INSERT), background execution, parallel rclone execution, dry-run behavior, logging, CSV outputs, and summary reporting. However, the declared purpose describes a much broader skill spanning both full and incremental migration, standalone Hive DDL conversion, direct-read mode, row-count verification, and several CLI features such as --auto-create-db, --force, and --verify. None of those broader/full-migration or converter capabilities appear in this code chunk. Since the task is to compare the declared description against the supplied code chunk itself, the description overstates what this particular code actually does. This is therefore a mismatch, though the implemented portion is strongly aligned with the incremental-migration subset of the description.
- Content
Ae1
High
- Category
- analysis-evasion
- Confidence
- 100% confidence
- Finding
Referenced artifact was not completely inspected
- Content
md | 3. Execute table-creation DDL | `step3_execute_ddl.py` | `ddl_result.csv` |
Ae1
High
- Category
- analysis-evasion
- Confidence
- 100% confidence
- Finding
Referenced artifact was not completely inspected
- Content
md The orchestrator `main.py` chains Step 1-5, supporting `--start-step` for resumable runs and `--skip-steps` to skip specific steps.
Ae1
High
- Category
- analysis-evasion
- Confidence
- 100% confidence
- Finding
Referenced artifact was not completely inspected
- Content
md The orchestrator `main.py` chains Step 1-5, supporting `--start-step` for resumable runs and `--skip-steps` to skip specific steps.
External Script Fetching
High
- Category
- Supply Chain
- Confidence
- 97% confidence
- Finding
The documentation recommends a one-line remote script fetch and execution pattern: curl ... | bash. This executes unpinned code directly from the network without signature or checksum verification, enabling supply-chain compromise, MITM abuse, or upstream-site compromise to result in arbitrary command execution on the host.
- Content
rclone (data sync tool) — the script auto-detects and tries to install it
CentOS/RHEL: yum install -y epel-release && yum install -y rclone
Debian/Ubuntu: apt-get install -y rclone
Generic: curl https://rclone.org/install.sh | bash
text ---
Chaining Abuse
High
- Category
- Tool Misuse
- Confidence
- 96% confidence
- Finding
The shell pipeline into bash is a classic unsafe command-chaining pattern because it takes untrusted network content and immediately executes it in a privileged local shell context. In this migration skill's environment, that could expose credentials, alter migration scripts, tamper with data paths, or fully compromise the host running production data operations.
- Content
rclone (data sync tool) — the script auto-detects and tries to install it
CentOS/RHEL: yum install -y epel-release && yum install -y rclone
Debian/Ubuntu: apt-get install -y rclone
Generic: curl https://rclone.org/install.sh | bash
text ---
Memory Manipulation
High
- Category
- Memory Poisoning
- Confidence
- 85% confidence
- Finding
Skill manipulates agent memory, state, or stored context. Memory corruption can alter personality, override safety rules, or cause unpredictable behavior.
- Content
md 1. **Phase 1 — DDL execution**: extract CREATE TABLE statements from `paimon_sync.sql` and create external + internal tables via Spark Thrift Server. 2. **Phase 2 — rclone sync**: extract rclone copy commands from `sync_commands.sh` and sync data in parallel (HDFS → OSS). 3. **Phase 3 — INSERT OVERWRITE**: extract INSERT OVERWRITE statements from `paimon_sync.sql` and load data. The order is fixed as DDL → rclone → DML, ensuring table creation precedes sync, and sync precedes data load.
Missing User Warnings
High
- Category
- Not specified by scanner
- Confidence
- 96% confidence
- Finding
Documenting an INSERT OVERWRITE load step without any warning is dangerous because INSERT OVERWRITE commonly replaces target table or partition contents, which can cause irreversible data loss if pointed at the wrong destination. In a migration skill, this is especially risky because users may assume the process is additive rather than destructive, and an agent could execute it at scale.
- Content
Context-Inappropriate Capability
High
- Category
- Not specified by scanner
- Confidence
- 98% confidence
- Finding
The common module can automatically install host software using system package managers or fallback script execution, expanding the skill from data migration into system modification. In production migration environments this is dangerous because it may run with elevated privileges, alter the host unexpectedly, and widen the blast radius if the installation path is abused or compromised.
- Content
External Script Fetching
High
- Category
- Supply Chain
- Confidence
- 100% confidence
- Finding
Fetching an external installer script and piping it directly to bash is a textbook remote code execution anti-pattern. It trusts live network content at execution time, so compromise of the remote site, DNS, TLS interception, or repository contents can immediately yield arbitrary code execution on the migration host.
- Content
python if shutil.which('curl'): print("尝试通过官方脚本安装 rclone...") result = _sp.run( 'curl -s https://rclone.org/install.sh | bash', shell=True, capture_output=True, text=True, timeout=300 ) if result.returncode == 0 and shutil.which('rclone'):
External Script Fetching
High
- Category
- Supply Chain
- Confidence
- 90% confidence
- Finding
Remote code is downloaded and executed. This bypasses code review and could introduce malicious code.
- Content
python print("错误: rclone 自动安装失败，请手动安装:") print(" CentOS/RHEL: yum install -y epel-release && yum install -y rclone") print(" Debian/Ubuntu: apt-get install -y rclone") print(" 通用: curl https://rclone.org/install.sh | bash") return False
Tool Parameter Abuse
High
- Category
- Tool Misuse
- Confidence
- 99% confidence
- Finding
This duplicate finding identifies the same core issue: execution of attacker-influenced shell text via Popen(..., shell=True). In a data-migration skill that processes external output directories, this creates a realistic path to arbitrary code execution under an operator account with access to HDFS, OSS, and Spark resources.
- Content
python lf.write(f"# 命令: {cmd}\n") lf.write(f"# 开始时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n\n") process = subprocess.Popen( cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True
Tool Parameter Abuse
High
- Category
- Tool Misuse
- Confidence
- 99% confidence
- Finding
This duplicate finding identifies the same core issue: execution of attacker-influenced shell text via Popen(..., shell=True). In a data-migration skill that processes external output directories, this creates a realistic path to arbitrary code execution under an operator account with access to HDFS, OSS, and Spark resources.
- Content
python lf.write(f"# 命令: {cmd}\n") lf.write(f"# 开始时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n\n") process = subprocess.Popen( cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True
Tool Parameter Abuse
High
- Category
- Tool Misuse
- Confidence
- 98% confidence
- Finding
The script abuses tool parameters by replaying parsed rclone config command strings through the shell. An attacker who can control migration artifacts can inject extra shell operations or alter remote definitions to redirect data and harvest credentials, causing host compromise and data exfiltration.
- Content
python for cmd in config_cmds: _safe_print(f" 执行: {cmd[:100]}{'...' if len(cmd) > 100 else ''}", master_log_f) result = subprocess.run(cmd, shell=True, capture_output=True, text=True) if result.returncode != 0: _safe_print(f" 错误: {result.stderr[:200]}", master_log_f) _safe_print("rclone 配置完成", master_log_f)
Tool Parameter Abuse
High
- Category
- Tool Misuse
- Confidence
- 99% confidence
- Finding
This duplicate finding points to the same underlying issue: shell execution of a command assembled from partially untrusted inputs. If exploited, an attacker could run arbitrary commands under the privileges of the migration process, potentially accessing credentials, altering migration results, or pivoting into connected storage environments.
- Content
python lf.write(f"# 命令: {cmd}\n") lf.write(f"# 开始时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n\n") process = subprocess.Popen( cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True
Tool Parameter Abuse
High
- Category
- Tool Misuse
- Confidence
- 99% confidence
- Finding
This duplicate finding points to the same underlying issue: shell execution of a command assembled from partially untrusted inputs. If exploited, an attacker could run arbitrary commands under the privileges of the migration process, potentially accessing credentials, altering migration results, or pivoting into connected storage environments.
- Content
python lf.write(f"# 命令: {cmd}\n") lf.write(f"# 开始时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n\n") process = subprocess.Popen( cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True
Memory Manipulation
High
- Category
- Memory Poisoning
- Confidence
- 85% confidence
- Finding
Skill manipulates agent memory, state, or stored context. Memory corruption can alter personality, override safety rules, or cause unpredictable behavior.
- Content
python output_dir = os.path.dirname(args.manifest) all_sql_path = os.path.join(output_dir, 'insert_overwrite_all.sql') with open(all_sql_path, 'w', encoding='utf-8') as f: f.write(f"-- INSERT OVERWRITE Statements\n") f.write(f"-- Generated at: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n") f.write(f"-- Total: {len(insert_tasks)} tables\n\n") for full_name, sql, fmt, _ in insert_tasks:
Unvalidated Output Injection
High
- Category
- Output Handling
- Confidence
- 100% confidence
- Finding
The function builds executable Hive SQL by directly interpolating untrusted identifiers such as result_table and table_name into statements without validation or quoting. If an attacker can influence the tables list or result table name, they can inject arbitrary SQL or redirect writes to unintended tables, which is especially risky in a migration tool that is expected to run with broad data-access privileges.
- Content
python if not partitions: # 无分区，计算全表 lines.append(f"INSERT INTO TABLE {result_table}") lines.append( f"SELECT '{db}' as db, '{tbl}' as tbl, 'all' as part, " f"count(*) as num FROM {table_name};"
Unvalidated Output Injection
High
- Category
- Output Handling
- Confidence
- 100% confidence
- Finding
This branch interpolates result_table, table_name, part_field, and partition_val directly into generated SQL, including both identifier and string literal contexts. An attacker controlling partition metadata or override values could inject SQL through crafted field names or quote-breaking partition values, leading to arbitrary query execution or incorrect verification results.
- Content
python partition_val = default_partition_values.get(part_field) if partition_val: lines.append(f"INSERT INTO TABLE {result_table}") lines.append( f"SELECT '{db}' as db, '{tbl}' as tbl, " f"'{part_field}={partition_val}' as part, "
Unvalidated Output Injection
High
- Category
- Output Handling
- Confidence
- 100% confidence
- Finding
Like the earlier branch, this path emits executable SQL using unvalidated result_table and table_name values. Because the skill processes migration metadata and CSV-derived inputs, a malicious or malformed source can cause SQL injection, write redirection, or execution of additional statements in a high-privilege data platform context.
- Content
python ) else: # 未知分区字段，计算全表 lines.append(f"INSERT INTO TABLE {result_table}") lines.append( f"SELECT '{db}' as db, '{tbl}' as tbl, 'all' as part, " f"count(*) as num FROM {table_name};"
Undeclared Tool Scope
Medium
- Category
- MCP Least Privilege
- Confidence
- 91% confidence
- Finding
The skill advertises and documents capabilities that involve shell execution, network access, file reads/writes, and environment-variable handling, but it does not declare an explicit tool scope such as permissions or allowed-tools. In an agent setting, that ambiguity increases the chance an agent can invoke powerful operations beyond what a reviewer expects, especially for a migration workflow that performs destructive writes.
- Content
Unsafe Defaults
Medium
- Category
- Tool Misuse
- Confidence
- 94% confidence
- Finding
The documented default config shows Spark Thrift authentication disabled (
auth = NONE) while also using plain HTTP, which encourages deployments without transport or identity protection. In a migration tool that connects to metastore, OSS, and Spark infrastructure, this can allow unauthorized query execution or credential/session exposure if the service is reachable over a network.- Content
username = <SPARK_USER> password = ${SPARK_PASSWORD} # recommended: read from env var database = default auth = NONE scheme = http
text
Missing User Warnings
Medium
- Category
- Not specified by scanner
- Confidence
- 88% confidence
- Finding
The overview explicitly promotes batch migration and incremental sync involving HDFS→OSS copying and table-loading operations, but it does not warn users that these actions can move large volumes of data, incur cost, or modify destination datasets. In an agent skill context, omission of safety warnings increases the chance that an automated system or user triggers impactful operations without understanding side effects.
- Content
Unsafe Defaults
Medium
- Category
- Tool Misuse
- Confidence
- 75% confidence
- Finding
Tool defaults are unsafe or overly permissive (e.g. disabled TLS verification, no authentication, world-writable permissions). Unsafe defaults widen the attack surface.
- Content
md | table_manifest.csv missing | Run Step 1 first, or check the --output-dir path | | pyhive install fails | `pip install pyhive[hive]`; confirm thrift/sasl dependencies | | DLS data unreachable by rclone | OSS-HDFS (DLS) and plain OSS are different storage layers; use `--direct-read` mode | | EMR Spark Gateway 401 | Check auth settings; EMR Gateway usually uses `auth=NONE`, `scheme=https`, `port=443` | | AK/SK leak risk | Never commit config.ini to version control; inject credentials via environment variables | ## Diagnostic Tips
subprocess module call
Medium
- Category
- Dangerous Code Execution
- Confidence
- 90% confidence
- Finding
The subprocess call itself is argv-based, but it embeds database and table names directly into a Hive CLI SQL string without escaping backticks or otherwise hardening identifier handling. If an attacker can influence metastore names or upstream inputs beyond the assumed \w-safe paths, this could cause Hive command/SQL injection or at minimum unreliable execution against unexpected objects; in a migration tool that queries external metadata sources, that trust assumption is risky.
- Content
python # 通过 hive CLI 获取 DDL try: result = subprocess.run( ['hive', '-S', '-e', f'SHOW CREATE TABLE `{db_name}`.`{tbl_name}`'], capture_output=True, text=True, timeout=60 )
Static analysis
No suspicious patterns detected.