huawei-cloud-skill-tester

End-to-end functional testing framework for Huawei Cloud skills — three-tier pipeline covering single-skill unit testing, multi-skill orchestration, and end-to-end full flow testing. Each phase produces structured JSON output with chain verification. Supports skill installation validation, functional analysis, CLI→SDK→API feasibility research, test case generation, real-environment execution with resource lifecycle, resource cleanup, multi-skill scenario orchestration, trigger-conflict detection, and consolidated reporting. Triggers include: 测试技能, 执行技能测试, 跑测试流程, 技能回归测试, skill test, run skill tests, test huawei cloud skill, verify skill, 测试华为云skill, 全流程测试, 编排测试, 技能完整性检查, skill-tester, 跑测试, 回归测试, 组合测试, 多skill编排, verification, e2e.

huaweicloud-skills-team@huaweiclouddev

Install

openclaw skills install @huaweiclouddev/huawei-cloud-skill-tester

Huawei Cloud Skill Tester — Three-Track Seven-Phase E2E Testing Pipeline

Independent, repeatable Huawei Cloud Skill testing framework. Does not depend on skill-creator; can test any existing Huawei Cloud Skill. Focuses on real-environment functional testing and multi-skill orchestration scenarios.

Overview

This Skill provides a three-track, seven-phase standardized testing pipeline:

Tier	Phases	Goal
Tier 1: Single-Skill Unit Testing	Phase 0~4	Verify each skill item by item: installation, feature extraction, technical research, test case generation, execution
Tier 2: Integration Testing	Phase 5~6	Multi-skill orchestration scenario derivation + end-to-end real-environment flow verification
Tier 3: Final Report	Phase 7	Consolidated report merging all phase outputs

Core Design Principles

Chain Verification — Before each Phase, check that the previous phase's JSON exists; if missing, refuse to execute
Agent-proof — Write operations require user confirmation for each item; automatic gate bypassing is not allowed
Three-Track Layering — Clear gates between Tiers; Tier 1 must be completed before entering Tier 2
Batch Repeatable — Supports --skills "skill-a,skill-b" or --all-installed
Fallback Strategy — When only 1 skill, Phase 5/6 automatically downgrade to single-skill lifecycle testing
Standardized JSON Output — All phases output in a unified schema; Phase 7 merges into a single report
Real-Environment First — All Tier 2 orchestrations execute against real Huawei Cloud; no mocks or simulations

Data Flow Diagram

text

User Input (--skills or --all-installed)
    │
    ├── Tier 1 ──── Iterate over each skill ────
    │   Phase 0 → 1 → 2 → 3 → 4
    │   (phase-N-summary.json chain validation)
    │
    ├── Tier 2 ──── Integration ────
    │   Phase 5 (orchestration scenario derivation + real-environment execution)
    │   Phase 6 (e2e full-flow: create→query→update→delete lifecycle)
    │   Only 1 skill → Downgrade single-skill closed loop
    │
    └── Tier 3 ──── Final ────
        Phase 7 (merge phase-0~6 JSON into consolidated report)

Prerequisites

hcloud CLI installed and authenticated (for Tier 2 CLI mode testing) — Reference: https://support.huaweicloud.com/qs-hcli/hcli_02_003.html
Python 3.8+ + huaweicloudsdk packages (for SDK mode testing) — SDK Reference: https://console.huaweicloud.com/apiexplorer/#/sdkcenter
Huawei Cloud AK/SK — 自动扫描所有以 HUAWEI / HW / HWC 开头的环境变量，匹配其中含 ACCESS_KEY / _AK / SECRET_KEY / _SK 的键值对。If missing, must prompt the user to provide them; if the user does not provide, terminate the process. Strictly prohibited from skipping
Target Skill must be under $HOME/.hermes/skills/ or a user-specified path
jq command (all JSON processing depends on it)
API Reference: https://console.huaweicloud.com/apiexplorer/#/openapi

Workflow — Three-Track Seven-Phase

text

Tier 1: Single-Skill Unit Testing
   Phase 0: Installation Verification (install/uninstall/reinstall)
   Phase 1: Feature Extraction (metadata + commands + resource types)
   Phase 2: Technical Research (CLI→SDK→API three-level availability)
   Phase 3: Test Case Generation (functional cases TC-F + API cases TC-A)
   Phase 4: Real-Environment Execution (read-only automatic + write operations require confirmation)

Tier 2: Integration Testing — Real-Environment Orchestration
   Phase 5: Multi-Skill Orchestration (scenario derivation → step execution → state verification)
   Phase 6: End-to-End Flow (resource lifecycle: create→query→update→delete)

Tier 3: Final Report
   Phase 7: Consolidated Report (merge phase-0~6 JSON into single report)

Phase 0: Installation Verification

Goal: Verify whether the skill can be installed/uninstalled/reinstalled normally.

Steps:

Check skill directory structure (SKILL.md exists, references/ exists)
Confirm whether Hermes has the skill installed (check if a directory with the same name exists under $HOME/.hermes/skills/)
Perform installation verification (simulated or actual installation)
Perform uninstallation verification (simulated or actual uninstallation)
Perform reinstallation verification (install → uninstall → install)
Record installation duration, directory integrity, and installation status

bash

# Simulated verification (directory structure check)
[ -f "${skill_path}/SKILL.md" ] && echo "SKILL.md exists"
[ -d "${skill_path}/scripts" ] && echo "scripts/ exists"
[ -d "${skill_path}/references" ] && echo "references/ exists"

# Installation status
hermes skills list | grep "${skill_name}"
echo $?  # 0=installed, 1=not installed

Output: phase-0-summary.json

json

{
  "install": {"status": "pass", "existing": true, "duration_s": 1.2},
  "uninstall": {"status": "skipped", "reason": "not installed"},
  "reinstall": {"status": "skipped", "reason": "not installed"},
  "directory_integrity": {"pass": true, "checks": {...}}
}

Phase 1: Feature Extraction

Goal: Extract structured feature information from SKILL.md as input for all subsequent phases.

Steps:

Read YAML frontmatter from SKILL.md → name, description, tags, triggers
Extract core commands table (Core Commands section)
Extract parameter confirmation table
Identify feature types (query/create/modify/delete)
Extract resource types involved (ECS instance, VPC, voucher, etc.)
Note whether there are write operations (Create/Update/Delete)
Read the list of test scripts under scripts/
Read the list of reference files under references/

Output: phase-1-summary.json

json

{
  "metadata": {
    "name": "huawei-cloud-bss-voucher-manage",
    "triggers": ["查代金券", "删除代金券", "list vouchers", ...],
    "tags": ["huawei-cloud", "bss", "voucher"]
  },
  "capabilities": {
    "list": ["查询代金券", "统计代金券"],
    "create": [],
    "update": [],
    "delete": ["删除代金券"]
  },
  "has_write_operations": true,
  "resource_types": ["bss_voucher"],
  "commands": [
    {"id": "CMD-01", "source": "SKILL.md", "description": "查询代金券列表", "executor": "sdk"},
    {"id": "CMD-02", "source": "SKILL.md", "description": "统计代金券", "executor": "sdk"},
    {"id": "CMD-03", "source": "SKILL.md", "description": "删除代金券", "executor": "sdk"}
  ],
  "scripts": ["scripts/test-cli-commands.sh"],
  "references": ["references/iam-policies.md", "references/api-paths.md"]
}

Phase 2: Technical Research

Goal: Perform CLI→SDK→API three-level fallback verification for each command extracted in Phase 1, determining the actual executable method.

Dependency: Phase 1 completed (phase-1-summary.json exists)

Research order (per command):

Priority	Method	Verification
1st	CLI	`hcloud <Service> <Operation> --cli-region=cn-north-4 --help`
2nd	SDK	`python3 -c "from huaweicloudsdk{service}.v2 import ..."`
3rd	API	Only from SDK source `_http_info` or Huawei Cloud API Explorer

Rule: API endpoints are strictly prohibited from being inferred; only allowed from SDK _http_info.resource_path or user confirmation from API Explorer.

Output: phase-2-summary.json

json

{
  "research": [
    {
      "cmd_id": "CMD-01",
      "description": "查询代金券列表",
      "cli": {"available": false, "reason": "BSS not in hcloud service list"},
      "sdk": {
        "available": true,
        "package": "huaweicloudsdkbss.v2",
        "method": "list_sub_customer_coupons",
        "api_path": "/v2/promotions/benefits/coupons"
      },
      "api": {"available": true, "endpoint": "/v2/promotions/benefits/coupons"},
      "recommended_executor": "sdk",
      "risk_level": "low"
    }
  ]
}

Phase 3: Test Case Generation

Goal: Generate two types of test cases based on Phase 1+2, present them to the user for confirmation.

Dependency: Phase 1+2 completed (phase-1-summary.json + phase-2-summary.json exist)

Functional case division rules:

Operation Type	Case Requirements	Risk Level
Query (List/Show/Get)	1 positive + 1 boundary (limit=0 or empty filter)	low
Create	1 standard + 1 parameter variant	high
Update	1 verification that modification took effect	medium
Delete	1 pre-deletion confirmation + 1 post-deletion verification	high
Statistics (count/aggregate)	1 positive + 1 time range boundary	low

Case IDs: TC-F-01 ~ TC-F-NN (functional), TC-A-01 ~ TC-A-NN (API/SDK)

Output: phase-3-summary.json

json

{
  "functional_cases": [
    {
      "id": "TC-F-01",
      "name": "List vouchers - positive",
      "command": "list_sub_customer_coupons(limit=10)",
      "expected": "Return voucher list, no more than 10 items",
      "is_write": false,
      "risk_level": "low",
      "executor": "sdk",
      "prerequisites": [],
      "verification": "resp.count >= 0"
    },
    {
      "id": "TC-F-03",
      "name": "Delete voucher - positive",
      "command": "reclaim_partner_coupons(coupon_id=...)",
      "expected": "Voucher status changed to reclaimed",
      "is_write": true,
      "risk_level": "high",
      "executor": "sdk",
      "prerequisites": ["TC-F-01 (provide valid coupon_id)"],
      "verification": "Query after delete to confirm status change"
    }
  ],
  "api_cases": [...]
}

Phase 4: Execution

Goal: Execute test cases from Phase 3 one by one; read-only runs automatically, write operations require per-case confirmation.

Dependency: Phase 3 completed (phase-3-summary.json exists)

Execution rules:

text

Pre-check: Is AK/SK set?
  if AK/SK not set:
    Prompt user to provide AK/SK
    if user does not provide:
      Terminate process, output "⛔ AK/SK missing, cannot execute tests"
    else:
      Set as environment variables, continue execution

Iterate over each test case:
  if is_write == false:
    Auto-execute → Record pass/fail
  if is_write == true:
    Show command and expected → Wait for user y/N confirmation
    if confirmed:
      Execute → Record pass/fail + resource changes
    else:
      Skip → Record skipped (user_cancelled)

Resource change record (key fields):

json

"resource_changes": [
  {
    "tc_id": "TC-F-03",
    "resource_type": "bss_voucher",
    "resource_id": "VCH-abc123",
    "change_type": "deleted",
    "cleanup_method": {"type": "sdk", "command": "already deleted, no cleanup needed"},
    "cleanup_required": false
  }
]

Output: phase-4-summary.json

json

{
  "execution_results": [
    {"tc_id": "TC-F-01", "status": "pass", "duration_s": 2.1, "output_snippet": "..."},
    {"tc_id": "TC-F-03", "status": "pass", "duration_s": 1.5, "output_snippet": "...",
     "resource_changes": [{"resource_type": "bss_voucher", "change_type": "deleted", ...}]}
  ],
  "statistics": {"total": 10, "pass": 9, "fail": 0, "skip": 1},
  "all_resources_changed": [...]
}

Phase 5: Multi-Skill Orchestration — Real-Environment Scenario Testing

Goal: Derive multi-skill business scenarios and execute them against the real Huawei Cloud environment to verify cross-skill integration.

Dependency: Phase 4 completed for all tested skills

Branch logic:

text

if skills_count == 1:
  Downgrade to single-skill orchestration:
    Extract all feature points from Phase 1
    Sort by CRUD lifecycle (query → create → update → delete)
    Execute sequentially against real environment
    Verify each step's output feeds correctly into the next

if skills_count >= 2:
  5a: Scenario derivation
      - Group feature points by resource type from each skill's Phase 1 output
      - Sort by dependency order (e.g., VPC must exist before ECS)
      - Auto-generate scenario chains with resource passing between skills
      - Present scenarios to user for confirmation
  5b: Real-environment execution
      - Execute each step in the confirmed scenario chain
      - Pass resource IDs/outputs between steps as runtime context
      - Record actual CLI/SDK output for each step
  5c: Cross-skill data flow verification
      - Verify output of skill A's operation can be consumed by skill B
      - Check data format compatibility (JSON field mapping)
  5d: Rollback on failure
      - If any step fails, execute rollback steps for already-created resources

Auto-derivation example (ECS + VPC + EIP):

text

Input skills: [ECS-manage, VPC-manage, EIP-manage]
Derived scenario:
  Step 1: VPC-manage.CreateVPC → outputs vpc_id
  Step 2: ECS-manage.CreateECS (in vpc_id) → outputs instance_id
  Step 3: EIP-manage.CreateEIP → outputs eip_id
  Step 4: EIP-manage.BindEIP (bind to instance_id) → outputs binding_status
  Step 5: ECS-manage.ListInstances (verify instance is running)
  Step 6: EIP-manage.UnbindEIP (unbind from instance_id)
  Step 7: ECS-manage.DeleteECS (instance_id)
  Step 8: EIP-manage.ReleaseEIP (eip_id)
  Step 9: VPC-manage.DeleteVPC (vpc_id)

Resource passing mechanism:

json

{
  "runtime_context": {
    "vpc_id": {"from_step": 1, "skill": "VPC-manage", "output_field": "vpc.id"},
    "instance_id": {"from_step": 2, "skill": "ECS-manage", "output_field": "server.id"}
  }
}

Execution mode: All steps run against real Huawei Cloud. Read-only steps (List/Show/Describe) run automatically. Write steps (Create/Delete/Update) require user confirmation for each step.

Output: phase-5-summary.json

json

{
  "mode": "full" | "downgraded_single",
  "scenario": {
    "name": "VPC-ECS-EIP lifecycle",
    "skills_involved": ["vpc-manage", "ecs-manage", "eip-manage"],
    "steps": [
      {"seq": 1, "skill": "vpc-manage", "action": "CreateVPC", "status": "pass", "output": {"vpc_id": "vpc-abc"}},
      {"seq": 2, "skill": "ecs-manage", "action": "CreateECS", "status": "pass", "output": {"instance_id": "srv-xyz"}},
      {"seq": 3, "skill": "eip-manage", "action": "CreateEIP", "status": "pass", "output": {"eip_id": "eip-123"}},
      {"seq": 4, "skill": "eip-manage", "action": "BindEIP", "status": "pass", "output": {}},
      {"seq": 5, "skill": "ecs-manage", "action": "ListInstances", "status": "pass", "output": {}},
      {"seq": 6, "skill": "eip-manage", "action": "UnbindEIP", "status": "pass", "output": {}},
      {"seq": 7, "skill": "ecs-manage", "action": "DeleteECS", "status": "pass", "output": {}},
      {"seq": 8, "skill": "eip-manage", "action": "ReleaseEIP", "status": "pass", "output": {}},
      {"seq": 9, "skill": "vpc-manage", "action": "DeleteVPC", "status": "pass", "output": {}}
    ]
  },
  "data_flow_verification": {"pass": true, "mismatches": []},
  "rollback_required": false
}

Phase 6: End-to-End Flow Testing — Real-Environment Lifecycle

Goal: End-to-end verification of complete resource lifecycles against real Huawei Cloud, automatically deriving scenario chains from Phase 1 feature lists and executing them with real API calls.

Dependency: Phase 5 completed (phase-5-summary.json exists)

Branch logic:

text

if skills_count == 1:
  Single-skill closed loop:
    Sort all feature points by create→list→show→update→delete
    Chain into a single-skill resource lifecycle
    Execute each step via real CLI/SDK against production
    Verify resource state after each mutation

if skills_count >= 2:
  6a: Scenario derivation (from Phase 5 output)
      - Use the confirmed scenario chain from Phase 5
      - Extend with additional verification steps
  6b: Lifecycle execution
      - Execute each step against real Huawei Cloud environment
      - Read-only steps auto-execute; write steps prompt user
  6c: State consistency verification
      - After each write operation, verify via read operation
      - Confirm resource state matches expected
  6d: Cross-step data validation
      - Verify output schema compatibility between steps
      - Detect field name mismatches, type mismatches
  6e: Cleanup verification
      - Verify all created resources are properly deleted
      - Check for orphaned resources

Single-skill closed loop example (RDS query skill):

text

Steps:
  Step 1: ListInstances (read-only, auto) → get instance count
  Step 2: ShowInstanceDetail (if instance exists) → get instance config
  Step 3: ListConfigurations (auto) → get parameter templates
  Step 4: ShowBackupPolicy (if instance exists) → get backup config
  Step 5: Analyze slow SQL (read-only, auto) → ListSlowLogs + ListTopSqls
  Step 6: Parameter tuning recommendation (analysis, no API call)
  Step 7: Backup strategy assessment (analysis, no API call)

Multi-skill E2E example (EC2 + EVS):

text

Steps:
  Step 1: Create EVS volume → verify volume exists
  Step 2: Attach volume to ECS instance → verify attachment
  Step 3: Query volume metrics → verify I/O
  Step 4: Detach volume → verify detached
  Step 5: Delete volume → verify deleted

Output: phase-6-summary.json

json

{
  "mode": "full" | "downgraded_single_skill_flow",
  "scenario": {
    "name": "RDS Intelligent Service Inspection",
    "skills_involved": ["huawei-cloud-rds-intelligent-service"],
    "steps": [
      {"seq": 1, "action": "ListInstances", "status": "pass", "output_summary": "3 instances found"},
      {"seq": 2, "action": "ShowInstanceDetail", "status": "pass", "output_summary": "instance config retrieved"},
      {"seq": 3, "action": "ListConfigurations", "status": "pass", "output_summary": "5 templates found"},
      {"seq": 4, "action": "ListDatastores", "status": "pass", "output_summary": "MySQL 8.0 available"},
      {"seq": 5, "action": "ShowBackupUsage", "status": "pass", "output_summary": "2.3GB used"},
      {"seq": 6, "action": "ListInstanceDiagnosis", "status": "pass", "output_summary": "No issues found"}
    ]
  },
  "state_consistency": {"pass": true},
  "cleanup_verification": {"pass": true, "orphaned": []}
}

Phase 7: Consolidated Report

Goal: Merge Phase 0~6 JSON outputs into a single, comprehensive test report.

Dependency: Phase 0~6 all exist

Output: phase-7-summary.json

json

{
  "test_id": "test-20260716-100300",
  "phases_summary": [
    {"phase": 0, "name": "install-check", "verdict": "pass", "duration_s": 3.5},
    {"phase": 1, "name": "skill-analysis", "verdict": "pass", "duration_s": 12.0},
    {"phase": 2, "name": "tech-research", "verdict": "pass", "duration_s": 45.0},
    {"phase": 3, "name": "test-generation", "verdict": "pass", "duration_s": 8.0},
    {"phase": 4, "name": "execution", "verdict": "pass", "duration_s": 120.0},
    {"phase": 5, "name": "orchestration", "verdict": "pass", "duration_s": 200.0},
    {"phase": 6, "name": "e2e-flow", "verdict": "pass", "duration_s": 180.0}
  ],
  "overall_statistics": {
    "total_phases": 7,
    "pass": 7,
    "fail": 0,
    "skipped": 0,
    "total_duration_s": 583.5,
    "test_cases_total": 24,
    "test_cases_pass": 22,
    "test_cases_fail": 1,
    "test_cases_skip": 1,
    "orchestration_scenarios": 2,
    "e2e_flows_executed": 1
  },
  "resources_created": 3,
  "resources_cleaned": 2,
  "resources_manual": 1,
  "manual_interventions": [
    {
      "phase": 5,
      "resource_type": "disk_volume",
      "resource_id": "vol-xyz789",
      "steps": ["hcloud EVS DeleteVolume --volume_id=vol-xyz789"]
    }
  ],
  "html_report": "reports/test-20260716-100300.html"
}

KooCLI Command Format Standard

This testing framework uses bash scripts as the primary execution mode, not direct hcloud CLI commands. However, when executing test cases, the framework constructs hcloud CLI commands in the following format:

bash

hcloud <Service> <Operation> --cli-region={region} [--param1=value1 ...]

Format Rules:

Rule	Description
Service name	Follows KooCLI Services (uppercase: ECS, VPC, OBS; title case: CloudPond, IAMAccessAnalyzer)
Operation name	PascalCase (e.g., ListServersDetails, ListBuckets)
Region	Always include `--cli-region={region}` parameter
Parameters	Use `--param=value` syntax
Read-only limit	Always append `--limit=1` for exploratory queries

For OBS service, the framework uses hcloud obs (obsutil) subsystem:

bash

hcloud obs <command> [args...] [options...]

Core Commands

Full Pipeline Run

bash

# Specify skills
bash scripts/run-test-pipeline.sh --skills "huawei-cloud-bss-voucher-manage"

# Specify multiple skills (comma-separated)
bash scripts/run-test-pipeline.sh --skills "huawei-cloud-bss-voucher-manage, huawei-cloud-ecs-manage"

# Scan all installed
bash scripts/run-test-pipeline.sh --all-installed

# Start from a specific phase (recovery scenarios only)
bash scripts/run-test-pipeline.sh --skills "bss-voucher" --phase 4

# Fresh mode
bash scripts/run-test-pipeline.sh --skills "bss-voucher" --fresh

Single Phase Run (Debug)

bash

bash scripts/tier1/phase-0-install-check.sh --skill "huawei-cloud-bss-voucher-manage"
bash scripts/tier1/phase-1-skill-analysis.sh --skill "huawei-cloud-bss-voucher-manage"
bash scripts/tier2/phase-5-orchestration.sh --skills "skill-a, skill-b"
bash scripts/tier3/phase-7-final-report.sh --skills "skill-a, skill-b"

Run Multi-Skill Orchestration

bash

# Derive and execute orchestration scenarios for 3 skills
bash scripts/tier2/phase-5-orchestration.sh --skills "ecs-manage, vpc-manage, eip-manage"

# Run E2E lifecycle test for a single skill
bash scripts/tier2/phase-6-full-flow.sh --skill "huawei-cloud-rds-intelligent-service"

Parameters

Parameter	Required	Description	Example
`--skills`	Mutually exclusive	Comma-separated skill names or directory names	`"bss-voucher-manage, ecs-manage"`
`--all-installed`	Mutually exclusive	Scan all `huawei-cloud-*` under $HOME/.hermes/skills/	—
`--phase`	No	Start from a specific Phase (defaults to resume from missing phase)	`--phase 0`
`--fresh`	No	Delete all existing phase-*.json and start from scratch	—
`--output`	No	Report output directory (default: reports/)	`--output ./test-reports`
`--skill-path`	No	Skill directory path (default: $HOME/.hermes/skills/huawei-cloud/)	`--skill-path ./skills`

References

references/architecture.md — Three-track seven-phase architecture diagram (Mermaid)
references/output-schema-spec.md — Complete JSON field specification for each phase
references/phase-transition-rules.md — Phase transition/fallback/skip rules

Output Format

All phases output phase-N-summary.json; Phase 7 merges them into a single report. See references/output-schema-spec.md for the JSON schema.

Phase 5 and 6 additionally output scenario execution logs with real CLI/SDK responses for auditability.

Best Practices

Complete Tier 1 before entering Tier 2 to ensure skills are individually functional before orchestration
Confirm write operations one by one in Phase 4 and Phase 5/6; do not batch-confirm to avoid misoperations
With only 1 skill, Phase 5/6 automatically downgrade to single-skill closed loop; no need to manually skip
When using --fresh to reset and rerun, confirm there are no uncleaned test resources
Review orchestration scenarios before execution to ensure resource dependency order is correct

Notes

Three-track seven-phase strictly follows sequential order; chain verification prevents skipping
API endpoints are strictly prohibited from being inferred; only obtain from SDK _http_info or API Explorer
Credentials are read from environment variables; hardcoding is prohibited
If AK/SK is missing, must prompt the user to provide them; if the user does not provide, terminate the process. Strictly prohibited from skipping any step that requires credentials
Resources created during testing must be tracked; if any are left behind, output manual cleanup instructions
Orchestration scenarios are auto-derived; user should review and confirm before execution
Write operations in orchestration scenarios require per-step user confirmation

Edge Cases

Scenario	Handling
Skill directory does not exist	Report error and terminate, output available skill list
AK/SK environment variables not set	Prompt user to provide AK/SK; if user does not provide, terminate process, strictly prohibited from skipping
User specifies skill name but not installed in Hermes	--fresh performs directory-level detection; if not found, report error with guidance
Some Phase JSON files deleted	Chain detection → Restart from the deleted Phase
Network interruption during Phase 4 execution	Already executed case results are not lost; on rerun, skip passed cases (via `--phase` flag)
User hits Ctrl+C mid-execution	Already output phase JSON is valid; next time `--resume` will recover from the current phase
Only 1 skill under test	Phase 5 → single-skill orchestration, Phase 6 → single-skill closed loop
User unsatisfied with derived orchestration scenarios	User can manually edit the scenario or choose to skip it
Multi-skill scenario step fails midway	Execute rollback steps for already-created resources, report partial failure
Cross-skill data flow mismatch	Log field mapping details, suggest adapter/fix
Orphaned resources detected after E2E flow	List in report with manual cleanup instructions

Design Principles

Chain Verification — Each Phase checks the previous phase's JSON to prevent skipping
Agent-proof — Write operations must be confirmed by the user; fake confirmations are not allowed
Data-Driven — All phases output in JSON format; Phase 7 merges
Batch Repeatable — The same set of skills can be tested repeatedly; --fresh resets
Real-Environment First — All orchestrations and E2E flows execute against real Huawei Cloud; no mocks
Degrade Without Losing Value — Single skill does not run empty orchestration phases; degrades to meaningful single-skill lifecycle tests
Resource Safety — Resources created during testing must be tracked; if any remain, output clear manual cleanup instructions
Credentials Mandatory — If AK/SK is missing, must prompt the user to provide; if not provided, terminate process. Strictly prohibited from skipping