Huawei Cloud Cce Workload Manager

Huawei Cloud CCE/UCS workload lifecycle management skill using hcloud CLI for kubeconfig acquisition and kubectl for Kubernetes resource operations. Use this skill when the user wants to: (1) obtain kubeconfig for CCE clusters, (2) obtain federation kubeconfig for UCS fleet (multi-cluster operations), (3) manage Deployment/StatefulSet/DaemonSet/Job/CronJob lifecycle, (4) configure HPA autoscaling, (5) manage Service/Ingress/ConfigMap/Secret/PVC, (6) observe Pod status/logs/events, (7) manage namespaces, (8) install and configure kubectl. Trigger: user mentions "CCE workload", "k8s workload", "UCS fleet workload", "Deployment", "StatefulSet", "DaemonSet", "Job", "CronJob", "HPA", "kubectl", "kubeconfig", "federation kubeconfig", "Pod logs", "CCE 负载", "UCS 联邦负载", "工作负载", "部署", "有状态副本集", "守护进程集", "任务", "定时任务", "弹性伸缩", "服务", "路由", "配置项", "密钥", "存储卷", "Pod 日志", "命名空间"

Huawei Cloud CCE/UCS Workload Manager

Overview

This skill provides workload lifecycle management capabilities for Huawei Cloud CCE and UCS-managed Kubernetes clusters using hcloud CLI for kubeconfig acquisition and kubectl for Kubernetes resource operations.

Architecture: hcloud CLI → kubeconfig YAML → kubectl --kubeconfig → k8s resources (Deployment/StatefulSet/DaemonSet/Job/CronJob/Service/Ingress/ConfigMap/Secret/PVC)

Related Skills:

• cce-cluster-management - CCE cluster infrastructure creation, scaling, and deletion
• ucs-cluster-onboarding-manager - UCS cluster onboarding, lifecycle, and fleet grouping
• ucs-policy-governor - UCS policy governance, compliance, and audit management

Capabilities:

• Obtain kubeconfig for CCE clusters (direct cluster access)
• Obtain federation kubeconfig for UCS fleet (multi-cluster fleet operations)
• Manage Deployment/StatefulSet/DaemonSet lifecycle (create, query, scale, update, delete)
• Manage Job/CronJob lifecycle (create, query, suspend, resume, delete)
• Configure HPA autoscaling for Deployments and StatefulSets
• Manage Service/Ingress networking, ConfigMap/Secret configuration, PVC storage
• Observe Pod status, logs, and events
• Manage namespaces
• Install and configure kubectl

Typical Use Cases:

• "Get kubeconfig for my CCE cluster"
• "Deploy an application to CCE"
• "Scale my Deployment to 5 replicas"
• "Roll back a Deployment update"
• "Create a StatefulSet for my database"
• "Deploy a logging agent as DaemonSet"
• "Create a CronJob for scheduled tasks"
• "Set up HPA for auto-scaling"
• "Check Pod logs for debugging"
• "Create a Service and Ingress for my app"
• "Manage ConfigMaps and Secrets"
• "Create a PersistentVolumeClaim"
• "List namespaces in my cluster"
• "Install kubectl on my machine"

Prerequisites

1. hcloud CLI Requirements (MANDATORY)

• hcloud CLI installed (version >= 7.2.2)
• Run hcloud version to verify installation
• First-time usage: printf "y\n" | hcloud version to accept privacy statement

2. kubectl Requirements (MANDATORY)

• kubectl installed (version compatible with cluster Kubernetes version)
• See Task: kubectl Setup for installation guidance
• Run kubectl version --client to verify installation

3. Credential Configuration

• Valid Huawei Cloud credentials (AK/SK mode)
• Security Rules:
  • 🚫 Never expose AK/SK values in code, conversation, or commands
  • 🚫 Never use echo $HUAWEI_CLOUD_AK or echo $HUAWEI_CLOUD_SK to check credentials
  • ✅ Use environment variables: HUAWEI_CLOUD_AK, HUAWEI_CLOUD_SK, HUAWEI_CLOUD_REGION
  • ✅ Prefer IAM users over root account for cloud operations
  • ✅ Enable MFA for sensitive operations

Configuration Method (Environment Variables Only):

bash

export HUAWEI_CLOUD_AK=<your-ak>
export HUAWEI_CLOUD_SK=<your-sk>
export HUAWEI_CLOUD_REGION=cn-north-4

⚠️ Important Security Notes:

• Never commit credentials to version control
• Use IAM users with minimal required permissions
• Enable MFA for sensitive operations
• Rotate AK/SK regularly

4. IAM Permission Requirements

API Action	Permission	Purpose
cce:cluster:get	Get cluster	View cluster details
cce:cluster:createCert	Create certificate	Obtain CCE cluster kubeconfig
ucs:kubeconfig:create	Create kubeconfig	Obtain UCS cluster kubeconfig
ucs:federationKubeconfig:get	Get federation	Download UCS federation kubeconfig

Two-Layer Permission Model:

1. Huawei Cloud IAM: Controls kubeconfig acquisition via hcloud CLI (who can get a kubeconfig for CCE or UCS fleet)
2. Kubernetes RBAC: Controls kubectl operations after kubeconfig is obtained (what the user can do in the cluster or fleet)

Permission Failure Handling:

1. When hcloud commands fail due to IAM permission errors, verify the IAM permissions listed above
2. When kubectl commands fail due to RBAC permission errors, the cluster administrator must configure appropriate RBAC roles
3. Guide the user to create custom policies in the IAM console for Huawei Cloud permissions
4. Guide the user to create ClusterRole/Role bindings for Kubernetes RBAC permissions
5. Pause execution and wait for user confirmation that permissions have been granted

Core Commands

1. Kubeconfig Acquisition

See Task: Kubeconfig Acquisition for detailed workflows.

bash

# CCE cluster kubeconfig (duration in days, 1-1827)
hcloud CCE CreateKubernetesClusterCert --cluster_id=<cce-cluster-id> --duration=1 --cli-region=cn-north-4

# UCS federation kubeconfig (duration in days, 1-1825)
# Used for multi-cluster fleet operations via UCS
hcloud UCS DownloadFederationKubeconfig --clustergroupid=<fleet-id> --duration=1 --cli-region=cn-north-4

Save kubeconfig and verify:

bash

# Save CCE kubeconfig to file
hcloud CCE CreateKubernetesClusterCert --cluster_id=<cce-cluster-id> --duration=1 --cli-region=cn-north-4 > ~/.kube/cce-kubeconfig.yaml

# Verify connection
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml cluster-info

Key Parameter Differences:

• CCE uses --cluster_id (with underscore) for cluster ID
• UCS uses --clusterid (no underscore) for cluster ID
• CCE CreateKubernetesClusterCert --duration is in days (1-1827)
• UCS DownloadFederationKubeconfig --duration is in days (1-1825)

2. Workload Management (Deployment/StatefulSet/DaemonSet)

See Task: Deployment Management and Task: StatefulSet/DaemonSet Management for detailed workflows.

All commands use kubectl --kubeconfig=<kubeconfig-file> -n <namespace> pattern.

bash

# Create Deployment from YAML file
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml apply -f deployment.yaml -n production

# Create Deployment inline
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml create deployment my-app --image=myapp:v1 --replicas=3 -n production

# Query Deployment status
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get deployments -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml describe deployment my-app -n production

# Scale Deployment
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml scale deployment my-app --replicas=5 -n production

# Update Deployment (rolling update)
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml set image deployment/my-app my-app=myapp:v2 -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml rollout status deployment/my-app -n production

# Rollback Deployment
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml rollout undo deployment/my-app -n production

# Delete Deployment
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml delete deployment my-app -n production

# StatefulSet operations (same pattern)
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml apply -f statefulset.yaml -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml scale statefulset my-db --replicas=3 -n production

# DaemonSet operations (same pattern)
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml apply -f daemonset.yaml -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml rollout status daemonset/log-agent -n production

3. Job/CronJob Management

bash

# Create Job from YAML
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml apply -f job.yaml -n production

# Create Job inline
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml create job data-migration --image=migrator:v1 -n production

# Query Job status
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get jobs -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml describe job data-migration -n production

# Delete Job
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml delete job data-migration -n production

# Create CronJob from YAML
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml apply -f cronjob.yaml -n production

# Create CronJob inline
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml create cronjob nightly-backup --image=backup:v1 --schedule="0 2 * * *" -n production

# Suspend CronJob
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml patch cronjob nightly-backup --type merge --patch-file=suspend.json -n production

# Resume CronJob
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml patch cronjob nightly-backup --type merge --patch-file=resume.json -n production

# Delete CronJob
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml delete cronjob nightly-backup -n production

4. Observability + Config

bash

# Pod status
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get pods -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get pods -o wide -n production

# Pod logs
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml logs my-app-pod -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml logs my-app-pod --tail=100 -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml logs my-app-pod -f -n production

# Pod events
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml describe pod my-app-pod -n production

# Namespace management
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get namespaces
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml create namespace staging
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml delete namespace staging

# Service
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml apply -f service.yaml -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get services -n production

# Ingress
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml apply -f ingress.yaml -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get ingress -n production

# ConfigMap
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml create configmap app-config --from-literal=key1=value1 -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get configmaps -n production

# Secret
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml create secret generic db-secret --from-literal=password=s3cret -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get secrets -n production

# PVC
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml apply -f pvc.yaml -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get pvc -n production

# HPA autoscaling
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml autoscale deployment my-app --min=2 --max=10 --cpu=80% -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get hpa -n production

5. kubectl Installation & Configuration

See Task: kubectl Setup for detailed installation and configuration guidance.

Parameter Reference

hcloud Parameters (Kubeconfig Acquisition)

Parameter	Command	Required	Description	Constraints
--cluster_id	CCE CreateCert	Yes	CCE cluster ID	Must reference existing CCE cluster
--clustergroupid	UCS DownloadFederation	Yes	Fleet group ID	Must reference existing fleet group
--duration	CCE CreateCert	Yes*	Certificate validity in days	1-1827 days; at least one of --duration/--expire_at required
--expire_at	CCE CreateCert	Yes*	Certificate expiry timestamp	ISO format; mutually exclusive with --duration
--duration	UCS DownloadFederation	Yes	Token validity in days (1-1825)	Integer (1-1825)
--project_id	CCE CreateCert	Required (auto-filled)	Project ID	Auto-filled from hcloud config if not specified
--cli-region	All hcloud commands	Required	Huawei Cloud region ID	Config value or HUAWEI_CLOUD_REGION

kubectl Flags

Flag	Description	Example
--kubeconfig	Path to kubeconfig file	--kubeconfig=~/.kube/cce-kubeconfig.yaml
-n	Namespace for the operation	-n production
-o	Output format (wide/yaml/json)	-o wide, -o yaml, -o json
-f	YAML/JSON file path	-f deployment.yaml
--replicas	Number of replicas for scaling	--replicas=5
--image	Container image for create/set	--image=myapp:v1
--tail	Number of log lines to show	--tail=100
-f (logs)	Follow log output (stream)	logs -f
--min/--max	HPA min/max replicas	--min=2 --max=10
--cpu-percent	HPA CPU target percentage	--cpu-percent=80

Output Format

CCE CreateKubernetesClusterCert (Kubeconfig YAML)

Returns a standard Kubernetes kubeconfig YAML document containing:

yaml

apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority-data: <base64-encoded-ca>
    server: https://<cluster-endpoint>:5443
  name: internalCluster
- cluster:
    server: https://<cluster-eip>:5443
    insecure-skip-tls-verify: true
  name: externalCluster
- cluster:
    certificate-authority-data: <base64-encoded-ca>
    server: https://<cluster-eip>:5443
  name: externalClusterTLSVerify
contexts:
- context:
    cluster: internalCluster
    user: user
  name: internal
- context:
    cluster: externalCluster
    user: user
  name: external
- context:
    cluster: externalClusterTLSVerify
    user: user
  name: externalTLSVerify
current-context: external

UCS DownloadFederationKubeconfig (Federation Kubeconfig YAML)

Returns a federation kubeconfig with two contexts for multi-cluster fleet operations:

yaml

apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority-data: <base64-encoded-ca>
    server: https://<fleet-name>.fleet.ucs.<region>.myhuaweicloud.com:5443
  name: federation
- cluster:
    certificate-authority-data: <base64-encoded-ca>
    server: https://<fleet-name>.fleet.ucs.<region>.myhuaweicloud.com:5443/apis/cluster.karmada.io/v1alpha1/clusters/*/proxy
  name: karmada-aggregated-apiserver
contexts:
- context:
    cluster: federation
    user: user
  name: federation
- context:
    cluster: karmada-aggregated-apiserver
    user: user
  name: karmada-aggregated-apiserver
current-context: federation

Federation vs Karmada context:

• federation: Operates on fleet-level resources (propagated workloads, fleet policies)
• karmada-aggregated-apiserver: Proxy access to individual member clusters via /clusters/<cluster-name>/proxy

Network prerequisite: UCS federation kubeconfig uses <fleet-name>.fleet.ucs.<region>.myhuaweicloud.com as the API server domain. This domain resolves via UCS VPC Endpoint (VPCEP). If DNS resolution fails, ensure your network can reach the UCS VPCEP (e.g., via VPC peering, VPN, or direct cloud network access).

text

### kubectl Output Formats

| `-o` flag        | Description                        | Use Case                                    |
| ---------------- | ---------------------------------- | ------------------------------------------- |
| (default)        | Tabular human-readable output      | Quick status check                          |
| `-o wide`        | Extended tabular with extra columns | Detailed Pod/IP info                       |
| `-o yaml`        | YAML format                        | Export/edit resource specs                  |
| `-o json`        | JSON format                        | Scripting/automation                        |
| `-o name`        | Resource name only                 | Quick list of names                         |

## Common Region IDs

| Region Name                    | Region ID        |
| ------------------------------ | ---------------- |
| North China - Beijing 4        | `cn-north-4`     |
| North China - Beijing 1        | `cn-north-1`     |
| East China - Shanghai 1        | `cn-east-3`      |
| East China - Shanghai 2        | `cn-east-2`      |
| South China - Guangzhou        | `cn-south-1`     |
| South China - Shenzhen         | `cn-south-4`     |
| Southwest China - Guiyang 1    | `cn-southwest-2` |
| Asia Pacific - Bangkok         | `ap-southeast-2` |
| Asia Pacific - Singapore       | `ap-southeast-1` |
| Asia Pacific - Hong Kong       | `ap-southeast-3` |
| Europe - Paris                 | `eu-west-0`      |

## Verification

### Kubeconfig Acquisition Verification

```bash
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml cluster-info
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get nodes

Workload Deployment Verification

bash

kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get deployments -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml rollout status deployment/my-app -n production
kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get pods -o wide -n production

HPA Verification

bash

kubectl --kubeconfig=~/.kube/cce-kubeconfig.yaml get hpa -n production

Best Practices

1. Namespace-First: Always specify -n <namespace> explicitly; never rely on default namespace
2. Kubeconfig Security: Store kubeconfig files with restricted file permissions (chmod 600); never expose them in public repositories or CI logs
3. Short-Duration Certificates: Use minimum --duration needed for CCE kubeconfig; prefer 1 day for interactive sessions
4. RBAC Alignment: Ensure Huawei Cloud IAM permissions and Kubernetes RBAC roles are aligned; IAM grants kubeconfig access, RBAC grants kubectl operations
5. YAML-Based Deployment: Prefer kubectl apply -f over inline kubectl create for reproducibility and auditability
6. Rollout Monitoring: Always check rollout status after updates; use rollout undo for emergency rollback
7. Namespace Isolation: Use separate namespaces for production, staging, and development workloads
8. HPA Baseline: Set HPA --min to your steady-state replica count; set --max based on resource budget

Reference Documents

Document	Description
Task: Kubeconfig Acquisition	Kubeconfig acquisition workflows
Task: kubectl Setup	kubectl installation and configuration
Task: Deployment Management	Deployment lifecycle workflows
Task: StatefulSet/DaemonSet Management	StatefulSet and DaemonSet workflows

Notes

• kubectl is the primary tool — all workload operations use kubectl --kubeconfig=<file> after kubeconfig acquisition via hcloud
• kubeconfig is a secret — treat it like a credential; never share or expose publicly
• RBAC governs kubectl access — even with a valid kubeconfig, Kubernetes RBAC controls what operations are permitted
• cce-cluster-management handles infrastructure — cluster creation, deletion, and node management belong to the cce-cluster-management skill
• Two-layer permission model — Huawei Cloud IAM controls kubeconfig acquisition, Kubernetes RBAC controls kubectl operations
• UCS is for fleet operations — UCS kubeconfig is only for federation (DownloadFederationKubeconfig), not single cluster (CreateClusterKubeconfig is out of scope for this skill)
• UCS federation kubeconfig provides two contexts — federation (fleet-level) and karmada-aggregated-apiserver (proxy to member clusters)
• UCS federation requires network access — fleet API server domain (<fleet>.fleet.ucs.<region>.myhuaweicloud.com) requires VPCEP access

Common Pitfalls

Pitfall	Symptom	Quick Fix
kubectl not installed	Command not found	Install kubectl (see references/task-kubectl-setup.md)
Wrong cluster_id	404 or kubeconfig for wrong cluster	Verify cluster ID with hcloud CCE ListClusters
Kubeconfig expired	Authentication failures	Re-acquire kubeconfig with CreateKubernetesClusterCert
RBAC insufficient	Forbidden errors in kubectl	Configure appropriate ClusterRole/Role bindings
Missing namespace flag	Resources in wrong namespace	Always specify -n <namespace> explicitly
CCE vs UCS param confusion	Parameter not recognized	CCE: --cluster_id, UCS: --clustergroupid for federation
Duration unit confusion	Certificate expires immediately	CCE: days (1-1827), UCS federation: days (1-1825)
UCS federation DNS unreachable	no such host on federation API	Ensure VPCEP/network access to UCS fleet domain
Inline create vs YAML apply	Hard to reproduce/audit	Prefer kubectl apply -f <yaml> for production
Rollout without status check	Unknown deployment state	Always run rollout status after updates
Kubeconfig file permissions	Security warning or access denied	Set file permissions to 600 (chmod 600)
Wrong StorageClass in PVC	PVC stuck Pending	Use csi-disk (not cce-standard); run kubectl get sc to verify
Metrics API not available	top pods/nodes fails	Install metrics-server addon via CCE console
PowerShell JSON patch escaping	patch -p fails with JSON errors	Use --patch-file instead of inline -p JSON
HPA --cpu-percent deprecated	Warning flag deprecated	Use --cpu=80% instead of --cpu-percent=80