k8s-aiops

Use this skill whenever the user needs to operate a Kubernetes cluster — list/inspect pods, deployments, statefulsets, daemonsets, replicasets, jobs, cronjobs, services, ingresses, endpoints, configmaps, secrets (names/keys only), PVCs/PVs/storageclasses, nodes, namespaces, and events; read pod logs; describe pods/nodes; pod/node top (metrics); read-only diagnostics / RCA (pod-health, workload-readiness); scale deployments/statefulsets; rollout status/history/undo/pause/resume and set image; delete pods/deployments/jobs; create/delete namespaces; and cordon/uncordon/drain nodes. Works with any kubeconfig-reachable cluster (standard Kubernetes, k3s, EKS, GKE, AKS). Always use this skill for "list k8s pods", "scale deployment", "kubernetes pod logs", "describe pod", "why is my pod crashing", "diagnose pods", "which deployments are unhealthy", "rollout undo", "set image", "top pods", "drain node", "cordon node", "restart deployment", "k3s", or "kubectl"-style tasks when the context is explicitly Kubernetes / a cluster. Do NOT use when the target is not a Kubernetes cluster (hypervisor VM lifecycle, backup products, or cloud-provider consoles are out of scope). Common Kubernetes operations with a built-in governance harness (audit, token budget, undo, risk-tier labels).

Disclaimer: This is a community-maintained open-source project and is not affiliated with, endorsed by, or sponsored by the Cloud Native Computing Foundation, the Kubernetes project, or k3s/Rancher. "Kubernetes" and "k3s" are trademarks of their respective owners. Source code is publicly auditable at github.com/AIops-tools/K8s-AIops under the MIT license.

Governed Kubernetes operations — 55 MCP tools, every one wrapped with the bundled @governed_tool harness: a local unified audit log under ~/.k8s-aiops/, a token/runaway budget guard, undo-token recording, and a descriptive risk-tier label on every audit row. Works with any kubeconfig-reachable cluster (standard Kubernetes, k3s, EKS, GKE, AKS). Run k8s-aiops init for a friendly onboarding wizard that registers your kube contexts as named targets.

Standalone: the governance harness is bundled in the package (k8s_aiops.governance) — k8s-aiops has no external skill-family dependency. Coverage focuses on common operations and is not yet exhaustive.

What This Skill Does

Category	Tools	Count	Read or Write
Pods	list, get, logs, describe, delete	5	4 read / 1 write
Deployments	list, get, scale, rollout restart, delete	5	2 read / 3 write
Rollout	status, history, undo, pause, resume, set-image	6	2 read / 4 write
StatefulSets	list, get, scale	3	2 read / 1 write
DaemonSets	list, get	2	2 read
ReplicaSets	list	1	1 read
Jobs / CronJobs	job list/get/delete, cronjob list/get	5	4 read / 1 write
Services / Ingress / Endpoints	service list, ingress list/get, endpoints list	4	4 read
Config / Secrets	configmap list/get, secret list (names/keys only)	3	3 read
Storage	pvc list/get, pv list, storageclass list	4	4 read
Nodes	list, describe, cordon, uncordon, drain	5	2 read / 3 write
Namespaces	list, create, delete	3	1 read / 2 write
Metrics (top)	pod, node	2	2 read
Cluster	cluster_info, api_resources	2	2 read
Events	list	1	1 read
Diagnostics / RCA	pod-health, workload-readiness	2	2 read

Quick Install

bash

uv tool install k8s-aiops
k8s-aiops init            # friendly wizard: register your kube contexts as targets
k8s-aiops doctor          # or skip init — works with your current kube-context too

When to Use This Skill

• List/inspect pods, deployments, services, nodes, namespaces and recent events
• Read a pod's recent log lines to diagnose a crash loop
• Run a read-only RCA sweep (diagnose pod-health / diagnose workload-readiness) to find the root cause worst-first
• Scale a deployment up/down, or trigger a rolling restart
• Delete a stuck pod (a controller recreates it) or a deployment
• Cordon a node before maintenance, then uncordon it after

Do NOT use when the target is not a Kubernetes cluster (hypervisor VM lifecycle, backup products, or cloud-provider consoles are out of scope for this skill).

Related Skills — Skill Routing

If the user wants…	Use
Kubernetes pods / deployments / nodes	k8s-aiops (this skill)
Hypervisor VM lifecycle (power, snapshot, migrate)	a hypervisor ops skill
Backup & restore	a backup ops skill

Common Workflows

Diagnose a crash-looping pod and restart its deployment

1. k8s-aiops pod list -n prod → find the pod with high restarts / non-Running phase
2. k8s-aiops pod logs <pod> -n prod --tail 200 → read the recent logs for the crash cause
3. k8s-aiops events -n prod → check for FailedScheduling / image-pull events
4. k8s-aiops deployment restart <deploy> -n prod → roll the deployment after fixing the cause
5. Failure branch: if logs/events show an RBAC 403, the kube context lacks the verb — run kubectl auth can-i get pods -n prod and switch to a context with adequate RBAC; the skill never retries a denied auth.

Triage an unhealthy namespace with RCA, then act on the worst finding

1. k8s-aiops diagnose pod-health -n prod → worst-first findings; a critical CrashLoopBackOff on prod/api cites restarts=9 and the exact kubectl logs … --previous action
2. k8s-aiops diagnose workload-readiness -n prod → confirm the blast radius: e.g. Deployment web ready 0/3 (critical, under-replicated)
3. k8s-aiops pod logs api-<hash> -n prod --tail 200 --previous-equivalent via k8s-aiops pod describe api-<hash> -n prod → read the crash cause the RCA pointed you at
4. k8s-aiops deployment restart web -n prod → roll the deployment once the root cause is fixed
5. Failure branch: if diagnose returns an RBAC 403, the kube context cannot list pods/deployments in that namespace — run kubectl auth can-i list pods -n prod and switch to a context with adequate RBAC; the RCA tools are read-only and never retry a denied auth.

Drain a node for maintenance, safely reversible

1. k8s-aiops node list → identify the node and confirm it is Ready/schedulable
2. k8s-aiops node cordon <node> --dry-run → preview, then k8s-aiops node cordon <node> (double confirm) — records an inverse uncordon_node undo descriptor
3. After maintenance: k8s-aiops node uncordon <node> → re-enable scheduling
4. Failure branch: if doctor shows the cluster unreachable, fix the kubeconfig context (kubectl config get-contexts) before retrying — cordon is never issued against an unauthenticated session.

Usage Mode

Scenario	Recommended	Why
Local/small models	CLI	fewer tokens than MCP
Cloud models (Claude, GPT)	Either	MCP gives structured JSON I/O
Automated pipelines	MCP	type-safe parameters, audited

MCP environment caveat: MCP clients spawn the server with a CLEAN environment — shell exports may not reach it. Set K8S_AIOPS_HOME, K8S_AUDIT_APPROVED_BY, K8S_AUDIT_RATIONALE (and KUBECONFIG when the kubeconfig is not at ~/.kube/config) in the MCP server config's env block, not just in your terminal.

MCP Tools (55 — 39 read, 16 write)

Category	Tools	R/W
Pods	pod_list, pod_get, pod_logs, pod_describe	Read
	delete_pod	Write
Deployments	deployment_list, deployment_get	Read
	scale_deployment, rollout_restart_deployment, delete_deployment	Write
Rollout	rollout_status, rollout_history	Read
	rollout_undo_deployment, rollout_pause, rollout_resume, set_deployment_image	Write
StatefulSets	statefulset_list, statefulset_get	Read
	scale_statefulset	Write
DaemonSets / ReplicaSets	daemonset_list, daemonset_get, replicaset_list	Read
Jobs / CronJobs	job_list, job_get, cronjob_list, cronjob_get	Read
	delete_job	Write
Services / Ingress	service_list, ingress_list, ingress_get, endpoints_list	Read
Config / Secrets	configmap_list, configmap_get, secret_list (names/keys only)	Read
Storage	pvc_list, pvc_get, pv_list, storageclass_list	Read
Nodes	node_list, node_describe	Read
	cordon_node, uncordon_node, drain_node	Write
Namespaces	namespace_list	Read
	create_namespace, delete_namespace	Write
Metrics (top)	node_top, pod_top	Read
Cluster	cluster_info, api_resources	Read
Events	event_list	Read
Diagnostics / RCA	pod_health_rca, workload_readiness_rca	Read
Undo	undo_list	Read
	undo_apply	Write

Security — secrets: secret_list returns secret names, types, and key NAMES only. Secret VALUES are never read, returned, or logged, and there is deliberately no tool that returns secret values.

Dry-run previews: every write tool takes dry_run: bool = False. A dry run returns a {"dryRun": true, "wouldX": ...} preview without touching the cluster, and no undo descriptor is recorded for a preview.

Harness features that light up: write tools with a clean inverse pass an undo= lambda so the harness records an inverse descriptor (with _undo_id) to the undo store — scale_deployment/scale_statefulset record a scale-back to their returned previous_replicas, set_deployment_image records a restore to the captured previous_image, cordon_node ↔ uncordon_node and rollout_pause ↔ rollout_resume are mutual inverses, and create_namespace records a delete_namespace. drain_node records a partial uncordon_node inverse (cordon is reversible; evictions are not). delete_* and rollout_undo_deployment declare no undo. risk_level=high: delete_deployment, delete_job, delete_namespace, drain_node, rollout_undo_deployment. undo_list (read) lists recorded reversible writes whose undo tokens have not been applied yet, and undo_apply (write) executes a recorded inverse — itself governed, single-use, and supports dry_run. All 55 tools are audit-logged under ~/.k8s-aiops/ and pass through the budget/runaway guard, each recorded with a descriptive risk-tier label. pod_top/node_top return a clear "metrics-server not installed" message (not an error) when metrics-server is absent. Avoid tight poll loops (re-listing pods every second) — the runaway breaker backs this up.

CLI Quick Reference

bash

k8s-aiops init                                            # interactive onboarding wizard
k8s-aiops pod list [-n <ns>] [-t <target>]
k8s-aiops pod get <name> [-n <ns>]
k8s-aiops pod describe <name> [-n <ns>]                   # status, container states, events
k8s-aiops pod logs <name> [-n <ns>] [--tail 200] [-c <container>]
k8s-aiops pod delete <name> [-n <ns>] [--dry-run]        # double confirm
k8s-aiops deployment list|get|scale|restart|delete ...    # scale/restart: single confirm + --dry-run; delete: double confirm
k8s-aiops rollout status|history|pause|resume <name> [-n <ns>]
k8s-aiops rollout set-image <name> <container> <image> [-n <ns>]
k8s-aiops rollout undo <name> [--to-revision N] [--dry-run]   # double confirm
k8s-aiops statefulset list|get|scale ...
k8s-aiops daemonset list|get ...
k8s-aiops job list|get|delete ...                         # delete: double confirm
k8s-aiops cronjob list|get ...
k8s-aiops service list [-n <ns>]
k8s-aiops ingress list|get [-n <ns>]
k8s-aiops configmap list|get [-n <ns>]
k8s-aiops secret list [-n <ns>]                           # names/keys only — never values
k8s-aiops storage pvc-list|pvc-get|pv-list|class-list
k8s-aiops top pod|node                                    # requires metrics-server
k8s-aiops node list|describe
k8s-aiops node cordon|drain <name> [--dry-run]           # double confirm
k8s-aiops node uncordon <name>
k8s-aiops namespace list|create
k8s-aiops namespace delete <name> [--dry-run]            # double confirm
k8s-aiops cluster-info
k8s-aiops api-resources
k8s-aiops events [-n <ns>]
k8s-aiops diagnose pod-health [-n <ns>] [-l <selector>]         # read-only RCA: crashloop/imagepull/OOM/unschedulable/restarts
k8s-aiops diagnose workload-readiness [-n <ns>]                 # read-only RCA: ready<desired / stuck rollouts
k8s-aiops doctor
k8s-aiops mcp                                             # start MCP server (stdio)

See references/cli-reference.md for the full command list.

Troubleshooting

"Could not load kubeconfig … context not found"

The named context does not exist in your kubeconfig. Run kubectl config get-contexts and set the target's context: to a listed name (or omit it to use current-context).

"Authentication/authorization failed (401/403)"

The kube context lacks the RBAC verb for the resource. Check with kubectl auth can-i <verb> <resource> -n <ns> and switch to a context/ServiceAccount with adequate roles. For EKS/GKE/AKS, confirm the exec-plugin (aws/gcloud/az CLI) is installed and logged in.

"Resource not found (404)"

The pod/deployment/node name or namespace is wrong, or the object was deleted. List the parent collection first (pod list, deployment list, node list) to get a current name. Remember most commands default to the default namespace unless -n is given.

"Conflict (409)"

The object changed concurrently (or already exists). Re-read it and retry the write.

Logs are empty or truncated

pod logs returns the trailing --tail lines (default 100); raise --tail. For a multi-container pod, pass -c <container> or the API returns an error naming the available containers.

Audit & Safety

All operations are automatically audited via the bundled @governed_tool decorator (k8s_aiops.governance):

• Every tool call logged to ~/.k8s-aiops/audit.db (local SQLite audit DB; relocate with K8S_AIOPS_HOME)
• Budget / runaway guard caps cumulative tool calls and wall-time, and trips on tight poll/retry loops — a safety backstop, not authorization
• Undo store records inverse descriptors for reversible writes (scale → previous replicas; cordon ↔ uncordon)
• Each write carries a descriptive risk-tier label into its audit row — a label, not a gate; K8S_AUDIT_APPROVED_BY / K8S_AUDIT_RATIONALE are optional annotations recorded when set, never required

Authorization is not this tool's job. There is no read-only switch, policy file, or approval gate. Whether a write is permitted is the agent's judgement or the RBAC of the kubeconfig context you connect with — give it a read-only ServiceAccount and writes fail at the apiserver, the place that owns the permission.

The harness is bundled in the package — no external dependency, no manual setup. See references/setup-guide.md for security details.

Driving these tools with a smaller / local model? See references/agent-guardrails.md — which guardrails the tool now enforces for you, plus a ready-to-paste system prompt.

Contributing & feature requests

Coverage is intentionally focused. Missing a device, action, or feature you need? Open an issue or pull request at github.com/AIops-tools/K8s-AIops — feature requests, contributions, and comments are all welcome.

License

MIT — github.com/AIops-tools/K8s-AIops