{"skill":{"slug":"ops-cert-check","displayName":"Ops Cert Check","summary":"SSL/TLS Certificate Check & Renewal SOP. Covers certificate validation (PEM/CRT/JKS), Nginx certificate update, Let's Encrypt wildcard application, and emerg...","description":"---\nname: ops-cert-check\ndescription: SSL/TLS Certificate Check & Renewal SOP. Covers certificate validation (PEM/CRT/JKS), Nginx certificate update, Let's Encrypt wildcard application, and emergency response for expired certificates.\ntriggers:\n  - \"ssl certificate\"\n  - \"tls certificate\"\n  - \"certificate expired\"\n  - \"https certificate\"\n  - \"jks certificate\"\n  - \"lets encrypt\"\n  - \"wildcard certificate\"\n  - \"certbot\"\n  - \"nginx ssl\"\n  - \"certificate renewal\"\ncategory: ops\ntags: [ssl, tls, certificate, nginx, certbot, lets-encrypt, security]\nversion: 1.0.0\ncreated: 2026-05-06\n---\n\n# SSL/TLS Certificate Check & Renewal SOP\n\n## Scenario 1: Validate Existing Certificate\n\n### Method A: Linux OpenSSL (Recommended)\n```bash\n# Check PEM/CRT certificate\nopenssl x509 -in certificate.crt -noout -dates\n\n# Check JKS certificate\nkeytool -list -v -keystore keystore.jks -storepass <password>\n```\n\n### Method B: Online Verification\nOpen the certificate file directly in browser to view validity period and issuer.\n\n### Method C: Remote Check via Request\n```bash\n# Check Nginx certificate (port 443)\necho \"\" | openssl s_client -connect domain:443 -servername domain 2>/dev/null | openssl x509 -noout -dates\n\n# Check game server JKS certificate (port +3 offset)\necho \"\" | openssl s_client -connect game.domain:port+3 2>/dev/null | openssl x509 -noout -dates\n```\n\n---\n\n## Scenario 2: Nginx Certificate Update (Standard Flow)\n\nApplicable to: Web services, reverse proxies, CDN frontends, and any Nginx-hosted SSL termination.\n\n### Standard Steps\n\n**1. Replace certificate files**\nLogin to target server, replace files in `/etc/nginx/ssl/`:\n- Certificate file (`.crt`)\n- Certificate key file (`.key`)\n\n**2. Validate configuration**\n```bash\nnginx -t\n```\n\n**3. Reload Nginx**\n```bash\nnginx -s reload\n```\n\n**4. Verify the update**\nSend HTTP/HTTPS request to confirm the new certificate is being served.\n\n**5. Update certificate monitoring records**\nLogin to ops backend → Asset Management → Domain Assets → Domain Monitoring → Add record → Upload new certificate.\n\n**6. Commit to version control**\n> ⚠️ **禁止将 `.key` 私钥文件提交到版本库**。私钥应保留在服务器安全路径（如 `/etc/nginx/ssl/`）或密钥管理系统中。仅提交证书文件（`.crt`）或元数据。\n\nSubmit updated certificates to the version control repository.\n\n### Role Responsibilities\n\n| Role | Scope |\n|------|-------|\n| Ops/Infra | Replace Nginx certificates, validate, reload, update monitoring |\n| Third-party vendor | Cloud services, load balancers, middleware certificates |\n\n---\n\n## Scenario 3: Game Server JKS Certificate Update\n\nApplicable to: Java-based game servers that use JKS (Java KeyStore) format certificates.\n\n### Flow\n\n```\nOps provides certificate and private key\n         ↓\nDev team compiles into Java certificate (JKS)\n         ↓\nOps uploads JKS to ops server\n         ↓\nGame server auto-replaces on next update\n         ↓\nVerify: request game server domain on port+3\n```\n\n### JKS Certificate Verification\n```bash\nkeytool -list -v -keystore game.jks -storepass <password>\n```\n\n---\n\n## Scenario 4: Let's Encrypt Wildcard Certificate\n\nApplicable to: Self-managed services using Let's Encrypt certificates (e.g., internal tools, dev environments, small-scale deployments).\n\n### Prerequisites\n- Linux server with internet access\n- Root/sudo access\n- Domain DNS already pointing to the server\n- DNS provider with API access (Alibaba Cloud DNS, Cloudflare, Route53, etc.)\n\n### DNS-01 Challenge for Wildcard\n```bash\n# Install certbot with DNS plugin (Alibaba Cloud example)\nyum install -y certbot-dns-aliyun\n\n# Configure Alibaba Cloud CLI authentication\n# Reference: https://help.aliyun.com/zh/cli/configure-credentials\n\n# Apply for wildcard certificate\ncertbot certonly \\\n  --manual \\\n  --preferred-challenges dns \\\n  --dns-aliyun \\\n  -d \"*.example.com\" \\\n  -d \"example.com\"\n```\n\nFor other DNS providers, use the corresponding plugin:\n- Cloudflare: `certbot-dns-cloudflare`\n- Route53: `certbot-dns-route53`\n- Generic: `--manual --preferred-challenges dns`\n\n### Install Certificate on Nginx\n```bash\n# Switch to cert directory\ncd /etc/letsencrypt\n\n# Generate DH parameters (enhanced security)\nopenssl dhparam -out dhparam.pem 2048\n\n# Add SSL configuration\nvim /etc/letsencrypt/options-ssl-nginx.conf\n\n# Configure Nginx\n# ... add ssl_certificate, ssl_certificate_key, etc. ...\n\n# Reload Nginx\nnginx -s reload\n```\n\n### Auto-Renewal\n```bash\n# Set up cron job (runs every 2 days)\ncrontab -e\n# 0 0 */2 * * certbot renew --deploy-hook \"/root/your-hook.sh\"\n\n# Manual dry-run (doesn't actually renew)\ncertbot renew --dry-run\n```\n\n**Note**: Let's Encrypt certificates are valid for 90 days. `certbot renew` only actually renews within 30 days of expiry.\n\n---\n\n## Scenario 5: Expired Certificate Emergency Response\n\n**Emergency**: Certificate expired, business impacted (e.g., payments failing, users locked out).\n\n**Immediate actions**:\n1. Identify which domains are affected: `openssl s_client -connect domain:443`\n2. Contact responsible party (internal team or third-party vendor)\n3. Replace certificate file and reload: `nginx -s reload`\n4. Notify stakeholders of recovery\n\n**Prevention**:\n- Monitoring alerts + notification (Slack/email/PagerDuty, etc.)\n- Cron job with certificate expiry check\n- Alert 30 days before expiry\n\n---\n\n## Certificate Quick Reference\n\n| Item | Type | Renewal | Notes |\n|------|------|---------|-------|\n| Project certificate (purchased) | Commercial CA | Vendor/Third-party | Contact issuer directly |\n| JKS certificate | Java KeyStore | Dev team compiles | Ops uploads to server |\n| Let's Encrypt | ACME | certbot auto-renewal | 90-day validity |\n| Wildcard certificate | DNS-validated | certbot with DNS plugin | Required for `*.domain.com` |\n\n---\n\n## Command Cheatsheet\n\n```bash\n# Check certificate validity\nopenssl x509 -in cert.crt -noout -dates\n\n# Check JKS certificate\nkeytool -list -v -keystore game.jks\n\n# Nginx config test\nnginx -t\n\n# Reload Nginx\nnginx -s reload\n\n# Apply Let's Encrypt (DNS challenge)\ncertbot certonly --manual --preferred-challenges dns -d \"*.domain.com\"\n\n# Auto-renewal dry-run\ncertbot renew --dry-run\n\n# Check remote certificate\necho \"\" | openssl s_client -connect domain:443 -servername domain 2>/dev/null | openssl x509 -noout -dates\n```\n\n---\n\n## Notes\n\n1. **JKS certificate update requires dev team involvement** — ops cannot do this alone\n2. **Third-party vendor projects** typically have the vendor handle certificate renewal\n3. **Let's Encrypt certificates are 90 days** — auto-renewal must be configured\n4. **Game server certificates and Nginx certificates are separate** — game servers use JKS, Nginx uses PEM/CRT\n5. **certbot certificate path**: `/etc/letsencrypt/live/domain/`\n","tags":{"latest":"1.0.1"},"stats":{"comments":0,"downloads":326,"installsAllTime":1,"installsCurrent":1,"stars":0,"versions":2},"createdAt":1778063486764,"updatedAt":1778492859420},"latestVersion":{"version":"1.0.1","createdAt":1778064261840,"changelog":"- Added a warning in the Nginx certificate update section to prohibit committing `.key` private key files to version control repositories; only certificate files (`.crt`) or metadata should be committed.\n- Minor update to the auto-renewal cron example for Let's Encrypt certificates.","license":"MIT-0"},"metadata":null,"owner":{"handle":"freepengyang","userId":"s17awhsb821pah3b7qbwfbwrm584yvyq","displayName":"freepengyang","image":"https://avatars.githubusercontent.com/u/18629902?v=4"},"moderation":null}