https://www.iceyao.com.cn/tags/%E6%95%85%E9%9A%9C%E6%8E%92%E6%9F%A5/ Recent content in 故障排查 on 爱折腾的工程师 Hugo en-us Thu, 10 Sep 2020 00:00:00 +0000 https://www.iceyao.com.cn/post/2020-09-10-tke-platform-controller_health_check/ Thu, 10 Sep 2020 00:00:00 +0000 https://www.iceyao.com.cn/post/2020-09-10-tke-platform-controller_health_check/ 环境 tke版本: v0.12.2 现象 Cluster对象错误日志 集群HealthCheck失败，集群状态为Failed, 且不会再恢复 # kubectl --kubeconfig=/etc/tke/tke-platform-config.yaml get cluster cls-bd46179d -o yaml - lastProbeTime: "2020-09-09T11:40:27Z" lastTransitionTime: https://www.iceyao.com.cn/post/2020-01-04-%E8%AE%B0%E4%B8%80%E6%AC%A1k8s_cgroup%E5%86%85%E5%AD%98%E6%B3%84%E9%9C%B2%E9%97%AE%E9%A2%98%E8%A7%A3%E5%86%B3/ Sat, 04 Jan 2020 00:00:00 +0000 https://www.iceyao.com.cn/post/2020-01-04-%E8%AE%B0%E4%B8%80%E6%AC%A1k8s_cgroup%E5%86%85%E5%AD%98%E6%B3%84%E9%9C%B2%E9%97%AE%E9%A2%98%E8%A7%A3%E5%86%B3/ 环境 arm ubuntu平台 Kubernetes v1.14.6 Etcd 3.3.12 Docker 18.09.9 Kernel 4.4.131 现象 现象如这个issue里面所描述的https://github.com/kubernetes/kub https://www.iceyao.com.cn/post/2019-12-28-%E8%AE%B0%E4%B8%80%E6%AC%A1k8s_apiserver%E8%BF%9E%E6%8E%A5%E6%95%B0%E8%BF%87%E5%A4%9A%E9%97%AE%E9%A2%98%E6%8E%92%E6%9F%A5/ Sat, 28 Dec 2019 00:00:00 +0000 https://www.iceyao.com.cn/post/2019-12-28-%E8%AE%B0%E4%B8%80%E6%AC%A1k8s_apiserver%E8%BF%9E%E6%8E%A5%E6%95%B0%E8%BF%87%E5%A4%9A%E9%97%AE%E9%A2%98%E6%8E%92%E6%9F%A5/ 环境 Kubernetes v1.14.6 Etcd 3.3.12 Docker 18.09.9 现象 查看不同状态的连接数数量 # netstat -ant | awk '/^tcp/ {++y[$NF]} END {for(w in y) print w, y[w]}' LISTEN 19 ESTABLISHED 1252 TIME_WAIT 17 发现有大量的ESTABLISHED连接 查看每个ip跟se