How is the lifecycle of Pods managed in Kubernetes?

30 Haz 05:37

Last year, while managing a customer's production Kubernetes cluster, I encountered a situation where a deployment's Pods were constantly stuck in a **"CrashLoopBackOff"** state. After each restart, I checked the pod logs and realized that the application's startup time was clashing with Kubernetes' **liveness probe** — the probe was marking the app as "failed" before it had fully initialized. The issue was that the **initialDelaySeconds** was set to 30 seconds, but the app took 45 seconds to be ready. A simple tweak to `initialDelaySeconds: 45` and `timeoutSeconds: 5` fixed the problem, and the pods stabilized. Another issue was **resource constraints** preventing the self-healing mechanism from working properly. When CPU throttling occurred on the nodes, pod restarts actually worsened the situation. The solution here was to properly configure CPU/memory limits at the namespace level using **LimitRange** and **ResourceQuota** objects. Once we balanced the `requests` and `limits`, Kubernetes' **Horizontal Pod Autoscaler (HPA)** started working correctly, scaling pods as needed. The most critical lesson, though, was ensuring **readiness and liveness probes are defined correctly**. I almost always use a **TCP readiness** and **HTTP liveness** combo — restart the pod if the app isn't responding at all, but avoid sending traffic if only specific endpoints aren't ready. This makes self-healing much smarter. One of the biggest improvements I’ve seen comes from monitoring **Kubernetes audit logs** to understand how probe settings should evolve over time. During high-traffic periods, we even implemented a **feedback system using a sidecar** to dynamically adjust probe timings — something I’d call a learned "hack" from experience.