<p>I am working on deploying a certain pod to GKE but I am having an unhealthy state for my backend services.</p>
<p>The deployment went through via <code>helm install process</code> but the ingress reports a certain warning error that says <code>Some backend services are in UNHEALTHY state</code>. I have tried to access the logs but do not know exactly what to look out for. Also, I already have liveness and readiness probes running.</p>
<p>What could I do to make the ingress come back to a healthy state? Thanks</p>
<p><a href="https://i.stack.imgur.com/hNcFg.png" rel="nofollow noreferrer">Picture of warning error on GKE UI</a></p>

<p>Without more details it is hard to determine the exact cause.</p>
<p>As first point I want to mention, that your error message is <code>Some backend services are in UNHEALTHY state</code>, <strong>not</strong> <code>All backend services are in UNHEALTHY state</code>. It indicates that only <strong>a few of your backends</strong> are affected.</p>
<p>There might be tons of reasons, if you are using <code>GCP Ingress</code> or <code>Nginx Ingress</code>, your configuration of <code>externalTrafficPolicy</code>, if you are using <code>preemptive nodes</code>, your <code>livenessProbe</code> and <code>readinessProbe</code>, <code>health checks</code>, etc.</p>
<p>In your scenario, only a few backends are affected, the only thing with current information I can suggest you some debug options.</p>
<ul>
<li>Using <code>$ kubectl get po -n &lt;namespace&gt;</code> check if all your pods are working correctly, that all containers within pods are <code>Ready</code> and pod status is <code>Running</code>. Eventually check logs of suspicious pod <code>$ kubectl logs &lt;podname&gt; -c &lt;containerName&gt;</code>. In general you should check all pods the load balancer is pointing to,</li>
<li>Confirm if <code>livenessProbe</code> and <code>readinessProbe</code> are configured properly and response is <code>200</code>,</li>
<li>Describe your ingress <code>$ kubectl describe ingress &lt;yourIngressName&gt;</code> and check <code>backends</code>,</li>
<li>Check if you've configured your <code>health checks</code> properly according to <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/ingress#health_checks" rel="nofollow noreferrer">GKE Ingress for HTTP(S) Load Balancing - Health Checks</a> guide.</li>
</ul>
<p>If you still won't be able to solve this issue with above debug options, please provide more details about your env with logs (without private information).</p>
<p><strong>Useful links</strong>:</p>
<ul>
<li><a href="https://stackoverflow.com/questions/39294305/">kubernetes unhealthy ingress backend</a></li>
<li><a href="https://stackoverflow.com/questions/63268552/gke-ingress-shows-unhealthy-backend-services">GKE Ingress shows unhealthy backend services</a></li>
</ul>
