<p>I have a k8s ingress with more than 5 backend services behind it. The ingress spawns a GoogleCloud LoadBalancer.</p>
<p>Each of the services is routed traffic to by an http <code>path</code> rule. Eg. one app is on <code>/foo</code>, another is on <code>/bar</code>, etc.
All of them work fine. Then I added a new app, with backend service and routing rule, all the same way as the others.</p>
<p>But I'm constantly getting this error when I hit the URL of the new app:</p>
<pre><code>Error: Server Error
The server encountered a temporary error and could not complete your request.
Please try again in 30 seconds.
</code></pre>
<p>When I open the ingress in GCP console, I can see this warning: <a href="https://i.stack.imgur.com/ZyA7I.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ZyA7I.png" alt="enter image description here" /></a></p>
<p>and the unhealthy service is the one from my newly added app.</p>
<p><strong>The weird thing is that the app actually does get traffic when I hit the URL. I can see it in the logs. But I still get that 502 error and the backend service is shown as unhealthy.</strong></p>
<p>I am not rly sure how to debug this in order to figure what's the issue.</p>

<p>So, the problem was that the LB health-check was hitting <code>/</code> which was a non-existing endpoint on the app (aka. it wasn't returning <code>OK 200</code>).</p>
<p>I added <code>readiness</code> probe to the k8s Deployment. According to the <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/ingress#health_checks" rel="noreferrer">GCP Ingress docs</a>, if there is a <code>readiness</code> probe, the ingress will pick it up and use it as the LB health-check.</p>
<p>I also had to manually update the path to which the health-check object of the backend service was hitting. I guess the pod with readiness probe should exist before the ingress is set up, otherwise it doesn't update the health-check object automatically.</p>
