<p>I managed to create a cluster using GKE using gce ingress successfully. However it takes a long time for Ingress to detect the service is ready (I already set both livenessProbe and readinessProbe).
My pods set up</p>

<pre><code>Containers:
...
  gateway:
    Liveness:   http-get http://:5100/api/v1/gateway/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:5100/api/v1/gateway/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
...
</code></pre>

<p>and ingress</p>

<pre><code>...
Name:             main-ingress
  Host                              Path  Backends
  ----                              ----  --------
  &lt;host&gt;
                                    /api/v1/gateway/    gateway:5100 (&lt;ip:5100&gt;)
                                    /api/v1/gateway/*   gateway:5100 (&lt;ip:5100&gt;)
                                                        web:80 (&lt;ip&gt;)
Annotations:
  ingress.kubernetes.io/backends:               {"k8s-be-***":"HEALTHY","k8s-be-***":"HEALTHY","k8s-be-***":"HEALTHY"}
  kubernetes.io/ingress.allow-http:             false
</code></pre>

<p>What I notice is that if I killed all the service and redeploy, the backend stays at <code>UNHEALTHY</code> for quite some time before it picks it up even though Kubernetes itself managed to pick up that pods/service are all running</p>

<p>I also noticed that when <code>livenessProbe</code> and <code>readinessProbe</code> is set, the Backend health check that's generated by ingress-gce is the following</p>

<pre><code>Backend
Timeout: 30 seconds

Backend Health check
Interval: 70 seconds
Timeout: 1 second
Unhealthy threshold: 10 consecutive failures
Healthy threshold: 1 success
</code></pre>

<p>Whereas if I just deploy a simple nginx pod without specifying <code>livenessProbe</code> and <code>readinessProbe</code>, the backend generated is the following</p>

<pre><code>Backend
Timeout: 30 seconds

Backend Health Check
Interval: 60 seconds
Timeout: 60 seconds
Unhealthy threshold: 10 consecutive failures
Healthy threshold: 1 success
</code></pre>

<p>Is the Backend health check the root cause of the slowness of picking things up? If so, any idea how to speed it up?</p>

<hr>

<p><strong>UPDATE</strong>
Wanted to clarify after reading <a href="https://stackoverflow.com/a/59374808/364569">@yyyyahir's answer below</a></p>

<p>I understand that when creating new ingress it will take much longer because the ingress controller needs to provision the new Load Balancer, backend and all the other related things.</p>

<p>However what I also notice is that when I release a new version of the service (through Helm - deployment is set to Recreate rather than RollingUpgrade) OR if the pod is died (out of memory) and restarted, it takes quite a while before the backend status is healthy again despite the Pod is already in running/healthy state (this is with existing Ingress and Load Balancer in GCP). Is there a way to speed this up?</p>

<p>When using GCE Ingress, you need to wait for the load balancer provisioning time before the <a href="https://cloud.google.com/load-balancing/docs/backend-service" rel="nofollow noreferrer">backend service</a> is deemed as healthy.</p>

<p>Consider that when you use this ingress class, you're relying on the GCE infrastructure that automatically has to provision an <a href="https://cloud.google.com/load-balancing/docs/https/" rel="nofollow noreferrer">HTTP(S) load balancer</a> and all of its components before sending requests into the cluster.</p>

<p>When you set up a deployment without <code>readinessProbe</code>, the default values are going to be applied to the load balancer health check:</p>

<pre><code>Backend Health Check
Interval: 60 seconds
Timeout: 60 seconds
Unhealthy threshold: 10 consecutive failures
Healthy threshold: 1 success
</code></pre>

<p>However, using the <code>readinessProbe</code> <a href="https://github.com/kubernetes/ingress-gce/blob/69ae0fe59e6ab69d8865339fa096e4cb142309c4/pkg/backends/syncer.go#L224" rel="nofollow noreferrer">will add the <code>periodSeconds</code> value to the default health check configuration</a>. So, in your case, you had <code>10</code> seconds + <code>60</code> by default = <code>70</code>.</p>

<pre><code>Backend Health check
Interval: 70 seconds
Timeout: 1 second
Unhealthy threshold: 10 consecutive failures
Healthy threshold: 1 success
</code></pre>

<p>Note that GKE will only use <code>readinessProbe</code> to set the health check in the load balancer. Liveness is never picked.</p>

<p>This means that, the lowest value will always be that of the default load balancer health check, <code>60</code>. Since these values are automatically set when the load balancer is invoked from GKE, there is no way to change them.</p>

<p>Wrapping up, you have to wait for the load balancer provisioing period (around ~1-3 minutes) plus the <code>periodSeconds</code> value set in your <code>readinessProbe</code>.</p>
