<p>In my Kubernetes cluster, I have a single pod (i.e. one replica) with two containers: <code>server</code> and <code>cache</code>.</p>
<p>I also have a Kubernetes <code>Service</code> that matches my pod.</p>
<p>If <code>cache</code> is crashing, when I try to send an HTTP request to <code>server</code> via my <code>Service</code>, I get a &quot;503 Service Temporarily Unavailable&quot;.</p>
<p>The HTTP request is going into the cluster via Nginx Ingress, and I suspect that the problem is that when <code>cache</code> is crashing, Kubernetes removes my one pod from the <code>Service</code> load balancers, as promised in the <a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/" rel="nofollow noreferrer">Kubernetes documentation</a>:</p>
<blockquote>
<p>The kubelet uses readiness probes to know when a container is ready to start accepting traffic. A Pod is considered ready when all of its containers are ready. One use of this signal is to control which Pods are used as backends for Services. When a Pod is not ready, it is removed from Service load balancers.</p>
</blockquote>
<p>I don't prefer this behavior, since I still want to be able <code>server</code> to respond to requests even if <code>cache</code> has failed. Is there any way to get this desired behavior?</p>

Matt S
<p>A POD is brought to the &quot;Failed&quot; state if one of the following conditions occur</p>
<ul>
<li>One of its containers exit with non-zero status</li>
<li>Kubernates terminates a container due to health checker failing</li>
</ul>
<p>So, if you need one of your containers to still respond when another one fails,</p>
<ol>
<li><p>Make sure your liveliness probe is pointed to the container you need to be continuing. The health checker will get success code always and will not mark the POD as &quot;Failed&quot;</p>
</li>
<li><p>Make sure the readiness probe is pointed to the container you neesd to be continuing. This will make sure that the load balancer will still send the traffic to your pod.</p>
</li>
<li><p>Make sure that you handle the container errors gracefully and make them exit with zero status code.</p>
</li>
</ol>
<p>In the following example readiness and liveliness probes, make sure that the port 8080 is handled by the <code>service</code> container and it has the <code>/healthz</code> and <code>/ready</code> routes active.</p>
<pre><code>    readinessProbe:
      httpGet:
        path: /healthz
        port: 8080
      initialDelaySeconds: 5
      periodSeconds: 5
    livenessProbe:
      httpGet:
        path: /ready
        port: 8080
      initialDelaySeconds: 5
      timeoutSeconds: 1
</code></pre>
