<p>I am trying to build and deploy microservices images to a single-node Kubernetes cluster running on my development machine using minikube. I am using the cloud-native microservices demo application Online Boutique by Google to understand the use of technologies like Kubernetes, Istio etc.</p>
<p>Link to github repo: <a href="https://github.com/GoogleCloudPlatform/microservices-demo" rel="nofollow noreferrer">microservices-demo</a></p>
<p>I have followed all the installation process to locally build and deploy the microservices, and am able to access the web frontend through my browser. However, when I click on any of the product images say, I see this error page.</p>
<p><a href="https://i.stack.imgur.com/Y5gs9.png" rel="nofollow noreferrer"> HTTP Status: 500 Internal Server Error</a></p>
<p>On doing a check using kubectl get pods
I realize that one of my pods( Recommendation service) has status CrashLoopBackOff.
Running <code>kubectl describe pods recommendationservice-55b4d6c477-kxv8r</code>:</p>
<pre><code>Namespace:    default
Priority:     0
Node:         minikube/192.168.99.116
Start Time:   Thu, 23 Jul 2020 19:58:38 +0530
Labels:       app=recommendationservice
              app.kubernetes.io/managed-by=skaffold-v1.11.0
              pod-template-hash=55b4d6c477
              skaffold.dev/builder=local
              skaffold.dev/cleanup=true
              skaffold.dev/deployer=kubectl
              skaffold.dev/docker-api-version=1.40
              skaffold.dev/run-id=49913ced-e8df-40a7-9336-a227b56bcb5f
              skaffold.dev/tag-policy=git-commit
Annotations:  &lt;none&gt;
Status:       Running
IP:           172.17.0.14
IPs:
  IP:           172.17.0.14
Controlled By:  ReplicaSet/recommendationservice-55b4d6c477
Containers:
  server:
    Container ID:   docker://2d92aa966a82fbe58c8f40f6ecf9d6d55c29f8081cb40e0423a2397e1419350f
    Image:          recommendationservice:2216d526d249cc8363129aed9a09d752f9ad8f458e61e50a2a99c59d000606cb
    Image ID:       docker://sha256:2216d526d249cc8363129aed9a09d752f9ad8f458e61e50a2a99c59d000606cb
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    137
      Started:      Thu, 23 Jul 2020 21:09:33 +0530
      Finished:     Thu, 23 Jul 2020 21:09:53 +0530
    Ready:          False
    Restart Count:  29
    Limits:
      cpu:     200m
      memory:  450Mi
    Requests:
      cpu:      100m
      memory:   220Mi
    Liveness:   exec [/bin/grpc_health_probe -addr=:8080] delay=0s timeout=1s period=5s #success=1 #failure=3
    Readiness:  exec [/bin/grpc_health_probe -addr=:8080] delay=0s timeout=1s period=5s #success=1 #failure=3
    Environment:
      PORT:                          8080
      PRODUCT_CATALOG_SERVICE_ADDR:  productcatalogservice:3550
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-sbpcx (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  default-token-sbpcx:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-sbpcx
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  &lt;none&gt;
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Pulled     44m (x15 over 74m)     kubelet, minikube  Container image &quot;recommendationservice:2216d526d249cc8363129aed9a09d752f9ad8f458e61e50a2a99c59d000606cb&quot; already present on machine
  Warning  Unhealthy  9m33s (x99 over 74m)   kubelet, minikube  Readiness probe failed: timeout: failed to connect service &quot;:8080&quot; within 1s
  Warning  BackOff    4m25s (x294 over 72m)  kubelet, minikube  Back-off restarting failed container
</code></pre>
<p>In Events, I see <code>Readiness probe failed: timeout: failed to connect service &quot;:8080&quot; within 1s</code>.
What is the reason and how can I resolve this?
Thanks for the help!</p>

Saranya Gupta
<h2>Answer</h2>
<p>The timeout of the Readiness Probe (1 second) was too short.</p>
<h2>More Info</h2>
<p>The relevant Readiness Probe is defined such that <code>/bin/grpc_health_probe -addr=:8080</code> is run inside the <code>server</code> container.</p>
<p>You would expect a 1 second timeout to be sufficient for such a probe but this is running on Minikube so that could be impacting the timeout of the probe.</p>
