<p>I have .netcore 2.2 API Pod on my K8S Farm i implement health check api to let k8s liveness check.
Here is My settings.</p>

<pre><code>     livenessProbe:
        httpGet:
          path: /api/Authentication/CheckLiveness
          port: 80
          scheme: HTTP
        initialDelaySeconds: 100
        timeoutSeconds: 60
        periodSeconds: 30
        successThreshold: 1
        failureThreshold: 1
      readinessProbe:
        httpGet:
          path: /api/Authentication/CheckReadiness
          port: 80
          scheme: HTTP
        initialDelaySeconds: 50
        timeoutSeconds: 30
        periodSeconds: 15
        successThreshold: 1
        failureThreshold: 1
</code></pre>

<p>The Problem is other worker node seem working find without problem except pod on worker node 1.</p>

<p><a href="https://i.stack.imgur.com/ZHeYK.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ZHeYK.png" alt="Error Pod on WorkerNode1"></a></p>

<p>Here is the error event.</p>

<pre><code>Liveness probe failed: Get 
http://10.244.3.218:80/api/Authentication/CheckLiveness:net/http: 
request canceled (Client.Timeout exceeded while awaiting headers)

Readiness probe failed: Get 
http://10.244.3.218:80/api/Authentication/CheckReadiness: net/http: 
request canceled (Client.Timeout exceeded while awaiting headers)
</code></pre>

<p>I can curl to my pod by exec it. But the pod keep restarting.</p>

<p>I think it a node Problem. This K8s Farm run on Vmware with centos7 os.
I tried this config on dev environment with the same infrastructure. and it all green without problem. 
Need any suggestion to debug or solve this problem.
Thank you.</p>

<p>@mWatney Edited</p>

<p>And here your result</p>

<pre><code>Name:           authenservice-dpm-7d468bfcc4-px44m
Namespace:      pbtsapi
Priority:       0
Node:           ptseclsbtwkn01/192.168.10.136
Start Time:     Fri, 12 Jun 2020 11:23:07 +0700
Labels:         app=authenservice-api
                pod-template-hash=7d468bfcc4
Annotations:    &lt;none&gt;
Status:         Running
IP:             10.244.3.218
Controlled By:  ReplicaSet/authenservice-dpm-7d468bfcc4
Containers:
  authenservice:
    Container ID:   docker://1b1acffeae54421201d1bbc54b8020a75db660e1ae1a0f0d18a56930bbca0d12
    Image:          10.99.21.89:5000/authenservice:v1.0.4
    Image ID:       docker-pullable://10.99.21.89:5000/authenservice@sha256:b9244059195edff3cc3592d3e19a94ac00e481e9936413a4315a3cf41b0023ea
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 12 Jun 2020 14:46:22 +0700
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 12 Jun 2020 14:37:52 +0700
      Finished:     Fri, 12 Jun 2020 14:46:21 +0700
    Ready:          False
    Restart Count:  28
    Limits:
      cpu:     500m
      memory:  400Mi
    Requests:
      cpu:      250m
      memory:   200Mi
    Liveness:   http-get http://:80/api/Authentication/CheckLiveness delay=300s timeout=60s period=30s #success=1 #failure=1
    Readiness:  http-get http://:80/api/Authentication/CheckReadiness delay=300s timeout=60s period=30s #success=1 #failure=1
    Environment:
      MSSQL_PORT:  1433
    Mounts:
      /app/appsettings.json from authen-v (rw,path="appsettings.json")
      /etc/localtime from tz-config (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-h8x2b (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  authen-v:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      authenservice-config
    Optional:  false
  tz-config:
    Type:          HostPath (bare host directory volume)
    Path:          /usr/share/zoneinfo/Asia/Bangkok
    HostPathType:  
  default-token-h8x2b:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-h8x2b
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  &lt;none&gt;
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                     From                     Message
  ----     ------     ----                    ----                     -------
  Normal   Killing    22m (x26 over 3h23m)    kubelet, ptseclsbtwkn01  Container authenservice failed liveness probe, will be restarted
  Warning  Unhealthy  6m42s (x28 over 3h23m)  kubelet, ptseclsbtwkn01  Liveness probe failed: Get http://10.244.3.218:80/api/Authentication/CheckLiveness: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
  Warning  Unhealthy  10s (x22 over 3h8m)     kubelet, ptseclsbtwkn01  Readiness probe failed: Get http://10.244.3.218:80/api/Authentication/CheckReadiness: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
</code></pre>

<p>Virtual Machines are never identical. You can setup Virtual Machines with identical specs, by doing this you make sure they will have similarities but it doesn't mean they will perform exactly the same. </p>

<p>Same thing happens to servers. You can buy two identical physical servers and they will perform similarly but never identically. </p>

<p>Logs are clearly saying that by the time that the application was tested, it wasn't ready yet. </p>

<p>As <a href="https://stackoverflow.com/users/251311/zerkms" title="217,534 reputation">zerkms</a> said, you can't have identical rediness and liveness probes. You definitely have  to review this. You have also to review your readiness probe <code>initialDelaySeconds</code>. Try increasing it to give more time for your application to start. </p>

<p>To troubleshoot, <a href="https://stackoverflow.com/users/5564578/suren" title="3,990 reputation">suren</a> suggestion and increase your timeout, I would increase the <code>initialDelaySeconds</code> instead.  </p>
