<p>I have setup kubenertes on ubuntu server using <a href="https://vitux.com/install-and-deploy-kubernetes-on-ubuntu/" rel="nofollow noreferrer">this</a> link.</p>

<p>Then I installed kubernetes dashboard using:</p>

<pre><code>kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-rc6/aio/deploy/recommended.yaml
</code></pre>

<p>Then I changed the ClusterIP to NodePort 32323, the service to NodePort.</p>

<p>But container is not running.</p>

<pre><code>uday@dockermaster:~$ kubectl -n kubernetes-dashboard get all
NAME                                             READY   STATUS             RESTARTS   AGE
pod/dashboard-metrics-scraper-779f5454cb-pqfrj   1/1     Running            0          50m
pod/kubernetes-dashboard-64686c4bf9-5jkwq        0/1     CrashLoopBackOff   14         50m

NAME                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)         AGE
service/dashboard-metrics-scraper   ClusterIP   10.103.22.252   &lt;none&gt;        8000/TCP        50m
service/kubernetes-dashboard        NodePort    10.102.48.80    &lt;none&gt;        443:32323/TCP   50m

NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/dashboard-metrics-scraper   1/1     1            1           50m
deployment.apps/kubernetes-dashboard        0/1     1            0           50m

NAME                                                   DESIRED   CURRENT   READY   AGE
replicaset.apps/dashboard-metrics-scraper-779f5454cb   1         1         1       50m
replicaset.apps/kubernetes-dashboard-64686c4bf9        1         1         0       50m

uday@dockermaster:~$ kubectl -n kubernetes-dashboard describe svc kubernetes-dashboard
Name:                     kubernetes-dashboard
Namespace:                kubernetes-dashboard
Labels:                   k8s-app=kubernetes-dashboard
Annotations:              Selector:  k8s-app=kubernetes-dashboard
Type:                     NodePort
IP:                       10.102.48.80
Port:                     &lt;unset&gt;  443/TCP
TargetPort:               8443/TCP
NodePort:                 &lt;unset&gt;  32323/TCP
Endpoints:
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   &lt;none&gt;
</code></pre>

<hr>

<p>Other apps are working fine with NodePort, whether Tomcat/nginx/databases.</p>

<p>But here, it is failing with container creation.</p>

<pre><code>C:\Users\uday\Desktop&gt;kubectl.exe get pods -n kubernetes-dashboard
NAME                                         READY   STATUS             RESTARTS   AGE
dashboard-metrics-scraper-779f5454cb-pqfrj   1/1     Running            1          20h
kubernetes-dashboard-64686c4bf9-g9z2k        0/1     CrashLoopBackOff   84         18h

C:\Users\uday\Desktop&gt;kubectl.exe describe pod kubernetes-dashboard-64686c4bf9-g9z2k -n kubernetes-dashboard
Name:         kubernetes-dashboard-64686c4bf9-g9z2k
Namespace:    kubernetes-dashboard
Priority:     0
Node:         slave-node/10.0.0.6
Start Time:   Sat, 28 Mar 2020 14:16:54 +0000
Labels:       k8s-app=kubernetes-dashboard
              pod-template-hash=64686c4bf9
Annotations:  &lt;none&gt;
Status:       Running
IP:           182.244.1.12
IPs:
  IP:           182.244.1.12
Controlled By:  ReplicaSet/kubernetes-dashboard-64686c4bf9
Containers:
  kubernetes-dashboard:
    Container ID:  docker://470ee8c61998c3c3dda86c58ad17817468f55aa73cd4feecf3b018977ce13ca3
    Image:         kubernetesui/dashboard:v2.0.0-rc6
    Image ID:      docker-pullable://kubernetesui/dashboard@sha256:61f9c378c427a3f8a9643f83baa9f96db1ae1357c67a93b533ae7b36d71c69dc
    Port:          8443/TCP
    Host Port:     0/TCP
    Args:
      --auto-generate-certificates
      --namespace=kubernetes-dashboard
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    2
      Started:      Sun, 29 Mar 2020 09:01:31 +0000
      Finished:     Sun, 29 Mar 2020 09:02:01 +0000
    Ready:          False
    Restart Count:  84
    Liveness:       http-get https://:8443/ delay=30s timeout=30s period=10s #success=1 #failure=3
    Environment:    &lt;none&gt;
    Mounts:
      /certs from kubernetes-dashboard-certs (rw)
      /tmp from tmp-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kubernetes-dashboard-token-pzfbl (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  kubernetes-dashboard-certs:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  kubernetes-dashboard-certs
    Optional:    false
  tmp-volume:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  &lt;unset&gt;
  kubernetes-dashboard-token-pzfbl:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  kubernetes-dashboard-token-pzfbl
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  beta.kubernetes.io/os=linux
Tolerations:     node-role.kubernetes.io/master:NoSchedule
                 node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason   Age                     From                 Message
  ----     ------   ----                    ----                 -------
  Warning  BackOff  4m49s (x501 over 123m)  kubelet, slave-node  Back-off restarting failed container

kubectl.exe logs kubernetes-dashboard-64686c4bf9-g9z2k -n kubernetes-dashboard
2020/03/29 09:01:31 Starting overwatch
2020/03/29 09:01:31 Using namespace: kubernetes-dashboard
2020/03/29 09:01:31 Using in-cluster config to connect to apiserver
2020/03/29 09:01:31 Using secret token for csrf signing
2020/03/29 09:01:31 Initializing csrf token from kubernetes-dashboard-csrf secret
panic: Get https://10.96.0.1:443/api/v1/namespaces/kubernetes-dashboard/secrets/kubernetes-dashboard-csrf: dial tcp 10.96.0.1:443: i/o timeout

goroutine 1 [running]:
github.com/kubernetes/dashboard/src/app/backend/client/csrf.(*csrfTokenManager).init(0xc0004e2dc0)
    /home/travis/build/kubernetes/dashboard/src/app/backend/client/csrf/manager.go:40 +0x3b0
github.com/kubernetes/dashboard/src/app/backend/client/csrf.NewCsrfTokenManager(...)
    /home/travis/build/kubernetes/dashboard/src/app/backend/client/csrf/manager.go:65
github.com/kubernetes/dashboard/src/app/backend/client.(*clientManager).initCSRFKey(0xc00043ae80)
    /home/travis/build/kubernetes/dashboard/src/app/backend/client/manager.go:499 +0xc6
github.com/kubernetes/dashboard/src/app/backend/client.(*clientManager).init(0xc00043ae80)
    /home/travis/build/kubernetes/dashboard/src/app/backend/client/manager.go:467 +0x47
github.com/kubernetes/dashboard/src/app/backend/client.NewClientManager(...)
    /home/travis/build/kubernetes/dashboard/src/app/backend/client/manager.go:548
main.main()
    /home/travis/build/kubernetes/dashboard/src/app/backend/dashboard.go:105 +0x20d
</code></pre>

<p><strong>The Problem</strong></p>

<p>The reason the application is not coming up is because the Dashboard container itself is not running. If you look at the output you provided you can see this:</p>

<p><code>pod/kubernetes-dashboard-64686c4bf9-5jkwq        0/1     CrashLoopBackOff   14</code></p>

<p>So how do we <a href="https://kubernetes.io/docs/tasks/debug-application-cluster/debug-application-introspection/" rel="nofollow noreferrer">troubleshoot</a> this? Well, there are three principle ways. One of which you'll probably be using more than the other two.</p>

<p><strong>Describe</strong></p>

<p><a href="https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#describe" rel="nofollow noreferrer">Describe</a> is a is a command that allows you to fetch details about a resource in kubernetes. This could be metadata information, the amount of replicas you assigned, or even some events depicting why a resource is failing to start. For example, a referenced Container Image in your Pod manifest can not be found in the usable container registries. The syntax for using Describe is like so:</p>

<p><code>kubectl describe pod -n kubernetes-dashboard kubernetes-dashboard-64686c4bf9-5jkwq</code></p>

<p>Here are some great docs on the tool as well.</p>

<p><strong>Logs</strong></p>

<p>The next troubleshooting step you'll likely take advantage of in Kubernetes is using the <a href="https://kubernetes.io/docs/concepts/cluster-administration/logging/" rel="nofollow noreferrer">logging architecture</a>. As you're probably aware, when a Docker container is spawned it is common practice to have the logs that are produced by the application be redirected to <code>STDOUT</code> or <code>STDERR</code> for the process. Kubernetes them captures this log data for you and provides an API abstraction layer with which you can interact with it. Sometimes your Describe events won't have any indication of why a process isn't running. However, you can then proceed with <a href="https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#logs" rel="nofollow noreferrer">grabbing logs</a> from the process to determine what is going wrong. An example syntax might look like:</p>

<p><code>kubectl logs -f -n kubernetes-dashboard kubernetes-dashboard-64686c4bf9-5jkwq</code></p>

<p><strong>Exec</strong></p>

<p>The last common troubleshooting technique is <a href="https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#exec" rel="nofollow noreferrer">Exec</a>. Exec effectively allows you to attach to the a shell in a running container so that you can <a href="https://kubernetes.io/docs/tasks/debug-application-cluster/get-shell-running-container/" rel="nofollow noreferrer">interact</a> with the live environment to troubleshoot the application. This allows you to do things like see if configuration files were properly staged on the container's filesystem, determine if environment variables were properly expanded and set, etc. An example syntax for Exec might look like:</p>

<p><code>kubectl exec -it -n kubernetes-dashboard kubernetes-dashboard-64686c4bf9-5jkwq sh</code></p>

<p>In this case, however, your pod is in a CrashLoopBackoff state. This means that you will not be able to exec into it due to the fact that the container is not running. The Kubernetes API Server recognized a pattern of failures and automatically reduced its attempts at scheduling the workload accordingly.</p>

<p>Here is a good <a href="https://stackoverflow.com/questions/41604499/my-kubernetes-pods-keep-crashing-with-crashloopbackoff-but-i-cant-find-any-lo">thread on how to troubleshoot</a> pods that enter this state.</p>

<p><strong>Summary</strong></p>

<p>So, now that I've said all of this. How do we answer your question? Well, we can't answer it directly. But I sort of did with my summary above. Because the real answer you're looking for is how to properly troubleshoot linux containers running in Kubernetes. These issues will be a reoccurring theme in your experience with Kubernetes so it's essential to develop debugging skills in the ecosystem as soon as possible. </p>

<p>If the <code>Describe</code>, <code>Logs</code>, and <code>Exec</code> command are unable to help you find out why the Dashboard pod is failing to come up, feel free to add a comment on this answer requesting additional support and I'll be happy to help where I can!</p>
