<p>I'm trying to create AWS ALB-Ingress through EKS following the steps in the document <a href="https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html" rel="nofollow noreferrer">https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html</a></p>
<p>I was successful till the step 7 in creating the controller:</p>
<pre><code>[ec2-user@ip-X-X-X-X eks-cluster]$ kubectl apply -f v2_0_0_full.yaml 
customresourcedefinition.apiextensions.k8s.io/targetgroupbindings.elbv2.k8s.aws created 
mutatingwebhookconfiguration.admissionregistration.k8s.io/aws-load-balancer-webhook created 
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply 
serviceaccount/aws-load-balancer-controller configured 
role.rbac.authorization.k8s.io/aws-load-balancer-controller-leader-election-role created 
clusterrole.rbac.authorization.k8s.io/aws-load-balancer-controller-role created 
rolebinding.rbac.authorization.k8s.io/aws-load-balancer-controller-leader-election-rolebinding created 
clusterrolebinding.rbac.authorization.k8s.io/aws-load-balancer-controller-rolebinding created 
service/aws-load-balancer-webhook-service created 
deployment.apps/aws-load-balancer-controller created 
certificate.cert-manager.io/aws-load-balancer-serving-cert created 
issuer.cert-manager.io/aws-load-balancer-selfsigned-issuer created 
validatingwebhookconfiguration.admissionregistration.k8s.io/aws-load-balancer-webhook created
</code></pre>
<p>However, the controller does NOT get to &quot;Ready&quot; status:</p>
<pre><code>[ec2-user@ip-X-X-X-X eks-cluster]$ kubectl get deployment -n kube-system aws-load-balancer-controller
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
aws-load-balancer-controller   0/1     1            0           29m

</code></pre>
<p>I'm also able to list the pod associated with the controller which also shows NOT READY:</p>
<pre><code>[ec2-user@ip-X-X-X-X eks-cluster]$ kubectl get pods -n kube-system
NAME                                            READY   STATUS    RESTARTS   AGE
aws-load-balancer-controller-XXXXXXXXXX-p4l7f   0/1     Pending   0          30m
</code></pre>
<p>I also can't seem to get its logs in order to try and debug the issue:</p>
<pre><code>[ec2-user@ip-X-X-X-X eks-cluster]$ kubectl -n kube-system logs aws-load-balancer-controller-XXXXXXXXXX-p4l7f
[ec2-user@ip-X-X-X-X eks-cluster]$
</code></pre>
<p>Furthermore, the /var/log directory also does not have any related logs.</p>
<p>Please help me understand why it is not coming to READY state. Also let me know how to enable logging to debug these kind of issues.</p>

<p>From the <a href="https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.1.3/docs/install/v2_1_3_full.yaml" rel="nofollow noreferrer">current LB conntroller manifest</a> I found out that LB controller Pod specification doesn't have <code>Readiness probe</code>, only <code>Liveness probe</code>. That means that the Pod becomes <code>Ready</code> as soon as it pass the Liveness probe:</p>
<pre><code>      livenessProbe:
        failureThreshold: 2
        httpGet:
          path: /healthz
          port: 61779
          scheme: HTTP
        initialDelaySeconds: 30
        timeoutSeconds: 10
</code></pre>
<p>But as we can see in the following output, LB controller's Pod is in <code>Pending</code> state:</p>
<pre><code>[ec2-user@ip-X-X-X-X eks-cluster]$ kubectl get pods -n kube-system
NAME                                            READY   STATUS    RESTARTS   AGE
aws-load-balancer-controller-XXXXXXXXXX-p4l7f   0/1     Pending   0          30m
</code></pre>
<p>If Pod stays in <code>Pending</code> state, it means that <code>kube-scheduler</code> is unable to bind the Pod to a cluster node for whatever reason.</p>
<p><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/" rel="nofollow noreferrer">Kube-scheduler</a> is a part of Kubernetes control plain that is responsible for assigning Pods to Nodes.</p>
<p>No Pod logs exist at this phase, because Pod's containers are not started yet.</p>
<p>The most convenient way to check the reason is using the <code>kubectl describe</code> command:</p>
<pre><code>kubectl describe pod/podname -n namespacename
</code></pre>
<p>On the bottom of the output there are list of events related to the Pod life cycle. Here is an example for the generic Ubuntu Pod:</p>
<pre><code>Events:
  Type    Reason     Age                From               Message
  ----    ------     ----               ----               -------
  Normal  Scheduled  37s                default-scheduler  Successfully assigned default/ubuntu to k8s-w1
  Normal  Pulling    25s (x2 over 35s)  kubelet, k8s-w1    Pulling image &quot;ubuntu&quot;
  Normal  Pulled     23s (x2 over 30s)  kubelet, k8s-w1    Successfully pulled image &quot;ubuntu&quot;
  Normal  Created    23s (x2 over 30s)  kubelet, k8s-w1    Created container ubuntu
  Normal  Started    23s (x2 over 29s)  kubelet, k8s-w1    Started container ubuntu
</code></pre>
<p><code>kubectl get events</code> command can also show the problem. For example:</p>
<pre><code>LAST SEEN   TYPE     REASON      OBJECT       MESSAGE
21s         Normal   Scheduled   pod/ubuntu   Successfully assigned default/ubuntu to k8s-w1
9s          Normal   Pulling     pod/ubuntu   Pulling image &quot;ubuntu&quot;
7s          Normal   Pulled      pod/ubuntu   Successfully pulled image &quot;ubuntu&quot;
7s          Normal   Created     pod/ubuntu   Created container ubuntu
7s          Normal   Started     pod/ubuntu   Started container ubuntu
</code></pre>
<p>or there could be a reason why Scheduler can't assign Pod to a Node:</p>
<pre><code>&quot;No nodes are available that match all of the predicates: Insufficient cpu (2), Insufficient memory (2)&quot;. 
</code></pre>
<p>In some cases errors could be found in <code>kube-scheduler</code> Pod logs in <code>kube-system</code> namespace. The logs could be listed using the following command:</p>
<pre><code>kubectl logs $(kubectl get pods -l component=kube-scheduler,tier=control-plane -n kube-system -o name) -n kube-system 
</code></pre>
<p>Most common reasons why pod isn't scheduled are the following:</p>
<ul>
<li>lack of CPU or memory <a href="https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/" rel="nofollow noreferrer">resources</a> requested by a Pod on the Nodes.</li>
<li>Pod cannot tolerate <a href="https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/" rel="nofollow noreferrer">Taints on the Nodes</a></li>
<li>Pod have <a href="https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/" rel="nofollow noreferrer">Affinity/AntiAffinity</a> configuration that prevents it from scheduling</li>
<li>Storage or <a href="https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/" rel="nofollow noreferrer">other</a> specific resource (like GPU) requirements in Pod spec cannot be satisfied</li>
</ul>
