<p>After giving nginx-controller an externalIPs list, I am able to hit the nginx load balancer but it gives me 503s. I've inspected the genereated nginx file and I cannot see any mention of the upstream servers, this part is not being populated in the nginx configuration file, I get placeholders for the upstream parts.</p>
<p>my deployment:</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

</code></pre>
<p>My ingress controller YAML:</p>
<pre><code>apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          serviceName: nginx-service
          servicePort: 80

</code></pre>
<p>My service:</p>
<pre><code>apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    name: nginx
spec:
  type: NodePort
  ports:
    - port: 80
      name: http
  selector:
    name: nginx

</code></pre>
<p>My logs from nginx:</p>
<pre><code>-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v0.34.1
  Build:         v20200715-ingress-nginx-2.11.0-8-gda5fa45e2
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.19.1

-------------------------------------------------------------------------------

I0808 10:03:07.530453       6 flags.go:205] Watching for Ingress class: nginx
W0808 10:03:07.530755       6 flags.go:250] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
W0808 10:03:07.530805       6 client_config.go:552] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0808 10:03:07.531018       6 main.go:231] Creating API client for https://10.96.0.1:443
I0808 10:03:07.535965       6 main.go:275] Running in Kubernetes cluster version v1.18 (v1.18.6) - git (clean) commit dff82dc0de47299ab66c83c626e08b245ab19037 - platform linux/amd64
I0808 10:03:07.661731       6 main.go:105] SSL fake certificate created /etc/ingress-controller/ssl/default-fake-certificate.pem
I0808 10:03:07.662262       6 main.go:113] Enabling new Ingress features available since Kubernetes v1.18
W0808 10:03:07.663809       6 main.go:125] No IngressClass resource with name nginx found. Only annotation will be used.
I0808 10:03:07.667982       6 ssl.go:528] loading tls certificate from certificate path /usr/local/certificates/cert and key path /usr/local/certificates/key
I0808 10:03:07.695581       6 nginx.go:263] Starting NGINX Ingress controller
I0808 10:03:07.699045       6 event.go:278] Event(v1.ObjectReference{Kind:&quot;ConfigMap&quot;, Namespace:&quot;ingress-nginx&quot;, Name:&quot;ingress-nginx-controller&quot;, UID:&quot;7f8efb11-d326-4a79-96e1-87e9e610d189&quot;, APIVersion:&quot;v1&quot;, ResourceVersion:&quot;11100&quot;, FieldPath:&quot;&quot;}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0808 10:03:08.799119       6 event.go:278] Event(v1.ObjectReference{Kind:&quot;Ingress&quot;, Namespace:&quot;default&quot;, Name:&quot;test-ingress&quot;, UID:&quot;5b3b5c37-d81e-4144-9c8e-d4ff424401ea&quot;, APIVersion:&quot;networking.k8s.io/v1beta1&quot;, ResourceVersion:&quot;31926&quot;, FieldPath:&quot;&quot;}): type: 'Normal' reason: 'CREATE' Ingress default/test-ingress
I0808 10:03:08.895939       6 nginx.go:307] Starting NGINX process
I0808 10:03:08.896060       6 leaderelection.go:242] attempting to acquire leader lease  ingress-nginx/ingress-controller-leader-nginx...
I0808 10:03:08.896177       6 nginx.go:327] Starting validation webhook on :8443 with keys /usr/local/certificates/cert /usr/local/certificates/key
I0808 10:03:08.896317       6 controller.go:141] Configuration changes detected, backend reload required.
I0808 10:03:08.897732       6 status.go:86] new leader elected: ingress-nginx-controller-7fd7d8df56-7kls4
I0808 10:03:08.943105       6 controller.go:157] Backend successfully reloaded.
I0808 10:03:08.943137       6 controller.go:166] Initial sync, sleeping for 1 second.
I0808 10:03:48.343214       6 leaderelection.go:252] successfully acquired lease ingress-nginx/ingress-controller-leader-nginx
I0808 10:03:48.343262       6 status.go:86] new leader elected: ingress-nginx-controller-7fd7d8df56-54v6b
I0808 10:03:48.353140       6 status.go:275] updating Ingress default/test-ingress status from [] to [{10.0.2.15 } {49.12.86.195 }]
I0808 10:03:48.355684       6 event.go:278] Event(v1.ObjectReference{Kind:&quot;Ingress&quot;, Namespace:&quot;default&quot;, Name:&quot;test-ingress&quot;, UID:&quot;5b3b5c37-d81e-4144-9c8e-d4ff424401ea&quot;, APIVersion:&quot;networking.k8s.io/v1beta1&quot;, ResourceVersion:&quot;33002&quot;, FieldPath:&quot;&quot;}): type: 'Normal' reason: 'UPDATE' Ingress default/test-ingress
10.32.0.1 - - [08/Aug/2020:18:37:39 +0000] &quot;GET / HTTP/1.1&quot; 503 197 &quot;-&quot; &quot;Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0&quot; 424 0.000 [default-nginx-1337] [] - - - - 68e58595b011156ca288edba56a53a99
10.32.0.1 - - [08/Aug/2020:18:37:42 +0000] &quot;GET / HTTP/1.1&quot; 503 197 &quot;-&quot; &quot;Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0&quot; 367 0.000 [default-nginx-1337] [] - - - - 77659ecd8f16cbc8a1d9f9e603dcba90
W0808 18:38:37.982021       6 controller.go:916] Service &quot;default/nginx&quot; does not have any active Endpoint.
I0808 18:38:38.010466       6 main.go:115] successfully validated configuration, accepting ingress test-ingress in namespace default
I0808 18:38:38.012769       6 event.go:278] Event(v1.ObjectReference{Kind:&quot;Ingress&quot;, Namespace:&quot;default&quot;, Name:&quot;test-ingress&quot;, UID:&quot;5b3b5c37-d81e-4144-9c8e-d4ff424401ea&quot;, APIVersion:&quot;networking.k8s.io/v1beta1&quot;, ResourceVersion:&quot;106029&quot;, FieldPath:&quot;&quot;}): type: 'Normal' reason: 'UPDATE' Ingress default/test-ingress
W0808 18:38:38.012885       6 controller.go:916] Service &quot;default/nginx&quot; does not have any active Endpoint.
I0808 18:38:38.012926       6 controller.go:141] Configuration changes detected, backend reload required.
I0808 18:38:38.060293       6 controller.go:157] Backend successfully reloaded.
10.32.0.1 - - [08/Aug/2020:18:38:39 +0000] &quot;GET / HTTP/1.1&quot; 503 197 &quot;-&quot; &quot;Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0&quot; 367 0.000 [default-nginx-80] [] - - - - 49267e557ee39cfed6c41b1592d2ceaf

</code></pre>
<p>I'm not sure what kind of service the nginx service is meant to be, I suspected NodePort.</p>
<p>The part that is suspicious to me is</p>
<pre><code>Service &quot;default/nginx&quot; does not have any active Endpoint.
</code></pre>
<p>But I cannot work out what it means.</p>
<p>EDIT: Output of <code>kubectl describe svc ngin</code></p>
<pre><code>Name:                     nginx
Namespace:                default
Labels:                   name=nginx
Annotations:              Selector:  name=nginx
Type:                     NodePort
IP:                       10.105.68.13
Port:                     http  80/TCP
TargetPort:               80/TCP
NodePort:                 http  32250/TCP
Endpoints:                &lt;none&gt;
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   &lt;none&gt;

</code></pre>
<p>kubectl describe pod nginx-deployment</p>
<pre><code>Name:                      nginx-deployment-68bd55b8f6-5gcpn
Namespace:                 default
Priority:                  0
Node:                      ubuntu/10.0.2.15
Start Time:                Sat, 08 Aug 2020 08:33:11 +0100
Labels:                    app=nginx
                           pod-template-hash=68bd55b8f6
Annotations:               &lt;none&gt;
Status:                    Terminating (lasts 3d5h)
Termination Grace Period:  30s
IP:                        10.44.0.2
IPs:
  IP:           10.44.0.2
Controlled By:  ReplicaSet/nginx-deployment-68bd55b8f6
Containers:
  nginx:
    Container ID:   docker://d6553f9fe848c4b535cbc05a3009accb06f7825f742f39db9e15d6511b0dba00
    Image:          nginx:1.14.2
    Image ID:       docker-pullable://nginx@sha256:f7988fb6c02e0ce69257d9bd9cf37ae20a60f1df7563c3a2a6abe24160306b8d
    Port:           1337/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Sat, 08 Aug 2020 08:33:22 +0100
    Ready:          True
    Restart Count:  0
    Environment:    &lt;none&gt;
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-8hfzj (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-8hfzj:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-8hfzj
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  &lt;none&gt;
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          &lt;none&gt;

Name:                      nginx-deployment-68bd55b8f6-zpzj5
Namespace:                 default
Priority:                  0
Node:                      ubuntu/10.0.2.15
Start Time:                Sat, 08 Aug 2020 08:33:11 +0100
Labels:                    app=nginx
                           pod-template-hash=68bd55b8f6
Annotations:               &lt;none&gt;
Status:                    Terminating (lasts 3d5h)
Termination Grace Period:  30s
IP:                        10.44.0.1
IPs:
  IP:           10.44.0.1
Controlled By:  ReplicaSet/nginx-deployment-68bd55b8f6
Containers:
  nginx:
    Container ID:   docker://4ed8b3e6896cc3923a4e446c102a4822e118eb66cb531df1e865ee4ad78e3fdb
    Image:          nginx:1.14.2
    Image ID:       docker-pullable://nginx@sha256:f7988fb6c02e0ce69257d9bd9cf37ae20a60f1df7563c3a2a6abe24160306b8d
    Port:           1337/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Sat, 08 Aug 2020 08:33:21 +0100
    Ready:          True
    Restart Count:  0
    Environment:    &lt;none&gt;
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-8hfzj (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-8hfzj:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-8hfzj
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  &lt;none&gt;
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          &lt;none&gt;

Name:         nginx-deployment-6b474476c4-9zsbx
Namespace:    default
Priority:     0
Node:         ubuntu-2004-focal-64-minimal/49.12.86.195
Start Time:   Tue, 11 Aug 2020 16:14:55 +0100
Labels:       app=nginx
              pod-template-hash=6b474476c4
Annotations:  &lt;none&gt;
Status:       Running
IP:           10.32.0.7
IPs:
  IP:           10.32.0.7
Controlled By:  ReplicaSet/nginx-deployment-6b474476c4
Containers:
  nginx:
    Container ID:   docker://c1610bd533ea78339aca54a252f4058e08805940dc2e55ab803728d00bda9274
    Image:          nginx:1.14.2
    Image ID:       docker-pullable://nginx@sha256:f7988fb6c02e0ce69257d9bd9cf37ae20a60f1df7563c3a2a6abe24160306b8d
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 11 Aug 2020 16:14:56 +0100
    Ready:          True
    Restart Count:  0
    Environment:    &lt;none&gt;
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-8hfzj (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-8hfzj:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-8hfzj
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  &lt;none&gt;
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age    From                                   Message
  ----    ------     ----   ----                                   -------
  Normal  Scheduled  9m51s  default-scheduler                      Successfully assigned default/nginx-deployment-6b474476c4-9zsbx to ubuntu-2004-focal-64-minimal
  Normal  Pulled     9m51s  kubelet, ubuntu-2004-focal-64-minimal  Container image &quot;nginx:1.14.2&quot; already present on machine
  Normal  Created    9m50s  kubelet, ubuntu-2004-focal-64-minimal  Created container nginx
  Normal  Started    9m50s  kubelet, ubuntu-2004-focal-64-minimal  Started container nginx

Name:         nginx-deployment-6b474476c4-xlj4m
Namespace:    default
Priority:     0
Node:         ubuntu-2004-focal-64-minimal/49.12.86.195
Start Time:   Tue, 11 Aug 2020 16:14:56 +0100
Labels:       app=nginx
              pod-template-hash=6b474476c4
Annotations:  &lt;none&gt;
Status:       Running
IP:           10.32.0.4
IPs:
  IP:           10.32.0.4
Controlled By:  ReplicaSet/nginx-deployment-6b474476c4
Containers:
  nginx:
    Container ID:   docker://05d30b9bcbb92bccfaec96dd8298a2ed1690394910db4703a3b7bb6624920071
    Image:          nginx:1.14.2
    Image ID:       docker-pullable://nginx@sha256:f7988fb6c02e0ce69257d9bd9cf37ae20a60f1df7563c3a2a6abe24160306b8d
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 11 Aug 2020 16:14:57 +0100
    Ready:          True
    Restart Count:  0
    Environment:    &lt;none&gt;
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-8hfzj (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-8hfzj:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-8hfzj
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  &lt;none&gt;
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age    From                                   Message
  ----    ------     ----   ----                                   -------
  Normal  Scheduled  9m50s  default-scheduler                      Successfully assigned default/nginx-deployment-6b474476c4-xlj4m to ubuntu-2004-focal-64-minimal
  Normal  Pulled     9m50s  kubelet, ubuntu-2004-focal-64-minimal  Container image &quot;nginx:1.14.2&quot; already present on machine
  Normal  Created    9m50s  kubelet, ubuntu-2004-focal-64-minimal  Created container nginx
  Normal  Started    9m49s  kubelet, ubuntu-2004-focal-64-minimal  Started container nginx

</code></pre>

<p>In this case you wanted to use <a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/" rel="nofollow noreferrer">service with selectors</a>. There is also option to create <a href="https://kubernetes.io/docs/concepts/services-networking/service/#services-without-selectors" rel="nofollow noreferrer">service without selectors</a>.</p>
<p>In your <code>Deployment</code> you have used <code>app: nginx</code>, however in <code>service</code> you used <code>name: nginx</code>. More details you can find in <a href="https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service" rel="nofollow noreferrer">Service Documentation</a>.</p>
<p>Second thing is that in <code>Ingress</code> you pointed to <code>serviceName</code> called <code>nginx-service</code> where your <code>service</code> name is <code>nginx</code>.
Another thing is that you have only specified <code>port</code> in your <code>service</code> manifest. In this case is no matter as Nginx is listening on port 80, however if you wouldn't specify it, it will assign <code>targetPort</code> with the same value as <code>port</code>.</p>
<blockquote>
<p>A Service can map any incoming <code>port</code> to a <code>targetPort</code>. By default and for convenience, the <code>targetPort</code> is set to the same value as the <code>port</code> field.</p>
</blockquote>
<p>This YAMLs should work in your case:</p>
<pre><code>apiVersion: apps/v1
kind:  Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.14.2
          ports:
          - containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  labels: 
    app: nginx
spec:
  type: NodePort 
  selector:
    app: nginx
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          serviceName: nginx-service
          servicePort: 80
      
</code></pre>
<p>As last thing, please keep in mind that <code>pathType: Prefix</code> was introduced in <a href="https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#better-path-matching-with-path-types" rel="nofollow noreferrer">Kubernetes 1.18</a>, so it will give error if you will use it in older versions.</p>
