<p>My question is built on the question and answers from this question - <a href="https://stackoverflow.com/questions/41509439/whats-the-difference-between-clusterip-nodeport-and-loadbalancer-service-types">What&#39;s the difference between ClusterIP, NodePort and LoadBalancer service types in Kubernetes?</a></p>
<p>The question might not be well-formed for some of you.</p>
<p>I am trying to understand the differences between <code>clusterIP</code>, <code>nodePort</code> and <code>Loadbalancer</code> and when to use these with an example. I suppose that my understanding of the following concept is correct
K8s consists of the following components</p>
<ul>
<li>Node - A VM or physical machine. Runs kubectl and docker process</li>
<li>Pod - unit which encapsulates container(s) and volumes (storage). If a pod contains multiple containers then shared volume could be the way for process communication</li>
<li>Node can have one or multiple pods. Each pod will have its own IP</li>
<li>Cluster - replicas of a Node. Each node in a cluster will contain same pods (instances, type)</li>
</ul>
<p>Here is the scenario:</p>
<p>My application has a <code>web server</code> (always returning 200OK) and a <code>database</code> (always returning the same value) for simplicity. Also, say I am on <code>GCP</code> and I make images of <code>webserver</code> and of the <code>database</code>. Each of these will be run in their own respective <code>pods</code> and will have 2 replicas.</p>
<p>I suppose I'll have two clusters (<code>cluster-webserver (node1-web (containing pod1-web), node2-web (containing pod2-web))</code> and <code>cluster-database (node1-db (containing pod1-db), node2-db (containing pod2-db))</code>. Each node will have its own <code>ip</code> address (<code>node1-webip, node2-webip, node1-dbip, node2-dbip</code>)</p>
<p>A client application (browser) should be able to access the web application from outside <code>web</code> cluster but the <code>database</code> shouldn't be accessible from outside <code>database</code> cluster. However <code>web</code> nodes should be able to access <code>database</code> nodes)</p>
<ul>
<li>Question 1 - Am I correct that if I create a service for <code>web</code> (<code>webServiceName</code>) and a service for <code>database</code> then by default, I'll get only <code>clusterIP</code> and a <code>port</code> (or <code>targetPort</code>).</li>
<li>Question 1.2 - Am I correct that <code>clusterIP</code> is an <code>IP</code> assigned to a <code>pod</code>, not the <code>node</code> i.e. in my example, clusterIP gets assigned to <code>pod1-web</code>, not <code>node1-web</code> even though <code>node1</code> has only <code>pod1</code>.</li>
<li>Question 1.3 - Am I correct that as <code>cluster IP</code> is accessible from only within the cluster, <code>pod1-web</code> and <code>pod2-web</code> can talk to each other and <code>pod1-db</code> and <code>pod2-db</code> can talk to each other using <code>clusterIP/dns:port</code> or <code>clusterIP/dns:targetPort</code> but <code>web</code> can't talk to <code>database</code> (and vice versa) and external client can't talk to web? Also, the <code>nodes</code> are not accessible using the <code>cluster IP</code>.</li>
<li>Question 1.4 - Am I correct that <code>dns</code> i.e. <code>servicename.namespace.svc.cluster.local</code> would map the <code>clusterIP</code>?</li>
<li>Question 1.5 - For which type of applications I might use only <code>clusterIP</code>? Where multiple instances of an application need to communicate with each other (eg master-slave configuration)?</li>
</ul>
<p>If I use <code>nodePort</code> then <code>K8s</code> will open a <code>port</code> on each of the <code>node</code> and will forward <code>nodeIP/nodePort</code> to <code>cluster IP (on pod)/Cluster Port</code></p>
<ul>
<li>Question 2 - Can <code>web</code> nodes now access <code>database</code> nodes using <code>nodeIP:nodePort</code> which will route the traffic to <code>database's</code> <code>clusterIP (on pod):clusterport/targertPort</code>? ( I have read that clusterIP/dns:nodePort will not work).</li>
<li>Question 2.1 - How do I get a <code>node's</code> <code>IP</code>? Is <code>nodeIP</code> the <code>IP</code> I'll get when I run <code>describe pods</code> command?</li>
<li>Question 2.2 - Is there a <code>dns</code> equivalent for the <code>node IP</code> as <code>node IP</code> could change during failovers. Or does <code>dns</code> now resolve to the node's IP instead of <code>clusterIP</code>?</li>
<li>Question 2.3 - I read that <code>K8s</code> will create <code>endpoints</code> for each <code>service</code>. Is <code>endpoint</code> same as <code>node</code> or is it same as <code>pod</code>? If I run <code>kubectl describe pods</code> or <code>kubectl get endpoints</code>, would I get same <code>IPs</code>)?</li>
</ul>
<p>As I am on <code>GCP</code>, I can use <code>Loadbalancer</code> for <code>web</code> cluster to get an external IP. Using the external IP, the client application can access the <code>web</code> service</p>
<p>I saw this configuration for a <code>LoadBalancer</code></p>
<pre><code>spec:
  selector:
    app: MyApp
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376
  type: LoadBalancer
</code></pre>
<p>Questi</p>
<ul>
<li>Question 3 - Is it exposing an external <code>IP</code> and port <code>80</code> to outside world? What would be the value of <code>nodePort</code> in this case?</li>
</ul>

Manu Chadha
<blockquote>
<p>My question is built on the question and answers from this question -
<a href="https://stackoverflow.com/questions/41509439/whats-the-difference-between-clusterip-nodeport-and-loadbalancer-service-types">What&#39;s the difference between ClusterIP, NodePort and LoadBalancer service types in Kubernetes?</a></p>
<p>The question might not be well-formed for some of you.</p>
</blockquote>
<p>It's ok but in my opinion it's a bit too extensive for a single question and it could be posted as a few separate questions as it touches quite a few different topics.</p>
<blockquote>
<p>I am trying to understand the differences between <code>clusterIP</code>,
<code>nodePort</code> and <code>Loadbalancer</code> and when to use these with an example. I
suppose that my understanding of the following concept is correct K8s
consists of the following components</p>
<ul>
<li>Node - A VM or physical machine. Runs kubectl and docker process</li>
</ul>
</blockquote>
<p>Not <code>kubectl</code> but <code>kubelet</code>. You can check it by <code>ssh-ing</code> into your node and runnign <code>systemctl status kubelet</code>. And yes, it runs also some sort of <strong>container runtime environment</strong>. It doesn't have to be exactly <strong>docker</strong>.</p>
<blockquote>
<ul>
<li>Pod - unit which encapsulates container(s) and volumes (storage). If a pod contains multiple containers then shared volume could be the way
for process communication</li>
<li>Node can have one or multiple pods. Each pod will have its own IP</li>
</ul>
</blockquote>
<p>That's correct.</p>
<blockquote>
<ul>
<li>Cluster - replicas of a Node. Each node in a cluster will contain same pods (instances, type)</li>
</ul>
</blockquote>
<p>Not really. Kubernetes <code>nodes</code> are not different replicas. They are part of the same kubernetes cluster but they are <strong>independent instances</strong>, which are capable of running your containerized apps. In <strong>kubernetes</strong> terminology it is called a <strong>workload</strong>. Workload isn't part of kubernetes cluster, it's something that you run on it. Your <code>Pods</code> can be scheduled on different nodes and it doesn't always have to be an even distribution. Suppose you have kubernetes cluster consisting of 3 worker nodes (nodes on which workload can be scheduled as opposed to master node, that usually runs only kubernetes control plane components). If you deploy your application as a <code>Deployment</code> e.g. 5 different replicas of the same <code>Pod</code> are created. Usually they are scheduled on different nodes, but situation when <strong>node1</strong> runs 2 replicas, <strong>node2</strong> 3 replicas and <strong>node3</strong> zero replicas is perfectly possible.</p>
<p>You need to keep in mind that there are different clustering levels. You have your <strong>kubernetes cluster</strong> which basically is an environment to run your containerized workload.</p>
<p>There are also clusters within this cluster i.e. it is perfectly possible that your workload forms clusters as well e.g. you can have a database deployed as a <code>StatefulSet</code> and it can run in a cluster. In such scenario, different stateful <code>Pods</code> will form members or nodes of such cluster.</p>
<p>Even if your <code>Pods</code> don't comunicate with each other but e.g. serve exactly the same content, <code>Deployment</code> resoure makes sure that there is always certain number of replicas of such a <code>Pod</code> that is up and running. If one kubernetes node for some reason becomes unavailable, it means such <code>Pod</code> needs to be re-scheduled on one of the available nodes. So the replication of your workload isn't achieved by deploying it on different kubernetes nodes but by assuring that certain amout of replicas of a <code>Pod</code> of a certain kind is always up and running, and it may be running on the same as well as on different kubernetes nodes.</p>
<blockquote>
<p>Here is the scenario:</p>
<p>My application has a <code>web server</code> (always returning 200OK) and a
<code>database</code> (always returning the same value) for simplicity. Also, say
I am on <code>GCP</code> and I make images of <code>webserver</code> and of the <code>database</code>.
Each of these will be run in their own respective <code>pods</code> and will have
2 replicas.</p>
<p>I suppose I'll have two clusters (<code>cluster-webserver (node1-web (containing pod1-web), node2-web (containing pod2-web))</code> and
<code>cluster-database (node1-db (containing pod1-db), node2-db (containing pod2-db))</code>. Each node will have its own <code>ip</code> address (<code>node1-webip, node2-webip, node1-dbip, node2-dbip</code>)</p>
</blockquote>
<p>See above what I wrote about different clustering levels. Clusters formed by your app have nothing to do with kubernetes cluster nor its nodes. And I would say you would rather have 2 different <strong>microservices</strong> comunicating with each other and in some way also dependent on one another. But yes, you may see your database as a separate db cluster deployed within kubernetes cluster.</p>
<blockquote>
<p>A client application (browser) should be able to access the web
application from outside <code>web</code> cluster but the <code>database</code> shouldn't be
accessible from outside <code>database</code> cluster. However <code>web</code> nodes should
be able to access <code>database</code> nodes)</p>
<ul>
<li>Question 1 - Am I correct that if I create a service for <code>web</code> (<code>webServiceName</code>) and a service for <code>database</code> then by default, I'll
get only <code>clusterIP</code> and a <code>port</code> (or <code>targetPort</code>).</li>
</ul>
</blockquote>
<p>Yes, <code>ClusterIP</code> service type is often simply called a <code>Service</code> because it's the default <code>Service</code> type. If you don't specify <code>type</code> like in <a href="https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service" rel="nofollow noreferrer">this example</a>, <code>ClusterIP</code> type is created. To understand the difference between <code>port</code> and <code>targetPort</code> you can take a look at <a href="https://stackoverflow.com/a/63472670/11714114">this answer</a> or <a href="https://stackoverflow.com/a/63472670/11714114">kubernetes official docs</a>.</p>
<blockquote>
<ul>
<li>Question 1.2 - Am I correct that <code>clusterIP</code> is an <code>IP</code> assigned to a <code>pod</code>, not the <code>node</code> i.e. in my example, clusterIP gets assigned to
<code>pod1-web</code>, not <code>node1-web</code> even though <code>node1</code> has only <code>pod1</code>.</li>
</ul>
</blockquote>
<p>Basically yes. <code>ClusterIP</code> is one of the things that can be easily misunderstood as it is used to denote also a specific <code>Service</code> type, but in this context yes, it's an internal IP assigned within a kubernetes cluster to a specific resource, in this case to a <code>Pod</code>, but <code>Service</code> object has it's own Cluster IP assigned. <code>Pods</code> as part of kubernetes cluster get their own internal IPs (from kubernetes cluster perspective) - cluster IPs. Nodes can have completely different addressing scheme. They can also be private IPs but they are not cluster IPs, in other words they are not internal kubernetes cluster IPs from cluster perspective. Apart from those external IPs (from kubernetes cluster perspective), kubernetes nodes as legitimate API resources / objects have also their own Cluster IPs assigned.</p>
<p>You can check it by running:</p>
<pre><code>kubectl get nodes --output wide
</code></pre>
<p>It will show you both internal and external nodes IPs.</p>
<blockquote>
<ul>
<li>Question 1.3 - Am I correct that as <code>cluster IP</code> is accessible from only within the cluster, <code>pod1-web</code> and <code>pod2-web</code> can talk to each
other and <code>pod1-db</code> and <code>pod2-db</code> can talk to each other using
<code>clusterIP/dns:port</code> or <code>clusterIP/dns:targetPort</code> but <code>web</code> can't
talk to <code>database</code> (and vice versa) and external client can't talk to
web? Also, the <code>nodes</code> are not accessible using the <code>cluster IP</code>.</li>
</ul>
</blockquote>
<p>Yes, cluster IPs are only accessible from within the cluster. And yes, web pods and db pods can communicate with each other (typically the communication is initiated from <code>web</code> pods) provided you exposed (in your case db pods) via <code>ClusterIP</code> <code>Service</code>. As already mentioned, this type of <code>Service</code> exposes some set of <code>Pods</code> forming one microservice to some other set of <code>Pods</code> which need to comunicate with them and it exposes them only internally, within the cluster so no external client has access to them. You expose your <code>Pods</code> externally by using <code>LoadBalancer</code>, <code>NodePort</code> or in many scenarios via <code>ingress</code> (which under the hood also uses loadbalancer).</p>
<p>this fragment is not very clear to me:</p>
<blockquote>
<p>but <code>web</code> can't
talk to <code>database</code> (and vice versa) and external client can't talk to
web? Also, the <code>nodes</code> are not accessible using the <code>cluster IP</code>.</p>
</blockquote>
<p>If you expose your db via <code>Service</code> to be accessible from <code>web</code> <code>Pods</code>, they will have access to it. And if your web <code>Pods</code> are exposed to the external world e.g. via <code>LoadBalancer</code> or <code>NodePort</code>, they will be accessible from outside. And yes, nodes won't be accessible from outside by their cluster IPs as they are private internal IPs of a kubernetes cluster.</p>
<blockquote>
<ul>
<li>Question 1.4 - Am I correct that <code>dns</code> i.e. <code>servicename.namespace.svc.cluster.local</code> would map the <code>clusterIP</code>?</li>
</ul>
</blockquote>
<p>Yes, specifically <code>cluster IP</code> of this <code>Service</code>. More on that you can find <a href="https://kubernetes.io/docs/concepts/services-networking/dns-pod-service" rel="nofollow noreferrer">here</a>.</p>
<blockquote>
<ul>
<li>Question 1.5 - For which type of applications I might use only <code>clusterIP</code>? Where multiple instances of an application need to
communicate with each other (eg master-slave configuration)?</li>
</ul>
</blockquote>
<p>For something that doesn't need to be exposed externally, like some backend services, that are accessible not directly from outside but through some frontend <code>Pods</code> which process external requests and pass them to the backend afterwards. It may be also used for database pods which practically never should be accessed directly from outside.</p>
<blockquote>
<p>If I use <code>nodePort</code> then <code>K8s</code> will open a <code>port</code> on each of the
<code>node</code> and will forward <code>nodeIP/nodePort</code> to <code>cluster IP (on pod)/Cluster Port</code></p>
</blockquote>
<p>Yes, in <code>NodePort</code> Service configuration this destination port exposed by a <code>Pod</code> is called <code>targetPort</code>. Somewhere in between there is also a <code>port</code>, wchich refers to a port of the <code>Service</code> itself. So the <code>Service</code> has its <code>ClusterIP</code> (different then backend <code>Pods</code> IPs) and its port which usually is the same as <code>targetPort</code> (targetPort defaults to the value set for <code>port</code>) but can be set to a different value.</p>
<blockquote>
<ul>
<li>Question 2 - Can <code>web</code> nodes now access <code>database</code> nodes using <code>nodeIP:nodePort</code> which will route the traffic to <code>database's</code>
<code>clusterIP (on pod):clusterport/targertPort</code>?</li>
</ul>
</blockquote>
<p>I think you've mixed it up a bit. If <code>web</code> is something external to the kubernetes cluster, it might have sense to access <code>Pods</code> deployed on kubernetes cluster via <code>nodeIP:nodePort</code> but if it's part of the same kubernetes cluster, it can use simple <code>ClusterIP</code> <code>Service</code>.</p>
<blockquote>
<p>( I have read that
clusterIP/dns:nodePort will not work).</p>
</blockquote>
<p>From the external world of cours it won't work as <code>Cluster IPs</code> are not accessible from outside, they are internal kubernetes IPs. But from within the cluster ? It's perfectly possible. As I said in different part of my answer, kubernetes nodes have also their cluster IPs and it's perfectly possible to access your app on <code>nodePort</code> but from within the cluster i.e. from some other <code>Pod</code>. So when you look at internal (cluster) IP addresses of the nodes in my example it is also perfectly possible to run:</p>
<pre><code>root@nginx-deployment-85ff79dd56-5lhsk:/# curl http://10.164.0.8:32641
&lt;!DOCTYPE html&gt;
&lt;html&gt;
&lt;head&gt;
&lt;title&gt;Welcome to nginx!&lt;/title&gt;
...
</code></pre>
<blockquote>
<ul>
<li>Question 2.1 - How do I get a <code>node's</code> <code>IP</code>? Is <code>nodeIP</code> the <code>IP</code> I'll get when I run <code>describe pods</code> command?</li>
</ul>
</blockquote>
<p>To check IPs of your nodes run:</p>
<pre><code>kubectl get nodes --output wide
</code></pre>
<p>It will show you both their internal (yes, nodes also have their ClusterIPs!) and external IPs.</p>
<blockquote>
<ul>
<li>Question 2.2 - Is there a <code>dns</code> equivalent for the <code>node IP</code> as <code>node IP</code> could change during failovers. Or does <code>dns</code> now resolve to
the node's IP instead of <code>clusterIP</code>?</li>
</ul>
</blockquote>
<p>No, there isn't. Take a look at <a href="https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#what-things-get-dns-names" rel="nofollow noreferrer">What things get DNS names?</a></p>
<blockquote>
<ul>
<li>Question 2.3 - I read that <code>K8s</code> will create <code>endpoints</code> for each <code>service</code>. Is <code>endpoint</code> same as <code>node</code> or is it same as <code>pod</code>? If I
run <code>kubectl describe pods</code> or <code>kubectl get endpoints</code>, would I get
same <code>IPs</code>)?</li>
</ul>
</blockquote>
<p>No, <code>endpoints</code> is another type of kubernetes API object / resource.</p>
<pre><code>$ kubectl api-resources | grep endpoints
endpoints                         ep                                          true         Endpoints
</code></pre>
<p>If you run:</p>
<pre><code>kubectl explain endpoints
</code></pre>
<p>you will get it's detailed description:</p>
<pre><code>KIND:     Endpoints
VERSION:  v1

DESCRIPTION:
     Endpoints is a collection of endpoints that implement the actual service.
     Example: Name: &quot;mysvc&quot;, Subsets: [
         {
           Addresses: [{&quot;ip&quot;: &quot;10.10.1.1&quot;}, {&quot;ip&quot;: &quot;10.10.2.2&quot;}],
           Ports: [{&quot;name&quot;: &quot;a&quot;, &quot;port&quot;: 8675}, {&quot;name&quot;: &quot;b&quot;, &quot;port&quot;: 309}]
         },
         {
           Addresses: [{&quot;ip&quot;: &quot;10.10.3.3&quot;}],
           Ports: [{&quot;name&quot;: &quot;a&quot;, &quot;port&quot;: 93}, {&quot;name&quot;: &quot;b&quot;, &quot;port&quot;: 76}]
         },
     ]
</code></pre>
<p>Usually you don't have to worry about creating <code>endpoints</code> resource as it is created automatically. So to answer your question, <code>endpoints</code> stores information about <code>Pods</code> IPs and keeps track on them as <code>Pods</code> can be destroyed and recreated and their IPs are subject to change. For a <code>Service</code> to keep routing the traffic properly, although <code>Pods</code> IPs change, an object like <code>endpoints</code> must exist which keeps track of those IPs.</p>
<p>You can easily check it by yourself. Simply create a deployment, consisting of 3 <code>Pods</code> and expose it as a simple <code>ClusterIP</code> <code>Service</code>. Check its <code>endpoint</code> object. Then delete one <code>Pod</code>, verify its IP has changed and check again its <code>endpoint</code> object. You can do it by running:</p>
<pre><code>kubectl get ep &lt;endpoints-object-name&gt; -o yaml
</code></pre>
<p>or</p>
<pre><code>kubectl describe ep &lt;endpoints-object-name&gt;
</code></pre>
<p>So basically different <em>endpoints</em> (as many as backend <code>Pods</code> exposed by a certain <code>Service</code>) are internal (ClusterIP) addresses of <code>Pods</code> exposed by the <code>Service</code> but <code>endpoints</code> <strong>object / API resource</strong> is a single kubernetes <strong>resource</strong> that keeps track of those <em>endpoints</em>. I hope this is clear.</p>
<blockquote>
<p>As I am on <code>GCP</code>, I can use <code>Loadbalancer</code> for <code>web</code> cluster to get an
external IP. Using the external IP, the client application can access
the <code>web</code> service</p>
<p>I saw this configuration for a <code>LoadBalancer</code></p>
<pre><code>spec:
  selector:
    app: MyApp
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376
  type: LoadBalancer
</code></pre>
<p>Questi</p>
<ul>
<li>Question 3 - Is it exposing an external <code>IP</code> and port <code>80</code> to outside world? What would be the value of <code>nodePort</code> in this case?</li>
</ul>
</blockquote>
<p>Yes, under the hood a call to GCP API is made so that external http/https loadbalancer with a public IP is created.</p>
<p>Suppose you have a <code>Deployment</code> called <code>nginx-deployment</code>. If you run:</p>
<pre><code>kubectl expose deployment nginx-deployment --type LoadBalancer
</code></pre>
<p>It will create a new <code>Service</code> of <code>LoadBalancer</code> type. If you then run:</p>
<pre><code>kubectl get svc
</code></pre>
<p>you will see your <code>LoadBalancer</code> <code>Service</code> has both external IP and cluster IP assigned.</p>
<pre><code>NAME               TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)        AGE
nginx-deployment   LoadBalancer   10.3.248.43   &lt;some external ip&gt; 80:32641/TCP   102s
</code></pre>
<p>If you run:</p>
<pre><code>$ kubectl get svc nginx-deployment
NAME               TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)        AGE
nginx-deployment   LoadBalancer   10.3.248.43   &lt;some external ip&gt;   80:32641/TCP 👈  16m
</code></pre>
<p>You'll notice that <code>nodePort</code> value for this <code>Service</code> has been also set, in this case to <code>32641</code>. If you want to dive into it even deeper, run:</p>
<pre><code>kubectl get svc nginx-deployment -o yaml
</code></pre>
<p>and you will see it in this section:</p>
<pre><code>...
spec:
  clusterIP: 10.3.248.43
  externalTrafficPolicy: Cluster
  ports:
  - nodePort: 32641 👈
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  sessionAffinity: None
  type: LoadBalancer 👈
...
</code></pre>
<p>As you can see although the <code>Service</code> type is <code>LoadBalancer</code> it also has its <code>nodePort</code> value set. And you can test that it works by accessing your <code>Deployment</code> using this port, but not on the <code>IP</code> of the <code>LoadBalancer</code> but on <code>IPs</code> of your nodes. I know it may seem pretty confusing as <code>LoadBalancer</code> and <code>NodePort</code> are two different <code>Service</code> types. <code>LB</code> needs to distrute the incoming traffic to some backend <code>Pods</code> (e.g. managed by a <code>Deployment</code>) and needs this <code>nodePort</code> value set in its own specification to be able to route the traffic to <code>Pods</code> scheduled on different nodes. I hope this is a bit clearer now.</p>
