<p>I have the following local 2-node kubernetes cluster-</p>
<pre><code>$ kubectl get nodes -o wide
NAME   STATUS   ROLES                  AGE     VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
srl1   Ready    control-plane,master   2d18h   v1.21.2   xxx.xxx.12.58   &lt;none&gt;        Ubuntu 20.04.2 LTS   5.4.0-80-generic   docker://20.10.7
srl2   Ready    &lt;none&gt;                 2d18h   v1.21.3   xxx.xxx.80.72   &lt;none&gt;        Ubuntu 18.04.2 LTS   5.4.0-80-generic   docker://20.10.2

</code></pre>
<p>I am trying to deploy an application on using a cluster creation python scirpt(<a href="https://github.com/hydro-project/cluster/blob/master/hydro/cluster/create_cluster.py" rel="nofollow noreferrer">https://github.com/hydro-project/cluster/blob/master/hydro/cluster/create_cluster.py</a>)</p>
<p>When it tries to create a routing node using           <code>apps_client.create_namespaced_daemon_set(namespace=util.NAMESPACE, body=yml)</code> it is expected that it should create a single pod from the routing-ds.yaml (given below) file and assign it to the routing daemonset (<strong>kind</strong>). However as you can see, <strong>it is creating two routing pods instead of one</strong> on every physical node. (FYI-my master can schedule pod)</p>
<pre><code>akazad@srl1:~/hydro-project/cluster$ kubectl get all -A -o wide
NAMESPACE        NAME                                           READY   STATUS    RESTARTS   AGE     IP               NODE   NOMINATED NODE   READINESS GATES
default          pod/management-pod                             1/1     Running   0          25m     192.168.190.77   srl2   &lt;none&gt;           &lt;none&gt;
default          pod/monitoring-pod                             1/1     Running   0          25m     192.168.120.71   srl1   &lt;none&gt;           &lt;none&gt;
default          pod/routing-nodes-9q7dr                        1/1     Running   0          24m    xxx.xxx.12.58    srl1   &lt;none&gt;           &lt;none&gt;
default          pod/routing-nodes-kfbnv                        1/1     Running   0          24m    xxx.xxx.80.72    srl2   &lt;none&gt;           &lt;none&gt;
kube-system      pod/calico-kube-controllers-7676785684-tpz7q   1/1     Running   0          2d19h   192.168.120.65   srl1   &lt;none&gt;           &lt;none&gt;
kube-system      pod/calico-node-lnxtb                          1/1     Running   0          2d19h   xxx.xxx.12.58    srl1   &lt;none&gt;           &lt;none&gt;
kube-system      pod/calico-node-mdvpd                          1/1     Running   0          2d19h   xxx.xxx.80.72    srl2   &lt;none&gt;           &lt;none&gt;
kube-system      pod/coredns-558bd4d5db-vfghf                   1/1     Running   0          2d19h   192.168.120.66   srl1   &lt;none&gt;           &lt;none&gt;
kube-system      pod/coredns-558bd4d5db-x7jhj                   1/1     Running   0          2d19h   xxx.xxx.120.67   srl1   &lt;none&gt;           &lt;none&gt;
kube-system      pod/etcd-srl1                                  1/1     Running   0          2d19h   xxx.xxx.12.58    srl1   &lt;none&gt;           &lt;none&gt;
kube-system      pod/kube-apiserver-srl1                        1/1     Running   0          2d19h   xxx.xxx.12.58    srl1   &lt;none&gt;           &lt;none&gt;
kube-system      pod/kube-controller-manager-srl1               1/1     Running   0          2d19h  xxx.xxx.12.58    srl1   &lt;none&gt;           &lt;none&gt;
kube-system      pod/kube-proxy-l8fds                           1/1     Running   0          2d19h   xxx.xxx.12.58    srl1   &lt;none&gt;           &lt;none&gt;
kube-system      pod/kube-proxy-szrng                           1/1     Running   0          2d19h   xxx.xxx.80.72    srl2   &lt;none&gt;           &lt;none&gt;
kube-system      pod/kube-scheduler-srl1                        1/1     Running   0          2d19h   xxx.xxx.12.58    srl1   &lt;none&gt;           &lt;none&gt;
metallb-system   pod/controller-6b78bff7d9-t7gjr                1/1     Running   0          2d19h   192.168.190.65   srl2   &lt;none&gt;           &lt;none&gt;
metallb-system   pod/speaker-qsqnc                              1/1     Running   0          2d19h   xxx.xxx.12.58    srl1   &lt;none&gt;           &lt;none&gt;
metallb-system   pod/speaker-s4pp8                              1/1     Running   0          2d19h   xxx.xxx.80.72    srl2   &lt;none&gt;           &lt;none&gt;

NAMESPACE     NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE     SELECTOR
default       service/kubernetes   ClusterIP   10.96.0.1    &lt;none&gt;        443/TCP                  2d19h   &lt;none&gt;
kube-system   service/kube-dns     ClusterIP   10.96.0.10   &lt;none&gt;        53/UDP,53/TCP,9153/TCP   2d19h   k8s-app=kube-dns

NAMESPACE        NAME                           DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE     CONTAINERS          IMAGES                            SELECTOR
default          daemonset.apps/routing-nodes   2         2         2       2            2           &lt;none&gt;                   24m     routing-container   akazad1/srlanna:v2                role=routing
kube-system      daemonset.apps/calico-node     2         2         2       2            2           kubernetes.io/os=linux   2d19h   calico-node         calico/node:v3.14.2               k8s-app=calico-node
kube-system      daemonset.apps/kube-proxy      2         2         2       2            2           kubernetes.io/os=linux   2d19h   kube-proxy          k8s.gcr.io/kube-proxy:v1.21.3     k8s-app=kube-proxy
metallb-system   daemonset.apps/speaker         2         2         2       2            2           kubernetes.io/os=linux   2d19h   speaker             quay.io/metallb/speaker:v0.10.2   app=metallb,component=speaker


</code></pre>
<p>However, when it is directly creating pods from the management-pod.yaml (given below), it is creating one as expected.</p>
<p><strong>Why the dasemonset is creating two pods instead of one?</strong></p>
<h2>Code segment where it is supposed to create a daemonset of type routing node</h2>
<pre><code>for i in range(len(kinds)):
        kind = kinds[i]

        # Create should only be true when the DaemonSet is being created for the
        # first time -- i.e., when this is called from create_cluster. After that,
        # we can basically ignore this because the DaemonSet will take care of
        # adding pods to created nodes.
        if create:
            fname = 'yaml/ds/%s-ds.yml' % kind
            yml = util.load_yaml(fname, prefix)

            for container in yml['spec']['template']['spec']['containers']:
                env = container['env']

                util.replace_yaml_val(env, 'ROUTING_IPS', route_str)
                util.replace_yaml_val(env, 'ROUTE_ADDR', route_addr)
                util.replace_yaml_val(env, 'SCHED_IPS', sched_str)
                util.replace_yaml_val(env, 'FUNCTION_ADDR', function_addr)
                util.replace_yaml_val(env, 'MON_IPS', mon_str)
                util.replace_yaml_val(env, 'MGMT_IP', management_ip)
                util.replace_yaml_val(env, 'SEED_IP', seed_ip)

            apps_client.create_namespaced_daemon_set(namespace=util.NAMESPACE,
                                                     body=yml)

        # Wait until all pods of this kind are running
        res = []
        while len(res) != expected_counts[i]:
            res = util.get_pod_ips(client, 'role='+kind, is_running=True)

        pods = client.list_namespaced_pod(namespace=util.NAMESPACE,
                                          label_selector='role=' +
                                          kind).items

        created_pods = get_current_pod_container_pairs(pods)

</code></pre>
<p>I have removed the <strong>nodeSelector</strong> from all the yaml files as I am running it on bare-metal cluster.</p>
<h2>1 routing-ds.yaml</h2>
<pre><code> 14 
 15 apiVersion: apps/v1
 16 kind: DaemonSet
 17 metadata:
 18   name: routing-nodes
 19   labels:
 20     role: routing
 21 spec:
 22   selector:
 23     matchLabels:
 24       role: routing
 25   template:
 26     metadata:
 27       labels:
 28         role: routing
 29     spec:
 30       #nodeSelector:
 31        # role: routing
 32      
 33       hostNetwork: true
 34       containers:
 35       - name: routing-container
 36         image: akazad1/srlanna:v2
 37         env:
 38         - name: SERVER_TYPE
 39           value: r
 40         - name: MON_IPS
 41           value: MON_IPS_DUMMY
 42         - name: REPO_ORG
 43           value: hydro-project
 44         - name: REPO_BRANCH
 45           value: master
</code></pre>
<h2>2 management-pod.yaml</h2>
<pre><code> 15 apiVersion: v1 
 16 kind: Pod 
 17 metadata: 
 18   name: management-pod 
 19   labels: 
 20     role: management 
 21 spec: 
 22   restartPolicy: Never 
 23   containers: 
 24   - name: management-container 
 25     image: hydroproject/management 
 26     env: 
 27             #- name: AWS_ACCESS_KEY_ID 
 28             #value: ACCESS_KEY_ID_DUMMY 
 29             #- name: AWS_SECRET_ACCESS_KEY 
 30             #value: SECRET_KEY_DUMMY 
 31             #- name: KOPS_STATE_STORE 
 32             # value: KOPS_BUCKET_DUMMY 
 33     - name: HYDRO_CLUSTER_NAME 
 34       value: CLUSTER_NAME 
 35     - name: REPO_ORG 
 36       value: hydro-project 
 37     - name: REPO_BRANCH 
 38       value: master 
 39     - name: ANNA_REPO_ORG 
 40       value: hydro-project 
 41     - name: ANNA_REPO_BRANCH 
 42       value: master 
 43       # nodeSelector: 
 44       #role: general

</code></pre>

<p>May you have misunderstanding you have to use the <code>kind: deployment</code> if you want to manage the replicas (PODs - 1,2,3...n) on Kubernetes.</p>
<p>Daemon set behavior is like it will run the POD on each available node in the cluster.</p>
<p>So inside your cluster, there are <strong>two nodes</strong> so <strong>daemon set</strong> will run the POD on each <strong>available node</strong>. If you will increase the <strong>Node</strong> <strong>deamon set</strong> will auto-create the POD on that node also.</p>
<p><strong>kind: Pod</strong></p>
<p>will create the single POD only which is its default behavior.</p>
<p>The following are some of the Kubernetes Objects:</p>
<pre><code>pods
ReplicationController (Manages Pods)
Deployment (Manages Pods)
StatefulSets
DaemonSets
</code></pre>
<p>You can read more at :  <a href="https://chkrishna.medium.com/kubernetes-objects-e0a8b93b5cdc" rel="nofollow noreferrer">https://chkrishna.medium.com/kubernetes-objects-e0a8b93b5cdc</a></p>
<p>Official document : <a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/" rel="nofollow noreferrer">https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/</a></p>
<p>If you want to manage <strong>POD</strong> using any type of controller <code>kind: deployment</code> is best. you can scale the replicas and scale down. You can also mention the replicas in YAML 1,2,3 and that way it will be running on cluster based on number.</p>
