What is the difference between a statefulset and a daemonset in Kuberneetes
https://cloudyuga.guru/hands_on_lab/Deployments-vs-StatefulSets-vs-DaemonSets
Kubernetes: Deployments vs StatefulSets vs DaemonSets
2022-01-27
None
<div><div><p><a href="https://kubernetes.io/docs/home/">Kubernetes</a> (K8s) is one of the popular <b>open-source container orchestration systems</b> for automating deployment, scaling, and management of containerized applications.</p><p>Kubernetes provides a basic resource called <a href="https://kubernetes.io/docs/concepts/workloads/pods/">Pod</a>. A pod is the <b>smallest deployable unit</b> in Kubernetes which has a <b>collection of containers</b>. As a pod can have one or more containers so different configurations can be passed to the container(s) using the pod’s configuration e.g. passing environment variables, mounting volumes, having health checks, etc.</p><p>In this hands-on lab, we will be going through three different ways to deploy applications (pods) on <a href="https://kind.sigs.k8s.io/">kind </a>multi-node Kubernetes cluster using three different Kubernetes resources such as </p><ol><li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">Deployments</a></li><li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/">StatefulSets</a></li><li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/">DaemonSets</a></li></ol><ul><li>Configure a multi-node kind cluster with two worker and one master node.</li></ul><div><pre><code>kind create cluster --config=kind-config.yaml</code></pre></div><p>It will take a few minutes to get the kind cluster configured and as it gets configured, the <b>Ready status</b> of nodes can be verified by</p><p>In the <a href="https://cloudyuga.guru/hands_on_lab/nfs-kind/">NFS hands-on lab</a>, we have seen how to set up <code>ReadWriteMany</code> access mode for persistent volume in a kind cluster as it comes with a standard storage class that has <code>ReadWriteOnce</code>access mode.  </p><ul><li>Configure NFS storage class with <code>ReadWriteMany</code> access-mode</li></ul><div><pre><code>kubectl apply -f setup_nfs.yaml</code></pre></div><ul><li>Check the NFS storage class</li></ul><p>For detailed differences between the three resources, we will be deploying a <a href="https://github.com/kahootali/counter-app">sample counter app</a>, which <b>logs and increments the count from a counter file like 1,2,3,….</b></p><p><b></b>For this, we will be using a <b>counter file from a Persistent Volume</b> to detail the differences between the <b>Deployments, StatefulSets, and DaemonSets</b>. </p><div><pre><code>kubectl apply -f counter-pvc.yaml</code></pre></div><h2>Deployments</h2><p><a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">Deployment </a>is the <i>easiest and most used resource for deploying your application.</i> <b>It is a Kubernetes controller that matches the current state of your cluster to the desired state</b> mentioned in the Deployment manifest. </p><p>For example, If you create a <b>deployment with 1 replica</b>, it will check that the <b>desired state of ReplicaSet is 1 and current state is 0</b>, so it will create a ReplicaSet, which will further create the pod.</p><p><i> If you create a deployment with name counter, it will create a ReplicaSet with name counter-&lt;replica-set-id&gt;, which will further create a Pod with name counter-&lt;replica-set-&gt;-&lt;pod-id&gt;.</i></p><p>Deployments are <b>usually used for stateless applications</b>. However, you can <i>save the state of deployment by attaching a Persistent Volume to it and making it statefu</i>l, but all the pods of deployment will be sharing the <b>same Volume</b>, and data across all of them will be the same which can <i>lead to data inconsistency</i>.</p><p>For deploying the <a href="https://github.com/kahootali/counter-app">sample counter app</a> using a deployment, we will be using the following manifest </p><div><pre><code>kubectl apply -f counter-deployment.yaml</code></pre></div><div><pre><code>kubectl logs -l app=counter</code></pre></div><figure><figcaption><em>Figure 1: Persistence in deployments</em></figcaption></figure><p>Now if you scale up the deployment to 3 by running</p><div><pre><code>kubectl scale deployment counter-deploy --replicas=3</code></pre></div><p>As a new pods gets created, if we check the logs of the new pods, their logs will not start from 1 and the counter index be would random. This is because <b>pods in the deployment are sharing the same volume</b>, and data is shared across all the pods.</p><p>Also if we check the Persistent Volume Claim(PVC), only <b>one PVC will be created </b>that both the pods will be sharing so it can cause Data Inconsistency.</p><figure><figcaption><em>Figure 2: Data inconsistency in deployments sharing single volume</em></figcaption></figure><p><b>Deployments create a ReplicaSet</b> which then creates a Pod so whenever you update the deployment using RollingUpdate(default) strategy, a new ReplicaSet is created and the Deployment moves the Pods from the old ReplicaSet to the new one at a controlled rate.</p><p><b> Rolling Update means that the previous ReplicaSet doesn’t scale to 0 unless the new ReplicaSet is up &amp; running ensuring 100% uptime.</b> If an error occurs while updating, the new ReplicaSet will never be in Ready state, so the old ReplicaSet will not terminate again ensuring 100% uptime in case of a failed update. In Deployments, you can also manually roll back to a previous ReplicaSet, if needed in case your new feature is not working as expected.</p><p>Let's remove the Deployment Object</p><div><pre><code>kubectl delete deployment counter-deploy</code></pre></div><h2>StatefulSets</h2><p><a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/">StatefulSet</a> is a Kubernetes resource used to <b>manage stateful applications</b>. It manages the deployment and scaling of a set of Pods and provides the guarantee about the <b>ordering and uniqueness of these Pods.</b></p><p>StatefulSet is also a Controller but unlike Deployments, it doesn’t create ReplicaSet rather itself but <b>creates the Pod with a unique naming convention</b>. e.g. If you create a StatefulSet with <i>name counter</i>, it will create a pod with name <i>counter-0,</i> and for multiple replicas of a stateful set, their names will increment like <i>counter-0, counter-1, counter-2</i>, etc</p><p><b>Every replica of a stateful set will have its own state, and each of the pods will be creating its own PVC(Persistent Volume Claim).</b> </p><p>For deploying the <a href="https://github.com/kahootali/counter-app">sample counter app</a> using a stateful set, we will be using the following manifest</p><div><pre><code>kubectl apply -f counter-statefulset.yaml</code></pre></div><p>Now if you scale up the stateful set to 3 replicas by running</p><div><pre><code>kubectl scale statefulsets counter --replicas=3</code></pre></div><p>It will first create a new pod, and once that <b>pod is ready</b>, then another pod <code>counter-2</code>will be created. The <b>new pods will have their own volume</b> and if you check the logs, the <i>count will again start from 1 for the new pods</i>, unlike in Deployments.</p><p>Also if we check the Persistent Volume Claim(PVC) once again then there will be three PVCs that will be created. </p><figure><figcaption><em>Figure 3: Persistence in statefulsets as each having its own volume</em></figcaption></figure><p><b>StatefulSets don’t create ReplicaSet </b>so you cant roll back a StatefulSet to a previous version. <i>You can only delete or scale up/down the Statefulse</i>t. If you update a StatefulSet, it also performs RollingUpdate i.e. one replica pod will go down and the updated pod will come up.</p><p>StatefulSets are useful in the case of Databases especially when we need <b>Highly Available Databases in production</b> as we create a cluster of Database replicas with one being the primary replica and others being the secondary replicas. The primary will be responsible for read/write operations and secondary for read-only operations and they will be syncing data with the primary one.</p><p>Let's remove the StatefuleSet Object</p><div><pre><code>kubectl delete statefulset counter</code></pre></div><h2>DaemonSet</h2><p>A <a href="https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/">DaemonSet</a> is a controller that ensures that the <b>pod runs on all the nodes of the cluster</b>. If a node is added/removed from a cluster, DaemonSet automatically adds/deletes the pod.</p><p>Some typical use cases of a DaemonSet is to run cluster level applications like:</p><ul><li>Monitoring Exporters: You would want to monitor all the nodes of your cluster so you will need to run a monitor on all the nodes of the cluster like NodeExporter.</li><li>Logs Collection Daemon: You would want to export logs from all nodes so you would need a DaemonSet of log collector like Fluentd to export logs from all your nodes.</li></ul><p><i>However, Daemonset automatically doesn’t run on nodes that have a taint e.g. Master. You will have to specify the tolerations for it on the pod.</i></p><p><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/">Taints</a><b> are a way of telling the nodes to repel the pods</b> i.e. no pods will be scheduled on this node unless the pod tolerates the node with the same toleration. The master node is already tainted by:</p><p>This means it will repel all pods that do not tolerate this taint. So for DaemonSet to run on all nodes, you would have to add the following tolerations on DaemonSet which means that it should tolerate all nodes.</p><p>For deploying the <a href="https://github.com/kahootali/counter-app">sample counter app</a> using a DaemonSet, we will be using the following manifest.</p><p>When DaemonSet is deployed, it will <b>create pods equal to the number of nodes</b>. In terms of behavior, it will behave the same as Deployments i.e.<b> all pods will share the same Persistent Volume.</b></p><div><pre><code>kubectl apply -f counter-daemonset.yaml</code></pre></div><div><pre><code>kubectl logs &lt;counter-podname&gt;</code></pre></div><p>Ongoing through the logs of a pod of DaemonSet, one can observe that the <b>logs are not in order, meaning that all pods are sharing the same volum</b>e. Also, only one PVC will be created that all pods will be sharing.</p><figure><figcaption><em>Figure 4: Daemonset run replica on each node sharing volume</em></figcaption></figure><p>If you update a DaemonSet, it also performs RollingUpdate i.e. <b>one pod will go down and the updated pod will come up</b>. If an error occurs while updating, so only one pod will be down, all other pods will still be up, running on the previous stable version. Unlike Deployments, you cannot roll back your DaemonSet to a previous version.</p><p>Let's remove the DaemonSet Object</p><div><pre><code>kubectl delete daemonset counter-app</code></pre></div><h2>Conclusion</h2><p>In this hands-on lab, we have seen different ways to deploy pods with the help of different Kubernetes resources.</p></div>||||I|||| * Hands-on Lab
 * Kubernetes: Deployments vs StatefulSets vs DaemonSets

 Table of Contents

 * Deployments
 * StatefulSets
 * DaemonSet
 * Conclusion

 Lab Infrastructure Sponsored By

 Kubernetes: Deployments vs StatefulSets vs DaemonSets

 27 January 2022
 daemonset
 deployment
 statefuleset
 kubernetes

 To understand different workload resources in Kubernetes

 Kubernetes (K8s) is one of the popular open-source container orchestration systems for automating deployment, scaling, and management of containerized applications.

 Kubernetes provides a basic resource called Pod. A pod is the smallest deployable unit in Kubernetes which has a collection of containers. As a pod can have one or more containers so different configurations can be passed to the container(s) using the pod’s configuration e.g. passing environment variables, mounting volumes, having health checks, etc.

 In this hands-on lab, we will be going through three different ways to deploy applications (pods) on kind multi-node Kubernetes cluster using three different Kubernetes resources such as

 1. Deployments
 2. StatefulSets
 3. DaemonSets
 * Configure a multi-node kind cluster with two worker and one master node.
 kind create cluster --config=kind-config.yaml

 It will take a few minutes to get the kind cluster configured and as it gets configured, the Ready status of nodes can be verified by

 kubectl get nodes

 In the NFS hands-on lab, we have seen how to set up ReadWriteMany access mode for persistent volume in a kind cluster as it comes with a standard storage class that has ReadWriteOnceaccess mode.

 * Configure NFS storage class with ReadWriteMany access-mode
 kubectl apply -f setup_nfs.yaml
 * Check the NFS storage class
 kubectl get storageclass

 For detailed differences between the three resources, we will be deploying a sample counter app, which logs and increments the count from a counter file like 1,2,3,….

 For this, we will be using a counter file from a Persistent Volume to detail the differences between the Deployments, StatefulSets, and DaemonSets.

 kubectl apply -f counter-pvc.yaml

 Deployments

 Deployment is the easiest and most used resource for deploying your application. It is a Kubernetes controller that matches the current state of your cluster to the desired state mentioned in the Deployment manifest.

 For example, If you create a deployment with 1 replica, it will check that the desired state of ReplicaSet is 1 and current state is 0, so it will create a ReplicaSet, which will further create the pod.

 If you create a deployment with name counter, it will create a ReplicaSet with name counter-<replica-set-id>, which will further create a Pod with name counter-<replica-set->-<pod-id>.

 Deployments are usually used for stateless applications. However, you can save the state of deployment by attaching a Persistent Volume to it and making it stateful, but all the pods of deployment will be sharing the same Volume, and data across all of them will be the same which can lead to data inconsistency.

 For deploying the sample counter app using a deployment, we will be using the following manifest

 kubectl apply -f counter-deployment.yaml
 kubectl get pods -o wide
 kubectl logs -l app=counter

 Figure 1: Persistence in deployments

 Now if you scale up the deployment to 3 by running

 kubectl scale deployment counter-deploy --replicas=3

 As a new pods gets created, if we check the logs of the new pods, their logs will not start from 1 and the counter index be would random. This is because pods in the deployment are sharing the same volume, and data is shared across all the pods.

 kubectl get pods -o wide
 kubectl logs <pod_name>

 Also if we check the Persistent Volume Claim(PVC), only one PVC will be created that both the pods will be sharing so it can cause Data Inconsistency.

 kubectl get pvc

 Figure 2: Data inconsistency in deployments sharing single volume

 Deployments create a ReplicaSet which then creates a Pod so whenever you update the deployment using RollingUpdate(default) strategy, a new ReplicaSet is created and the Deployment moves the Pods from the old ReplicaSet to the new one at a controlled rate.

 Rolling Update means that the previous ReplicaSet doesn’t scale to 0 unless the new ReplicaSet is up & running ensuring 100% uptime. If an error occurs while updating, the new ReplicaSet will never be in Ready state, so the old ReplicaSet will not terminate again ensuring 100% uptime in case of a failed update. In Deployments, you can also manually roll back to a previous ReplicaSet, if needed in case your new feature is not working as expected.

 Let's remove the Deployment Object

 kubectl delete deployment counter-deploy

 StatefulSets

 StatefulSet is a Kubernetes resource used to manage stateful applications. It manages the deployment and scaling of a set of Pods and provides the guarantee about the ordering and uniqueness of these Pods.

 StatefulSet is also a Controller but unlike Deployments, it doesn’t create ReplicaSet rather itself but creates the Pod with a unique naming convention. e.g. If you create a StatefulSet with name counter, it will create a pod with name counter-0, and for multiple replicas of a stateful set, their names will increment like counter-0, counter-1, counter-2, etc

 Every replica of a stateful set will have its own state, and each of the pods will be creating its own PVC(Persistent Volume Claim).

 For deploying the sample counter app using a stateful set, we will be using the following manifest

 kubectl apply -f counter-statefulset.yaml
 kubectl get pods -o wide
 kubectl logs counter-0

 Now if you scale up the stateful set to 3 replicas by running

 kubectl scale statefulsets counter --replicas=3
 kubectl get pods -o wide

 It will first create a new pod, and once that pod is ready, then another pod counter-2will be created. The new pods will have their own volume and if you check the logs, the count will again start from 1 for the new pods, unlike in Deployments.

 kubectl logs counter-1
 kubectl logs counter-2

 Also if we check the Persistent Volume Claim(PVC) once again then there will be three PVCs that will be created.

 kubectl get pvc

 Figure 3: Persistence in statefulsets as each having its own volume

 StatefulSets don’t create ReplicaSet so you cant roll back a StatefulSet to a previous version. You can only delete or scale up/down the Statefulset. If you update a StatefulSet, it also performs RollingUpdate i.e. one replica pod will go down and the updated pod will come up.

 StatefulSets are useful in the case of Databases especially when we need Highly Available Databases in production as we create a cluster of Database replicas with one being the primary replica and others being the secondary replicas. The primary will be responsible for read/write operations and secondary for read-only operations and they will be syncing data with the primary one.

 Let's remove the StatefuleSet Object

 kubectl delete statefulset counter

 DaemonSet

 A DaemonSet is a controller that ensures that the pod runs on all the nodes of the cluster. If a node is added/removed from a cluster, DaemonSet automatically adds/deletes the pod.

 Some typical use cases of a DaemonSet is to run cluster level applications like:

 * Monitoring Exporters: You would want to monitor all the nodes of your cluster so you will need to run a monitor on all the nodes of the cluster like NodeExporter.
 * Logs Collection Daemon: You would want to export logs from all nodes so you would need a DaemonSet of log collector like Fluentd to export logs from all your nodes.

 However, Daemonset automatically doesn’t run on nodes that have a taint e.g. Master. You will have to specify the tolerations for it on the pod.

 Taints are a way of telling the nodes to repel the pods i.e. no pods will be scheduled on this node unless the pod tolerates the node with the same toleration. The master node is already tainted by:

 This means it will repel all pods that do not tolerate this taint. So for DaemonSet to run on all nodes, you would have to add the following tolerations on DaemonSet which means that it should tolerate all nodes.

 For deploying the sample counter app using a DaemonSet, we will be using the following manifest.

 When DaemonSet is deployed, it will create pods equal to the number of nodes. In terms of behavior, it will behave the same as Deployments i.e. all pods will share the same Persistent Volume.

 kubectl apply -f counter-daemonset.yaml
 kubectl get pods -o wide
 kubectl logs <counter-podname>

 Ongoing through the logs of a pod of DaemonSet, one can observe that the logs are not in order, meaning that all pods are sharing the same volume. Also, only one PVC will be created that all pods will be sharing.

 kubectl get pvc

 Figure 4: Daemonset run replica on each node sharing volume

 If you update a DaemonSet, it also performs RollingUpdate i.e. one pod will go down and the updated pod will come up. If an error occurs while updating, so only one pod will be down, all other pods will still be up, running on the previous stable version. Unlike Deployments, you cannot roll back your DaemonSet to a previous version.

 Let's remove the DaemonSet Object

 kubectl delete daemonset counter-app

 Conclusion

 In this hands-on lab, we have seen different ways to deploy pods with the help of different Kubernetes resources.

 How likely are you going to recommend this lab to your friends or colleagues?

 Unlikely
 Likely

 Share this article:

 Leave a comment:

 About the Author

 Ali Kahoot

 Senior DevOps Engineer, Tarabut Gateway

 Ali is a Senior DevOps Engineer & Trainer with more than 5 years of hands-on experience of architecting, developing, automating and optimising production grade deployments on Kubernetes & Openshift using best DevOps practices.

Kubernetes (K8s) is one of the popular open-source container orchestration systems for automating deployment, scaling, and management of containerized applications. Kubernetes provides a basic resource called Pod. A pod is the smallest deployable unit in Kubernetes which has a collection of containers. As a pod can have one or more containers so different configurations can be passed to the container(s) using the pod’s configuration e.g. passing environment variables, mounting volumes, having health checks, etc. In this hands-on lab, we will be going through three different ways to deploy applications (pods) on kind multi-node Kubernetes cluster using three different Kubernetes resources such as    Deployments   StatefulSets   DaemonSets    Configure a multi-node kind cluster with two worker and one master node.    kind create cluster --config=kind-config.yaml   It will take a few minutes to get the kind cluster configured and as it gets configured, the Ready status of nodes can be verified by In the NFS hands-on lab, we have seen how to set up ReadWriteMany access mode for persistent volume in a kind cluster as it comes with a standard storage class that has ReadWriteOnceaccess mode.    Configure NFS storage class with ReadWriteMany access-mode    kubectl apply -f setup_nfs.yaml    Check the NFS storage class  For detailed differences between the three resources, we will be deploying a sample counter app, which logs and increments the count from a counter file like 1,2,3,….    For this, we will be using a counter file from a Persistent Volume to detail the differences between the Deployments, StatefulSets, and DaemonSets.    kubectl apply -f counter-pvc.yaml   Deployments  Deployment is the easiest and most used resource for deploying your application. It is a Kubernetes controller that matches the current state of your cluster to the desired state mentioned in the Deployment manifest.  For example, If you create a deployment with 1 replica, it will check that the desired state of ReplicaSet is 1 and current state is 0, so it will create a ReplicaSet, which will further create the pod.   If you create a deployment with name counter, it will create a ReplicaSet with name counter-&lt;replica-set-id&gt;, which will further create a Pod with name counter-&lt;replica-set-&gt;-&lt;pod-id&gt;.  Deployments are usually used for stateless applications. However, you can save the state of deployment by attaching a Persistent Volume to it and making it stateful, but all the pods of deployment will be sharing the same Volume, and data across all of them will be the same which can lead to data inconsistency. For deploying the sample counter app using a deployment, we will be using the following manifest    kubectl apply -f counter-deployment.yaml     kubectl logs -l app=counter     Figure 1: Persistence in deployments   Now if you scale up the deployment to 3 by running   kubectl scale deployment counter-deploy --replicas=3   As a new pods gets created, if we check the logs of the new pods, their logs will not start from 1 and the counter index be would random. This is because pods in the deployment are sharing the same volume, and data is shared across all the pods. Also if we check the Persistent Volume Claim(PVC), only one PVC will be created that both the pods will be sharing so it can cause Data Inconsistency.   Figure 2: Data inconsistency in deployments sharing single volume    Deployments create a ReplicaSet which then creates a Pod so whenever you update the deployment using RollingUpdate(default) strategy, a new ReplicaSet is created and the Deployment moves the Pods from the old ReplicaSet to the new one at a controlled rate.   Rolling Update means that the previous ReplicaSet doesn’t scale to 0 unless the new ReplicaSet is up &amp; running ensuring 100% uptime. If an error occurs while updating, the new ReplicaSet will never be in Ready state, so the old ReplicaSet will not terminate again ensuring 100% uptime in case of a failed update. In Deployments, you can also manually roll back to a previous ReplicaSet, if needed in case your new feature is not working as expected. Let's remove the Deployment Object   kubectl delete deployment counter-deploy   StatefulSets  StatefulSet is a Kubernetes resource used to manage stateful applications. It manages the deployment and scaling of a set of Pods and provides the guarantee about the ordering and uniqueness of these Pods.  StatefulSet is also a Controller but unlike Deployments, it doesn’t create ReplicaSet rather itself but creates the Pod with a unique naming convention. e.g. If you create a StatefulSet with name counter, it will create a pod with name counter-0, and for multiple replicas of a stateful set, their names will increment like counter-0, counter-1, counter-2, etc  Every replica of a stateful set will have its own state, and each of the pods will be creating its own PVC(Persistent Volume Claim).  For deploying the sample counter app using a stateful set, we will be using the following manifest   kubectl apply -f counter-statefulset.yaml   Now if you scale up the stateful set to 3 replicas by running   kubectl scale statefulsets counter --replicas=3   It will first create a new pod, and once that pod is ready, then another pod counter-2will be created. The new pods will have their own volume and if you check the logs, the count will again start from 1 for the new pods, unlike in Deployments. Also if we check the Persistent Volume Claim(PVC) once again then there will be three PVCs that will be created.    Figure 3: Persistence in statefulsets as each having its own volume    StatefulSets don’t create ReplicaSet so you cant roll back a StatefulSet to a previous version. You can only delete or scale up/down the Statefulset. If you update a StatefulSet, it also performs RollingUpdate i.e. one replica pod will go down and the updated pod will come up. StatefulSets are useful in the case of Databases especially when we need Highly Available Databases in production as we create a cluster of Database replicas with one being the primary replica and others being the secondary replicas. The primary will be responsible for read/write operations and secondary for read-only operations and they will be syncing data with the primary one. Let's remove the StatefuleSet Object   kubectl delete statefulset counter   DaemonSet A DaemonSet is a controller that ensures that the pod runs on all the nodes of the cluster. If a node is added/removed from a cluster, DaemonSet automatically adds/deletes the pod. Some typical use cases of a DaemonSet is to run cluster level applications like:  Monitoring Exporters: You would want to monitor all the nodes of your cluster so you will need to run a monitor on all the nodes of the cluster like NodeExporter. Logs Collection Daemon: You would want to export logs from all nodes so you would need a DaemonSet of log collector like Fluentd to export logs from all your nodes.   However, Daemonset automatically doesn’t run on nodes that have a taint e.g. Master. You will have to specify the tolerations for it on the pod.   Taints  are a way of telling the nodes to repel the pods i.e. no pods will be scheduled on this node unless the pod tolerates the node with the same toleration. The master node is already tainted by: This means it will repel all pods that do not tolerate this taint. So for DaemonSet to run on all nodes, you would have to add the following tolerations on DaemonSet which means that it should tolerate all nodes. For deploying the sample counter app using a DaemonSet, we will be using the following manifest. When DaemonSet is deployed, it will create pods equal to the number of nodes. In terms of behavior, it will behave the same as Deployments i.e. all pods will share the same Persistent Volume.    kubectl apply -f counter-daemonset.yaml     kubectl logs &lt;counter-podname&gt;   Ongoing through the logs of a pod of DaemonSet, one can observe that the logs are not in order, meaning that all pods are sharing the same volume. Also, only one PVC will be created that all pods will be sharing.   Figure 4: Daemonset run replica on each node sharing volume   If you update a DaemonSet, it also performs RollingUpdate i.e. one pod will go down and the updated pod will come up. If an error occurs while updating, so only one pod will be down, all other pods will still be up, running on the previous stable version. Unlike Deployments, you cannot roll back your DaemonSet to a previous version. Let's remove the DaemonSet Object   kubectl delete daemonset counter-app   Conclusion In this hands-on lab, we have seen different ways to deploy pods with the help of different Kubernetes resources.
Kubernetes: Deployments vs StatefulSets vs DaemonSets
https://cloudyuga.guru/hands_on_lab/Deployments-vs-StatefulSets-vs-DaemonSets
https://stackoverflow.com/questions/41583672/kubernetes-deployments-vs-statefulsets
Kubernetes Deployments vs StatefulSets
2021-02-19
SS
<div><div>
<p><strong>What is StatefulSet?</strong></p>
<p>It's a Kubernetes component that is used specifically for stateful applications.</p>
<p><strong>What is a Stateful application?</strong></p>
<p>Any application that stores data to keep track of its state. Or we can say that the applications that track state by saving information in some storage.</p>
<p><strong>Examples of Stateful applications are all kinds of Databases</strong></p>
<ul>
<li>MongoDB</li>
<li>MySQL</li>
<li>ElasticSearch</li>
</ul>
<p><em>Deployment == Stateless</em></p>
<p><strong>What is a Stateless application?</strong></p>
<p>Those applications that do not need to keep records of previous request and interaction is handled as completely new and isolated based on information that comes with it.</p>
<p>Deployment of Stateful and Stateless application
Stateful application is used to deploy using <strong>Statefulset</strong> component of Kubernetes.</p>
<p>Stateless application is used to deploy using <strong>Deployment</strong> component Kubernetes.</p>
<ul>
<li><p>Just like deployment statefulset makes it possible to replicate application pods or to run multiple replicas of it.</p>
</li>
<li><p>You can also configure storage with both of them equally in the same way.</p>
</li>
</ul>
<h3>So what is the difference between Statefulset and Deployment or why we should use different ones for each type of application</h3>
<h2>StatefulSet vs Deployment</h2>
<ul>
<li>Replicating stateful applications is more difficult and has a couple of requirements that stateless applications do not have.</li>
</ul>
<p><em>Let's discuss an example</em></p>
<p>Let's say we have one MongoDB pod that handles requests from the NodeJs application pod which is deployed using deployment. Let's say we have scaled NodeJs application pod from 1 to 3 so they can handle more client requests and in parallel, you scale MongoDB pod so that they can handle more NodeJs request.</p>
<p>Scaling is your NodeJs application is pretty straightforward, pods will be identical and interchangeable so that to scale up deployment is pretty easy. The deployment will create the pods in any random order and they will get random hashes at the end of pod name <code>NodeApp-f5cdee</code>, <code>NodeApp-fasx34</code>, <code>NodeApp-ax7jds</code>.</p>
<p>The deployment will get one <code>svc</code> which helps to load balance to any pod of any request.</p>
<p>When you delete or scale down the deployment they will delete them in any random order at the same time.</p>
<p>If we talk about MongoDB pod replicas that were deployed using statefulset can not be created and deleted at the same time in any order and con not be randomly addressed. The reason behind this is replica pods of statefulset are not identical because they each have their own additional identity of the pods.</p>
<p><strong>Note-: Giving each pod its own required identity makes the difference between stateful and deployment</strong></p>
<p>Statefulset maintains a sticky identity for each pod so <em>they are created from the same specification but are not interchangeable!</em></p>
<p>It has a persistent identifier across any re-scheduling which means that when a pod dies it is replaced by a new pod and keeps the same identity.</p>
<p><strong>But why is this identity nescessary ?</strong></p>
<p>If we talk about a single MongoDB pod that used to be both reading and writing the data but if you add the second pod of MongoDB this can not act as the same way because if we allow instances of MongoDB to change the data that will end up with data inconsistency.</p>
<p>So instead there is a mechanism that decides that only the pod is allowed to write or change the data which is shared for multiple MongoDB instances for reading so the pod which allows changing the data is called <strong>master</strong> and others are called <strong>slave</strong>.</p>
<p><strong>Note-: Master and slaves don't use the same physical storage even though they use the same data.</strong></p>
<h2>Pod Identity</h2>
<ul>
<li><p>In stateful every pod has its own identifier and gets a fixed order name but not the same in the case for deployment.</p>
</li>
<li><p>If we create a satefulset replicas of 3 then it will create like <code>MongoDB-0</code>, <code>MongoDB-1</code>, <code>MongoDB-2</code> here the first one is master in next are slaves.</p>
</li>
</ul>
<p><strong>Note-: The statefulset will not create the next pod in the replica of the previous pod is not already running and up and the same order is for deletion but in reverse order.</strong></p>
<h2>Pod Endpoints</h2>
<ul>
<li>Each pod in statefulset gets its own DNS endpoint from the service.</li>
</ul>
<p>So finally we can say that the Statefulset application has 2 characters</p>
<ol>
<li>Predictable pod name</li>
<li>Fixed individual DNS name</li>
</ol>
<p>When pods restart the IP address will change but the name and endpoints still the same.</p>
 </div>||||I|||| Stack Overflow
 1. About
 2. Products
 3. For Teams
 1. Stack Overflow Public questions & answers
 2. Stack Overflow for Teams Where developers & technologists share private knowledge with coworkers
 3. Talent Build your employer brand
 4. Advertising Reach developers & technologists worldwide
 5. About the company
 Loading…

 1. current community

 + Stack Overflow
 help chat
 + Meta Stack Overflow

 your communities

 Sign up or log in to customize your list.

 more stack exchange communities

 company blog
 2. 
 3. Log in
 4. Sign up
 1. Home
 2. 
 1. Public
 2. Questions
 3. Tags
 4. Users
 5. Companies
 6. Collectives
 7. Explore Collectives
 3. 
 1. Teams
 Stack Overflow for Teams – Start collaborating and sharing organizational knowledge. Create a free Team Why Teams?
 2. Teams
 3. Create free Team

 Collectives™ on Stack Overflow

 Find centralized, trusted content and collaborate around the technologies you use most.

 Learn more about Collectives

 Teams

 Q&A for work

 Connect and share knowledge within a single location that is structured and easy to search.

 Learn more about Teams

 Kubernetes Deployments vs StatefulSets

 Ask Question
 Asked 6 years, 1 month ago
 Modified 5 months ago
 Viewed 114k times
 190

 I've been doing a lot of digging on Kubernetes, and I'm liking what I see a lot! One thing I've been unable to get a clear idea about is what the exact distinctions are between the Deployment and StatefulSet resources and in which scenarios would you use each (or is one generally preferred over the other).

 * kubernetes
 Share
 Improve this question
 Follow
 edited Feb 19, 2021 at 21:07
 030
 10.3k 12 12 gold badges 74 74 silver badges 122 122 bronze badges
 asked Jan 11, 2017 at 5:57
 SS781 SS781
 2,417 2 2 gold badges 13 13 silver badges 17 17 bronze badges
 Add a comment |

 7 Answers

 Sorted by: Reset to default
 Highest score (default) Trending (recent votes count more) Date modified (newest first) Date created (oldest first)
 168

 Deployments and ReplicationControllers are meant for stateless usage and are rather lightweight. StatefulSets are used when state has to be persisted. Therefore the latter use volumeClaimTemplates / claims on persistent volumes to ensure they can keep the state across component restarts.

 So if your application is stateful or if you want to deploy stateful storage on top of Kubernetes use a StatefulSet.

 If your application is stateless or if state can be built up from backend-systems during the start then use Deployments.

 Further details about running stateful application can be found in 2016 kubernetes' blog entry about stateful applications

 Share
 Improve this answer
 Follow
 edited Jan 30, 2019 at 7:49
 Mike Easley
 121 7 7 bronze badges
 answered Jan 17, 2017 at 21:39
 pagid pagid
 13.3k 11 11 gold badges 77 77 silver badges 104 104 bronze badges
 0
 Add a comment |
 147

 * Deployment - You specify a PersistentVolumeClaim that is shared by all pod replicas. In other words, shared volume.

 The backing storage obviously must have ReadWriteMany or ReadOnlyMany accessMode if you have more than one replica pod.

 * StatefulSet - You specify a volumeClaimTemplates so that each replica pod gets a unique PersistentVolumeClaim associated with it. In other words, no shared volume.

 Here, the backing storage can have ReadWriteOnce accessMode.

 StatefulSet is useful for running things in cluster e.g Hadoop cluster, MySQL cluster, where each node has its own storage.

 Share
 Improve this answer
 Follow
 edited Nov 17, 2018 at 11:53
 answered Nov 17, 2018 at 11:22
 Emmanuel Osimosu Emmanuel Osimosu
 5,417 2 2 gold badges 37 37 silver badges 39 39 bronze badges
 0
 Add a comment |
 117

 TL;DR

 Deployment is a resource to deploy a stateless application, if using a PVC, all replicas will be using the same Volume and none of it will have its own state.

 Statefulsets is used for Stateful applications, each replica of the pod will have its own state, and will be using its own Volume.

 DaemonSet is a controller similar to ReplicaSet that ensures that the pod runs on all the nodes of the cluster. If a node is added/removed from a cluster, DaemonSet automatically adds/deletes the pod.

 I have written about the detailed differences between Deployments, StatefulSets & Daemonsets, and how to deploy a sample application using these Resources K8s: Deployments vs StatefulSets vs DaemonSets.

 Share
 Improve this answer
 Follow
 edited Jun 29, 2021 at 10:58
 answered Jul 3, 2019 at 7:29
 Ali Kahoot Ali Kahoot
 3,061 2 2 gold badges 20 20 silver badges 25 25 bronze badges
 0
 Add a comment |
 21

 StatefulSet

 Use 'StatefulSet' with Stateful Distributed Applications, that require each node to have a persistent state. StatefulSet provides the ability to configure an arbitrary number of nodes, for a stateful application/component, through a configuration (replicas = N).

 There are two kinds of stateful distributed applications: Master-Master and Master-Slave. All nodes in a Master-Master configuration and Slave nodes in a Master-Slave configuration can make use of a StatefulSet.
 Examples:
 Master-Slave -> Datanodes (slaves) in a Hadoop cluster
 Master-Master -> Database nodes (master-master) in a Cassandra cluster

 Each Pod (replica/node) in a StatefulSet has a Unique and Stable network identity. For example in a Cassandra StatefulSet with name as 'cassandra' and number of replica nodes as N, each Cassandra pod (node) has:

 * Ordinal Index for each pod: 0,1,..,N-1
 * Stable network id: cassandra-0, cassandra-1,.., cassandra-N-1
 * A separate persistent volume for each pod against a volume claim template i.e a separate storage for every pod (node)
 * Pods are created in the order 0 to N-1 and terminated in the reverse order N-1 to 0

 Refer: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/

 Deployment

 'Deployment' on the other hand is suitable for stateless applications/services where the nodes do not require any special identity. A load balancer can reach any node that it chooses. All nodes are equal. A Deployment is useful for creating any number of arbitrary nodes, through a configuration (replicas = N).

 Share
 Improve this answer
 Follow
 edited Mar 19, 2020 at 9:35
 answered Dec 28, 2017 at 10:57
 Anurag Anurag
 866 11 11 silver badges 16 16 bronze badges
 Add a comment |
 12

 The difference between StatefulSet and deployment

 StatefulSet is equivalent to a special deployment. Each pod in StatefulSet has a stable, unique network identifier that can be used to discover other members in the cluster. If the name of StatefulSet is Kafka, then the first pod is called Kafka-0, the second Kafka-1, and so on; the start and stop sequence of the pod copy controlled by the StatefulSet is controlled. When the nth pod is operated, the first N-1 pods are already running and ready Good state; the pod in the StatefulSet uses a stable persistent storage volume, implemented by PV or PVC. When deleting the pod, the storage volume associated with the StatefulSet is not deleted by default (for data security); the StatefulSet is bound to be bound to the PV volume. Used to store pod state data, and also used in conjunction with headless services, declared to belong to that headless service;

 Share
 Improve this answer
 Follow
 answered Sep 17, 2019 at 8:30
 Adler.Liu Adler.Liu
 121 1 1 silver badge 4 4 bronze badges
 0
 Add a comment |
 2

 Comparing StatefulSets with ReplicaSets

 Feature StatefulSets    Deployment       
 State Statefull    Stateless       
 Definition Stateful app: Stateful applications typically involve some database, such as Cassandra, MongoDB, or MySQL, and processes a read and/or write to it.  Usually, frontend components have completely different scaling requirements than the backends, so we tend to scale them individually. Not to mention the fact that backends such as databases are usually much harder to scale compared to (stateless) frontend web servers. Yes, the term “stateless” means that no past data nor state is stored or needs to be persistent when a new container is created
 Behaviour When a stateful pod instance dies (or the node it’s running on fails), the pod instance needs to be resurrected on another node, new instance get the same name, network identity, and state as the one it’s replacing. Pod replicas managed by a Deployment; they’re mostly stateless, they can be replaced with a completely new pod replica at any time.     
 Pod Mechanism Pods created by the StatefulSet aren’t exact replicas of each other. Each can have its own set of volumes—in other words, storage (and thus persistent state)—which differentiates it from its peers. When a Deployment replaces a pod, the new pod is a completely new pod with a new hostname and IP     
 
 Share
 Improve this answer
 Follow
 answered Jan 30, 2022 at 18:25
 Gupta Gupta
 8,176 4 4 gold badges 45 45 silver badges 58 58 bronze badges
 Add a comment |
 2

 What is StatefulSet?

 It's a Kubernetes component that is used specifically for stateful applications.

 What is a Stateful application?

 Any application that stores data to keep track of its state. Or we can say that the applications that track state by saving information in some storage.

 Examples of Stateful applications are all kinds of Databases

 * MongoDB
 * MySQL
 * ElasticSearch

 Deployment == Stateless

 What is a Stateless application?

 Those applications that do not need to keep records of previous request and interaction is handled as completely new and isolated based on information that comes with it.

 Deployment of Stateful and Stateless application Stateful application is used to deploy using Statefulset component of Kubernetes.

 Stateless application is used to deploy using Deployment component Kubernetes.

 * Just like deployment statefulset makes it possible to replicate application pods or to run multiple replicas of it.

 * You can also configure storage with both of them equally in the same way.

 So what is the difference between Statefulset and Deployment or why we should use different ones for each type of application

 StatefulSet vs Deployment

 * Replicating stateful applications is more difficult and has a couple of requirements that stateless applications do not have.

 Let's discuss an example

 Let's say we have one MongoDB pod that handles requests from the NodeJs application pod which is deployed using deployment. Let's say we have scaled NodeJs application pod from 1 to 3 so they can handle more client requests and in parallel, you scale MongoDB pod so that they can handle more NodeJs request.

 Scaling is your NodeJs application is pretty straightforward, pods will be identical and interchangeable so that to scale up deployment is pretty easy. The deployment will create the pods in any random order and they will get random hashes at the end of pod name NodeApp-f5cdee, NodeApp-fasx34, NodeApp-ax7jds.

 The deployment will get one svc which helps to load balance to any pod of any request.

 When you delete or scale down the deployment they will delete them in any random order at the same time.

 If we talk about MongoDB pod replicas that were deployed using statefulset can not be created and deleted at the same time in any order and con not be randomly addressed. The reason behind this is replica pods of statefulset are not identical because they each have their own additional identity of the pods.

 Note-: Giving each pod its own required identity makes the difference between stateful and deployment

 Statefulset maintains a sticky identity for each pod so they are created from the same specification but are not interchangeable!

 It has a persistent identifier across any re-scheduling which means that when a pod dies it is replaced by a new pod and keeps the same identity.

 But why is this identity nescessary ?

 If we talk about a single MongoDB pod that used to be both reading and writing the data but if you add the second pod of MongoDB this can not act as the same way because if we allow instances of MongoDB to change the data that will end up with data inconsistency.

 So instead there is a mechanism that decides that only the pod is allowed to write or change the data which is shared for multiple MongoDB instances for reading so the pod which allows changing the data is called master and others are called slave.

 Note-: Master and slaves don't use the same physical storage even though they use the same data.

 Pod Identity

 * In stateful every pod has its own identifier and gets a fixed order name but not the same in the case for deployment.

 * If we create a satefulset replicas of 3 then it will create like MongoDB-0, MongoDB-1, MongoDB-2 here the first one is master in next are slaves.

 Note-: The statefulset will not create the next pod in the replica of the previous pod is not already running and up and the same order is for deletion but in reverse order.

 Pod Endpoints

 * Each pod in statefulset gets its own DNS endpoint from the service.

 So finally we can say that the Statefulset application has 2 characters

 1. Predictable pod name
 2. Fixed individual DNS name

 When pods restart the IP address will change but the name and endpoints still the same.

 Share
 Improve this answer
 Follow
 answered Sep 21, 2022 at 4:37
 Yogendra Pratap Singh Yogendra Pratap Singh
 131 1 1 silver badge 5 5 bronze badges
 2
 * Followup question on your great answer. Does it mean that for an app to write data to the MongoDB in the example, does it have to connect to the master necessarily, or can slaves somehow propagate write requests to the master?
 – cwj
 Feb 13 at 7:38
 * Great answer. I'll add that the primary difference is that a statefulset is needed when pods need to maintain an "internal addressing scheme or structure (unknown to k8s)" that would be disrupted if they were re-scheduled and named randomly.
 – S.D.
 Feb 14 at 16:09
 Add a comment |
 Highly active question. Earn 10 reputation (not counting the association bonus) in order to answer this question. The reputation requirement helps protect this question from spam and non-answer activity.

 Not the answer you're looking for? Browse other questions tagged
 * kubernetes
 or ask your own question.

 * The Overflow Blog
 * ML and AI consulting-as-a-service (Ep. 541)
 * Why governments need open source more than ever
 * Featured on Meta
 * Ticket smash for [status-review] tag: Part Deux
 * We've added a "Necessary cookies only" option to the cookie consent popup
 * The [amazon] tag is being burninated
 * Microsoft Azure Collective launch and proposed tag changes
 * Temporary policy: ChatGPT is banned
 Visit chat

 Linked

 1
 What is the difference between StatefulSet and Deployment with respect to Volumes?
 4
 StatefulSet, ReplicaSet or DaemonSet. What is the best for a single Pod?
 2
 Does the storage class dynamically provision persistent volume per pod?
 2
 How to update DNS configuration of K8S Pod
 1
 What is the VM folder when using Linux as OS and kvm as driver in kubernetes?

 Related

 13
 Best CD strategy for Kubernetes Deployments
 17
 How should I manage deployments with kubernetes
 61
 Can Kubernetes be used like Docker Compose?
 37
 Difference between Google App Engine Flexible and Google Container Engine?
 0
 Automate deployments on Kubernetes
 19
 What exactly Kubernetes Services are and how they are different from Deployments
 1
 check kubernetes node\cluster resource before creating kubernetes resources
 0
 Kubernetes workload for stateful application but no need of persistent disk

 Hot Network Questions

 * In Romans 8:18, how is the aorist ἀποκαλυφθῆναι translated as a future tense?
 * Will floating pantry shelves built with 2x2s be ok?
 * How did Leo Tolstoy approve Maude's English translation of War and Peace? Did he speak good English already?
 * Why was intuitionist logic abandoned?
 * How to make indices (math mode super- and sub-script) uniform size?
 * Shortcut of finding a tangent line and tangent point to a curve that's explicitly or implicitly defined
 * Which philosophers have considered irrational conviction
 * Repeating text watermark in background?
 * Looking for a SF/alternate history book about a 19th century style explorer who navigates lands with corrupted fauna and flora
 * Did the residents of Aneyoshi survive the 2011 tsunami thanks to the warnings of a stone marker?
 * What led to the fall of shell accounts?
 * Intel "Kernel processor" or "OS Kernel"
 * Arithmetic progression in LaTeX
 * Does the discount from Mindsplice Apparatus apply to flashback costs?
 * What is an important mathematical question?
 * Ungrounded receptacle in unusual 1950s metal outlet box - i was told to add a ground to box. GFCI outlet protected. Do i have to remove wires?
 * Identify and split data in other languages using QGIS
 * Do 'ration' and 'station' rhyme in any English accents?
 * How did this ceiling exhaust fan get stuck on (and why did disconnecting this wire fix it)?
 * Why do game developers not make games with cheat codes or god mode any more?
 * Bumping Series Implementation
 * Is there a colloquial word/expression for a push that helps you to start to do something?
 * Short story featuring a detective and his alien sidekick
 * How important is it to clearly understand requirements and architecture before starting to code?
 more hot questions
 Question feed

 Subscribe to RSS

 Question feed

 To subscribe to this RSS feed, copy and paste this URL into your RSS reader.

 Stack Overflow

 * Questions
 * Help

 Products

 * Teams
 * Advertising
 * Collectives
 * Talent

 Company

 * About
 * Press
 * Work Here
 * Legal
 * Privacy Policy
 * Terms of Service
 * Contact Us
 * Cookie Settings
 * Cookie Policy

 Stack Exchange Network

 * Technology
 * Culture & recreation
 * Life & arts
 * Science
 * Professional
 * Business
 * API
 * Data
 * Blog
 * Facebook
 * Twitter
 * LinkedIn
 * Instagram

 Site design / logo © 2023 Stack Exchange Inc; user contributions licensed under CC BY-SA. rev 2023.2.24.43259

 Your privacy

 By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.

 Accept all cookies Necessary cookies only
 Customize settings
What is StatefulSet? 
It's a Kubernetes component that is used specifically for stateful applications.
 What is a Stateful application? 
Any application that stores data to keep track of its state. Or we can say that the applications that track state by saving information in some storage.
 Examples of Stateful applications are all kinds of Databases 
MongoDB
MySQL
ElasticSearch
 Deployment == Stateless 
 What is a Stateless application? 
Those applications that do not need to keep records of previous request and interaction is handled as completely new and isolated based on information that comes with it.
Deployment of Stateful and Stateless application
Stateful application is used to deploy using Statefulset component of Kubernetes.
Stateless application is used to deploy using Deployment component Kubernetes.
 Just like deployment statefulset makes it possible to replicate application pods or to run multiple replicas of it.
 You can also configure storage with both of them equally in the same way.
So what is the difference between Statefulset and Deployment or why we should use different ones for each type of application
StatefulSet vs Deployment
Replicating stateful applications is more difficult and has a couple of requirements that stateless applications do not have.
 Let's discuss an example 
Let's say we have one MongoDB pod that handles requests from the NodeJs application pod which is deployed using deployment. Let's say we have scaled NodeJs application pod from 1 to 3 so they can handle more client requests and in parallel, you scale MongoDB pod so that they can handle more NodeJs request.
Scaling is your NodeJs application is pretty straightforward, pods will be identical and interchangeable so that to scale up deployment is pretty easy. The deployment will create the pods in any random order and they will get random hashes at the end of pod name NodeApp-f5cdee, NodeApp-fasx34, NodeApp-ax7jds.
The deployment will get one svc which helps to load balance to any pod of any request.
When you delete or scale down the deployment they will delete them in any random order at the same time.
If we talk about MongoDB pod replicas that were deployed using statefulset can not be created and deleted at the same time in any order and con not be randomly addressed. The reason behind this is replica pods of statefulset are not identical because they each have their own additional identity of the pods.
 Note-: Giving each pod its own required identity makes the difference between stateful and deployment 
Statefulset maintains a sticky identity for each pod so they are created from the same specification but are not interchangeable! 
It has a persistent identifier across any re-scheduling which means that when a pod dies it is replaced by a new pod and keeps the same identity.
 But why is this identity nescessary ? 
If we talk about a single MongoDB pod that used to be both reading and writing the data but if you add the second pod of MongoDB this can not act as the same way because if we allow instances of MongoDB to change the data that will end up with data inconsistency.
So instead there is a mechanism that decides that only the pod is allowed to write or change the data which is shared for multiple MongoDB instances for reading so the pod which allows changing the data is called master and others are called slave.
 Note-: Master and slaves don't use the same physical storage even though they use the same data. 
Pod Identity
 In stateful every pod has its own identifier and gets a fixed order name but not the same in the case for deployment.
 If we create a satefulset replicas of 3 then it will create like MongoDB-0, MongoDB-1, MongoDB-2 here the first one is master in next are slaves.
 Note-: The statefulset will not create the next pod in the replica of the previous pod is not already running and up and the same order is for deletion but in reverse order. 
Pod Endpoints
Each pod in statefulset gets its own DNS endpoint from the service.
So finally we can say that the Statefulset application has 2 characters
Predictable pod name
Fixed individual DNS name
When pods restart the IP address will change but the name and endpoints still the same.
Kubernetes Deployments vs StatefulSets
https://stackoverflow.com/questions/41583672/kubernetes-deployments-vs-statefulsets#:~:text=Statefulsets%20is%20used%20for%20Stateful,the%20nodes%20of%20the%20cluster.
https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
StatefulSets
2022-12-15
None
<div><div><p>StatefulSet is the workload API object used to manage stateful applications.</p><p>Manages the deployment and scaling of a set of <a href="/docs/concepts/workloads/pods/">Pods</a>, <em>and provides guarantees about the ordering and uniqueness</em> of these Pods.</p><p>Like a <a href="/docs/concepts/workloads/controllers/deployment/">Deployment</a>, a StatefulSet manages Pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable: each has a persistent identifier that it maintains across any rescheduling.</p><p>If you want to use storage volumes to provide persistence for your workload, you can use a StatefulSet as part of the solution. Although individual Pods in a StatefulSet are susceptible to failure, the persistent Pod identifiers make it easier to match existing volumes to the new Pods that replace any that have failed.</p><h2>Using StatefulSets</h2><p>StatefulSets are valuable for applications that require one or more of the
following.</p><ul><li>Stable, unique network identifiers.</li><li>Stable, persistent storage.</li><li>Ordered, graceful deployment and scaling.</li><li>Ordered, automated rolling updates.</li></ul><p>In the above, stable is synonymous with persistence across Pod (re)scheduling.
If an application doesn't require any stable identifiers or ordered deployment,
deletion, or scaling, you should deploy your application using a workload object
that provides a set of stateless replicas.
<a href="/docs/concepts/workloads/controllers/deployment/">Deployment</a> or
<a href="/docs/concepts/workloads/controllers/replicaset/">ReplicaSet</a> may be better suited to your stateless needs.</p><h2>Limitations</h2><ul><li>The storage for a given Pod must either be provisioned by a
<a href="https://github.com/kubernetes/examples/tree/master/staging/persistent-volume-provisioning/README.md">PersistentVolume Provisioner</a>
based on the requested <code>storage class</code>, or pre-provisioned by an admin.</li><li>Deleting and/or scaling a StatefulSet down will <em>not</em> delete the volumes associated with the
StatefulSet. This is done to ensure data safety, which is generally more valuable than an
automatic purge of all related StatefulSet resources.</li><li>StatefulSets currently require a <a href="/docs/concepts/services-networking/service/#headless-services">Headless Service</a>
to be responsible for the network identity of the Pods. You are responsible for creating this
Service.</li><li>StatefulSets do not provide any guarantees on the termination of pods when a StatefulSet is
deleted. To achieve ordered and graceful termination of the pods in the StatefulSet, it is
possible to scale the StatefulSet down to 0 prior to deletion.</li><li>When using <a href="#rolling-updates">Rolling Updates</a> with the default
<a href="#pod-management-policies">Pod Management Policy</a> (<code>OrderedReady</code>),
it's possible to get into a broken state that requires
<a href="#forced-rollback">manual intervention to repair</a>.</li></ul><h2>Components</h2><p>The example below demonstrates the components of a StatefulSet.</p><div><pre><code>apiVersion: v1
kind: Service
metadata:
 name: nginx
 labels:
 app: nginx
spec:
 ports:
 - port: 80
 name: web
 clusterIP: None
 selector:
 app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
 name: web
spec:
 selector:
 matchLabels:
 app: nginx # has to match .spec.template.metadata.labels
 serviceName: "nginx"
 replicas: 3 # by default is 1
 minReadySeconds: 10 # by default is 0
 template:
 metadata:
 labels:
 app: nginx # has to match .spec.selector.matchLabels
 spec:
 terminationGracePeriodSeconds: 10
 containers:
 - name: nginx
 image: registry.k8s.io/nginx-slim:0.8
 ports:
 - containerPort: 80
 name: web
 volumeMounts:
 - name: www
 mountPath: /usr/share/nginx/html
 volumeClaimTemplates:
 - metadata:
 name: www
 spec:
 accessModes: [ "ReadWriteOnce" ]
 storageClassName: "my-storage-class"
 resources:
 requests:
 storage: 1Gi
</code></pre></div><p>In the above example:</p><ul><li>A Headless Service, named <code>nginx</code>, is used to control the network domain.</li><li>The StatefulSet, named <code>web</code>, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods.</li><li>The <code>volumeClaimTemplates</code> will provide stable storage using
<a href="/docs/concepts/storage/persistent-volumes/">PersistentVolumes</a> provisioned by a
PersistentVolume Provisioner.</li></ul><p>The name of a StatefulSet object must be a valid
<a href="/docs/concepts/overview/working-with-objects/names#dns-label-names">DNS label</a>.</p><h3>Pod Selector</h3><p>You must set the <code>.spec.selector</code> field of a StatefulSet to match the labels of its
<code>.spec.template.metadata.labels</code>. Failing to specify a matching Pod Selector will result in a
validation error during StatefulSet creation.</p><h3>Volume Claim Templates</h3><p>You can set the <code>.spec.volumeClaimTemplates</code> which can provide stable storage using
<a href="/docs/concepts/storage/persistent-volumes/">PersistentVolumes</a> provisioned by a PersistentVolume
Provisioner.</p><h3>Minimum ready seconds</h3><p><b>FEATURE STATE:</b> <code>Kubernetes v1.25 [stable]</code></p><p><code>.spec.minReadySeconds</code> is an optional field that specifies the minimum number of seconds for which a newly
created Pod should be running and ready without any of its containers crashing, for it to be considered available.
This is used to check progression of a rollout when using a <a href="#rolling-updates">Rolling Update</a> strategy.
This field defaults to 0 (the Pod will be considered available as soon as it is ready). To learn more about when
a Pod is considered ready, see <a href="/docs/concepts/workloads/pods/pod-lifecycle/#container-probes">Container Probes</a>.</p><h2>Pod Identity</h2><p>StatefulSet Pods have a unique identity that consists of an ordinal, a
stable network identity, and stable storage. The identity sticks to the Pod,
regardless of which node it's (re)scheduled on.</p><h3>Ordinal Index</h3><p>For a StatefulSet with N <a href="#replicas">replicas</a>, each Pod in the StatefulSet
will be assigned an integer ordinal, that is unique over the Set. By default,
pods will be assigned ordinals from 0 up through N-1.</p><h3>Start ordinal</h3><p><b>FEATURE STATE:</b> <code>Kubernetes v1.26 [alpha]</code></p><p><code>.spec.ordinals</code> is an optional field that allows you to configure the integer
ordinals assigned to each Pod. It defaults to nil. You must enable the
<code>StatefulSetStartOrdinal</code>
<a href="/docs/reference/command-line-tools-reference/feature-gates/">feature gate</a> to
use this field. Once enabled, you can configure the following options:</p><ul><li><code>.spec.ordinals.start</code>: If the <code>.spec.ordinals.start</code> field is set, Pods will
be assigned ordinals from <code>.spec.ordinals.start</code> up through
<code>.spec.ordinals.start + .spec.replicas - 1</code>.</li></ul><h3>Stable Network ID</h3><p>Each Pod in a StatefulSet derives its hostname from the name of the StatefulSet
and the ordinal of the Pod. The pattern for the constructed hostname
is <code>$(statefulset name)-$(ordinal)</code>. The example above will create three Pods
named <code>web-0,web-1,web-2</code>.
A StatefulSet can use a <a href="/docs/concepts/services-networking/service/#headless-services">Headless Service</a>
to control the domain of its Pods. The domain managed by this Service takes the form:
<code>$(service name).$(namespace).svc.cluster.local</code>, where "cluster.local" is the
cluster domain.
As each Pod is created, it gets a matching DNS subdomain, taking the form:
<code>$(podname).$(governing service domain)</code>, where the governing service is defined
by the <code>serviceName</code> field on the StatefulSet.</p><p>Depending on how DNS is configured in your cluster, you may not be able to look up the DNS
name for a newly-run Pod immediately. This behavior can occur when other clients in the
cluster have already sent queries for the hostname of the Pod before it was created.
Negative caching (normal in DNS) means that the results of previous failed lookups are
remembered and reused, even after the Pod is running, for at least a few seconds.</p><p>If you need to discover Pods promptly after they are created, you have a few options:</p><ul><li>Query the Kubernetes API directly (for example, using a watch) rather than relying on DNS lookups.</li><li>Decrease the time of caching in your Kubernetes DNS provider (typically this means editing the
config map for CoreDNS, which currently caches for 30 seconds).</li></ul><p>As mentioned in the <a href="#limitations">limitations</a> section, you are responsible for
creating the <a href="/docs/concepts/services-networking/service/#headless-services">Headless Service</a>
responsible for the network identity of the pods.</p><p>Here are some examples of choices for Cluster Domain, Service name,
StatefulSet name, and how that affects the DNS names for the StatefulSet's Pods.</p><table><thead><tr><th>Cluster Domain</th><th>Service (ns/name)</th><th>StatefulSet (ns/name)</th><th>StatefulSet Domain</th><th>Pod DNS</th><th>Pod Hostname</th></tr></thead><tbody><tr><td>cluster.local</td><td>default/nginx</td><td>default/web</td><td>nginx.default.svc.cluster.local</td><td>web-{0..N-1}.nginx.default.svc.cluster.local</td><td>web-{0..N-1}</td></tr><tr><td>cluster.local</td><td>foo/nginx</td><td>foo/web</td><td>nginx.foo.svc.cluster.local</td><td>web-{0..N-1}.nginx.foo.svc.cluster.local</td><td>web-{0..N-1}</td></tr><tr><td>kube.local</td><td>foo/nginx</td><td>foo/web</td><td>nginx.foo.svc.kube.local</td><td>web-{0..N-1}.nginx.foo.svc.kube.local</td><td>web-{0..N-1}</td></tr></tbody></table><h3>Stable Storage</h3><p>For each VolumeClaimTemplate entry defined in a StatefulSet, each Pod receives one
PersistentVolumeClaim. In the nginx example above, each Pod receives a single PersistentVolume
with a StorageClass of <code>my-storage-class</code> and 1 Gib of provisioned storage. If no StorageClass
is specified, then the default StorageClass will be used. When a Pod is (re)scheduled
onto a node, its <code>volumeMounts</code> mount the PersistentVolumes associated with its
PersistentVolume Claims. Note that, the PersistentVolumes associated with the
Pods' PersistentVolume Claims are not deleted when the Pods, or StatefulSet are deleted.
This must be done manually.</p><h3>Pod Name Label</h3><p>When the StatefulSet <a href="/docs/concepts/architecture/controller/">controller</a> creates a Pod,
it adds a label, <code>statefulset.kubernetes.io/pod-name</code>, that is set to the name of
the Pod. This label allows you to attach a Service to a specific Pod in
the StatefulSet.</p><h2>Deployment and Scaling Guarantees</h2><ul><li>For a StatefulSet with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}.</li><li>When Pods are being deleted, they are terminated in reverse order, from {N-1..0}.</li><li>Before a scaling operation is applied to a Pod, all of its predecessors must be Running and Ready.</li><li>Before a Pod is terminated, all of its successors must be completely shutdown.</li></ul><p>The StatefulSet should not specify a <code>pod.Spec.TerminationGracePeriodSeconds</code> of 0. This practice
is unsafe and strongly discouraged. For further explanation, please refer to
<a href="/docs/tasks/run-application/force-delete-stateful-set-pod/">force deleting StatefulSet Pods</a>.</p><p>When the nginx example above is created, three Pods will be deployed in the order
web-0, web-1, web-2. web-1 will not be deployed before web-0 is
<a href="/docs/concepts/workloads/pods/pod-lifecycle/">Running and Ready</a>, and web-2 will not be deployed until
web-1 is Running and Ready. If web-0 should fail, after web-1 is Running and Ready, but before
web-2 is launched, web-2 will not be launched until web-0 is successfully relaunched and
becomes Running and Ready.</p><p>If a user were to scale the deployed example by patching the StatefulSet such that
<code>replicas=1</code>, web-2 would be terminated first. web-1 would not be terminated until web-2
is fully shutdown and deleted. If web-0 were to fail after web-2 has been terminated and
is completely shutdown, but prior to web-1's termination, web-1 would not be terminated
until web-0 is Running and Ready.</p><h3>Pod Management Policies</h3><p>StatefulSet allows you to relax its ordering guarantees while
preserving its uniqueness and identity guarantees via its <code>.spec.podManagementPolicy</code> field.</p><h4>OrderedReady Pod Management</h4><p><code>OrderedReady</code> pod management is the default for StatefulSets. It implements the behavior
described <a href="#deployment-and-scaling-guarantees">above</a>.</p><h4>Parallel Pod Management</h4><p><code>Parallel</code> pod management tells the StatefulSet controller to launch or
terminate all Pods in parallel, and to not wait for Pods to become Running
and Ready or completely terminated prior to launching or terminating another
Pod. This option only affects the behavior for scaling operations. Updates are not
affected.</p><h2>Update strategies</h2><p>A StatefulSet's <code>.spec.updateStrategy</code> field allows you to configure
and disable automated rolling updates for containers, labels, resource request/limits, and
annotations for the Pods in a StatefulSet. There are two possible values:</p><dl><dt><code>OnDelete</code></dt><dd>When a StatefulSet's <code>.spec.updateStrategy.type</code> is set to <code>OnDelete</code>,
the StatefulSet controller will not automatically update the Pods in a
StatefulSet. Users must manually delete Pods to cause the controller to
create new Pods that reflect modifications made to a StatefulSet's <code>.spec.template</code>.</dd><dt><code>RollingUpdate</code></dt><dd>The <code>RollingUpdate</code> update strategy implements automated, rolling updates for the Pods in a
StatefulSet. This is the default update strategy.</dd></dl><h2>Rolling Updates</h2><p>When a StatefulSet's <code>.spec.updateStrategy.type</code> is set to <code>RollingUpdate</code>, the
StatefulSet controller will delete and recreate each Pod in the StatefulSet. It will proceed
in the same order as Pod termination (from the largest ordinal to the smallest), updating
each Pod one at a time.</p><p>The Kubernetes control plane waits until an updated Pod is Running and Ready prior
to updating its predecessor. If you have set <code>.spec.minReadySeconds</code> (see
<a href="#minimum-ready-seconds">Minimum Ready Seconds</a>), the control plane additionally waits that
amount of time after the Pod turns ready, before moving on.</p><h3>Partitioned rolling updates</h3><p>The <code>RollingUpdate</code> update strategy can be partitioned, by specifying a
<code>.spec.updateStrategy.rollingUpdate.partition</code>. If a partition is specified, all Pods with an
ordinal that is greater than or equal to the partition will be updated when the StatefulSet's
<code>.spec.template</code> is updated. All Pods with an ordinal that is less than the partition will not
be updated, and, even if they are deleted, they will be recreated at the previous version. If a
StatefulSet's <code>.spec.updateStrategy.rollingUpdate.partition</code> is greater than its <code>.spec.replicas</code>,
updates to its <code>.spec.template</code> will not be propagated to its Pods.
In most cases you will not need to use a partition, but they are useful if you want to stage an
update, roll out a canary, or perform a phased roll out.</p><h3>Maximum unavailable Pods</h3><p><b>FEATURE STATE:</b> <code>Kubernetes v1.24 [alpha]</code></p><p>You can control the maximum number of Pods that can be unavailable during an update
by specifying the <code>.spec.updateStrategy.rollingUpdate.maxUnavailable</code> field.
The value can be an absolute number (for example, <code>5</code>) or a percentage of desired
Pods (for example, <code>10%</code>). Absolute number is calculated from the percentage value
by rounding it up. This field cannot be 0. The default setting is 1.</p><p>This field applies to all Pods in the range <code>0</code> to <code>replicas - 1</code>. If there is any
unavailable Pod in the range <code>0</code> to <code>replicas - 1</code>, it will be counted towards
<code>maxUnavailable</code>.</p><h3>Forced rollback</h3><p>When using <a href="#rolling-updates">Rolling Updates</a> with the default
<a href="#pod-management-policies">Pod Management Policy</a> (<code>OrderedReady</code>),
it's possible to get into a broken state that requires manual intervention to repair.</p><p>If you update the Pod template to a configuration that never becomes Running and
Ready (for example, due to a bad binary or application-level configuration error),
StatefulSet will stop the rollout and wait.</p><p>In this state, it's not enough to revert the Pod template to a good configuration.
Due to a <a href="https://github.com/kubernetes/kubernetes/issues/67250">known issue</a>,
StatefulSet will continue to wait for the broken Pod to become Ready
(which never happens) before it will attempt to revert it back to the working
configuration.</p><p>After reverting the template, you must also delete any Pods that StatefulSet had
already attempted to run with the bad configuration.
StatefulSet will then begin to recreate the Pods using the reverted template.</p><h2>PersistentVolumeClaim retention</h2><p><b>FEATURE STATE:</b> <code>Kubernetes v1.23 [alpha]</code></p><p>The optional <code>.spec.persistentVolumeClaimRetentionPolicy</code> field controls if
and how PVCs are deleted during the lifecycle of a StatefulSet. You must enable the
<code>StatefulSetAutoDeletePVC</code> <a href="/docs/reference/command-line-tools-reference/feature-gates/">feature gate</a>
on the API server and the controller manager to use this field.
Once enabled, there are two policies you can configure for each StatefulSet:</p><dl><dt><code>whenDeleted</code></dt><dd>configures the volume retention behavior that applies when the StatefulSet is deleted</dd><dt><code>whenScaled</code></dt><dd>configures the volume retention behavior that applies when the replica count of
the StatefulSet is reduced; for example, when scaling down the set.</dd></dl><p>For each policy that you can configure, you can set the value to either <code>Delete</code> or <code>Retain</code>.</p><dl><dt><code>Delete</code></dt><dd>The PVCs created from the StatefulSet <code>volumeClaimTemplate</code> are deleted for each Pod
affected by the policy. With the <code>whenDeleted</code> policy all PVCs from the
<code>volumeClaimTemplate</code> are deleted after their Pods have been deleted. With the
<code>whenScaled</code> policy, only PVCs corresponding to Pod replicas being scaled down are
deleted, after their Pods have been deleted.</dd><dt><code>Retain</code> (default)</dt><dd>PVCs from the <code>volumeClaimTemplate</code> are not affected when their Pod is
deleted. This is the behavior before this new feature.</dd></dl><p>Bear in mind that these policies <strong>only</strong> apply when Pods are being removed due to the
StatefulSet being deleted or scaled down. For example, if a Pod associated with a StatefulSet
fails due to node failure, and the control plane creates a replacement Pod, the StatefulSet
retains the existing PVC. The existing volume is unaffected, and the cluster will attach it to
the node where the new Pod is about to launch.</p><p>The default for policies is <code>Retain</code>, matching the StatefulSet behavior before this new feature.</p><p>Here is an example policy.</p><div><pre><code>apiVersion: apps/v1
kind: StatefulSet
...
spec:
 persistentVolumeClaimRetentionPolicy:
 whenDeleted: Retain
 whenScaled: Delete
...
</code></pre></div><p>The StatefulSet <a href="/docs/concepts/architecture/controller/">controller</a> adds
<a href="/docs/concepts/overview/working-with-objects/owners-dependents/#owner-references-in-object-specifications">owner references</a>
to its PVCs, which are then deleted by the <a href="/docs/concepts/architecture/garbage-collection/">garbage collector</a> after the Pod is terminated. This enables the Pod to
cleanly unmount all volumes before the PVCs are deleted (and before the backing PV and
volume are deleted, depending on the retain policy). When you set the <code>whenDeleted</code>
policy to <code>Delete</code>, an owner reference to the StatefulSet instance is placed on all PVCs
associated with that StatefulSet.</p><p>The <code>whenScaled</code> policy must delete PVCs only when a Pod is scaled down, and not when a
Pod is deleted for another reason. When reconciling, the StatefulSet controller compares
its desired replica count to the actual Pods present on the cluster. Any StatefulSet Pod
whose id greater than the replica count is condemned and marked for deletion. If the
<code>whenScaled</code> policy is <code>Delete</code>, the condemned Pods are first set as owners to the
associated StatefulSet template PVCs, before the Pod is deleted. This causes the PVCs
to be garbage collected after only the condemned Pods have terminated.</p><p>This means that if the controller crashes and restarts, no Pod will be deleted before its
owner reference has been updated appropriate to the policy. If a condemned Pod is
force-deleted while the controller is down, the owner reference may or may not have been
set up, depending on when the controller crashed. It may take several reconcile loops to
update the owner references, so some condemned Pods may have set up owner references and
others may not. For this reason we recommend waiting for the controller to come back up,
which will verify owner references before terminating Pods. If that is not possible, the
operator should verify the owner references on PVCs to ensure the expected objects are
deleted when Pods are force-deleted.</p><h3>Replicas</h3><p><code>.spec.replicas</code> is an optional field that specifies the number of desired Pods. It defaults to 1.</p><p>Should you manually scale a deployment, example via <code>kubectl scale statefulset statefulset --replicas=X</code>, and then you update that StatefulSet
based on a manifest (for example: by running <code>kubectl apply -f statefulset.yaml</code>), then applying that manifest overwrites the manual scaling
that you previously did.</p><p>If a <a href="/docs/tasks/run-application/horizontal-pod-autoscale/">HorizontalPodAutoscaler</a>
(or any similar API for horizontal scaling) is managing scaling for a
Statefulset, don't set <code>.spec.replicas</code>. Instead, allow the Kubernetes
<a href="/docs/reference/glossary/?all=true#term-control-plane">control plane</a> to manage
the <code>.spec.replicas</code> field automatically.</p><h2>What's next</h2><ul><li>Learn about <a href="/docs/concepts/workloads/pods">Pods</a>.</li><li>Find out how to use StatefulSets<ul><li>Follow an example of <a href="/docs/tutorials/stateful-application/basic-stateful-set/">deploying a stateful application</a>.</li><li>Follow an example of <a href="/docs/tutorials/stateful-application/cassandra/">deploying Cassandra with Stateful Sets</a>.</li><li>Follow an example of <a href="/docs/tasks/run-application/run-replicated-stateful-application/">running a replicated stateful application</a>.</li><li>Learn how to <a href="/docs/tasks/run-application/scale-stateful-set/">scale a StatefulSet</a>.</li><li>Learn what's involved when you <a href="/docs/tasks/run-application/delete-stateful-set/">delete a StatefulSet</a>.</li><li>Learn how to <a href="/docs/tasks/configure-pod-container/configure-volume-storage/">configure a Pod to use a volume for storage</a>.</li><li>Learn how to <a href="/docs/tasks/configure-pod-container/configure-persistent-volume-storage/">configure a Pod to use a PersistentVolume for storage</a>.</li></ul></li><li><code>StatefulSet</code> is a top-level resource in the Kubernetes REST API.
Read the
<a href="/docs/reference/kubernetes-api/workload-resources/stateful-set-v1/">StatefulSet</a>
object definition to understand the API for stateful sets.</li><li>Read about <a href="/docs/concepts/workloads/pods/disruptions/">PodDisruptionBudget</a> and how
you can use it to manage application availability during disruptions.</li></ul></div>||||I|||| * Documentation
 * Kubernetes Blog
 * Training
 * Partners
 * Community
 * Case Studies
 * Versions
 Release Information v1.26 v1.25 v1.24 v1.23 v1.22
 * English
 中文 (Chinese) 한국어 (Korean) 日本語 (Japanese) Français (French) Español (Spanish) Bahasa Indonesia
 * 
 + Home
 o Available Documentation Versions
 + Getting started
 o Learning environment
 o Production environment
 - Container Runtimes
 - Installing Kubernetes with deployment tools
 * Bootstrapping clusters with kubeadm
 + Installing kubeadm
 + Troubleshooting kubeadm
 + Creating a cluster with kubeadm
 + Customizing components with the kubeadm API
 + Options for Highly Available Topology
 + Creating Highly Available Clusters with kubeadm
 + Set up a High Availability etcd Cluster with kubeadm
 + Configuring each kubelet in your cluster using kubeadm
 + Dual-stack support with kubeadm
 * Installing Kubernetes with kOps
 * Installing Kubernetes with Kubespray
 - Turnkey Cloud Solutions
 o Best practices
 - Considerations for large clusters
 - Running in multiple zones
 - Validate node setup
 - Enforcing Pod Security Standards
 - PKI certificates and requirements
 + Concepts
 o Overview
 - Kubernetes Components
 - The Kubernetes API
 - Working with Kubernetes Objects
 * Understanding Kubernetes Objects
 * Kubernetes Object Management
 * Object Names and IDs
 * Labels and Selectors
 * Namespaces
 * Annotations
 * Field Selectors
 * Finalizers
 * Owners and Dependents
 * Recommended Labels
 o Cluster Architecture
 - Nodes
 - Communication between Nodes and the Control Plane
 - Controllers
 - Leases
 - Cloud Controller Manager
 - About cgroup v2
 - Container Runtime Interface (CRI)
 - Garbage Collection
 o Containers
 - Images
 - Container Environment
 - Runtime Class
 - Container Lifecycle Hooks
 o Windows in Kubernetes
 - Windows containers in Kubernetes
 - Guide for scheduling Windows containers in Kubernetes
 o Workloads
 - Pods
 * Pod Lifecycle
 * Init Containers
 * Disruptions
 * Ephemeral Containers
 * User Namespaces
 * Downward API
 - Workload Resources
 * Deployments
 * ReplicaSet
 * StatefulSets
 * DaemonSet
 * Jobs
 * Automatic Cleanup for Finished Jobs
 * CronJob
 * ReplicationController
 o Services, Load Balancing, and Networking
 - Service
 - Ingress
 - Ingress Controllers
 - EndpointSlices
 - Network Policies
 - DNS for Services and Pods
 - IPv4/IPv6 dual-stack
 - Topology Aware Hints
 - Networking on Windows
 - Service ClusterIP allocation
 - Service Internal Traffic Policy
 - Topology-aware traffic routing with topology keys
 o Storage
 - Volumes
 - Persistent Volumes
 - Projected Volumes
 - Ephemeral Volumes
 - Storage Classes
 - Dynamic Volume Provisioning
 - Volume Snapshots
 - Volume Snapshot Classes
 - CSI Volume Cloning
 - Storage Capacity
 - Node-specific Volume Limits
 - Volume Health Monitoring
 - Windows Storage
 o Configuration
 - Configuration Best Practices
 - ConfigMaps
 - Secrets
 - Resource Management for Pods and Containers
 - Organizing Cluster Access Using kubeconfig Files
 - Resource Management for Windows nodes
 o Security
 - Overview of Cloud Native Security
 - Pod Security Standards
 - Pod Security Admission
 - Pod Security Policies
 - Security For Windows Nodes
 - Controlling Access to the Kubernetes API
 - Role Based Access Control Good Practices
 - Good practices for Kubernetes Secrets
 - Multi-tenancy
 - Kubernetes API Server Bypass Risks
 - Security Checklist
 o Policies
 - Limit Ranges
 - Resource Quotas
 - Process ID Limits And Reservations
 - Node Resource Managers
 o Scheduling, Preemption and Eviction
 - Kubernetes Scheduler
 - Assigning Pods to Nodes
 - Pod Overhead
 - Pod Scheduling Readiness
 - Pod Topology Spread Constraints
 - Taints and Tolerations
 - Scheduling Framework
 - Dynamic Resource Allocation
 - Scheduler Performance Tuning
 - Resource Bin Packing
 - Pod Priority and Preemption
 - Node-pressure Eviction
 - API-initiated Eviction
 o Cluster Administration
 - Certificates
 - Managing Resources
 - Cluster Networking
 - Logging Architecture
 - Metrics For Kubernetes System Components
 - System Logs
 - Traces For Kubernetes System Components
 - Proxies in Kubernetes
 - API Priority and Fairness
 - Installing Addons
 o Extending Kubernetes
 - Compute, Storage, and Networking Extensions
 * Network Plugins
 * Device Plugins
 - Extending the Kubernetes API
 * Custom Resources
 * Kubernetes API Aggregation Layer
 - Operator pattern
 + Tasks
 o Install Tools
 - Install and Set Up kubectl on Linux
 - Install and Set Up kubectl on macOS
 - Install and Set Up kubectl on Windows
 o Administer a Cluster
 - Administration with kubeadm
 * Certificate Management with kubeadm
 * Configuring a cgroup driver
 * Reconfiguring a kubeadm cluster
 * Upgrading kubeadm clusters
 * Upgrading Windows nodes
 - Migrating from dockershim
 * Changing the Container Runtime on a Node from Docker Engine to containerd
 * Migrate Docker Engine nodes from dockershim to cri-dockerd
 * Find Out What Container Runtime is Used on a Node
 * Troubleshooting CNI plugin-related errors
 * Check whether dockershim removal affects you
 * Migrating telemetry and security agents from dockershim
 - Generate Certificates Manually
 - Manage Memory, CPU, and API Resources
 * Configure Default Memory Requests and Limits for a Namespace
 * Configure Default CPU Requests and Limits for a Namespace
 * Configure Minimum and Maximum Memory Constraints for a Namespace
 * Configure Minimum and Maximum CPU Constraints for a Namespace
 * Configure Memory and CPU Quotas for a Namespace
 * Configure a Pod Quota for a Namespace
 - Install a Network Policy Provider
 * Use Antrea for NetworkPolicy
 * Use Calico for NetworkPolicy
 * Use Cilium for NetworkPolicy
 * Use Kube-router for NetworkPolicy
 * Romana for NetworkPolicy
 * Weave Net for NetworkPolicy
 - Access Clusters Using the Kubernetes API
 - Advertise Extended Resources for a Node
 - Autoscale the DNS Service in a Cluster
 - Change the default StorageClass
 - Change the Reclaim Policy of a PersistentVolume
 - Cloud Controller Manager Administration
 - Configure a kubelet image credential provider
 - Configure Quotas for API Objects
 - Control CPU Management Policies on the Node
 - Control Topology Management Policies on a node
 - Customizing DNS Service
 - Debugging DNS Resolution
 - Declare Network Policy
 - Developing Cloud Controller Manager
 - Enable Or Disable A Kubernetes API
 - Encrypting Secret Data at Rest
 - Guaranteed Scheduling For Critical Add-On Pods
 - IP Masquerade Agent User Guide
 - Limit Storage Consumption
 - Migrate Replicated Control Plane To Use Cloud Controller Manager
 - Namespaces Walkthrough
 - Operating etcd clusters for Kubernetes
 - Reconfigure a Node's Kubelet in a Live Cluster
 - Reserve Compute Resources for System Daemons
 - Running Kubernetes Node Components as a Non-root User
 - Safely Drain a Node
 - Securing a Cluster
 - Set Kubelet parameters via a config file
 - Share a Cluster with Namespaces
 - Upgrade A Cluster
 - Use Cascading Deletion in a Cluster
 - Using a KMS provider for data encryption
 - Using CoreDNS for Service Discovery
 - Using NodeLocal DNSCache in Kubernetes Clusters
 - Using sysctls in a Kubernetes Cluster
 - Utilizing the NUMA-aware Memory Manager
 - Verify Signed Kubernetes Artifacts
 o Configure Pods and Containers
 - Assign Memory Resources to Containers and Pods
 - Assign CPU Resources to Containers and Pods
 - Configure GMSA for Windows Pods and containers
 - Configure RunAsUserName for Windows pods and containers
 - Create a Windows HostProcess Pod
 - Configure Quality of Service for Pods
 - Assign Extended Resources to a Container
 - Configure a Pod to Use a Volume for Storage
 - Configure a Pod to Use a PersistentVolume for Storage
 - Configure a Pod to Use a Projected Volume for Storage
 - Configure a Security Context for a Pod or Container
 - Configure Service Accounts for Pods
 - Pull an Image from a Private Registry
 - Configure Liveness, Readiness and Startup Probes
 - Assign Pods to Nodes
 - Assign Pods to Nodes using Node Affinity
 - Configure Pod Initialization
 - Attach Handlers to Container Lifecycle Events
 - Configure a Pod to Use a ConfigMap
 - Share Process Namespace between Containers in a Pod
 - Use a User Namespace With a Pod
 - Create static Pods
 - Translate a Docker Compose File to Kubernetes Resources
 - Enforce Pod Security Standards by Configuring the Built-in Admission Controller
 - Enforce Pod Security Standards with Namespace Labels
 - Migrate from PodSecurityPolicy to the Built-In PodSecurity Admission Controller
 o Monitoring, Logging, and Debugging
 - Troubleshooting Applications
 * Debug Pods
 * Debug Services
 * Debug a StatefulSet
 * Determine the Reason for Pod Failure
 * Debug Init Containers
 * Debug Running Pods
 * Get a Shell to a Running Container
 - Troubleshooting Clusters
 * Resource metrics pipeline
 * Tools for Monitoring Resources
 * Monitor Node Health
 * Debugging Kubernetes nodes with crictl
 * Auditing
 * Debugging Kubernetes Nodes With Kubectl
 * Developing and debugging services locally using telepresence
 * Windows debugging tips
 o Manage Kubernetes Objects
 - Declarative Management of Kubernetes Objects Using Configuration Files
 - Declarative Management of Kubernetes Objects Using Kustomize
 - Managing Kubernetes Objects Using Imperative Commands
 - Imperative Management of Kubernetes Objects Using Configuration Files
 - Update API Objects in Place Using kubectl patch
 o Managing Secrets
 - Managing Secrets using kubectl
 - Managing Secrets using Configuration File
 - Managing Secrets using Kustomize
 o Inject Data Into Applications
 - Define a Command and Arguments for a Container
 - Define Dependent Environment Variables
 - Define Environment Variables for a Container
 - Expose Pod Information to Containers Through Environment Variables
 - Expose Pod Information to Containers Through Files
 - Distribute Credentials Securely Using Secrets
 o Run Applications
 - Run a Stateless Application Using a Deployment
 - Run a Single-Instance Stateful Application
 - Run a Replicated Stateful Application
 - Scale a StatefulSet
 - Delete a StatefulSet
 - Force Delete StatefulSet Pods
 - Horizontal Pod Autoscaling
 - HorizontalPodAutoscaler Walkthrough
 - Specifying a Disruption Budget for your Application
 - Accessing the Kubernetes API from a Pod
 o Run Jobs
 - Running Automated Tasks with a CronJob
 - Coarse Parallel Processing Using a Work Queue
 - Fine Parallel Processing Using a Work Queue
 - Indexed Job for Parallel Processing with Static Work Assignment
 - Job with Pod-to-Pod Communication
 - Parallel Processing using Expansions
 - Handling retriable and non-retriable pod failures with Pod failure policy
 o Access Applications in a Cluster
 - Deploy and Access the Kubernetes Dashboard
 - Accessing Clusters
 - Configure Access to Multiple Clusters
 - Use Port Forwarding to Access Applications in a Cluster
 - Use a Service to Access an Application in a Cluster
 - Connect a Frontend to a Backend Using Services
 - Create an External Load Balancer
 - List All Container Images Running in a Cluster
 - Set up Ingress on Minikube with the NGINX Ingress Controller
 - Communicate Between Containers in the Same Pod Using a Shared Volume
 - Configure DNS for a Cluster
 - Access Services Running on Clusters
 o Extend Kubernetes
 - Configure the Aggregation Layer
 - Use Custom Resources
 * Extend the Kubernetes API with CustomResourceDefinitions
 * Versions in CustomResourceDefinitions
 - Set up an Extension API Server
 - Configure Multiple Schedulers
 - Use an HTTP Proxy to Access the Kubernetes API
 - Use a SOCKS5 Proxy to Access the Kubernetes API
 - Set up Konnectivity service
 o TLS
 - Configure Certificate Rotation for the Kubelet
 - Manage TLS Certificates in a Cluster
 - Manual Rotation of CA Certificates
 o Manage Cluster Daemons
 - Perform a Rolling Update on a DaemonSet
 - Perform a Rollback on a DaemonSet
 o Networking
 - Adding entries to Pod /etc/hosts with HostAliases
 - Validate IPv4/IPv6 dual-stack
 o Extend kubectl with plugins
 o Manage HugePages
 o Schedule GPUs
 + Tutorials
 o Hello Minikube
 o Learn Kubernetes Basics
 - Create a Cluster
 * Using Minikube to Create a Cluster
 * Interactive Tutorial - Creating a Cluster
 - Deploy an App
 * Using kubectl to Create a Deployment
 * Interactive Tutorial - Deploying an App
 - Explore Your App
 * Viewing Pods and Nodes
 * Interactive Tutorial - Exploring Your App
 - Expose Your App Publicly
 * Using a Service to Expose Your App
 * Interactive Tutorial - Exposing Your App
 - Scale Your App
 * Running Multiple Instances of Your App
 * Interactive Tutorial - Scaling Your App
 - Update Your App
 * Performing a Rolling Update
 * Interactive Tutorial - Updating Your App
 o Configuration
 - Example: Configuring a Java Microservice
 * Externalizing config using MicroProfile, ConfigMaps and Secrets
 * Interactive Tutorial - Configuring a Java Microservice
 - Configuring Redis using a ConfigMap
 o Security
 - Apply Pod Security Standards at the Cluster Level
 - Apply Pod Security Standards at the Namespace Level
 - Restrict a Container's Access to Resources with AppArmor
 - Restrict a Container's Syscalls with seccomp
 o Stateless Applications
 - Exposing an External IP Address to Access an Application in a Cluster
 - Example: Deploying PHP Guestbook application with Redis
 o Stateful Applications
 - StatefulSet Basics
 - Example: Deploying WordPress and MySQL with Persistent Volumes
 - Example: Deploying Cassandra with a StatefulSet
 - Running ZooKeeper, A Distributed System Coordinator
 o Services
 - Connecting Applications with Services
 - Using Source IP
 + Reference
 o Glossary
 o API Overview
 - Kubernetes API Concepts
 - Server-Side Apply
 - Client Libraries
 - Kubernetes Deprecation Policy
 - Deprecated API Migration Guide
 - Kubernetes API health endpoints
 o API Access Control
 - Authenticating
 - Authenticating with Bootstrap Tokens
 - Certificate Signing Requests
 - Admission Controllers
 - Dynamic Admission Control
 - Managing Service Accounts
 - Authorization Overview
 - Using RBAC Authorization
 - Using ABAC Authorization
 - Using Node Authorization
 - Mapping PodSecurityPolicies to Pod Security Standards
 - Webhook Mode
 - Kubelet authentication/authorization
 - TLS bootstrapping
 - Validating Admission Policy
 o Well-Known Labels, Annotations and Taints
 - Audit Annotations
 o Kubernetes API
 - Workload Resources
 * Pod
 * PodTemplate
 * ReplicationController
 * ReplicaSet
 * Deployment
 * StatefulSet
 * ControllerRevision
 * DaemonSet
 * Job
 * CronJob
 * HorizontalPodAutoscaler
 * HorizontalPodAutoscaler
 * PriorityClass
 * PodScheduling v1alpha1
 * ResourceClaim v1alpha1
 * ResourceClaimTemplate v1alpha1
 * ResourceClass v1alpha1
 - Service Resources
 * Service
 * Endpoints
 * EndpointSlice
 * Ingress
 * IngressClass
 - Config and Storage Resources
 * ConfigMap
 * Secret
 * Volume
 * PersistentVolumeClaim
 * PersistentVolume
 * StorageClass
 * VolumeAttachment
 * CSIDriver
 * CSINode
 * CSIStorageCapacity
 - Authentication Resources
 * ServiceAccount
 * TokenRequest
 * TokenReview
 * CertificateSigningRequest
 - Authorization Resources
 * LocalSubjectAccessReview
 * SelfSubjectAccessReview
 * SelfSubjectRulesReview
 * SubjectAccessReview
 * SelfSubjectReview v1alpha1
 * ClusterRole
 * ClusterRoleBinding
 * Role
 * RoleBinding
 - Policy Resources
 * LimitRange
 * ResourceQuota
 * NetworkPolicy
 * PodDisruptionBudget
 - Extend Resources
 * CustomResourceDefinition
 * MutatingWebhookConfiguration
 * ValidatingWebhookConfiguration
 * ValidatingAdmissionPolicy v1alpha1
 - Cluster Resources
 * Node
 * Namespace
 * Event
 * APIService
 * Lease
 * RuntimeClass
 * FlowSchema v1beta3
 * PriorityLevelConfiguration v1beta3
 * Binding
 * ComponentStatus
 * ClusterCIDR v1alpha1
 - Common Definitions
 * DeleteOptions
 * LabelSelector
 * ListMeta
 * LocalObjectReference
 * NodeSelectorRequirement
 * ObjectFieldSelector
 * ObjectMeta
 * ObjectReference
 * Patch
 * Quantity
 * ResourceFieldSelector
 * Status
 * TypedLocalObjectReference
 - Other Resources
 * ValidatingAdmissionPolicyBindingList v1alpha1
 - Common Parameters
 o Instrumentation
 - Service Level Indicator Metrics
 - Node metrics data
 - Kubernetes Metrics Reference
 o Kubernetes Issues and Security
 - Kubernetes Issue Tracker
 - Kubernetes Security and Disclosure Information
 - Official CVE Feed
 o Node Reference Information
 - Kubelet Checkpoint API
 - Kubelet Device Manager API Versions
 - Articles on dockershim Removal and on Using CRI-compatible Runtimes
 o Networking Reference
 - Protocols for Services
 - Ports and Protocols
 - Virtual IPs and Service Proxies
 o Setup tools
 - Kubeadm
 * kubeadm init
 * kubeadm join
 * kubeadm upgrade
 * kubeadm config
 * kubeadm reset
 * kubeadm token
 * kubeadm version
 * kubeadm alpha
 * kubeadm certs
 * kubeadm init phase
 * kubeadm join phase
 * kubeadm kubeconfig
 * kubeadm reset phase
 * kubeadm upgrade phase
 * Implementation details
 o Command line tool (kubectl)
 - kubectl Cheat Sheet
 - kubectl Commands
 - kubectl
 - JSONPath Support
 - kubectl for Docker Users
 - kubectl Usage Conventions
 o Component tools
 - Feature Gates
 - Feature Gates (removed)
 - kubelet
 - kube-apiserver
 - kube-controller-manager
 - kube-proxy
 - kube-scheduler
 o Configuration APIs
 - Client Authentication (v1)
 - Client Authentication (v1beta1)
 - Event Rate Limit Configuration (v1alpha1)
 - Image Policy API (v1alpha1)
 - kube-apiserver Admission (v1)
 - kube-apiserver Audit Configuration (v1)
 - kube-apiserver Configuration (v1)
 - kube-apiserver Configuration (v1alpha1)
 - kube-apiserver Encryption Configuration (v1)
 - kube-proxy Configuration (v1alpha1)
 - kube-scheduler Configuration (v1)
 - kube-scheduler Configuration (v1beta2)
 - kube-scheduler Configuration (v1beta3)
 - kubeadm Configuration (v1beta2)
 - kubeadm Configuration (v1beta3)
 - kubeconfig (v1)
 - Kubelet Configuration (v1)
 - Kubelet Configuration (v1alpha1)
 - Kubelet Configuration (v1beta1)
 - Kubelet CredentialProvider (v1)
 - Kubelet CredentialProvider (v1alpha1)
 - Kubelet CredentialProvider (v1beta1)
 - WebhookAdmission Configuration (v1)
 o Scheduling
 - Scheduler Configuration
 - Scheduling Policies
 o Other Tools
 - Mapping from dockercli to crictl
 + Contribute
 o Suggesting content improvements
 o Contributing new content
 - Opening a pull request
 - Documenting for a release
 - Blogs and case studies
 o Reviewing changes
 - Reviewing pull requests
 - For approvers and reviewers
 o Localizing Kubernetes documentation
 o Participating in SIG Docs
 - Roles and responsibilities
 - PR wranglers
 o Documentation style overview
 - Content guide
 - Style guide
 - Diagram guide
 - Writing a new topic
 - Page content types
 - Content organization
 - Custom Hugo Shortcodes
 o Updating Reference Documentation
 - Quickstart
 - Contributing to the Upstream Kubernetes Code
 - Generating Reference Documentation for the Kubernetes API
 - Generating Reference Documentation for kubectl Commands
 - Generating Reference Pages for Kubernetes Components and Tools
 - 
 o Advanced contributing
 o Viewing Site Analytics
 + Docs smoke test page
 1. Kubernetes Documentation
 2. Concepts
 3. Workloads
 4. Workload Resources
 5. StatefulSets

 StatefulSets

 StatefulSet is the workload API object used to manage stateful applications.

 Manages the deployment and scaling of a set of Pods, and provides guarantees about the ordering and uniqueness of these Pods.

 Like a Deployment, a StatefulSet manages Pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable: each has a persistent identifier that it maintains across any rescheduling.

 If you want to use storage volumes to provide persistence for your workload, you can use a StatefulSet as part of the solution. Although individual Pods in a StatefulSet are susceptible to failure, the persistent Pod identifiers make it easier to match existing volumes to the new Pods that replace any that have failed.

 Using StatefulSets

 StatefulSets are valuable for applications that require one or more of the following.

 * Stable, unique network identifiers.
 * Stable, persistent storage.
 * Ordered, graceful deployment and scaling.
 * Ordered, automated rolling updates.

 In the above, stable is synonymous with persistence across Pod (re)scheduling. If an application doesn't require any stable identifiers or ordered deployment, deletion, or scaling, you should deploy your application using a workload object that provides a set of stateless replicas. Deployment or ReplicaSet may be better suited to your stateless needs.

 Limitations

 * The storage for a given Pod must either be provisioned by a PersistentVolume Provisioner based on the requested storage class, or pre-provisioned by an admin.
 * Deleting and/or scaling a StatefulSet down will not delete the volumes associated with the StatefulSet. This is done to ensure data safety, which is generally more valuable than an automatic purge of all related StatefulSet resources.
 * StatefulSets currently require a Headless Service to be responsible for the network identity of the Pods. You are responsible for creating this Service.
 * StatefulSets do not provide any guarantees on the termination of pods when a StatefulSet is deleted. To achieve ordered and graceful termination of the pods in the StatefulSet, it is possible to scale the StatefulSet down to 0 prior to deletion.
 * When using Rolling Updates with the default Pod Management Policy (OrderedReady), it's possible to get into a broken state that requires manual intervention to repair.

 Components

 The example below demonstrates the components of a StatefulSet.

 apiVersion: v1
 kind: Service
 metadata:
 name: nginx
 labels:
 app: nginx
 spec:
 ports:
 - port: 80
 name: web
 clusterIP: None
 selector:
 app: nginx
 ---
 apiVersion: apps/v1
 kind: StatefulSet
 metadata:
 name: web
 spec:
 selector:
 matchLabels:
 app: nginx # has to match .spec.template.metadata.labels
 serviceName: "nginx"
 replicas: 3 # by default is 1
 minReadySeconds: 10 # by default is 0
 template:
 metadata:
 labels:
 app: nginx # has to match .spec.selector.matchLabels
 spec:
 terminationGracePeriodSeconds: 10
 containers:
 - name: nginx
 image: registry.k8s.io/nginx-slim:0.8
 ports:
 - containerPort: 80
 name: web
 volumeMounts:
 - name: www
 mountPath: /usr/share/nginx/html
 volumeClaimTemplates:
 - metadata:
 name: www
 spec:
 accessModes: [ "ReadWriteOnce" ]
 storageClassName: "my-storage-class"
 resources:
 requests:
 storage: 1Gi
 

 In the above example:

 * A Headless Service, named nginx, is used to control the network domain.
 * The StatefulSet, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods.
 * The volumeClaimTemplates will provide stable storage using PersistentVolumes provisioned by a PersistentVolume Provisioner.

 The name of a StatefulSet object must be a valid DNS label.

 Pod Selector

 You must set the .spec.selector field of a StatefulSet to match the labels of its .spec.template.metadata.labels. Failing to specify a matching Pod Selector will result in a validation error during StatefulSet creation.

 Volume Claim Templates

 You can set the .spec.volumeClaimTemplates which can provide stable storage using PersistentVolumes provisioned by a PersistentVolume Provisioner.

 Minimum ready seconds

 FEATURE STATE: Kubernetes v1.25 [stable]

 .spec.minReadySeconds is an optional field that specifies the minimum number of seconds for which a newly created Pod should be running and ready without any of its containers crashing, for it to be considered available. This is used to check progression of a rollout when using a Rolling Update strategy. This field defaults to 0 (the Pod will be considered available as soon as it is ready). To learn more about when a Pod is considered ready, see Container Probes.

 Pod Identity

 StatefulSet Pods have a unique identity that consists of an ordinal, a stable network identity, and stable storage. The identity sticks to the Pod, regardless of which node it's (re)scheduled on.

 Ordinal Index

 For a StatefulSet with N replicas, each Pod in the StatefulSet will be assigned an integer ordinal, that is unique over the Set. By default, pods will be assigned ordinals from 0 up through N-1.

 Start ordinal

 FEATURE STATE: Kubernetes v1.26 [alpha]

 .spec.ordinals is an optional field that allows you to configure the integer ordinals assigned to each Pod. It defaults to nil. You must enable the StatefulSetStartOrdinal feature gate to use this field. Once enabled, you can configure the following options:

 * .spec.ordinals.start: If the .spec.ordinals.start field is set, Pods will be assigned ordinals from .spec.ordinals.start up through .spec.ordinals.start + .spec.replicas - 1.

 Stable Network ID

 Each Pod in a StatefulSet derives its hostname from the name of the StatefulSet and the ordinal of the Pod. The pattern for the constructed hostname is $(statefulset name)-$(ordinal). The example above will create three Pods named web-0,web-1,web-2. A StatefulSet can use a Headless Service to control the domain of its Pods. The domain managed by this Service takes the form: $(service name).$(namespace).svc.cluster.local, where "cluster.local" is the cluster domain. As each Pod is created, it gets a matching DNS subdomain, taking the form: $(podname).$(governing service domain), where the governing service is defined by the serviceName field on the StatefulSet.

 Depending on how DNS is configured in your cluster, you may not be able to look up the DNS name for a newly-run Pod immediately. This behavior can occur when other clients in the cluster have already sent queries for the hostname of the Pod before it was created. Negative caching (normal in DNS) means that the results of previous failed lookups are remembered and reused, even after the Pod is running, for at least a few seconds.

 If you need to discover Pods promptly after they are created, you have a few options:

 * Query the Kubernetes API directly (for example, using a watch) rather than relying on DNS lookups.
 * Decrease the time of caching in your Kubernetes DNS provider (typically this means editing the config map for CoreDNS, which currently caches for 30 seconds).

 As mentioned in the limitations section, you are responsible for creating the Headless Service responsible for the network identity of the pods.

 Here are some examples of choices for Cluster Domain, Service name, StatefulSet name, and how that affects the DNS names for the StatefulSet's Pods.

 Cluster Domain Service (ns/name) StatefulSet (ns/name) StatefulSet Domain Pod DNS Pod Hostname
 cluster.local default/nginx default/web nginx.default.svc.cluster.local web-{0..N-1}.nginx.default.svc.cluster.local web-{0..N-1}
 cluster.local foo/nginx foo/web nginx.foo.svc.cluster.local web-{0..N-1}.nginx.foo.svc.cluster.local web-{0..N-1}
 kube.local foo/nginx foo/web nginx.foo.svc.kube.local web-{0..N-1}.nginx.foo.svc.kube.local web-{0..N-1}
 
 Note: Cluster Domain will be set to cluster.local unless otherwise configured.

 Stable Storage

 For each VolumeClaimTemplate entry defined in a StatefulSet, each Pod receives one PersistentVolumeClaim. In the nginx example above, each Pod receives a single PersistentVolume with a StorageClass of my-storage-class and 1 Gib of provisioned storage. If no StorageClass is specified, then the default StorageClass will be used. When a Pod is (re)scheduled onto a node, its volumeMounts mount the PersistentVolumes associated with its PersistentVolume Claims. Note that, the PersistentVolumes associated with the Pods' PersistentVolume Claims are not deleted when the Pods, or StatefulSet are deleted. This must be done manually.

 Pod Name Label

 When the StatefulSet controller creates a Pod, it adds a label, statefulset.kubernetes.io/pod-name, that is set to the name of the Pod. This label allows you to attach a Service to a specific Pod in the StatefulSet.

 Deployment and Scaling Guarantees

 * For a StatefulSet with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}.
 * When Pods are being deleted, they are terminated in reverse order, from {N-1..0}.
 * Before a scaling operation is applied to a Pod, all of its predecessors must be Running and Ready.
 * Before a Pod is terminated, all of its successors must be completely shutdown.

 The StatefulSet should not specify a pod.Spec.TerminationGracePeriodSeconds of 0. This practice is unsafe and strongly discouraged. For further explanation, please refer to force deleting StatefulSet Pods.

 When the nginx example above is created, three Pods will be deployed in the order web-0, web-1, web-2. web-1 will not be deployed before web-0 is Running and Ready, and web-2 will not be deployed until web-1 is Running and Ready. If web-0 should fail, after web-1 is Running and Ready, but before web-2 is launched, web-2 will not be launched until web-0 is successfully relaunched and becomes Running and Ready.

 If a user were to scale the deployed example by patching the StatefulSet such that replicas=1, web-2 would be terminated first. web-1 would not be terminated until web-2 is fully shutdown and deleted. If web-0 were to fail after web-2 has been terminated and is completely shutdown, but prior to web-1's termination, web-1 would not be terminated until web-0 is Running and Ready.

 Pod Management Policies

 StatefulSet allows you to relax its ordering guarantees while preserving its uniqueness and identity guarantees via its .spec.podManagementPolicy field.

 OrderedReady Pod Management

 OrderedReady pod management is the default for StatefulSets. It implements the behavior described above.

 Parallel Pod Management

 Parallel pod management tells the StatefulSet controller to launch or terminate all Pods in parallel, and to not wait for Pods to become Running and Ready or completely terminated prior to launching or terminating another Pod. This option only affects the behavior for scaling operations. Updates are not affected.

 Update strategies

 A StatefulSet's .spec.updateStrategy field allows you to configure and disable automated rolling updates for containers, labels, resource request/limits, and annotations for the Pods in a StatefulSet. There are two possible values:

 OnDeleteWhen a StatefulSet's .spec.updateStrategy.type is set to OnDelete, the StatefulSet controller will not automatically update the Pods in a StatefulSet. Users must manually delete Pods to cause the controller to create new Pods that reflect modifications made to a StatefulSet's .spec.template.RollingUpdateThe RollingUpdate update strategy implements automated, rolling updates for the Pods in a StatefulSet. This is the default update strategy.

 Rolling Updates

 When a StatefulSet's .spec.updateStrategy.type is set to RollingUpdate, the StatefulSet controller will delete and recreate each Pod in the StatefulSet. It will proceed in the same order as Pod termination (from the largest ordinal to the smallest), updating each Pod one at a time.

 The Kubernetes control plane waits until an updated Pod is Running and Ready prior to updating its predecessor. If you have set .spec.minReadySeconds (see Minimum Ready Seconds), the control plane additionally waits that amount of time after the Pod turns ready, before moving on.

 Partitioned rolling updates

 The RollingUpdate update strategy can be partitioned, by specifying a .spec.updateStrategy.rollingUpdate.partition. If a partition is specified, all Pods with an ordinal that is greater than or equal to the partition will be updated when the StatefulSet's .spec.template is updated. All Pods with an ordinal that is less than the partition will not be updated, and, even if they are deleted, they will be recreated at the previous version. If a StatefulSet's .spec.updateStrategy.rollingUpdate.partition is greater than its .spec.replicas, updates to its .spec.template will not be propagated to its Pods. In most cases you will not need to use a partition, but they are useful if you want to stage an update, roll out a canary, or perform a phased roll out.

 Maximum unavailable Pods

 FEATURE STATE: Kubernetes v1.24 [alpha]

 You can control the maximum number of Pods that can be unavailable during an update by specifying the .spec.updateStrategy.rollingUpdate.maxUnavailable field. The value can be an absolute number (for example, 5) or a percentage of desired Pods (for example, 10%). Absolute number is calculated from the percentage value by rounding it up. This field cannot be 0. The default setting is 1.

 This field applies to all Pods in the range 0 to replicas - 1. If there is any unavailable Pod in the range 0 to replicas - 1, it will be counted towards maxUnavailable.

 Note: The maxUnavailable field is in Alpha stage and it is honored only by API servers that are running with the MaxUnavailableStatefulSet feature gate enabled.

 Forced rollback

 When using Rolling Updates with the default Pod Management Policy (OrderedReady), it's possible to get into a broken state that requires manual intervention to repair.

 If you update the Pod template to a configuration that never becomes Running and Ready (for example, due to a bad binary or application-level configuration error), StatefulSet will stop the rollout and wait.

 In this state, it's not enough to revert the Pod template to a good configuration. Due to a known issue, StatefulSet will continue to wait for the broken Pod to become Ready (which never happens) before it will attempt to revert it back to the working configuration.

 After reverting the template, you must also delete any Pods that StatefulSet had already attempted to run with the bad configuration. StatefulSet will then begin to recreate the Pods using the reverted template.

 PersistentVolumeClaim retention

 FEATURE STATE: Kubernetes v1.23 [alpha]

 The optional .spec.persistentVolumeClaimRetentionPolicy field controls if and how PVCs are deleted during the lifecycle of a StatefulSet. You must enable the StatefulSetAutoDeletePVC feature gate on the API server and the controller manager to use this field. Once enabled, there are two policies you can configure for each StatefulSet:

 whenDeletedconfigures the volume retention behavior that applies when the StatefulSet is deletedwhenScaledconfigures the volume retention behavior that applies when the replica count of the StatefulSet is reduced; for example, when scaling down the set.

 For each policy that you can configure, you can set the value to either Delete or Retain.

 DeleteThe PVCs created from the StatefulSet volumeClaimTemplate are deleted for each Pod affected by the policy. With the whenDeleted policy all PVCs from the volumeClaimTemplate are deleted after their Pods have been deleted. With the whenScaled policy, only PVCs corresponding to Pod replicas being scaled down are deleted, after their Pods have been deleted.Retain (default)PVCs from the volumeClaimTemplate are not affected when their Pod is deleted. This is the behavior before this new feature.

 Bear in mind that these policies only apply when Pods are being removed due to the StatefulSet being deleted or scaled down. For example, if a Pod associated with a StatefulSet fails due to node failure, and the control plane creates a replacement Pod, the StatefulSet retains the existing PVC. The existing volume is unaffected, and the cluster will attach it to the node where the new Pod is about to launch.

 The default for policies is Retain, matching the StatefulSet behavior before this new feature.

 Here is an example policy.

 apiVersion: apps/v1
 kind: StatefulSet
 ...
 spec:
 persistentVolumeClaimRetentionPolicy:
 whenDeleted: Retain
 whenScaled: Delete
 ...
 

 The StatefulSet controller adds owner references to its PVCs, which are then deleted by the garbage collector after the Pod is terminated. This enables the Pod to cleanly unmount all volumes before the PVCs are deleted (and before the backing PV and volume are deleted, depending on the retain policy). When you set the whenDeleted policy to Delete, an owner reference to the StatefulSet instance is placed on all PVCs associated with that StatefulSet.

 The whenScaled policy must delete PVCs only when a Pod is scaled down, and not when a Pod is deleted for another reason. When reconciling, the StatefulSet controller compares its desired replica count to the actual Pods present on the cluster. Any StatefulSet Pod whose id greater than the replica count is condemned and marked for deletion. If the whenScaled policy is Delete, the condemned Pods are first set as owners to the associated StatefulSet template PVCs, before the Pod is deleted. This causes the PVCs to be garbage collected after only the condemned Pods have terminated.

 This means that if the controller crashes and restarts, no Pod will be deleted before its owner reference has been updated appropriate to the policy. If a condemned Pod is force-deleted while the controller is down, the owner reference may or may not have been set up, depending on when the controller crashed. It may take several reconcile loops to update the owner references, so some condemned Pods may have set up owner references and others may not. For this reason we recommend waiting for the controller to come back up, which will verify owner references before terminating Pods. If that is not possible, the operator should verify the owner references on PVCs to ensure the expected objects are deleted when Pods are force-deleted.

 Replicas

 .spec.replicas is an optional field that specifies the number of desired Pods. It defaults to 1.

 Should you manually scale a deployment, example via kubectl scale statefulset statefulset --replicas=X, and then you update that StatefulSet based on a manifest (for example: by running kubectl apply -f statefulset.yaml), then applying that manifest overwrites the manual scaling that you previously did.

 If a HorizontalPodAutoscaler (or any similar API for horizontal scaling) is managing scaling for a Statefulset, don't set .spec.replicas. Instead, allow the Kubernetes control plane to manage the .spec.replicas field automatically.

 What's next

 * Learn about Pods.
 * Find out how to use StatefulSets
 + Follow an example of deploying a stateful application.
 + Follow an example of deploying Cassandra with Stateful Sets.
 + Follow an example of running a replicated stateful application.
 + Learn how to scale a StatefulSet.
 + Learn what's involved when you delete a StatefulSet.
 + Learn how to configure a Pod to use a volume for storage.
 + Learn how to configure a Pod to use a PersistentVolume for storage.
 * StatefulSet is a top-level resource in the Kubernetes REST API. Read the StatefulSet object definition to understand the API for stateful sets.
 * Read about PodDisruptionBudget and how you can use it to manage application availability during disruptions.

 Feedback

 Was this page helpful?

 Yes No

 Thanks for the feedback. If you have a specific, answerable question about how to use Kubernetes, ask it on Stack Overflow. Open an issue in the GitHub repo if you want to report a problem or suggest an improvement.

 Last modified December 15, 2022 at 10:37 AM PST: Recommend DNS Label for workload names (d3c4fe6759)
 Edit this page Create child page Create an issue Print entire section
 * Using StatefulSets
 * Limitations
 * Components
 + Pod Selector
 + Volume Claim Templates
 + Minimum ready seconds
 * Pod Identity
 + Ordinal Index
 + Start ordinal
 + Stable Network ID
 + Stable Storage
 + Pod Name Label
 * Deployment and Scaling Guarantees
 + Pod Management Policies
 * Update strategies
 * Rolling Updates
 + Partitioned rolling updates
 + Maximum unavailable Pods
 + Forced rollback
 * PersistentVolumeClaim retention
 + Replicas
 * What's next
 Home Blog Training Partners Community Case Studies
 * 
 * 
 * 
 * 
 * 
 * 
 * 
 * 
 © 2023 The Kubernetes Authors | Documentation Distributed under CC BY 4.0
 Copyright © 2023 The Linux Foundation ®. All rights reserved. The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our Trademark Usage page
 ICP license: 京ICP备17074266号-3
StatefulSet is the workload API object used to manage stateful applications. Manages the deployment and scaling of a set of Pods, and provides guarantees about the ordering and uniqueness of these Pods. Like a Deployment, a StatefulSet manages Pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable: each has a persistent identifier that it maintains across any rescheduling. If you want to use storage volumes to provide persistence for your workload, you can use a StatefulSet as part of the solution. Although individual Pods in a StatefulSet are susceptible to failure, the persistent Pod identifiers make it easier to match existing volumes to the new Pods that replace any that have failed. Using StatefulSets StatefulSets are valuable for applications that require one or more of the
following.  Stable, unique network identifiers. Stable, persistent storage. Ordered, graceful deployment and scaling. Ordered, automated rolling updates.  In the above, stable is synonymous with persistence across Pod (re)scheduling.
If an application doesn't require any stable identifiers or ordered deployment,
deletion, or scaling, you should deploy your application using a workload object
that provides a set of stateless replicas.
Deployment or
ReplicaSet may be better suited to your stateless needs. Limitations  The storage for a given Pod must either be provisioned by a
PersistentVolume Provisioner
based on the requested storage class, or pre-provisioned by an admin. Deleting and/or scaling a StatefulSet down will not delete the volumes associated with the
StatefulSet. This is done to ensure data safety, which is generally more valuable than an
automatic purge of all related StatefulSet resources. StatefulSets currently require a Headless Service
to be responsible for the network identity of the Pods. You are responsible for creating this
Service. StatefulSets do not provide any guarantees on the termination of pods when a StatefulSet is
deleted. To achieve ordered and graceful termination of the pods in the StatefulSet, it is
possible to scale the StatefulSet down to 0 prior to deletion. When using Rolling Updates with the default
Pod Management Policy (OrderedReady),
it's possible to get into a broken state that requires
manual intervention to repair.  Components The example below demonstrates the components of a StatefulSet.   apiVersion: v1
kind: Service
metadata:
 name: nginx
 labels:
 app: nginx
spec:
 ports:
 - port: 80
 name: web
 clusterIP: None
 selector:
 app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
 name: web
spec:
 selector:
 matchLabels:
 app: nginx # has to match .spec.template.metadata.labels
 serviceName: "nginx"
 replicas: 3 # by default is 1
 minReadySeconds: 10 # by default is 0
 template:
 metadata:
 labels:
 app: nginx # has to match .spec.selector.matchLabels
 spec:
 terminationGracePeriodSeconds: 10
 containers:
 - name: nginx
 image: registry.k8s.io/nginx-slim:0.8
 ports:
 - containerPort: 80
 name: web
 volumeMounts:
 - name: www
 mountPath: /usr/share/nginx/html
 volumeClaimTemplates:
 - metadata:
 name: www
 spec:
 accessModes: [ "ReadWriteOnce" ]
 storageClassName: "my-storage-class"
 resources:
 requests:
 storage: 1Gi
   In the above example:  A Headless Service, named nginx, is used to control the network domain. The StatefulSet, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods. The volumeClaimTemplates will provide stable storage using
PersistentVolumes provisioned by a
PersistentVolume Provisioner.  The name of a StatefulSet object must be a valid
DNS label. Pod Selector You must set the .spec.selector field of a StatefulSet to match the labels of its
.spec.template.metadata.labels. Failing to specify a matching Pod Selector will result in a
validation error during StatefulSet creation. Volume Claim Templates You can set the .spec.volumeClaimTemplates which can provide stable storage using
PersistentVolumes provisioned by a PersistentVolume
Provisioner. Minimum ready seconds  FEATURE STATE: Kubernetes v1.25 [stable]   .spec.minReadySeconds is an optional field that specifies the minimum number of seconds for which a newly
created Pod should be running and ready without any of its containers crashing, for it to be considered available.
This is used to check progression of a rollout when using a Rolling Update strategy.
This field defaults to 0 (the Pod will be considered available as soon as it is ready). To learn more about when
a Pod is considered ready, see Container Probes. Pod Identity StatefulSet Pods have a unique identity that consists of an ordinal, a
stable network identity, and stable storage. The identity sticks to the Pod,
regardless of which node it's (re)scheduled on. Ordinal Index For a StatefulSet with N replicas, each Pod in the StatefulSet
will be assigned an integer ordinal, that is unique over the Set. By default,
pods will be assigned ordinals from 0 up through N-1. Start ordinal  FEATURE STATE: Kubernetes v1.26 [alpha]   .spec.ordinals is an optional field that allows you to configure the integer
ordinals assigned to each Pod. It defaults to nil. You must enable the
StatefulSetStartOrdinal
feature gate to
use this field. Once enabled, you can configure the following options:   .spec.ordinals.start: If the .spec.ordinals.start field is set, Pods will
be assigned ordinals from .spec.ordinals.start up through
.spec.ordinals.start + .spec.replicas - 1.  Stable Network ID Each Pod in a StatefulSet derives its hostname from the name of the StatefulSet
and the ordinal of the Pod. The pattern for the constructed hostname
is $(statefulset name)-$(ordinal). The example above will create three Pods
named web-0,web-1,web-2.
A StatefulSet can use a Headless Service
to control the domain of its Pods. The domain managed by this Service takes the form:
$(service name).$(namespace).svc.cluster.local, where "cluster.local" is the
cluster domain.
As each Pod is created, it gets a matching DNS subdomain, taking the form:
$(podname).$(governing service domain), where the governing service is defined
by the serviceName field on the StatefulSet. Depending on how DNS is configured in your cluster, you may not be able to look up the DNS
name for a newly-run Pod immediately. This behavior can occur when other clients in the
cluster have already sent queries for the hostname of the Pod before it was created.
Negative caching (normal in DNS) means that the results of previous failed lookups are
remembered and reused, even after the Pod is running, for at least a few seconds. If you need to discover Pods promptly after they are created, you have a few options:  Query the Kubernetes API directly (for example, using a watch) rather than relying on DNS lookups. Decrease the time of caching in your Kubernetes DNS provider (typically this means editing the
config map for CoreDNS, which currently caches for 30 seconds).  As mentioned in the limitations section, you are responsible for
creating the Headless Service
responsible for the network identity of the pods. Here are some examples of choices for Cluster Domain, Service name,
StatefulSet name, and how that affects the DNS names for the StatefulSet's Pods.    Cluster Domain Service (ns/name) StatefulSet (ns/name) StatefulSet Domain Pod DNS Pod Hostname     cluster.local default/nginx default/web nginx.default.svc.cluster.local web-{0..N-1}.nginx.default.svc.cluster.local web-{0..N-1}   cluster.local foo/nginx foo/web nginx.foo.svc.cluster.local web-{0..N-1}.nginx.foo.svc.cluster.local web-{0..N-1}   kube.local foo/nginx foo/web nginx.foo.svc.kube.local web-{0..N-1}.nginx.foo.svc.kube.local web-{0..N-1}    Stable Storage For each VolumeClaimTemplate entry defined in a StatefulSet, each Pod receives one
PersistentVolumeClaim. In the nginx example above, each Pod receives a single PersistentVolume
with a StorageClass of my-storage-class and 1 Gib of provisioned storage. If no StorageClass
is specified, then the default StorageClass will be used. When a Pod is (re)scheduled
onto a node, its volumeMounts mount the PersistentVolumes associated with its
PersistentVolume Claims. Note that, the PersistentVolumes associated with the
Pods' PersistentVolume Claims are not deleted when the Pods, or StatefulSet are deleted.
This must be done manually. Pod Name Label When the StatefulSet controller creates a Pod,
it adds a label, statefulset.kubernetes.io/pod-name, that is set to the name of
the Pod. This label allows you to attach a Service to a specific Pod in
the StatefulSet. Deployment and Scaling Guarantees  For a StatefulSet with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}. When Pods are being deleted, they are terminated in reverse order, from {N-1..0}. Before a scaling operation is applied to a Pod, all of its predecessors must be Running and Ready. Before a Pod is terminated, all of its successors must be completely shutdown.  The StatefulSet should not specify a pod.Spec.TerminationGracePeriodSeconds of 0. This practice
is unsafe and strongly discouraged. For further explanation, please refer to
force deleting StatefulSet Pods. When the nginx example above is created, three Pods will be deployed in the order
web-0, web-1, web-2. web-1 will not be deployed before web-0 is
Running and Ready, and web-2 will not be deployed until
web-1 is Running and Ready. If web-0 should fail, after web-1 is Running and Ready, but before
web-2 is launched, web-2 will not be launched until web-0 is successfully relaunched and
becomes Running and Ready. If a user were to scale the deployed example by patching the StatefulSet such that
replicas=1, web-2 would be terminated first. web-1 would not be terminated until web-2
is fully shutdown and deleted. If web-0 were to fail after web-2 has been terminated and
is completely shutdown, but prior to web-1's termination, web-1 would not be terminated
until web-0 is Running and Ready. Pod Management Policies StatefulSet allows you to relax its ordering guarantees while
preserving its uniqueness and identity guarantees via its .spec.podManagementPolicy field. OrderedReady Pod Management  OrderedReady pod management is the default for StatefulSets. It implements the behavior
described above. Parallel Pod Management  Parallel pod management tells the StatefulSet controller to launch or
terminate all Pods in parallel, and to not wait for Pods to become Running
and Ready or completely terminated prior to launching or terminating another
Pod. This option only affects the behavior for scaling operations. Updates are not
affected. Update strategies A StatefulSet's .spec.updateStrategy field allows you to configure
and disable automated rolling updates for containers, labels, resource request/limits, and
annotations for the Pods in a StatefulSet. There are two possible values:   OnDelete  When a StatefulSet's .spec.updateStrategy.type is set to OnDelete,
the StatefulSet controller will not automatically update the Pods in a
StatefulSet. Users must manually delete Pods to cause the controller to
create new Pods that reflect modifications made to a StatefulSet's .spec.template.  RollingUpdate  The RollingUpdate update strategy implements automated, rolling updates for the Pods in a
StatefulSet. This is the default update strategy.  Rolling Updates When a StatefulSet's .spec.updateStrategy.type is set to RollingUpdate, the
StatefulSet controller will delete and recreate each Pod in the StatefulSet. It will proceed
in the same order as Pod termination (from the largest ordinal to the smallest), updating
each Pod one at a time. The Kubernetes control plane waits until an updated Pod is Running and Ready prior
to updating its predecessor. If you have set .spec.minReadySeconds (see
Minimum Ready Seconds), the control plane additionally waits that
amount of time after the Pod turns ready, before moving on. Partitioned rolling updates The RollingUpdate update strategy can be partitioned, by specifying a
.spec.updateStrategy.rollingUpdate.partition. If a partition is specified, all Pods with an
ordinal that is greater than or equal to the partition will be updated when the StatefulSet's
.spec.template is updated. All Pods with an ordinal that is less than the partition will not
be updated, and, even if they are deleted, they will be recreated at the previous version. If a
StatefulSet's .spec.updateStrategy.rollingUpdate.partition is greater than its .spec.replicas,
updates to its .spec.template will not be propagated to its Pods.
In most cases you will not need to use a partition, but they are useful if you want to stage an
update, roll out a canary, or perform a phased roll out. Maximum unavailable Pods  FEATURE STATE: Kubernetes v1.24 [alpha]  You can control the maximum number of Pods that can be unavailable during an update
by specifying the .spec.updateStrategy.rollingUpdate.maxUnavailable field.
The value can be an absolute number (for example, 5) or a percentage of desired
Pods (for example, 10%). Absolute number is calculated from the percentage value
by rounding it up. This field cannot be 0. The default setting is 1. This field applies to all Pods in the range 0 to replicas - 1. If there is any
unavailable Pod in the range 0 to replicas - 1, it will be counted towards
maxUnavailable. Forced rollback When using Rolling Updates with the default
Pod Management Policy (OrderedReady),
it's possible to get into a broken state that requires manual intervention to repair. If you update the Pod template to a configuration that never becomes Running and
Ready (for example, due to a bad binary or application-level configuration error),
StatefulSet will stop the rollout and wait. In this state, it's not enough to revert the Pod template to a good configuration.
Due to a known issue,
StatefulSet will continue to wait for the broken Pod to become Ready
(which never happens) before it will attempt to revert it back to the working
configuration. After reverting the template, you must also delete any Pods that StatefulSet had
already attempted to run with the bad configuration.
StatefulSet will then begin to recreate the Pods using the reverted template. PersistentVolumeClaim retention  FEATURE STATE: Kubernetes v1.23 [alpha]  The optional .spec.persistentVolumeClaimRetentionPolicy field controls if
and how PVCs are deleted during the lifecycle of a StatefulSet. You must enable the
StatefulSetAutoDeletePVC feature gate
on the API server and the controller manager to use this field.
Once enabled, there are two policies you can configure for each StatefulSet:   whenDeleted  configures the volume retention behavior that applies when the StatefulSet is deleted  whenScaled  configures the volume retention behavior that applies when the replica count of
the StatefulSet is reduced; for example, when scaling down the set.  For each policy that you can configure, you can set the value to either Delete or Retain.   Delete  The PVCs created from the StatefulSet volumeClaimTemplate are deleted for each Pod
affected by the policy. With the whenDeleted policy all PVCs from the
volumeClaimTemplate are deleted after their Pods have been deleted. With the
whenScaled policy, only PVCs corresponding to Pod replicas being scaled down are
deleted, after their Pods have been deleted.  Retain (default) PVCs from the volumeClaimTemplate are not affected when their Pod is
deleted. This is the behavior before this new feature.  Bear in mind that these policies only apply when Pods are being removed due to the
StatefulSet being deleted or scaled down. For example, if a Pod associated with a StatefulSet
fails due to node failure, and the control plane creates a replacement Pod, the StatefulSet
retains the existing PVC. The existing volume is unaffected, and the cluster will attach it to
the node where the new Pod is about to launch. The default for policies is Retain, matching the StatefulSet behavior before this new feature. Here is an example policy.   apiVersion: apps/v1
kind: StatefulSet
...
spec:
 persistentVolumeClaimRetentionPolicy:
 whenDeleted: Retain
 whenScaled: Delete
...
   The StatefulSet controller adds
owner references
to its PVCs, which are then deleted by the garbage collector after the Pod is terminated. This enables the Pod to
cleanly unmount all volumes before the PVCs are deleted (and before the backing PV and
volume are deleted, depending on the retain policy). When you set the whenDeleted
policy to Delete, an owner reference to the StatefulSet instance is placed on all PVCs
associated with that StatefulSet. The whenScaled policy must delete PVCs only when a Pod is scaled down, and not when a
Pod is deleted for another reason. When reconciling, the StatefulSet controller compares
its desired replica count to the actual Pods present on the cluster. Any StatefulSet Pod
whose id greater than the replica count is condemned and marked for deletion. If the
whenScaled policy is Delete, the condemned Pods are first set as owners to the
associated StatefulSet template PVCs, before the Pod is deleted. This causes the PVCs
to be garbage collected after only the condemned Pods have terminated. This means that if the controller crashes and restarts, no Pod will be deleted before its
owner reference has been updated appropriate to the policy. If a condemned Pod is
force-deleted while the controller is down, the owner reference may or may not have been
set up, depending on when the controller crashed. It may take several reconcile loops to
update the owner references, so some condemned Pods may have set up owner references and
others may not. For this reason we recommend waiting for the controller to come back up,
which will verify owner references before terminating Pods. If that is not possible, the
operator should verify the owner references on PVCs to ensure the expected objects are
deleted when Pods are force-deleted. Replicas  .spec.replicas is an optional field that specifies the number of desired Pods. It defaults to 1. Should you manually scale a deployment, example via kubectl scale statefulset statefulset --replicas=X, and then you update that StatefulSet
based on a manifest (for example: by running kubectl apply -f statefulset.yaml), then applying that manifest overwrites the manual scaling
that you previously did. If a HorizontalPodAutoscaler
(or any similar API for horizontal scaling) is managing scaling for a
Statefulset, don't set .spec.replicas. Instead, allow the Kubernetes
control plane to manage
the .spec.replicas field automatically. What's next  Learn about Pods. Find out how to use StatefulSets Follow an example of deploying a stateful application. Follow an example of deploying Cassandra with Stateful Sets. Follow an example of running a replicated stateful application. Learn how to scale a StatefulSet. Learn what's involved when you delete a StatefulSet. Learn how to configure a Pod to use a volume for storage. Learn how to configure a Pod to use a PersistentVolume for storage.    StatefulSet is a top-level resource in the Kubernetes REST API.
Read the
StatefulSet
object definition to understand the API for stateful sets. Read about PodDisruptionBudget and how
you can use it to manage application availability during disruptions.
StatefulSets
https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#:~:text=StatefulSet%20is%20the%20workload%20API,on%20an%20identical%20container%20spec.
https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
DaemonSet
2022-08-31
None
<div><div><p>A <em>DaemonSet</em> ensures that all (or some) Nodes run a copy of a Pod. As nodes are added to the
cluster, Pods are added to them. As nodes are removed from the cluster, those Pods are garbage
collected. Deleting a DaemonSet will clean up the Pods it created.</p><p>Some typical uses of a DaemonSet are:</p><ul><li>running a cluster storage daemon on every node</li><li>running a logs collection daemon on every node</li><li>running a node monitoring daemon on every node</li></ul><p>In a simple case, one DaemonSet, covering all nodes, would be used for each type of daemon.
A more complex setup might use multiple DaemonSets for a single type of daemon, but with
different flags and/or different memory and cpu requests for different hardware types.</p><h2>Writing a DaemonSet Spec</h2><h3>Create a DaemonSet</h3><p>You can describe a DaemonSet in a YAML file. For example, the <code>daemonset.yaml</code> file below
describes a DaemonSet that runs the fluentd-elasticsearch Docker image:</p><div><pre><code>apiVersion: apps/v1
kind: DaemonSet
metadata:
 name: fluentd-elasticsearch
 namespace: kube-system
 labels:
 k8s-app: fluentd-logging
spec:
 selector:
 matchLabels:
 name: fluentd-elasticsearch
 template:
 metadata:
 labels:
 name: fluentd-elasticsearch
 spec:
 tolerations:
 # these tolerations are to have the daemonset runnable on control plane nodes
 # remove them if your control plane nodes should not run pods
 - key: node-role.kubernetes.io/control-plane
 operator: Exists
 effect: NoSchedule
 - key: node-role.kubernetes.io/master
 operator: Exists
 effect: NoSchedule
 containers:
 - name: fluentd-elasticsearch
 image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2
 resources:
 limits:
 memory: 200Mi
 requests:
 cpu: 100m
 memory: 200Mi
 volumeMounts:
 - name: varlog
 mountPath: /var/log
 terminationGracePeriodSeconds: 30
 volumes:
 - name: varlog
 hostPath:
 path: /var/log
</code></pre></div><p>Create a DaemonSet based on the YAML file:</p><pre><code>kubectl apply -f https://k8s.io/examples/controllers/daemonset.yaml
</code></pre><h3>Required Fields</h3><p>As with all other Kubernetes config, a DaemonSet needs <code>apiVersion</code>, <code>kind</code>, and <code>metadata</code> fields. For
general information about working with config files, see
<a href="/docs/tasks/run-application/run-stateless-application-deployment/">running stateless applications</a>
and <a href="/docs/concepts/overview/working-with-objects/object-management/">object management using kubectl</a>.</p><p>The name of a DaemonSet object must be a valid
<a href="/docs/concepts/overview/working-with-objects/names#dns-subdomain-names">DNS subdomain name</a>.</p><p>A DaemonSet also needs a
<a href="https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status"><code>.spec</code></a>
section.</p><h3>Pod Template</h3><p>The <code>.spec.template</code> is one of the required fields in <code>.spec</code>.</p><p>The <code>.spec.template</code> is a <a href="/docs/concepts/workloads/pods/#pod-templates">pod template</a>.
It has exactly the same schema as a <a href="/docs/concepts/workloads/pods/">Pod</a>,
except it is nested and does not have an <code>apiVersion</code> or <code>kind</code>.</p><p>In addition to required fields for a Pod, a Pod template in a DaemonSet has to specify appropriate
labels (see <a href="#pod-selector">pod selector</a>).</p><p>A Pod Template in a DaemonSet must have a <a href="/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy"><code>RestartPolicy</code></a>
equal to <code>Always</code>, or be unspecified, which defaults to <code>Always</code>.</p><h3>Pod Selector</h3><p>The <code>.spec.selector</code> field is a pod selector. It works the same as the <code>.spec.selector</code> of
a <a href="/docs/concepts/workloads/controllers/job/">Job</a>.</p><p>You must specify a pod selector that matches the labels of the
<code>.spec.template</code>.
Also, once a DaemonSet is created,
its <code>.spec.selector</code> can not be mutated. Mutating the pod selector can lead to the
unintentional orphaning of Pods, and it was found to be confusing to users.</p><p>The <code>.spec.selector</code> is an object consisting of two fields:</p><ul><li><code>matchLabels</code> - works the same as the <code>.spec.selector</code> of a
<a href="/docs/concepts/workloads/controllers/replicationcontroller/">ReplicationController</a>.</li><li><code>matchExpressions</code> - allows to build more sophisticated selectors by specifying key,
list of values and an operator that relates the key and values.</li></ul><p>When the two are specified the result is ANDed.</p><p>The <code>.spec.selector</code> must match the <code>.spec.template.metadata.labels</code>.
Config with these two not matching will be rejected by the API.</p><h3>Running Pods on select Nodes</h3><p>If you specify a <code>.spec.template.spec.nodeSelector</code>, then the DaemonSet controller will
create Pods on nodes which match that <a href="/docs/concepts/scheduling-eviction/assign-pod-node/">node selector</a>.
Likewise if you specify a <code>.spec.template.spec.affinity</code>,
then DaemonSet controller will create Pods on nodes which match that
<a href="/docs/concepts/scheduling-eviction/assign-pod-node/">node affinity</a>.
If you do not specify either, then the DaemonSet controller will create Pods on all nodes.</p><h2>How Daemon Pods are scheduled</h2><h3>Scheduled by default scheduler</h3><p><b>FEATURE STATE:</b> <code>Kubernetes 1.17 [stable]</code></p><p>A DaemonSet ensures that all eligible nodes run a copy of a Pod. Normally, the
node that a Pod runs on is selected by the Kubernetes scheduler. However,
DaemonSet pods are created and scheduled by the DaemonSet controller instead.
That introduces the following issues:</p><ul><li>Inconsistent Pod behavior: Normal Pods waiting to be scheduled are created
and in <code>Pending</code> state, but DaemonSet pods are not created in <code>Pending</code>
state. This is confusing to the user.</li><li><a href="/docs/concepts/scheduling-eviction/pod-priority-preemption/">Pod preemption</a>
is handled by default scheduler. When preemption is enabled, the DaemonSet controller
will make scheduling decisions without considering pod priority and preemption.</li></ul><p><code>ScheduleDaemonSetPods</code> allows you to schedule DaemonSets using the default
scheduler instead of the DaemonSet controller, by adding the <code>NodeAffinity</code> term
to the DaemonSet pods, instead of the <code>.spec.nodeName</code> term. The default
scheduler is then used to bind the pod to the target host. If node affinity of
the DaemonSet pod already exists, it is replaced (the original node affinity was
taken into account before selecting the target host). The DaemonSet controller only
performs these operations when creating or modifying DaemonSet pods, and no
changes are made to the <code>spec.template</code> of the DaemonSet.</p><div><pre><code>nodeAffinity:
 requiredDuringSchedulingIgnoredDuringExecution:
 nodeSelectorTerms:
 - matchFields:
 - key: metadata.name
 operator: In
 values:
 - target-host-name
</code></pre></div><p>In addition, <code>node.kubernetes.io/unschedulable:NoSchedule</code> toleration is added
automatically to DaemonSet Pods. The default scheduler ignores
<code>unschedulable</code> Nodes when scheduling DaemonSet Pods.</p><h3>Taints and Tolerations</h3><p>Although Daemon Pods respect
<a href="/docs/concepts/scheduling-eviction/taint-and-toleration/">taints and tolerations</a>,
the following tolerations are added to DaemonSet Pods automatically according to
the related features.</p><table><thead><tr><th>Toleration Key</th><th>Effect</th><th>Version</th><th>Description</th></tr></thead><tbody><tr><td><code>node.kubernetes.io/not-ready</code></td><td>NoExecute</td><td>1.13+</td><td>DaemonSet pods will not be evicted when there are node problems such as a network partition.</td></tr><tr><td><code>node.kubernetes.io/unreachable</code></td><td>NoExecute</td><td>1.13+</td><td>DaemonSet pods will not be evicted when there are node problems such as a network partition.</td></tr><tr><td><code>node.kubernetes.io/disk-pressure</code></td><td>NoSchedule</td><td>1.8+</td><td>DaemonSet pods tolerate disk-pressure attributes by default scheduler.</td></tr><tr><td><code>node.kubernetes.io/memory-pressure</code></td><td>NoSchedule</td><td>1.8+</td><td>DaemonSet pods tolerate memory-pressure attributes by default scheduler.</td></tr><tr><td><code>node.kubernetes.io/unschedulable</code></td><td>NoSchedule</td><td>1.12+</td><td>DaemonSet pods tolerate unschedulable attributes by default scheduler.</td></tr><tr><td><code>node.kubernetes.io/network-unavailable</code></td><td>NoSchedule</td><td>1.12+</td><td>DaemonSet pods, who uses host network, tolerate network-unavailable attributes by default scheduler.</td></tr></tbody></table><h2>Communicating with Daemon Pods</h2><p>Some possible patterns for communicating with Pods in a DaemonSet are:</p><ul><li><strong>Push</strong>: Pods in the DaemonSet are configured to send updates to another service, such
as a stats database. They do not have clients.</li><li><strong>NodeIP and Known Port</strong>: Pods in the DaemonSet can use a <code>hostPort</code>, so that the pods
are reachable via the node IPs.
Clients know the list of node IPs somehow, and know the port by convention.</li><li><strong>DNS</strong>: Create a <a href="/docs/concepts/services-networking/service/#headless-services">headless service</a>
with the same pod selector, and then discover DaemonSets using the <code>endpoints</code>
resource or retrieve multiple A records from DNS.</li><li><strong>Service</strong>: Create a service with the same Pod selector, and use the service to reach a
daemon on a random node. (No way to reach specific node.)</li></ul><h2>Updating a DaemonSet</h2><p>If node labels are changed, the DaemonSet will promptly add Pods to newly matching nodes and delete
Pods from newly not-matching nodes.</p><p>You can modify the Pods that a DaemonSet creates. However, Pods do not allow all
fields to be updated. Also, the DaemonSet controller will use the original template the next
time a node (even with the same name) is created.</p><p>You can delete a DaemonSet. If you specify <code>--cascade=orphan</code> with <code>kubectl</code>, then the Pods
will be left on the nodes. If you subsequently create a new DaemonSet with the same selector,
the new DaemonSet adopts the existing Pods. If any Pods need replacing the DaemonSet replaces
them according to its <code>updateStrategy</code>.</p><p>You can <a href="/docs/tasks/manage-daemon/update-daemon-set/">perform a rolling update</a> on a DaemonSet.</p><h2>Alternatives to DaemonSet</h2><h3>Init scripts</h3><p>It is certainly possible to run daemon processes by directly starting them on a node (e.g. using
<code>init</code>, <code>upstartd</code>, or <code>systemd</code>). This is perfectly fine. However, there are several advantages to
running such processes via a DaemonSet:</p><ul><li>Ability to monitor and manage logs for daemons in the same way as applications.</li><li>Same config language and tools (e.g. Pod templates, <code>kubectl</code>) for daemons and applications.</li><li>Running daemons in containers with resource limits increases isolation between daemons from app
containers. However, this can also be accomplished by running the daemons in a container but not in a Pod.</li></ul><h3>Bare Pods</h3><p>It is possible to create Pods directly which specify a particular node to run on. However,
a DaemonSet replaces Pods that are deleted or terminated for any reason, such as in the case of
node failure or disruptive node maintenance, such as a kernel upgrade. For this reason, you should
use a DaemonSet rather than creating individual Pods.</p><h3>Static Pods</h3><p>It is possible to create Pods by writing a file to a certain directory watched by Kubelet. These
are called <a href="/docs/tasks/configure-pod-container/static-pod/">static pods</a>.
Unlike DaemonSet, static Pods cannot be managed with kubectl
or other Kubernetes API clients. Static Pods do not depend on the apiserver, making them useful
in cluster bootstrapping cases. Also, static Pods may be deprecated in the future.</p><h3>Deployments</h3><p>DaemonSets are similar to <a href="/docs/concepts/workloads/controllers/deployment/">Deployments</a> in that
they both create Pods, and those Pods have processes which are not expected to terminate (e.g. web servers,
storage servers).</p><p>Use a Deployment for stateless services, like frontends, where scaling up and down the
number of replicas and rolling out updates are more important than controlling exactly which host
the Pod runs on. Use a DaemonSet when it is important that a copy of a Pod always run on
all or certain hosts, if the DaemonSet provides node-level functionality that allows other Pods to run correctly on that particular node.</p><p>For example, <a href="/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/">network plugins</a> often include a component that runs as a DaemonSet. The DaemonSet component makes sure that the node where it's running has working cluster networking.</p><h2>What's next</h2><ul><li>Learn about <a href="/docs/concepts/workloads/pods">Pods</a>.<ul><li>Learn about <a href="#static-pods">static Pods</a>, which are useful for running Kubernetes
<a href="/docs/reference/glossary/?all=true#term-control-plane">control plane</a> components.</li></ul></li><li>Find out how to use DaemonSets<ul><li><a href="/docs/tasks/manage-daemon/update-daemon-set/">Perform a rolling update on a DaemonSet</a></li><li><a href="/docs/tasks/manage-daemon/rollback-daemon-set/">Perform a rollback on a DaemonSet</a>
(for example, if a roll out didn't work how you expected).</li></ul></li><li>Understand <a href="/docs/concepts/scheduling-eviction/assign-pod-node/">how Kubernetes assigns Pods to Nodes</a>.</li><li>Learn about <a href="/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/">device plugins</a> and
<a href="/docs/concepts/cluster-administration/addons/">add ons</a>, which often run as DaemonSets.</li><li><code>DaemonSet</code> is a top-level resource in the Kubernetes REST API.
Read the
<a href="/docs/reference/kubernetes-api/workload-resources/daemon-set-v1/">DaemonSet</a>
object definition to understand the API for daemon sets.</li></ul></div>||||I|||| * Documentation
 * Kubernetes Blog
 * Training
 * Partners
 * Community
 * Case Studies
 * Versions
 Release Information v1.26 v1.25 v1.24 v1.23 v1.22
 * English
 中文 (Chinese) 한국어 (Korean) 日本語 (Japanese) Español (Spanish) Bahasa Indonesia
 * 
 + Home
 o Available Documentation Versions
 + Getting started
 o Learning environment
 o Production environment
 - Container Runtimes
 - Installing Kubernetes with deployment tools
 * Bootstrapping clusters with kubeadm
 + Installing kubeadm
 + Troubleshooting kubeadm
 + Creating a cluster with kubeadm
 + Customizing components with the kubeadm API
 + Options for Highly Available Topology
 + Creating Highly Available Clusters with kubeadm
 + Set up a High Availability etcd Cluster with kubeadm
 + Configuring each kubelet in your cluster using kubeadm
 + Dual-stack support with kubeadm
 * Installing Kubernetes with kOps
 * Installing Kubernetes with Kubespray
 - Turnkey Cloud Solutions
 o Best practices
 - Considerations for large clusters
 - Running in multiple zones
 - Validate node setup
 - Enforcing Pod Security Standards
 - PKI certificates and requirements
 + Concepts
 o Overview
 - Kubernetes Components
 - The Kubernetes API
 - Working with Kubernetes Objects
 * Understanding Kubernetes Objects
 * Kubernetes Object Management
 * Object Names and IDs
 * Labels and Selectors
 * Namespaces
 * Annotations
 * Field Selectors
 * Finalizers
 * Owners and Dependents
 * Recommended Labels
 o Cluster Architecture
 - Nodes
 - Communication between Nodes and the Control Plane
 - Controllers
 - Leases
 - Cloud Controller Manager
 - About cgroup v2
 - Container Runtime Interface (CRI)
 - Garbage Collection
 o Containers
 - Images
 - Container Environment
 - Runtime Class
 - Container Lifecycle Hooks
 o Windows in Kubernetes
 - Windows containers in Kubernetes
 - Guide for scheduling Windows containers in Kubernetes
 o Workloads
 - Pods
 * Pod Lifecycle
 * Init Containers
 * Disruptions
 * Ephemeral Containers
 * User Namespaces
 * Downward API
 - Workload Resources
 * Deployments
 * ReplicaSet
 * StatefulSets
 * DaemonSet
 * Jobs
 * Automatic Cleanup for Finished Jobs
 * CronJob
 * ReplicationController
 o Services, Load Balancing, and Networking
 - Service
 - Ingress
 - Ingress Controllers
 - EndpointSlices
 - Network Policies
 - DNS for Services and Pods
 - IPv4/IPv6 dual-stack
 - Topology Aware Hints
 - Networking on Windows
 - Service ClusterIP allocation
 - Service Internal Traffic Policy
 - Topology-aware traffic routing with topology keys
 o Storage
 - Volumes
 - Persistent Volumes
 - Projected Volumes
 - Ephemeral Volumes
 - Storage Classes
 - Dynamic Volume Provisioning
 - Volume Snapshots
 - Volume Snapshot Classes
 - CSI Volume Cloning
 - Storage Capacity
 - Node-specific Volume Limits
 - Volume Health Monitoring
 - Windows Storage
 o Configuration
 - Configuration Best Practices
 - ConfigMaps
 - Secrets
 - Resource Management for Pods and Containers
 - Organizing Cluster Access Using kubeconfig Files
 - Resource Management for Windows nodes
 o Security
 - Overview of Cloud Native Security
 - Pod Security Standards
 - Pod Security Admission
 - Pod Security Policies
 - Security For Windows Nodes
 - Controlling Access to the Kubernetes API
 - Role Based Access Control Good Practices
 - Good practices for Kubernetes Secrets
 - Multi-tenancy
 - Kubernetes API Server Bypass Risks
 - Security Checklist
 o Policies
 - Limit Ranges
 - Resource Quotas
 - Process ID Limits And Reservations
 - Node Resource Managers
 o Scheduling, Preemption and Eviction
 - Kubernetes Scheduler
 - Assigning Pods to Nodes
 - Pod Overhead
 - Pod Scheduling Readiness
 - Pod Topology Spread Constraints
 - Taints and Tolerations
 - Scheduling Framework
 - Dynamic Resource Allocation
 - Scheduler Performance Tuning
 - Resource Bin Packing
 - Pod Priority and Preemption
 - Node-pressure Eviction
 - API-initiated Eviction
 o Cluster Administration
 - Certificates
 - Managing Resources
 - Cluster Networking
 - Logging Architecture
 - Metrics For Kubernetes System Components
 - System Logs
 - Traces For Kubernetes System Components
 - Proxies in Kubernetes
 - API Priority and Fairness
 - Installing Addons
 o Extending Kubernetes
 - Compute, Storage, and Networking Extensions
 * Network Plugins
 * Device Plugins
 - Extending the Kubernetes API
 * Custom Resources
 * Kubernetes API Aggregation Layer
 - Operator pattern
 + Tasks
 o Install Tools
 - Install and Set Up kubectl on Linux
 - Install and Set Up kubectl on macOS
 - Install and Set Up kubectl on Windows
 o Administer a Cluster
 - Administration with kubeadm
 * Certificate Management with kubeadm
 * Configuring a cgroup driver
 * Reconfiguring a kubeadm cluster
 * Upgrading kubeadm clusters
 * Upgrading Windows nodes
 - Migrating from dockershim
 * Changing the Container Runtime on a Node from Docker Engine to containerd
 * Migrate Docker Engine nodes from dockershim to cri-dockerd
 * Find Out What Container Runtime is Used on a Node
 * Troubleshooting CNI plugin-related errors
 * Check whether dockershim removal affects you
 * Migrating telemetry and security agents from dockershim
 - Generate Certificates Manually
 - Manage Memory, CPU, and API Resources
 * Configure Default Memory Requests and Limits for a Namespace
 * Configure Default CPU Requests and Limits for a Namespace
 * Configure Minimum and Maximum Memory Constraints for a Namespace
 * Configure Minimum and Maximum CPU Constraints for a Namespace
 * Configure Memory and CPU Quotas for a Namespace
 * Configure a Pod Quota for a Namespace
 - Install a Network Policy Provider
 * Use Antrea for NetworkPolicy
 * Use Calico for NetworkPolicy
 * Use Cilium for NetworkPolicy
 * Use Kube-router for NetworkPolicy
 * Romana for NetworkPolicy
 * Weave Net for NetworkPolicy
 - Access Clusters Using the Kubernetes API
 - Advertise Extended Resources for a Node
 - Autoscale the DNS Service in a Cluster
 - Change the default StorageClass
 - Change the Reclaim Policy of a PersistentVolume
 - Cloud Controller Manager Administration
 - Configure a kubelet image credential provider
 - Configure Quotas for API Objects
 - Control CPU Management Policies on the Node
 - Control Topology Management Policies on a node
 - Customizing DNS Service
 - Debugging DNS Resolution
 - Declare Network Policy
 - Developing Cloud Controller Manager
 - Enable Or Disable A Kubernetes API
 - Encrypting Secret Data at Rest
 - Guaranteed Scheduling For Critical Add-On Pods
 - IP Masquerade Agent User Guide
 - Limit Storage Consumption
 - Migrate Replicated Control Plane To Use Cloud Controller Manager
 - Namespaces Walkthrough
 - Operating etcd clusters for Kubernetes
 - Reconfigure a Node's Kubelet in a Live Cluster
 - Reserve Compute Resources for System Daemons
 - Running Kubernetes Node Components as a Non-root User
 - Safely Drain a Node
 - Securing a Cluster
 - Set Kubelet parameters via a config file
 - Share a Cluster with Namespaces
 - Upgrade A Cluster
 - Use Cascading Deletion in a Cluster
 - Using a KMS provider for data encryption
 - Using CoreDNS for Service Discovery
 - Using NodeLocal DNSCache in Kubernetes Clusters
 - Using sysctls in a Kubernetes Cluster
 - Utilizing the NUMA-aware Memory Manager
 - Verify Signed Kubernetes Artifacts
 o Configure Pods and Containers
 - Assign Memory Resources to Containers and Pods
 - Assign CPU Resources to Containers and Pods
 - Configure GMSA for Windows Pods and containers
 - Configure RunAsUserName for Windows pods and containers
 - Create a Windows HostProcess Pod
 - Configure Quality of Service for Pods
 - Assign Extended Resources to a Container
 - Configure a Pod to Use a Volume for Storage
 - Configure a Pod to Use a PersistentVolume for Storage
 - Configure a Pod to Use a Projected Volume for Storage
 - Configure a Security Context for a Pod or Container
 - Configure Service Accounts for Pods
 - Pull an Image from a Private Registry
 - Configure Liveness, Readiness and Startup Probes
 - Assign Pods to Nodes
 - Assign Pods to Nodes using Node Affinity
 - Configure Pod Initialization
 - Attach Handlers to Container Lifecycle Events
 - Configure a Pod to Use a ConfigMap
 - Share Process Namespace between Containers in a Pod
 - Use a User Namespace With a Pod
 - Create static Pods
 - Translate a Docker Compose File to Kubernetes Resources
 - Enforce Pod Security Standards by Configuring the Built-in Admission Controller
 - Enforce Pod Security Standards with Namespace Labels
 - Migrate from PodSecurityPolicy to the Built-In PodSecurity Admission Controller
 o Monitoring, Logging, and Debugging
 - Troubleshooting Applications
 * Debug Pods
 * Debug Services
 * Debug a StatefulSet
 * Determine the Reason for Pod Failure
 * Debug Init Containers
 * Debug Running Pods
 * Get a Shell to a Running Container
 - Troubleshooting Clusters
 * Resource metrics pipeline
 * Tools for Monitoring Resources
 * Monitor Node Health
 * Debugging Kubernetes nodes with crictl
 * Auditing
 * Debugging Kubernetes Nodes With Kubectl
 * Developing and debugging services locally using telepresence
 * Windows debugging tips
 o Manage Kubernetes Objects
 - Declarative Management of Kubernetes Objects Using Configuration Files
 - Declarative Management of Kubernetes Objects Using Kustomize
 - Managing Kubernetes Objects Using Imperative Commands
 - Imperative Management of Kubernetes Objects Using Configuration Files
 - Update API Objects in Place Using kubectl patch
 o Managing Secrets
 - Managing Secrets using kubectl
 - Managing Secrets using Configuration File
 - Managing Secrets using Kustomize
 o Inject Data Into Applications
 - Define a Command and Arguments for a Container
 - Define Dependent Environment Variables
 - Define Environment Variables for a Container
 - Expose Pod Information to Containers Through Environment Variables
 - Expose Pod Information to Containers Through Files
 - Distribute Credentials Securely Using Secrets
 o Run Applications
 - Run a Stateless Application Using a Deployment
 - Run a Single-Instance Stateful Application
 - Run a Replicated Stateful Application
 - Scale a StatefulSet
 - Delete a StatefulSet
 - Force Delete StatefulSet Pods
 - Horizontal Pod Autoscaling
 - HorizontalPodAutoscaler Walkthrough
 - Specifying a Disruption Budget for your Application
 - Accessing the Kubernetes API from a Pod
 o Run Jobs
 - Running Automated Tasks with a CronJob
 - Coarse Parallel Processing Using a Work Queue
 - Fine Parallel Processing Using a Work Queue
 - Indexed Job for Parallel Processing with Static Work Assignment
 - Job with Pod-to-Pod Communication
 - Parallel Processing using Expansions
 - Handling retriable and non-retriable pod failures with Pod failure policy
 o Access Applications in a Cluster
 - Deploy and Access the Kubernetes Dashboard
 - Accessing Clusters
 - Configure Access to Multiple Clusters
 - Use Port Forwarding to Access Applications in a Cluster
 - Use a Service to Access an Application in a Cluster
 - Connect a Frontend to a Backend Using Services
 - Create an External Load Balancer
 - List All Container Images Running in a Cluster
 - Set up Ingress on Minikube with the NGINX Ingress Controller
 - Communicate Between Containers in the Same Pod Using a Shared Volume
 - Configure DNS for a Cluster
 - Access Services Running on Clusters
 o Extend Kubernetes
 - Configure the Aggregation Layer
 - Use Custom Resources
 * Extend the Kubernetes API with CustomResourceDefinitions
 * Versions in CustomResourceDefinitions
 - Set up an Extension API Server
 - Configure Multiple Schedulers
 - Use an HTTP Proxy to Access the Kubernetes API
 - Use a SOCKS5 Proxy to Access the Kubernetes API
 - Set up Konnectivity service
 o TLS
 - Configure Certificate Rotation for the Kubelet
 - Manage TLS Certificates in a Cluster
 - Manual Rotation of CA Certificates
 o Manage Cluster Daemons
 - Perform a Rolling Update on a DaemonSet
 - Perform a Rollback on a DaemonSet
 o Networking
 - Adding entries to Pod /etc/hosts with HostAliases
 - Validate IPv4/IPv6 dual-stack
 o Extend kubectl with plugins
 o Manage HugePages
 o Schedule GPUs
 + Tutorials
 o Hello Minikube
 o Learn Kubernetes Basics
 - Create a Cluster
 * Using Minikube to Create a Cluster
 * Interactive Tutorial - Creating a Cluster
 - Deploy an App
 * Using kubectl to Create a Deployment
 * Interactive Tutorial - Deploying an App
 - Explore Your App
 * Viewing Pods and Nodes
 * Interactive Tutorial - Exploring Your App
 - Expose Your App Publicly
 * Using a Service to Expose Your App
 * Interactive Tutorial - Exposing Your App
 - Scale Your App
 * Running Multiple Instances of Your App
 * Interactive Tutorial - Scaling Your App
 - Update Your App
 * Performing a Rolling Update
 * Interactive Tutorial - Updating Your App
 o Configuration
 - Example: Configuring a Java Microservice
 * Externalizing config using MicroProfile, ConfigMaps and Secrets
 * Interactive Tutorial - Configuring a Java Microservice
 - Configuring Redis using a ConfigMap
 o Security
 - Apply Pod Security Standards at the Cluster Level
 - Apply Pod Security Standards at the Namespace Level
 - Restrict a Container's Access to Resources with AppArmor
 - Restrict a Container's Syscalls with seccomp
 o Stateless Applications
 - Exposing an External IP Address to Access an Application in a Cluster
 - Example: Deploying PHP Guestbook application with Redis
 o Stateful Applications
 - StatefulSet Basics
 - Example: Deploying WordPress and MySQL with Persistent Volumes
 - Example: Deploying Cassandra with a StatefulSet
 - Running ZooKeeper, A Distributed System Coordinator
 o Services
 - Connecting Applications with Services
 - Using Source IP
 + Reference
 o Glossary
 o API Overview
 - Kubernetes API Concepts
 - Server-Side Apply
 - Client Libraries
 - Kubernetes Deprecation Policy
 - Deprecated API Migration Guide
 - Kubernetes API health endpoints
 o API Access Control
 - Authenticating
 - Authenticating with Bootstrap Tokens
 - Certificate Signing Requests
 - Admission Controllers
 - Dynamic Admission Control
 - Managing Service Accounts
 - Authorization Overview
 - Using RBAC Authorization
 - Using ABAC Authorization
 - Using Node Authorization
 - Mapping PodSecurityPolicies to Pod Security Standards
 - Webhook Mode
 - Kubelet authentication/authorization
 - TLS bootstrapping
 - Validating Admission Policy
 o Well-Known Labels, Annotations and Taints
 - Audit Annotations
 o Kubernetes API
 - Workload Resources
 * Pod
 * PodTemplate
 * ReplicationController
 * ReplicaSet
 * Deployment
 * StatefulSet
 * ControllerRevision
 * DaemonSet
 * Job
 * CronJob
 * HorizontalPodAutoscaler
 * HorizontalPodAutoscaler
 * PriorityClass
 * PodScheduling v1alpha1
 * ResourceClaim v1alpha1
 * ResourceClaimTemplate v1alpha1
 * ResourceClass v1alpha1
 - Service Resources
 * Service
 * Endpoints
 * EndpointSlice
 * Ingress
 * IngressClass
 - Config and Storage Resources
 * ConfigMap
 * Secret
 * Volume
 * PersistentVolumeClaim
 * PersistentVolume
 * StorageClass
 * VolumeAttachment
 * CSIDriver
 * CSINode
 * CSIStorageCapacity
 - Authentication Resources
 * ServiceAccount
 * TokenRequest
 * TokenReview
 * CertificateSigningRequest
 - Authorization Resources
 * LocalSubjectAccessReview
 * SelfSubjectAccessReview
 * SelfSubjectRulesReview
 * SubjectAccessReview
 * SelfSubjectReview v1alpha1
 * ClusterRole
 * ClusterRoleBinding
 * Role
 * RoleBinding
 - Policy Resources
 * LimitRange
 * ResourceQuota
 * NetworkPolicy
 * PodDisruptionBudget
 - Extend Resources
 * CustomResourceDefinition
 * MutatingWebhookConfiguration
 * ValidatingWebhookConfiguration
 * ValidatingAdmissionPolicy v1alpha1
 - Cluster Resources
 * Node
 * Namespace
 * Event
 * APIService
 * Lease
 * RuntimeClass
 * FlowSchema v1beta3
 * PriorityLevelConfiguration v1beta3
 * Binding
 * ComponentStatus
 * ClusterCIDR v1alpha1
 - Common Definitions
 * DeleteOptions
 * LabelSelector
 * ListMeta
 * LocalObjectReference
 * NodeSelectorRequirement
 * ObjectFieldSelector
 * ObjectMeta
 * ObjectReference
 * Patch
 * Quantity
 * ResourceFieldSelector
 * Status
 * TypedLocalObjectReference
 - Other Resources
 * ValidatingAdmissionPolicyBindingList v1alpha1
 - Common Parameters
 o Instrumentation
 - Service Level Indicator Metrics
 - Node metrics data
 - Kubernetes Metrics Reference
 o Kubernetes Issues and Security
 - Kubernetes Issue Tracker
 - Kubernetes Security and Disclosure Information
 - Official CVE Feed
 o Node Reference Information
 - Kubelet Checkpoint API
 - Kubelet Device Manager API Versions
 - Articles on dockershim Removal and on Using CRI-compatible Runtimes
 o Networking Reference
 - Protocols for Services
 - Ports and Protocols
 - Virtual IPs and Service Proxies
 o Setup tools
 - Kubeadm
 * kubeadm init
 * kubeadm join
 * kubeadm upgrade
 * kubeadm config
 * kubeadm reset
 * kubeadm token
 * kubeadm version
 * kubeadm alpha
 * kubeadm certs
 * kubeadm init phase
 * kubeadm join phase
 * kubeadm kubeconfig
 * kubeadm reset phase
 * kubeadm upgrade phase
 * Implementation details
 o Command line tool (kubectl)
 - kubectl Cheat Sheet
 - kubectl Commands
 - kubectl
 - JSONPath Support
 - kubectl for Docker Users
 - kubectl Usage Conventions
 o Component tools
 - Feature Gates
 - Feature Gates (removed)
 - kubelet
 - kube-apiserver
 - kube-controller-manager
 - kube-proxy
 - kube-scheduler
 o Configuration APIs
 - Client Authentication (v1)
 - Client Authentication (v1beta1)
 - Event Rate Limit Configuration (v1alpha1)
 - Image Policy API (v1alpha1)
 - kube-apiserver Admission (v1)
 - kube-apiserver Audit Configuration (v1)
 - kube-apiserver Configuration (v1)
 - kube-apiserver Configuration (v1alpha1)
 - kube-apiserver Encryption Configuration (v1)
 - kube-proxy Configuration (v1alpha1)
 - kube-scheduler Configuration (v1)
 - kube-scheduler Configuration (v1beta2)
 - kube-scheduler Configuration (v1beta3)
 - kubeadm Configuration (v1beta2)
 - kubeadm Configuration (v1beta3)
 - kubeconfig (v1)
 - Kubelet Configuration (v1)
 - Kubelet Configuration (v1alpha1)
 - Kubelet Configuration (v1beta1)
 - Kubelet CredentialProvider (v1)
 - Kubelet CredentialProvider (v1alpha1)
 - Kubelet CredentialProvider (v1beta1)
 - WebhookAdmission Configuration (v1)
 o Scheduling
 - Scheduler Configuration
 - Scheduling Policies
 o Other Tools
 - Mapping from dockercli to crictl
 + Contribute
 o Suggesting content improvements
 o Contributing new content
 - Opening a pull request
 - Documenting for a release
 - Blogs and case studies
 o Reviewing changes
 - Reviewing pull requests
 - For approvers and reviewers
 o Localizing Kubernetes documentation
 o Participating in SIG Docs
 - Roles and responsibilities
 - PR wranglers
 o Documentation style overview
 - Content guide
 - Style guide
 - Diagram guide
 - Writing a new topic
 - Page content types
 - Content organization
 - Custom Hugo Shortcodes
 o Updating Reference Documentation
 - Quickstart
 - Contributing to the Upstream Kubernetes Code
 - Generating Reference Documentation for the Kubernetes API
 - Generating Reference Documentation for kubectl Commands
 - Generating Reference Pages for Kubernetes Components and Tools
 - 
 o Advanced contributing
 o Viewing Site Analytics
 + Docs smoke test page
 1. Kubernetes Documentation
 2. Concepts
 3. Workloads
 4. Workload Resources
 5. DaemonSet

 DaemonSet

 A DaemonSet ensures that all (or some) Nodes run a copy of a Pod. As nodes are added to the cluster, Pods are added to them. As nodes are removed from the cluster, those Pods are garbage collected. Deleting a DaemonSet will clean up the Pods it created.

 Some typical uses of a DaemonSet are:

 * running a cluster storage daemon on every node
 * running a logs collection daemon on every node
 * running a node monitoring daemon on every node

 In a simple case, one DaemonSet, covering all nodes, would be used for each type of daemon. A more complex setup might use multiple DaemonSets for a single type of daemon, but with different flags and/or different memory and cpu requests for different hardware types.

 Writing a DaemonSet Spec

 Create a DaemonSet

 You can describe a DaemonSet in a YAML file. For example, the daemonset.yaml file below describes a DaemonSet that runs the fluentd-elasticsearch Docker image:

 controllers/daemonset.yaml
 apiVersion: apps/v1
 kind: DaemonSet
 metadata:
 name: fluentd-elasticsearch
 namespace: kube-system
 labels:
 k8s-app: fluentd-logging
 spec:
 selector:
 matchLabels:
 name: fluentd-elasticsearch
 template:
 metadata:
 labels:
 name: fluentd-elasticsearch
 spec:
 tolerations:
 # these tolerations are to have the daemonset runnable on control plane nodes
 # remove them if your control plane nodes should not run pods
 - key: node-role.kubernetes.io/control-plane
 operator: Exists
 effect: NoSchedule
 - key: node-role.kubernetes.io/master
 operator: Exists
 effect: NoSchedule
 containers:
 - name: fluentd-elasticsearch
 image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2
 resources:
 limits:
 memory: 200Mi
 requests:
 cpu: 100m
 memory: 200Mi
 volumeMounts:
 - name: varlog
 mountPath: /var/log
 terminationGracePeriodSeconds: 30
 volumes:
 - name: varlog
 hostPath:
 path: /var/log
 

 Create a DaemonSet based on the YAML file:

 kubectl apply -f https://k8s.io/examples/controllers/daemonset.yaml
 

 Required Fields

 As with all other Kubernetes config, a DaemonSet needs apiVersion, kind, and metadata fields. For general information about working with config files, see running stateless applications and object management using kubectl.

 The name of a DaemonSet object must be a valid DNS subdomain name.

 A DaemonSet also needs a .spec section.

 Pod Template

 The .spec.template is one of the required fields in .spec.

 The .spec.template is a pod template. It has exactly the same schema as a Pod, except it is nested and does not have an apiVersion or kind.

 In addition to required fields for a Pod, a Pod template in a DaemonSet has to specify appropriate labels (see pod selector).

 A Pod Template in a DaemonSet must have a RestartPolicy equal to Always, or be unspecified, which defaults to Always.

 Pod Selector

 The .spec.selector field is a pod selector. It works the same as the .spec.selector of a Job.

 You must specify a pod selector that matches the labels of the .spec.template. Also, once a DaemonSet is created, its .spec.selector can not be mutated. Mutating the pod selector can lead to the unintentional orphaning of Pods, and it was found to be confusing to users.

 The .spec.selector is an object consisting of two fields:

 * matchLabels - works the same as the .spec.selector of a ReplicationController.
 * matchExpressions - allows to build more sophisticated selectors by specifying key, list of values and an operator that relates the key and values.

 When the two are specified the result is ANDed.

 The .spec.selector must match the .spec.template.metadata.labels. Config with these two not matching will be rejected by the API.

 Running Pods on select Nodes

 If you specify a .spec.template.spec.nodeSelector, then the DaemonSet controller will create Pods on nodes which match that node selector. Likewise if you specify a .spec.template.spec.affinity, then DaemonSet controller will create Pods on nodes which match that node affinity. If you do not specify either, then the DaemonSet controller will create Pods on all nodes.

 How Daemon Pods are scheduled

 Scheduled by default scheduler

 FEATURE STATE: Kubernetes 1.17 [stable]

 A DaemonSet ensures that all eligible nodes run a copy of a Pod. Normally, the node that a Pod runs on is selected by the Kubernetes scheduler. However, DaemonSet pods are created and scheduled by the DaemonSet controller instead. That introduces the following issues:

 * Inconsistent Pod behavior: Normal Pods waiting to be scheduled are created and in Pending state, but DaemonSet pods are not created in Pending state. This is confusing to the user.
 * Pod preemption is handled by default scheduler. When preemption is enabled, the DaemonSet controller will make scheduling decisions without considering pod priority and preemption.

 ScheduleDaemonSetPods allows you to schedule DaemonSets using the default scheduler instead of the DaemonSet controller, by adding the NodeAffinity term to the DaemonSet pods, instead of the .spec.nodeName term. The default scheduler is then used to bind the pod to the target host. If node affinity of the DaemonSet pod already exists, it is replaced (the original node affinity was taken into account before selecting the target host). The DaemonSet controller only performs these operations when creating or modifying DaemonSet pods, and no changes are made to the spec.template of the DaemonSet.

 nodeAffinity:
 requiredDuringSchedulingIgnoredDuringExecution:
 nodeSelectorTerms:
 - matchFields:
 - key: metadata.name
 operator: In
 values:
 - target-host-name
 

 In addition, node.kubernetes.io/unschedulable:NoSchedule toleration is added automatically to DaemonSet Pods. The default scheduler ignores unschedulable Nodes when scheduling DaemonSet Pods.

 Taints and Tolerations

 Although Daemon Pods respect taints and tolerations, the following tolerations are added to DaemonSet Pods automatically according to the related features.

 Toleration Key Effect Version Description  
 node.kubernetes.io/not-ready NoExecute 1.13+ DaemonSet pods will not be evicted when there are node problems such as a network partition. 
 node.kubernetes.io/unreachable NoExecute 1.13+ DaemonSet pods will not be evicted when there are node problems such as a network partition. 
 node.kubernetes.io/disk-pressure NoSchedule 1.8+ DaemonSet pods tolerate disk-pressure attributes by default scheduler. 
 node.kubernetes.io/memory-pressure NoSchedule 1.8+ DaemonSet pods tolerate memory-pressure attributes by default scheduler. 
 node.kubernetes.io/unschedulable NoSchedule 1.12+ DaemonSet pods tolerate unschedulable attributes by default scheduler. 
 node.kubernetes.io/network-unavailable NoSchedule 1.12+ DaemonSet pods, who uses host network, tolerate network-unavailable attributes by default scheduler.
 

 Communicating with Daemon Pods

 Some possible patterns for communicating with Pods in a DaemonSet are:

 * Push: Pods in the DaemonSet are configured to send updates to another service, such as a stats database. They do not have clients.
 * NodeIP and Known Port: Pods in the DaemonSet can use a hostPort, so that the pods are reachable via the node IPs. Clients know the list of node IPs somehow, and know the port by convention.
 * DNS: Create a headless service with the same pod selector, and then discover DaemonSets using the endpoints resource or retrieve multiple A records from DNS.
 * Service: Create a service with the same Pod selector, and use the service to reach a daemon on a random node. (No way to reach specific node.)

 Updating a DaemonSet

 If node labels are changed, the DaemonSet will promptly add Pods to newly matching nodes and delete Pods from newly not-matching nodes.

 You can modify the Pods that a DaemonSet creates. However, Pods do not allow all fields to be updated. Also, the DaemonSet controller will use the original template the next time a node (even with the same name) is created.

 You can delete a DaemonSet. If you specify --cascade=orphan with kubectl, then the Pods will be left on the nodes. If you subsequently create a new DaemonSet with the same selector, the new DaemonSet adopts the existing Pods. If any Pods need replacing the DaemonSet replaces them according to its updateStrategy.

 You can perform a rolling update on a DaemonSet.

 Alternatives to DaemonSet

 Init scripts

 It is certainly possible to run daemon processes by directly starting them on a node (e.g. using init, upstartd, or systemd). This is perfectly fine. However, there are several advantages to running such processes via a DaemonSet:

 * Ability to monitor and manage logs for daemons in the same way as applications.
 * Same config language and tools (e.g. Pod templates, kubectl) for daemons and applications.
 * Running daemons in containers with resource limits increases isolation between daemons from app containers. However, this can also be accomplished by running the daemons in a container but not in a Pod.

 Bare Pods

 It is possible to create Pods directly which specify a particular node to run on. However, a DaemonSet replaces Pods that are deleted or terminated for any reason, such as in the case of node failure or disruptive node maintenance, such as a kernel upgrade. For this reason, you should use a DaemonSet rather than creating individual Pods.

 Static Pods

 It is possible to create Pods by writing a file to a certain directory watched by Kubelet. These are called static pods. Unlike DaemonSet, static Pods cannot be managed with kubectl or other Kubernetes API clients. Static Pods do not depend on the apiserver, making them useful in cluster bootstrapping cases. Also, static Pods may be deprecated in the future.

 Deployments

 DaemonSets are similar to Deployments in that they both create Pods, and those Pods have processes which are not expected to terminate (e.g. web servers, storage servers).

 Use a Deployment for stateless services, like frontends, where scaling up and down the number of replicas and rolling out updates are more important than controlling exactly which host the Pod runs on. Use a DaemonSet when it is important that a copy of a Pod always run on all or certain hosts, if the DaemonSet provides node-level functionality that allows other Pods to run correctly on that particular node.

 For example, network plugins often include a component that runs as a DaemonSet. The DaemonSet component makes sure that the node where it's running has working cluster networking.

 What's next

 * Learn about Pods.
 + Learn about static Pods, which are useful for running Kubernetes control plane components.
 * Find out how to use DaemonSets
 + Perform a rolling update on a DaemonSet
 + Perform a rollback on a DaemonSet (for example, if a roll out didn't work how you expected).
 * Understand how Kubernetes assigns Pods to Nodes.
 * Learn about device plugins and add ons, which often run as DaemonSets.
 * DaemonSet is a top-level resource in the Kubernetes REST API. Read the DaemonSet object definition to understand the API for daemon sets.

 Feedback

 Was this page helpful?

 Yes No

 Thanks for the feedback. If you have a specific, answerable question about how to use Kubernetes, ask it on Stack Overflow. Open an issue in the GitHub repo if you want to report a problem or suggest an improvement.

 Last modified August 31, 2022 at 6:22 PM PST: Removed docker usage from daemontsets (a174afba76)
 Edit this page Create child page Create an issue Print entire section
 * Writing a DaemonSet Spec
 + Create a DaemonSet
 + Required Fields
 + Pod Template
 + Pod Selector
 + Running Pods on select Nodes
 * How Daemon Pods are scheduled
 + Scheduled by default scheduler
 + Taints and Tolerations
 * Communicating with Daemon Pods
 * Updating a DaemonSet
 * Alternatives to DaemonSet
 + Init scripts
 + Bare Pods
 + Static Pods
 + Deployments
 * What's next
 Home Blog Training Partners Community Case Studies
 * 
 * 
 * 
 * 
 * 
 * 
 * 
 * 
 © 2023 The Kubernetes Authors | Documentation Distributed under CC BY 4.0
 Copyright © 2023 The Linux Foundation ®. All rights reserved. The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our Trademark Usage page
 ICP license: 京ICP备17074266号-3
A DaemonSet ensures that all (or some) Nodes run a copy of a Pod. As nodes are added to the
cluster, Pods are added to them. As nodes are removed from the cluster, those Pods are garbage
collected. Deleting a DaemonSet will clean up the Pods it created. Some typical uses of a DaemonSet are:  running a cluster storage daemon on every node running a logs collection daemon on every node running a node monitoring daemon on every node  In a simple case, one DaemonSet, covering all nodes, would be used for each type of daemon.
A more complex setup might use multiple DaemonSets for a single type of daemon, but with
different flags and/or different memory and cpu requests for different hardware types. Writing a DaemonSet Spec Create a DaemonSet You can describe a DaemonSet in a YAML file. For example, the daemonset.yaml file below
describes a DaemonSet that runs the fluentd-elasticsearch Docker image:   apiVersion: apps/v1
kind: DaemonSet
metadata:
 name: fluentd-elasticsearch
 namespace: kube-system
 labels:
 k8s-app: fluentd-logging
spec:
 selector:
 matchLabels:
 name: fluentd-elasticsearch
 template:
 metadata:
 labels:
 name: fluentd-elasticsearch
 spec:
 tolerations:
 # these tolerations are to have the daemonset runnable on control plane nodes
 # remove them if your control plane nodes should not run pods
 - key: node-role.kubernetes.io/control-plane
 operator: Exists
 effect: NoSchedule
 - key: node-role.kubernetes.io/master
 operator: Exists
 effect: NoSchedule
 containers:
 - name: fluentd-elasticsearch
 image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2
 resources:
 limits:
 memory: 200Mi
 requests:
 cpu: 100m
 memory: 200Mi
 volumeMounts:
 - name: varlog
 mountPath: /var/log
 terminationGracePeriodSeconds: 30
 volumes:
 - name: varlog
 hostPath:
 path: /var/log
   Create a DaemonSet based on the YAML file:  kubectl apply -f https://k8s.io/examples/controllers/daemonset.yaml
  Required Fields As with all other Kubernetes config, a DaemonSet needs apiVersion, kind, and metadata fields. For
general information about working with config files, see
running stateless applications
and object management using kubectl. The name of a DaemonSet object must be a valid
DNS subdomain name. A DaemonSet also needs a
 .spec 
section. Pod Template The .spec.template is one of the required fields in .spec. The .spec.template is a pod template.
It has exactly the same schema as a Pod,
except it is nested and does not have an apiVersion or kind. In addition to required fields for a Pod, a Pod template in a DaemonSet has to specify appropriate
labels (see pod selector). A Pod Template in a DaemonSet must have a  RestartPolicy 
equal to Always, or be unspecified, which defaults to Always. Pod Selector The .spec.selector field is a pod selector. It works the same as the .spec.selector of
a Job. You must specify a pod selector that matches the labels of the
.spec.template.
Also, once a DaemonSet is created,
its .spec.selector can not be mutated. Mutating the pod selector can lead to the
unintentional orphaning of Pods, and it was found to be confusing to users. The .spec.selector is an object consisting of two fields:   matchLabels - works the same as the .spec.selector of a
ReplicationController.  matchExpressions - allows to build more sophisticated selectors by specifying key,
list of values and an operator that relates the key and values.  When the two are specified the result is ANDed. The .spec.selector must match the .spec.template.metadata.labels.
Config with these two not matching will be rejected by the API. Running Pods on select Nodes If you specify a .spec.template.spec.nodeSelector, then the DaemonSet controller will
create Pods on nodes which match that node selector.
Likewise if you specify a .spec.template.spec.affinity,
then DaemonSet controller will create Pods on nodes which match that
node affinity.
If you do not specify either, then the DaemonSet controller will create Pods on all nodes. How Daemon Pods are scheduled Scheduled by default scheduler  FEATURE STATE: Kubernetes 1.17 [stable]  A DaemonSet ensures that all eligible nodes run a copy of a Pod. Normally, the
node that a Pod runs on is selected by the Kubernetes scheduler. However,
DaemonSet pods are created and scheduled by the DaemonSet controller instead.
That introduces the following issues:  Inconsistent Pod behavior: Normal Pods waiting to be scheduled are created
and in Pending state, but DaemonSet pods are not created in Pending
state. This is confusing to the user.  Pod preemption
is handled by default scheduler. When preemption is enabled, the DaemonSet controller
will make scheduling decisions without considering pod priority and preemption.   ScheduleDaemonSetPods allows you to schedule DaemonSets using the default
scheduler instead of the DaemonSet controller, by adding the NodeAffinity term
to the DaemonSet pods, instead of the .spec.nodeName term. The default
scheduler is then used to bind the pod to the target host. If node affinity of
the DaemonSet pod already exists, it is replaced (the original node affinity was
taken into account before selecting the target host). The DaemonSet controller only
performs these operations when creating or modifying DaemonSet pods, and no
changes are made to the spec.template of the DaemonSet.   nodeAffinity:
 requiredDuringSchedulingIgnoredDuringExecution:
 nodeSelectorTerms:
 - matchFields:
 - key: metadata.name
 operator: In
 values:
 - target-host-name
   In addition, node.kubernetes.io/unschedulable:NoSchedule toleration is added
automatically to DaemonSet Pods. The default scheduler ignores
unschedulable Nodes when scheduling DaemonSet Pods. Taints and Tolerations Although Daemon Pods respect
taints and tolerations,
the following tolerations are added to DaemonSet Pods automatically according to
the related features.    Toleration Key Effect Version Description      node.kubernetes.io/not-ready  NoExecute 1.13+ DaemonSet pods will not be evicted when there are node problems such as a network partition.    node.kubernetes.io/unreachable  NoExecute 1.13+ DaemonSet pods will not be evicted when there are node problems such as a network partition.    node.kubernetes.io/disk-pressure  NoSchedule 1.8+ DaemonSet pods tolerate disk-pressure attributes by default scheduler.    node.kubernetes.io/memory-pressure  NoSchedule 1.8+ DaemonSet pods tolerate memory-pressure attributes by default scheduler.    node.kubernetes.io/unschedulable  NoSchedule 1.12+ DaemonSet pods tolerate unschedulable attributes by default scheduler.    node.kubernetes.io/network-unavailable  NoSchedule 1.12+ DaemonSet pods, who uses host network, tolerate network-unavailable attributes by default scheduler.    Communicating with Daemon Pods Some possible patterns for communicating with Pods in a DaemonSet are:   Push: Pods in the DaemonSet are configured to send updates to another service, such
as a stats database. They do not have clients.  NodeIP and Known Port: Pods in the DaemonSet can use a hostPort, so that the pods
are reachable via the node IPs.
Clients know the list of node IPs somehow, and know the port by convention.  DNS: Create a headless service
with the same pod selector, and then discover DaemonSets using the endpoints
resource or retrieve multiple A records from DNS.  Service: Create a service with the same Pod selector, and use the service to reach a
daemon on a random node. (No way to reach specific node.)  Updating a DaemonSet If node labels are changed, the DaemonSet will promptly add Pods to newly matching nodes and delete
Pods from newly not-matching nodes. You can modify the Pods that a DaemonSet creates. However, Pods do not allow all
fields to be updated. Also, the DaemonSet controller will use the original template the next
time a node (even with the same name) is created. You can delete a DaemonSet. If you specify --cascade=orphan with kubectl, then the Pods
will be left on the nodes. If you subsequently create a new DaemonSet with the same selector,
the new DaemonSet adopts the existing Pods. If any Pods need replacing the DaemonSet replaces
them according to its updateStrategy. You can perform a rolling update on a DaemonSet. Alternatives to DaemonSet Init scripts It is certainly possible to run daemon processes by directly starting them on a node (e.g. using
init, upstartd, or systemd). This is perfectly fine. However, there are several advantages to
running such processes via a DaemonSet:  Ability to monitor and manage logs for daemons in the same way as applications. Same config language and tools (e.g. Pod templates, kubectl) for daemons and applications. Running daemons in containers with resource limits increases isolation between daemons from app
containers. However, this can also be accomplished by running the daemons in a container but not in a Pod.  Bare Pods It is possible to create Pods directly which specify a particular node to run on. However,
a DaemonSet replaces Pods that are deleted or terminated for any reason, such as in the case of
node failure or disruptive node maintenance, such as a kernel upgrade. For this reason, you should
use a DaemonSet rather than creating individual Pods. Static Pods It is possible to create Pods by writing a file to a certain directory watched by Kubelet. These
are called static pods.
Unlike DaemonSet, static Pods cannot be managed with kubectl
or other Kubernetes API clients. Static Pods do not depend on the apiserver, making them useful
in cluster bootstrapping cases. Also, static Pods may be deprecated in the future. Deployments DaemonSets are similar to Deployments in that
they both create Pods, and those Pods have processes which are not expected to terminate (e.g. web servers,
storage servers). Use a Deployment for stateless services, like frontends, where scaling up and down the
number of replicas and rolling out updates are more important than controlling exactly which host
the Pod runs on. Use a DaemonSet when it is important that a copy of a Pod always run on
all or certain hosts, if the DaemonSet provides node-level functionality that allows other Pods to run correctly on that particular node. For example, network plugins often include a component that runs as a DaemonSet. The DaemonSet component makes sure that the node where it's running has working cluster networking. What's next  Learn about Pods. Learn about static Pods, which are useful for running Kubernetes
control plane components.   Find out how to use DaemonSets  Perform a rolling update on a DaemonSet   Perform a rollback on a DaemonSet
(for example, if a roll out didn't work how you expected).   Understand how Kubernetes assigns Pods to Nodes. Learn about device plugins and
add ons, which often run as DaemonSets.  DaemonSet is a top-level resource in the Kubernetes REST API.
Read the
DaemonSet
object definition to understand the API for daemon sets.
DaemonSet
https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
https://loft.sh/blog/kubernetes-statefulset-examples-and-best-practices/
Kubernetes StatefulSet - Examples & Best Practices
2021-11-23
Bharathiraja Shanmugam
<div><div><p>Are you planning to deploy a database in the Kubernetes cluster? If so, then you’ve come to the right place. <a href="https://kubernetes.io/">Kubernetes</a> is a container orchestration tool that uses many controllers to run applications as containers (Pods). One of these controllers is called <em><a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/">StatefulSet</a></em>, which is used to run stateful applications.</p>
<p>Deploying stateful applications in the Kubernetes cluster can be a tedious task. This is because the stateful application expects primary-replica architecture and a fixed Pod name. The StatefulSets controller addresses this problem while deploying the stateful application in the Kubernetes cluster.</p>
<p>In this article, you will learn more about what StatefulSets in the Kubernetes cluster are and when to use them, as well as how to deploy the stateful application using the StatefulSets controller in a step-by-step example that includes all the manifest (YAML) files. Furthermore, the StatefulSets concept will be demonstrated using the MySQL database, which covers how to create a fixed Pod name for each MySQL replication and how to access the replicated Pods using the Services object.</p>
<h2><a href="#what-are-stateful-applications">#</a>What Are Stateful Applications?</h2>
<p>Stateful applications are applications that store data and keep tracking it. All databases, such as MySQL, Oracle, and PostgreSQL, are examples of stateful applications. Stateless applications, on the other hand, do not keep the data. Node.js and Nginx are examples of stateless applications. For each request, the stateless application will receive new data and process it.</p>
<p>In a modern web application, the stateless application connects with stateful applications to serve the user’s request. A Node.js application is a stateless application that receives new data on each request from the user. This application is then connected with a stateful application, such as a MySQL database, to process the data. MySQL stores data and keeps updating the data based on the user’s request.</p>
<p></p>
<p>Read on to learn more about StatefulSets in the Kubernetes cluster—what they are, when to use them, how to create them, and what the best practices are.</p>
<h2><a href="#what-are-statefulsets">#</a>What Are StatefulSets?</h2>
<p>A StatefulSet is the Kubernetes controller used to run the stateful application as containers (Pods) in the Kubernetes cluster. StatefulSets assign a sticky identity—an ordinal number starting from zero—to each Pod instead of assigning random IDs for each replica Pod. A new Pod is created by cloning the previous Pod’s data. If the previous Pod is in the pending state, then the new Pod will not be created. If you delete a Pod, it will delete the Pod in reverse order, not in random order. For example, if you had four replicas and you scaled down to three, it will delete the Pod numbered 3.</p>
<p>The diagram below shows how the Pod is numbered from zero and how the persistent volume is attached to the Pod in the StatefulSets.</p>
<p></p>
<h2><a href="#when-to-use-statefulsets">#</a>When to Use StatefulSets</h2>
<p>There are several reasons to consider using StatefulSets. Here are two examples:</p>
<ol>
<li>Assume you deployed a MySQL database in the Kubernetes cluster and scaled this to three replicas, and a frontend application wants to access the MySQL cluster to read and write data. The read request will be forwarded to three Pods. However, the write request will only be forwarded to the first (primary) Pod, and the data will be synced with the other Pods. You can achieve this by using StatefulSets.</li>
<li>Deleting or scaling down a StatefulSet will not delete the volumes associated with the stateful application. This gives you your data safety. If you delete the MySQL Pod or if the MySQL Pod restarts, you can have access to the data in the same volume.</li>
</ol>
<h3><a href="#deployment-vs-statefulsets">#</a>Deployment vs. StatefulSets</h3>
<p>You can also create Pods (containers) using the Deployment object in the Kubernetes cluster. This allows you to easily replicate Pods and attach a storage volume to the Pods. The same thing can be done by using StatefulSets. What then is the advantage of using StatefulSets?</p>
<p>Well, the Pods created using the Deployment object are assigned random IDs. For example, you are creating a Pod named “my-app”, and you are scaling it to three replicas. The names of the Pods are created like this:</p>
<div><pre><code>my-app-123ab
my-app-098bd
my-app-890yt
</code></pre></div><p>After the name “my-app”, random IDs are added. If the Pod restarts or you scale it down, then again, the Kubernetes Deployment object will assign different random IDs for each Pod. After restarting, the names of all Pods appear like this:</p>
<div><pre><code>my-app-jk879
my-app-kl097
my-app-76hf7
</code></pre></div><p>All these Pods are associated with one load balancer service. So in a stateless application, changes in the Pod name are easily identified, and the service object easily handles the random IDs of Pods and distributes the load. This type of deployment is very suitable for stateless applications.</p>
<p></p>
<p>However, stateful applications cannot be deployed like this. The stateful application needs a sticky identity for each Pod because replica Pods are not identical Pods.</p>
<p>Take a look at the MySQL database deployment. Assume you are creating Pods for the MySQL database using the Kubernetes Deployment object and scaling the Pods. If you are writing data on one MySQL Pod, do not replicate the same data on another MySQL Pod if the Pod is restarted. This is the first problem with the Kubernetes Deployment object for the stateful application.</p>
<p></p>
<p>Stateful applications always need a sticky identity. While the Kubernetes Deployment object offers random IDs for each Pod, the Kubernetes StatefulSets controller offers an ordinal number for each Pod starting from zero, such as mysql-0, mysql-1, mysql-2, and so forth.</p>
<p></p>
<p>For stateful applications with a StatefulSet controller, it is possible to set the first Pod as primary and other Pods as replicas—the first Pod will handle both read and write requests from the user, and other Pods always sync with the first Pod for data replication. If the Pod dies, a new Pod is created with the same name.</p>
<p>The diagram below shows a MySQL primary and replica architecture with persistent volume and data replication architecture.</p>
<p></p>
<p>Now, add another Pod to that. The fourth Pod will only be created if the third Pod is up and running, and it will clone the data from the previous Pod.</p>
<p></p>
<p>In summary, StatefulSets provide the following advantages when compared to Deployment objects:</p>
<ol>
<li>Ordered numbers for each Pod</li>
<li>The first Pod can be a primary, which makes it a good choice when creating a replicated database setup, which handles both reading and writing</li>
<li>Other Pods act as replicas</li>
<li>New Pods will only be created if the previous Pod is in running state and will clone the previous Pod’s data</li>
<li>Deletion of Pods occurs in reverse order</li>
</ol>
<h2><a href="#how-to-create-a-statefulset-in-kubernetes">#</a>How to Create a StatefulSet in Kubernetes</h2>
<p>In this section, you will learn how to create a Pod for MySQL database using the StatefulSets controller.</p>
<h3><a href="#create-a-secret">#</a>Create a Secret</h3>
<p>To start, you will need to create a Secret for the MySQL application that will store sensitive information, such as usernames and passwords. Here, I am creating a simple Secret. However, in a production environment, using the HashiCorp Vault is recommended. Use the following code to create a Secret for MySQL:</p>
<div><pre><code>apiVersion: v1
kind: Secret
metadata:
 name: mysql-password
type: opaque
stringData:
 MYSQL_ROOT_PASSWORD: password
</code></pre></div><p>Save the code using the file name <code>mysql-secret.yaml</code> and execute the code using the following command on your Kubernetes cluster:</p>
<div><pre><code>kubectl apply -f mysql-secret.yaml
</code></pre></div><p>Get the list of Secrets:</p>
<h3><a href="#create-a-mysql-statefulset-application">#</a>Create a MySQL StatefulSet Application</h3>
<p>Before creating a StatefulSet application, check your volumes by getting the persistent volume list:</p>
<div><pre><code>kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM STATUS
pvc-e0567 	10Gi 	RWO 	Retain Bound
</code></pre></div><p>Next, get the persistent volume claim list:</p>
<div><pre><code>kubectl get pvc
NAME 	 STATUS VOLUME 	 CAPACITY ACCESS
mysql-store-mysql-set-0 Bound 	pvc-e0567d43ffc6405b 10Gi 	RWO
</code></pre></div><p>Last, get the storage class list:</p>
<div><pre><code>kubectl get storageclass
NAME 	PROVISIONER 	RECLAIMPOLICY
linode-block-storage 	linodebs.csi.linode.com 	Delete
linode-block-storage-retain (default) linodebs.csi.linode.com 	Retain
</code></pre></div><p>Then use the following code to create a MySQL StatefulSet application in the Kubernetes cluster:</p>
<div><pre><code>apiVersion: apps/v1
kind: StatefulSet
metadata:
 name: mysql-set
spec:
 selector:
 matchLabels:
 app: mysql
 serviceName: "mysql"
 replicas: 3
 template:
 metadata:
 labels:
 app: mysql
 spec:
 terminationGracePeriodSeconds: 10
 containers:
 - name: mysql
 image: mysql:5.7
 ports:
 - containerPort: 3306
 volumeMounts:
 - name: mysql-store
 mountPath: /var/lib/mysql
 env:
 - name: MYSQL_ROOT_PASSWORD
 valueFrom:
 secretKeyRef:
 name: mysql-password
 key: MYSQL_ROOT_PASSWORD
 volumeClaimTemplates:
 - metadata:
 name: mysql-store
 spec:
 accessModes: ["ReadWriteOnce"]
 storageClassName: "linode-block-storage-retain"
 resources:
 requests:
 storage: 5Gi
</code></pre></div><p>Here are a few things to note:</p>
<ol>
<li>The <code>kind</code> is a StatefulSet. <code>kind</code> tells Kubernetes to create a MySQL application with the stateful feature.</li>
<li>The password is taken from the Secret object using the <code>secretKeyRef</code>.</li>
<li>The Linode block storage was used in the <code>volumeClaimTemplates</code>. If you are not mentioning any storage class name here, then it will take the default storage class in your cluster.</li>
<li>The replication count here is 3 (using the replica parameter), so it will create three Pods named <code>mysql-set-0</code>, <code>mysql-set-1</code>, and <code>mysql-set-2</code>.</li>
</ol>
<p>Next, save the code using the file name <code>mysql.yaml</code> and execute using the following command:</p>
<div><pre><code>kubectl apply -f mysql.yaml
</code></pre></div><p>Now that the MySQL Pods are created, get the Pods list:</p>
<div><pre><code>kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-set-0 1/1 Running 0 142s
mysql-set-1 1/1 Running 0 132s
mysql-set-2 1/1 Running 0 120s
</code></pre></div><h3><a href="#create-a-service-for-the-statefulset-application">#</a>Create a Service for the StatefulSet Application</h3>
<p>Now, create the service for the MySQL Pod. Do not use the load balancer service for a stateful application, but instead, create a headless service for the MySQL application using the following code:</p>
<div><pre><code>apiVersion: v1
kind: Service
metadata:
 name: mysql
 labels:
 app: mysql
spec:
 ports:
 - port: 3306
 clusterIP: None
 selector:
 app: mysql
</code></pre></div><p>Save the code using the file name <code>mysql-service.yaml</code> and execute using the following command:</p>
<div><pre><code>kubectl apply -f mysql-service.yaml
</code></pre></div><p>Get the list of running services:</p>
<h3><a href="#create-a-client-for-mysql">#</a>Create a Client for MySQL</h3>
<p>If you want to access MySQL, then you will need a MySQL client tool. Deploy a MySQL client using the following manifest code:</p>
<div><pre><code>apiVersion: v1
kind: Pod
metadata:
 name: mysql-client
spec:
 containers:
 - name: mysql-container
 image: alpine
 command: ['sh','-c', "sleep 1800m"]
 imagePullPolicy: IfNotPresent
</code></pre></div><p>Save the code using the file name <code>mysql-client.yaml</code> and execute using the following command:</p>
<div><pre><code>kubectl apply -f mysql-client.yaml
</code></pre></div><p>Then enter this into the MySQL client:</p>
<div><pre><code>kubectl exec --stdin --tty mysql-client -- sh
</code></pre></div><p>Finally, install the MySQL client tool:</p>
<h3><a href="#access-the-mysql-application-using-the-mysql-client">#</a>Access the MySQL Application Using the MySQL Client</h3>
<p>Next, access the MySQL application using the MySQL client and create databases on the Pods.</p>
<p>If you are not already in the MySQL client Pod, enter it now:</p>
<div><pre><code>kubectl exec -it mysql-client /bin/sh
</code></pre></div><p>To access MySQL, you can use the same standard MySQL command to connect with the MySQL server:</p>
<div><pre><code>mysql -u root -p -h host-server-name
</code></pre></div><p>For access, you will need a MySQL server name. The syntax of the MySQL server in the Kubernetes cluster is given below:</p>
<div><pre><code>stateful_name-ordinal_number.mysql.default.svc.cluster.local
#Example
mysql-set-0.mysql.default.svc.cluster.local
</code></pre></div><p>Connect with the MySQL primary Pod using the following command. When asked for a password, enter the one you made in the “Create a Secret” section above.</p>
<div><pre><code>mysql -u root -p -h mysql-set-0.mysql.default.svc.cluster.local
</code></pre></div><p>Next, create a database on the MySQL primary, then exit:</p>
<div><pre><code>create database erp;
exit;
</code></pre></div><p>Now connect the other Pods and create the database like above:</p>
<div><pre><code>mysql -u root -p -h mysql-set-1.mysql.default.svc.cluster.local
mysql -u root -p -h mysql-set-2.mysql.default.svc.cluster.local
</code></pre></div><blockquote>
<p>Remember that while Kubernetes helps you set up a stateful application, you will need to set up the data cloning and data sync by yourself. This cannot be done by the StatefulSets.</p>
</blockquote>
<h2><a href="#best-practices">#</a>Best Practices</h2>
<p>If you are planning to deploy stateful applications, such as Oracle, MySQL, Elasticsearch, and MongoDB, then using StatefulSets is a great option. The following points need to be considered while creating stateful applications:</p>
<ol>
<li>Create a separate namespace for databases.</li>
<li>Place all the needed components for stateful applications, such as ConfigMaps, Secrets, and Services, in the particular namespace.</li>
<li>Put your custom scripts in the ConfigMaps.</li>
<li>Use headless service instead of load balancer service while creating Service objects.</li>
<li>Use the HashiCorp Vault for storing your Secrets.</li>
<li>Use the persistent volume storage for storing the data. Then your data won’t be deleted even if the Pod dies or crashes.</li>
</ol>
<p>Deployment objects are the most used controller to create Pods in Kubernetes. You can easily scale these Pods by mentioning replication count in the manifest file. For stateless applications, using Deployment objects is most suitable. For example, assume you are planning to deploy your Node.js application and you want to scale the Node.js application to five replicas. In this case, the Deployment object is well suited.</p>
<p>The diagram below shows how Deployment and StatefulSets assign names to the Pods.</p>
<p></p>
<p>StatefulSets create ordinal service endpoints for each Pod created using the replica option. The diagram below shows how the stateful Pod endpoints are created with ordinal numbering and how they communicate with each other.</p>
<p></p>
<h2><a href="#conclusion">#</a>Conclusion</h2>
<p>In this article, you learned about Kubernetes’s two main controllers for creating Pods: Deployments and StatefulSets. A Deployment object is well suited for stateless applications, and the StatefulSets controller is well suited for stateful applications. If you are planning to deploy stateful applications, such as MySQL and Oracle, then you should use the StatefulSets controller instead of the Deployment object.</p>
<p>The StatefulSets controller offers an ordinal number feature for each Pod starting from zero. This helps stateful applications easily set up a primary-replica architecture, and if a Pod dies, a new Pod is re-created using the same name. This is a very useful feature and does not break the chain of stateful application clusters. If you are scaling down, then it deletes in the reverse order.</p>
<p>Use the StatefulSets controller in the Kubernetes cluster for deploying stateful applications, such as Oracle, MySQL, Elasticsearch, and MongoDB. While cloning and syncing data must still be completed manually, StatefulSets go a long way in easing the complexity involved in deploying stateful applications.</p>
<p><em>Photo by <a href="https://unsplash.com/@photoholgic?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Photoholgic</a> on <a href="https://unsplash.com/s/photos/pebbles?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></em></p>
</div>||||I|||| Kubernetes StatefulSet - Examples & Best Practices

 Bharathiraja Shanmugam
 Nov 23, 2021
 11 min read
 Blog
 Go to loft.sh
 Featured
 By Topic
 Access Control
 Authentication
 CI/CD Pipelines
 Collaboration
 Community
 Comparisons
 Cost Optimization
 Dashboards
 Developer Experience
 Development
 Enterprise
 GitOps
 Kubernetes Insights
 Monitoring
 Multi-Tenancy
 Open Source
 Platform Engineering
 Security
 Self-Service
 Tools
 Tutorials
 Use Cases
 Video
 By Language
 By Product
 Loft
 DevSpace
 vcluster
 jsPolicy
 kiosk
 Search
 Subscribe
 Go to loft.sh
 Table of Contents
 * What Are Stateful Applications?
 * What Are StatefulSets?
 * When to Use StatefulSets
 + Deployment vs. StatefulSets
 * How to Create a StatefulSet in Kubernetes
 + Create a Secret
 + Create a MySQL StatefulSet Application
 + Create a Service for the StatefulSet Application
 + Create a Client for MySQL
 + Access the MySQL Application Using the MySQL Client
 * Best Practices
 * Conclusion

 Are you planning to deploy a database in the Kubernetes cluster? If so, then you’ve come to the right place. Kubernetes is a container orchestration tool that uses many controllers to run applications as containers (Pods). One of these controllers is called StatefulSet, which is used to run stateful applications.

 Deploying stateful applications in the Kubernetes cluster can be a tedious task. This is because the stateful application expects primary-replica architecture and a fixed Pod name. The StatefulSets controller addresses this problem while deploying the stateful application in the Kubernetes cluster.

 In this article, you will learn more about what StatefulSets in the Kubernetes cluster are and when to use them, as well as how to deploy the stateful application using the StatefulSets controller in a step-by-step example that includes all the manifest (YAML) files. Furthermore, the StatefulSets concept will be demonstrated using the MySQL database, which covers how to create a fixed Pod name for each MySQL replication and how to access the replicated Pods using the Services object.

 #What Are Stateful Applications?

 Stateful applications are applications that store data and keep tracking it. All databases, such as MySQL, Oracle, and PostgreSQL, are examples of stateful applications. Stateless applications, on the other hand, do not keep the data. Node.js and Nginx are examples of stateless applications. For each request, the stateless application will receive new data and process it.

 In a modern web application, the stateless application connects with stateful applications to serve the user’s request. A Node.js application is a stateless application that receives new data on each request from the user. This application is then connected with a stateful application, such as a MySQL database, to process the data. MySQL stores data and keeps updating the data based on the user’s request.

 Read on to learn more about StatefulSets in the Kubernetes cluster—what they are, when to use them, how to create them, and what the best practices are.

 #What Are StatefulSets?

 A StatefulSet is the Kubernetes controller used to run the stateful application as containers (Pods) in the Kubernetes cluster. StatefulSets assign a sticky identity—an ordinal number starting from zero—to each Pod instead of assigning random IDs for each replica Pod. A new Pod is created by cloning the previous Pod’s data. If the previous Pod is in the pending state, then the new Pod will not be created. If you delete a Pod, it will delete the Pod in reverse order, not in random order. For example, if you had four replicas and you scaled down to three, it will delete the Pod numbered 3.

 The diagram below shows how the Pod is numbered from zero and how the persistent volume is attached to the Pod in the StatefulSets.

 #When to Use StatefulSets

 There are several reasons to consider using StatefulSets. Here are two examples:

 1. Assume you deployed a MySQL database in the Kubernetes cluster and scaled this to three replicas, and a frontend application wants to access the MySQL cluster to read and write data. The read request will be forwarded to three Pods. However, the write request will only be forwarded to the first (primary) Pod, and the data will be synced with the other Pods. You can achieve this by using StatefulSets.
 2. Deleting or scaling down a StatefulSet will not delete the volumes associated with the stateful application. This gives you your data safety. If you delete the MySQL Pod or if the MySQL Pod restarts, you can have access to the data in the same volume.

 #Deployment vs. StatefulSets

 You can also create Pods (containers) using the Deployment object in the Kubernetes cluster. This allows you to easily replicate Pods and attach a storage volume to the Pods. The same thing can be done by using StatefulSets. What then is the advantage of using StatefulSets?

 Well, the Pods created using the Deployment object are assigned random IDs. For example, you are creating a Pod named “my-app”, and you are scaling it to three replicas. The names of the Pods are created like this:

 my-app-123ab
 my-app-098bd
 my-app-890yt
 

 After the name “my-app”, random IDs are added. If the Pod restarts or you scale it down, then again, the Kubernetes Deployment object will assign different random IDs for each Pod. After restarting, the names of all Pods appear like this:

 my-app-jk879
 my-app-kl097
 my-app-76hf7
 

 All these Pods are associated with one load balancer service. So in a stateless application, changes in the Pod name are easily identified, and the service object easily handles the random IDs of Pods and distributes the load. This type of deployment is very suitable for stateless applications.

 However, stateful applications cannot be deployed like this. The stateful application needs a sticky identity for each Pod because replica Pods are not identical Pods.

 Take a look at the MySQL database deployment. Assume you are creating Pods for the MySQL database using the Kubernetes Deployment object and scaling the Pods. If you are writing data on one MySQL Pod, do not replicate the same data on another MySQL Pod if the Pod is restarted. This is the first problem with the Kubernetes Deployment object for the stateful application.

 Stateful applications always need a sticky identity. While the Kubernetes Deployment object offers random IDs for each Pod, the Kubernetes StatefulSets controller offers an ordinal number for each Pod starting from zero, such as mysql-0, mysql-1, mysql-2, and so forth.

 For stateful applications with a StatefulSet controller, it is possible to set the first Pod as primary and other Pods as replicas—the first Pod will handle both read and write requests from the user, and other Pods always sync with the first Pod for data replication. If the Pod dies, a new Pod is created with the same name.

 The diagram below shows a MySQL primary and replica architecture with persistent volume and data replication architecture.

 Now, add another Pod to that. The fourth Pod will only be created if the third Pod is up and running, and it will clone the data from the previous Pod.

 In summary, StatefulSets provide the following advantages when compared to Deployment objects:

 1. Ordered numbers for each Pod
 2. The first Pod can be a primary, which makes it a good choice when creating a replicated database setup, which handles both reading and writing
 3. Other Pods act as replicas
 4. New Pods will only be created if the previous Pod is in running state and will clone the previous Pod’s data
 5. Deletion of Pods occurs in reverse order

 #How to Create a StatefulSet in Kubernetes

 In this section, you will learn how to create a Pod for MySQL database using the StatefulSets controller.

 #Create a Secret

 To start, you will need to create a Secret for the MySQL application that will store sensitive information, such as usernames and passwords. Here, I am creating a simple Secret. However, in a production environment, using the HashiCorp Vault is recommended. Use the following code to create a Secret for MySQL:

 apiVersion: v1
 kind: Secret
 metadata:
 name: mysql-password
 type: opaque
 stringData:
 MYSQL_ROOT_PASSWORD: password
 

 Save the code using the file name mysql-secret.yaml and execute the code using the following command on your Kubernetes cluster:

 kubectl apply -f mysql-secret.yaml
 

 Get the list of Secrets:

 kubectl get secrets
 

 #Create a MySQL StatefulSet Application

 Before creating a StatefulSet application, check your volumes by getting the persistent volume list:

 kubectl get pv
 NAME CAPACITY ACCESS MODES RECLAIM STATUS
 pvc-e0567 		10Gi 	RWO 	Retain Bound
 

 Next, get the persistent volume claim list:

 kubectl get pvc
 
 NAME 	 STATUS VOLUME 	 CAPACITY ACCESS
 mysql-store-mysql-set-0 Bound 	pvc-e0567d43ffc6405b 10Gi 	RWO
 

 Last, get the storage class list:

 kubectl get storageclass
 
 NAME 			PROVISIONER 	RECLAIMPOLICY
 linode-block-storage 		linodebs.csi.linode.com 	Delete
 linode-block-storage-retain (default) linodebs.csi.linode.com 	Retain
 

 Then use the following code to create a MySQL StatefulSet application in the Kubernetes cluster:

 apiVersion: apps/v1
 kind: StatefulSet
 metadata:
 name: mysql-set
 spec:
 selector:
 matchLabels:
 app: mysql
 serviceName: "mysql"
 replicas: 3
 template:
 metadata:
 labels:
 app: mysql
 spec:
 terminationGracePeriodSeconds: 10
 containers:
 - name: mysql
 image: mysql:5.7
 ports:
 - containerPort: 3306
 volumeMounts:
 - name: mysql-store
 mountPath: /var/lib/mysql
 env:
 - name: MYSQL_ROOT_PASSWORD
 valueFrom:
 secretKeyRef:
 name: mysql-password
 key: MYSQL_ROOT_PASSWORD
 volumeClaimTemplates:
 - metadata:
 name: mysql-store
 spec:
 accessModes: ["ReadWriteOnce"]
 storageClassName: "linode-block-storage-retain"
 resources:
 requests:
 storage: 5Gi
 

 Here are a few things to note:

 1. The kind is a StatefulSet. kind tells Kubernetes to create a MySQL application with the stateful feature.
 2. The password is taken from the Secret object using the secretKeyRef.
 3. The Linode block storage was used in the volumeClaimTemplates. If you are not mentioning any storage class name here, then it will take the default storage class in your cluster.
 4. The replication count here is 3 (using the replica parameter), so it will create three Pods named mysql-set-0, mysql-set-1, and mysql-set-2.

 Next, save the code using the file name mysql.yaml and execute using the following command:

 kubectl apply -f mysql.yaml
 

 Now that the MySQL Pods are created, get the Pods list:

 kubectl get pods
 
 NAME READY STATUS RESTARTS AGE
 mysql-set-0 1/1 Running 0 142s
 mysql-set-1 1/1 Running 0 132s
 mysql-set-2 1/1 Running 0 120s
 

 #Create a Service for the StatefulSet Application

 Now, create the service for the MySQL Pod. Do not use the load balancer service for a stateful application, but instead, create a headless service for the MySQL application using the following code:

 apiVersion: v1
 kind: Service
 metadata:
 name: mysql
 labels:
 app: mysql
 spec:
 ports:
 - port: 3306
 clusterIP: None
 selector:
 app: mysql
 

 Save the code using the file name mysql-service.yaml and execute using the following command:

 kubectl apply -f mysql-service.yaml
 

 Get the list of running services:

 kubectl get svc
 

 #Create a Client for MySQL

 If you want to access MySQL, then you will need a MySQL client tool. Deploy a MySQL client using the following manifest code:

 apiVersion: v1
 kind: Pod
 metadata:
 name: mysql-client
 spec:
 containers:
 - name: mysql-container
 image: alpine
 command: ['sh','-c', "sleep 1800m"]
 imagePullPolicy: IfNotPresent
 

 Save the code using the file name mysql-client.yaml and execute using the following command:

 kubectl apply -f mysql-client.yaml
 

 Then enter this into the MySQL client:

 kubectl exec --stdin --tty mysql-client -- sh
 

 Finally, install the MySQL client tool:

 apk add mysql-client
 

 #Access the MySQL Application Using the MySQL Client

 Next, access the MySQL application using the MySQL client and create databases on the Pods.

 If you are not already in the MySQL client Pod, enter it now:

 kubectl exec -it mysql-client /bin/sh
 

 To access MySQL, you can use the same standard MySQL command to connect with the MySQL server:

 mysql -u root -p -h host-server-name
 

 For access, you will need a MySQL server name. The syntax of the MySQL server in the Kubernetes cluster is given below:

 stateful_name-ordinal_number.mysql.default.svc.cluster.local
 
 #Example
 mysql-set-0.mysql.default.svc.cluster.local
 

 Connect with the MySQL primary Pod using the following command. When asked for a password, enter the one you made in the “Create a Secret” section above.

 mysql -u root -p -h mysql-set-0.mysql.default.svc.cluster.local
 

 Next, create a database on the MySQL primary, then exit:

 create database erp;
 exit;
 

 Now connect the other Pods and create the database like above:

 mysql -u root -p -h mysql-set-1.mysql.default.svc.cluster.local
 
 mysql -u root -p -h mysql-set-2.mysql.default.svc.cluster.local
 

 Remember that while Kubernetes helps you set up a stateful application, you will need to set up the data cloning and data sync by yourself. This cannot be done by the StatefulSets.

 #Best Practices

 If you are planning to deploy stateful applications, such as Oracle, MySQL, Elasticsearch, and MongoDB, then using StatefulSets is a great option. The following points need to be considered while creating stateful applications:

 1. Create a separate namespace for databases.
 2. Place all the needed components for stateful applications, such as ConfigMaps, Secrets, and Services, in the particular namespace.
 3. Put your custom scripts in the ConfigMaps.
 4. Use headless service instead of load balancer service while creating Service objects.
 5. Use the HashiCorp Vault for storing your Secrets.
 6. Use the persistent volume storage for storing the data. Then your data won’t be deleted even if the Pod dies or crashes.

 Deployment objects are the most used controller to create Pods in Kubernetes. You can easily scale these Pods by mentioning replication count in the manifest file. For stateless applications, using Deployment objects is most suitable. For example, assume you are planning to deploy your Node.js application and you want to scale the Node.js application to five replicas. In this case, the Deployment object is well suited.

 The diagram below shows how Deployment and StatefulSets assign names to the Pods.

 StatefulSets create ordinal service endpoints for each Pod created using the replica option. The diagram below shows how the stateful Pod endpoints are created with ordinal numbering and how they communicate with each other.

 #Conclusion

 In this article, you learned about Kubernetes’s two main controllers for creating Pods: Deployments and StatefulSets. A Deployment object is well suited for stateless applications, and the StatefulSets controller is well suited for stateful applications. If you are planning to deploy stateful applications, such as MySQL and Oracle, then you should use the StatefulSets controller instead of the Deployment object.

 The StatefulSets controller offers an ordinal number feature for each Pod starting from zero. This helps stateful applications easily set up a primary-replica architecture, and if a Pod dies, a new Pod is re-created using the same name. This is a very useful feature and does not break the chain of stateful application clusters. If you are scaling down, then it deletes in the reverse order.

 Use the StatefulSets controller in the Kubernetes cluster for deploying stateful applications, such as Oracle, MySQL, Elasticsearch, and MongoDB. While cloning and syncing data must still be completed manually, StatefulSets go a long way in easing the complexity involved in deploying stateful applications.

 Photo by Photoholgic on Unsplash

 Guides
 Kubernetes Insights
 Tools
 Tutorials
 Platform Engineering

 Related blog posts

 View all posts
 Nov 8, 2021
 •
 8 Minute Read

 Kubernetes NGINX Ingress: 10 Useful Configuration Options

 KubeCon + CloudNativeCon North America 2022 was held October 26-28 in Detroit, Michigan. It was the second in-person KubeCon NA since
 Nov 3, 2021
 •
 12 Minute Read

 Kubernetes Persistent Volumes: Examples & Best Practices

 KubeCon + CloudNativeCon North America 2022 was held October 26-28 in Detroit, Michigan. It was the second in-person KubeCon NA since
 Nov 1, 2021
 •
 7 Minute Read

 Harbor + Kubernetes = Self-Hosted Container Registry

 KubeCon + CloudNativeCon North America 2022 was held October 26-28 in Detroit, Michigan. It was the second in-person KubeCon NA since

 Sign up for our newsletter

 Be the first to know about new features, announcements and industry insights.
 eu1
 26230392
 15314855-8a4c-4751-ad29-605050b96984
 loft-sh
 false
 Join our newsletter to stay up to date.
 eu1
 26230392
 15314855-8a4c-4751-ad29-605050b96984
 loft-sh
 true
 By subscribing you agree to with our Privacy Policy and provide consent to receive updates from our company.
 Resources
 Customer StoriesSlack CommunityBlog
 Product
 Getting StartedDocumentationPricing
 Company
 About UsCareers - We're hiring!Press & Media
 Terms of ServicePrivacy PolicyCookie Policy
 © Loft Labs, Inc. All right reserved.
Are you planning to deploy a database in the Kubernetes cluster? If so, then you’ve come to the right place. Kubernetes is a container orchestration tool that uses many controllers to run applications as containers (Pods). One of these controllers is called  StatefulSet , which is used to run stateful applications.
Deploying stateful applications in the Kubernetes cluster can be a tedious task. This is because the stateful application expects primary-replica architecture and a fixed Pod name. The StatefulSets controller addresses this problem while deploying the stateful application in the Kubernetes cluster.
In this article, you will learn more about what StatefulSets in the Kubernetes cluster are and when to use them, as well as how to deploy the stateful application using the StatefulSets controller in a step-by-step example that includes all the manifest (YAML) files. Furthermore, the StatefulSets concept will be demonstrated using the MySQL database, which covers how to create a fixed Pod name for each MySQL replication and how to access the replicated Pods using the Services object.
 #What Are Stateful Applications?
Stateful applications are applications that store data and keep tracking it. All databases, such as MySQL, Oracle, and PostgreSQL, are examples of stateful applications. Stateless applications, on the other hand, do not keep the data. Node.js and Nginx are examples of stateless applications. For each request, the stateless application will receive new data and process it.
In a modern web application, the stateless application connects with stateful applications to serve the user’s request. A Node.js application is a stateless application that receives new data on each request from the user. This application is then connected with a stateful application, such as a MySQL database, to process the data. MySQL stores data and keeps updating the data based on the user’s request.
 
Read on to learn more about StatefulSets in the Kubernetes cluster—what they are, when to use them, how to create them, and what the best practices are.
 #What Are StatefulSets?
A StatefulSet is the Kubernetes controller used to run the stateful application as containers (Pods) in the Kubernetes cluster. StatefulSets assign a sticky identity—an ordinal number starting from zero—to each Pod instead of assigning random IDs for each replica Pod. A new Pod is created by cloning the previous Pod’s data. If the previous Pod is in the pending state, then the new Pod will not be created. If you delete a Pod, it will delete the Pod in reverse order, not in random order. For example, if you had four replicas and you scaled down to three, it will delete the Pod numbered 3.
The diagram below shows how the Pod is numbered from zero and how the persistent volume is attached to the Pod in the StatefulSets.
 
 #When to Use StatefulSets
There are several reasons to consider using StatefulSets. Here are two examples:
Assume you deployed a MySQL database in the Kubernetes cluster and scaled this to three replicas, and a frontend application wants to access the MySQL cluster to read and write data. The read request will be forwarded to three Pods. However, the write request will only be forwarded to the first (primary) Pod, and the data will be synced with the other Pods. You can achieve this by using StatefulSets.
Deleting or scaling down a StatefulSet will not delete the volumes associated with the stateful application. This gives you your data safety. If you delete the MySQL Pod or if the MySQL Pod restarts, you can have access to the data in the same volume.
 #Deployment vs. StatefulSets
You can also create Pods (containers) using the Deployment object in the Kubernetes cluster. This allows you to easily replicate Pods and attach a storage volume to the Pods. The same thing can be done by using StatefulSets. What then is the advantage of using StatefulSets?
Well, the Pods created using the Deployment object are assigned random IDs. For example, you are creating a Pod named “my-app”, and you are scaling it to three replicas. The names of the Pods are created like this:
  my-app-123ab
my-app-098bd
my-app-890yt
   After the name “my-app”, random IDs are added. If the Pod restarts or you scale it down, then again, the Kubernetes Deployment object will assign different random IDs for each Pod. After restarting, the names of all Pods appear like this:
  my-app-jk879
my-app-kl097
my-app-76hf7
   All these Pods are associated with one load balancer service. So in a stateless application, changes in the Pod name are easily identified, and the service object easily handles the random IDs of Pods and distributes the load. This type of deployment is very suitable for stateless applications.
 
However, stateful applications cannot be deployed like this. The stateful application needs a sticky identity for each Pod because replica Pods are not identical Pods.
Take a look at the MySQL database deployment. Assume you are creating Pods for the MySQL database using the Kubernetes Deployment object and scaling the Pods. If you are writing data on one MySQL Pod, do not replicate the same data on another MySQL Pod if the Pod is restarted. This is the first problem with the Kubernetes Deployment object for the stateful application.
 
Stateful applications always need a sticky identity. While the Kubernetes Deployment object offers random IDs for each Pod, the Kubernetes StatefulSets controller offers an ordinal number for each Pod starting from zero, such as mysql-0, mysql-1, mysql-2, and so forth.
 
For stateful applications with a StatefulSet controller, it is possible to set the first Pod as primary and other Pods as replicas—the first Pod will handle both read and write requests from the user, and other Pods always sync with the first Pod for data replication. If the Pod dies, a new Pod is created with the same name.
The diagram below shows a MySQL primary and replica architecture with persistent volume and data replication architecture.
 
Now, add another Pod to that. The fourth Pod will only be created if the third Pod is up and running, and it will clone the data from the previous Pod.
 
In summary, StatefulSets provide the following advantages when compared to Deployment objects:
Ordered numbers for each Pod
The first Pod can be a primary, which makes it a good choice when creating a replicated database setup, which handles both reading and writing
Other Pods act as replicas
New Pods will only be created if the previous Pod is in running state and will clone the previous Pod’s data
Deletion of Pods occurs in reverse order
 #How to Create a StatefulSet in Kubernetes
In this section, you will learn how to create a Pod for MySQL database using the StatefulSets controller.
 #Create a Secret
To start, you will need to create a Secret for the MySQL application that will store sensitive information, such as usernames and passwords. Here, I am creating a simple Secret. However, in a production environment, using the HashiCorp Vault is recommended. Use the following code to create a Secret for MySQL:
  apiVersion: v1
kind: Secret
metadata:
 name: mysql-password
type: opaque
stringData:
 MYSQL_ROOT_PASSWORD: password
   Save the code using the file name mysql-secret.yaml and execute the code using the following command on your Kubernetes cluster:
  kubectl apply -f mysql-secret.yaml
   Get the list of Secrets:
 #Create a MySQL StatefulSet Application
Before creating a StatefulSet application, check your volumes by getting the persistent volume list:
  kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM STATUS
pvc-e0567 	10Gi 	RWO 	Retain Bound
   Next, get the persistent volume claim list:
  kubectl get pvc
NAME 	 STATUS VOLUME 	 CAPACITY ACCESS
mysql-store-mysql-set-0 Bound 	pvc-e0567d43ffc6405b 10Gi 	RWO
   Last, get the storage class list:
  kubectl get storageclass
NAME 	PROVISIONER 	RECLAIMPOLICY
linode-block-storage 	linodebs.csi.linode.com 	Delete
linode-block-storage-retain (default) linodebs.csi.linode.com 	Retain
   Then use the following code to create a MySQL StatefulSet application in the Kubernetes cluster:
  apiVersion: apps/v1
kind: StatefulSet
metadata:
 name: mysql-set
spec:
 selector:
 matchLabels:
 app: mysql
 serviceName: "mysql"
 replicas: 3
 template:
 metadata:
 labels:
 app: mysql
 spec:
 terminationGracePeriodSeconds: 10
 containers:
 - name: mysql
 image: mysql:5.7
 ports:
 - containerPort: 3306
 volumeMounts:
 - name: mysql-store
 mountPath: /var/lib/mysql
 env:
 - name: MYSQL_ROOT_PASSWORD
 valueFrom:
 secretKeyRef:
 name: mysql-password
 key: MYSQL_ROOT_PASSWORD
 volumeClaimTemplates:
 - metadata:
 name: mysql-store
 spec:
 accessModes: ["ReadWriteOnce"]
 storageClassName: "linode-block-storage-retain"
 resources:
 requests:
 storage: 5Gi
   Here are a few things to note:
The kind is a StatefulSet. kind tells Kubernetes to create a MySQL application with the stateful feature.
The password is taken from the Secret object using the secretKeyRef.
The Linode block storage was used in the volumeClaimTemplates. If you are not mentioning any storage class name here, then it will take the default storage class in your cluster.
The replication count here is 3 (using the replica parameter), so it will create three Pods named mysql-set-0, mysql-set-1, and mysql-set-2.
Next, save the code using the file name mysql.yaml and execute using the following command:
  kubectl apply -f mysql.yaml
   Now that the MySQL Pods are created, get the Pods list:
  kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-set-0 1/1 Running 0 142s
mysql-set-1 1/1 Running 0 132s
mysql-set-2 1/1 Running 0 120s
    #Create a Service for the StatefulSet Application
Now, create the service for the MySQL Pod. Do not use the load balancer service for a stateful application, but instead, create a headless service for the MySQL application using the following code:
  apiVersion: v1
kind: Service
metadata:
 name: mysql
 labels:
 app: mysql
spec:
 ports:
 - port: 3306
 clusterIP: None
 selector:
 app: mysql
   Save the code using the file name mysql-service.yaml and execute using the following command:
  kubectl apply -f mysql-service.yaml
   Get the list of running services:
 #Create a Client for MySQL
If you want to access MySQL, then you will need a MySQL client tool. Deploy a MySQL client using the following manifest code:
  apiVersion: v1
kind: Pod
metadata:
 name: mysql-client
spec:
 containers:
 - name: mysql-container
 image: alpine
 command: ['sh','-c', "sleep 1800m"]
 imagePullPolicy: IfNotPresent
   Save the code using the file name mysql-client.yaml and execute using the following command:
  kubectl apply -f mysql-client.yaml
   Then enter this into the MySQL client:
  kubectl exec --stdin --tty mysql-client -- sh
   Finally, install the MySQL client tool:
 #Access the MySQL Application Using the MySQL Client
Next, access the MySQL application using the MySQL client and create databases on the Pods.
If you are not already in the MySQL client Pod, enter it now:
  kubectl exec -it mysql-client /bin/sh
   To access MySQL, you can use the same standard MySQL command to connect with the MySQL server:
  mysql -u root -p -h host-server-name
   For access, you will need a MySQL server name. The syntax of the MySQL server in the Kubernetes cluster is given below:
  stateful_name-ordinal_number.mysql.default.svc.cluster.local
#Example
mysql-set-0.mysql.default.svc.cluster.local
   Connect with the MySQL primary Pod using the following command. When asked for a password, enter the one you made in the “Create a Secret” section above.
  mysql -u root -p -h mysql-set-0.mysql.default.svc.cluster.local
   Next, create a database on the MySQL primary, then exit:
  create database erp;
exit;
   Now connect the other Pods and create the database like above:
  mysql -u root -p -h mysql-set-1.mysql.default.svc.cluster.local
mysql -u root -p -h mysql-set-2.mysql.default.svc.cluster.local
   
Remember that while Kubernetes helps you set up a stateful application, you will need to set up the data cloning and data sync by yourself. This cannot be done by the StatefulSets.
 #Best Practices
If you are planning to deploy stateful applications, such as Oracle, MySQL, Elasticsearch, and MongoDB, then using StatefulSets is a great option. The following points need to be considered while creating stateful applications:
Create a separate namespace for databases.
Place all the needed components for stateful applications, such as ConfigMaps, Secrets, and Services, in the particular namespace.
Put your custom scripts in the ConfigMaps.
Use headless service instead of load balancer service while creating Service objects.
Use the HashiCorp Vault for storing your Secrets.
Use the persistent volume storage for storing the data. Then your data won’t be deleted even if the Pod dies or crashes.
Deployment objects are the most used controller to create Pods in Kubernetes. You can easily scale these Pods by mentioning replication count in the manifest file. For stateless applications, using Deployment objects is most suitable. For example, assume you are planning to deploy your Node.js application and you want to scale the Node.js application to five replicas. In this case, the Deployment object is well suited.
The diagram below shows how Deployment and StatefulSets assign names to the Pods.
 
StatefulSets create ordinal service endpoints for each Pod created using the replica option. The diagram below shows how the stateful Pod endpoints are created with ordinal numbering and how they communicate with each other.
 
 #Conclusion
In this article, you learned about Kubernetes’s two main controllers for creating Pods: Deployments and StatefulSets. A Deployment object is well suited for stateless applications, and the StatefulSets controller is well suited for stateful applications. If you are planning to deploy stateful applications, such as MySQL and Oracle, then you should use the StatefulSets controller instead of the Deployment object.
The StatefulSets controller offers an ordinal number feature for each Pod starting from zero. This helps stateful applications easily set up a primary-replica architecture, and if a Pod dies, a new Pod is re-created using the same name. This is a very useful feature and does not break the chain of stateful application clusters. If you are scaling down, then it deletes in the reverse order.
Use the StatefulSets controller in the Kubernetes cluster for deploying stateful applications, such as Oracle, MySQL, Elasticsearch, and MongoDB. While cloning and syncing data must still be completed manually, StatefulSets go a long way in easing the complexity involved in deploying stateful applications.
 Photo by Photoholgic on Unsplash
Kubernetes StatefulSet - Examples & Best Practices
https://loft.sh/blog/kubernetes-statefulset-examples-and-best-practices/