<p>I have a Kubernetes cluster with three control and three worker nodes.
To install Ceph I'm using rook. After the installation, only the worker nodes are included in the Ceph cluster - but not the controller nodes.</p>
<p>The configuration snippet:</p>
<pre><code>storage:                                                                                                                                                                       
  useAllNodes: true
  useAllDevices: true
</code></pre>
<p>Is there an easy way to also include the controller nodes in the Ceph cluster?
Or: do I need to explicitly list all controller and worker nodes in the storage nodes list?</p>

<p>The root cause for the problem was, that the master nodes were tainted in a way that no 'normal' pod was allowed to run there.</p>
<p>Adapting the <code>cluster.yaml</code> configuration of rook and adding the following lines which explicitly allowed the ceph pods run also on master nodes solved the problem:</p>
<pre><code>placement:
  all:
    tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
</code></pre>
<p>Thanks a lot to Hackaholic who pointed me in this direction.</p>
<p><strong>Update 2024-03:</strong> The above solution stopped working for me using kubernetes 1.26.3 and kubespray 1.10.13.  IMHO one of the reasons is, that <a href="https://kubernetes.io/docs/reference/labels-annotations-taints/#node-role-kubernetes-io-master-taint" rel="nofollow noreferrer">the <code>master</code> label is deprecated</a>.  Using the deprecated master and new control-plane label works for me - specifying only one does not. So the current solution looks like:</p>
<pre><code>placement:                                                                                                                                       
  all:                                                                                                                                           
    tolerations:                                                                                                                                 
      - effect: NoSchedule                                                                                                                       
        key: node-role.kubernetes.io/control-plane                                                                                               
      - effect: NoSchedule                                                                                                                       
        key: node-role.kubernetes.io/master
</code></pre>
