<p>I'm using rook-ceph in kubernetes.
I deployed rook-ceph-operator and rook-ceph-cluster Helm charts.</p>
<p>I have 5 worker nodes with 2 OSD each one. Each OSD has 100GB. So in total, I have 5*200GB = 1TB raw storage. In CephBlockPool I set <code>replicated=3 failureDomain=host</code>.</p>
<p>I created a POD with a PVC which ask for 500Gi and it works.
Then I can create more PODs with PVC &gt; 500Gi and it works also.
Basically I can create hundreds PVC and I have hundreds of PV Bounded, but the TOTAL space is more than 1TB, which is the maximum.</p>
<p>What I expected? If I create PVs, and they exceeds 1TB, they SHOULD NOT be created. I don't want to use thin provisioning.</p>
<p>Last thing, my REAL storage should be 1TB/3 because I have a replica of 3.</p>
<p>I tried using <code>ResourceQuotas</code> with <code>requests.storage</code> and it works but the problem is that it's applied per namespace. And I have a lot of namespaces. I need to limit the PCV requests to all the cluster.</p>
<p>My question is: How can I limit the storage requests in k8s to avoid request more than the maximum storage capacity for ALL the cluster?</p>

mrk
<blockquote>
<p>they SHOULD NOT be created</p>
</blockquote>
<p>Says who? You did not consume all that space yet, did you?</p>
<p>With Ceph, you can allocate more storage space than you actually have.</p>
<p>Reaching your nearfull-ratio (~80 capacity), you should start seeing warnings. Reaching full-ratio (~90%), you won't be able to write anymore. Up to you, to monitor your cluster capacity, add OSDs or purge volumes when needed.</p>
<p>To my knowledge, there is no thick provisioning with RBD.</p>
<p>You could use ResourceQuotas limiting the allocatable storage space per Namespace. On OpenShift, they also have ClusterResourceQuotas, which could ensure no Namespace escapes that quota.</p>
