<p>What is the difference between used and hard quota for memory in kubernetes? How to write correct values in quota of limits.memory &amp; requests.memory? i dont understand it &amp; i didn't find any explanation of this</p>

Epic555
<h2>Background</h2>
<p><a href="https://kubernetes.io/docs/concepts/policy/resource-quotas/" rel="nofollow noreferrer">Resource Quota</a> is limit of specific resource which is namespace wide.</p>
<blockquote>
<p>A resource quota, defined by a ResourceQuota object, provides constraints that limit aggregate resource consumption per namespace.</p>
</blockquote>
<p>As I mention in comments it's very similar to <code>Requests</code> and <code>Limits</code> which are used in <code>Pods</code>. <code>Request</code> is value which is requested by pod on the beginning and <code>limit</code> is maximum value which pod can use.
In <code>Quota</code>, <code>hard</code> value is like <code>limit</code>, it cannot have more than this value. <code>Used</code> is value of current use of this particular resource. Quota can be used to limit number of <code>jobs</code>, <code>pods</code>, <code>services</code>, <code>cpu</code> or <code>memory</code>.</p>
<h2>Tests</h2>
<p>To list all your <code>quotas</code>, you can use command <code>$ kubectl get quota -A</code>. If you miss <code>-A</code> or <code>--all-namespaces</code> you will get only quota from <code>default</code> namespace.</p>
<p>I've tested this on <code>GKE</code> cluster and <code>Minikube</code>.</p>
<p>As default, <code>GKE</code> cluster have a few <code>Quotas</code>. You can have more than 1 <code>Quota</code> in <code>namespace</code>. In below example you have 2 <code>quotas</code> for <code>kube-system</code> namespace.</p>
<pre><code>$ kubectl get quota -A
NAMESPACE         NAME                  AGE   REQUEST                      LIMIT
default           gke-resource-quotas   11m   count/ingresses.extensions: 0/100, count/ingresses.networking.k8s.io: 0/100, count/jobs.batch: 0/5k, pods: 0/1500, services: 1/500
kube-node-lease   gke-resource-quotas   11m   count/ingresses.extensions: 0/100, count/ingresses.networking.k8s.io: 0/100, count/jobs.batch: 0/5k, pods: 0/1500, services: 0/500
kube-public       gke-resource-quotas   11m   count/ingresses.extensions: 0/100, count/ingresses.networking.k8s.io: 0/100, count/jobs.batch: 0/5k, pods: 0/1500, services: 0/500
kube-system       gcp-critical-pods     18m   pods: 13/1G 
kube-system       gke-resource-quotas   11m   count/ingresses.extensions: 0/100, count/ingresses.networking.k8s.io: 0/100, count/jobs.batch: 0/5k, pods: 15/1500, services: 3/500
</code></pre>
<p>But <code>Minikube</code> don't have any at the beginning. To enable <code>Resource Quota</code>, please check <a href="https://kubernetes.io/docs/concepts/policy/resource-quotas/#enabling-resource-quota" rel="nofollow noreferrer">this docs</a></p>
<p>You can describe <code>Quota</code> to check which resources are going to be limited.</p>
<pre><code>$ kubectl describe quota gke-resource-quotas -n kube-system
Name:                              gke-resource-quotas
Namespace:                         kube-system
Resource                           Used  Hard
--------                           ----  ----
count/ingresses.extensions         0     100
count/ingresses.networking.k8s.io  0     100
count/jobs.batch                   0     5k
pods                               15    1500
services                           3     500
</code></pre>
<p>Based on the above output:</p>
<p>In my current <code>GKE</code> cluster I have 15 pods running (<code>Used</code>) but maximum number of pods I can have in this namespace is <code>1500</code>. I won't be able to deploy in this <code>namespace</code> 1501 pods.</p>
<p><strong>Create namespace</strong></p>
<pre><code>$ kubectl create ns test
</code></pre>
<p><strong>Creating Quota</strong></p>
<pre><code>apiVersion: v1
kind: ResourceQuota
metadata:
  name: test
  namespace: test
spec:
  hard:
    requests.memory: &quot;100Mi&quot;  
    limits.memory: &quot;200Mi&quot;

resourcequota/test created

$ kubectl describe quota -n test
Name:            test
Namespace:       test
Resource         Used  Hard
--------         ----  ----
limits.memory    0     200Mi
requests.memory  0     100Mi
</code></pre>
<p>In above example I don't have any pods whih are using memory.</p>
<p><strong>Scenario</strong></p>
<p>2 pods with <code>50Mi requests</code> and <code>100Mi</code> Limit each.</p>
<p><em><strong>pod.yaml</strong></em></p>
<pre><code>apiVersion: v1
kind: Pod
metadata:
  name: nginx-1
  namespace: test
spec:
  containers:
  - name: nginx
    image: nginx
    resources:
      requests:
        memory: &quot;50Mi&quot;
      limits:
        memory: &quot;100Mi&quot;
</code></pre>
<p>After that, <code>Used</code> value of <code>Quota</code> has been chagnged.</p>
<pre><code>$ kubectl describe quota -n test
Name:            test
Namespace:       test
Resource         Used   Hard
--------         ----   ----
limits.memory    100Mi  200Mi
requests.memory  50Mi   100Mi
</code></pre>
<p>If I would create second pod with the same values but with name <code>nginx-2</code>, Kubernetes allows me to do it as <code>Hard</code> values won't be exceeded. <code>Used</code> and <code>Hard</code> values will be the same</p>
<pre><code>$ kubectl describe quota -n test
Name:            test
Namespace:       test
Resource         Used   Hard
--------         ----   ----
limits.memory    200Mi  200Mi
requests.memory  100Mi  100Mi
</code></pre>
<p>However, I won't be able to create <code>nginx-3</code> as it would exceeded <code>Hard</code> value. You will receive error:</p>
<pre><code>Error from server (Forbidden): error when creating &quot;pod.yaml&quot;: pods &quot;nginx-3&quot; is forbidden: exceeded quota: test, requested: limits.memory=100Mi,requests.memory=50Mi, used: limits.memory=200Mi,requests.memory=100Mi, limited: limits.memory=200Mi,requests.memory=100Mi
</code></pre>
<p>In above example <code>resourcequota/test</code> set limitations on <code>test</code> namespace.</p>
<ul>
<li><code>limits.memory</code> is referring to pod's <code>spec.containers.resources.limits.memory</code>.</li>
<li><code>requests.memory</code> is referring to pod's <code>spec.containers.resources.requests.memory</code>.</li>
</ul>
<p>Created <code>quota</code> won't allow to create pods if <code>limits</code> and <code>requests</code> will exceed <code>Hard</code> value in the <code>quota</code>.</p>
<p>In <code>$ kubectl describe quota test -n test</code>, <code>Usage</code> value is sum of all pod's limits in this <code>namespace</code> (currently in use). <code>Hard</code> is maximum value which cannot be exceeded.</p>
<h2>Conclusion</h2>
<p><code>Used</code> value of <code>Quota</code> is number of resources which are already using in this <code>namespace</code>. It can be number of <code>pods</code> or amount of <code>memory</code> or <code>cpu</code>.
<code>Hard</code> value of <code>Quota</code> is maximum number of <code>pods</code> or maximum threshold for <code>memory</code> to use. If it will exceed <code>Hard</code> value, particular resource won't be created.</p>
<h2>Useful links</h2>
<ul>
<li><a href="https://opensource.com/article/20/12/kubernetes-resource-quotas" rel="nofollow noreferrer">How to use Kubernetes resource quotas</a></li>
<li><a href="https://kubernetes.io/docs/concepts/policy/limit-range/" rel="nofollow noreferrer">Limit Ranges</a></li>
</ul>
<p>Please let me know if you still have some questions.</p>
