<p>I am trying to limit the resource usage of each container in a pod dynamically. For docker, I used --cgroup-parent to put containers in a specific cgroup directory. However, in Kubernetes, I haven't found any option that I can do this.</p>

<p>You can use the <a href="https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/#using-the-cgroupfs-driver" rel="nofollow noreferrer">cgroupfs</a> driver supported by Kubernetes.</p>
<p>But perhaps you just want to <a href="https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/" rel="nofollow noreferrer">set the <code>Pod</code> and <code>Container</code> resources</a> alone? Kubernetes allows setting the limits for specific containers <a href="https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memory" rel="nofollow noreferrer">as well</a>, therefore you can have app with <code>500Mi</code> RAM and logs with <code>50Mi</code> RAM (<code>requests</code>), but expand both to <code>1Gi</code> if necessary (<code>limits</code>):</p>
<pre class="lang-yaml prettyprint-override"><code>spec:
  containers:
  - name: app
    image: images.my-company.example/app:v4
    resources:
      requests:
        memory: &quot;64Mi&quot;
        cpu: &quot;250m&quot;
      limits:
        memory: &quot;128Mi&quot;
        cpu: &quot;500m&quot;
  - name: log-aggregator
    image: images.my-company.example/log-aggregator:v6
    resources:
      requests:
        memory: &quot;64Mi&quot;
        cpu: &quot;250m&quot;
      limits:
        memory: &quot;128Mi&quot;
        cpu: &quot;500m&quot;
</code></pre>
<hr />
<p>Update:</p>
<p>As mentioned by <a href="https://stackoverflow.com/users/10008173">David Maze</a>, you can also utilize autoscalers such as <a href="https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler" rel="nofollow noreferrer">Vertical Pod Autoscaler</a> which will adjust the Pod <em>resources</em>. For that however a <a href="https://kubernetes.io/docs/concepts/api-extension/custom-resources/" rel="nofollow noreferrer"><code>CustomResourceDefinition</code></a> is necessary, therefore even the appropriate permissions to the cluster for creating CRDs and deploying it (thus may not work if you have limited access and you would need to contact the cluster admin).</p>
<blockquote>
<p>... it will set the requests automatically based on usage ... It will also maintain ratios between limits and requests that were specified in initial containers configuration.</p>
</blockquote>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: my-app-vpa
spec:
  targetRef:
    apiVersion: &quot;apps/v1&quot;
    kind:       Deployment
    name:       my-app
  updatePolicy:
    updateMode: &quot;Auto&quot;
</code></pre>
<p>or even <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/" rel="nofollow noreferrer">Horizontal autoscaling</a> in case you need to have separate <em>instances</em> of the applications (thus is managing the <code>Pod</code> <em>count</em>).</p>
