<p>I followed this tutorial to create a Kubernetes cluster on Azure to run build agents: <a href="http://www.chrisjohnson.io/2018/07/07/using-azure-kubernetes-service-aks-for-your-vsts-build-agents/" rel="nofollow noreferrer">http://www.chrisjohnson.io/2018/07/07/using-azure-kubernetes-service-aks-for-your-vsts-build-agents/</a></p>

<p>To recap what is there: a helm chart to do a deployment with a secret and a config map. For this deployment, I created a kubernetes cluster on Azure with all default settings and it is pulling an image from the docker hub with vsts build agent installed.</p>

<p>All was working fine, but recently pods started to be evicted pretty regularly, the message on them is:</p>

<blockquote>
  <p>Message:            Pod The node was low on resource: [DiskPressure].</p>
</blockquote>

<p>How can I fix this issue?</p>

<p>Either/Or:</p>

<ul>
<li><p>You upgrade the size of your main node disks with something like <a href="https://learn.microsoft.com/en-us/azure/virtual-machines/linux/expand-disks" rel="nofollow noreferrer">this</a>.</p></li>
<li><p>Check what pods are taking up space. Is it logs? Is it cached data? is it swap? Every application is different so you will have to go case by case.</p></li>
<li><p>Set <a href="https://learn.microsoft.com/en-us/azure/virtual-machines/linux/expand-disks" rel="nofollow noreferrer">local ephemeral</a> storage at the pod level for your workloads so that they don't go over. Pods using a lot will get evicted.</p></li>
<li><p>Use <a href="https://kubernetes.io/docs/concepts/storage/persistent-volumes/" rel="nofollow noreferrer">Persistent Volumes</a> for your workloads, especially some that are not local and just reserved for your applications.</p></li>
</ul>
