<p>I have tried to deploy so far in the below 2 ways:</p>

<pre><code>1. Deploy it as a statefulset on Kubernetes and using persistent 
volume of nfs as its storage . But then I learnt that we shouldn't 
be running minio backed by NAS as erasure code should not be run on NAS data.

2. Deploy it as Daemonset using local volumes. This time I attached 
separate disks to my nodes and labelled them such 
that minio is scheduled to run on these nodes. The disks are mounted on 
/data/minio on each of the nodes
</code></pre>

<p>But now everyday, the nodes are coming under disk pressure and minio pods are 
getting evicted. When I check the kubelet logs:</p>

<pre><code>Aug 13 21:05:45 staging-node2 kubelet[2188]: I0813 21:05:45.968179    2188 kubelet_pods.go:1073] Killing unwanted pod "minio-kjrkc"   
Aug 13 21:05:45 staging-node2 kubelet[2188]: I0813 21:05:45.975372    2188 kuberuntime_container.go:559] Killing container "docker://6da1247718f8e6c92399e231f8c31ff1c510737c658ac2aca87c1659aa6b51cc" with 30 second grace period
</code></pre>

<p>It tries to kill the pods but the container never dies. Even if minio gets TERMINATED signal, container is still up.</p>

<ul>
<li><blockquote>
  <p>What other options are left for an on-prem minio installations?</p>
</blockquote></li>
<li><blockquote>
  <p>Is using local storage somehow not recommended or am I using it incorrectly ?</p>
</blockquote></li>
<li><blockquote>
  <p>Any idea if I have to explicitly configure any pre-hook for minio to accept the terminate signal ?</p>
</blockquote></li>
</ul>

swetad90
<p>Right approach to deploy MinIO on Kubernetes is using local volumes on multiple distributed nodes using a StatefulSet. You can do this via Yaml files or Helm Chart or via our MinIO Operator. Docs are available here <a href="https://github.com/minio/minio/tree/master/docs/orchestration/kubernetes" rel="nofollow noreferrer">https://github.com/minio/minio/tree/master/docs/orchestration/kubernetes</a></p>

<blockquote>
  <p>What other options are left for an on-prem minio installations?</p>
</blockquote>

<p>A StatefulSet with atleast 4 Pods and storage via <code>hostPath</code> volumes is generally the right way to deploy. Depending on the use case and existing infrastructure there may be other relevant approaches too.</p>

<blockquote>
  <p>Is using local storage somehow not recommended or am I using it incorrectly ?</p>
</blockquote>

<p>Local storage is recommended. I'll need more info to see what exactly is going wrong here, but if you have dedicated local storage drives, you should be fine.</p>

<blockquote>
  <p>Any idea if I have to explicitly configure any pre-hook for minio to accept the terminate signal ?</p>
</blockquote>

<p>MinIO handles signals to terminate etc. There is no pre-hook needed as such.</p>

<p>Hope this helps, please join us on our Slack channel @ <a href="https://slack.min.io" rel="nofollow noreferrer">https://slack.min.io</a> for detailed discussion etc.</p>
