<p>i'm new in spark-operator i'm confused how to setup the resources request and limits in YAML file for example in my case i have request 512m of memory for driver pod, but what about limits, it is unbounded ?</p>

<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-html lang-html prettyprint-override"><code>spec:
  driver:
    cores: 1
    coreLimit: 200m
    memory: 512m
    labels:
      version: 2.4.5
    serviceAccount: spark</code></pre>
</div>
</div>
</p>

<p>spec:
  driver:
    cores: 1
    coreLimit: 200m
    memory: 512m
    labels:
      version: 2.4.5
    serviceAccount: spark</p>

<p>It is good practice to set limits when defining your yaml file. If you do not do so you run the risk of using all resources on node as per this <a href="https://kubernetes.io/docs/tasks/configure-pod-container/assign-memory-resource/#if-you-do-not-specify-a-memory-limit" rel="nofollow noreferrer">doc</a> since there is no upper bound.</p>

<hr>

<p>Memory limits for the driver and executor pods are set internally by the Kubernetes scheduler backend in Spark, and calculated as the value of spark.{driver|executor}.memory + memory overhead.</p>
