title: Container killed by YARN for exceeding memory limits
body: <p>I am creating a cluster in google dataproc with the following characteristics:</p>
  
  <pre><code>Master Standard (1 master, N workers)
    Machine       n1-highmem-2 (2 vCPU, 13.0 GB memory)
    Primary disk  250 GB
  
  Worker nodes    2
    Machine type  n1-highmem-2 (2 vCPU, 13.0 GB memory)
    Primary disk  size    250 GB
  </code></pre>
  
  <p>I am also adding in <code>Initialization actions</code> the <code>.sh</code> file from this <a href="https://github.com/GoogleCloudPlatform/dataproc-initialization-actions/tree/master/zeppelin" rel="noreferrer">repository</a> in order to use zeppelin. </p>
  
  <p>The code that I use works fine with some data but if I use bigger amount of, I got the following error:</p>
  
  <pre><code>Container killed by YARN for exceeding memory limits. 4.0 GB of 4 GB physical memory used. Consider boosting spark.yarn.executor.memoryOverhead.
  </code></pre>
  
  <p>I have seen posts such as this one: <a href="https://stackoverflow.com/questions/40781354/container-killed-by-yarn-for-exceeding-memory-limits-10-4-gb-of-10-4-gb-physic">Container killed by YARN for exceeding memory...</a> where it is recommended to change <code>yarn.nodemanager.vmem-check-enabled</code> to <code>false</code>. </p>
  
  <p>I am a bit confused though. Are all these configurations happening when I initialize the cluster or not? </p>
  
  <p>Also where exactly is <code>yarn-site.xml</code> located? I am unable to find it in the master(cant find it in <code>/usr/lib/zeppelin/conf/</code>, <code>/usr/lib/spark/conf</code>, <code>/usr/lib/hadoop-yar/</code>) in order to change it, and if changed what do i need to 'restart'?</p>