title: Difference between 3 memory parameters in Hadoop 2?
body: <p>I'm using Hadoop 2.0.5 (Alpha) to run relatively big jobs, and I've ran into these errors:</p>
  
  <blockquote>
    <p>Container
    [pid=15023,containerID=container_1378641992707_0002_01_000029] is
    running beyond virtual memory limits. Current usage: 492.4 MB of <strong>1 GB</strong>
    physical memory used; 3.3 GB of <strong>2.1 GB</strong> virtual memory used. Killing
    container.</p>
  </blockquote>
  
  <p>I then learned about these two parameters:</p>
  
  <p><strong>yarn.nodemanager.vmem-pmem-ratio property</strong> which is set to 2.1 by default.</p>
  
  <p><strong>yarn.app.mapreduce.am.command-opts</strong> which is set to -Xmx1024mb (=1GB) by default.</p>
  
  <p>That explained the limits marked above. </p>
  
  <p>Setting these parameters to a higher value did help, but then I found this parameter:
  <strong>yarn.app.mapreduce.am.resource.mb</strong> which is set to 1536 by default.</p>
  
  <p>And I can't quite tell the difference between the 3 from the description given in Hadoop's default XMLs, nor how should I properly set them in means of optimization.</p>
  
  <p>An explanation or a good reference would be much appreciated</p>