title: Hadoop - Insufficient memory for the Java Runtime Environment while starting YARN service
body: <p>I have set up a cluster (1-master &amp; 2-slaves(slave1, slave2)) based on the tutorial <a href="http://pingax.com/install-apache-hadoop-ubuntu-cluster-setup" rel="nofollow noreferrer">http://pingax.com/install-apache-hadoop-ubuntu-cluster-setup</a>. When I ran for the first time both <code>HDFS</code> &amp; <code>YARN</code> services ran without any problem. But when I stopped an ran them again, I got the following while running <code>YARN</code> service (<code>start-yarn.sh</code>) from the master.</p>
  
  <pre><code># starting yarn daemons
  # starting resourcemanager, logging to /local/hadoop/logs/yarn-dev-resourcemanager-login200.out
  # 
  # There is insufficient memory for the Java Runtime Environment to continue.
  # Native memory allocation (malloc) failed to allocate 168 bytes for AllocateHeap
  # An error report file with more information is saved as: /local/hadoop/hs_err_pid21428.log
  
  Compiler replay data is saved as: /local/hadoop/replay_pid21428.log
  slave1: starting nodemanager, logging to /local/hadoop/logs/yarn-dev-nodemanager-login198.out
  slave2: starting nodemanager, logging to /local/hadoop/logs/yarn-dev-nodemanager-login199.out
  slave2: #
  slave2: # There is insufficient memory for the Java Runtime Environment to continue.
  slave2: # Native memory allocation (malloc) failed to allocate 168 bytes for AllocateHeap
  slave2: # An error report file with more information is saved as:
  slave2: # /local/hadoop/hs_err_pid27199.log
  slave2: #
  slave2: # Compiler replay data is saved as:
  slave2: # /local/hadoop/replay_pid27199.log
  </code></pre>
  
  <p>Based on the suggestions from <a href="https://stackoverflow.com/questions/8464048/out-of-memory-error-in-hadoop">out of Memory Error in Hadoop</a> and <a href="https://stackoverflow.com/questions/30295606/java-heap-space-out-of-memory-error-while-running-a-mapreduce-program">&quot;Java Heap space Out Of Memory Error&quot; while running a mapreduce program</a>, I varied the <code>heap memory</code> size limit to 256, 512, 1024 &amp; 2048 in all the 3 files <code>~/.bashrc</code>, <code>hadoop-env.sh</code> and <code>mapred-site.sh</code> but nothing worked.</p>
  
  <p>Note: I'm not an expert on Linux nor JVM.</p>
  
  <p>Log file contents from one of the nodes:</p>
  
  <pre><code># There is insufficient memory for the Java Runtime Environment to continue.
  # Native memory allocation (malloc) failed to allocate 32784 bytes for Chunk::new
  # Possible reasons:
  #   The system is out of physical RAM or swap space
  #   In 32 bit mode, the process size limit was hit
  # Possible solutions:
  #   Reduce memory load on the system
  #   Increase physical memory or swap space
  #   Check if swap backing store is full
  #   Use 64 bit Java on a 64 bit OS
  #   Decrease Java heap size (-Xmx/-Xms)
  #   Decrease number of Java threads
  #   Decrease Java thread stack sizes (-Xss)
  #   Set larger code cache with -XX:ReservedCodeCacheSize=
  # This output file may be truncated or incomplete.
  #
  #  Out of Memory Error (allocation.cpp:390), pid=16375, tid=0x00007f39a352c700
  #
  # JRE version: Java(TM) SE Runtime Environment (8.0_102-b14) (build 1.8.0_102-b14)
  # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.102-b14 mixed mode linux-amd64 compressed oops)
  # Core dump written. Default location: /local/hadoop/core or core.16375 (max size 1 kB). To ensure a full core dump, try "ulimit -c unlimited" before starting Java again
  
  CPU:total 1 (1 cores per cpu, 1 threads per core) family 6 model 45 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, aes, clmul, tsc, tscinvbit, tscinv
  
  Memory: 4k page, physical 2051532k(254660k free), swap 1051644k(1051324k free)
  </code></pre>