title: java.lang.OutOfMemoryError: unable to create new native thread for big data set
body: <p>I have running hive query which running fine for small dataset. but i am running for 250 million records i have getting below errors in logs</p>
  
  <pre><code> FATAL org.apache.hadoop.mapred.Child: Error running child : java.lang.OutOfMemoryError:   unable to create new native thread
      at java.lang.Thread.start0(Native Method)
      at java.lang.Thread.start(Thread.java:640)
      at org.apache.hadoop.mapred.Task$TaskReporter.startCommunicationThread(Task.java:725)
      at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:362)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
      at org.apache.hadoop.mapred.Child.main(Child.java:249)
  
  
  
   2013-03-18 14:12:58,907 WARN org.apache.hadoop.mapred.Child: Error running child
   java.io.IOException: Cannot run program "ln": java.io.IOException: error=11, Resource temporarily unavailable
      at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
      at java.lang.Runtime.exec(Runtime.java:593)
      at java.lang.Runtime.exec(Runtime.java:431)
      at java.lang.Runtime.exec(Runtime.java:369)
      at org.apache.hadoop.fs.FileUtil.symLink(FileUtil.java:567)
      at org.apache.hadoop.mapred.TaskRunner.symlink(TaskRunner.java:787)
      at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:752)
      at org.apache.hadoop.mapred.Child.main(Child.java:225)
   Caused by: java.io.IOException: java.io.IOException: error=11, Resource temporarily unavailable
      at java.lang.UNIXProcess.&lt;init&gt;(UNIXProcess.java:148)
      at java.lang.ProcessImpl.start(ProcessImpl.java:65)
      at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
      ... 7 more
  2013-03-18 14:12:58,911 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task
  2013-03-18 14:12:58,911 INFO org.apache.hadoop.mapred.Child: Error cleaning up
    java.lang.NullPointerException
      at org.apache.hadoop.mapred.Task.taskCleanup(Task.java:1048)
      at org.apache.hadoop.mapred.Child.main(Child.java:281)
  </code></pre>
  
  <p>need help on this.</p>