title: Streaming Command Failed! in RHADOOP
body: <p>I have installed RHADOOP in Hortonwork VM.  when I am running mapreduce code to verify it is throwing an error saying </p>
  
  <p>I am using user as :rstudio (not root.but has access to sudoer)</p>
  
  <blockquote>
    <p>Streaming Command Failed!</p>
  </blockquote>
  
  <p>Can anybody help me understanding the issue.I am not getting much idea to solve thios issue.</p>
  
  <pre><code>Sys.setenv(HADOOP_HOME="/usr/hdp/2.2.0.0-2041/hadoop")
  
      Sys.setenv(HADOOP_CMD="/usr/bin/hadoop")
      Sys.setenv(HADOOP_STREAMING="/usr/hdp/2.2.0.0-2041/hadoop-mapreduce/hadoop-streaming.jar")
      library(rhdfs)
      hdfs.init()
      library(rmr2)
      ints = to.dfs(1:10)
      calc = mapreduce(input = ints, map = function(k, v) cbind(v, 2*v)
  
  )
  </code></pre>
  
  <p>I am getting the error and below is the error in rhadoop</p>
  
  <pre><code>Error in mr(map = map, reduce = reduce, combine = combine, vectorized.reduce, : hadoop streaming failed with error code 1
  
  4
  stop("hadoop streaming failed with error code ", retval, "\n")
  3
  mr(map = map, reduce = reduce, combine = combine, vectorized.reduce, in.folder = if (is.list(input)) { lapply(input, to.dfs.path) } else to.dfs.path(input), out.folder = to.dfs.path(output), ...
  2
  mapreduce(input = input, output = output, input.format = "text", map = map)
  1
  wordcount(hdfs.data, hdfs.out)
  
  
  
  packageJobJar: [] [/usr/hdp/2.2.0.0-2041/hadoop-mapreduce/hadoop-streaming-2.6.0.2.2.0.0-2041.jar] /tmp/streamjob3075733686753367992.jar tmpDir=null
  15/04/07 21:43:10 INFO impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
  15/04/07 21:43:10 INFO client.RMProxy: Connecting to ResourceManager at localhost/127.0.0.1:8050
  15/04/07 21:43:11 INFO impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
  15/04/07 21:43:11 INFO client.RMProxy: Connecting to ResourceManager at localhost/127.0.0.1:8050
  15/04/07 21:43:11 INFO mapred.FileInputFormat: Total input paths to process : 1
  15/04/07 21:43:11 INFO mapreduce.JobSubmitter: number of splits:2
  15/04/07 21:43:12 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1428440418649_0006
  15/04/07 21:43:12 INFO impl.YarnClientImpl: Submitted application application_1428440418649_0006
  15/04/07 21:43:12 INFO mapreduce.Job: The url to track the job: http://sandbox.hortonworks.com:8088/proxy/application_1428440418649_0006/
  15/04/07 21:43:12 INFO mapreduce.Job: Running job: job_1428440418649_0006
  15/04/07 21:43:19 INFO mapreduce.Job: Job job_1428440418649_0006 running in uber mode : false
  15/04/07 21:43:19 INFO mapreduce.Job:  map 0% reduce 0%
  15/04/07 21:43:27 INFO mapreduce.Job: Task Id : attempt_1428440418649_0006_m_000001_0, Status : FAILED
  Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1
      at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)
      at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)
      at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
      at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
      at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
      at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  
  Container killed by the ApplicationMaster.
  Container killed on request. Exit code is 143
  Container exited with a non-zero exit code 143
  
  15/04/07 21:43:27 INFO mapreduce.Job: Task Id : attempt_1428440418649_0006_m_000000_0, Status : FAILED
  Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1
      at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)
      at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)
      at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
      at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
      at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
      at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  
  15/04/07 21:43:35 INFO mapreduce.Job: Task Id : attempt_1428440418649_0006_m_000001_1, Status : FAILED
  Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1
      at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)
      at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)
      at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
      at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
      at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
      at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  
  15/04/07 21:43:35 INFO mapreduce.Job: Task Id : attempt_1428440418649_0006_m_000000_1, Status : FAILED
  Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1
      at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)
      at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)
      at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
      at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
      at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
      at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  
  15/04/07 21:43:43 INFO mapreduce.Job: Task Id : attempt_1428440418649_0006_m_000001_2, Status : FAILED
  Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1
      at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)
      at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)
      at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
      at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
      at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
      at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  
  15/04/07 21:43:44 INFO mapreduce.Job: Task Id : attempt_1428440418649_0006_m_000000_2, Status : FAILED
  Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1
      at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)
      at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)
      at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
      at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
      at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
      at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  
  15/04/07 21:43:52 INFO mapreduce.Job:  map 100% reduce 0%
  15/04/07 21:43:53 INFO mapreduce.Job: Job job_1428440418649_0006 failed with state FAILED due to: Task failed task_1428440418649_0006_m_000001
  Job failed as tasks failed. failedMaps:1 failedReduces:0
  
  15/04/07 21:43:54 INFO mapreduce.Job: Counters: 13
      Job Counters 
          Failed map tasks=7
          Killed map tasks=1
          Launched map tasks=8
          Other local map tasks=6
          Data-local map tasks=2
          Total time spent by all maps in occupied slots (ms)=49670
          Total time spent by all reduces in occupied slots (ms)=0
          Total time spent by all map tasks (ms)=49670
          Total vcore-seconds taken by all map tasks=49670
          Total megabyte-seconds taken by all map tasks=12417500
      Map-Reduce Framework
          CPU time spent (ms)=0
          Physical memory (bytes) snapshot=0
          Virtual memory (bytes) snapshot=0
  15/04/07 21:43:54 ERROR streaming.StreamJob: Job not successful!
  Streaming Command Failed!
  Error in mr(map = map, reduce = reduce, combine = combine, vectorized.reduce,  : 
    hadoop streaming failed with error code 1
  </code></pre>