title: Unable to run mapreduce wordcount
body: <p>I am trying to teach myself some hadoop basics and so have build a simple hadoop cluster. This works and I can put, ls, cat from the hdfs filesystem without any issues.
  So I took the next step and tried to do a wordcount on a file I had put into hadoop, but I get the following error</p>
  
  <pre><code> $ hadoop jar /home/hadoop/share/hadoop/mapreduce/*examples*.jar wordcount     data/sectors.txt results
  2018-06-06 07:57:36,936 INFO client.RMProxy: Connecting to ResourceManager     at ansdb1/10.49.17.12:8040
  2018-06-06 07:57:37,404 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/hadoop/.staging/job_1528191458385_0014
  2018-06-06 07:57:37,734 INFO input.FileInputFormat: Total input files to process : 1
  2018-06-06 07:57:37,869 INFO mapreduce.JobSubmitter: number of splits:1
  2018-06-06 07:57:37,923 INFO Configuration.deprecation: yarn.resourcemanager.system-metrics-publisher.enabled is deprecated. Instead, use yarn.system-metrics-publisher.enabled
  2018-06-06 07:57:38,046 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1528191458385_0014
  2018-06-06 07:57:38,048 INFO mapreduce.JobSubmitter: Executing with tokens: []
  2018-06-06 07:57:38,284 INFO conf.Configuration: resource-types.xml not found
  2018-06-06 07:57:38,284 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
  2018-06-06 07:57:38,382 INFO impl.YarnClientImpl: Submitted application application_1528191458385_0014
  2018-06-06 07:57:38,445 INFO mapreduce.Job: The url to track the job: http://ansdb1:8088/proxy/application_1528191458385_0014/
  2018-06-06 07:57:38,446 INFO mapreduce.Job: Running job: job_1528191458385_0014
  2018-06-06 07:57:45,499 INFO mapreduce.Job: Job job_1528191458385_0014 running in uber mode : false
  2018-06-06 07:57:45,501 INFO mapreduce.Job:  map 0% reduce 0%
  2018-06-06 07:57:45,521 INFO mapreduce.Job: Job job_1528191458385_0014 failed with state FAILED due to: Application application_1528191458385_0014 failed 2 times due to AM Container for appattempt_1528191458385_0014_000002 exited with  exitCode: 1
  Failing this attempt.Diagnostics: [2018-06-06 07:57:43.301]Exception from container-launch.
  Container id: container_1528191458385_0014_02_000001
  Exit code: 1
  
  [2018-06-06 07:57:43.304]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
  Last 4096 bytes of prelaunch.err :
  Last 4096 bytes of stderr :
  Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
  
  Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:
  &lt;property&gt;
    &lt;name&gt;yarn.app.mapreduce.am.env&lt;/name&gt;
    &lt;value&gt;HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
    &lt;name&gt;mapreduce.map.env&lt;/name&gt;
    &lt;value&gt;HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
    &lt;name&gt;mapreduce.reduce.env&lt;/name&gt;
    &lt;value&gt;HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}&lt;/value&gt;
  &lt;/property&gt;
  
  [2018-06-06 07:57:43.304]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
  Last 4096 bytes of prelaunch.err :
  Last 4096 bytes of stderr :
  Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
  
  Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:
  &lt;property&gt;
    &lt;name&gt;yarn.app.mapreduce.am.env&lt;/name&gt;
    &lt;value&gt;HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
    &lt;name&gt;mapreduce.map.env&lt;/name&gt;
    &lt;value&gt;HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
    &lt;name&gt;mapreduce.reduce.env&lt;/name&gt;
    &lt;value&gt;HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}&lt;/value&gt;
  &lt;/property&gt;
  
  For more detailed output, check the application tracking page: http://ansdb1:8088/cluster/app/application_1528191458385_0014 Then click on links to logs of each attempt.
  . Failing the application.
  2018-06-06 07:57:45,558 INFO mapreduce.Job: Counters: 0
  </code></pre>
  
  <p>I have searched lots of website and they seem to say that my environment isn't right. I have tried many of the suggested fixes, but nothing has worked.</p>
  
  <p>Everything is running on both nodes:</p>
  
  <pre><code>$ jps
  31858 ResourceManager
  31544 SecondaryNameNode
  6152 Jps
  31275 DataNode
  31132 NameNode
  $ ssh ansdb2 jps
  16615 NodeManager
  21290 Jps
  16478 DataNode
  </code></pre>
  
  <p>I can ls hadoop:</p>
  
  <pre><code>$ hadoop fs -ls /
  Found 3 items
  drwxrwxrwt   - hadoop supergroup          0 2018-06-06 07:58 /tmp
  drwxr-xr-x   - hadoop supergroup          0 2018-06-05 11:46 /user
  drwxr-xr-x   - hadoop supergroup          0 2018-06-05 07:50 /usr
  </code></pre>
  
  <p>hadoop version:</p>
  
  <pre><code>$ hadoop version
  Hadoop 3.1.0
  Source code repository https://github.com/apache/hadoop -r 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d
  Compiled by centos on 2018-03-30T00:00Z
  Compiled with protoc 2.5.0
  From source with checksum 14182d20c972b3e2105580a1ad6990
  This command was run using /home/hadoop/share/hadoop/common/hadoop-common-3.1.0.jar
  </code></pre>
  
  <p>hadoop classpath:</p>
  
  <pre><code>$ hadoop classpath
  /home/hadoop/etc/hadoop:/home/hadoop/share/hadoop/common/lib/*:/home/hadoop/share/hadoop/common/*:/home/hadoop/share/hadoop/hdfs:/home/hadoop/share/hadoop/hdfs/lib/*:/home/hadoop/share/hadoop/hdfs/*:/home/hadoop/share/hadoop/mapreduce/*:/home/hadoop/share/hadoop/yarn:/home/hadoop/share/hadoop/yarn/lib/*:/home/hadoop/share/hadoop/yarn/*
  </code></pre>
  
  <p>my environment is setup:</p>
  
  <pre><code># hadoop
  ## JAVA env variables
  export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.el7.x86_64
  export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar
  
  ## HADOOP env variables
  export HADOOP_HOME=/home/hadoop
  export HADOOP_MAPRED_HOME=$HADOOP_HOME
  export HADOOP_COMMON_HOME=$HADOOP_HOME
  export HADOOP_HDFS_HOME=$HADOOP_HOME
  export HADOOP_YARN_HOME=$HADOOP_HOME
  export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"
  export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
  export YARN_HOME=$HADOOP_HOME
  export HADOOP_INSTALL=$HADOOP_HOME
  export HADOOP_CONF_DIR=$HADOOP_HOME
  export HADOOP_LIBEXEC_DIR=$HADOOP_HOME/libexec
  export JAVA_LIBRARY_PATH=$HADOOP_HOME/lib/native:$JAVA_LIBRARY_PATH
  export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
  export HADOOP_INSTALL=$HADOOP_HOME
  
  PATH=$PATH:$JAVA_HOME/bin
  export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
  </code></pre>
  
  <p>My hadoop xml files</p>
  
  <p>core-site.xml:</p>
  
  <pre><code>$ cat $HADOOP_HOME/etc/hadoop/core-site.xml
  &lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;?xml-stylesheet type="text/xsl" href="configuration.xsl"?&gt;
  &lt;configuration&gt;
   &lt;property&gt;
    &lt;name&gt;fs.defaultFS&lt;/name&gt;
    &lt;value&gt;hdfs://ansdb1:9000/&lt;/value&gt;
   &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>hdfs-site.xml:</p>
  
  <pre><code>$ cat $HADOOP_HOME/etc/hadoop/hdfs-site.xml
  &lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;?xml-stylesheet type="text/xsl" href="configuration.xsl"?&gt;
  &lt;configuration&gt;
   &lt;property&gt;
    &lt;name&gt;dfs.data.dir&lt;/name&gt;
    &lt;value&gt;/data/hadoop/datanode&lt;/value&gt;
   &lt;/property&gt;
   &lt;property&gt;
    &lt;name&gt;dfs.name.dir&lt;/name&gt;
    &lt;value&gt;/data/hadoop/namenode&lt;/value&gt;
   &lt;/property&gt;
   &lt;property&gt;
    &lt;name&gt;dfs.checkpoint.dir&lt;/name&gt;
    &lt;value&gt;/data/hadoop/secondarynamenode&lt;/value&gt;
   &lt;/property&gt;
   &lt;property&gt;
    &lt;name&gt;dfs.replication&lt;/name&gt;
    &lt;value&gt;2&lt;/value&gt;
   &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>yarn-site.xml:</p>
  
  <pre><code>$ cat $HADOOP_HOME/etc/hadoop/yarn-site.xml
  &lt;?xml version="1.0"?&gt;
  &lt;configuration&gt;
   &lt;property&gt;
    &lt;name&gt;yarn.resourcemanager.hostname&lt;/name&gt;
    &lt;value&gt;ansdb1&lt;/value&gt;
   &lt;/property&gt;
   &lt;property&gt;
    &lt;name&gt;yarn.nodemanager.aux-services&lt;/name&gt;
    &lt;value&gt;mapreduce_shuffle&lt;/value&gt;
   &lt;/property&gt;
   &lt;property&gt;
    &lt;name&gt;yarn.nodemanager.aux-services.mapreduce.shuffle.class&lt;/name&gt;
    &lt;value&gt;org.apache.hadoop.mapred.ShuffleHandler&lt;/value&gt;
   &lt;/property&gt;
   &lt;property&gt;
    &lt;name&gt;yarn.resourcemanager.resource-tracker.address&lt;/name&gt;
    &lt;value&gt;ansdb1:8025&lt;/value&gt;
   &lt;/property&gt;
   &lt;property&gt;
    &lt;name&gt;yarn.resourcemanager.scheduler.address&lt;/name&gt;
    &lt;value&gt;ansdb1:8030&lt;/value&gt;
   &lt;/property&gt;
   &lt;property&gt;
    &lt;name&gt;yarn.resourcemanager.address&lt;/name&gt;
    &lt;value&gt;ansdb1:8040&lt;/value&gt;
   &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>mapred-site.xml:</p>
  
  <pre><code>$ cat $HADOOP_HOME/etc/hadoop/mapred-site.xml
  &lt;?xml version="1.0"?&gt;
  &lt;?xml-stylesheet type="text/xsl" href="configuration.xsl"?&gt;
  &lt;configuration&gt;
   &lt;property&gt;
    &lt;name&gt;mapreduce.framework.name&lt;/name&gt;
    &lt;value&gt;yarn&lt;/value&gt;
   &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>I have checked which jar file contains MRAppMaster:</p>
  
  <pre><code>$ find /home/hadoop -name '*.jar' -exec grep -Hls MRAppMaster {} \;
  /home/hadoop/share/hadoop/mapreduce/sources/hadoop-mapreduce-client-app-3.1.0-sources.jar
  /home/hadoop/share/hadoop/mapreduce/sources/hadoop-mapreduce-client-app-3.1.0-test-sources.jar
  /home/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.0.jar 
  </code></pre>
  
  <p>Clearly I am missing something, so could somebody please point me the right direction.</p>