title: Hadoop: datanode process running but not working?
body: <p>We have been trying to install hadoop cluster these days. Sometimes succeed but most times failed. I make the configuration according to official document and some seems-high-quality blogs.</p>
  
  <p>The problem I meet is:
  All the process (including namenode,datanode, nodemanager,resourcemanager) can be seen by the command:<code>jps</code></p>
  
  <p>But the slaves are actually not working. I cannot see them in web interface master:8088 or master:50070</p>
  
  <p>Someone said that is caused by repeated namenode format and the id conflict. I think that is not to my problem since the datanodes don't work from the very beginning and the datanode folder are always empty.</p>
  
  <p>Is there any other possible causes to this phenomenon?
  I am really suffering to setup the cluster. </p>
  
  <p>Detail:</p>
  
  <p>Hadoop version:3.0.0-alpha2</p>
  
  <p>Output of hdfs dfsadmin -report is :</p>
  
  <pre><code>Configured Capacity: 492017770496 (458.23 GB)
  Present Capacity: 461047037952 (429.38 GB)
  DFS Remaining: 460770820096 (429.13 GB)
  DFS Used: 276217856 (263.42 MB)
  DFS Used%: 0.06%
  Under replicated blocks: 10069
  Blocks with corrupt replicas: 0
  Missing blocks: 0
  Missing blocks (with replication factor 1): 0
  Pending deletion blocks: 0
  
  -------------------------------------------------
  Live datanodes (1):
  
  Name: 127.0.0.1:9866 (localhost)
  Hostname: sr145.local.lan
  Decommission Status : Normal
  Configured Capacity: 492017770496 (458.23 GB)
  DFS Used: 276217856 (263.42 MB)
  Non DFS Used: 5954019328 (5.55 GB)
  DFS Remaining: 460770820096 (429.13 GB)
  DFS Used%: 0.06%
  DFS Remaining%: 93.65%
  Configured Cache Capacity: 0 (0 B)
  Cache Used: 0 (0 B)
  Cache Remaining: 0 (0 B)
  Cache Used%: 100.00%
  Cache Remaining%: 0.00%
  Xceivers: 1
  Last contact: Sun Jan 02 02:52:57 CST 2000
  </code></pre>
  
  <p>**The only one live datanode is the same node of master node.**All the other slaves node are not live.</p>
  
  <p>Configuration detail:
  1,hdfs-site.xml:</p>
  
  <pre><code>&lt;configuration&gt;
      &lt;property&gt;
          &lt;name&gt;dfs.namenode.secondary.http-address&lt;/name&gt;
          &lt;value&gt;sr145:9001&lt;/value&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;dfs.webhdfs.enabled&lt;/name&gt;
          &lt;value&gt;true&lt;/value&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;dfs.replication&lt;/name&gt;
          &lt;value&gt;3&lt;/value&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;dfs.namenode.name.dir&lt;/name&gt;
          &lt;value&gt;file:/opt/hadoop/hdfs/namenode&lt;/value&gt;
      &lt;/property&gt;
  
      &lt;property&gt;
          &lt;name&gt;dfs.datanode.data.dir&lt;/name&gt;
          &lt;value&gt;file:/opt/hadoop/hdfs/datanode&lt;/value&gt;
      &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>2,core-site.xml</p>
  
  <pre><code>&lt;configuration&gt;
      &lt;property&gt;
          &lt;name&gt;fs.defaultFS&lt;/name&gt;
          &lt;value&gt;hdfs://localhost:9000&lt;/value&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;hadoop.tmp.dir&lt;/name&gt;
          &lt;value&gt;file:/opt/hadoop/tmp&lt;/value&gt;
          &lt;description&gt;Abasefor other temporary directories.&lt;/description&gt;
      &lt;/property&gt;
  
  &lt;/configuration&gt;
  </code></pre>
  
  <p>3,yarn-site.xml</p>
  
  <pre><code>&lt;configuration&gt;
  
  &lt;!-- Site specific YARN configuration properties --&gt;
          &lt;property&gt;
                  &lt;name&gt;yarn.nodemanager.aux-services&lt;/name&gt;
                  &lt;value&gt;mapreduce_shuffle&lt;/value&gt;
          &lt;/property&gt;
          &lt;property&gt;
                  &lt;name&gt;yarn.manager.aux-services.mapreduce.shuffle.class&lt;/name&gt;
                  &lt;value&gt;org.apache.hadoop.mapred.ShuffleHandler&lt;/value&gt;
          &lt;/property&gt;
          &lt;property&gt;
                  &lt;name&gt;yarn.resourcemanager.hostname&lt;/name&gt;
                  &lt;value&gt;sr145&lt;/value&gt;
          &lt;/property&gt;
          &lt;property&gt;
             &lt;name&gt;yarn.resourcemanager.address&lt;/name&gt;
             &lt;value&gt;sr145:8032&lt;/value&gt;
          &lt;/property&gt;
          &lt;property&gt;
             &lt;name&gt;yarn.resourcemanager.scheduler.address&lt;/name&gt;
             &lt;value&gt;sr145:8030&lt;/value&gt;
          &lt;/property&gt;
          &lt;property&gt;
             &lt;name&gt;yarn.resourcemanager.resource-tracker.address&lt;/name&gt;
             &lt;value&gt;sr145:8035&lt;/value&gt;
          &lt;/property&gt;
          &lt;property&gt;
             &lt;name&gt;yarn.resourcemanager.admin.address&lt;/name&gt;
             &lt;value&gt;sr145:8033&lt;/value&gt;
          &lt;/property&gt;
          &lt;property&gt;
             &lt;name&gt;yarn.resourcemanager.webapp.address&lt;/name&gt;
             &lt;value&gt;sr145:8088&lt;/value&gt;
          &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>The configuration at all nodes are almost the same, only differs in the hdfs-site.xml natenode or datanode configuration. 
  The <code>workers</code> and <code>slaves</code> files in $HADOOP_HOME/etc/hadoop are also edited. No files are moved compared to the raw status. </p>