title: Problem with copying local data onto HDFS on a Hadoop cluster using Amazon EC2/ S3
body: <p>I have setup a Hadoop cluster containing 5 nodes on Amazon EC2. Now, when i login into the Master node and submit the following command</p>
  
  <pre><code>bin/hadoop jar &lt;program&gt;.jar &lt;arg1&gt; &lt;arg2&gt; &lt;path/to/input/file/on/S3&gt;
  </code></pre>
  
  <p>It throws the following errors (not at the same time.) The first error is thrown when i don't replace the slashes with '%2F' and the second is thrown when i replace them with '%2F':</p>
  
  <pre><code>1) Java.lang.IllegalArgumentException: Invalid hostname in URI S3://&lt;ID&gt;:&lt;SECRETKEY&gt;@&lt;BUCKET&gt;/&lt;path-to-inputfile&gt;
  2) org.apache.hadoop.fs.S3.S3Exception: org.jets3t.service.S3ServiceException: S3 PUT failed for '/' XML Error Message: The request signature we calculated does not match the signature you provided. check your key and signing method.
  </code></pre>
  
  <p>Note: </p>
  
  <p>1)when i submitted jps to see what tasks were running on the Master, it just showed</p>
  
  <pre><code>1116 NameNode
  1699 Jps
  1180 JobTracker
  </code></pre>
  
  <p>leaving DataNode and TaskTracker.</p>
  
  <p>2)My Secret key contains two '/' (forward slashes). And i replace them with '%2F' in the S3 URI.</p>
  
  <p>PS: The program runs fine on EC2 when run on a single node. Its only when i launch a cluster, i run into issues related to copying data to/from S3 from/to HDFS. And, what does distcp do? Do i need to distribute the data even after i copy the data from S3 to HDFS?(I thought, HDFS took care of that internally)</p>
  
  <p>IF you could direct me to a link that explains running Map/reduce programs on a hadoop cluster using Amazon EC2/S3. That would be great.</p>
  
  <p>Regards,</p>
  
  <p>Deepak.</p>