title: Getting data in and out of Elastic MapReduce HDFS
body: <p>I've written a Hadoop program which requires a certain layout within HDFS, and which afterwards, I need to get the files out of HDFS.   It works on my single-node Hadoop setup and I'm eager to get it working on 10's of nodes within Elastic MapReduce.</p>
  
  <p>What I've been doing is something like this:</p>
  
  <pre><code>./elastic-mapreduce --create --alive
  JOBID="j-XXX" # output from creation
  ./elastic-mapreduce -j $JOBID --ssh "hadoop fs -cp s3://bucket-id/XXX /XXX"
  ./elastic-mapreduce -j $JOBID --jar s3://bucket-id/jars/hdeploy.jar --main-class com.ranjan.HadoopMain --arg /XXX
  </code></pre>
  
  <p>This is asynchronous, but when the job's completed, I can do this</p>
  
  <pre><code>./elastic-mapreduce -j $JOBID --ssh "hadoop fs -cp /XXX s3://bucket-id/XXX-output"
  ./elastic-mapreduce -j $JOBID --terminate
  </code></pre>
  
  <p>So while this <em>sort-of</em> works, but it's clunky and not what I'd like.  Is there cleaner way to do this?</p>
  
  <p>Thanks!</p>