title: hadoop copying the result from hdfs to S3
body: <p>I've successfully completed my job on Amazon EMR, Now I want to copy results from <code>HDFS</code> to <code>S3</code>,but I have some problem </p>
  
  <p>this is the code (--steps)</p>
  
  <pre><code>{
      "Name":"AAAAA",
      "Type":"CUSTOM_JAR",
      "Jar":"command-runner.jar",
      "ActionOnFailure":"CONTINUE",
      "Args": [
          "s3-dist-cp",
          "--src", "hdfs:///seqaddid_output",
          "--dest", "s3://wuda-notebook/seqaddid"
      ]  
  }
  </code></pre>
  
  <p>this is the logs:</p>
  
  <blockquote>
    <p>2019-04-12 03:01:23,571 INFO com.amazon.elasticmapreduce.s3distcp.S3DistCp (main): Running with args: -libjars /usr/share/aws/emr/s3-dist-cp/lib/commons-httpclient-3.1.jar,/usr/share/aws/emr/s3-dist-cp/lib/commons-logging-1.0.4.jar,/usr/share/aws/emr/s3-dist-cp/lib/guava-18.0.jar,/usr/share/aws/emr/s3-dist-cp/lib/s3-dist-cp-2.10.0.jar,/usr/share/aws/emr/s3-dist-cp/lib/s3-dist-cp.jar --src hdfs:///seqaddid_output/ --dest s3://wuda-notebook/seqaddid 
    2019-04-12 03:01:24,196 INFO com.amazon.elasticmapreduce.s3distcp.S3DistCp (main): S3DistCp args: --src hdfs:///seqaddid_output/ --dest s3://wuda-notebook/seqaddid 
    2019-04-12 03:01:24,203 INFO com.amazon.elasticmapreduce.s3distcp.S3DistCp (main): Using output path 'hdfs:/tmp/4f93d497-fade-4c78-86b9-59fc3da35b4e/output'
    2019-04-12 03:01:24,263 INFO com.amazon.elasticmapreduce.s3distcp.S3DistCp (main): GET <a href="http://169.254.169.254/latest/meta-data/placement/availability-zone" rel="nofollow noreferrer">http://169.254.169.254/latest/meta-data/placement/availability-zone</a> result: us-east-1f
    2019-04-12 03:01:24,664 FATAL com.amazon.elasticmapreduce.s3distcp.S3DistCp (main): Failed to get source file system
    java.io.FileNotFoundException: File does not exist: hdfs:/seqaddid_output
        at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1444)
        at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1437)
        at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
        at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1452)
        at com.amazon.elasticmapreduce.s3distcp.S3DistCp.run(S3DistCp.java:795)
        at com.amazon.elasticmapreduce.s3distcp.S3DistCp.run(S3DistCp.java:705)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
        at com.amazon.elasticmapreduce.s3distcp.Main.main(Main.java:22)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:148)</p>
  </blockquote>