title: Running EMR Spark With Multiple S3 Accounts
body: <p>I have an EMR Spark Job that needs to read data from S3 on one account and write to another.<br>
  I split my job into two steps.  </p>
  
  <ol>
  <li><p>read data from the S3 (no credentials required because my EMR cluster is in the same account).</p></li>
  <li><p>read data in the local HDFS created by step 1 and write it to an S3 bucket in another account.</p></li>
  </ol>
  
  <p>I've attempted setting the <code>hadoopConfiguration</code>:</p>
  
  <pre><code>sc.hadoopConfiguration.set("fs.s3n.awsAccessKeyId", "&lt;your access key&gt;")
  sc.hadoopConfiguration.set("fs.s3n.awsSecretAccessKey","&lt;your secretkey&gt;")
  </code></pre>
  
  <p>And exporting the keys on the cluster:</p>
  
  <pre><code>$ export AWS_SECRET_ACCESS_KEY=
  $ export AWS_ACCESS_KEY_ID=
  </code></pre>
  
  <p>I've tried both <strong>cluster</strong> and <strong>client</strong> mode as well as <strong>spark-shell</strong> with no luck.</p>
  
  <p>Each of them returns an error:</p>
  
  <pre><code>ERROR ApplicationMaster: User class threw exception: com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.s3.model.AmazonS3Exception: 
  Access Denied
  </code></pre>