title: Issues creating a Hive table backed by Avro files in Amazon S3
body: <p>I've been trying to create a Hive table backed by Avro files in S3. Initially, I thought this might be relatively simple to do, but I ran into the following error.</p>
  
  <p>Here's the create table command:</p>
  
  <pre><code>set fs.s3.awsAccessKeyId=ACCESS_KEY_ID;
  set fs.s3.awsSecretAccessKey=SECRET_ACCESS_KEY;
  use some_database;
  CREATE EXTERNAL TABLE experiment_with_s3_backed_data
  ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
  WITH SERDEPROPERTIES (
          'avro.schema.literal'='{
          "namespace": "",
          "type": "record",
          "name": "SomeAvroSchema",
          "fields": [
              {"name": "someVariable","type":"string"}
          ]
  }')
  STORED AS INPUTFORMAT
      'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
  OUTPUTFORMAT
      'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
  LOCATION 's3://MY_BUCKET/some/data/'
  ;
  </code></pre>
  
  <p>And here's the error I get:</p>
  
  <pre><code>AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3 URL, or by setting the fs.s3.awsAccessKeyId or fs.s3.awsSecretAccessKey properties (respectively).
  </code></pre>
  
  <p>I tried using both <code>s3</code> and <code>s3n</code> URLs and parameters, with the same result. I noticed related questions, which recommend adding the keys to <code>core-site.xml</code>, but I can't do this for two reasons:</p>
  
  <ol>
  <li>I can't change the Hadoop configuration, due to access restrictions.</li>
  <li>I might have different tables with different access privileges to S3, so I'm generally interested in providing users with the ability to load their S3 data into Hive tables.</li>
  </ol>
  
  <p>See <a href="https://stackoverflow.com/questions/3009792/problem-with-copying-local-data-onto-hdfs-on-a-hadoop-cluster-using-amazon-ec2">Problem with copying local data onto HDFS on a Hadoop cluster using Amazon EC2/ S3</a></p>