title: How to create AWS Athena table via Glue crawler when the s3 data store has both json and .gz compressed files?
body: <p>I have two problems in my intended solution:</p>
  
  <p>1.
  My S3 store structure is as following:</p>
  
  <pre><code>mainfolder/date=2019-01-01/hour=14/abcd.json
  mainfolder/date=2019-01-01/hour=13/abcd2.json.gz
  ...
  mainfolder/date=2019-01-15/hour=13/abcd74.json.gz
  </code></pre>
  
  <p>All json files have the same schema and I want to make a crawler pointing to <em>mainfolder/</em> which can then create a table in Athena for querying.</p>
  
  <p>I have already tried with just one file format, e.g. if the files are just <em>json</em> or just <em>gz</em> then the crawler works perfectly but I am looking for a solution through which I can automate either type of file processing. I am open to write a custom script or any out of the box solution but need pointers where to start.</p>
  
  <p>2.
  The second issue that my json data has a field(column) which the crawler interprets as <em>struct</em> data but I want to make that field type as <em>string</em>. Reason being that if the type remains <em>struct</em> the date/hour partitions get a mismatch error as obviously struct data has not the same internal schema across the files. I have tried to make a custom classifier but there are no options there to describe data types.</p>