title: Writing sub directories in Hive partitions
body: <p>Problem statement</p>
  
  <p>I have files as follows with the schema Event_Time, AD_id</p>
  
  <pre><code>file_20170102-May have records with event_time for 20170101,20170102,20170103
  file_20170103-May have records with event_time for 20170102,20170103,20170104
  </code></pre>
  
  <p>Here event time is the time when the event occurred and the timestamp on filename is when the events were collected.So the timestamp on the filename and the event_time inside the file are out of sync.</p>
  
  <p>When i write this data to hive i definitely need to write data based on event_time partition because the users are interested in queries based on event_time.</p>
  
  <p>so my out put looks as follow</p>
  
  <pre><code>/path/to/output/event_time=20170102/....parquet
  /path/to/output/event_time=20170103/....parquet
  </code></pre>
  
  <p>However i need to be able to keep track of file timestamp because some times a file gets reposted and we want to go delete already processed files based on file timestamp.</p>
  
  <p>is there a way i could write this
  /path/to/output/event_time=20170101/20170202(file_tiemstamp)</p>
  
  <p>Please note that in the above 20170102(file_timestamp) is a directory and not hive partition.</p>
  
  <p>ALternately can i control the name of the parquet file so when i want to delete a file name its easy to figure out which files to delete </p>