title: best way to preprocess the data
body: <p>I have data which has following schema</p>
  
  <p><code>person_id | category_id | date | type</code></p>
  
  <p>Each day data collected is huge close to 95GB on average.
  My use-case is to fetch all the person_ids for given category_id in provided date range.
  This is present on hdfs in csv files.
  When using spark to process for 3 weeks it takes half an hour to run on actual dump.
  How do I preprocess it to increase performance of my spark job ?
  I tried grouping on date but did not help much.</p>