title: Parquet vs ORC vs ORC with Snappy
body: <p>I am running a few tests on the storage formats available with Hive and using Parquet and ORC as major options. I included ORC once with default compression and once with Snappy.</p>
  
  <p>I have read many a documents that state Parquet to be better in time/space complexity as compared to ORC but my tests are opposite to the documents I went through.</p>
  
  <p>Follows some details of my data.</p>
  
  <pre><code>Table A- Text File Format- 2.5GB
  
  Table B - ORC - 652MB
  
  Table C - ORC with Snappy - 802MB
  
  Table D - Parquet - 1.9 GB
  </code></pre>
  
  <p>Parquet was worst as far as compression for my table is concerned.</p>
  
  <p>My tests with the above tables yielded following results.</p>
  
  <p><strong>Row count operation</strong></p>
  
  <pre><code>Text Format Cumulative CPU - 123.33 sec
  
  Parquet Format Cumulative CPU - 204.92 sec
  
  ORC Format Cumulative CPU - 119.99 sec 
  
  ORC with SNAPPY Cumulative CPU - 107.05 sec
  </code></pre>
  
  <p><strong>Sum of a column operation</strong></p>
  
  <pre><code>Text Format Cumulative CPU - 127.85 sec   
  
  Parquet Format Cumulative CPU - 255.2 sec   
  
  ORC Format Cumulative CPU - 120.48 sec   
  
  ORC with SNAPPY Cumulative CPU - 98.27 sec
  </code></pre>
  
  <p><strong>Average of a column operation</strong></p>
  
  <pre><code>Text Format Cumulative CPU - 128.79 sec
  
  Parquet Format Cumulative CPU - 211.73 sec    
  
  ORC Format Cumulative CPU - 165.5 sec   
  
  ORC with SNAPPY Cumulative CPU - 135.45 sec 
  </code></pre>
  
  <p><strong>Selecting 4 columns from a given range using where clause</strong></p>
  
  <pre><code>Text Format Cumulative CPU -  72.48 sec 
  
  Parquet Format Cumulative CPU - 136.4 sec       
  
  ORC Format Cumulative CPU - 96.63 sec 
  
  ORC with SNAPPY Cumulative CPU - 82.05 sec 
  </code></pre>
  
  <p>Does that mean ORC is faster then Parquet? Or there is something that I can do to make it work better with query response time and compression ratio?</p>
  
  <p>Thanks!   </p>