title: Spark performance enhancements by storing sorted Parquet files
body: <p>Will data extracts run quicker if a DataFrame is sorted before being persisted as Parquet files.</p>
  
  <p>Suppose we have the following <code>peopleDf</code> DataFrame (pretend this a sample and the real one has 20 billion rows):</p>
  
  <pre><code>+-----+----------------+
  | age | favorite_color |
  +-----+----------------+
  |  54 | blue           |
  |  10 | black          |
  |  13 | blue           |
  |  19 | red            |
  |  89 | blue           |
  +-----+----------------+
  </code></pre>
  
  <p>Let's write out sorted and unsorted versions of this DataFrame to Parquet files.</p>
  
  <pre><code>peopleDf.write.parquet("s3a://some-bucket/unsorted/")
  peopleDf.sort($"favorite_color").write.parquet("s3a://some-bucket/sorted/")
  </code></pre>
  
  <p>Are there any performance gains when reading in the sorted data and doing a data extract based on <code>favorite_color</code>?</p>
  
  <pre><code>val pBlue1 = spark.read.parquet("s3a://some-bucket/unsorted/").filter($"favorite_color" === "blue")
  
  // is this faster?
  
  val pBlue2 = spark.read.parquet("s3a://some-bucket/sorted/").filter($"favorite_color" === "blue")
  </code></pre>