title: Does Spark support true column scans over parquet files in S3?
body: <p>One of the great benefits of the Parquet data storage format is that <a href="https://stackoverflow.com/a/36831549/189336">it's columnar</a>.  If I've got a 'wide' dataset with hundreds of columns, but my query only touches a few of those, then it's possible read only the data that stores those few columns, and skip the rest.</p>
  
  <p>Presumably this feature works by reading a bit of metadata at the head of a parquet file that indicates the locations on the filesystem for each column.  The reader can then seek on disk to read in only the necessary columns.</p>
  
  <p>Does anyone know whether spark's default parquet reader correctly implements this kind of selective seeking on S3?  I think <a href="https://stackoverflow.com/questions/36436057/s3-how-to-do-a-partial-read-seek-without-downloading-the-complete-file">it's supported by S3</a>, but there's a big difference between theoretical support and an implementation that properly exploits that support.</p>