title: What is the difference between Spark's Partition Pruning and Predicate Pushdown?
body: <p>I was going through Spark optimization methods and came across various ways to implement to achieve optimization. But two names caught my eyes.</p>
  
  <ol>
  <li>Partition Pruning </li>
  <li>Predicate Pushdown</li>
  </ol>
  
  <p>They say:</p>
  
  <p>Partition Pruning:</p>
  
  <blockquote>
    <p>Partition pruning is a performance optimization that limits the number
    of files and partitions that Spark reads when querying. After
    partitioning the data, queries that match certain partition filter
    criteria improve performance by allowing Spark to only read a subset
    of the directories and files.</p>
  </blockquote>
  
  <p>Predicate Pushdown:</p>
  
  <blockquote>
    <p>Spark will attempt to move filtering of data as close to the source as
    possible to avoid loading unnecessary data into memory. Parquet and
    ORC files maintain various stats about each column in different chunks
    aof data (such as min and max values). Programs reading these files
    can use these indexes to determine if certain chunks, and even entire
    files, need to be read at all.  This allows programs to potentially
    skip over huge portions of the data during processing.</p>
  </blockquote>
  
  <p>By reading the above concepts, they appear to do the same thing which is to apply read statements (queries) that satisfy the predicates given in the query.
  Are Partition Pruning and Predicate Pushdown different concepts or I'm looking at them in a wrong way?</p>