title: spark read parquet with partition filters vs complete path
body: <p>I have a partitioned parquet data in hdfs example:
  hdfs://cluster/stage/data/datawarehouse/table=metrics_data/country=india/year=2020/month=06/day=30/hour=23/&lt;part-files.parquet&gt;</p>
  <p>I would like to understand which is the best way to read the data:</p>
  <p><strong>df = spark.read.parquet(&quot;hdfs://cluster/stage/data/datawarehouse/table=metrics_data/country=india/year=2020/month=06/day=30/&quot;).where(col('hour') == &quot;23&quot;)</strong></p>
  <p>OR</p>
  <p><strong>df = spark.read.parquet(&quot;hdfs://cluster/stage/data/datawarehouse/table=metrics_data/country=india/year=2020/month=06/day=30/hour=23&quot;)</strong></p>
  <p>I would like to understand more in terms of performance and other advantages if any.</p>