title: Spark: Difference between numPartitions in read.jdbc(..numPartitions..) and repartition(..numPartitions..)
body: <p>I'm perplexed between the behaviour of <code>numPartitions</code> parameter in the following methods:</p>
  
  <ol>
  <li><code>DataFrameReader.jdbc</code></li>
  <li><code>Dataset.repartition</code></li>
  </ol>
  
  <p>The <a href="https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameReader@jdbc(url:String,table:String,columnName:String,lowerBound:Long,upperBound:Long,numPartitions:Int,connectionProperties:java.util.Properties):org.apache.spark.sql.DataFrame" rel="noreferrer">official docs</a> of <code>DataFrameReader.jdbc</code> say following regarding <code>numPartitions</code> parameter</p>
  
  <blockquote>
    <p><strong>numPartitions</strong>:
    the number of partitions. This, along with lowerBound (inclusive), upperBound (exclusive), form partition strides for generated WHERE clause expressions used to split the column columnName evenly.</p>
  </blockquote>
  
  <p>And <a href="https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.Dataset@repartition(numPartitions:Int):org.apache.spark.sql.Dataset[T]" rel="noreferrer">official docs</a> of <code>Dataset.repartition</code> say</p>
  
  <blockquote>
    <p>Returns a new Dataset that has exactly <code>numPartitions</code> partitions.</p>
  </blockquote>
  
  <hr>
  
  <p>My current understanding:</p>
  
  <ol>
  <li>The <code>numPartition</code> parameter in <code>DataFrameReader.jdbc</code> method controls the <em>degree of parallelism</em> in reading the data from database</li>
  <li>The <code>numPartition</code> parameter in <code>Dataset.repartition</code> controls the <em>number of output files</em> that will be generated when this <code>DataFrame</code> would be written to disk</li>
  </ol>
  
  <hr>
  
  <p>My questions:</p>
  
  <ol>
  <li>If I read <code>DataFrame</code> via <code>DataFrameReader.jdbc</code> and then write it to disk (without invoking <code>repartition</code> method), then would there still be as many files in output as there would've been had I written out a <code>DataFrame</code> to disk after having invoked <code>repartition</code> on it?</li>
  <li>If the answer to the above question is:
  
  <ul>
  <li>Yes: Then is it redundant to invoke <code>repartition</code> method on a <code>DataFrame</code> that was read using <code>DataFrameReader.jdbc</code> method (with <code>numPartitions</code> parameter)?</li>
  <li>No: Then please correct the lapses in my understanding. Also in that case shouldn't the <code>numPartitions</code> parameter of <code>DataFrameReader.jdbc</code> method be called something like <em>'parallelism'</em>?</li>
  </ul></li>
  </ol>