title: Spark: efficiency of dataframe checkpoint vs. explicitly writing to disk
body: <p>Checkpoint version:</p>
  <pre><code>val savePath = &quot;/some/path&quot;
  spark.sparkContext.setCheckpointDir(savePath)
  df.checkpoint()
  </code></pre>
  <p>Write to disk version:</p>
  <pre><code>df.write.parquet(savePath)
  val df = spark.read.parquet(savePath)
  </code></pre>
  <p>I think both break the lineage in the same way.</p>
  <p>In my experiments checkpoint is almost 30 bigger on disk than parquet (689GB vs. 24GB). In terms of running time, checkpoint takes 1.5 times longer (10.5 min vs 7.5 min).</p>
  <p>Considering all this, what would be the point of using checkpoint instead of saving to file? Am I missing something?</p>