title: Cache() in Pyspark Dataframe
body: <p>I have a dataframe and I need to include several transformations on it. I thought of performing all the actions in the same dataframe. So if I need to use cache Should I cache the dataframe after every action performed in it ?</p>
  <pre><code>df=df.selectExpr(&quot;*&quot;,&quot;explode(area)&quot;).select(&quot;*&quot;,&quot;col.*&quot;).drop(*['col','area'])
  df.cache()
  df=df.withColumn('full_name',f.concat(f.col('first_name'),f.lit(' '),f.col('last_name'))).drop('first_name','last_name')
  df.cache()
  df=df.withColumn(&quot;cleaned_map&quot;, regexp_replace(&quot;date&quot;, &quot;[^0-9T]&quot;, &quot;&quot;)).withColumn(&quot;date_type&quot;, to_date(&quot;cleaned_map&quot;, &quot;ddMMyyyy&quot;)).drop('date','cleaned_map')
  df.cache()
  df=df.filter(df.date_type.isNotNull())
  df.show()
  </code></pre>
  <p>Should I add like this or caching once is enough ?</p>
  <p>Also I want to know if I use multiple dataframes instead of one for the above code should I include cache at every transformation. Thanks a lot !</p>