title: When can I unpersist my RDD?
body: <p>I have a method that gets called several times. This method looks like the following:</p>
  
  <pre><code>def separateGoodAndBad(myRDD: RDD[String]): RDD[String] = {
      val newRDD = myRDD.map(......)  //do stuff
      newRDD.cache  //newRDD has 2 actions performed on it
  
      val badRDD = newRDD.filter(row =&gt; row.contains("bad"))
      badRDD.count
  
      val goodRDD = newRDD.filter(row =&gt; row.contains("good"))
      goodRDD.count
  
      newRDD.unpersist // I am unpersisting because this method gets called several times
  
      goodRDD
  }
  </code></pre>
  
  <p>Like I said, I want to unpersist <code>newRDD</code> because the method gets called several times and I don't want 4 copies of different cached <code>newRDDs</code>. Here is a code sample:</p>
  
  <pre><code>val firstRDD = separateGoodAndBad(originalRDD)
  val firstRDDTransformed = doStuffToFirstRDD(firstRDD)
  
  val secondRDD = separateGoodAndBad(firstRDDTransformed)
  val secondRDDTransformed = doStuffToSecondRDD(secondRDD)
  
  val thirdRDD = separateGoodAndBad(secondRDDTransformed)
  val thirdRDDTransformed = doStuffToThirdRDD(thirdRDD)
  </code></pre>
  
  <p>However, <code>secondRDD</code> and <code>thirdRDD</code> are taking MUCH longer now that I added the unpersist (see above in <code>separateGoodAndBad()</code>. It seems that they are having to recompute <code>newRDD</code>. </p>
  
  <p>When can I unpersist <code>newRDD</code> so that it never has to get recomputed?</p>