title: How does spark streaming cache work?
body: <p>Through searching the internet, I learned that by default spark streaming will cache the rdd itself for a batch time, after that, it will be deleted. So we do not need to call rdd.cache() explicitly. Is my understanding correct? Also I have some doubt about the spark cache mechanism, for example:</p>
  
  <pre><code>val rdd = some method to generate rdd
  rdd.cache()  //will this rdd still be cached in memory?
  rdd = rdd.union(another rdd)
  </code></pre>
  
  <p>Any body can explain this question to me please, any help is appreciated!</p>