title: relationship between RDD , partitions and nodes
body: <p>I have been reading about RDDs and how various transformations are affected by partitions, and how some transformations affect partitions themselves. While I understand this, I am not able to relate it to the bigger picture as to how this fits in a cluster where we have multiple nodes.</p>
  
  <p>Is there one to one correspondence between a partition and a node? I mean if there is a single partition per node ideally? And if not, how does Spark decide how many partitions for a specific RDD have to reside on the same node?</p>
  
  <p>More specifically, I can think of one of the following:-</p>
  
  <p>1) All the partitions for a given RDD on the same node
  2) All partitions of the same RDD could reside on different nodes (but what is the basis of split?)
  3) Partitions of the same node are scattered across cluster, some of them on the same node, some of them on different nodes (again, what is the basis of this distribution?)</p>
  
  <p>Can someone please explain or at least point me to some specific link which answers exactly this?</p>