title: Understanding parallelism in Spark and Scala
body: <p>I have some confusion about parallelism in Spark and Scala. I am running an experiment in which I have to read many (csv) files from the disk change/ process certain columns and then write it back to the disk.</p>
  
  <p>In my experiments, if I use <strong>SparkContext's parallelize method only then it does not seem to have any impact on the performance. However simply using Scala's parallel collections  (through par) reduces the time almost to half.</strong></p>
  
  <p>I am running my experiments in localhost mode with the arguments local[2] for the spark context.</p>
  
  <p>My question is when should I use scala's parallel collections and when to use spark context's parallelize? </p>