title: How to construct (key, value) list from parallelized list in scala spark?
body: <p>I am new to Scala and I have some questions about how it works.
  I want to do the next thing : given list of values, I want to construct some imitation of dictionary in parallel, something like that: <code>(1,2,3,4) -&gt; ((1,1), (2,2), (3,3), (4,4) )</code>. I know that if we deal with parallelized collections we should use accumulators. So here is my attempt:</p>
  <pre><code>import org.apache.spark.{SparkConf, SparkContext}
  import org.apache.spark.util.AccumulatorV2
  import scala.collection.mutable.ListBuffer
  
  class DictAccumulatorV2 extends AccumulatorV2[Int, ListBuffer[(Int, Int)]] {
    private var dict:ListBuffer[(Int, Int)]= new ListBuffer[(Int, Int)]
  
    def reset(): Unit = {
      dict.clear()
    }
  
    def add(v: Int): Unit = {
      dict.append((v, v))
    }
    def value():ListBuffer[(Int, Int)] = {
      return dict
    }
    def isZero(): Boolean = {
      return dict.isEmpty
    }
    def copy() : AccumulatorV2[Int, ListBuffer[(Int, Int)]] = {
      // I do not understand how to code it correctly
      return new DictAccumulatorV2
    }
    def merge(other:AccumulatorV2[Int, ListBuffer[(Int, Int)]]): Unit = {
      // I do not understand how to code it correctly without reinitializing dict from val to var
      dict = dict ++ other.value
    }
  }
  object FirstSparkApplication {
    def main(args: Array[String]): Unit = {
      val conf = new SparkConf().setAppName(&quot;MyFirstApp&quot;).setMaster(&quot;local&quot;)
      val sc = new SparkContext(conf)
  
      val accum = new DictAccumulatorV2()
      sc.register(accum, &quot;mydictacc&quot;)
      val data = Array(1, 2, 3, 4, 5)
      val distData = sc.parallelize(data)
      var res = distData.map(x =&gt; accum.add(x))
      res.count()
      println(accum)
    }
  }
  </code></pre>
  <p>So I wonder if I do it right or there are any mistakes.</p>
  <p>In general I also have questions about how <code>sc.parallelize</code> works. Does it actually parallelize job on my machine or it's just fictional string of code? What should I put instead of <code>&quot;local&quot;</code> in <code>setMaster</code>? How can I see on which nodes is the task been performing? Is the task performed on the all of the nodes at the same time or there is some sequence?</p>