title: Converting a Spark Dataframe to a mutable Map
body: <p>I am new to spark and scala. I am trying to query a table in hive(select 2 columns from the table) and convert the resulting dataframe into a Map. I am using Spark 1.6 with Scala 2.10.6.</p>
  
  <p>Ex:</p>
  
  <pre><code>Dataframe:
  +--------+-------+
  | address| exists|
  +--------+-------+
  |address1|   1   |
  |address2|   0   |
  |address3|   1   |
  +--------+-------+ 
  should be converted to: Map("address1" -&gt; 1, "address2" -&gt; 0, "address3" -&gt; 1)
  </code></pre>
  
  <p>This is the code I am using:</p>
  
  <pre><code>val testMap: scala.collection.mutable.Map[String,Any] = Map()
  val df= hiveContext.sql("select address,exists from testTable")
  qualys.foreach( r =&gt; {
    val key = r(0).toString
    val value = r(1)
    testMap+=(key -&gt; value)
    }
  )
  testMap.foreach(println)
  </code></pre>
  
  <p>When I run the above code, I get this error:</p>
  
  <pre><code>java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object;
  </code></pre>
  
  <p>It is throwing this error at the line where I am trying to add the key value pair to the Map. i.e. <code>testMap+=(key -&gt; value)</code></p>
  
  <p>I know that there is a better and simpler way of doing this using the <code>org.apache.spark.sql.functions.map</code>. However, I am using Spark 1.6 and I don't think this function is available. I tried doing the <code>import</code> and I didn't find it in the list of available functions.</p>
  
  <p>why is my approach giving me an error? and is there a better/elegant way of achieving this with spark 1.6? </p>
  
  <p>any help would be appreciated. Thank you!</p>
  
  <p><strong>UPDATE:</strong></p>
  
  <p>I changed the way the elements are being added to the Map to the following:     <code>testMap.put(key, value)</code>. </p>
  
  <p>I was previously using the <code>+=</code> for adding the elements. Now i don't get the <code>java.lang.NoSuchMethodError</code> anymore. However, no elements are getting added to the <code>testMap</code>. After the foreach step is complete, I tried to print the size of the map and all the elements in it and I see that there are <code>zero</code> elements. </p>
  
  <p>Why are the elements not getting added? I am also open to any other better approach. Thank you!!</p>