title: Spark broadcast error: exceeds spark.akka.frameSize Consider using broadcast
body: <p>I have a large data called "edges"</p>
  
  <pre><code>org.apache.spark.rdd.RDD[org.apache.spark.graphx.Edge[(String, Int)]] = MappedRDD[27] at map at &lt;console&gt;:52
  </code></pre>
  
  <p>When I was working in standalone mode, I was able to collect, count and save this file.  Now, on a cluster, I'm getting this error</p>
  
  <pre><code>edges.count
  ...
  Serialized task 28:0 was 12519797 bytes which exceeds spark.akka.frameSize
    (10485760 bytes). Consider using broadcast variables for large values.
  </code></pre>
  
  <p>Same with .saveAsTextFile("edges")</p>
  
  <p>This is from the spark-shell.  I have tried using the option<br>
      --driver-java-options "-Dspark.akka.frameSize=15"</p>
  
  <p>But when I do that, it just hangs indefinitely. Any help would be appreciated.  </p>
  
  <p>** EDIT **</p>
  
  <p>My standalone mode was on Spark 1.1.0 and my cluster is Spark 1.0.1.</p>
  
  <p>Also, the hanging occurs when I go to count, collect or saveAs* the RDD, but defining it or doing filters on it work just fine.</p>