title: Serialized task exceeds max allowed, Spark - Cluster
body: <p>So I'm working with java to deploy in spark, this was my original code:</p>
  
  <pre><code>List&lt;Float&gt; data = some_data;
  JavaRDD&lt;Float&gt; dataAsRDD = javaSparkContext.parallelize(data);
  JavaRDD&lt;Float&gt; dataWithoutNaN = dataAsRDD.filter(number -&gt; !number.isNaN());
  JavaDoubleRDD dataAsDouble = dataWithoutNaN.mapToDouble(number -&gt; (double) number);
  logger.info("\t\t\tMean: " + dataAsDouble.mean());
  </code></pre>
  
  <p>So, this can run in apache-spark stand alone mode with warning but in cluster mode keeps runing into an error: (line 86 is <em>dataAsDouble.mean()</em>)</p>
  
  <pre><code>17/01/06 17:54:24 INFO DAGScheduler: Job 0 failed: mean at Cluster.java:86, took 12.678086 s
  Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Serialized task 0:0 was 45337325 bytes, which exceeds max allowed: spark.akka.frameSize (10485760 bytes) - reserved (204800 bytes). Consider increasing spark.akka.frameSize or using broadcast variables for large values.
  </code></pre>
  
  <p>I follow the instructions suggested in the exception, <a href="https://stackoverflow.com/questions/36692386/exceeding-spark-akka-framesize-when-saving-word2vecmodel">Exceeding spark.akka.frameSize when saving Word2VecModel</a> and <a href="https://stackoverflow.com/questions/27218472/spark-broadcast-error-exceeds-spark-akka-framesize-consider-using-broadcast">Spark broadcast error: exceeds spark.akka.frameSize Consider using broadcast</a>, I used a broadcast variable:</p>
  
  <pre><code>List&lt;Float&gt; dataAsList = some_data;
  Broadcast&lt;List&lt;Float&gt;&gt; broadcast = javaSparkContext.broadcast(dataAsList);
  JavaRDD&lt;Float&gt; dataAsRDD = javaSparkContext.parallelize(broadcast.value());
  JavaRDD&lt;Float&gt; dataWithoutNaN = dataAsRDD.filter(number -&gt; !number.isNaN());
  JavaDoubleRDD dataAsDouble = dataWithoutNaN.mapToDouble(number -&gt; (double) number);
  logger.info("\t\t\tMean: " + dataAsDouble.mean());
  </code></pre>
  
  <p>But I keep runing in the same error, ¿what am I doing wrong?</p>
  
  <p>Thanks in advance!</p>