title: Large task size for simplest program
body: <p>I am trying to run the simplest program with Spark</p>
  
  <pre><code>import org.apache.spark.{SparkContext, SparkConf}
  
  object LargeTaskTest {
  
    def main(args: Array[String]) {
      val conf = new SparkConf().setAppName("DataTest").setMaster("local[*]")
      val sc = new SparkContext(conf)
      val dat = (1 to 10000000).toList
      val data = sc.parallelize(dat).cache()
      for(i &lt;- 1 to 100){
        println(data.reduce(_ + _))
      }
    }   
  }
  </code></pre>
  
  <p>I get the following error message, after each iteration :  </p>
  
  <blockquote>
    <p>WARN TaskSetManager: Stage 0 contains a task of very large size (9767
    KB). The maximum recommended task size is 100 KB.</p>
  </blockquote>
  
  <p>Increasing the data size increases said task size. This suggests to me that the driver is shipping the "dat" object to all executors, but I can't for the life of me see why, as the only operation on my RDD is reduce, which basically has no closure. Any ideas ? </p>