title: spark executor lost failure
body: <p>I am using the databricks spark cluster (AWS), and testing on my scala experiment.
  I have some issue when training on a 10 GB data with LogisticRegressionWithLBFGS algorithm.
  The code block where I met the issue is as follows:</p>
  
  <pre><code>import org.apache.spark.mllib.classification.LogisticRegressionWithLBFGS
  val algorithm = new LogisticRegressionWithLBFGS()
  algorithm.run(training_set)
  </code></pre>
  
  <p>First I got a lot executor lost failure and java out of memory issues, then I repartitioned my training_set with more partitions and the out of memory issues are gone, but Still get executor lost failure.</p>
  
  <p>My cluster has 72 cores and 500GB ram in total.
  Can any one give some idea on this?</p>