{"action":"opened","issue":{"url":"https://api.github.com/repos/intel-analytics/BigDL/issues/2628","repository_url":"https://api.github.com/repos/intel-analytics/BigDL","labels_url":"https://api.github.com/repos/intel-analytics/BigDL/issues/2628/labels{/name}","comments_url":"https://api.github.com/repos/intel-analytics/BigDL/issues/2628/comments","events_url":"https://api.github.com/repos/intel-analytics/BigDL/issues/2628/events","html_url":"https://github.com/intel-analytics/BigDL/issues/2628","id":358874881,"node_id":"MDU6SXNzdWUzNTg4NzQ4ODE=","number":2628,"title":"BigDL compatibility issue on CDH","user":{"login":"ZeweiChen11","id":17589160,"node_id":"MDQ6VXNlcjE3NTg5MTYw","avatar_url":"https://avatars2.githubusercontent.com/u/17589160?v=4","gravatar_id":"","url":"https://api.github.com/users/ZeweiChen11","html_url":"https://github.com/ZeweiChen11","followers_url":"https://api.github.com/users/ZeweiChen11/followers","following_url":"https://api.github.com/users/ZeweiChen11/following{/other_user}","gists_url":"https://api.github.com/users/ZeweiChen11/gists{/gist_id}","starred_url":"https://api.github.com/users/ZeweiChen11/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ZeweiChen11/subscriptions","organizations_url":"https://api.github.com/users/ZeweiChen11/orgs","repos_url":"https://api.github.com/users/ZeweiChen11/repos","events_url":"https://api.github.com/users/ZeweiChen11/events{/privacy}","received_events_url":"https://api.github.com/users/ZeweiChen11/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-09-11T03:40:51Z","updated_at":"2018-09-11T03:40:51Z","closed_at":null,"author_association":"NONE","body":"SW stack: BigDL 0.6, cdh 6.0\r\nRunning command:\r\n```\r\nspark-submit \\\r\n--master yarn \\\r\n--deploy-mode client \\\r\n--conf \"spark.yarn.executor.memoryOverhead=60G\" \\\r\n--conf \"spark.blacklist.enabled=false\" \\\r\n--conf \"spark.dynamicAllocation.enabled=false\" \\\r\n--executor-cores 36  \\\r\n--num-executors 4 \\\r\n--driver-memory 60g \\\r\n--executor-memory 180G \\\r\n--driver-class-path /root/BigDL-0.6.0/dist/lib/bigdl-*-jar-with-dependencies.jar \\\r\n--class com.intel.analytics.bigdl.models.inception.TrainInceptionV1 \\\r\n/root/BigDL-0.6.0/dist/lib/bigdl-*-jar-with-dependencies.jar \\\r\n--batchSize  1152 \\\r\n--learningRate 0.0896 \\\r\n-f hdfs:///user/root/sequence \\\r\n--maxEpoch 3 \\\r\n--checkpoint /mnt/disk2/ckp/inceptionv1\r\n\r\n```\r\n\r\nNPE Error \r\n```\r\n2018-09-10 23:21:09 INFO  DistriOptimizer$:672 - Cache thread models...\r\n2018-09-10 23:21:09 ERROR TaskSetManager:70 - Task 1 in stage 3.0 failed 1 times; aborting job\r\nException in thread \"main\" org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 3.0 failed 1 times, most recent failure: Lost task 1.0 in stage 3.0 (TID 152, hsw-node9, executor 1): java.lang.NullPointerException\r\n        at com.intel.analytics.bigdl.models.utils.ModelBroadcast.value(ModelBroadcast.scala:120)\r\n        at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$13$$anonfun$14.apply(DistriOptimizer.scala:643)\r\n        at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$13$$anonfun$14.apply(DistriOptimizer.scala:642)\r\n        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)\r\n        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)\r\n        at scala.collection.immutable.Range.foreach(Range.scala:160)\r\n        at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)\r\n        at scala.collection.AbstractTraversable.map(Traversable.scala:104)\r\n        at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$13.apply(DistriOptimizer.scala:642)\r\n        at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$13.apply(DistriOptimizer.scala:625)\r\n```"}}
{"actor":{"display_login":"ZeweiChen11"}}