{"action":"created","issue":{"url":"https://api.github.com/repos/dmlc/xgboost/issues/2821","repository_url":"https://api.github.com/repos/dmlc/xgboost","labels_url":"https://api.github.com/repos/dmlc/xgboost/issues/2821/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/xgboost/issues/2821/comments","events_url":"https://api.github.com/repos/dmlc/xgboost/issues/2821/events","html_url":"https://github.com/dmlc/xgboost/issues/2821","id":267669782,"number":2821,"title":"[jvm-package] spark predict cost too much time that throw a exception ","user":{"login":"hzliang","id":12317389,"avatar_url":"https://avatars2.githubusercontent.com/u/12317389?v=4","gravatar_id":"","url":"https://api.github.com/users/hzliang","html_url":"https://github.com/hzliang","followers_url":"https://api.github.com/users/hzliang/followers","following_url":"https://api.github.com/users/hzliang/following{/other_user}","gists_url":"https://api.github.com/users/hzliang/gists{/gist_id}","starred_url":"https://api.github.com/users/hzliang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hzliang/subscriptions","organizations_url":"https://api.github.com/users/hzliang/orgs","repos_url":"https://api.github.com/users/hzliang/repos","events_url":"https://api.github.com/users/hzliang/events{/privacy}","received_events_url":"https://api.github.com/users/hzliang/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-10-23T13:34:04Z","updated_at":"2018-03-21T02:25:53Z","closed_at":null,"author_association":"NONE","body":"### Spark xgboost predict 300000 data，cost too much time that SparkContex shut down\r\n '''\r\n17/10/23 21:28:40 INFO DAGScheduler: Job 2 failed: foreach at XGBTrainDist.scala:40, took 1004.596912 s\r\nException in thread \"main\" org.apache.spark.SparkException: Job 2 cancelled because SparkContext was shut down\r\n\tat org.apache.spark.scheduler.DAGScheduler$$anonfun$cleanUpAfterSchedulerStop$1.apply(DAGScheduler.scala:808)\r\n\tat org.apache.spark.scheduler.DAGScheduler$$anonfun$cleanUpAfterSchedulerStop$1.apply(DAGScheduler.scala:806)\r\n\tat scala.collection.mutable.HashSet.foreach(HashSet.scala:78)\r\n\tat org.apache.spark.scheduler.DAGScheduler.cleanUpAfterSchedulerStop(DAGScheduler.scala:806)\r\n\tat org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onStop(DAGScheduler.scala:1668)\r\n\tat org.apache.spark.util.EventLoop.stop(EventLoop.scala:83)\r\n\tat org.apache.spark.scheduler.DAGScheduler.stop(DAGScheduler.scala:1587)\r\n\tat org.apache.spark.SparkContext$$anonfun$stop$8.apply$mcV$sp(SparkContext.scala:1826)\r\n\tat org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1283)\r\n\tat org.apache.spark.SparkContext.stop(SparkContext.scala:1825)\r\n\tat org.apache.spark.SparkContext$$anonfun$2.apply$mcV$sp(SparkContext.scala:581)\r\n\tat org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:216)\r\n\tat org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ShutdownHookManager.scala:188)\r\n\tat org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:188)\r\n\tat org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:188)\r\n\tat org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1951)\r\n\tat org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply$mcV$sp(ShutdownHookManager.scala:188)\r\n\tat org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply(ShutdownHookManager.scala:188)\r\n\tat org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply(ShutdownHookManager.scala:188)\r\n\tat scala.util.Try$.apply(Try.scala:192)\r\n\tat org.apache.spark.util.SparkShutdownHookManager.runAll(ShutdownHookManager.scala:188)\r\n\tat org.apache.spark.util.SparkShutdownHookManager$$anon$2.run(ShutdownHookManager.scala:178)\r\n\tat org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)\r\n\tat org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:628)\r\n\tat org.apache.spark.SparkContext.runJob(SparkContext.scala:1918)\r\n\tat org.apache.spark.SparkContext.runJob(SparkContext.scala:1931)\r\n\tat org.apache.spark.SparkContext.runJob(SparkContext.scala:1944)\r\n\tat org.apache.spark.SparkContext.runJob(SparkContext.scala:1958)\r\n\tat org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:917)\r\n\tat org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:915)\r\n\tat org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)\r\n\tat org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)\r\n\tat org.apache.spark.rdd.RDD.withScope(RDD.scala:362)\r\n\tat org.apache.spark.rdd.RDD.foreach(RDD.scala:915)\r\n\tat XGBTrainDist$.main(XGBTrainDist.scala:40)\r\n\tat XGBTrainDist.main(XGBTrainDist.scala)\r\n'''"},"comment":{"url":"https://api.github.com/repos/dmlc/xgboost/issues/comments/374817816","html_url":"https://github.com/dmlc/xgboost/issues/2821#issuecomment-374817816","issue_url":"https://api.github.com/repos/dmlc/xgboost/issues/2821","id":374817816,"user":{"login":"jinlianch","id":4799292,"avatar_url":"https://avatars1.githubusercontent.com/u/4799292?v=4","gravatar_id":"","url":"https://api.github.com/users/jinlianch","html_url":"https://github.com/jinlianch","followers_url":"https://api.github.com/users/jinlianch/followers","following_url":"https://api.github.com/users/jinlianch/following{/other_user}","gists_url":"https://api.github.com/users/jinlianch/gists{/gist_id}","starred_url":"https://api.github.com/users/jinlianch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jinlianch/subscriptions","organizations_url":"https://api.github.com/users/jinlianch/orgs","repos_url":"https://api.github.com/users/jinlianch/repos","events_url":"https://api.github.com/users/jinlianch/events{/privacy}","received_events_url":"https://api.github.com/users/jinlianch/received_events","type":"User","site_admin":false},"created_at":"2018-03-21T02:25:53Z","updated_at":"2018-03-21T02:25:53Z","author_association":"NONE","body":"Have you found a solution for this?"}}
{"actor":{"display_login":"jinlianch"}}