{"action":"opened","issue":{"url":"https://api.github.com/repos/dmlc/xgboost/issues/1053","repository_url":"https://api.github.com/repos/dmlc/xgboost","labels_url":"https://api.github.com/repos/dmlc/xgboost/issues/1053/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/xgboost/issues/1053/comments","events_url":"https://api.github.com/repos/dmlc/xgboost/issues/1053/events","html_url":"https://github.com/dmlc/xgboost/issues/1053","id":143974807,"number":1053,"title":"Spark 1.4.1 java.lang.NoSuchMethodError: org.apache.spark.TaskContext$.getPartitionId()I","user":{"login":"edi-bice","id":1819450,"avatar_url":"https://avatars.githubusercontent.com/u/1819450?v=3","gravatar_id":"","url":"https://api.github.com/users/edi-bice","html_url":"https://github.com/edi-bice","followers_url":"https://api.github.com/users/edi-bice/followers","following_url":"https://api.github.com/users/edi-bice/following{/other_user}","gists_url":"https://api.github.com/users/edi-bice/gists{/gist_id}","starred_url":"https://api.github.com/users/edi-bice/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edi-bice/subscriptions","organizations_url":"https://api.github.com/users/edi-bice/orgs","repos_url":"https://api.github.com/users/edi-bice/repos","events_url":"https://api.github.com/users/edi-bice/events{/privacy}","received_events_url":"https://api.github.com/users/edi-bice/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-03-28T13:13:12Z","updated_at":"2016-03-28T13:13:12Z","closed_at":null,"body":"I'm running Spark 1.4.1 shell as follows\r\n\r\nspark-shell --master yarn-client --jars xgboost-latest/jvm-packages/xgboost4j-spark/target/xgboost4j-spark-0.5-jar-with-dependencies.jar\r\n\r\n... and \r\n\r\nsc.getConf.registerKryoClasses(Array(classOf[Booster]))\r\nval model = XGBoost.train(training, paramMap, 10, nWorkers = 9)\r\n\r\nfails with following stacktrace:\r\n\r\n16/03/25 20:23:10 WARN TaskSetManager: Lost task 0.0 in stage 2.0 (TID 471, lvsdevfrsa06.gspt.net): java.lang.NoSuchMethodError: org.apache.spark.TaskContext$.getPartitionId()I\r\n        at ml.dmlc.xgboost4j.scala.spark.XGBoost$$anonfun$buildDistributedBoosters$1.apply(XGBoost.scala:59)\r\n        at ml.dmlc.xgboost4j.scala.spark.XGBoost$$anonfun$buildDistributedBoosters$1.apply(XGBoost.scala:58)\r\n        at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$17.apply(RDD.scala:686)\r\n        at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$17.apply(RDD.scala:686)\r\n        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)\r\n        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)\r\n        at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:69)\r\n        at org.apache.spark.rdd.RDD.iterator(RDD.scala:242)\r\n        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)\r\n        at org.apache.spark.scheduler.Task.run(Task.scala:70)\r\n        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)\r\n        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n        at java.lang.Thread.run(Thread.java:745)"}}