{"action":"closed","issue":{"url":"https://api.github.com/repos/dmlc/xgboost/issues/1635","repository_url":"https://api.github.com/repos/dmlc/xgboost","labels_url":"https://api.github.com/repos/dmlc/xgboost/issues/1635/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/xgboost/issues/1635/comments","events_url":"https://api.github.com/repos/dmlc/xgboost/issues/1635/events","html_url":"https://github.com/dmlc/xgboost/issues/1635","id":180864460,"number":1635,"title":"[jvm-packages] XGBoost in spark-shell: SIGSEGV at libjvm.so","user":{"login":"tcarette","id":2481066,"avatar_url":"https://avatars.githubusercontent.com/u/2481066?v=3","gravatar_id":"","url":"https://api.github.com/users/tcarette","html_url":"https://github.com/tcarette","followers_url":"https://api.github.com/users/tcarette/followers","following_url":"https://api.github.com/users/tcarette/following{/other_user}","gists_url":"https://api.github.com/users/tcarette/gists{/gist_id}","starred_url":"https://api.github.com/users/tcarette/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tcarette/subscriptions","organizations_url":"https://api.github.com/users/tcarette/orgs","repos_url":"https://api.github.com/users/tcarette/repos","events_url":"https://api.github.com/users/tcarette/events{/privacy}","received_events_url":"https://api.github.com/users/tcarette/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-10-04T11:09:07Z","updated_at":"2016-10-05T13:44:46Z","closed_at":"2016-10-05T13:44:46Z","body":"I have the same problem as in issue #1061 at XGBoost.train on a dataset of 12k (1% of my full dataset for the test) rows with about 4500 features. But the fix doesn't seem to work for me: I start my spark-shell (v2.0.0) in local mode on a 16Gb ram MacBook Pro with\r\n\r\nspark-shell --jars xgboost4j-spark-0.7-jar-with-dependencies.jar --master local[*] --driver-memory 12G --conf \"spark.serializer=org.apache.spark.serializer.KryoSerializer\" --conf \"spark.kryo.classesToRegister=ml.dmlc.xgboost4j.java.Booster\"\r\n\r\nHere is the way I call XGBoost.train (it's just a test, so I didn't really pay attention to the options)\r\n```\r\nval data = MLUtils.loadLibSVMFile(sc,outputFolder + modelFolder + \"training\")\r\n  .sample(withReplacement=false,fraction=0.01)\r\n  .persist()\r\n\r\nval paramMap = Map(\r\n  \"eta\" -> 0.1f,\r\n  \"max_depth\" -> 2,\r\n  \"objective\" -> \"binary:logistic\",\r\n  \"eval_metric\" -> \"merror\")\r\n\r\nval model = XGBoost.train(\r\n  data,\r\n  paramMap,\r\n  round = 5,\r\n  nWorkers = 8,\r\n  useExternalMemory = true)      \r\n```             \r\n\r\nThe jar is built on commit 37bc122c90b7c1337403f797fb7b01fc6abd5b36. \r\n\r\nHere is the error\r\n\r\n```\r\n#\r\n# A fatal error has been detected by the Java Runtime Environment:\r\n#\r\n#  SIGSEGV (0xb) at pc=0x000000010e17cacc, pid=61751, tid=43267\r\n#\r\n# JRE version: Java(TM) SE Runtime Environment (8.0_91-b14) (build 1.8.0_91-b14)\r\n# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.91-b14 mixed mode bsd-amd64 compressed oops)\r\n# Problematic frame:\r\n# V  [libjvm.dylib+0x57cacc]  UTF8::unicode_length(char const*)+0x4\r\n[hs_err_pid61751.log.zip](https://github.com/dmlc/xgboost/files/508252/hs_err_pid61751.log.zip)\r\n\r\n\r\n```\r\n\r\nlogs: [hs_err_pid61751.log.zip](https://github.com/dmlc/xgboost/files/508264/hs_err_pid61751.log.zip)\r\n\r\n\r\nFor the sake of completeness (but I guess it's unrelated), I have a lot of warnings of the type\r\n```\r\n16/10/04 12:50:33 WARN BlockManager: Putting block rdd_4_183 failed\r\n16/10/04 12:50:33 WARN MemoryStore: Not enough space to cache rdd_4_182 in memory! (computed 7.1 MB so far)\r\n```\r\nwhen loading the data and running the model."}}
{"actor":{"display_login":"CodingCat"}}