{"action":"opened","issue":{"url":"https://api.github.com/repos/Azure/mmlspark/issues/290","repository_url":"https://api.github.com/repos/Azure/mmlspark","labels_url":"https://api.github.com/repos/Azure/mmlspark/issues/290/labels{/name}","comments_url":"https://api.github.com/repos/Azure/mmlspark/issues/290/comments","events_url":"https://api.github.com/repos/Azure/mmlspark/issues/290/events","html_url":"https://github.com/Azure/mmlspark/issues/290","id":315820384,"number":290,"title":"Error training LightGBMClassifier in Spark (OS X)","user":{"login":"julioasotodv","id":20630819,"avatar_url":"https://avatars2.githubusercontent.com/u/20630819?v=4","gravatar_id":"","url":"https://api.github.com/users/julioasotodv","html_url":"https://github.com/julioasotodv","followers_url":"https://api.github.com/users/julioasotodv/followers","following_url":"https://api.github.com/users/julioasotodv/following{/other_user}","gists_url":"https://api.github.com/users/julioasotodv/gists{/gist_id}","starred_url":"https://api.github.com/users/julioasotodv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/julioasotodv/subscriptions","organizations_url":"https://api.github.com/users/julioasotodv/orgs","repos_url":"https://api.github.com/users/julioasotodv/repos","events_url":"https://api.github.com/users/julioasotodv/events{/privacy}","received_events_url":"https://api.github.com/users/julioasotodv/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-04-19T10:34:36Z","updated_at":"2018-04-19T10:34:36Z","closed_at":null,"author_association":"NONE","body":"I believe that the mmlspark jar pushed for spark packages (version 0.12) has C libraries inside, and perhaps are just .so files for Linux.\r\n\r\nTherefore, when using LightGBMClassifier on spark on OS X, the following error is raised:\r\n```\r\n18/04/19 12:21:34 WARN TaskSetManager: Lost task 2.0 in stage 4.0 (TID 41, localhost, \r\nexecutor driver): \r\njava.lang.UnsatisfiedLinkError: Could not load the native libraries \r\nbecause we encountered the following problems: no _lightgbm in java.library.path and \r\nCould not find resource /com/microsoft/ml/lightgbm/osx/x86_64/lib_lightgbm.dylib in jar\r\n```\r\nGiven that IMO the best idea is that every user builds is own LightGBM with the SWIG wrapper (just to avoid creating multiple jars for every mmlspark version and OS), wouldn't it be a better idea to put LightGBM on Apache Spark on a separate repo and add it to mmlspark as such?\r\n\r\nThat way LightGBM on Apache Spark could be built with less dependencies on any OS, while still being part of mmlspark for those that need the whole project\r\n\r\nEither that or I am doing something wrong somewhere :)"}}
{"actor":{"display_login":"julioasotodv"}}