{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/11119","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/11119/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/11119/comments","events_url":"https://api.github.com/repos/apache/spark/issues/11119/events","html_url":"https://github.com/apache/spark/pull/11119","id":132245928,"number":11119,"title":"[SPARK-10780][ML][WIP] Add initial model to kmeans","user":{"login":"yinxusen","id":2637239,"avatar_url":"https://avatars.githubusercontent.com/u/2637239?v=3","gravatar_id":"","url":"https://api.github.com/users/yinxusen","html_url":"https://github.com/yinxusen","followers_url":"https://api.github.com/users/yinxusen/followers","following_url":"https://api.github.com/users/yinxusen/following{/other_user}","gists_url":"https://api.github.com/users/yinxusen/gists{/gist_id}","starred_url":"https://api.github.com/users/yinxusen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yinxusen/subscriptions","organizations_url":"https://api.github.com/users/yinxusen/orgs","repos_url":"https://api.github.com/users/yinxusen/repos","events_url":"https://api.github.com/users/yinxusen/events{/privacy}","received_events_url":"https://api.github.com/users/yinxusen/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2016-02-08T20:16:46Z","updated_at":"2016-02-22T20:13:43Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/11119","html_url":"https://github.com/apache/spark/pull/11119","diff_url":"https://github.com/apache/spark/pull/11119.diff","patch_url":"https://github.com/apache/spark/pull/11119.patch"},"body":"https://issues.apache.org/jira/browse/SPARK-10780\r\n\r\nI mark it as WIP because there are several issues need to discuss:\r\n\r\n1. Codegen for shared params doesn't support general type. But IMHO there is no need to rewrite the codegen to support general type params, for now. So I add an `extras` in `SharedParamsCodeGen` to support this kind of exception. We may modify the codegen later if there are more general type shared parameters.\r\n\r\n2. Since we add `Model` in parameters of every `Estimator`, the save/load of estimators should be modified to support model persistence, refer to https://github.com/apache/spark/pull/9971. I ignore it in this version of the code because the save/load works well only if we don't set `initialModel`. I'll add save/load for the new parameter after the discussion.\r\n\r\n3. I don't add model check (e.g. number of centers equals to k, the dimension of centers equals to data points') here because the inner `MLlibKMeans` checks it for us. For those estimators whose MLlib companions don't provide warm-starts, we need to check the initial model in ML package.\r\n\r\n4. I add three different setters for `KMeans`. The first one uses for collaboration with model save/load, the second one is for accepting generalized model, and the third one is more user-friendly."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/187365420","html_url":"https://github.com/apache/spark/pull/11119#issuecomment-187365420","issue_url":"https://api.github.com/repos/apache/spark/issues/11119","id":187365420,"user":{"login":"yinxusen","id":2637239,"avatar_url":"https://avatars.githubusercontent.com/u/2637239?v=3","gravatar_id":"","url":"https://api.github.com/users/yinxusen","html_url":"https://github.com/yinxusen","followers_url":"https://api.github.com/users/yinxusen/followers","following_url":"https://api.github.com/users/yinxusen/following{/other_user}","gists_url":"https://api.github.com/users/yinxusen/gists{/gist_id}","starred_url":"https://api.github.com/users/yinxusen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yinxusen/subscriptions","organizations_url":"https://api.github.com/users/yinxusen/orgs","repos_url":"https://api.github.com/users/yinxusen/repos","events_url":"https://api.github.com/users/yinxusen/events{/privacy}","received_events_url":"https://api.github.com/users/yinxusen/received_events","type":"User","site_admin":false},"created_at":"2016-02-22T20:13:43Z","updated_at":"2016-02-22T20:13:43Z","body":"Ping @dbtsai Coming back? :)"}}