{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/15277","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/15277/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/15277/comments","events_url":"https://api.github.com/repos/apache/spark/issues/15277/events","html_url":"https://github.com/apache/spark/pull/15277","id":179706781,"number":15277,"title":"[SPARK-17704][ML][MLlib] ChiSqSelector performance improvement.","user":{"login":"yanboliang","id":1962026,"avatar_url":"https://avatars.githubusercontent.com/u/1962026?v=3","gravatar_id":"","url":"https://api.github.com/users/yanboliang","html_url":"https://github.com/yanboliang","followers_url":"https://api.github.com/users/yanboliang/followers","following_url":"https://api.github.com/users/yanboliang/following{/other_user}","gists_url":"https://api.github.com/users/yanboliang/gists{/gist_id}","starred_url":"https://api.github.com/users/yanboliang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yanboliang/subscriptions","organizations_url":"https://api.github.com/users/yanboliang/orgs","repos_url":"https://api.github.com/users/yanboliang/repos","events_url":"https://api.github.com/users/yanboliang/events{/privacy}","received_events_url":"https://api.github.com/users/yanboliang/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2016-09-28T08:10:38Z","updated_at":"2016-09-29T11:49:09Z","closed_at":"2016-09-29T11:33:06Z","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/15277","html_url":"https://github.com/apache/spark/pull/15277","diff_url":"https://github.com/apache/spark/pull/15277.diff","patch_url":"https://github.com/apache/spark/pull/15277.patch"},"body":"## What changes were proposed in this pull request?\r\nSeveral performance improvement for ```ChiSqSelector```:\r\n1, Keep ```selectedFeatures``` ordered ascendent.\r\n```ChiSqSelectorModel.transform``` need ```selectedFeatures``` ordered to make prediction. We should sort it when training model rather than making prediction, since users usually train model once and use the model to do prediction multiple times. \r\n2, When training ```fpr``` type ```ChiSqSelectorModel```, it's not necessary to sort the ChiSq test result by statistic.\r\n\r\n## How was this patch tested?\r\nExisting unit tests.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/250444446","html_url":"https://github.com/apache/spark/pull/15277#issuecomment-250444446","issue_url":"https://api.github.com/repos/apache/spark/issues/15277","id":250444446,"user":{"login":"srowen","id":822522,"avatar_url":"https://avatars.githubusercontent.com/u/822522?v=3","gravatar_id":"","url":"https://api.github.com/users/srowen","html_url":"https://github.com/srowen","followers_url":"https://api.github.com/users/srowen/followers","following_url":"https://api.github.com/users/srowen/following{/other_user}","gists_url":"https://api.github.com/users/srowen/gists{/gist_id}","starred_url":"https://api.github.com/users/srowen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/srowen/subscriptions","organizations_url":"https://api.github.com/users/srowen/orgs","repos_url":"https://api.github.com/users/srowen/repos","events_url":"https://api.github.com/users/srowen/events{/privacy}","received_events_url":"https://api.github.com/users/srowen/received_events","type":"User","site_admin":false},"created_at":"2016-09-29T11:49:08Z","updated_at":"2016-09-29T11:49:08Z","body":"OK thank you. I'd like to keep the ability to accept unsorted indices, because it really isn't something the model needs to demand of its input. It's an implementation detail. You can just sort and store the indices.\r\n\r\nOptionally deprecating isSorted sounds good to me, but not essential. I don't see why anyone would use it though."}}
{"actor":{"display_login":"srowen"}}