{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/15214","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/15214/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/15214/comments","events_url":"https://api.github.com/repos/apache/spark/issues/15214/events","html_url":"https://github.com/apache/spark/pull/15214","id":178859490,"number":15214,"title":"[SPARK-17017][Follow-up][ML] Refactor of ChiSqSelector and add ML Python API.","user":{"login":"yanboliang","id":1962026,"avatar_url":"https://avatars.githubusercontent.com/u/1962026?v=3","gravatar_id":"","url":"https://api.github.com/users/yanboliang","html_url":"https://github.com/yanboliang","followers_url":"https://api.github.com/users/yanboliang/followers","following_url":"https://api.github.com/users/yanboliang/following{/other_user}","gists_url":"https://api.github.com/users/yanboliang/gists{/gist_id}","starred_url":"https://api.github.com/users/yanboliang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yanboliang/subscriptions","organizations_url":"https://api.github.com/users/yanboliang/orgs","repos_url":"https://api.github.com/users/yanboliang/repos","events_url":"https://api.github.com/users/yanboliang/events{/privacy}","received_events_url":"https://api.github.com/users/yanboliang/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-09-23T12:03:53Z","updated_at":"2016-09-23T13:03:30Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/15214","html_url":"https://github.com/apache/spark/pull/15214","diff_url":"https://github.com/apache/spark/pull/15214.diff","patch_url":"https://github.com/apache/spark/pull/15214.patch"},"body":"## What changes were proposed in this pull request?\r\n#14597 modified ```ChiSqSelector``` to support ```fpr``` type selector, however, it left some issue need to be addressed:\r\n* We should allow users to set selector type explicitly rather than switching them by using different setting function, since the setting order will involves some unexpected issue. For example, if users both set ```numTopFeatures``` and ```percentile```, it will train ```kbest``` or ```percentile``` model based on the order of setting (the latter setting one will be trained). This make users confused, and we should allow users to set selector type explicitly. We handle similar issues at other place of ML code base such as ```GeneralizedLinearRegression``` and ```LogisticRegression```.\r\n* Meanwhile, if there are more than one parameter except ```alpha``` can be set for ```fpr``` model, we can not handle it elegantly in the existing framework. And similar issues for ```kbest``` and ```percentile``` model. Setting selector type explicitly can solve this issue also.\r\n* If setting selector type explicitly by users is allowed, we should handle param interaction such as if users set ```selectorType = percentile``` and ```alpha = 0.1```, we should notify users the parameter ```alpha``` will take no effect. We should handle complex parameter interaction checks at ```transformSchema```. (FYI #11620)\r\n* We should use lower case of the selector type names to follow MLlib convention.\r\n* Add ML Python API.\r\n\r\n## How was this patch tested?\r\nUnit test.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/249186447","html_url":"https://github.com/apache/spark/pull/15214#issuecomment-249186447","issue_url":"https://api.github.com/repos/apache/spark/issues/15214","id":249186447,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2016-09-23T13:03:30Z","updated_at":"2016-09-23T13:03:30Z","body":"Merged build finished. Test PASSed."}}
{"actor":{"display_login":"AmplabJenkins"}}