{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/15314","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/15314/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/15314/comments","events_url":"https://api.github.com/repos/apache/spark/issues/15314/events","html_url":"https://github.com/apache/spark/pull/15314","id":180314464,"number":15314,"title":"[SPARK-17747][ML] WeightCol support non-double datetypes","user":{"login":"zhengruifeng","id":7322292,"avatar_url":"https://avatars.githubusercontent.com/u/7322292?v=3","gravatar_id":"","url":"https://api.github.com/users/zhengruifeng","html_url":"https://github.com/zhengruifeng","followers_url":"https://api.github.com/users/zhengruifeng/followers","following_url":"https://api.github.com/users/zhengruifeng/following{/other_user}","gists_url":"https://api.github.com/users/zhengruifeng/gists{/gist_id}","starred_url":"https://api.github.com/users/zhengruifeng/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhengruifeng/subscriptions","organizations_url":"https://api.github.com/users/zhengruifeng/orgs","repos_url":"https://api.github.com/users/zhengruifeng/repos","events_url":"https://api.github.com/users/zhengruifeng/events{/privacy}","received_events_url":"https://api.github.com/users/zhengruifeng/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2016-09-30T14:14:30Z","updated_at":"2016-10-03T21:27:41Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/15314","html_url":"https://github.com/apache/spark/pull/15314","diff_url":"https://github.com/apache/spark/pull/15314.diff","patch_url":"https://github.com/apache/spark/pull/15314.patch"},"body":"## What changes were proposed in this pull request?\r\nWeightCol support non-double datetypes\r\n\r\n\r\n## How was this patch tested?\r\nlocal tests in spark-shell and unit tests\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/251232501","html_url":"https://github.com/apache/spark/pull/15314#issuecomment-251232501","issue_url":"https://api.github.com/repos/apache/spark/issues/15314","id":251232501,"user":{"login":"sethah","id":7275795,"avatar_url":"https://avatars.githubusercontent.com/u/7275795?v=3","gravatar_id":"","url":"https://api.github.com/users/sethah","html_url":"https://github.com/sethah","followers_url":"https://api.github.com/users/sethah/followers","following_url":"https://api.github.com/users/sethah/following{/other_user}","gists_url":"https://api.github.com/users/sethah/gists{/gist_id}","starred_url":"https://api.github.com/users/sethah/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sethah/subscriptions","organizations_url":"https://api.github.com/users/sethah/orgs","repos_url":"https://api.github.com/users/sethah/repos","events_url":"https://api.github.com/users/sethah/events{/privacy}","received_events_url":"https://api.github.com/users/sethah/received_events","type":"User","site_admin":false},"created_at":"2016-10-03T21:27:41Z","updated_at":"2016-10-03T21:27:41Z","body":"@zhengruifeng From a testing perspective, if we are going to accept arbitrary numeric column types, then we should have unit tests which validate that _all_ of the numeric types are handled correctly, not just `Int` and `Long`. Right now, there is no quick way I can see to implement this. Ideally, we'd write a helper method that we can easily apply in all the test suites, accepting arbitrary estimators with `HasWeightCol` trait. I'm hesitant to suggest that here, since we really need an exhaustive helper test that we can apply to all aspects of instance weighting in ML algos. The other option is to test all numeric types inside every test suite that has instance weighting, but this is going to duplicate a lot of code. [I've created a Jira](https://issues.apache.org/jira/browse/SPARK-17772) for weighted instance testing, we can either block this PR until its implemented, or put in some temporary tests for now. Thoughts?"}}
{"actor":{"display_login":"sethah"}}