{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/15261","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/15261/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/15261/comments","events_url":"https://api.github.com/repos/apache/spark/issues/15261/events","html_url":"https://github.com/apache/spark/pull/15261","id":179464704,"number":15261,"title":"[SPARK-16356][Follow-up][ML] Enforce ML test of exception for local/distributed Dataset.","user":{"login":"yanboliang","id":1962026,"avatar_url":"https://avatars.githubusercontent.com/u/1962026?v=3","gravatar_id":"","url":"https://api.github.com/users/yanboliang","html_url":"https://github.com/yanboliang","followers_url":"https://api.github.com/users/yanboliang/followers","following_url":"https://api.github.com/users/yanboliang/following{/other_user}","gists_url":"https://api.github.com/users/yanboliang/gists{/gist_id}","starred_url":"https://api.github.com/users/yanboliang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yanboliang/subscriptions","organizations_url":"https://api.github.com/users/yanboliang/orgs","repos_url":"https://api.github.com/users/yanboliang/repos","events_url":"https://api.github.com/users/yanboliang/events{/privacy}","received_events_url":"https://api.github.com/users/yanboliang/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2016-09-27T11:15:17Z","updated_at":"2016-09-29T07:53:54Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/15261","html_url":"https://github.com/apache/spark/pull/15261","diff_url":"https://github.com/apache/spark/pull/15261.diff","patch_url":"https://github.com/apache/spark/pull/15261.patch"},"body":"## What changes were proposed in this pull request?\r\n#14035 added ```testImplicits``` to ML unit tests and promoted ```toDF()```, but left one minor issue at ```VectorIndexerSuite```. If we create the DataFrame by ```Seq(...).toDF()```, it will throw different error/exception compared with ```sc.parallelize(Seq(...)).toDF()``` for one of the test cases.\r\nAfter in-depth study, I found it was caused by different behavior of local and distributed Dataset if the UDF failed at ```assert```. If the data is local Dataset, it throws ```AssertionError``` directly; If the data is distributed Dataset, it throws ```SparkException``` which is the wrapper of ```AssertionError```. I think we should enforce this test to cover both case.\r\n\r\n## How was this patch tested?\r\nUnit test.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/250396400","html_url":"https://github.com/apache/spark/pull/15261#issuecomment-250396400","issue_url":"https://api.github.com/repos/apache/spark/issues/15261","id":250396400,"user":{"login":"yanboliang","id":1962026,"avatar_url":"https://avatars.githubusercontent.com/u/1962026?v=3","gravatar_id":"","url":"https://api.github.com/users/yanboliang","html_url":"https://github.com/yanboliang","followers_url":"https://api.github.com/users/yanboliang/followers","following_url":"https://api.github.com/users/yanboliang/following{/other_user}","gists_url":"https://api.github.com/users/yanboliang/gists{/gist_id}","starred_url":"https://api.github.com/users/yanboliang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yanboliang/subscriptions","organizations_url":"https://api.github.com/users/yanboliang/orgs","repos_url":"https://api.github.com/users/yanboliang/repos","events_url":"https://api.github.com/users/yanboliang/events{/privacy}","received_events_url":"https://api.github.com/users/yanboliang/received_events","type":"User","site_admin":false},"created_at":"2016-09-29T07:53:54Z","updated_at":"2016-09-29T07:53:54Z","body":"Merged into master, thanks for review."}}
{"actor":{"display_login":"yanboliang"}}