{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/21416","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/21416/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/21416/comments","events_url":"https://api.github.com/repos/apache/spark/issues/21416/events","html_url":"https://github.com/apache/spark/pull/21416","id":325866805,"number":21416,"title":"[SPARK-24371] [SQL] Added isInCollection in DataFrame API for Scala and Java.","user":{"login":"dbtsai","id":1134574,"avatar_url":"https://avatars1.githubusercontent.com/u/1134574?v=4","gravatar_id":"","url":"https://api.github.com/users/dbtsai","html_url":"https://github.com/dbtsai","followers_url":"https://api.github.com/users/dbtsai/followers","following_url":"https://api.github.com/users/dbtsai/following{/other_user}","gists_url":"https://api.github.com/users/dbtsai/gists{/gist_id}","starred_url":"https://api.github.com/users/dbtsai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dbtsai/subscriptions","organizations_url":"https://api.github.com/users/dbtsai/orgs","repos_url":"https://api.github.com/users/dbtsai/repos","events_url":"https://api.github.com/users/dbtsai/events{/privacy}","received_events_url":"https://api.github.com/users/dbtsai/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":29,"created_at":"2018-05-23T20:45:31Z","updated_at":"2018-05-29T07:05:02Z","closed_at":null,"author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/21416","html_url":"https://github.com/apache/spark/pull/21416","diff_url":"https://github.com/apache/spark/pull/21416.diff","patch_url":"https://github.com/apache/spark/pull/21416.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nImplemented **`isInCollection `** in DataFrame API for both Scala and Java, so users can do\r\n\r\n```scala\r\nval profileDF = Seq(\r\n  Some(1), Some(2), Some(3), Some(4),\r\n  Some(5), Some(6), Some(7), None\r\n).toDF(\"profileID\")\r\n\r\nval validUsers: Seq[Any] = Seq(6, 7.toShort, 8L, \"3\")\r\n\r\nval result = profileDF.withColumn(\"isValid\", $\"profileID\". isInCollection(validUsers))\r\n\r\nresult.show(10)\r\n\"\"\"\r\n+---------+-------+\r\n|profileID|isValid|\r\n+---------+-------+\r\n|        1|  false|\r\n|        2|  false|\r\n|        3|   true|\r\n|        4|  false|\r\n|        5|  false|\r\n|        6|   true|\r\n|        7|   true|\r\n|     null|   null|\r\n+---------+-------+\r\n \"\"\".stripMargin\r\n```\r\n## How was this patch tested?\r\n\r\nSeveral unit tests are added."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/392673840","html_url":"https://github.com/apache/spark/pull/21416#issuecomment-392673840","issue_url":"https://api.github.com/repos/apache/spark/issues/21416","id":392673840,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars1.githubusercontent.com/u/8108735?v=4","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2018-05-29T07:05:02Z","updated_at":"2018-05-29T07:05:02Z","author_association":"NONE","body":"**[Test build #91242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91242/testReport)** for PR 21416 at commit [`fed2846`](https://github.com/apache/spark/commit/fed2846fe7c9ca2cb4534b23803cd29d5a18d4f9).\n * This patch **fails due to an unknown error code, -9**.\n * This patch merges cleanly.\n * This patch adds no public classes."}}
{"actor":{"display_login":"SparkQA"}}