{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/15432","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/15432/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/15432/comments","events_url":"https://api.github.com/repos/apache/spark/issues/15432/events","html_url":"https://github.com/apache/spark/pull/15432","id":182216651,"number":15432,"title":"[SPARK-17854][SQL] rand/randn allows null/long as input seed","user":{"login":"HyukjinKwon","id":6477701,"avatar_url":"https://avatars.githubusercontent.com/u/6477701?v=3","gravatar_id":"","url":"https://api.github.com/users/HyukjinKwon","html_url":"https://github.com/HyukjinKwon","followers_url":"https://api.github.com/users/HyukjinKwon/followers","following_url":"https://api.github.com/users/HyukjinKwon/following{/other_user}","gists_url":"https://api.github.com/users/HyukjinKwon/gists{/gist_id}","starred_url":"https://api.github.com/users/HyukjinKwon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HyukjinKwon/subscriptions","organizations_url":"https://api.github.com/users/HyukjinKwon/orgs","repos_url":"https://api.github.com/users/HyukjinKwon/repos","events_url":"https://api.github.com/users/HyukjinKwon/events{/privacy}","received_events_url":"https://api.github.com/users/HyukjinKwon/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2016-10-11T09:29:02Z","updated_at":"2016-10-12T14:01:33Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/15432","html_url":"https://github.com/apache/spark/pull/15432","diff_url":"https://github.com/apache/spark/pull/15432.diff","patch_url":"https://github.com/apache/spark/pull/15432.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis PR proposes `rand`/`randn` accept `null` as input in Scala/SQL and `LongType` as input in SQL. In this case, it treats the values as `0`.\r\n\r\nSo, this PR includes both changes below:\r\n\r\n- `null` support\r\n\r\n  It seems MySQL also accepts this.\r\n\r\n\r\n  ```sql\r\nmysql> select rand(0);\r\n+---------------------+\r\n| rand(0)             |\r\n+---------------------+\r\n| 0.15522042769493574 |\r\n+---------------------+\r\n1 row in set (0.00 sec)\r\n\r\nmysql> select rand(NULL);\r\n+---------------------+\r\n| rand(NULL)          |\r\n+---------------------+\r\n| 0.15522042769493574 |\r\n+---------------------+\r\n1 row in set (0.00 sec)\r\n```\r\n\r\n  and also Hive does according to [HIVE-14694](https://issues.apache.org/jira/browse/HIVE-14694)\r\n\r\n  So the codes below:\r\n\r\n  ```scala\r\n  spark.range(1).selectExpr(\"rand(null)\").show()\r\n  ```\r\n\r\n  prints..\r\n\r\n  **Before**\r\n\r\n    ```\r\n    Input argument to rand must be an integer literal.;; line 1 pos 0\r\norg.apache.spark.sql.AnalysisException: Input argument to rand must be an integer literal.;; line 1 pos 0\r\n\tat org.apache.spark.sql.catalyst.analysis.FunctionRegistry$$anonfun$5.apply(FunctionRegistry.scala:465)\r\n\tat org.apache.spark.sql.catalyst.analysis.FunctionRegistry$$anonfun$5.apply(FunctionRegistry.scala:444)\r\n```\r\n\r\n  **After**\r\n\r\n    ```\r\n    +-----------------------+\r\n    |rand(CAST(NULL AS INT))|\r\n    +-----------------------+\r\n    |    0.13385709732307427|\r\n    +-----------------------+\r\n    ```\r\n\r\n- `LongType` support in SQL.\r\n\r\n  In addition, it make the function allows to take `LongType` consistently within Scala/SQL.\r\n\r\n  In more details, the codes below:\r\n\r\n  ```scala\r\nspark.range(1).select(rand(1), rand(1L)).show()\r\nspark.range(1).selectExpr(\"rand(1)\", \"rand(1L)\").show()\r\n```\r\n\r\n  prints..\r\n\r\n  **Before**\r\n\r\n    ```\r\n+------------------+------------------+\r\n|           rand(1)|           rand(1)|\r\n+------------------+------------------+\r\n|0.2630967864682161|0.2630967864682161|\r\n+------------------+------------------+\r\n\r\n\r\nInput argument to rand must be an integer literal.;; line 1 pos 0\r\norg.apache.spark.sql.AnalysisException: Input argument to rand must be an integer literal.;; line 1 pos 0\r\n\tat org.apache.spark.sql.catalyst.analysis.FunctionRegistry$$anonfun$5.apply(FunctionRegistry.scala:465)\r\n\tat\r\n```\r\n\r\n  **After**\r\n\r\n    ```\r\n+------------------+------------------+\r\n|           rand(1)|           rand(1)|\r\n+------------------+------------------+\r\n|0.2630967864682161|0.2630967864682161|\r\n+------------------+------------------+\r\n\r\n+------------------+------------------+\r\n|           rand(1)|           rand(1)|\r\n+------------------+------------------+\r\n|0.2630967864682161|0.2630967864682161|\r\n+------------------+------------------+\r\n```\r\n\r\n\r\n## How was this patch tested?\r\n\r\nUnit tests in `DataFrameSuite.scala` and `RandomSuite.scala`.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/253221795","html_url":"https://github.com/apache/spark/pull/15432#issuecomment-253221795","issue_url":"https://api.github.com/repos/apache/spark/issues/15432","id":253221795,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-10-12T14:01:33Z","updated_at":"2016-10-12T14:01:33Z","body":"**[Test build #66815 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66815/consoleFull)** for PR 15432 at commit [`6f8f3f3`](https://github.com/apache/spark/commit/6f8f3f33f9b67d77285048bfd7d794990e072b8a).\n * This patch **fails Spark unit tests**.\n * This patch merges cleanly.\n * This patch adds the following public classes _(experimental)_:\n  * `abstract class RDG extends UnaryExpression with ExpectsInputTypes with Nondeterministic `\n  * `case class Rand(child: Expression) extends RDG `\n  * `case class Randn(child: Expression) extends RDG `"}}
{"actor":{"display_login":"SparkQA"}}