{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/15677","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/15677/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/15677/comments","events_url":"https://api.github.com/repos/apache/spark/issues/15677/events","html_url":"https://github.com/apache/spark/pull/15677","id":186048836,"number":15677,"title":"[SPARK-17963][SQL][Documentation] Add examples (extend) in each expression and improve documentation","user":{"login":"HyukjinKwon","id":6477701,"avatar_url":"https://avatars.githubusercontent.com/u/6477701?v=3","gravatar_id":"","url":"https://api.github.com/users/HyukjinKwon","html_url":"https://github.com/HyukjinKwon","followers_url":"https://api.github.com/users/HyukjinKwon/followers","following_url":"https://api.github.com/users/HyukjinKwon/following{/other_user}","gists_url":"https://api.github.com/users/HyukjinKwon/gists{/gist_id}","starred_url":"https://api.github.com/users/HyukjinKwon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HyukjinKwon/subscriptions","organizations_url":"https://api.github.com/users/HyukjinKwon/orgs","repos_url":"https://api.github.com/users/HyukjinKwon/repos","events_url":"https://api.github.com/users/HyukjinKwon/events{/privacy}","received_events_url":"https://api.github.com/users/HyukjinKwon/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":54,"created_at":"2016-10-29T00:36:24Z","updated_at":"2016-11-02T23:22:40Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/15677","html_url":"https://github.com/apache/spark/pull/15677","diff_url":"https://github.com/apache/spark/pull/15677.diff","patch_url":"https://github.com/apache/spark/pull/15677.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis PR proposes to change the documentation for functions. Please refer the discussion from https://github.com/apache/spark/pull/15513\r\n\r\nThe changes include\r\n- Re-indent the documentation\r\n- Add examples/arguments in `extended` where the arguments are multiple or specific format (e.g. xml/ json).\r\n\r\nFor examples, the documentation was updated as below:\r\n### Functions with single line usage\r\n\r\n**Before**\r\n- `pow`\r\n  \r\n  ``` sql\r\n  Usage: pow(x1, x2) - Raise x1 to the power of x2.\r\n  Extended Usage:\r\n  > SELECT pow(2, 3);\r\n   8.0\r\n  ```\r\n- `current_timestamp`\r\n  \r\n  ``` sql\r\n  Usage: current_timestamp() - Returns the current timestamp at the start of query evaluation.\r\n  Extended Usage:\r\n  No example for current_timestamp.\r\n  ```\r\n\r\n**After**\r\n- `pow`\r\n  \r\n  ``` sql\r\n  Usage: pow(expr1, expr2) - Raises `expr1` to the power of `expr2`.\r\n  Extended Usage:\r\n      Examples:\r\n        > SELECT pow(2, 3);\r\n         8.0\r\n  ```\r\n\r\n- `current_timestamp`\r\n  \r\n  ``` sql\r\n  Usage: current_timestamp() - Returns the current timestamp at the start of query evaluation.\r\n  Extended Usage:\r\n      No example/argument for current_timestamp.\r\n  ```\r\n### Functions with (already) multiple line usage\r\n\r\n**Before**\r\n- `approx_count_distinct`\r\n  \r\n  ``` sql\r\n  Usage: approx_count_distinct(expr) - Returns the estimated cardinality by HyperLogLog++.\r\n      approx_count_distinct(expr, relativeSD=0.05) - Returns the estimated cardinality by HyperLogLog++\r\n        with relativeSD, the maximum estimation error allowed.\r\n  \r\n  Extended Usage:\r\n  No example for approx_count_distinct.\r\n  ```\r\n- `percentile_approx`\r\n  \r\n  ``` sql\r\n  Usage:\r\n        percentile_approx(col, percentage [, accuracy]) - Returns the approximate percentile value of numeric\r\n        column `col` at the given percentage. The value of percentage must be between 0.0\r\n        and 1.0. The `accuracy` parameter (default: 10000) is a positive integer literal which\r\n        controls approximation accuracy at the cost of memory. Higher value of `accuracy` yields\r\n        better accuracy, `1.0/accuracy` is the relative error of the approximation.\r\n  \r\n        percentile_approx(col, array(percentage1 [, percentage2]...) [, accuracy]) - Returns the approximate\r\n        percentile array of column `col` at the given percentage array. Each value of the\r\n        percentage array must be between 0.0 and 1.0. The `accuracy` parameter (default: 10000) is\r\n        a positive integer literal which controls approximation accuracy at the cost of memory.\r\n        Higher value of `accuracy` yields better accuracy, `1.0/accuracy` is the relative error of\r\n        the approximation.\r\n  \r\n  Extended Usage:\r\n  No example for percentile_approx.\r\n  ```\r\n\r\n**After**\r\n- `approx_count_distinct`\r\n  \r\n  ``` sql\r\n  Usage:\r\n      approx_count_distinct(expr[, relativeSD]) - Returns the estimated cardinality by HyperLogLog++.\r\n        `relativeSD` defines the maximum estimation error allowed.\r\n\r\n  Extended Usage:\r\n      No example/argument for approx_count_distinct.\r\n  ```\r\n\r\n- `percentile_approx`\r\n  \r\n  ``` sql\r\n  Usage:\r\n      percentile_approx(col, percentage [, accuracy]) - Returns the approximate percentile value of numeric\r\n        column `col` at the given percentage. The value of percentage must be between 0.0\r\n        and 1.0. The `accuracy` parameter (default: 10000) is a positive numeric literal which\r\n        controls approximation accuracy at the cost of memory. Higher value of `accuracy` yields\r\n        better accuracy, `1.0/accuracy` is the relative error of the approximation.\r\n        When `percentage` is an array, each value of the percentage array must be between 0.0 and 1.0.\r\n        In this case, returns the approximate percentile array of column `col` at the given\r\n        percentage array.\r\n\r\n  Extended Usage:\r\n      Examples:\r\n        > SELECT percentile_approx(10.0, array(0.5, 0.4, 0.1), 100);\r\n         [10.0,10.0,10.0]\r\n        > SELECT percentile_approx(10.0, 0.5, 100);\r\n         10.0\r\n  ```\r\n## How was this patch tested?\r\n\r\nManually tested\r\n\r\n**When examples are multiple**\r\n\r\n``` sql\r\nspark-sql> describe function extended reflect;\r\nFunction: reflect\r\nClass: org.apache.spark.sql.catalyst.expressions.CallMethodViaReflection\r\nUsage: reflect(class, method[, arg1[, arg2 ..]]) - Calls a method with reflection.\r\nExtended Usage:\r\n    Examples:\r\n      > SELECT reflect('java.util.UUID', 'randomUUID');\r\n       c33fb387-8500-4bfa-81d2-6e0e3e930df2\r\n      > SELECT reflect('java.util.UUID', 'fromString', 'a5cf6c42-0c85-418f-af6c-3e4e5b1328f2');\r\n       a5cf6c42-0c85-418f-af6c-3e4e5b1328f2\r\n```\r\n\r\n**When `Usage` is in single line**\r\n\r\n``` sql\r\nspark-sql> describe function extended min;\r\nFunction: min\r\nClass: org.apache.spark.sql.catalyst.expressions.aggregate.Min\r\nUsage: min(expr) - Returns the minimum value of `expr`.\r\nExtended Usage:\r\n    No example/argument for min.\r\n```\r\n\r\n**When `Usage` is already in multiple lines**\r\n\r\n``` sql\r\nspark-sql> describe function extended percentile_approx;\r\nFunction: percentile_approx\r\nClass: org.apache.spark.sql.catalyst.expressions.aggregate.ApproximatePercentile\r\nUsage:\r\n    percentile_approx(col, percentage [, accuracy]) - Returns the approximate percentile value of numeric\r\n      column `col` at the given percentage. The value of percentage must be between 0.0\r\n      and 1.0. The `accuracy` parameter (default: 10000) is a positive numeric literal which\r\n      controls approximation accuracy at the cost of memory. Higher value of `accuracy` yields\r\n      better accuracy, `1.0/accuracy` is the relative error of the approximation.\r\n      When `percentage` is an array, each value of the percentage array must be between 0.0 and 1.0.\r\n      In this case, returns the approximate percentile array of column `col` at the given\r\n      percentage array.\r\n\r\nExtended Usage:\r\n    Examples:\r\n      > SELECT percentile_approx(10.0, array(0.5, 0.4, 0.1), 100);\r\n       [10.0,10.0,10.0]\r\n      > SELECT percentile_approx(10.0, 0.5, 100);\r\n       10.0\r\n```\r\n\r\n**When example/argument is missing**\r\n\r\n``` sql\r\nspark-sql> describe function extended rank;\r\nFunction: rank\r\nClass: org.apache.spark.sql.catalyst.expressions.Rank\r\nUsage:\r\n    rank() - Computes the rank of a value in a group of values. The result is one plus the number\r\n      of rows preceding or equal to the current row in the ordering of the partition. The values\r\n      will produce gaps in the sequence.\r\n\r\nExtended Usage:\r\n    No example/argument for rank.\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/258029404","html_url":"https://github.com/apache/spark/pull/15677#issuecomment-258029404","issue_url":"https://api.github.com/repos/apache/spark/issues/15677","id":258029404,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-11-02T23:22:40Z","updated_at":"2016-11-02T23:22:40Z","body":"**[Test build #68033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68033/consoleFull)** for PR 15677 at commit [`6ad2068`](https://github.com/apache/spark/commit/6ad2068bae420ab8f61b20cb165038d841352436)."}}
{"actor":{"display_login":"SparkQA"}}