{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/15513","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/15513/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/15513/comments","events_url":"https://api.github.com/repos/apache/spark/issues/15513/events","html_url":"https://github.com/apache/spark/pull/15513","id":183415516,"number":15513,"title":"[WIP][SPARK-17963][SQL][Documentation] Add examples (extend) in each expression and improve documentation with arguments","user":{"login":"HyukjinKwon","id":6477701,"avatar_url":"https://avatars.githubusercontent.com/u/6477701?v=3","gravatar_id":"","url":"https://api.github.com/users/HyukjinKwon","html_url":"https://github.com/HyukjinKwon","followers_url":"https://api.github.com/users/HyukjinKwon/followers","following_url":"https://api.github.com/users/HyukjinKwon/following{/other_user}","gists_url":"https://api.github.com/users/HyukjinKwon/gists{/gist_id}","starred_url":"https://api.github.com/users/HyukjinKwon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HyukjinKwon/subscriptions","organizations_url":"https://api.github.com/users/HyukjinKwon/orgs","repos_url":"https://api.github.com/users/HyukjinKwon/repos","events_url":"https://api.github.com/users/HyukjinKwon/events{/privacy}","received_events_url":"https://api.github.com/users/HyukjinKwon/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":28,"created_at":"2016-10-17T13:26:12Z","updated_at":"2016-10-21T10:07:31Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/15513","html_url":"https://github.com/apache/spark/pull/15513","diff_url":"https://github.com/apache/spark/pull/15513.diff","patch_url":"https://github.com/apache/spark/pull/15513.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis PR proposes to change the documentation for functions.\r\n\r\nThe changes include\r\n\r\n - Re-indent the documentation\r\n - Add examples/arguments in `extended` where the arguments are multiple or specific format (e.g. xml/ json).\r\n\r\nFor examples, the documentation was updated as below:\r\n\r\n### Functions with single line usage\r\n\r\n**Before**\r\n  - `pow`\r\n\r\n    ```sql\r\n    Usage: pow(x1, x2) - Raise x1 to the power of x2.\r\n    Extended Usage:\r\n    > SELECT pow(2, 3);\r\n     8.0\r\n    ```\r\n\r\n  - `current_timestamp`\r\n\r\n    ```sql\r\n    Usage: current_timestamp() - Returns the current timestamp at the start of query evaluation.\r\n    Extended Usage:\r\n    No example for current_timestamp.\r\n    ```\r\n\r\n**After**\r\n\r\n  - `pow`\r\n\r\n    ```sql\r\n    Usage: pow(expr1, expr2) - Raise expr1 to the power of expr2.\r\n    Extended Usage:\r\n        pow(expr1, expr2)\r\n\r\n           Arguments:\r\n             expr1 - any numeric type or any nonnumeric type expression that can be implicitly\r\n               converted to double type\r\n             expr2 - any numeric type or any nonnumeric type expression that can be implicitly\r\n               converted to double type\r\n\r\n           Examples:\r\n             > SELECT pow(2, 3);\r\n              8.0\r\n    ```\r\n\r\n  - `current_timestamp`\r\n\r\n    ```sql\r\n    Usage: current_timestamp() - Returns the current timestamp at the start of query evaluation.\r\n    Extended Usage:\r\n        No example/arguemnt for current_timestamp.\r\n    ```\r\n\r\n\r\n### Functions with (already) multiple line usage\r\n\r\n**Before**\r\n\r\n  - `approx_count_distinct`\r\n\r\n    ```sql\r\n    Usage: approx_count_distinct(expr) - Returns the estimated cardinality by HyperLogLog++.\r\n        approx_count_distinct(expr, relativeSD=0.05) - Returns the estimated cardinality by HyperLogLog++\r\n          with relativeSD, the maximum estimation error allowed.\r\n\r\n    Extended Usage:\r\n    No example for approx_count_distinct.\r\n    ```\r\n\r\n  - `percentile_approx`\r\n\r\n    ```sql\r\n    Usage:\r\n          percentile_approx(col, percentage [, accuracy]) - Returns the approximate percentile value of numeric\r\n          column `col` at the given percentage. The value of percentage must be between 0.0\r\n          and 1.0. The `accuracy` parameter (default: 10000) is a positive integer literal which\r\n          controls approximation accuracy at the cost of memory. Higher value of `accuracy` yields\r\n          better accuracy, `1.0/accuracy` is the relative error of the approximation.\r\n\r\n          percentile_approx(col, array(percentage1 [, percentage2]...) [, accuracy]) - Returns the approximate\r\n          percentile array of column `col` at the given percentage array. Each value of the\r\n          percentage array must be between 0.0 and 1.0. The `accuracy` parameter (default: 10000) is\r\n          a positive integer literal which controls approximation accuracy at the cost of memory.\r\n          Higher value of `accuracy` yields better accuracy, `1.0/accuracy` is the relative error of\r\n          the approximation.\r\n\r\nExtended Usage:\r\nNo example for percentile_approx.\r\n```\r\n\r\n**After**\r\n\r\n  - `approx_count_distinct`\r\n\r\n    ```sql\r\n    Usage:\r\n        approx_count_distinct(expr) - Returns the estimated cardinality by HyperLogLog++.\r\n\r\n        approx_count_distinct(expr, relativeSD) - Returns the estimated cardinality by HyperLogLog++\r\n          with relativeSD, the maximum estimation error allowed.\r\n\r\n  Extended Usage:\r\n      approx_count_distinct(expr)\r\n\r\n        Arguments:\r\n          expr - any type expression that represents data to collect the first.\r\n\r\n      approx_count_distinct(expr, relativeSD)\r\n\r\n        Arguments:\r\n          expr - any type expression that represents data to collect the first.\r\n          relativeSD - any numeric type or any nonnumeric type literal that can be implicitly\r\n            converted to double type, that represents maximum estimation error allowed\r\n            (default = 0.05).\r\n```\r\n\r\n  - `percentile_approx`\r\n\r\n    ```sql\r\n    Usage:\r\n        percentile_approx(col, percentage [, accuracy]) - Returns the approximate percentile value of numeric\r\n          column `col` at the given percentage. The value of percentage must be between 0.0\r\n          and 1.0. The `accuracy` parameter (default: 10000) is a positive integer literal which\r\n          controls approximation accuracy at the cost of memory. Higher value of `accuracy` yields\r\n          better accuracy, `1.0/accuracy` is the relative error of the approximation.\r\n\r\n        percentile_approx(col, array(percentage1 [, percentage2]...) [, accuracy]) - Returns the approximate\r\n          percentile array of column `col` at the given percentage array. Each value of the\r\n          percentage array must be between 0.0 and 1.0. The `accuracy` parameter (default: 10000) is\r\n          a positive integer literal which controls approximation accuracy at the cost of memory.\r\n          Higher value of `accuracy` yields better accuracy, `1.0/accuracy` is the relative error of\r\n          the approximation.\r\n\r\n    Extended Usage:\r\n        percentile_approx(col, percentage [, accuracy])\r\n\r\n          Arguments:\r\n            col - any numeric type or any nonnumeric type expression that can be implicitly\r\n              converted to double type.\r\n            percentage - any numeric type or any nonnumeric type literal that can be\r\n              implicitly converted to double type.\r\n            accuracy - any numeric type or any nonnumeric type literal that can be implicitly\r\n              converted to int type.\r\n\r\n          Examples:\r\n            > SELECT percentile_approx(10.0, 0.5, 100);\r\n             10.0\r\n\r\n        percentile_approx(col, array(percentage1 [, percentage2]...) [, accuracy]) \r\n\r\n          Arguments:\r\n            col - any numeric type or any nonnumeric type expression that can be implicitly\r\n              converted to double type.\r\n            array(...) - an array that contains any numeric type literal that can be implicitly\r\n              converted to double type.\r\n            accuracy - any numeric type or any nonnumeric type literal that can be implicitly\r\n              converted to int type.\r\n\r\n          Examples:\r\n            > SELECT percentile_approx(10.0, array(0.5, 0.4, 0.1), 100);\r\n             [10.0,10.0,10.0]\r\n    ```\r\n\r\n## How was this patch tested?\r\n\r\nManually tested\r\n\r\n**When examples are multiple**\r\n\r\n```sql\r\nspark-sql> describe function extended reflect;\r\nFunction: reflect\r\nClass: org.apache.spark.sql.catalyst.expressions.CallMethodViaReflection\r\nUsage: reflect(class, method[, arg1[, arg2 ..]]) calls method with reflection\r\nExtended Usage:\r\n    reflect(class, method[, arg1[, arg2 ..]])\r\n\r\n      Arguments:\r\n        class - string type literal that represents full-qualified class name.\r\n        method - string type literal that represents method name.\r\n        arg - string type literal that represents arguments for the method.\r\n\r\n      Examples:\r\n        > SELECT reflect('java.util.UUID', 'randomUUID');\r\n         c33fb387-8500-4bfa-81d2-6e0e3e930df2\r\n        > SELECT reflect('java.util.UUID', 'fromString', 'a5cf6c42-0c85-418f-af6c-3e4e5b1328f2');\r\n         a5cf6c42-0c85-418f-af6c-3e4e5b1328f2\r\n```\r\n\r\n**When `Usage` is in single line**\r\n\r\n```sql\r\nspark-sql> describe function extended min;\r\nFunction: min\r\nClass: org.apache.spark.sql.catalyst.expressions.aggregate.Min\r\nUsage: min(expr) - Returns the minimum value of expr.\r\nExtended Usage:\r\n    min(expr)\r\n\r\n      Arguments:\r\n        expr - any type expression.\r\n```\r\n\r\n**When `Usage` is already in multiple lines**\r\n\r\n```sql\r\nspark-sql> describe function extended percentile_approx;\r\nFunction: percentile_approx\r\nClass: org.apache.spark.sql.catalyst.expressions.aggregate.ApproximatePercentile\r\nUsage:\r\n    percentile_approx(col, percentage [, accuracy]) - Returns the approximate percentile value of numeric\r\n      column `col` at the given percentage. The value of percentage must be between 0.0\r\n      and 1.0. The `accuracy` parameter (default: 10000) is a positive integer literal which\r\n      controls approximation accuracy at the cost of memory. Higher value of `accuracy` yields\r\n      better accuracy, `1.0/accuracy` is the relative error of the approximation.\r\n\r\n    percentile_approx(col, array(percentage1 [, percentage2]...) [, accuracy]) - Returns the approximate\r\n      percentile array of column `col` at the given percentage array. Each value of the\r\n      percentage array must be between 0.0 and 1.0. The `accuracy` parameter (default: 10000) is\r\n      a positive integer literal which controls approximation accuracy at the cost of memory.\r\n      Higher value of `accuracy` yields better accuracy, `1.0/accuracy` is the relative error of\r\n      the approximation.\r\n\r\nExtended Usage:\r\n    percentile_approx(col, percentage [, accuracy])\r\n\r\n      Arguments:\r\n        col - any numeric type or any nonnumeric type expression that can be implicitly\r\n          converted to numeric type.\r\n        array(...) - an array that contains any numeric type literal that can be implicitly\r\n          converted to numeric type.\r\n        accuracy - any numeric type or any nonnumeric type literal that can be implicitly\r\n          converted to numeric type.\r\n\r\n      Examples:\r\n        > SELECT percentile_approx(10.0, array(0.5, 0.4, 0.1), 100);\r\n         [10.0,10.0,10.0]\r\n\r\n    percentile_approx(col, array(percentage1 [, percentage2]...) [, accuracy])\r\n\r\n      Arguments:\r\n        col - any numeric type or any nonnumeric type expression that can be implicitly\r\n          converted to numeric type.\r\n        percentage - any numeric type or any nonnumeric type literal that can be\r\n          implicitly converted to numeric type.\r\n        accuracy - any numeric type or any nonnumeric type literal that can be implicitly\r\n          converted to numeric type.\r\n\r\n      Examples:\r\n        > SELECT percentile_approx(10.0, 0.5, 100);\r\n         10.0\r\n```\r\n\r\n**When example/argument is missing**\r\n\r\n```sql\r\nspark-sql> describe function extended rank;\r\nFunction: rank\r\nClass: org.apache.spark.sql.catalyst.expressions.Rank\r\nUsage:\r\n    rank() -  RANK() computes the rank of a value in a group of values. The result\r\n      is one plus the number of rows preceding or equal to the current row in the\r\n      ordering of the partition. Tie values will produce gaps in the sequence.\r\n\r\nExtended Usage:\r\n    No example/argument for rank.\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/255344062","html_url":"https://github.com/apache/spark/pull/15513#issuecomment-255344062","issue_url":"https://api.github.com/repos/apache/spark/issues/15513","id":255344062,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-10-21T10:07:31Z","updated_at":"2016-10-21T10:07:31Z","body":"**[Test build #67334 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67334/consoleFull)** for PR 15513 at commit [`f71e39e`](https://github.com/apache/spark/commit/f71e39eb6145cc9f1d3265573bfe245e202d70dc)."}}
{"actor":{"display_login":"SparkQA"}}