{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/11555","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/11555/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/11555/comments","events_url":"https://api.github.com/repos/apache/spark/issues/11555/events","html_url":"https://github.com/apache/spark/pull/11555","id":138905053,"number":11555,"title":"[SPARK-12718][SPARK-13720][SQL] SQL generation support for window functions","user":{"login":"cloud-fan","id":3182036,"avatar_url":"https://avatars.githubusercontent.com/u/3182036?v=3","gravatar_id":"","url":"https://api.github.com/users/cloud-fan","html_url":"https://github.com/cloud-fan","followers_url":"https://api.github.com/users/cloud-fan/followers","following_url":"https://api.github.com/users/cloud-fan/following{/other_user}","gists_url":"https://api.github.com/users/cloud-fan/gists{/gist_id}","starred_url":"https://api.github.com/users/cloud-fan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cloud-fan/subscriptions","organizations_url":"https://api.github.com/users/cloud-fan/orgs","repos_url":"https://api.github.com/users/cloud-fan/repos","events_url":"https://api.github.com/users/cloud-fan/events{/privacy}","received_events_url":"https://api.github.com/users/cloud-fan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":15,"created_at":"2016-03-07T07:08:43Z","updated_at":"2016-03-07T16:44:34Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/11555","html_url":"https://github.com/apache/spark/pull/11555","diff_url":"https://github.com/apache/spark/pull/11555.diff","patch_url":"https://github.com/apache/spark/pull/11555.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nAdd SQL generation support for window functions. The idea is simple, just treat `Window` operator like `Project`, i.e. add subquery to its child when necessary, generate a `SELECT ... FROM ...` SQL string, implement `sql` method for window related expressions, e.g. `WindowSpecDefinition`, `WindowFrame`, etc.\r\n\r\nThis PR also fixed SPARK-13720 by improving the process of adding extra `SubqueryAlias`(the `RecoverScopingInfo` rule). Before this PR, we update the qualifiers in project list while adding the subquery. However, this is incomplete as we need to update qualifiers in all ancestors that refer attributes here. In this PR, we split `RecoverScopingInfo` into 2 rules: `AddSubQuery` and `UpdateQualifier`. `AddSubQuery` only add subquery if necessary, and `UpdateQualifier` will re-propagate and update qualifiers bottom up.\r\n\r\nIdeally we should put the bug fix part in an individual PR, but this bug also blocks the window stuff, so I put them together here.\r\n\r\nMany thanks to @gatorsmile for the initial discussion and test cases!\r\n\r\n## How was this patch tested?\r\n\r\nnew tests in `LogicalPlanToSQLSuite`"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/193339120","html_url":"https://github.com/apache/spark/pull/11555#issuecomment-193339120","issue_url":"https://api.github.com/repos/apache/spark/issues/11555","id":193339120,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-03-07T16:44:33Z","updated_at":"2016-03-07T16:44:33Z","body":"**[Test build #52566 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52566/consoleFull)** for PR 11555 at commit [`276a870`](https://github.com/apache/spark/commit/276a870dee9d150c35220c391d8d41acd463c314).\n * This patch **fails Spark unit tests**.\n * This patch merges cleanly.\n * This patch adds no public classes."}}