{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/10733","labels_url":"https://api.github.com/repos/apache/spark/issues/10733/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/10733/comments","events_url":"https://api.github.com/repos/apache/spark/issues/10733/events","html_url":"https://github.com/apache/spark/pull/10733","id":126324874,"number":10733,"title":"[SPARK-12728][SQL] Integrates SQL generation with native view","user":{"login":"liancheng","id":230655,"avatar_url":"https://avatars.githubusercontent.com/u/230655?v=3","gravatar_id":"","url":"https://api.github.com/users/liancheng","html_url":"https://github.com/liancheng","followers_url":"https://api.github.com/users/liancheng/followers","following_url":"https://api.github.com/users/liancheng/following{/other_user}","gists_url":"https://api.github.com/users/liancheng/gists{/gist_id}","starred_url":"https://api.github.com/users/liancheng/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liancheng/subscriptions","organizations_url":"https://api.github.com/users/liancheng/orgs","repos_url":"https://api.github.com/users/liancheng/repos","events_url":"https://api.github.com/users/liancheng/events{/privacy}","received_events_url":"https://api.github.com/users/liancheng/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":17,"created_at":"2016-01-13T02:20:11Z","updated_at":"2016-01-27T00:12:00Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/10733","html_url":"https://github.com/apache/spark/pull/10733","diff_url":"https://github.com/apache/spark/pull/10733.diff","patch_url":"https://github.com/apache/spark/pull/10733.patch"},"body":"This PR is a follow-up of PR #10541. It integrates the newly introduced SQL generation feature with native view to make native view canonical.\r\n\r\nIn this PR, a new SQL option `spark.sql.nativeView.canonical` is added.  When this option and `spark.sql.nativeView` are both `true`, Spark SQL tries to handle `CREATE VIEW` DDL statements using SQL query strings generated from view definition logical plans. If we failed to map the plan to SQL, we fallback to the original native view approach.\r\n\r\nOne important issue this PR fixes is that, now we can use CTE when defining a view.  Originally, when native view is turned on, we wrap the view definition text with an extra `SELECT`.  However, HiveQL parser doesn't allow CTE appearing as a subquery.  Namely, something like this is disallowed:\r\n\r\n```sql\r\nSELECT n\r\nFROM (\r\n  WITH w AS (SELECT 1 AS n)\r\n  SELECT * FROM w\r\n) v\r\n```\r\n\r\nThis PR fixes this issue because the extra `SELECT` is no longer needed (also, CTE expressions are inlined as subqueries during analysis phase, thus there won't be CTE expressions in the generated SQL query string)."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/175306181","html_url":"https://github.com/apache/spark/pull/10733#issuecomment-175306181","issue_url":"https://api.github.com/repos/apache/spark/issues/10733","id":175306181,"user":{"login":"liancheng","id":230655,"avatar_url":"https://avatars.githubusercontent.com/u/230655?v=3","gravatar_id":"","url":"https://api.github.com/users/liancheng","html_url":"https://github.com/liancheng","followers_url":"https://api.github.com/users/liancheng/followers","following_url":"https://api.github.com/users/liancheng/following{/other_user}","gists_url":"https://api.github.com/users/liancheng/gists{/gist_id}","starred_url":"https://api.github.com/users/liancheng/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liancheng/subscriptions","organizations_url":"https://api.github.com/users/liancheng/orgs","repos_url":"https://api.github.com/users/liancheng/repos","events_url":"https://api.github.com/users/liancheng/events{/privacy}","received_events_url":"https://api.github.com/users/liancheng/received_events","type":"User","site_admin":false},"created_at":"2016-01-27T00:12:00Z","updated_at":"2016-01-27T00:12:00Z","body":"retest this please"}}