{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/12719","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/12719/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/12719/comments","events_url":"https://api.github.com/repos/apache/spark/issues/12719/events","html_url":"https://github.com/apache/spark/pull/12719","id":151262663,"number":12719,"title":"[SPARK-14939][SQL] Add FoldablePropagation optimizer","user":{"login":"dongjoon-hyun","id":9700541,"avatar_url":"https://avatars.githubusercontent.com/u/9700541?v=3","gravatar_id":"","url":"https://api.github.com/users/dongjoon-hyun","html_url":"https://github.com/dongjoon-hyun","followers_url":"https://api.github.com/users/dongjoon-hyun/followers","following_url":"https://api.github.com/users/dongjoon-hyun/following{/other_user}","gists_url":"https://api.github.com/users/dongjoon-hyun/gists{/gist_id}","starred_url":"https://api.github.com/users/dongjoon-hyun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dongjoon-hyun/subscriptions","organizations_url":"https://api.github.com/users/dongjoon-hyun/orgs","repos_url":"https://api.github.com/users/dongjoon-hyun/repos","events_url":"https://api.github.com/users/dongjoon-hyun/events{/privacy}","received_events_url":"https://api.github.com/users/dongjoon-hyun/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":42,"created_at":"2016-04-26T23:26:23Z","updated_at":"2016-05-10T08:33:26Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/12719","html_url":"https://github.com/apache/spark/pull/12719","diff_url":"https://github.com/apache/spark/pull/12719.diff","patch_url":"https://github.com/apache/spark/pull/12719.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis PR aims to add new **FoldablePropagation** optimizer that propagates foldable expressions by replacing all attributes with the aliases of original foldable expression. Other optimizations will take advantage of the propagated foldable expressions: e.g. `EliminateSorts` optimizer now can handle the following Case 2 and 3. (Case 1 is the previous implementation.)\r\n\r\n1. Literals and foldable expression, e.g. \"ORDER BY 1.0, 'abc', Now()\"\r\n2. Foldable ordinals, e.g. \"SELECT 1.0, 'abc', Now() ORDER BY 1, 2, 3\"\r\n3. Foldable aliases, e.g. \"SELECT 1.0 x, 'abc' y, Now() z ORDER BY x, y, z\"\r\n\r\nThis PR has been generalized based on @cloud-fan 's key ideas many times; he should be credited for the work he did.\r\n\r\n**Before**\r\n```\r\nscala> sql(\"SELECT 1.0, Now() x ORDER BY 1, x\").explain\r\n== Physical Plan ==\r\nWholeStageCodegen\r\n:  +- Sort [1.0#5 ASC,x#0 ASC], true, 0\r\n:     +- INPUT\r\n+- Exchange rangepartitioning(1.0#5 ASC, x#0 ASC, 200), None\r\n   +- WholeStageCodegen\r\n      :  +- Project [1.0 AS 1.0#5,1461873043577000 AS x#0]\r\n      :     +- INPUT\r\n      +- Scan OneRowRelation[]\r\n```\r\n\r\n**After**\r\n```\r\nscala> sql(\"SELECT 1.0, Now() x ORDER BY 1, x\").explain\r\n== Physical Plan ==\r\nWholeStageCodegen\r\n:  +- Project [1.0 AS 1.0#5,1461873079484000 AS x#0]\r\n:     +- INPUT\r\n+- Scan OneRowRelation[]\r\n```\r\n\r\n## How was this patch tested?\r\n\r\nPass the Jenkins tests including a new test case."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/218092620","html_url":"https://github.com/apache/spark/pull/12719#issuecomment-218092620","issue_url":"https://api.github.com/repos/apache/spark/issues/12719","id":218092620,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2016-05-10T08:33:26Z","updated_at":"2016-05-10T08:33:26Z","body":"Test PASSed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58220/\nTest PASSed."}}