{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/12850","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/12850/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/12850/comments","events_url":"https://api.github.com/repos/apache/spark/issues/12850/events","html_url":"https://github.com/apache/spark/pull/12850","id":152662508,"number":12850,"title":"[SPARK-15076][SQL] Add ReorderAssociativeOperator optimizer","user":{"login":"dongjoon-hyun","id":9700541,"avatar_url":"https://avatars.githubusercontent.com/u/9700541?v=3","gravatar_id":"","url":"https://api.github.com/users/dongjoon-hyun","html_url":"https://github.com/dongjoon-hyun","followers_url":"https://api.github.com/users/dongjoon-hyun/followers","following_url":"https://api.github.com/users/dongjoon-hyun/following{/other_user}","gists_url":"https://api.github.com/users/dongjoon-hyun/gists{/gist_id}","starred_url":"https://api.github.com/users/dongjoon-hyun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dongjoon-hyun/subscriptions","organizations_url":"https://api.github.com/users/dongjoon-hyun/orgs","repos_url":"https://api.github.com/users/dongjoon-hyun/repos","events_url":"https://api.github.com/users/dongjoon-hyun/events{/privacy}","received_events_url":"https://api.github.com/users/dongjoon-hyun/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":22,"created_at":"2016-05-02T22:41:51Z","updated_at":"2016-06-01T02:00:18Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/12850","html_url":"https://github.com/apache/spark/pull/12850","diff_url":"https://github.com/apache/spark/pull/12850.diff","patch_url":"https://github.com/apache/spark/pull/12850.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis issue add a new optimizer `ReorderAssociativeOperator` by taking advantage of integral associative property. Currently, Spark works like the following.\r\n\r\n1) Can optimize `1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + a` into `45 + a`.\r\n2) Cannot optimize `a + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9`.\r\n\r\nThis PR can handle Case 2 for **Add/Multiply** expression whose data types are `ByteType`, `ShortType`, `IntegerType`, and `LongType`. The followings are the plan comparison between `before` and `after` this issue.\r\n\r\n**Before**\r\n```scala\r\nscala> sql(\"select a+1+2+3+4+5+6+7+8+9 from (select explode(array(1)) a)\").explain\r\n== Physical Plan ==\r\nWholeStageCodegen\r\n:  +- Project [(((((((((a#7 + 1) + 2) + 3) + 4) + 5) + 6) + 7) + 8) + 9) AS (((((((((a + 1) + 2) + 3) + 4) + 5) + 6) + 7) + 8) + 9)#8]\r\n:     +- INPUT\r\n+- Generate explode([1]), false, false, [a#7]\r\n   +- Scan OneRowRelation[]\r\nscala> sql(\"select a*1*2*3*4*5*6*7*8*9 from (select explode(array(1)) a)\").explain\r\n== Physical Plan ==\r\n*Project [(((((((((a#18 * 1) * 2) * 3) * 4) * 5) * 6) * 7) * 8) * 9) AS (((((((((a * 1) * 2) * 3) * 4) * 5) * 6) * 7) * 8) * 9)#19]\r\n+- Generate explode([1]), false, false, [a#18]\r\n   +- Scan OneRowRelation[]\r\n```\r\n\r\n**After**\r\n```scala\r\nscala> sql(\"select a+1+2+3+4+5+6+7+8+9 from (select explode(array(1)) a)\").explain\r\n== Physical Plan ==\r\nWholeStageCodegen\r\n:  +- Project [(a#7 + 45) AS (((((((((a + 1) + 2) + 3) + 4) + 5) + 6) + 7) + 8) + 9)#8]\r\n:     +- INPUT\r\n+- Generate explode([1]), false, false, [a#7]\r\n   +- Scan OneRowRelation[]\r\nscala> sql(\"select a*1*2*3*4*5*6*7*8*9 from (select explode(array(1)) a)\").explain\r\n== Physical Plan ==\r\n*Project [(a#18 * 362880) AS (((((((((a * 1) * 2) * 3) * 4) * 5) * 6) * 7) * 8) * 9)#19]\r\n+- Generate explode([1]), false, false, [a#18]\r\n   +- Scan OneRowRelation[]\r\n```\r\n\r\n## How was this patch tested?\r\n\r\nPass the Jenkins tests including new testsuite."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/222872750","html_url":"https://github.com/apache/spark/pull/12850#issuecomment-222872750","issue_url":"https://api.github.com/repos/apache/spark/issues/12850","id":222872750,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2016-06-01T02:00:18Z","updated_at":"2016-06-01T02:00:18Z","body":"Merged build finished. Test PASSed."}}
{"actor":{"display_login":"AmplabJenkins"}}