{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/10896","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/10896/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/10896/comments","events_url":"https://api.github.com/repos/apache/spark/issues/10896/events","html_url":"https://github.com/apache/spark/pull/10896","id":128484689,"number":10896,"title":"[SPARK-12978][SQL] Skip unnecessary final group-by when input data already clustered with group-by keys","user":{"login":"maropu","id":692303,"avatar_url":"https://avatars.githubusercontent.com/u/692303?v=3","gravatar_id":"","url":"https://api.github.com/users/maropu","html_url":"https://github.com/maropu","followers_url":"https://api.github.com/users/maropu/followers","following_url":"https://api.github.com/users/maropu/following{/other_user}","gists_url":"https://api.github.com/users/maropu/gists{/gist_id}","starred_url":"https://api.github.com/users/maropu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/maropu/subscriptions","organizations_url":"https://api.github.com/users/maropu/orgs","repos_url":"https://api.github.com/users/maropu/repos","events_url":"https://api.github.com/users/maropu/events{/privacy}","received_events_url":"https://api.github.com/users/maropu/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":26,"created_at":"2016-01-25T08:14:25Z","updated_at":"2016-06-07T11:34:30Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/10896","html_url":"https://github.com/apache/spark/pull/10896","diff_url":"https://github.com/apache/spark/pull/10896.diff","patch_url":"https://github.com/apache/spark/pull/10896.patch"},"body":"This ticket targets the optimization to skip an unnecessary group-by operation below;\r\n\r\nWithout opt.:\r\n```\r\n== Physical Plan ==\r\nTungstenAggregate(key=[col0#159], functions=[(sum(col1#160),mode=Final,isDistinct=false),(avg(col2#161),mode=Final,isDistinct=false)], output=[col0#159,sum(col1)#177,avg(col2)#178])\r\n+- TungstenAggregate(key=[col0#159], functions=[(sum(col1#160),mode=Partial,isDistinct=false),(avg(col2#161),mode=Partial,isDistinct=false)], output=[col0#159,sum#200,sum#201,count#202L])\r\n   +- TungstenExchange hashpartitioning(col0#159,200), None\r\n      +- InMemoryColumnarTableScan [col0#159,col1#160,col2#161], InMemoryRelation [col0#159,col1#160,col2#161], true, 10000, StorageLevel(true, true, false, true, 1), ConvertToUnsafe, None\r\n```\r\n\r\nWith opt.:\r\n```\r\n== Physical Plan ==\r\nTungstenAggregate(key=[col0#159], functions=[(sum(col1#160),mode=Complete,isDistinct=false),(avg(col2#161),mode=Final,isDistinct=false)], output=[col0#159,sum(col1)#177,avg(col2)#178])\r\n+- TungstenExchange hashpartitioning(col0#159,200), None\r\n  +- InMemoryColumnarTableScan [col0#159,col1#160,col2#161], InMemoryRelation [col0#159,col1#160,col2#161], true, 10000, StorageLevel(true, true, false, true, 1), ConvertToUnsafe, None\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/224254259","html_url":"https://github.com/apache/spark/pull/10896#issuecomment-224254259","issue_url":"https://api.github.com/repos/apache/spark/issues/10896","id":224254259,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-06-07T11:34:30Z","updated_at":"2016-06-07T11:34:30Z","body":"**[Test build #60121 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60121/consoleFull)** for PR 10896 at commit [`2007caa`](https://github.com/apache/spark/commit/2007caaa712439926d8a0d30df1bffe472bf50d3).\n * This patch passes all tests.\n * This patch merges cleanly.\n * This patch adds no public classes."}}
{"actor":{"display_login":"SparkQA"}}