{"action":"created","comment":{"url":"https://api.github.com/repos/citusdata/citus/pulls/comments/64725104","id":64725104,"diff_hunk":"@@ -0,0 +1,129 @@\n+--\n+-- COMPLEX_COUNT_DISTINCT\n+--\n+\n+CREATE TABLE lineitem_hash (\n+\tl_orderkey bigint not null,\n+\tl_partkey integer not null,\n+\tl_suppkey integer not null,\n+\tl_linenumber integer not null,\n+\tl_quantity decimal(15, 2) not null,\n+\tl_extendedprice decimal(15, 2) not null,\n+\tl_discount decimal(15, 2) not null,\n+\tl_tax decimal(15, 2) not null,\n+\tl_returnflag char(1) not null,\n+\tl_linestatus char(1) not null,\n+\tl_shipdate date not null,\n+\tl_commitdate date not null,\n+\tl_receiptdate date not null,\n+\tl_shipinstruct char(25) not null,\n+\tl_shipmode char(10) not null,\n+\tl_comment varchar(44) not null,\n+\tPRIMARY KEY(l_orderkey, l_linenumber) );\n+\t\n+SELECT master_create_distributed_table('lineitem_hash', 'l_orderkey', 'hash');\n+SELECT master_create_worker_shards('lineitem_hash', 8, 1);\n+\n+\\COPY lineitem_hash FROM '@abs_srcdir@/data/lineitem.1.data' with delimiter '|'\n+\\COPY lineitem_hash FROM '@abs_srcdir@/data/lineitem.2.data' with delimiter '|'\n+\n+SET citus.task_executor_type to \"task-tracker\";\n+\n+-- count(distinct) is supported on top level query if there\n+-- is a grouping on the partition key\t\n+SELECT \n+\tl_orderkey, count(DISTINCT l_partkey) \n+\tFROM lineitem_hash \n+\tGROUP BY l_orderkey \n+\tORDER BY 2 DESC \n+\tLIMIT 10;\n+\n+-- it is not supported if there is no grouping or grouping is on non-partition field\n+SELECT \n+\tcount(DISTINCT l_partkey) \n+\tFROM lineitem_hash \n+\tORDER BY 1 DESC \n+\tLIMIT 10;\n+\n+SELECT \n+\tl_shipmode, count(DISTINCT l_partkey) \n+\tFROM lineitem_hash \n+\tGROUP BY l_shipmode \n+\tORDER BY 2 DESC \n+\tLIMIT 10;\n+\n+-- count distinct is supported on single table subqueries\n+SELECT *\n+\tFROM (\n+\t\tSELECT \n+\t\t\tl_orderkey, count(DISTINCT l_partkey) \n+\t\t\tFROM lineitem_hash \n+\t\t\tGROUP BY l_orderkey) sub\n+\tORDER BY 2 DESC ","path":"src/test/regress/input/multi_complex_count_distinct.source","position":62,"original_position":62,"commit_id":"03d5c6e4771d5b36f4882f6e67b3ead87740a565","original_commit_id":"03d5c6e4771d5b36f4882f6e67b3ead87740a565","user":{"login":"mtuncer","id":9413470,"avatar_url":"https://avatars.githubusercontent.com/u/9413470?v=3","gravatar_id":"","url":"https://api.github.com/users/mtuncer","html_url":"https://github.com/mtuncer","followers_url":"https://api.github.com/users/mtuncer/followers","following_url":"https://api.github.com/users/mtuncer/following{/other_user}","gists_url":"https://api.github.com/users/mtuncer/gists{/gist_id}","starred_url":"https://api.github.com/users/mtuncer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mtuncer/subscriptions","organizations_url":"https://api.github.com/users/mtuncer/orgs","repos_url":"https://api.github.com/users/mtuncer/repos","events_url":"https://api.github.com/users/mtuncer/events{/privacy}","received_events_url":"https://api.github.com/users/mtuncer/received_events","type":"User","site_admin":false},"body":"does not make much difference but ok.","created_at":"2016-05-26T10:39:57Z","updated_at":"2016-05-26T10:39:57Z","html_url":"https://github.com/citusdata/citus/pull/516#discussion_r64725104","pull_request_url":"https://api.github.com/repos/citusdata/citus/pulls/516","_links":{"self":{"href":"https://api.github.com/repos/citusdata/citus/pulls/comments/64725104"},"html":{"href":"https://github.com/citusdata/citus/pull/516#discussion_r64725104"},"pull_request":{"href":"https://api.github.com/repos/citusdata/citus/pulls/516"}}},"pull_request":{"url":"https://api.github.com/repos/citusdata/citus/pulls/516","id":69673670,"html_url":"https://github.com/citusdata/citus/pull/516","diff_url":"https://github.com/citusdata/citus/pull/516.diff","patch_url":"https://github.com/citusdata/citus/pull/516.patch","issue_url":"https://api.github.com/repos/citusdata/citus/issues/516","number":516,"state":"open","locked":false,"title":"Add complex count distinct support","user":{"login":"mtuncer","id":9413470,"avatar_url":"https://avatars.githubusercontent.com/u/9413470?v=3","gravatar_id":"","url":"https://api.github.com/users/mtuncer","html_url":"https://github.com/mtuncer","followers_url":"https://api.github.com/users/mtuncer/followers","following_url":"https://api.github.com/users/mtuncer/following{/other_user}","gists_url":"https://api.github.com/users/mtuncer/gists{/gist_id}","starred_url":"https://api.github.com/users/mtuncer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mtuncer/subscriptions","organizations_url":"https://api.github.com/users/mtuncer/orgs","repos_url":"https://api.github.com/users/mtuncer/repos","events_url":"https://api.github.com/users/mtuncer/events{/privacy}","received_events_url":"https://api.github.com/users/mtuncer/received_events","type":"User","site_admin":false},"body":"We can now support complex count distinct in queries in single table subqueries with repartitioning, i.e. queries like\r\n\r\n```sql\r\nSELECT *\r\nFROM (\r\n\tSELECT \r\n\t\tl_orderkey, count(DISTINCT CASE WHEN l_shipmode = 'AIR' THEN l_partkey ELSE NULL END) as count \r\n\tFROM \r\n\t\tlineitem \r\n\tGROUP BY\r\n\t\tl_orderkey) sub\r\nWHERE count > 0\r\nORDER BY 2 DESC, 1 DESC \r\nLIMIT 10;\r\n```\r\n\r\nFixes #434 \r\n\r\nSolution involves adding columns in count distinct aggregate to target list, and to the group by list of worker re-partition query. Actual count distinct operation is performed while merging the partitions during reduce.\r\n\r\nNote that the fix is specific to single table subqueries, single level repartition queries can not have complex count distinct aggregates yet.\r\n","created_at":"2016-05-11T13:40:25Z","updated_at":"2016-05-26T10:39:57Z","closed_at":null,"merged_at":null,"merge_commit_sha":"2828026c693f2fdcdb0bedead09aac4d229ef87d","assignee":{"login":"onderkalaci","id":7829470,"avatar_url":"https://avatars.githubusercontent.com/u/7829470?v=3","gravatar_id":"","url":"https://api.github.com/users/onderkalaci","html_url":"https://github.com/onderkalaci","followers_url":"https://api.github.com/users/onderkalaci/followers","following_url":"https://api.github.com/users/onderkalaci/following{/other_user}","gists_url":"https://api.github.com/users/onderkalaci/gists{/gist_id}","starred_url":"https://api.github.com/users/onderkalaci/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/onderkalaci/subscriptions","organizations_url":"https://api.github.com/users/onderkalaci/orgs","repos_url":"https://api.github.com/users/onderkalaci/repos","events_url":"https://api.github.com/users/onderkalaci/events{/privacy}","received_events_url":"https://api.github.com/users/onderkalaci/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/citusdata/citus/pulls/516/commits","review_comments_url":"https://api.github.com/repos/citusdata/citus/pulls/516/comments","review_comment_url":"https://api.github.com/repos/citusdata/citus/pulls/comments{/number}","comments_url":"https://api.github.com/repos/citusdata/citus/issues/516/comments","statuses_url":"https://api.github.com/repos/citusdata/citus/statuses/03d5c6e4771d5b36f4882f6e67b3ead87740a565","head":{"label":"citusdata:feature/fix_434_support_count_distinct","ref":"feature/fix_434_support_count_distinct","sha":"03d5c6e4771d5b36f4882f6e67b3ead87740a565","user":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars.githubusercontent.com/u/2545424?v=3","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"repo":{"id":50874442,"name":"citus","full_name":"citusdata/citus","owner":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars.githubusercontent.com/u/2545424?v=3","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/citusdata/citus","description":"Scalable PostgreSQL for real-time workloads","fork":false,"url":"https://api.github.com/repos/citusdata/citus","forks_url":"https://api.github.com/repos/citusdata/citus/forks","keys_url":"https://api.github.com/repos/citusdata/citus/keys{/key_id}","collaborators_url":"https://api.github.com/repos/citusdata/citus/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/citusdata/citus/teams","hooks_url":"https://api.github.com/repos/citusdata/citus/hooks","issue_events_url":"https://api.github.com/repos/citusdata/citus/issues/events{/number}","events_url":"https://api.github.com/repos/citusdata/citus/events","assignees_url":"https://api.github.com/repos/citusdata/citus/assignees{/user}","branches_url":"https://api.github.com/repos/citusdata/citus/branches{/branch}","tags_url":"https://api.github.com/repos/citusdata/citus/tags","blobs_url":"https://api.github.com/repos/citusdata/citus/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/citusdata/citus/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/citusdata/citus/git/refs{/sha}","trees_url":"https://api.github.com/repos/citusdata/citus/git/trees{/sha}","statuses_url":"https://api.github.com/repos/citusdata/citus/statuses/{sha}","languages_url":"https://api.github.com/repos/citusdata/citus/languages","stargazers_url":"https://api.github.com/repos/citusdata/citus/stargazers","contributors_url":"https://api.github.com/repos/citusdata/citus/contributors","subscribers_url":"https://api.github.com/repos/citusdata/citus/subscribers","subscription_url":"https://api.github.com/repos/citusdata/citus/subscription","commits_url":"https://api.github.com/repos/citusdata/citus/commits{/sha}","git_commits_url":"https://api.github.com/repos/citusdata/citus/git/commits{/sha}","comments_url":"https://api.github.com/repos/citusdata/citus/comments{/number}","issue_comment_url":"https://api.github.com/repos/citusdata/citus/issues/comments{/number}","contents_url":"https://api.github.com/repos/citusdata/citus/contents/{+path}","compare_url":"https://api.github.com/repos/citusdata/citus/compare/{base}...{head}","merges_url":"https://api.github.com/repos/citusdata/citus/merges","archive_url":"https://api.github.com/repos/citusdata/citus/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/citusdata/citus/downloads","issues_url":"https://api.github.com/repos/citusdata/citus/issues{/number}","pulls_url":"https://api.github.com/repos/citusdata/citus/pulls{/number}","milestones_url":"https://api.github.com/repos/citusdata/citus/milestones{/number}","notifications_url":"https://api.github.com/repos/citusdata/citus/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/citusdata/citus/labels{/name}","releases_url":"https://api.github.com/repos/citusdata/citus/releases{/id}","deployments_url":"https://api.github.com/repos/citusdata/citus/deployments","created_at":"2016-02-01T21:58:39Z","updated_at":"2016-05-25T20:08:19Z","pushed_at":"2016-05-26T09:04:13Z","git_url":"git://github.com/citusdata/citus.git","ssh_url":"git@github.com:citusdata/citus.git","clone_url":"https://github.com/citusdata/citus.git","svn_url":"https://github.com/citusdata/citus","homepage":"https://www.citusdata.com","size":2538,"stargazers_count":1310,"watchers_count":1310,"language":"C","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":72,"mirror_url":null,"open_issues_count":361,"forks":72,"open_issues":361,"watchers":1310,"default_branch":"master"}},"base":{"label":"citusdata:master","ref":"master","sha":"ad61d4ae06d868796e1cbd67f67f6e8207ee0ae9","user":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars.githubusercontent.com/u/2545424?v=3","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"repo":{"id":50874442,"name":"citus","full_name":"citusdata/citus","owner":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars.githubusercontent.com/u/2545424?v=3","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/citusdata/citus","description":"Scalable PostgreSQL for real-time workloads","fork":false,"url":"https://api.github.com/repos/citusdata/citus","forks_url":"https://api.github.com/repos/citusdata/citus/forks","keys_url":"https://api.github.com/repos/citusdata/citus/keys{/key_id}","collaborators_url":"https://api.github.com/repos/citusdata/citus/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/citusdata/citus/teams","hooks_url":"https://api.github.com/repos/citusdata/citus/hooks","issue_events_url":"https://api.github.com/repos/citusdata/citus/issues/events{/number}","events_url":"https://api.github.com/repos/citusdata/citus/events","assignees_url":"https://api.github.com/repos/citusdata/citus/assignees{/user}","branches_url":"https://api.github.com/repos/citusdata/citus/branches{/branch}","tags_url":"https://api.github.com/repos/citusdata/citus/tags","blobs_url":"https://api.github.com/repos/citusdata/citus/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/citusdata/citus/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/citusdata/citus/git/refs{/sha}","trees_url":"https://api.github.com/repos/citusdata/citus/git/trees{/sha}","statuses_url":"https://api.github.com/repos/citusdata/citus/statuses/{sha}","languages_url":"https://api.github.com/repos/citusdata/citus/languages","stargazers_url":"https://api.github.com/repos/citusdata/citus/stargazers","contributors_url":"https://api.github.com/repos/citusdata/citus/contributors","subscribers_url":"https://api.github.com/repos/citusdata/citus/subscribers","subscription_url":"https://api.github.com/repos/citusdata/citus/subscription","commits_url":"https://api.github.com/repos/citusdata/citus/commits{/sha}","git_commits_url":"https://api.github.com/repos/citusdata/citus/git/commits{/sha}","comments_url":"https://api.github.com/repos/citusdata/citus/comments{/number}","issue_comment_url":"https://api.github.com/repos/citusdata/citus/issues/comments{/number}","contents_url":"https://api.github.com/repos/citusdata/citus/contents/{+path}","compare_url":"https://api.github.com/repos/citusdata/citus/compare/{base}...{head}","merges_url":"https://api.github.com/repos/citusdata/citus/merges","archive_url":"https://api.github.com/repos/citusdata/citus/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/citusdata/citus/downloads","issues_url":"https://api.github.com/repos/citusdata/citus/issues{/number}","pulls_url":"https://api.github.com/repos/citusdata/citus/pulls{/number}","milestones_url":"https://api.github.com/repos/citusdata/citus/milestones{/number}","notifications_url":"https://api.github.com/repos/citusdata/citus/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/citusdata/citus/labels{/name}","releases_url":"https://api.github.com/repos/citusdata/citus/releases{/id}","deployments_url":"https://api.github.com/repos/citusdata/citus/deployments","created_at":"2016-02-01T21:58:39Z","updated_at":"2016-05-25T20:08:19Z","pushed_at":"2016-05-26T09:04:13Z","git_url":"git://github.com/citusdata/citus.git","ssh_url":"git@github.com:citusdata/citus.git","clone_url":"https://github.com/citusdata/citus.git","svn_url":"https://github.com/citusdata/citus","homepage":"https://www.citusdata.com","size":2538,"stargazers_count":1310,"watchers_count":1310,"language":"C","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":72,"mirror_url":null,"open_issues_count":361,"forks":72,"open_issues":361,"watchers":1310,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/citusdata/citus/pulls/516"},"html":{"href":"https://github.com/citusdata/citus/pull/516"},"issue":{"href":"https://api.github.com/repos/citusdata/citus/issues/516"},"comments":{"href":"https://api.github.com/repos/citusdata/citus/issues/516/comments"},"review_comments":{"href":"https://api.github.com/repos/citusdata/citus/pulls/516/comments"},"review_comment":{"href":"https://api.github.com/repos/citusdata/citus/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/citusdata/citus/pulls/516/commits"},"statuses":{"href":"https://api.github.com/repos/citusdata/citus/statuses/03d5c6e4771d5b36f4882f6e67b3ead87740a565"}}}}
{"actor":{"display_login":"mtuncer"}}