{"action":"created","comment":{"url":"https://api.github.com/repos/prestodb/presto/pulls/comments/67504105","id":67504105,"diff_hunk":"@@ -225,6 +228,20 @@ public SystemSessionProperties(\n                         COLOCATED_JOIN,\n                         \"Experimental: Use a colocated join when possible\",\n                         featuresConfig.isColocatedJoinsEnabled(),\n+                        false),\n+                new PropertyMetadata<>(\n+                        OPERATOR_MEMORY_LIMIT_BEFORE_SPILL,\n+                        \"Experimental: Operator memory limit before spill\",\n+                        VARCHAR,\n+                        DataSize.class,\n+                        featuresConfig.getOperatorMemoryLimitBeforeSpill(),\n+                        false,\n+                        value -> DataSize.valueOf((String) value),\n+                        DataSize::toString),\n+                longSessionProperty(\n+                        MAX_ENTRIES_BEFORE_SPILL,\n+                        \"Experimental: Limit max number of entries before spill\",","path":"presto-main/src/main/java/com/facebook/presto/SystemSessionProperties.java","position":33,"original_position":33,"commit_id":"06b6369b573c17a3b226592e6cb72052eee65367","original_commit_id":"ffba1736e10205f4c2fd6c0a4638e69489d5672f","user":{"login":"sopel39","id":5989165,"avatar_url":"https://avatars.githubusercontent.com/u/5989165?v=3","gravatar_id":"","url":"https://api.github.com/users/sopel39","html_url":"https://github.com/sopel39","followers_url":"https://api.github.com/users/sopel39/followers","following_url":"https://api.github.com/users/sopel39/following{/other_user}","gists_url":"https://api.github.com/users/sopel39/gists{/gist_id}","starred_url":"https://api.github.com/users/sopel39/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sopel39/subscriptions","organizations_url":"https://api.github.com/users/sopel39/orgs","repos_url":"https://api.github.com/users/sopel39/repos","events_url":"https://api.github.com/users/sopel39/events{/privacy}","received_events_url":"https://api.github.com/users/sopel39/received_events","type":"User","site_admin":false},"body":"this isn't grammatically correct English, I think.","created_at":"2016-06-17T12:45:03Z","updated_at":"2016-06-17T12:45:03Z","html_url":"https://github.com/prestodb/presto/pull/5142#discussion_r67504105","pull_request_url":"https://api.github.com/repos/prestodb/presto/pulls/5142","_links":{"self":{"href":"https://api.github.com/repos/prestodb/presto/pulls/comments/67504105"},"html":{"href":"https://github.com/prestodb/presto/pull/5142#discussion_r67504105"},"pull_request":{"href":"https://api.github.com/repos/prestodb/presto/pulls/5142"}}},"pull_request":{"url":"https://api.github.com/repos/prestodb/presto/pulls/5142","id":68211155,"html_url":"https://github.com/prestodb/presto/pull/5142","diff_url":"https://github.com/prestodb/presto/pull/5142.diff","patch_url":"https://github.com/prestodb/presto/pull/5142.patch","issue_url":"https://api.github.com/repos/prestodb/presto/issues/5142","number":5142,"state":"open","locked":false,"title":"Spilling to disk for aggregation","user":{"login":"pnowojski","id":8957547,"avatar_url":"https://avatars.githubusercontent.com/u/8957547?v=3","gravatar_id":"","url":"https://api.github.com/users/pnowojski","html_url":"https://github.com/pnowojski","followers_url":"https://api.github.com/users/pnowojski/followers","following_url":"https://api.github.com/users/pnowojski/following{/other_user}","gists_url":"https://api.github.com/users/pnowojski/gists{/gist_id}","starred_url":"https://api.github.com/users/pnowojski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pnowojski/subscriptions","organizations_url":"https://api.github.com/users/pnowojski/orgs","repos_url":"https://api.github.com/users/pnowojski/repos","events_url":"https://api.github.com/users/pnowojski/events{/privacy}","received_events_url":"https://api.github.com/users/pnowojski/received_events","type":"User","site_admin":false},"body":"This code have some issues, but I think it's some starting point which is worth showing before resolving them. On our test cluster this PoC allowed to pass multiple queries which were previously failing due to lack of memory, for example:\r\n\r\n```\r\nselect avg(quantity) as aq, avg(extendedprice) as ae, avg(discount) as ad, avg(tax) as at from lineitem group by orderkey order by aq desc, ae desc limit 10;\r\n```\r\n\r\nSome of the issues with this code:\r\n\r\n1. it only supports one Spiller plugin in the class path \r\n2. decision what to spill is naive - spill everything and during spilling block aggregation operator\r\n3. Currently spillilng aggregations on array, map and row columns is not supported","created_at":"2016-04-28T12:23:49Z","updated_at":"2016-06-17T12:45:03Z","closed_at":null,"merged_at":null,"merge_commit_sha":"69489dcd06a40cc42d44c5eff188d9c381d809b4","assignee":{"login":"cberner","id":842688,"avatar_url":"https://avatars.githubusercontent.com/u/842688?v=3","gravatar_id":"","url":"https://api.github.com/users/cberner","html_url":"https://github.com/cberner","followers_url":"https://api.github.com/users/cberner/followers","following_url":"https://api.github.com/users/cberner/following{/other_user}","gists_url":"https://api.github.com/users/cberner/gists{/gist_id}","starred_url":"https://api.github.com/users/cberner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cberner/subscriptions","organizations_url":"https://api.github.com/users/cberner/orgs","repos_url":"https://api.github.com/users/cberner/repos","events_url":"https://api.github.com/users/cberner/events{/privacy}","received_events_url":"https://api.github.com/users/cberner/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/prestodb/presto/pulls/5142/commits","review_comments_url":"https://api.github.com/repos/prestodb/presto/pulls/5142/comments","review_comment_url":"https://api.github.com/repos/prestodb/presto/pulls/comments{/number}","comments_url":"https://api.github.com/repos/prestodb/presto/issues/5142/comments","statuses_url":"https://api.github.com/repos/prestodb/presto/statuses/06b6369b573c17a3b226592e6cb72052eee65367","head":{"label":"Teradata:spill","ref":"spill","sha":"06b6369b573c17a3b226592e6cb72052eee65367","user":{"login":"Teradata","id":1615979,"avatar_url":"https://avatars.githubusercontent.com/u/1615979?v=3","gravatar_id":"","url":"https://api.github.com/users/Teradata","html_url":"https://github.com/Teradata","followers_url":"https://api.github.com/users/Teradata/followers","following_url":"https://api.github.com/users/Teradata/following{/other_user}","gists_url":"https://api.github.com/users/Teradata/gists{/gist_id}","starred_url":"https://api.github.com/users/Teradata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Teradata/subscriptions","organizations_url":"https://api.github.com/users/Teradata/orgs","repos_url":"https://api.github.com/users/Teradata/repos","events_url":"https://api.github.com/users/Teradata/events{/privacy}","received_events_url":"https://api.github.com/users/Teradata/received_events","type":"Organization","site_admin":false},"repo":{"id":36903368,"name":"presto","full_name":"Teradata/presto","owner":{"login":"Teradata","id":1615979,"avatar_url":"https://avatars.githubusercontent.com/u/1615979?v=3","gravatar_id":"","url":"https://api.github.com/users/Teradata","html_url":"https://github.com/Teradata","followers_url":"https://api.github.com/users/Teradata/followers","following_url":"https://api.github.com/users/Teradata/following{/other_user}","gists_url":"https://api.github.com/users/Teradata/gists{/gist_id}","starred_url":"https://api.github.com/users/Teradata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Teradata/subscriptions","organizations_url":"https://api.github.com/users/Teradata/orgs","repos_url":"https://api.github.com/users/Teradata/repos","events_url":"https://api.github.com/users/Teradata/events{/privacy}","received_events_url":"https://api.github.com/users/Teradata/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/Teradata/presto","description":"Teradata's Certified Version of Presto -- A Distributed SQL Query Engine for Big Data","fork":true,"url":"https://api.github.com/repos/Teradata/presto","forks_url":"https://api.github.com/repos/Teradata/presto/forks","keys_url":"https://api.github.com/repos/Teradata/presto/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Teradata/presto/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Teradata/presto/teams","hooks_url":"https://api.github.com/repos/Teradata/presto/hooks","issue_events_url":"https://api.github.com/repos/Teradata/presto/issues/events{/number}","events_url":"https://api.github.com/repos/Teradata/presto/events","assignees_url":"https://api.github.com/repos/Teradata/presto/assignees{/user}","branches_url":"https://api.github.com/repos/Teradata/presto/branches{/branch}","tags_url":"https://api.github.com/repos/Teradata/presto/tags","blobs_url":"https://api.github.com/repos/Teradata/presto/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Teradata/presto/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Teradata/presto/git/refs{/sha}","trees_url":"https://api.github.com/repos/Teradata/presto/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Teradata/presto/statuses/{sha}","languages_url":"https://api.github.com/repos/Teradata/presto/languages","stargazers_url":"https://api.github.com/repos/Teradata/presto/stargazers","contributors_url":"https://api.github.com/repos/Teradata/presto/contributors","subscribers_url":"https://api.github.com/repos/Teradata/presto/subscribers","subscription_url":"https://api.github.com/repos/Teradata/presto/subscription","commits_url":"https://api.github.com/repos/Teradata/presto/commits{/sha}","git_commits_url":"https://api.github.com/repos/Teradata/presto/git/commits{/sha}","comments_url":"https://api.github.com/repos/Teradata/presto/comments{/number}","issue_comment_url":"https://api.github.com/repos/Teradata/presto/issues/comments{/number}","contents_url":"https://api.github.com/repos/Teradata/presto/contents/{+path}","compare_url":"https://api.github.com/repos/Teradata/presto/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Teradata/presto/merges","archive_url":"https://api.github.com/repos/Teradata/presto/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Teradata/presto/downloads","issues_url":"https://api.github.com/repos/Teradata/presto/issues{/number}","pulls_url":"https://api.github.com/repos/Teradata/presto/pulls{/number}","milestones_url":"https://api.github.com/repos/Teradata/presto/milestones{/number}","notifications_url":"https://api.github.com/repos/Teradata/presto/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Teradata/presto/labels{/name}","releases_url":"https://api.github.com/repos/Teradata/presto/releases{/id}","deployments_url":"https://api.github.com/repos/Teradata/presto/deployments","created_at":"2015-06-05T00:38:37Z","updated_at":"2016-06-14T07:29:49Z","pushed_at":"2016-06-17T11:49:29Z","git_url":"git://github.com/Teradata/presto.git","ssh_url":"git@github.com:Teradata/presto.git","clone_url":"https://github.com/Teradata/presto.git","svn_url":"https://github.com/Teradata/presto","homepage":"http://www.teradata.com/presto","size":142247,"stargazers_count":51,"watchers_count":51,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":8,"mirror_url":null,"open_issues_count":12,"forks":8,"open_issues":12,"watchers":51,"default_branch":"master"}},"base":{"label":"prestodb:master","ref":"master","sha":"912934766c4de6a5fb1972a927d19993ec21c6b8","user":{"login":"prestodb","id":6882181,"avatar_url":"https://avatars.githubusercontent.com/u/6882181?v=3","gravatar_id":"","url":"https://api.github.com/users/prestodb","html_url":"https://github.com/prestodb","followers_url":"https://api.github.com/users/prestodb/followers","following_url":"https://api.github.com/users/prestodb/following{/other_user}","gists_url":"https://api.github.com/users/prestodb/gists{/gist_id}","starred_url":"https://api.github.com/users/prestodb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/prestodb/subscriptions","organizations_url":"https://api.github.com/users/prestodb/orgs","repos_url":"https://api.github.com/users/prestodb/repos","events_url":"https://api.github.com/users/prestodb/events{/privacy}","received_events_url":"https://api.github.com/users/prestodb/received_events","type":"Organization","site_admin":false},"repo":{"id":5349565,"name":"presto","full_name":"prestodb/presto","owner":{"login":"prestodb","id":6882181,"avatar_url":"https://avatars.githubusercontent.com/u/6882181?v=3","gravatar_id":"","url":"https://api.github.com/users/prestodb","html_url":"https://github.com/prestodb","followers_url":"https://api.github.com/users/prestodb/followers","following_url":"https://api.github.com/users/prestodb/following{/other_user}","gists_url":"https://api.github.com/users/prestodb/gists{/gist_id}","starred_url":"https://api.github.com/users/prestodb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/prestodb/subscriptions","organizations_url":"https://api.github.com/users/prestodb/orgs","repos_url":"https://api.github.com/users/prestodb/repos","events_url":"https://api.github.com/users/prestodb/events{/privacy}","received_events_url":"https://api.github.com/users/prestodb/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/prestodb/presto","description":"Distributed SQL query engine for big data","fork":false,"url":"https://api.github.com/repos/prestodb/presto","forks_url":"https://api.github.com/repos/prestodb/presto/forks","keys_url":"https://api.github.com/repos/prestodb/presto/keys{/key_id}","collaborators_url":"https://api.github.com/repos/prestodb/presto/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/prestodb/presto/teams","hooks_url":"https://api.github.com/repos/prestodb/presto/hooks","issue_events_url":"https://api.github.com/repos/prestodb/presto/issues/events{/number}","events_url":"https://api.github.com/repos/prestodb/presto/events","assignees_url":"https://api.github.com/repos/prestodb/presto/assignees{/user}","branches_url":"https://api.github.com/repos/prestodb/presto/branches{/branch}","tags_url":"https://api.github.com/repos/prestodb/presto/tags","blobs_url":"https://api.github.com/repos/prestodb/presto/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/prestodb/presto/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/prestodb/presto/git/refs{/sha}","trees_url":"https://api.github.com/repos/prestodb/presto/git/trees{/sha}","statuses_url":"https://api.github.com/repos/prestodb/presto/statuses/{sha}","languages_url":"https://api.github.com/repos/prestodb/presto/languages","stargazers_url":"https://api.github.com/repos/prestodb/presto/stargazers","contributors_url":"https://api.github.com/repos/prestodb/presto/contributors","subscribers_url":"https://api.github.com/repos/prestodb/presto/subscribers","subscription_url":"https://api.github.com/repos/prestodb/presto/subscription","commits_url":"https://api.github.com/repos/prestodb/presto/commits{/sha}","git_commits_url":"https://api.github.com/repos/prestodb/presto/git/commits{/sha}","comments_url":"https://api.github.com/repos/prestodb/presto/comments{/number}","issue_comment_url":"https://api.github.com/repos/prestodb/presto/issues/comments{/number}","contents_url":"https://api.github.com/repos/prestodb/presto/contents/{+path}","compare_url":"https://api.github.com/repos/prestodb/presto/compare/{base}...{head}","merges_url":"https://api.github.com/repos/prestodb/presto/merges","archive_url":"https://api.github.com/repos/prestodb/presto/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/prestodb/presto/downloads","issues_url":"https://api.github.com/repos/prestodb/presto/issues{/number}","pulls_url":"https://api.github.com/repos/prestodb/presto/pulls{/number}","milestones_url":"https://api.github.com/repos/prestodb/presto/milestones{/number}","notifications_url":"https://api.github.com/repos/prestodb/presto/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/prestodb/presto/labels{/name}","releases_url":"https://api.github.com/repos/prestodb/presto/releases{/id}","deployments_url":"https://api.github.com/repos/prestodb/presto/deployments","created_at":"2012-08-09T01:03:37Z","updated_at":"2016-06-17T10:48:35Z","pushed_at":"2016-06-17T11:49:29Z","git_url":"git://github.com/prestodb/presto.git","ssh_url":"git@github.com:prestodb/presto.git","clone_url":"https://github.com/prestodb/presto.git","svn_url":"https://github.com/prestodb/presto","homepage":"https://prestodb.io","size":154910,"stargazers_count":4827,"watchers_count":4827,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1443,"mirror_url":null,"open_issues_count":567,"forks":1443,"open_issues":567,"watchers":4827,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/prestodb/presto/pulls/5142"},"html":{"href":"https://github.com/prestodb/presto/pull/5142"},"issue":{"href":"https://api.github.com/repos/prestodb/presto/issues/5142"},"comments":{"href":"https://api.github.com/repos/prestodb/presto/issues/5142/comments"},"review_comments":{"href":"https://api.github.com/repos/prestodb/presto/pulls/5142/comments"},"review_comment":{"href":"https://api.github.com/repos/prestodb/presto/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/prestodb/presto/pulls/5142/commits"},"statuses":{"href":"https://api.github.com/repos/prestodb/presto/statuses/06b6369b573c17a3b226592e6cb72052eee65367"}}}}
{"actor":{"display_login":"sopel39"}}