{"action":"created","issue":{"url":"https://api.github.com/repos/prestodb/presto/issues/6232","repository_url":"https://api.github.com/repos/prestodb/presto","labels_url":"https://api.github.com/repos/prestodb/presto/issues/6232/labels{/name}","comments_url":"https://api.github.com/repos/prestodb/presto/issues/6232/comments","events_url":"https://api.github.com/repos/prestodb/presto/issues/6232/events","html_url":"https://github.com/prestodb/presto/pull/6232","id":180306371,"number":6232,"title":"Use columnar processing in projection operator without filter","user":{"login":"sopel39","id":5989165,"avatar_url":"https://avatars.githubusercontent.com/u/5989165?v=3","gravatar_id":"","url":"https://api.github.com/users/sopel39","html_url":"https://github.com/sopel39","followers_url":"https://api.github.com/users/sopel39/followers","following_url":"https://api.github.com/users/sopel39/following{/other_user}","gists_url":"https://api.github.com/users/sopel39/gists{/gist_id}","starred_url":"https://api.github.com/users/sopel39/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sopel39/subscriptions","organizations_url":"https://api.github.com/users/sopel39/orgs","repos_url":"https://api.github.com/users/sopel39/repos","events_url":"https://api.github.com/users/sopel39/events{/privacy}","received_events_url":"https://api.github.com/users/sopel39/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/prestodb/presto/labels/CLA%20Signed","name":"CLA Signed","color":"ededed"}],"state":"open","locked":false,"assignee":{"login":"dain","id":58298,"avatar_url":"https://avatars.githubusercontent.com/u/58298?v=3","gravatar_id":"","url":"https://api.github.com/users/dain","html_url":"https://github.com/dain","followers_url":"https://api.github.com/users/dain/followers","following_url":"https://api.github.com/users/dain/following{/other_user}","gists_url":"https://api.github.com/users/dain/gists{/gist_id}","starred_url":"https://api.github.com/users/dain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dain/subscriptions","organizations_url":"https://api.github.com/users/dain/orgs","repos_url":"https://api.github.com/users/dain/repos","events_url":"https://api.github.com/users/dain/events{/privacy}","received_events_url":"https://api.github.com/users/dain/received_events","type":"User","site_admin":false},"assignees":[{"login":"dain","id":58298,"avatar_url":"https://avatars.githubusercontent.com/u/58298?v=3","gravatar_id":"","url":"https://api.github.com/users/dain","html_url":"https://github.com/dain","followers_url":"https://api.github.com/users/dain/followers","following_url":"https://api.github.com/users/dain/following{/other_user}","gists_url":"https://api.github.com/users/dain/gists{/gist_id}","starred_url":"https://api.github.com/users/dain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dain/subscriptions","organizations_url":"https://api.github.com/users/dain/orgs","repos_url":"https://api.github.com/users/dain/repos","events_url":"https://api.github.com/users/dain/events{/privacy}","received_events_url":"https://api.github.com/users/dain/received_events","type":"User","site_admin":false},{"login":"losipiuk","id":890111,"avatar_url":"https://avatars.githubusercontent.com/u/890111?v=3","gravatar_id":"","url":"https://api.github.com/users/losipiuk","html_url":"https://github.com/losipiuk","followers_url":"https://api.github.com/users/losipiuk/followers","following_url":"https://api.github.com/users/losipiuk/following{/other_user}","gists_url":"https://api.github.com/users/losipiuk/gists{/gist_id}","starred_url":"https://api.github.com/users/losipiuk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/losipiuk/subscriptions","organizations_url":"https://api.github.com/users/losipiuk/orgs","repos_url":"https://api.github.com/users/losipiuk/repos","events_url":"https://api.github.com/users/losipiuk/events{/privacy}","received_events_url":"https://api.github.com/users/losipiuk/received_events","type":"User","site_admin":false}],"milestone":null,"comments":14,"created_at":"2016-09-30T13:40:11Z","updated_at":"2016-10-25T19:16:40Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/prestodb/presto/pulls/6232","html_url":"https://github.com/prestodb/presto/pull/6232","diff_url":"https://github.com/prestodb/presto/pull/6232.diff","patch_url":"https://github.com/prestodb/presto/pull/6232.patch"},"body":"When there is no filter condition in `FilterAndProjectOperator` it is\r\noptimal to use columnar version of `PageProcessor` so that for\r\nidentity expressions no data is copied. Additionally, this\r\noptimizes memory usage when `FilterAndProjectOperator` output\r\nis used in `InnerJoin` as build table.\r\n\r\nFYI: @dain @martint "},"comment":{"url":"https://api.github.com/repos/prestodb/presto/issues/comments/256147199","html_url":"https://github.com/prestodb/presto/pull/6232#issuecomment-256147199","issue_url":"https://api.github.com/repos/prestodb/presto/issues/6232","id":256147199,"user":{"login":"dain","id":58298,"avatar_url":"https://avatars.githubusercontent.com/u/58298?v=3","gravatar_id":"","url":"https://api.github.com/users/dain","html_url":"https://github.com/dain","followers_url":"https://api.github.com/users/dain/followers","following_url":"https://api.github.com/users/dain/following{/other_user}","gists_url":"https://api.github.com/users/dain/gists{/gist_id}","starred_url":"https://api.github.com/users/dain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dain/subscriptions","organizations_url":"https://api.github.com/users/dain/orgs","repos_url":"https://api.github.com/users/dain/repos","events_url":"https://api.github.com/users/dain/events{/privacy}","received_events_url":"https://api.github.com/users/dain/received_events","type":"User","site_admin":false},"created_at":"2016-10-25T19:16:39Z","updated_at":"2016-10-25T19:16:39Z","body":"Instead of this smaller change, I think we should switch to always using columnar, and only use the row oriented filter/project when we have a row oriented table scan.  As @fmeiser pointed out, the column oriented version is always faster (except for lazy row oriented datasources), and the reason we haven't flipped the default is the current code does not handle the expansion of pages in the columnar processor.  In the page processor we check the page size after each row is processes, and when it is the right size, we stop processing.\r\n\r\nFor columnar processing, I propose we do something similar to the row processor.  First we change the columnar processor to run over a fixed number of positions, say 1024, for each call.  Then after each column is projected we check the size, and if it is too big, we permanently reduce the number by half for that operator.  That way we only reduce when we find a problem, and at the worse case we end up processing a single 1024 segment twice.\r\n\r\nWhat do you think?"}}
{"actor":{"display_login":"dain"}}