{"action":"created","issue":{"url":"https://api.github.com/repos/apache/carbondata/issues/2823","repository_url":"https://api.github.com/repos/apache/carbondata","labels_url":"https://api.github.com/repos/apache/carbondata/issues/2823/labels{/name}","comments_url":"https://api.github.com/repos/apache/carbondata/issues/2823/comments","events_url":"https://api.github.com/repos/apache/carbondata/issues/2823/events","html_url":"https://github.com/apache/carbondata/pull/2823","id":370605774,"node_id":"MDExOlB1bGxSZXF1ZXN0MjIzMjQ0OTg1","number":2823,"title":"[CARBONDATA-3015]  Support Lazy load in carbon vector","user":{"login":"ravipesala","id":8134086,"node_id":"MDQ6VXNlcjgxMzQwODY=","avatar_url":"https://avatars2.githubusercontent.com/u/8134086?v=4","gravatar_id":"","url":"https://api.github.com/users/ravipesala","html_url":"https://github.com/ravipesala","followers_url":"https://api.github.com/users/ravipesala/followers","following_url":"https://api.github.com/users/ravipesala/following{/other_user}","gists_url":"https://api.github.com/users/ravipesala/gists{/gist_id}","starred_url":"https://api.github.com/users/ravipesala/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ravipesala/subscriptions","organizations_url":"https://api.github.com/users/ravipesala/orgs","repos_url":"https://api.github.com/users/ravipesala/repos","events_url":"https://api.github.com/users/ravipesala/events{/privacy}","received_events_url":"https://api.github.com/users/ravipesala/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":47,"created_at":"2018-10-16T13:13:34Z","updated_at":"2018-10-26T17:12:47Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/carbondata/pulls/2823","html_url":"https://github.com/apache/carbondata/pull/2823","diff_url":"https://github.com/apache/carbondata/pull/2823.diff","patch_url":"https://github.com/apache/carbondata/pull/2823.patch"},"body":"This PR depends on PR https://github.com/apache/carbondata/pull/2822\r\n\r\nEven though we prune the pages as per min/max there is a high chance of false positives in case of filters on high cardinality columns. So to avoid that we can use the lazy loading design. It does not read/decompresses data and fill the vector immediately when the call comes for data filling from spark/presto.\r\nFirst only reads the required filter columns give back to execution engine, execution engine starts filtering on the filtered column vector and if it finds some data need to be read from projection columns then only it starts reads the projection columns and fills the vector on demand. It is the concept of presto and same is integrated with spark 2.3. Older versions of spark cannot use this advantage as ColumnVector interfaces are non-extendable.\r\nFor the above purpose added new classes 'LazyBlockletLoad' and 'LazyPageLoad' and changed the carbon vector interfaces.\r\n\r\n\r\nBe sure to do all of the following checklist to help us incorporate \r\nyour contribution quickly and easily:\r\n\r\n - [ ] Any interfaces changed?\r\n \r\n - [ ] Any backward compatibility impacted?\r\n \r\n - [ ] Document update required?\r\n\r\n - [ ] Testing done\r\n        Please provide details on \r\n        - Whether new unit test cases have been added or why no new tests are required?\r\n        - How it is tested? Please attach test report.\r\n        - Is it a performance related change? Please attach the performance test report.\r\n        - Any additional information to help reviewers in testing this change.\r\n       \r\n - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. \r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/carbondata/issues/comments/433479261","html_url":"https://github.com/apache/carbondata/pull/2823#issuecomment-433479261","issue_url":"https://api.github.com/repos/apache/carbondata/issues/2823","id":433479261,"node_id":"MDEyOklzc3VlQ29tbWVudDQzMzQ3OTI2MQ==","user":{"login":"CarbonDataQA","id":24402133,"node_id":"MDQ6VXNlcjI0NDAyMTMz","avatar_url":"https://avatars0.githubusercontent.com/u/24402133?v=4","gravatar_id":"","url":"https://api.github.com/users/CarbonDataQA","html_url":"https://github.com/CarbonDataQA","followers_url":"https://api.github.com/users/CarbonDataQA/followers","following_url":"https://api.github.com/users/CarbonDataQA/following{/other_user}","gists_url":"https://api.github.com/users/CarbonDataQA/gists{/gist_id}","starred_url":"https://api.github.com/users/CarbonDataQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/CarbonDataQA/subscriptions","organizations_url":"https://api.github.com/users/CarbonDataQA/orgs","repos_url":"https://api.github.com/users/CarbonDataQA/repos","events_url":"https://api.github.com/users/CarbonDataQA/events{/privacy}","received_events_url":"https://api.github.com/users/CarbonDataQA/received_events","type":"User","site_admin":false},"created_at":"2018-10-26T17:12:47Z","updated_at":"2018-10-26T17:12:47Z","author_association":"NONE","body":"Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9324/\n"}}
{"actor":{"display_login":"CarbonDataQA"}}