{"action":"created","comment":{"url":"https://api.github.com/repos/apache/incubator-pinot/pulls/comments/243365133","pull_request_review_id":187116357,"id":243365133,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDI0MzM2NTEzMw==","diff_hunk":"@@ -27,6 +27,9 @@\n public interface DataTable {\n   String EXCEPTION_METADATA_KEY = \"Exception\";\n   String NUM_DOCS_SCANNED_METADATA_KEY = \"numDocsScanned\";\n+  String NUM_INDICES_LOADED_KEY = \"numIndicesLoaded\";","path":"pinot-common/src/main/java/com/linkedin/pinot/common/utils/DataTable.java","position":4,"original_position":4,"commit_id":"0badef06b99c0b72f71f732cd29753c5a6827a08","original_commit_id":"0badef06b99c0b72f71f732cd29753c5a6827a08","user":{"login":"mayankshriv","id":12615885,"node_id":"MDQ6VXNlcjEyNjE1ODg1","avatar_url":"https://avatars0.githubusercontent.com/u/12615885?v=4","gravatar_id":"","url":"https://api.github.com/users/mayankshriv","html_url":"https://github.com/mayankshriv","followers_url":"https://api.github.com/users/mayankshriv/followers","following_url":"https://api.github.com/users/mayankshriv/following{/other_user}","gists_url":"https://api.github.com/users/mayankshriv/gists{/gist_id}","starred_url":"https://api.github.com/users/mayankshriv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mayankshriv/subscriptions","organizations_url":"https://api.github.com/users/mayankshriv/orgs","repos_url":"https://api.github.com/users/mayankshriv/repos","events_url":"https://api.github.com/users/mayankshriv/events{/privacy}","received_events_url":"https://api.github.com/users/mayankshriv/received_events","type":"User","site_admin":false},"body":"If this does not represent num_indices, then could we name it appropriately?","created_at":"2018-12-20T18:01:23Z","updated_at":"2018-12-20T18:24:55Z","html_url":"https://github.com/apache/incubator-pinot/pull/3511#discussion_r243365133","pull_request_url":"https://api.github.com/repos/apache/incubator-pinot/pulls/3511","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/apache/incubator-pinot/pulls/comments/243365133"},"html":{"href":"https://github.com/apache/incubator-pinot/pull/3511#discussion_r243365133"},"pull_request":{"href":"https://api.github.com/repos/apache/incubator-pinot/pulls/3511"}}},"pull_request":{"url":"https://api.github.com/repos/apache/incubator-pinot/pulls/3511","id":231689283,"node_id":"MDExOlB1bGxSZXF1ZXN0MjMxNjg5Mjgz","html_url":"https://github.com/apache/incubator-pinot/pull/3511","diff_url":"https://github.com/apache/incubator-pinot/pull/3511.diff","patch_url":"https://github.com/apache/incubator-pinot/pull/3511.patch","issue_url":"https://api.github.com/repos/apache/incubator-pinot/issues/3511","number":3511,"state":"open","locked":false,"title":"[PINOT-7370] Track bytes read per query","user":{"login":"sunithabeeram","id":3123798,"node_id":"MDQ6VXNlcjMxMjM3OTg=","avatar_url":"https://avatars2.githubusercontent.com/u/3123798?v=4","gravatar_id":"","url":"https://api.github.com/users/sunithabeeram","html_url":"https://github.com/sunithabeeram","followers_url":"https://api.github.com/users/sunithabeeram/followers","following_url":"https://api.github.com/users/sunithabeeram/following{/other_user}","gists_url":"https://api.github.com/users/sunithabeeram/gists{/gist_id}","starred_url":"https://api.github.com/users/sunithabeeram/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sunithabeeram/subscriptions","organizations_url":"https://api.github.com/users/sunithabeeram/orgs","repos_url":"https://api.github.com/users/sunithabeeram/repos","events_url":"https://api.github.com/users/sunithabeeram/events{/privacy}","received_events_url":"https://api.github.com/users/sunithabeeram/received_events","type":"User","site_admin":false},"body":"WIP - Code is mostly done; need to validate with more tests and ensure all paths are covered.\r\n\r\nThis change introduces a few main changes:\r\n- Ability to track the size of the inverted-index posting list size. This is achieved by adding a wrapper for ImmutableRoaringBitMap; the wrapper just caches the bitmap size (its immutable after all!).\r\n- Tracks the bytes read in the scan-based filters. The current implementation is OK for dictionary based scans; For string/byte raw store, it might be off.\r\n- Tracks byes projected via DataBlockCache/DataFetcher classes. ","created_at":"2018-11-17T00:13:20Z","updated_at":"2018-12-20T18:24:55Z","closed_at":null,"merged_at":null,"merge_commit_sha":"93bdfed210e5eb6e921f62c0173cf7af01879618","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/incubator-pinot/pulls/3511/commits","review_comments_url":"https://api.github.com/repos/apache/incubator-pinot/pulls/3511/comments","review_comment_url":"https://api.github.com/repos/apache/incubator-pinot/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/incubator-pinot/issues/3511/comments","statuses_url":"https://api.github.com/repos/apache/incubator-pinot/statuses/0badef06b99c0b72f71f732cd29753c5a6827a08","head":{"label":"apache:trackIndexWork","ref":"trackIndexWork","sha":"0badef06b99c0b72f71f732cd29753c5a6827a08","user":{"login":"apache","id":47359,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ3MzU5","avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":19961085,"node_id":"MDEwOlJlcG9zaXRvcnkxOTk2MTA4NQ==","name":"incubator-pinot","full_name":"apache/incubator-pinot","private":false,"owner":{"login":"apache","id":47359,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ3MzU5","avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/apache/incubator-pinot","description":"Apache Pinot (Incubating) - A realtime distributed OLAP datastore","fork":false,"url":"https://api.github.com/repos/apache/incubator-pinot","forks_url":"https://api.github.com/repos/apache/incubator-pinot/forks","keys_url":"https://api.github.com/repos/apache/incubator-pinot/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/incubator-pinot/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/incubator-pinot/teams","hooks_url":"https://api.github.com/repos/apache/incubator-pinot/hooks","issue_events_url":"https://api.github.com/repos/apache/incubator-pinot/issues/events{/number}","events_url":"https://api.github.com/repos/apache/incubator-pinot/events","assignees_url":"https://api.github.com/repos/apache/incubator-pinot/assignees{/user}","branches_url":"https://api.github.com/repos/apache/incubator-pinot/branches{/branch}","tags_url":"https://api.github.com/repos/apache/incubator-pinot/tags","blobs_url":"https://api.github.com/repos/apache/incubator-pinot/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/incubator-pinot/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/incubator-pinot/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/incubator-pinot/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/incubator-pinot/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/incubator-pinot/languages","stargazers_url":"https://api.github.com/repos/apache/incubator-pinot/stargazers","contributors_url":"https://api.github.com/repos/apache/incubator-pinot/contributors","subscribers_url":"https://api.github.com/repos/apache/incubator-pinot/subscribers","subscription_url":"https://api.github.com/repos/apache/incubator-pinot/subscription","commits_url":"https://api.github.com/repos/apache/incubator-pinot/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/incubator-pinot/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/incubator-pinot/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/incubator-pinot/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/incubator-pinot/contents/{+path}","compare_url":"https://api.github.com/repos/apache/incubator-pinot/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/incubator-pinot/merges","archive_url":"https://api.github.com/repos/apache/incubator-pinot/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/incubator-pinot/downloads","issues_url":"https://api.github.com/repos/apache/incubator-pinot/issues{/number}","pulls_url":"https://api.github.com/repos/apache/incubator-pinot/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/incubator-pinot/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/incubator-pinot/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/incubator-pinot/labels{/name}","releases_url":"https://api.github.com/repos/apache/incubator-pinot/releases{/id}","deployments_url":"https://api.github.com/repos/apache/incubator-pinot/deployments","created_at":"2014-05-19T23:27:48Z","updated_at":"2018-12-20T07:37:51Z","pushed_at":"2018-12-20T03:35:40Z","git_url":"git://github.com/apache/incubator-pinot.git","ssh_url":"git@github.com:apache/incubator-pinot.git","clone_url":"https://github.com/apache/incubator-pinot.git","svn_url":"https://github.com/apache/incubator-pinot","homepage":"https://pinot.apache.org","size":159378,"stargazers_count":2065,"watchers_count":2065,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":375,"mirror_url":null,"archived":false,"open_issues_count":199,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":375,"open_issues":199,"watchers":2065,"default_branch":"master"}},"base":{"label":"apache:master","ref":"master","sha":"39e3360d5f63eb3b5bddc878706e174a2901b6c7","user":{"login":"apache","id":47359,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ3MzU5","avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":19961085,"node_id":"MDEwOlJlcG9zaXRvcnkxOTk2MTA4NQ==","name":"incubator-pinot","full_name":"apache/incubator-pinot","private":false,"owner":{"login":"apache","id":47359,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ3MzU5","avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/apache/incubator-pinot","description":"Apache Pinot (Incubating) - A realtime distributed OLAP datastore","fork":false,"url":"https://api.github.com/repos/apache/incubator-pinot","forks_url":"https://api.github.com/repos/apache/incubator-pinot/forks","keys_url":"https://api.github.com/repos/apache/incubator-pinot/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/incubator-pinot/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/incubator-pinot/teams","hooks_url":"https://api.github.com/repos/apache/incubator-pinot/hooks","issue_events_url":"https://api.github.com/repos/apache/incubator-pinot/issues/events{/number}","events_url":"https://api.github.com/repos/apache/incubator-pinot/events","assignees_url":"https://api.github.com/repos/apache/incubator-pinot/assignees{/user}","branches_url":"https://api.github.com/repos/apache/incubator-pinot/branches{/branch}","tags_url":"https://api.github.com/repos/apache/incubator-pinot/tags","blobs_url":"https://api.github.com/repos/apache/incubator-pinot/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/incubator-pinot/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/incubator-pinot/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/incubator-pinot/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/incubator-pinot/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/incubator-pinot/languages","stargazers_url":"https://api.github.com/repos/apache/incubator-pinot/stargazers","contributors_url":"https://api.github.com/repos/apache/incubator-pinot/contributors","subscribers_url":"https://api.github.com/repos/apache/incubator-pinot/subscribers","subscription_url":"https://api.github.com/repos/apache/incubator-pinot/subscription","commits_url":"https://api.github.com/repos/apache/incubator-pinot/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/incubator-pinot/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/incubator-pinot/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/incubator-pinot/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/incubator-pinot/contents/{+path}","compare_url":"https://api.github.com/repos/apache/incubator-pinot/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/incubator-pinot/merges","archive_url":"https://api.github.com/repos/apache/incubator-pinot/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/incubator-pinot/downloads","issues_url":"https://api.github.com/repos/apache/incubator-pinot/issues{/number}","pulls_url":"https://api.github.com/repos/apache/incubator-pinot/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/incubator-pinot/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/incubator-pinot/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/incubator-pinot/labels{/name}","releases_url":"https://api.github.com/repos/apache/incubator-pinot/releases{/id}","deployments_url":"https://api.github.com/repos/apache/incubator-pinot/deployments","created_at":"2014-05-19T23:27:48Z","updated_at":"2018-12-20T07:37:51Z","pushed_at":"2018-12-20T03:35:40Z","git_url":"git://github.com/apache/incubator-pinot.git","ssh_url":"git@github.com:apache/incubator-pinot.git","clone_url":"https://github.com/apache/incubator-pinot.git","svn_url":"https://github.com/apache/incubator-pinot","homepage":"https://pinot.apache.org","size":159378,"stargazers_count":2065,"watchers_count":2065,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":375,"mirror_url":null,"archived":false,"open_issues_count":199,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":375,"open_issues":199,"watchers":2065,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/incubator-pinot/pulls/3511"},"html":{"href":"https://github.com/apache/incubator-pinot/pull/3511"},"issue":{"href":"https://api.github.com/repos/apache/incubator-pinot/issues/3511"},"comments":{"href":"https://api.github.com/repos/apache/incubator-pinot/issues/3511/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/incubator-pinot/pulls/3511/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/incubator-pinot/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/incubator-pinot/pulls/3511/commits"},"statuses":{"href":"https://api.github.com/repos/apache/incubator-pinot/statuses/0badef06b99c0b72f71f732cd29753c5a6827a08"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"mayankshriv"}}