{"action":"created","comment":{"url":"https://api.github.com/repos/palantir/atlasdb/pulls/comments/206864547","pull_request_review_id":142358375,"id":206864547,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIwNjg2NDU0Nw==","diff_hunk":"@@ -684,9 +686,26 @@ private void cellsCleanedUp() {\n                     line(\"for (Cell cell : cells) {\"); {\n                         line(\"rows.add(\", StreamIdxRow, \".BYTES_HYDRATOR.hydrateFromBytes(cell.getRowName()));\");\n                     } line(\"}\");\n-                    line(\"Multimap<\", StreamIdxRow, \", \", StreamIdxColumnValue, \"> rowsInDb = usersIndex.getRowsMultimap(rows);\");\n-                    line(\"Set<\", StreamId, \"> toDelete = Sets.newHashSetWithExpectedSize(rows.size() - rowsInDb.keySet().size());\");\n-                    line(\"for (\", StreamIdxRow, \" rowToDelete : Sets.difference(rows, rowsInDb.keySet())) {\"); {\n+\n+                    line(\"BatchColumnRangeSelection oneColumn = BatchColumnRangeSelection.create(\");\n+                    line(\"        PtBytes.EMPTY_BYTE_ARRAY, PtBytes.EMPTY_BYTE_ARRAY, 1);\");\n+                    line(\"Map<\", StreamIdxRow, \", BatchingVisitable<\", StreamIdxColumnValue, \">> existentRows\");\n+                    line(\"        = usersIndex.getRowsColumnRange(rows, oneColumn);\");\n+\n+                    line(\"Set<\", StreamIdxRow, \"> rowsInDb = Sets.newHashSetWithExpectedSize(cells.size());\");\n+\n+                    line(\"for (Map.Entry<\", StreamIdxRow, \", BatchingVisitable<\", StreamIdxColumnValue, \">> rowVisitable\");\n+                    line(\"        : existentRows.entrySet())\"); {","path":"atlasdb-client/src/main/java/com/palantir/atlasdb/table/description/render/StreamStoreRenderer.java","position":null,"original_position":36,"commit_id":"899e789e2d4afa35e2cc7539c4ae68bf4835e0eb","original_commit_id":"b158edabf526ed231813f7a59e0ed76c48cd80e3","user":{"login":"jeremyk-91","id":5955510,"node_id":"MDQ6VXNlcjU5NTU1MTA=","avatar_url":"https://avatars2.githubusercontent.com/u/5955510?v=4","gravatar_id":"","url":"https://api.github.com/users/jeremyk-91","html_url":"https://github.com/jeremyk-91","followers_url":"https://api.github.com/users/jeremyk-91/followers","following_url":"https://api.github.com/users/jeremyk-91/following{/other_user}","gists_url":"https://api.github.com/users/jeremyk-91/gists{/gist_id}","starred_url":"https://api.github.com/users/jeremyk-91/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeremyk-91/subscriptions","organizations_url":"https://api.github.com/users/jeremyk-91/orgs","repos_url":"https://api.github.com/users/jeremyk-91/repos","events_url":"https://api.github.com/users/jeremyk-91/events{/privacy}","received_events_url":"https://api.github.com/users/jeremyk-91/received_events","type":"User","site_admin":false},"body":"Good call - the indentation is taken care of by the braces in the raw code-strings being there or not there. I left this out by accident (but the tests still passed, because the scope of the for loop happened to be 1 statement. New version should be OK (see generated code)","created_at":"2018-08-01T12:47:39Z","updated_at":"2018-08-01T12:50:52Z","html_url":"https://github.com/palantir/atlasdb/pull/3423#discussion_r206864547","pull_request_url":"https://api.github.com/repos/palantir/atlasdb/pulls/3423","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/palantir/atlasdb/pulls/comments/206864547"},"html":{"href":"https://github.com/palantir/atlasdb/pull/3423#discussion_r206864547"},"pull_request":{"href":"https://api.github.com/repos/palantir/atlasdb/pulls/3423"}},"in_reply_to_id":206344439},"pull_request":{"url":"https://api.github.com/repos/palantir/atlasdb/pulls/3423","id":204449818,"node_id":"MDExOlB1bGxSZXF1ZXN0MjA0NDQ5ODE4","html_url":"https://github.com/palantir/atlasdb/pull/3423","diff_url":"https://github.com/palantir/atlasdb/pull/3423.diff","patch_url":"https://github.com/palantir/atlasdb/pull/3423.patch","issue_url":"https://api.github.com/repos/palantir/atlasdb/issues/3423","number":3423,"state":"open","locked":false,"title":"getRowsColumnRange for IndexCleanupTasks","user":{"login":"jeremyk-91","id":5955510,"node_id":"MDQ6VXNlcjU5NTU1MTA=","avatar_url":"https://avatars2.githubusercontent.com/u/5955510?v=4","gravatar_id":"","url":"https://api.github.com/users/jeremyk-91","html_url":"https://github.com/jeremyk-91","followers_url":"https://api.github.com/users/jeremyk-91/followers","following_url":"https://api.github.com/users/jeremyk-91/following{/other_user}","gists_url":"https://api.github.com/users/jeremyk-91/gists{/gist_id}","starred_url":"https://api.github.com/users/jeremyk-91/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeremyk-91/subscriptions","organizations_url":"https://api.github.com/users/jeremyk-91/orgs","repos_url":"https://api.github.com/users/jeremyk-91/repos","events_url":"https://api.github.com/users/jeremyk-91/events{/privacy}","received_events_url":"https://api.github.com/users/jeremyk-91/received_events","type":"User","site_admin":false},"body":"**Goals (and why)**:\r\n- Stream store index cleanup tasks read the whole row, which can be wide in certain cases, to check if _any_ reference to the stream remains (which gates deleting the actual stream blocks). This is inefficient; reading one column should suffice.\r\n\r\n**Implementation Description (bullets)**:\r\n- Use `getRowsColumnRange()` instead of `getRowsMultimap()` to load one column per row\r\n- Regenerate schemas.\r\n\r\n**Testing (What was existing testing like?  What have you done to improve it?)**:\r\n- This code path is tested through `TargetedSweepEteTest` which checks deletes are correctly propagated.\r\n- @snegrus-adb and I were working on an ETE test for the multitenancy, but that proved to be quite unwieldy.\r\n\r\n**Concerns (what feedback would you like?)**:\r\n- Is this actually more performant?\r\n\r\n**Where should we start reviewing?**: Pick any one of the index cleanup tasks, which shows what the renderer is trying to do.\r\n\r\n**Priority (whenever / two weeks / yesterday)**: Whenever. The heart of the issue that spawned this was with a downstream product abusing the stream store; this change makes Atlas more resilient to badness and possibly more performant in certain internal use cases as well.\n\n<!-- Reviewable:start -->\n---\nThis change is [<img src=\"https://reviewable.io/review_button.svg\" height=\"34\" align=\"absmiddle\" alt=\"Reviewable\"/>](https://reviewable.io/reviews/palantir/atlasdb/3423)\n<!-- Reviewable:end -->\n","created_at":"2018-07-27T14:33:36Z","updated_at":"2018-08-01T12:50:52Z","closed_at":null,"merged_at":null,"merge_commit_sha":"5cf92e08a0d7920e47104bbaff010bdb29c0900e","assignee":null,"assignees":[],"requested_reviewers":[{"login":"sandorw","id":8271534,"node_id":"MDQ6VXNlcjgyNzE1MzQ=","avatar_url":"https://avatars1.githubusercontent.com/u/8271534?v=4","gravatar_id":"","url":"https://api.github.com/users/sandorw","html_url":"https://github.com/sandorw","followers_url":"https://api.github.com/users/sandorw/followers","following_url":"https://api.github.com/users/sandorw/following{/other_user}","gists_url":"https://api.github.com/users/sandorw/gists{/gist_id}","starred_url":"https://api.github.com/users/sandorw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sandorw/subscriptions","organizations_url":"https://api.github.com/users/sandorw/orgs","repos_url":"https://api.github.com/users/sandorw/repos","events_url":"https://api.github.com/users/sandorw/events{/privacy}","received_events_url":"https://api.github.com/users/sandorw/received_events","type":"User","site_admin":false},{"login":"tboam","id":24569178,"node_id":"MDQ6VXNlcjI0NTY5MTc4","avatar_url":"https://avatars1.githubusercontent.com/u/24569178?v=4","gravatar_id":"","url":"https://api.github.com/users/tboam","html_url":"https://github.com/tboam","followers_url":"https://api.github.com/users/tboam/followers","following_url":"https://api.github.com/users/tboam/following{/other_user}","gists_url":"https://api.github.com/users/tboam/gists{/gist_id}","starred_url":"https://api.github.com/users/tboam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tboam/subscriptions","organizations_url":"https://api.github.com/users/tboam/orgs","repos_url":"https://api.github.com/users/tboam/repos","events_url":"https://api.github.com/users/tboam/events{/privacy}","received_events_url":"https://api.github.com/users/tboam/received_events","type":"User","site_admin":false},{"login":"snegrus-adb","id":37536200,"node_id":"MDQ6VXNlcjM3NTM2MjAw","avatar_url":"https://avatars0.githubusercontent.com/u/37536200?v=4","gravatar_id":"","url":"https://api.github.com/users/snegrus-adb","html_url":"https://github.com/snegrus-adb","followers_url":"https://api.github.com/users/snegrus-adb/followers","following_url":"https://api.github.com/users/snegrus-adb/following{/other_user}","gists_url":"https://api.github.com/users/snegrus-adb/gists{/gist_id}","starred_url":"https://api.github.com/users/snegrus-adb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/snegrus-adb/subscriptions","organizations_url":"https://api.github.com/users/snegrus-adb/orgs","repos_url":"https://api.github.com/users/snegrus-adb/repos","events_url":"https://api.github.com/users/snegrus-adb/events{/privacy}","received_events_url":"https://api.github.com/users/snegrus-adb/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/palantir/atlasdb/pulls/3423/commits","review_comments_url":"https://api.github.com/repos/palantir/atlasdb/pulls/3423/comments","review_comment_url":"https://api.github.com/repos/palantir/atlasdb/pulls/comments{/number}","comments_url":"https://api.github.com/repos/palantir/atlasdb/issues/3423/comments","statuses_url":"https://api.github.com/repos/palantir/atlasdb/statuses/899e789e2d4afa35e2cc7539c4ae68bf4835e0eb","head":{"label":"palantir:jkong/rows-column-range-for-cleanup-task","ref":"jkong/rows-column-range-for-cleanup-task","sha":"899e789e2d4afa35e2cc7539c4ae68bf4835e0eb","user":{"login":"palantir","id":303157,"node_id":"MDEyOk9yZ2FuaXphdGlvbjMwMzE1Nw==","avatar_url":"https://avatars0.githubusercontent.com/u/303157?v=4","gravatar_id":"","url":"https://api.github.com/users/palantir","html_url":"https://github.com/palantir","followers_url":"https://api.github.com/users/palantir/followers","following_url":"https://api.github.com/users/palantir/following{/other_user}","gists_url":"https://api.github.com/users/palantir/gists{/gist_id}","starred_url":"https://api.github.com/users/palantir/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/palantir/subscriptions","organizations_url":"https://api.github.com/users/palantir/orgs","repos_url":"https://api.github.com/users/palantir/repos","events_url":"https://api.github.com/users/palantir/events{/privacy}","received_events_url":"https://api.github.com/users/palantir/received_events","type":"Organization","site_admin":false},"repo":{"id":36960637,"node_id":"MDEwOlJlcG9zaXRvcnkzNjk2MDYzNw==","name":"atlasdb","full_name":"palantir/atlasdb","owner":{"login":"palantir","id":303157,"node_id":"MDEyOk9yZ2FuaXphdGlvbjMwMzE1Nw==","avatar_url":"https://avatars0.githubusercontent.com/u/303157?v=4","gravatar_id":"","url":"https://api.github.com/users/palantir","html_url":"https://github.com/palantir","followers_url":"https://api.github.com/users/palantir/followers","following_url":"https://api.github.com/users/palantir/following{/other_user}","gists_url":"https://api.github.com/users/palantir/gists{/gist_id}","starred_url":"https://api.github.com/users/palantir/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/palantir/subscriptions","organizations_url":"https://api.github.com/users/palantir/orgs","repos_url":"https://api.github.com/users/palantir/repos","events_url":"https://api.github.com/users/palantir/events{/privacy}","received_events_url":"https://api.github.com/users/palantir/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/palantir/atlasdb","description":"Transactional Distributed Database Layer","fork":false,"url":"https://api.github.com/repos/palantir/atlasdb","forks_url":"https://api.github.com/repos/palantir/atlasdb/forks","keys_url":"https://api.github.com/repos/palantir/atlasdb/keys{/key_id}","collaborators_url":"https://api.github.com/repos/palantir/atlasdb/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/palantir/atlasdb/teams","hooks_url":"https://api.github.com/repos/palantir/atlasdb/hooks","issue_events_url":"https://api.github.com/repos/palantir/atlasdb/issues/events{/number}","events_url":"https://api.github.com/repos/palantir/atlasdb/events","assignees_url":"https://api.github.com/repos/palantir/atlasdb/assignees{/user}","branches_url":"https://api.github.com/repos/palantir/atlasdb/branches{/branch}","tags_url":"https://api.github.com/repos/palantir/atlasdb/tags","blobs_url":"https://api.github.com/repos/palantir/atlasdb/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/palantir/atlasdb/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/palantir/atlasdb/git/refs{/sha}","trees_url":"https://api.github.com/repos/palantir/atlasdb/git/trees{/sha}","statuses_url":"https://api.github.com/repos/palantir/atlasdb/statuses/{sha}","languages_url":"https://api.github.com/repos/palantir/atlasdb/languages","stargazers_url":"https://api.github.com/repos/palantir/atlasdb/stargazers","contributors_url":"https://api.github.com/repos/palantir/atlasdb/contributors","subscribers_url":"https://api.github.com/repos/palantir/atlasdb/subscribers","subscription_url":"https://api.github.com/repos/palantir/atlasdb/subscription","commits_url":"https://api.github.com/repos/palantir/atlasdb/commits{/sha}","git_commits_url":"https://api.github.com/repos/palantir/atlasdb/git/commits{/sha}","comments_url":"https://api.github.com/repos/palantir/atlasdb/comments{/number}","issue_comment_url":"https://api.github.com/repos/palantir/atlasdb/issues/comments{/number}","contents_url":"https://api.github.com/repos/palantir/atlasdb/contents/{+path}","compare_url":"https://api.github.com/repos/palantir/atlasdb/compare/{base}...{head}","merges_url":"https://api.github.com/repos/palantir/atlasdb/merges","archive_url":"https://api.github.com/repos/palantir/atlasdb/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/palantir/atlasdb/downloads","issues_url":"https://api.github.com/repos/palantir/atlasdb/issues{/number}","pulls_url":"https://api.github.com/repos/palantir/atlasdb/pulls{/number}","milestones_url":"https://api.github.com/repos/palantir/atlasdb/milestones{/number}","notifications_url":"https://api.github.com/repos/palantir/atlasdb/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/palantir/atlasdb/labels{/name}","releases_url":"https://api.github.com/repos/palantir/atlasdb/releases{/id}","deployments_url":"https://api.github.com/repos/palantir/atlasdb/deployments","created_at":"2015-06-05T23:42:44Z","updated_at":"2018-08-01T12:50:51Z","pushed_at":"2018-08-01T12:50:47Z","git_url":"git://github.com/palantir/atlasdb.git","ssh_url":"git@github.com:palantir/atlasdb.git","clone_url":"https://github.com/palantir/atlasdb.git","svn_url":"https://github.com/palantir/atlasdb","homepage":"http://palantir.github.io/atlasdb/","size":72005,"stargazers_count":611,"watchers_count":611,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":111,"mirror_url":null,"archived":false,"open_issues_count":316,"license":{"key":"other","name":"Other","spdx_id":null,"url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":111,"open_issues":316,"watchers":611,"default_branch":"develop"}},"base":{"label":"palantir:develop","ref":"develop","sha":"45bea4bfae575c0d4ab3ebae6e2fcaa0cc16e1a6","user":{"login":"palantir","id":303157,"node_id":"MDEyOk9yZ2FuaXphdGlvbjMwMzE1Nw==","avatar_url":"https://avatars0.githubusercontent.com/u/303157?v=4","gravatar_id":"","url":"https://api.github.com/users/palantir","html_url":"https://github.com/palantir","followers_url":"https://api.github.com/users/palantir/followers","following_url":"https://api.github.com/users/palantir/following{/other_user}","gists_url":"https://api.github.com/users/palantir/gists{/gist_id}","starred_url":"https://api.github.com/users/palantir/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/palantir/subscriptions","organizations_url":"https://api.github.com/users/palantir/orgs","repos_url":"https://api.github.com/users/palantir/repos","events_url":"https://api.github.com/users/palantir/events{/privacy}","received_events_url":"https://api.github.com/users/palantir/received_events","type":"Organization","site_admin":false},"repo":{"id":36960637,"node_id":"MDEwOlJlcG9zaXRvcnkzNjk2MDYzNw==","name":"atlasdb","full_name":"palantir/atlasdb","owner":{"login":"palantir","id":303157,"node_id":"MDEyOk9yZ2FuaXphdGlvbjMwMzE1Nw==","avatar_url":"https://avatars0.githubusercontent.com/u/303157?v=4","gravatar_id":"","url":"https://api.github.com/users/palantir","html_url":"https://github.com/palantir","followers_url":"https://api.github.com/users/palantir/followers","following_url":"https://api.github.com/users/palantir/following{/other_user}","gists_url":"https://api.github.com/users/palantir/gists{/gist_id}","starred_url":"https://api.github.com/users/palantir/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/palantir/subscriptions","organizations_url":"https://api.github.com/users/palantir/orgs","repos_url":"https://api.github.com/users/palantir/repos","events_url":"https://api.github.com/users/palantir/events{/privacy}","received_events_url":"https://api.github.com/users/palantir/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/palantir/atlasdb","description":"Transactional Distributed Database Layer","fork":false,"url":"https://api.github.com/repos/palantir/atlasdb","forks_url":"https://api.github.com/repos/palantir/atlasdb/forks","keys_url":"https://api.github.com/repos/palantir/atlasdb/keys{/key_id}","collaborators_url":"https://api.github.com/repos/palantir/atlasdb/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/palantir/atlasdb/teams","hooks_url":"https://api.github.com/repos/palantir/atlasdb/hooks","issue_events_url":"https://api.github.com/repos/palantir/atlasdb/issues/events{/number}","events_url":"https://api.github.com/repos/palantir/atlasdb/events","assignees_url":"https://api.github.com/repos/palantir/atlasdb/assignees{/user}","branches_url":"https://api.github.com/repos/palantir/atlasdb/branches{/branch}","tags_url":"https://api.github.com/repos/palantir/atlasdb/tags","blobs_url":"https://api.github.com/repos/palantir/atlasdb/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/palantir/atlasdb/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/palantir/atlasdb/git/refs{/sha}","trees_url":"https://api.github.com/repos/palantir/atlasdb/git/trees{/sha}","statuses_url":"https://api.github.com/repos/palantir/atlasdb/statuses/{sha}","languages_url":"https://api.github.com/repos/palantir/atlasdb/languages","stargazers_url":"https://api.github.com/repos/palantir/atlasdb/stargazers","contributors_url":"https://api.github.com/repos/palantir/atlasdb/contributors","subscribers_url":"https://api.github.com/repos/palantir/atlasdb/subscribers","subscription_url":"https://api.github.com/repos/palantir/atlasdb/subscription","commits_url":"https://api.github.com/repos/palantir/atlasdb/commits{/sha}","git_commits_url":"https://api.github.com/repos/palantir/atlasdb/git/commits{/sha}","comments_url":"https://api.github.com/repos/palantir/atlasdb/comments{/number}","issue_comment_url":"https://api.github.com/repos/palantir/atlasdb/issues/comments{/number}","contents_url":"https://api.github.com/repos/palantir/atlasdb/contents/{+path}","compare_url":"https://api.github.com/repos/palantir/atlasdb/compare/{base}...{head}","merges_url":"https://api.github.com/repos/palantir/atlasdb/merges","archive_url":"https://api.github.com/repos/palantir/atlasdb/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/palantir/atlasdb/downloads","issues_url":"https://api.github.com/repos/palantir/atlasdb/issues{/number}","pulls_url":"https://api.github.com/repos/palantir/atlasdb/pulls{/number}","milestones_url":"https://api.github.com/repos/palantir/atlasdb/milestones{/number}","notifications_url":"https://api.github.com/repos/palantir/atlasdb/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/palantir/atlasdb/labels{/name}","releases_url":"https://api.github.com/repos/palantir/atlasdb/releases{/id}","deployments_url":"https://api.github.com/repos/palantir/atlasdb/deployments","created_at":"2015-06-05T23:42:44Z","updated_at":"2018-08-01T12:50:51Z","pushed_at":"2018-08-01T12:50:47Z","git_url":"git://github.com/palantir/atlasdb.git","ssh_url":"git@github.com:palantir/atlasdb.git","clone_url":"https://github.com/palantir/atlasdb.git","svn_url":"https://github.com/palantir/atlasdb","homepage":"http://palantir.github.io/atlasdb/","size":72005,"stargazers_count":611,"watchers_count":611,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":111,"mirror_url":null,"archived":false,"open_issues_count":316,"license":{"key":"other","name":"Other","spdx_id":null,"url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":111,"open_issues":316,"watchers":611,"default_branch":"develop"}},"_links":{"self":{"href":"https://api.github.com/repos/palantir/atlasdb/pulls/3423"},"html":{"href":"https://github.com/palantir/atlasdb/pull/3423"},"issue":{"href":"https://api.github.com/repos/palantir/atlasdb/issues/3423"},"comments":{"href":"https://api.github.com/repos/palantir/atlasdb/issues/3423/comments"},"review_comments":{"href":"https://api.github.com/repos/palantir/atlasdb/pulls/3423/comments"},"review_comment":{"href":"https://api.github.com/repos/palantir/atlasdb/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/palantir/atlasdb/pulls/3423/commits"},"statuses":{"href":"https://api.github.com/repos/palantir/atlasdb/statuses/899e789e2d4afa35e2cc7539c4ae68bf4835e0eb"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"jeremyk-91"}}