{"action":"created","issue":{"url":"https://api.github.com/repos/palantir/atlasdb/issues/3181","repository_url":"https://api.github.com/repos/palantir/atlasdb","labels_url":"https://api.github.com/repos/palantir/atlasdb/issues/3181/labels{/name}","comments_url":"https://api.github.com/repos/palantir/atlasdb/issues/3181/comments","events_url":"https://api.github.com/repos/palantir/atlasdb/issues/3181/events","html_url":"https://github.com/palantir/atlasdb/issues/3181","id":325263818,"number":3181,"title":"Consider using cell size as a heuristic for sweep batch sizes","user":{"login":"gsheasby","id":15610492,"avatar_url":"https://avatars2.githubusercontent.com/u/15610492?v=4","gravatar_id":"","url":"https://api.github.com/users/gsheasby","html_url":"https://github.com/gsheasby","followers_url":"https://api.github.com/users/gsheasby/followers","following_url":"https://api.github.com/users/gsheasby/following{/other_user}","gists_url":"https://api.github.com/users/gsheasby/gists{/gist_id}","starred_url":"https://api.github.com/users/gsheasby/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gsheasby/subscriptions","organizations_url":"https://api.github.com/users/gsheasby/orgs","repos_url":"https://api.github.com/users/gsheasby/repos","events_url":"https://api.github.com/users/gsheasby/events{/privacy}","received_events_url":"https://api.github.com/users/gsheasby/received_events","type":"User","site_admin":false},"labels":[{"id":538403919,"url":"https://api.github.com/repos/palantir/atlasdb/labels/component:%20sweep","name":"component: sweep","color":"cccccc","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-05-22T12:04:52Z","updated_at":"2018-05-23T15:22:16Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"In PDS-68325, sweep was failing to delete values from the stream store table, because the values in this table are very large.\r\n\r\nCan we investigate what it would look like to use cell size as a heuristic vs just timeouts? Something like \"I only want to load ~50M of data into sweep at once\" or something.\r\n\r\nThis would be an alternative to blindly using lower delete batch size or higher pause time for sweeping value tables.\r\n\r\ncc @aggie05engineer, who suggested this."},"comment":{"url":"https://api.github.com/repos/palantir/atlasdb/issues/comments/391388086","html_url":"https://github.com/palantir/atlasdb/issues/3181#issuecomment-391388086","issue_url":"https://api.github.com/repos/palantir/atlasdb/issues/3181","id":391388086,"user":{"login":"gsheasby","id":15610492,"avatar_url":"https://avatars2.githubusercontent.com/u/15610492?v=4","gravatar_id":"","url":"https://api.github.com/users/gsheasby","html_url":"https://github.com/gsheasby","followers_url":"https://api.github.com/users/gsheasby/followers","following_url":"https://api.github.com/users/gsheasby/following{/other_user}","gists_url":"https://api.github.com/users/gsheasby/gists{/gist_id}","starred_url":"https://api.github.com/users/gsheasby/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gsheasby/subscriptions","organizations_url":"https://api.github.com/users/gsheasby/orgs","repos_url":"https://api.github.com/users/gsheasby/repos","events_url":"https://api.github.com/users/gsheasby/events{/privacy}","received_events_url":"https://api.github.com/users/gsheasby/received_events","type":"User","site_admin":false},"created_at":"2018-05-23T15:22:16Z","updated_at":"2018-05-23T15:22:16Z","author_association":"CONTRIBUTOR","body":"In the support ticket, high pressure in Cassandra was caused by a sustained high volume of deletes, and was fixed by reducing the delete batch size and increasing the sweep pause millis.\r\n\r\nSo this is memory pressure in the _delete_ stage, and not in the read stage."}}
{"actor":{"display_login":"gsheasby"}}