{"action":"created","issue":{"url":"https://api.github.com/repos/dimagi/commcare-hq/issues/20834","repository_url":"https://api.github.com/repos/dimagi/commcare-hq","labels_url":"https://api.github.com/repos/dimagi/commcare-hq/issues/20834/labels{/name}","comments_url":"https://api.github.com/repos/dimagi/commcare-hq/issues/20834/comments","events_url":"https://api.github.com/repos/dimagi/commcare-hq/issues/20834/events","html_url":"https://github.com/dimagi/commcare-hq/pull/20834","id":330609978,"node_id":"MDExOlB1bGxSZXF1ZXN0MTkzNTc2NjY1","number":20834,"title":"batch ucrpillow","user":{"login":"sravfeyn","id":829142,"node_id":"MDQ6VXNlcjgyOTE0Mg==","avatar_url":"https://avatars0.githubusercontent.com/u/829142?v=4","gravatar_id":"","url":"https://api.github.com/users/sravfeyn","html_url":"https://github.com/sravfeyn","followers_url":"https://api.github.com/users/sravfeyn/followers","following_url":"https://api.github.com/users/sravfeyn/following{/other_user}","gists_url":"https://api.github.com/users/sravfeyn/gists{/gist_id}","starred_url":"https://api.github.com/users/sravfeyn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sravfeyn/subscriptions","organizations_url":"https://api.github.com/users/sravfeyn/orgs","repos_url":"https://api.github.com/users/sravfeyn/repos","events_url":"https://api.github.com/users/sravfeyn/events{/privacy}","received_events_url":"https://api.github.com/users/sravfeyn/received_events","type":"User","site_admin":false},"labels":[{"id":87923342,"node_id":"MDU6TGFiZWw4NzkyMzM0Mg==","url":"https://api.github.com/repos/dimagi/commcare-hq/labels/awaiting%20QA","name":"awaiting QA","color":"fbca04","default":false},{"id":724787015,"node_id":"MDU6TGFiZWw3MjQ3ODcwMTU=","url":"https://api.github.com/repos/dimagi/commcare-hq/labels/product/invisible","name":"product/invisible","color":"cfd2d6","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":10,"created_at":"2018-06-08T10:45:27Z","updated_at":"2018-08-15T21:44:49Z","closed_at":null,"author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/dimagi/commcare-hq/pulls/20834","html_url":"https://github.com/dimagi/commcare-hq/pull/20834","diff_url":"https://github.com/dimagi/commcare-hq/pull/20834.diff","patch_url":"https://github.com/dimagi/commcare-hq/pull/20834.patch"},"body":"This PR\r\n1. Introduces a `BulkPillowProcessor` class\r\n2. Extends the current `PillowBase` to pass changes in chunks if the attached processor is `BulkPillowProcessor` (and couple of other conditions, see code for details)\r\n3. Implements `process_change_chunk` for the UCR `ConfigurableReportKafkaPillow`\r\n "},"comment":{"url":"https://api.github.com/repos/dimagi/commcare-hq/issues/comments/413348048","html_url":"https://github.com/dimagi/commcare-hq/pull/20834#issuecomment-413348048","issue_url":"https://api.github.com/repos/dimagi/commcare-hq/issues/20834","id":413348048,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMzM0ODA0OA==","user":{"login":"emord","id":1471773,"node_id":"MDQ6VXNlcjE0NzE3NzM=","avatar_url":"https://avatars0.githubusercontent.com/u/1471773?v=4","gravatar_id":"","url":"https://api.github.com/users/emord","html_url":"https://github.com/emord","followers_url":"https://api.github.com/users/emord/followers","following_url":"https://api.github.com/users/emord/following{/other_user}","gists_url":"https://api.github.com/users/emord/gists{/gist_id}","starred_url":"https://api.github.com/users/emord/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/emord/subscriptions","organizations_url":"https://api.github.com/users/emord/orgs","repos_url":"https://api.github.com/users/emord/repos","events_url":"https://api.github.com/users/emord/events{/privacy}","received_events_url":"https://api.github.com/users/emord/received_events","type":"User","site_admin":false},"created_at":"2018-08-15T21:44:49Z","updated_at":"2018-08-15T21:44:49Z","author_association":"CONTRIBUTOR","body":"> One other point, since we're now including multiple processors in a pillow we should change the datadog metrics to be tagged with the processor name and not the pillow name. @emord do you agree?\r\n\r\nI think that this might depend on the metric? For example a processing time metric could be tagged with processor name, but the checkpoint would still be on the pillow level so it should stay pillow name\r\n\r\n> If we add that restriction, we won't be able to combine processors without implementing bulk support for all processors\r\n\r\nWe don't need to implement \"smart\" bulk support for the processors. As long as there's an appropriate interface, I think that the following change wouldn't be difficult to add to the base class:\r\n\r\n```\r\ndef process_changes_bulk(changes):\r\n   for change in changes:\r\n      process_change(change)\r\n```"}}
{"actor":{"display_login":"emord"}}