{"action":"created","comment":{"url":"https://api.github.com/repos/spinnaker/echo/pulls/comments/212470144","pull_request_review_id":149113346,"id":212470144,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIxMjQ3MDE0NA==","diff_hunk":"@@ -112,11 +122,29 @@ private void setMessageHandled(String messageKey, String identifier, Long retent\n     });\n   }\n \n+  private void setMessageComplete(String messageKey, String payload, Long retentionDeadlineMillis) {\n+    redisClientDelegate.withCommandsClient(c -> {\n+      c.psetex(messageKey, retentionDeadlineMillis, payload);\n+    });\n+  }\n+\n+  private Boolean messageComplete(String messageKey, String value) {\n+    Boolean exists = redisClientDelegate.withCommandsClient(c -> {","path":"echo-pubsub-core/src/main/java/com/netflix/spinnaker/echo/pubsub/PubsubMessageHandler.java","position":null,"original_position":81,"commit_id":"eac058b449e03ca2d0885ad4621f821eac0d5f25","original_commit_id":"a73588b9af82c725595c011adc1c37ad88d06de1","user":{"login":"emjburns","id":8454927,"node_id":"MDQ6VXNlcjg0NTQ5Mjc=","avatar_url":"https://avatars1.githubusercontent.com/u/8454927?v=4","gravatar_id":"","url":"https://api.github.com/users/emjburns","html_url":"https://github.com/emjburns","followers_url":"https://api.github.com/users/emjburns/followers","following_url":"https://api.github.com/users/emjburns/following{/other_user}","gists_url":"https://api.github.com/users/emjburns/gists{/gist_id}","starred_url":"https://api.github.com/users/emjburns/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/emjburns/subscriptions","organizations_url":"https://api.github.com/users/emjburns/orgs","repos_url":"https://api.github.com/users/emjburns/repos","events_url":"https://api.github.com/users/emjburns/events{/privacy}","received_events_url":"https://api.github.com/users/emjburns/received_events","type":"User","site_admin":false},"body":"Ah good catch. Fixed.","created_at":"2018-08-23T21:57:32Z","updated_at":"2018-08-23T21:57:32Z","html_url":"https://github.com/spinnaker/echo/pull/327#discussion_r212470144","pull_request_url":"https://api.github.com/repos/spinnaker/echo/pulls/327","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/spinnaker/echo/pulls/comments/212470144"},"html":{"href":"https://github.com/spinnaker/echo/pull/327#discussion_r212470144"},"pull_request":{"href":"https://api.github.com/repos/spinnaker/echo/pulls/327"}},"in_reply_to_id":212466996},"pull_request":{"url":"https://api.github.com/repos/spinnaker/echo/pulls/327","id":210570586,"node_id":"MDExOlB1bGxSZXF1ZXN0MjEwNTcwNTg2","html_url":"https://github.com/spinnaker/echo/pull/327","diff_url":"https://github.com/spinnaker/echo/pull/327.diff","patch_url":"https://github.com/spinnaker/echo/pull/327.patch","issue_url":"https://api.github.com/repos/spinnaker/echo/issues/327","number":327,"state":"open","locked":false,"title":"fix(pubsub): don't infinitely nack duplicate messages","user":{"login":"emjburns","id":8454927,"node_id":"MDQ6VXNlcjg0NTQ5Mjc=","avatar_url":"https://avatars1.githubusercontent.com/u/8454927?v=4","gravatar_id":"","url":"https://api.github.com/users/emjburns","html_url":"https://github.com/emjburns","followers_url":"https://api.github.com/users/emjburns/followers","following_url":"https://api.github.com/users/emjburns/following{/other_user}","gists_url":"https://api.github.com/users/emjburns/gists{/gist_id}","starred_url":"https://api.github.com/users/emjburns/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/emjburns/subscriptions","organizations_url":"https://api.github.com/users/emjburns/orgs","repos_url":"https://api.github.com/users/emjburns/repos","events_url":"https://api.github.com/users/emjburns/events{/privacy}","received_events_url":"https://api.github.com/users/emjburns/received_events","type":"User","site_admin":false},"body":"For pubsub, `nack()`-ing tells the system to redeliver the message (optionally after some time period).\r\nWe store only one key in redis about whether someone is currently processing a message. Timeout is 7 days for google for this.\r\n \r\nIf we get a duplicate message, the current implementation will continue to `nack()` the message (and cause it to be redelivered) until either the system stops doing that, or the timeout expires and the duplicate will be processed.\r\n\r\nThis PR introduces a second redis key of 'completed' messages with the message body as the value. Before processing a new message, we check to see if the message was already processed (`messageComplete(completedKey, payload)`). If it was processed, we `ack()` the message and move on - no need to process the same message more than once.\r\n\r\nIf the message was not fully processed, we try to get the lock. If we get the lock, we process the message and then store the message in the completed map (`setMessageComplete(...)`).\r\n\r\nIf we don't get the lock, we `nack` the message. When the message is re-delivered, we try again.\r\n\r\nAlso emitting metrics for number of duplicates, for funsies. ","created_at":"2018-08-23T20:44:10Z","updated_at":"2018-08-23T21:57:32Z","closed_at":null,"merged_at":null,"merge_commit_sha":"461188fcacedc97af49f11397782a98b1f7bd631","assignee":null,"assignees":[],"requested_reviewers":[{"login":"robzienert","id":108741,"node_id":"MDQ6VXNlcjEwODc0MQ==","avatar_url":"https://avatars0.githubusercontent.com/u/108741?v=4","gravatar_id":"","url":"https://api.github.com/users/robzienert","html_url":"https://github.com/robzienert","followers_url":"https://api.github.com/users/robzienert/followers","following_url":"https://api.github.com/users/robzienert/following{/other_user}","gists_url":"https://api.github.com/users/robzienert/gists{/gist_id}","starred_url":"https://api.github.com/users/robzienert/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/robzienert/subscriptions","organizations_url":"https://api.github.com/users/robzienert/orgs","repos_url":"https://api.github.com/users/robzienert/repos","events_url":"https://api.github.com/users/robzienert/events{/privacy}","received_events_url":"https://api.github.com/users/robzienert/received_events","type":"User","site_admin":false},{"login":"lwander","id":4874941,"node_id":"MDQ6VXNlcjQ4NzQ5NDE=","avatar_url":"https://avatars3.githubusercontent.com/u/4874941?v=4","gravatar_id":"","url":"https://api.github.com/users/lwander","html_url":"https://github.com/lwander","followers_url":"https://api.github.com/users/lwander/followers","following_url":"https://api.github.com/users/lwander/following{/other_user}","gists_url":"https://api.github.com/users/lwander/gists{/gist_id}","starred_url":"https://api.github.com/users/lwander/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lwander/subscriptions","organizations_url":"https://api.github.com/users/lwander/orgs","repos_url":"https://api.github.com/users/lwander/repos","events_url":"https://api.github.com/users/lwander/events{/privacy}","received_events_url":"https://api.github.com/users/lwander/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/spinnaker/echo/pulls/327/commits","review_comments_url":"https://api.github.com/repos/spinnaker/echo/pulls/327/comments","review_comment_url":"https://api.github.com/repos/spinnaker/echo/pulls/comments{/number}","comments_url":"https://api.github.com/repos/spinnaker/echo/issues/327/comments","statuses_url":"https://api.github.com/repos/spinnaker/echo/statuses/eac058b449e03ca2d0885ad4621f821eac0d5f25","head":{"label":"emjburns:dedup","ref":"dedup","sha":"eac058b449e03ca2d0885ad4621f821eac0d5f25","user":{"login":"emjburns","id":8454927,"node_id":"MDQ6VXNlcjg0NTQ5Mjc=","avatar_url":"https://avatars1.githubusercontent.com/u/8454927?v=4","gravatar_id":"","url":"https://api.github.com/users/emjburns","html_url":"https://github.com/emjburns","followers_url":"https://api.github.com/users/emjburns/followers","following_url":"https://api.github.com/users/emjburns/following{/other_user}","gists_url":"https://api.github.com/users/emjburns/gists{/gist_id}","starred_url":"https://api.github.com/users/emjburns/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/emjburns/subscriptions","organizations_url":"https://api.github.com/users/emjburns/orgs","repos_url":"https://api.github.com/users/emjburns/repos","events_url":"https://api.github.com/users/emjburns/events{/privacy}","received_events_url":"https://api.github.com/users/emjburns/received_events","type":"User","site_admin":false},"repo":{"id":102141043,"node_id":"MDEwOlJlcG9zaXRvcnkxMDIxNDEwNDM=","name":"echo","full_name":"emjburns/echo","owner":{"login":"emjburns","id":8454927,"node_id":"MDQ6VXNlcjg0NTQ5Mjc=","avatar_url":"https://avatars1.githubusercontent.com/u/8454927?v=4","gravatar_id":"","url":"https://api.github.com/users/emjburns","html_url":"https://github.com/emjburns","followers_url":"https://api.github.com/users/emjburns/followers","following_url":"https://api.github.com/users/emjburns/following{/other_user}","gists_url":"https://api.github.com/users/emjburns/gists{/gist_id}","starred_url":"https://api.github.com/users/emjburns/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/emjburns/subscriptions","organizations_url":"https://api.github.com/users/emjburns/orgs","repos_url":"https://api.github.com/users/emjburns/repos","events_url":"https://api.github.com/users/emjburns/events{/privacy}","received_events_url":"https://api.github.com/users/emjburns/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/emjburns/echo","description":"Spinnaker Eventing Service","fork":true,"url":"https://api.github.com/repos/emjburns/echo","forks_url":"https://api.github.com/repos/emjburns/echo/forks","keys_url":"https://api.github.com/repos/emjburns/echo/keys{/key_id}","collaborators_url":"https://api.github.com/repos/emjburns/echo/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/emjburns/echo/teams","hooks_url":"https://api.github.com/repos/emjburns/echo/hooks","issue_events_url":"https://api.github.com/repos/emjburns/echo/issues/events{/number}","events_url":"https://api.github.com/repos/emjburns/echo/events","assignees_url":"https://api.github.com/repos/emjburns/echo/assignees{/user}","branches_url":"https://api.github.com/repos/emjburns/echo/branches{/branch}","tags_url":"https://api.github.com/repos/emjburns/echo/tags","blobs_url":"https://api.github.com/repos/emjburns/echo/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/emjburns/echo/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/emjburns/echo/git/refs{/sha}","trees_url":"https://api.github.com/repos/emjburns/echo/git/trees{/sha}","statuses_url":"https://api.github.com/repos/emjburns/echo/statuses/{sha}","languages_url":"https://api.github.com/repos/emjburns/echo/languages","stargazers_url":"https://api.github.com/repos/emjburns/echo/stargazers","contributors_url":"https://api.github.com/repos/emjburns/echo/contributors","subscribers_url":"https://api.github.com/repos/emjburns/echo/subscribers","subscription_url":"https://api.github.com/repos/emjburns/echo/subscription","commits_url":"https://api.github.com/repos/emjburns/echo/commits{/sha}","git_commits_url":"https://api.github.com/repos/emjburns/echo/git/commits{/sha}","comments_url":"https://api.github.com/repos/emjburns/echo/comments{/number}","issue_comment_url":"https://api.github.com/repos/emjburns/echo/issues/comments{/number}","contents_url":"https://api.github.com/repos/emjburns/echo/contents/{+path}","compare_url":"https://api.github.com/repos/emjburns/echo/compare/{base}...{head}","merges_url":"https://api.github.com/repos/emjburns/echo/merges","archive_url":"https://api.github.com/repos/emjburns/echo/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/emjburns/echo/downloads","issues_url":"https://api.github.com/repos/emjburns/echo/issues{/number}","pulls_url":"https://api.github.com/repos/emjburns/echo/pulls{/number}","milestones_url":"https://api.github.com/repos/emjburns/echo/milestones{/number}","notifications_url":"https://api.github.com/repos/emjburns/echo/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/emjburns/echo/labels{/name}","releases_url":"https://api.github.com/repos/emjburns/echo/releases{/id}","deployments_url":"https://api.github.com/repos/emjburns/echo/deployments","created_at":"2017-09-01T18:18:49Z","updated_at":"2017-09-01T18:18:51Z","pushed_at":"2018-08-23T21:57:24Z","git_url":"git://github.com/emjburns/echo.git","ssh_url":"git@github.com:emjburns/echo.git","clone_url":"https://github.com/emjburns/echo.git","svn_url":"https://github.com/emjburns/echo","homepage":"","size":1410,"stargazers_count":0,"watchers_count":0,"language":"Groovy","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"spinnaker:master","ref":"master","sha":"f29036381134a2d2077374eb411d0db74cc1d619","user":{"login":"spinnaker","id":7634182,"node_id":"MDEyOk9yZ2FuaXphdGlvbjc2MzQxODI=","avatar_url":"https://avatars1.githubusercontent.com/u/7634182?v=4","gravatar_id":"","url":"https://api.github.com/users/spinnaker","html_url":"https://github.com/spinnaker","followers_url":"https://api.github.com/users/spinnaker/followers","following_url":"https://api.github.com/users/spinnaker/following{/other_user}","gists_url":"https://api.github.com/users/spinnaker/gists{/gist_id}","starred_url":"https://api.github.com/users/spinnaker/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/spinnaker/subscriptions","organizations_url":"https://api.github.com/users/spinnaker/orgs","repos_url":"https://api.github.com/users/spinnaker/repos","events_url":"https://api.github.com/users/spinnaker/events{/privacy}","received_events_url":"https://api.github.com/users/spinnaker/received_events","type":"Organization","site_admin":false},"repo":{"id":21145185,"node_id":"MDEwOlJlcG9zaXRvcnkyMTE0NTE4NQ==","name":"echo","full_name":"spinnaker/echo","owner":{"login":"spinnaker","id":7634182,"node_id":"MDEyOk9yZ2FuaXphdGlvbjc2MzQxODI=","avatar_url":"https://avatars1.githubusercontent.com/u/7634182?v=4","gravatar_id":"","url":"https://api.github.com/users/spinnaker","html_url":"https://github.com/spinnaker","followers_url":"https://api.github.com/users/spinnaker/followers","following_url":"https://api.github.com/users/spinnaker/following{/other_user}","gists_url":"https://api.github.com/users/spinnaker/gists{/gist_id}","starred_url":"https://api.github.com/users/spinnaker/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/spinnaker/subscriptions","organizations_url":"https://api.github.com/users/spinnaker/orgs","repos_url":"https://api.github.com/users/spinnaker/repos","events_url":"https://api.github.com/users/spinnaker/events{/privacy}","received_events_url":"https://api.github.com/users/spinnaker/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/spinnaker/echo","description":"Spinnaker Eventing Service","fork":false,"url":"https://api.github.com/repos/spinnaker/echo","forks_url":"https://api.github.com/repos/spinnaker/echo/forks","keys_url":"https://api.github.com/repos/spinnaker/echo/keys{/key_id}","collaborators_url":"https://api.github.com/repos/spinnaker/echo/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/spinnaker/echo/teams","hooks_url":"https://api.github.com/repos/spinnaker/echo/hooks","issue_events_url":"https://api.github.com/repos/spinnaker/echo/issues/events{/number}","events_url":"https://api.github.com/repos/spinnaker/echo/events","assignees_url":"https://api.github.com/repos/spinnaker/echo/assignees{/user}","branches_url":"https://api.github.com/repos/spinnaker/echo/branches{/branch}","tags_url":"https://api.github.com/repos/spinnaker/echo/tags","blobs_url":"https://api.github.com/repos/spinnaker/echo/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/spinnaker/echo/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/spinnaker/echo/git/refs{/sha}","trees_url":"https://api.github.com/repos/spinnaker/echo/git/trees{/sha}","statuses_url":"https://api.github.com/repos/spinnaker/echo/statuses/{sha}","languages_url":"https://api.github.com/repos/spinnaker/echo/languages","stargazers_url":"https://api.github.com/repos/spinnaker/echo/stargazers","contributors_url":"https://api.github.com/repos/spinnaker/echo/contributors","subscribers_url":"https://api.github.com/repos/spinnaker/echo/subscribers","subscription_url":"https://api.github.com/repos/spinnaker/echo/subscription","commits_url":"https://api.github.com/repos/spinnaker/echo/commits{/sha}","git_commits_url":"https://api.github.com/repos/spinnaker/echo/git/commits{/sha}","comments_url":"https://api.github.com/repos/spinnaker/echo/comments{/number}","issue_comment_url":"https://api.github.com/repos/spinnaker/echo/issues/comments{/number}","contents_url":"https://api.github.com/repos/spinnaker/echo/contents/{+path}","compare_url":"https://api.github.com/repos/spinnaker/echo/compare/{base}...{head}","merges_url":"https://api.github.com/repos/spinnaker/echo/merges","archive_url":"https://api.github.com/repos/spinnaker/echo/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/spinnaker/echo/downloads","issues_url":"https://api.github.com/repos/spinnaker/echo/issues{/number}","pulls_url":"https://api.github.com/repos/spinnaker/echo/pulls{/number}","milestones_url":"https://api.github.com/repos/spinnaker/echo/milestones{/number}","notifications_url":"https://api.github.com/repos/spinnaker/echo/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/spinnaker/echo/labels{/name}","releases_url":"https://api.github.com/repos/spinnaker/echo/releases{/id}","deployments_url":"https://api.github.com/repos/spinnaker/echo/deployments","created_at":"2014-06-23T22:59:57Z","updated_at":"2018-08-23T21:11:03Z","pushed_at":"2018-08-23T21:57:26Z","git_url":"git://github.com/spinnaker/echo.git","ssh_url":"git@github.com:spinnaker/echo.git","clone_url":"https://github.com/spinnaker/echo.git","svn_url":"https://github.com/spinnaker/echo","homepage":"","size":1409,"stargazers_count":28,"watchers_count":28,"language":"Groovy","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":237,"mirror_url":null,"archived":false,"open_issues_count":1,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":237,"open_issues":1,"watchers":28,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/spinnaker/echo/pulls/327"},"html":{"href":"https://github.com/spinnaker/echo/pull/327"},"issue":{"href":"https://api.github.com/repos/spinnaker/echo/issues/327"},"comments":{"href":"https://api.github.com/repos/spinnaker/echo/issues/327/comments"},"review_comments":{"href":"https://api.github.com/repos/spinnaker/echo/pulls/327/comments"},"review_comment":{"href":"https://api.github.com/repos/spinnaker/echo/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/spinnaker/echo/pulls/327/commits"},"statuses":{"href":"https://api.github.com/repos/spinnaker/echo/statuses/eac058b449e03ca2d0885ad4621f821eac0d5f25"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"emjburns"}}