{"action":"created","comment":{"url":"https://api.github.com/repos/apache/kafka/pulls/comments/93344584","pull_request_review_id":13869268,"id":93344584,"diff_hunk":"@@ -788,9 +793,9 @@ protected StreamTask createStreamTask(TaskId id, Collection<TopicPartition> part\n \n         sensors.taskCreationSensor.record();\n \n-        ProcessorTopology topology = builder.build(id.topicGroupId);\n-\n-        return new StreamTask(id, applicationId, partitions, topology, consumer, producer, restoreConsumer, config, sensors, stateDirectory, cache);\n+        final ProcessorTopology topology = builder.build(id.topicGroupId);\n+        final RecordCollector recordCollector = new RecordCollectorImpl(producer, id.toString());\n+        return new StreamTask(id, applicationId, partitions, topology, consumer, restoreConsumer, config, sensors, stateDirectory, cache, recordCollector);","path":"streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java","position":224,"original_position":224,"commit_id":"a1386034399dc63126b9a1261713ca140fb30c07","original_commit_id":"a1386034399dc63126b9a1261713ca140fb30c07","user":{"login":"guozhangwang","id":10729597,"avatar_url":"https://avatars.githubusercontent.com/u/10729597?v=3","gravatar_id":"","url":"https://api.github.com/users/guozhangwang","html_url":"https://github.com/guozhangwang","followers_url":"https://api.github.com/users/guozhangwang/followers","following_url":"https://api.github.com/users/guozhangwang/following{/other_user}","gists_url":"https://api.github.com/users/guozhangwang/gists{/gist_id}","starred_url":"https://api.github.com/users/guozhangwang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guozhangwang/subscriptions","organizations_url":"https://api.github.com/users/guozhangwang/orgs","repos_url":"https://api.github.com/users/guozhangwang/repos","events_url":"https://api.github.com/users/guozhangwang/events{/privacy}","received_events_url":"https://api.github.com/users/guozhangwang/received_events","type":"User","site_admin":false},"body":"Why we want to move the construction of record collector outside stream task?","created_at":"2016-12-20T23:04:38Z","updated_at":"2016-12-20T23:14:01Z","html_url":"https://github.com/apache/kafka/pull/2281#discussion_r93344584","pull_request_url":"https://api.github.com/repos/apache/kafka/pulls/2281","_links":{"self":{"href":"https://api.github.com/repos/apache/kafka/pulls/comments/93344584"},"html":{"href":"https://github.com/apache/kafka/pull/2281#discussion_r93344584"},"pull_request":{"href":"https://api.github.com/repos/apache/kafka/pulls/2281"}}},"pull_request":{"url":"https://api.github.com/repos/apache/kafka/pulls/2281","id":98806698,"html_url":"https://github.com/apache/kafka/pull/2281","diff_url":"https://github.com/apache/kafka/pull/2281.diff","patch_url":"https://github.com/apache/kafka/pull/2281.patch","issue_url":"https://api.github.com/repos/apache/kafka/issues/2281","number":2281,"state":"open","locked":false,"title":"KAFKA-4561: Ordering of operations in StreamThread.shutdownTasksAndState may void at-least-once guarantees","user":{"login":"dguy","id":54829,"avatar_url":"https://avatars.githubusercontent.com/u/54829?v=3","gravatar_id":"","url":"https://api.github.com/users/dguy","html_url":"https://github.com/dguy","followers_url":"https://api.github.com/users/dguy/followers","following_url":"https://api.github.com/users/dguy/following{/other_user}","gists_url":"https://api.github.com/users/dguy/gists{/gist_id}","starred_url":"https://api.github.com/users/dguy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dguy/subscriptions","organizations_url":"https://api.github.com/users/dguy/orgs","repos_url":"https://api.github.com/users/dguy/repos","events_url":"https://api.github.com/users/dguy/events{/privacy}","received_events_url":"https://api.github.com/users/dguy/received_events","type":"User","site_admin":false},"body":"In `shutdownTasksAndState` and `suspendTasksAndState` we commit offsets BEFORE we flush any state. This is wrong as if an exception occurs during a flush, we may violate the at-least-once guarantees, that is we would have committed some offsets but NOT sent the processed data on to other Sinks.\r\nAlso during suspend and shutdown, we should try and complete all tasks even when exceptions occur. We should just keep track of the exception and rethrow it at the end if necessary. This helps with ensuring that StateStores etc are closed.","created_at":"2016-12-20T19:04:59Z","updated_at":"2016-12-20T23:14:01Z","closed_at":null,"merged_at":null,"merge_commit_sha":"2183650a6b26dde82eb0aac9a88c365b37c72465","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/kafka/pulls/2281/commits","review_comments_url":"https://api.github.com/repos/apache/kafka/pulls/2281/comments","review_comment_url":"https://api.github.com/repos/apache/kafka/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/kafka/issues/2281/comments","statuses_url":"https://api.github.com/repos/apache/kafka/statuses/a1386034399dc63126b9a1261713ca140fb30c07","head":{"label":"dguy:kafka-4561","ref":"kafka-4561","sha":"a1386034399dc63126b9a1261713ca140fb30c07","user":{"login":"dguy","id":54829,"avatar_url":"https://avatars.githubusercontent.com/u/54829?v=3","gravatar_id":"","url":"https://api.github.com/users/dguy","html_url":"https://github.com/dguy","followers_url":"https://api.github.com/users/dguy/followers","following_url":"https://api.github.com/users/dguy/following{/other_user}","gists_url":"https://api.github.com/users/dguy/gists{/gist_id}","starred_url":"https://api.github.com/users/dguy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dguy/subscriptions","organizations_url":"https://api.github.com/users/dguy/orgs","repos_url":"https://api.github.com/users/dguy/repos","events_url":"https://api.github.com/users/dguy/events{/privacy}","received_events_url":"https://api.github.com/users/dguy/received_events","type":"User","site_admin":false},"repo":{"id":56393114,"name":"kafka","full_name":"dguy/kafka","owner":{"login":"dguy","id":54829,"avatar_url":"https://avatars.githubusercontent.com/u/54829?v=3","gravatar_id":"","url":"https://api.github.com/users/dguy","html_url":"https://github.com/dguy","followers_url":"https://api.github.com/users/dguy/followers","following_url":"https://api.github.com/users/dguy/following{/other_user}","gists_url":"https://api.github.com/users/dguy/gists{/gist_id}","starred_url":"https://api.github.com/users/dguy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dguy/subscriptions","organizations_url":"https://api.github.com/users/dguy/orgs","repos_url":"https://api.github.com/users/dguy/repos","events_url":"https://api.github.com/users/dguy/events{/privacy}","received_events_url":"https://api.github.com/users/dguy/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/dguy/kafka","description":"Mirror of Apache Kafka","fork":true,"url":"https://api.github.com/repos/dguy/kafka","forks_url":"https://api.github.com/repos/dguy/kafka/forks","keys_url":"https://api.github.com/repos/dguy/kafka/keys{/key_id}","collaborators_url":"https://api.github.com/repos/dguy/kafka/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/dguy/kafka/teams","hooks_url":"https://api.github.com/repos/dguy/kafka/hooks","issue_events_url":"https://api.github.com/repos/dguy/kafka/issues/events{/number}","events_url":"https://api.github.com/repos/dguy/kafka/events","assignees_url":"https://api.github.com/repos/dguy/kafka/assignees{/user}","branches_url":"https://api.github.com/repos/dguy/kafka/branches{/branch}","tags_url":"https://api.github.com/repos/dguy/kafka/tags","blobs_url":"https://api.github.com/repos/dguy/kafka/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/dguy/kafka/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/dguy/kafka/git/refs{/sha}","trees_url":"https://api.github.com/repos/dguy/kafka/git/trees{/sha}","statuses_url":"https://api.github.com/repos/dguy/kafka/statuses/{sha}","languages_url":"https://api.github.com/repos/dguy/kafka/languages","stargazers_url":"https://api.github.com/repos/dguy/kafka/stargazers","contributors_url":"https://api.github.com/repos/dguy/kafka/contributors","subscribers_url":"https://api.github.com/repos/dguy/kafka/subscribers","subscription_url":"https://api.github.com/repos/dguy/kafka/subscription","commits_url":"https://api.github.com/repos/dguy/kafka/commits{/sha}","git_commits_url":"https://api.github.com/repos/dguy/kafka/git/commits{/sha}","comments_url":"https://api.github.com/repos/dguy/kafka/comments{/number}","issue_comment_url":"https://api.github.com/repos/dguy/kafka/issues/comments{/number}","contents_url":"https://api.github.com/repos/dguy/kafka/contents/{+path}","compare_url":"https://api.github.com/repos/dguy/kafka/compare/{base}...{head}","merges_url":"https://api.github.com/repos/dguy/kafka/merges","archive_url":"https://api.github.com/repos/dguy/kafka/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/dguy/kafka/downloads","issues_url":"https://api.github.com/repos/dguy/kafka/issues{/number}","pulls_url":"https://api.github.com/repos/dguy/kafka/pulls{/number}","milestones_url":"https://api.github.com/repos/dguy/kafka/milestones{/number}","notifications_url":"https://api.github.com/repos/dguy/kafka/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/dguy/kafka/labels{/name}","releases_url":"https://api.github.com/repos/dguy/kafka/releases{/id}","deployments_url":"https://api.github.com/repos/dguy/kafka/deployments","created_at":"2016-04-16T16:12:53Z","updated_at":"2016-04-16T16:12:57Z","pushed_at":"2016-12-20T19:12:30Z","git_url":"git://github.com/dguy/kafka.git","ssh_url":"git@github.com:dguy/kafka.git","clone_url":"https://github.com/dguy/kafka.git","svn_url":"https://github.com/dguy/kafka","homepage":null,"size":38215,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"trunk"}},"base":{"label":"apache:trunk","ref":"trunk","sha":"7f4b278c030f6e5cff8caca6428f0e631c8918ec","user":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":2211243,"name":"kafka","full_name":"apache/kafka","owner":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/apache/kafka","description":"Mirror of Apache Kafka","fork":false,"url":"https://api.github.com/repos/apache/kafka","forks_url":"https://api.github.com/repos/apache/kafka/forks","keys_url":"https://api.github.com/repos/apache/kafka/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/kafka/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/kafka/teams","hooks_url":"https://api.github.com/repos/apache/kafka/hooks","issue_events_url":"https://api.github.com/repos/apache/kafka/issues/events{/number}","events_url":"https://api.github.com/repos/apache/kafka/events","assignees_url":"https://api.github.com/repos/apache/kafka/assignees{/user}","branches_url":"https://api.github.com/repos/apache/kafka/branches{/branch}","tags_url":"https://api.github.com/repos/apache/kafka/tags","blobs_url":"https://api.github.com/repos/apache/kafka/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/kafka/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/kafka/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/kafka/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/kafka/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/kafka/languages","stargazers_url":"https://api.github.com/repos/apache/kafka/stargazers","contributors_url":"https://api.github.com/repos/apache/kafka/contributors","subscribers_url":"https://api.github.com/repos/apache/kafka/subscribers","subscription_url":"https://api.github.com/repos/apache/kafka/subscription","commits_url":"https://api.github.com/repos/apache/kafka/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/kafka/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/kafka/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/kafka/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/kafka/contents/{+path}","compare_url":"https://api.github.com/repos/apache/kafka/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/kafka/merges","archive_url":"https://api.github.com/repos/apache/kafka/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/kafka/downloads","issues_url":"https://api.github.com/repos/apache/kafka/issues{/number}","pulls_url":"https://api.github.com/repos/apache/kafka/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/kafka/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/kafka/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/kafka/labels{/name}","releases_url":"https://api.github.com/repos/apache/kafka/releases{/id}","deployments_url":"https://api.github.com/repos/apache/kafka/deployments","created_at":"2011-08-15T18:06:16Z","updated_at":"2016-12-20T19:33:51Z","pushed_at":"2016-12-20T22:46:03Z","git_url":"git://github.com/apache/kafka.git","ssh_url":"git@github.com:apache/kafka.git","clone_url":"https://github.com/apache/kafka.git","svn_url":"https://github.com/apache/kafka","homepage":null,"size":38039,"stargazers_count":4312,"watchers_count":4312,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":2679,"mirror_url":"git://git.apache.org/kafka.git","open_issues_count":309,"forks":2679,"open_issues":309,"watchers":4312,"default_branch":"trunk"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/kafka/pulls/2281"},"html":{"href":"https://github.com/apache/kafka/pull/2281"},"issue":{"href":"https://api.github.com/repos/apache/kafka/issues/2281"},"comments":{"href":"https://api.github.com/repos/apache/kafka/issues/2281/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/kafka/pulls/2281/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/kafka/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/kafka/pulls/2281/commits"},"statuses":{"href":"https://api.github.com/repos/apache/kafka/statuses/a1386034399dc63126b9a1261713ca140fb30c07"}}}}
{"actor":{"display_login":"guozhangwang"}}