{"action":"created","comment":{"url":"https://api.github.com/repos/apache/beam/pulls/comments/226353525","pull_request_review_id":166156615,"id":226353525,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIyNjM1MzUyNQ==","diff_hunk":"@@ -159,49 +160,16 @@ private StateRequestHandler getStateRequestHandler(ExecutableStage executableSta\n     }\n   }\n \n-  // TODO: currently assumes that every element is a separate bundle,\n-  // but this can be changed by pushing some of this logic into the \"DoFnRunner\"\n-  private void processElementWithSdkHarness(WindowedValue<InputT> element) throws Exception {\n-    checkState(\n-        stageBundleFactory != null, \"%s not yet prepared\", StageBundleFactory.class.getName());\n-    checkState(\n-        stateRequestHandler != null, \"%s not yet prepared\", StateRequestHandler.class.getName());\n-\n-    OutputReceiverFactory receiverFactory =\n-        new OutputReceiverFactory() {\n-          @Override\n-          public FnDataReceiver<OutputT> create(String pCollectionId) {\n-            return (receivedElement) -> {\n-              // handover to queue, do not block the grpc thread\n-              outputQueue.put(KV.of(pCollectionId, receivedElement));\n-            };\n-          }\n-        };\n-\n-    try (RemoteBundle bundle =\n-        stageBundleFactory.getBundle(receiverFactory, stateRequestHandler, progressHandler)) {\n-      LOG.debug(String.format(\"Sending value: %s\", element));\n-      // TODO(BEAM-4681): Add support to Flink to support portable timers.\n-      Iterables.getOnlyElement(bundle.getInputReceivers().values()).accept(element);\n-      // TODO: it would be nice to emit results as they arrive, can thread wait non-blocking?\n-    }\n-\n-    // RemoteBundle close blocks until all results are received\n-    KV<String, OutputT> result;\n-    while ((result = outputQueue.poll()) != null) {\n-      outputManager.output(outputMap.get(result.getKey()), (WindowedValue) result.getValue());\n-    }\n-  }\n-\n   @Override\n-  public void close() throws Exception {\n+  public void dispose() throws Exception {","path":"runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/ExecutableStageDoFnOperator.java","position":48,"original_position":48,"commit_id":"9a748117cabac160cd3f237b18f01850ecd8ef95","original_commit_id":"1e350b49390a2103ea9ae1720b679eebef4c31bb","user":{"login":"tweise","id":263695,"node_id":"MDQ6VXNlcjI2MzY5NQ==","avatar_url":"https://avatars1.githubusercontent.com/u/263695?v=4","gravatar_id":"","url":"https://api.github.com/users/tweise","html_url":"https://github.com/tweise","followers_url":"https://api.github.com/users/tweise/followers","following_url":"https://api.github.com/users/tweise/following{/other_user}","gists_url":"https://api.github.com/users/tweise/gists{/gist_id}","starred_url":"https://api.github.com/users/tweise/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tweise/subscriptions","organizations_url":"https://api.github.com/users/tweise/orgs","repos_url":"https://api.github.com/users/tweise/repos","events_url":"https://api.github.com/users/tweise/events{/privacy}","received_events_url":"https://api.github.com/users/tweise/received_events","type":"User","site_admin":false},"body":"That's because DoFnOperator actually generates a bundle in `dispose`. That looks really funky at maybe should be looked at separately.","created_at":"2018-10-18T15:28:51Z","updated_at":"2018-10-18T15:28:51Z","html_url":"https://github.com/apache/beam/pull/6723#discussion_r226353525","pull_request_url":"https://api.github.com/repos/apache/beam/pulls/6723","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/apache/beam/pulls/comments/226353525"},"html":{"href":"https://github.com/apache/beam/pull/6723#discussion_r226353525"},"pull_request":{"href":"https://api.github.com/repos/apache/beam/pulls/6723"}},"in_reply_to_id":226012199},"pull_request":{"url":"https://api.github.com/repos/apache/beam/pulls/6723","id":223640605,"node_id":"MDExOlB1bGxSZXF1ZXN0MjIzNjQwNjA1","html_url":"https://github.com/apache/beam/pull/6723","diff_url":"https://github.com/apache/beam/pull/6723.diff","patch_url":"https://github.com/apache/beam/pull/6723.patch","issue_url":"https://api.github.com/repos/apache/beam/issues/6723","number":6723,"state":"open","locked":false,"title":"[WIP] [BEAM-5760] Add support for multi-element bundles to portable Flink runner.","user":{"login":"tweise","id":263695,"node_id":"MDQ6VXNlcjI2MzY5NQ==","avatar_url":"https://avatars1.githubusercontent.com/u/263695?v=4","gravatar_id":"","url":"https://api.github.com/users/tweise","html_url":"https://github.com/tweise","followers_url":"https://api.github.com/users/tweise/followers","following_url":"https://api.github.com/users/tweise/following{/other_user}","gists_url":"https://api.github.com/users/tweise/gists{/gist_id}","starred_url":"https://api.github.com/users/tweise/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tweise/subscriptions","organizations_url":"https://api.github.com/users/tweise/orgs","repos_url":"https://api.github.com/users/tweise/repos","events_url":"https://api.github.com/users/tweise/events{/privacy}","received_events_url":"https://api.github.com/users/tweise/received_events","type":"User","site_admin":false},"body":"This makes the operator compatible with the bundle control of the underlying `DoFnOperator`. Specifically, bundle size can be controlled via the maxBundleSize/maxBundleMillis pipeline options.\r\n\r\n------------------------\r\n\r\nFollow this checklist to help us incorporate your contribution quickly and easily:\r\n\r\n - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.\r\n - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).\r\n\r\nIt will help us expedite review of your Pull Request if you tag someone (e.g. `@username`) to look at it.\r\n\r\nPost-Commit Tests Status (on master branch)\r\n------------------------------------------------------------------------------------------------\r\n\r\nLang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark\r\n--- | --- | --- | --- | --- | --- | --- | ---\r\nGo | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/) | --- | --- | --- | --- | --- | ---\r\nJava | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)\r\nPython | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) </br> [![Build Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | --- | ---\r\n\r\n\r\n\r\n\r\n","created_at":"2018-10-17T15:02:39Z","updated_at":"2018-10-18T15:28:51Z","closed_at":null,"merged_at":null,"merge_commit_sha":"336d7ed47bb096ef9317a01d6ed30976307b2713","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/beam/pulls/6723/commits","review_comments_url":"https://api.github.com/repos/apache/beam/pulls/6723/comments","review_comment_url":"https://api.github.com/repos/apache/beam/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/beam/issues/6723/comments","statuses_url":"https://api.github.com/repos/apache/beam/statuses/9a748117cabac160cd3f237b18f01850ecd8ef95","head":{"label":"tweise:BEAM-5760.streamingBundle","ref":"BEAM-5760.streamingBundle","sha":"9a748117cabac160cd3f237b18f01850ecd8ef95","user":{"login":"tweise","id":263695,"node_id":"MDQ6VXNlcjI2MzY5NQ==","avatar_url":"https://avatars1.githubusercontent.com/u/263695?v=4","gravatar_id":"","url":"https://api.github.com/users/tweise","html_url":"https://github.com/tweise","followers_url":"https://api.github.com/users/tweise/followers","following_url":"https://api.github.com/users/tweise/following{/other_user}","gists_url":"https://api.github.com/users/tweise/gists{/gist_id}","starred_url":"https://api.github.com/users/tweise/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tweise/subscriptions","organizations_url":"https://api.github.com/users/tweise/orgs","repos_url":"https://api.github.com/users/tweise/repos","events_url":"https://api.github.com/users/tweise/events{/privacy}","received_events_url":"https://api.github.com/users/tweise/received_events","type":"User","site_admin":false},"repo":{"id":59848217,"node_id":"MDEwOlJlcG9zaXRvcnk1OTg0ODIxNw==","name":"beam","full_name":"tweise/beam","private":false,"owner":{"login":"tweise","id":263695,"node_id":"MDQ6VXNlcjI2MzY5NQ==","avatar_url":"https://avatars1.githubusercontent.com/u/263695?v=4","gravatar_id":"","url":"https://api.github.com/users/tweise","html_url":"https://github.com/tweise","followers_url":"https://api.github.com/users/tweise/followers","following_url":"https://api.github.com/users/tweise/following{/other_user}","gists_url":"https://api.github.com/users/tweise/gists{/gist_id}","starred_url":"https://api.github.com/users/tweise/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tweise/subscriptions","organizations_url":"https://api.github.com/users/tweise/orgs","repos_url":"https://api.github.com/users/tweise/repos","events_url":"https://api.github.com/users/tweise/events{/privacy}","received_events_url":"https://api.github.com/users/tweise/received_events","type":"User","site_admin":false},"html_url":"https://github.com/tweise/beam","description":"Mirror of Apache Beam (Incubating)","fork":true,"url":"https://api.github.com/repos/tweise/beam","forks_url":"https://api.github.com/repos/tweise/beam/forks","keys_url":"https://api.github.com/repos/tweise/beam/keys{/key_id}","collaborators_url":"https://api.github.com/repos/tweise/beam/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/tweise/beam/teams","hooks_url":"https://api.github.com/repos/tweise/beam/hooks","issue_events_url":"https://api.github.com/repos/tweise/beam/issues/events{/number}","events_url":"https://api.github.com/repos/tweise/beam/events","assignees_url":"https://api.github.com/repos/tweise/beam/assignees{/user}","branches_url":"https://api.github.com/repos/tweise/beam/branches{/branch}","tags_url":"https://api.github.com/repos/tweise/beam/tags","blobs_url":"https://api.github.com/repos/tweise/beam/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/tweise/beam/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/tweise/beam/git/refs{/sha}","trees_url":"https://api.github.com/repos/tweise/beam/git/trees{/sha}","statuses_url":"https://api.github.com/repos/tweise/beam/statuses/{sha}","languages_url":"https://api.github.com/repos/tweise/beam/languages","stargazers_url":"https://api.github.com/repos/tweise/beam/stargazers","contributors_url":"https://api.github.com/repos/tweise/beam/contributors","subscribers_url":"https://api.github.com/repos/tweise/beam/subscribers","subscription_url":"https://api.github.com/repos/tweise/beam/subscription","commits_url":"https://api.github.com/repos/tweise/beam/commits{/sha}","git_commits_url":"https://api.github.com/repos/tweise/beam/git/commits{/sha}","comments_url":"https://api.github.com/repos/tweise/beam/comments{/number}","issue_comment_url":"https://api.github.com/repos/tweise/beam/issues/comments{/number}","contents_url":"https://api.github.com/repos/tweise/beam/contents/{+path}","compare_url":"https://api.github.com/repos/tweise/beam/compare/{base}...{head}","merges_url":"https://api.github.com/repos/tweise/beam/merges","archive_url":"https://api.github.com/repos/tweise/beam/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/tweise/beam/downloads","issues_url":"https://api.github.com/repos/tweise/beam/issues{/number}","pulls_url":"https://api.github.com/repos/tweise/beam/pulls{/number}","milestones_url":"https://api.github.com/repos/tweise/beam/milestones{/number}","notifications_url":"https://api.github.com/repos/tweise/beam/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/tweise/beam/labels{/name}","releases_url":"https://api.github.com/repos/tweise/beam/releases{/id}","deployments_url":"https://api.github.com/repos/tweise/beam/deployments","created_at":"2016-05-27T16:24:56Z","updated_at":"2017-01-03T06:01:01Z","pushed_at":"2018-10-18T15:24:30Z","git_url":"git://github.com/tweise/beam.git","ssh_url":"git@github.com:tweise/beam.git","clone_url":"https://github.com/tweise/beam.git","svn_url":"https://github.com/tweise/beam","homepage":null,"size":107212,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"apache:master","ref":"master","sha":"d17d34d2c8fb9e44f3ec3491eef142b75c119a03","user":{"login":"apache","id":47359,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ3MzU5","avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":50904245,"node_id":"MDEwOlJlcG9zaXRvcnk1MDkwNDI0NQ==","name":"beam","full_name":"apache/beam","private":false,"owner":{"login":"apache","id":47359,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ3MzU5","avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/apache/beam","description":"Apache Beam","fork":false,"url":"https://api.github.com/repos/apache/beam","forks_url":"https://api.github.com/repos/apache/beam/forks","keys_url":"https://api.github.com/repos/apache/beam/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/beam/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/beam/teams","hooks_url":"https://api.github.com/repos/apache/beam/hooks","issue_events_url":"https://api.github.com/repos/apache/beam/issues/events{/number}","events_url":"https://api.github.com/repos/apache/beam/events","assignees_url":"https://api.github.com/repos/apache/beam/assignees{/user}","branches_url":"https://api.github.com/repos/apache/beam/branches{/branch}","tags_url":"https://api.github.com/repos/apache/beam/tags","blobs_url":"https://api.github.com/repos/apache/beam/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/beam/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/beam/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/beam/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/beam/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/beam/languages","stargazers_url":"https://api.github.com/repos/apache/beam/stargazers","contributors_url":"https://api.github.com/repos/apache/beam/contributors","subscribers_url":"https://api.github.com/repos/apache/beam/subscribers","subscription_url":"https://api.github.com/repos/apache/beam/subscription","commits_url":"https://api.github.com/repos/apache/beam/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/beam/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/beam/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/beam/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/beam/contents/{+path}","compare_url":"https://api.github.com/repos/apache/beam/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/beam/merges","archive_url":"https://api.github.com/repos/apache/beam/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/beam/downloads","issues_url":"https://api.github.com/repos/apache/beam/issues{/number}","pulls_url":"https://api.github.com/repos/apache/beam/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/beam/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/beam/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/beam/labels{/name}","releases_url":"https://api.github.com/repos/apache/beam/releases{/id}","deployments_url":"https://api.github.com/repos/apache/beam/deployments","created_at":"2016-02-02T08:00:06Z","updated_at":"2018-10-18T13:46:54Z","pushed_at":"2018-10-18T15:24:32Z","git_url":"git://github.com/apache/beam.git","ssh_url":"git@github.com:apache/beam.git","clone_url":"https://github.com/apache/beam.git","svn_url":"https://github.com/apache/beam","homepage":"https://beam.apache.org","size":108818,"stargazers_count":2248,"watchers_count":2248,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":1393,"mirror_url":null,"archived":false,"open_issues_count":75,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":1393,"open_issues":75,"watchers":2248,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/beam/pulls/6723"},"html":{"href":"https://github.com/apache/beam/pull/6723"},"issue":{"href":"https://api.github.com/repos/apache/beam/issues/6723"},"comments":{"href":"https://api.github.com/repos/apache/beam/issues/6723/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/beam/pulls/6723/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/beam/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/beam/pulls/6723/commits"},"statuses":{"href":"https://api.github.com/repos/apache/beam/statuses/9a748117cabac160cd3f237b18f01850ecd8ef95"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"tweise"}}