{"action":"created","comment":{"url":"https://api.github.com/repos/twitter/heron/pulls/comments/82492556","pull_request_review_id":3399752,"id":82492556,"diff_hunk":"@@ -82,13 +82,36 @@ public static int runSyncProcess(\n   }\n \n   /**\n+   * Start a daemon thread to read data from \"input\" to \"out\".\n+   */\n+  private static Thread asyncProcessStream(final InputStream input, final StringBuilder out) {\n+    Thread thread = new Thread() {\n+      @Override\n+      public void run() {\n+        try {\n+          out.append(inputstreamToString(input));\n+        } finally {\n+          try {\n+            input.close();\n+          } catch (IOException e) {\n+            LOG.log(Level.SEVERE, \"Failed to close the input stream\", e);","path":"heron/spi/src/java/com/twitter/heron/spi/utils/ShellUtils.java","position":null,"original_position":16,"commit_id":"6c5852bdb295bb308ca0df8c36258b44111d272f","original_commit_id":"eb0009fd7c2a5a3b95bcbf0756e1c08ba29a88a4","user":{"login":"windie","id":4434300,"avatar_url":"https://avatars.githubusercontent.com/u/4434300?v=3","gravatar_id":"","url":"https://api.github.com/users/windie","html_url":"https://github.com/windie","followers_url":"https://api.github.com/users/windie/followers","following_url":"https://api.github.com/users/windie/following{/other_user}","gists_url":"https://api.github.com/users/windie/gists{/gist_id}","starred_url":"https://api.github.com/users/windie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/windie/subscriptions","organizations_url":"https://api.github.com/users/windie/orgs","repos_url":"https://api.github.com/users/windie/repos","events_url":"https://api.github.com/users/windie/events{/privacy}","received_events_url":"https://api.github.com/users/windie/received_events","type":"User","site_admin":false},"body":"> since we're logging this and moving on without throwing, we should change this to Level.WARN actually.\r\n\r\nFixed.","created_at":"2016-10-08T01:44:19Z","updated_at":"2016-10-08T01:44:19Z","html_url":"https://github.com/twitter/heron/pull/1477#discussion_r82492556","pull_request_url":"https://api.github.com/repos/twitter/heron/pulls/1477","_links":{"self":{"href":"https://api.github.com/repos/twitter/heron/pulls/comments/82492556"},"html":{"href":"https://github.com/twitter/heron/pull/1477#discussion_r82492556"},"pull_request":{"href":"https://api.github.com/repos/twitter/heron/pulls/1477"}}},"pull_request":{"url":"https://api.github.com/repos/twitter/heron/pulls/1477","id":88198402,"html_url":"https://github.com/twitter/heron/pull/1477","diff_url":"https://github.com/twitter/heron/pull/1477.diff","patch_url":"https://github.com/twitter/heron/pull/1477.patch","issue_url":"https://api.github.com/repos/twitter/heron/issues/1477","number":1477,"state":"open","locked":false,"title":"Do not let external processes block on IO buffers in ShellUtils.runSyncProcess","user":{"login":"windie","id":4434300,"avatar_url":"https://avatars.githubusercontent.com/u/4434300?v=3","gravatar_id":"","url":"https://api.github.com/users/windie","html_url":"https://github.com/windie","followers_url":"https://api.github.com/users/windie/followers","following_url":"https://api.github.com/users/windie/following{/other_user}","gists_url":"https://api.github.com/users/windie/gists{/gist_id}","starred_url":"https://api.github.com/users/windie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/windie/subscriptions","organizations_url":"https://api.github.com/users/windie/orgs","repos_url":"https://api.github.com/users/windie/repos","events_url":"https://api.github.com/users/windie/events{/privacy}","received_events_url":"https://api.github.com/users/windie/received_events","type":"User","site_admin":false},"body":"In ShellUtils.runSyncProcess, \"output from the external process can exhaust the available buffer for the output or error stream because neither stream is read while waiting for the process to complete. If either buffer becomes full, it can block the external process as well, preventing all progress for both the external process and the Java program.\" [1]\r\n\r\nThis PR just launches threads to read outputs asynchronously to fix this issue. The added unit tests can reproduce this bug.\r\n\r\n[1] https://www.securecoding.cert.org/confluence/display/java/FIO07-J.+Do+not+let+external+processes+block+on+IO+buffers","created_at":"2016-10-06T04:48:35Z","updated_at":"2016-10-08T01:44:19Z","closed_at":null,"merged_at":null,"merge_commit_sha":"3fbfbb6a92e4b1bc30d5014a21fe60f5188d0613","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/twitter/heron/pulls/1477/commits","review_comments_url":"https://api.github.com/repos/twitter/heron/pulls/1477/comments","review_comment_url":"https://api.github.com/repos/twitter/heron/pulls/comments{/number}","comments_url":"https://api.github.com/repos/twitter/heron/issues/1477/comments","statuses_url":"https://api.github.com/repos/twitter/heron/statuses/6c5852bdb295bb308ca0df8c36258b44111d272f","head":{"label":"windie:sync-process","ref":"sync-process","sha":"6c5852bdb295bb308ca0df8c36258b44111d272f","user":{"login":"windie","id":4434300,"avatar_url":"https://avatars.githubusercontent.com/u/4434300?v=3","gravatar_id":"","url":"https://api.github.com/users/windie","html_url":"https://github.com/windie","followers_url":"https://api.github.com/users/windie/followers","following_url":"https://api.github.com/users/windie/following{/other_user}","gists_url":"https://api.github.com/users/windie/gists{/gist_id}","starred_url":"https://api.github.com/users/windie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/windie/subscriptions","organizations_url":"https://api.github.com/users/windie/orgs","repos_url":"https://api.github.com/users/windie/repos","events_url":"https://api.github.com/users/windie/events{/privacy}","received_events_url":"https://api.github.com/users/windie/received_events","type":"User","site_admin":false},"repo":{"id":62265809,"name":"heron","full_name":"windie/heron","owner":{"login":"windie","id":4434300,"avatar_url":"https://avatars.githubusercontent.com/u/4434300?v=3","gravatar_id":"","url":"https://api.github.com/users/windie","html_url":"https://github.com/windie","followers_url":"https://api.github.com/users/windie/followers","following_url":"https://api.github.com/users/windie/following{/other_user}","gists_url":"https://api.github.com/users/windie/gists{/gist_id}","starred_url":"https://api.github.com/users/windie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/windie/subscriptions","organizations_url":"https://api.github.com/users/windie/orgs","repos_url":"https://api.github.com/users/windie/repos","events_url":"https://api.github.com/users/windie/events{/privacy}","received_events_url":"https://api.github.com/users/windie/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/windie/heron","description":"Heron is a realtime, distributed, fault-tolerant stream processing engine from Twitter","fork":true,"url":"https://api.github.com/repos/windie/heron","forks_url":"https://api.github.com/repos/windie/heron/forks","keys_url":"https://api.github.com/repos/windie/heron/keys{/key_id}","collaborators_url":"https://api.github.com/repos/windie/heron/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/windie/heron/teams","hooks_url":"https://api.github.com/repos/windie/heron/hooks","issue_events_url":"https://api.github.com/repos/windie/heron/issues/events{/number}","events_url":"https://api.github.com/repos/windie/heron/events","assignees_url":"https://api.github.com/repos/windie/heron/assignees{/user}","branches_url":"https://api.github.com/repos/windie/heron/branches{/branch}","tags_url":"https://api.github.com/repos/windie/heron/tags","blobs_url":"https://api.github.com/repos/windie/heron/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/windie/heron/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/windie/heron/git/refs{/sha}","trees_url":"https://api.github.com/repos/windie/heron/git/trees{/sha}","statuses_url":"https://api.github.com/repos/windie/heron/statuses/{sha}","languages_url":"https://api.github.com/repos/windie/heron/languages","stargazers_url":"https://api.github.com/repos/windie/heron/stargazers","contributors_url":"https://api.github.com/repos/windie/heron/contributors","subscribers_url":"https://api.github.com/repos/windie/heron/subscribers","subscription_url":"https://api.github.com/repos/windie/heron/subscription","commits_url":"https://api.github.com/repos/windie/heron/commits{/sha}","git_commits_url":"https://api.github.com/repos/windie/heron/git/commits{/sha}","comments_url":"https://api.github.com/repos/windie/heron/comments{/number}","issue_comment_url":"https://api.github.com/repos/windie/heron/issues/comments{/number}","contents_url":"https://api.github.com/repos/windie/heron/contents/{+path}","compare_url":"https://api.github.com/repos/windie/heron/compare/{base}...{head}","merges_url":"https://api.github.com/repos/windie/heron/merges","archive_url":"https://api.github.com/repos/windie/heron/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/windie/heron/downloads","issues_url":"https://api.github.com/repos/windie/heron/issues{/number}","pulls_url":"https://api.github.com/repos/windie/heron/pulls{/number}","milestones_url":"https://api.github.com/repos/windie/heron/milestones{/number}","notifications_url":"https://api.github.com/repos/windie/heron/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/windie/heron/labels{/name}","releases_url":"https://api.github.com/repos/windie/heron/releases{/id}","deployments_url":"https://api.github.com/repos/windie/heron/deployments","created_at":"2016-06-30T00:05:51Z","updated_at":"2016-06-30T00:05:55Z","pushed_at":"2016-10-08T01:43:38Z","git_url":"git://github.com/windie/heron.git","ssh_url":"git@github.com:windie/heron.git","clone_url":"https://github.com/windie/heron.git","svn_url":"https://github.com/windie/heron","homepage":"http://heronstreaming.io","size":113534,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"twitter:master","ref":"master","sha":"7d65a48c7d878c2fbcb380f79cf504217b4fa12a","user":{"login":"twitter","id":50278,"avatar_url":"https://avatars.githubusercontent.com/u/50278?v=3","gravatar_id":"","url":"https://api.github.com/users/twitter","html_url":"https://github.com/twitter","followers_url":"https://api.github.com/users/twitter/followers","following_url":"https://api.github.com/users/twitter/following{/other_user}","gists_url":"https://api.github.com/users/twitter/gists{/gist_id}","starred_url":"https://api.github.com/users/twitter/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twitter/subscriptions","organizations_url":"https://api.github.com/users/twitter/orgs","repos_url":"https://api.github.com/users/twitter/repos","events_url":"https://api.github.com/users/twitter/events{/privacy}","received_events_url":"https://api.github.com/users/twitter/received_events","type":"Organization","site_admin":false},"repo":{"id":43158694,"name":"heron","full_name":"twitter/heron","owner":{"login":"twitter","id":50278,"avatar_url":"https://avatars.githubusercontent.com/u/50278?v=3","gravatar_id":"","url":"https://api.github.com/users/twitter","html_url":"https://github.com/twitter","followers_url":"https://api.github.com/users/twitter/followers","following_url":"https://api.github.com/users/twitter/following{/other_user}","gists_url":"https://api.github.com/users/twitter/gists{/gist_id}","starred_url":"https://api.github.com/users/twitter/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twitter/subscriptions","organizations_url":"https://api.github.com/users/twitter/orgs","repos_url":"https://api.github.com/users/twitter/repos","events_url":"https://api.github.com/users/twitter/events{/privacy}","received_events_url":"https://api.github.com/users/twitter/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/twitter/heron","description":"Heron is a realtime, distributed, fault-tolerant stream processing engine from Twitter","fork":false,"url":"https://api.github.com/repos/twitter/heron","forks_url":"https://api.github.com/repos/twitter/heron/forks","keys_url":"https://api.github.com/repos/twitter/heron/keys{/key_id}","collaborators_url":"https://api.github.com/repos/twitter/heron/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/twitter/heron/teams","hooks_url":"https://api.github.com/repos/twitter/heron/hooks","issue_events_url":"https://api.github.com/repos/twitter/heron/issues/events{/number}","events_url":"https://api.github.com/repos/twitter/heron/events","assignees_url":"https://api.github.com/repos/twitter/heron/assignees{/user}","branches_url":"https://api.github.com/repos/twitter/heron/branches{/branch}","tags_url":"https://api.github.com/repos/twitter/heron/tags","blobs_url":"https://api.github.com/repos/twitter/heron/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/twitter/heron/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/twitter/heron/git/refs{/sha}","trees_url":"https://api.github.com/repos/twitter/heron/git/trees{/sha}","statuses_url":"https://api.github.com/repos/twitter/heron/statuses/{sha}","languages_url":"https://api.github.com/repos/twitter/heron/languages","stargazers_url":"https://api.github.com/repos/twitter/heron/stargazers","contributors_url":"https://api.github.com/repos/twitter/heron/contributors","subscribers_url":"https://api.github.com/repos/twitter/heron/subscribers","subscription_url":"https://api.github.com/repos/twitter/heron/subscription","commits_url":"https://api.github.com/repos/twitter/heron/commits{/sha}","git_commits_url":"https://api.github.com/repos/twitter/heron/git/commits{/sha}","comments_url":"https://api.github.com/repos/twitter/heron/comments{/number}","issue_comment_url":"https://api.github.com/repos/twitter/heron/issues/comments{/number}","contents_url":"https://api.github.com/repos/twitter/heron/contents/{+path}","compare_url":"https://api.github.com/repos/twitter/heron/compare/{base}...{head}","merges_url":"https://api.github.com/repos/twitter/heron/merges","archive_url":"https://api.github.com/repos/twitter/heron/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/twitter/heron/downloads","issues_url":"https://api.github.com/repos/twitter/heron/issues{/number}","pulls_url":"https://api.github.com/repos/twitter/heron/pulls{/number}","milestones_url":"https://api.github.com/repos/twitter/heron/milestones{/number}","notifications_url":"https://api.github.com/repos/twitter/heron/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/twitter/heron/labels{/name}","releases_url":"https://api.github.com/repos/twitter/heron/releases{/id}","deployments_url":"https://api.github.com/repos/twitter/heron/deployments","created_at":"2015-09-25T16:09:24Z","updated_at":"2016-10-07T18:53:59Z","pushed_at":"2016-10-08T01:43:38Z","git_url":"git://github.com/twitter/heron.git","ssh_url":"git@github.com:twitter/heron.git","clone_url":"https://github.com/twitter/heron.git","svn_url":"https://github.com/twitter/heron","homepage":"http://heronstreaming.io","size":147681,"stargazers_count":2408,"watchers_count":2408,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":401,"mirror_url":null,"open_issues_count":202,"forks":401,"open_issues":202,"watchers":2408,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/twitter/heron/pulls/1477"},"html":{"href":"https://github.com/twitter/heron/pull/1477"},"issue":{"href":"https://api.github.com/repos/twitter/heron/issues/1477"},"comments":{"href":"https://api.github.com/repos/twitter/heron/issues/1477/comments"},"review_comments":{"href":"https://api.github.com/repos/twitter/heron/pulls/1477/comments"},"review_comment":{"href":"https://api.github.com/repos/twitter/heron/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/twitter/heron/pulls/1477/commits"},"statuses":{"href":"https://api.github.com/repos/twitter/heron/statuses/6c5852bdb295bb308ca0df8c36258b44111d272f"}}}}
{"actor":{"display_login":"windie"}}