{"action":"created","comment":{"url":"https://api.github.com/repos/apache/storm/pulls/comments/69497481","id":69497481,"diff_hunk":"@@ -425,6 +433,13 @@\n          \". State: \" state\n          \", Heartbeat: \" (pr-str heartbeat))\n         (shutdown-worker supervisor id)))\n+\n+    (doseq [storm-id all-downloaded-storm-ids]","path":"storm-core/src/clj/org/apache/storm/daemon/supervisor.clj","position":53,"original_position":53,"commit_id":"39a9092367983d0b7bb9e902892498aa58d26997","original_commit_id":"39a9092367983d0b7bb9e902892498aa58d26997","user":{"login":"HeartSaVioR","id":1317309,"avatar_url":"https://avatars.githubusercontent.com/u/1317309?v=3","gravatar_id":"","url":"https://api.github.com/users/HeartSaVioR","html_url":"https://github.com/HeartSaVioR","followers_url":"https://api.github.com/users/HeartSaVioR/followers","following_url":"https://api.github.com/users/HeartSaVioR/following{/other_user}","gists_url":"https://api.github.com/users/HeartSaVioR/gists{/gist_id}","starred_url":"https://api.github.com/users/HeartSaVioR/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HeartSaVioR/subscriptions","organizations_url":"https://api.github.com/users/HeartSaVioR/orgs","repos_url":"https://api.github.com/users/HeartSaVioR/repos","events_url":"https://api.github.com/users/HeartSaVioR/events{/privacy}","received_events_url":"https://api.github.com/users/HeartSaVioR/received_events","type":"User","site_admin":false},"body":"@harshach Would I file a new issue regarding your suggestion, or do you want to file it?","created_at":"2016-07-05T00:52:49Z","updated_at":"2016-07-05T00:52:49Z","html_url":"https://github.com/apache/storm/pull/1528#discussion_r69497481","pull_request_url":"https://api.github.com/repos/apache/storm/pulls/1528","_links":{"self":{"href":"https://api.github.com/repos/apache/storm/pulls/comments/69497481"},"html":{"href":"https://github.com/apache/storm/pull/1528#discussion_r69497481"},"pull_request":{"href":"https://api.github.com/repos/apache/storm/pulls/1528"}}},"pull_request":{"url":"https://api.github.com/repos/apache/storm/pulls/1528","id":75604014,"html_url":"https://github.com/apache/storm/pull/1528","diff_url":"https://github.com/apache/storm/pull/1528.diff","patch_url":"https://github.com/apache/storm/pull/1528.patch","issue_url":"https://api.github.com/repos/apache/storm/issues/1528","number":1528,"state":"open","locked":false,"title":"STORM-1934 Fix race condition between sync-supervisor and sync-processes (1.x)","user":{"login":"HeartSaVioR","id":1317309,"avatar_url":"https://avatars.githubusercontent.com/u/1317309?v=3","gravatar_id":"","url":"https://api.github.com/users/HeartSaVioR","html_url":"https://github.com/HeartSaVioR","followers_url":"https://api.github.com/users/HeartSaVioR/followers","following_url":"https://api.github.com/users/HeartSaVioR/following{/other_user}","gists_url":"https://api.github.com/users/HeartSaVioR/gists{/gist_id}","starred_url":"https://api.github.com/users/HeartSaVioR/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HeartSaVioR/subscriptions","organizations_url":"https://api.github.com/users/HeartSaVioR/orgs","repos_url":"https://api.github.com/users/HeartSaVioR/repos","events_url":"https://api.github.com/users/HeartSaVioR/events{/privacy}","received_events_url":"https://api.github.com/users/HeartSaVioR/received_events","type":"User","site_admin":false},"body":"Please refer [STORM-1933](https://issues.apache.org/jira/browse/STORM-1933) and [STORM-1934](https://issues.apache.org/jira/browse/STORM-1934) to see reason of this issue and concept to fix.\r\n\r\n* sync-supervisor just downloads new topology code and writes new local assignment\r\n  * shutting down workers and removing topology code is moved to sync-processes\r\n* sync-processes does all of jobs based on local assignment and allocated workers\r\n* remove unused / unneeded codes\r\n\r\nPR for master: #1529 \r\n\r\nHere's my test result for this patch:\r\n\r\n* `mvn clean install` 5 times: not met supervisor intermittent failure (STORM-1933)\r\n  * will try more times\r\n* kill worker via `kill`, `kill -9`, `restart worker` from UI: no issue on restarting worker\r\n* rebalance topology to change workers (3 -> 2): to test that new assignment has same worker port but different executors compared to assigned workers\r\n  * worker is recognized as :disallowed, and killed & relaunched\r\n\r\nRebalance test in details:\r\n\r\n- Writing new assignment\r\n```\r\n6701 {:storm-id \"test-topology2-4-1467185073\", :executors ([7 7] [5 5] [3 3] [1 1]), :resources [0.0 0.0 0.0]}, \r\n6702 {:storm-id \"test-topology2-4-1467185073\", :executors ([6 6] [4 4] [2 2]), :resources [0.0 0.0 0.0]}\r\n```\r\n\r\n- Assigned executors:\r\n```\r\n6701 {:storm-id \"test-topology2-4-1467185073\", :executors [[7 7] [5 5] [3 3] [1 1]], :resources #object[org.apache.storm.generated.WorkerResources 0x40c4d31c \"WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)\"]}, \r\n6702 {:storm-id \"test-topology2-4-1467185073\", :executors [[6 6] [4 4] [2 2]], :resources #object[org.apache.storm.generated.WorkerResources 0x4ba861f4 \"WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)\"]}}\r\n```\r\n\r\n- Allocated:\r\n```\r\n\"2e9bea10-02b7-4e55-88e7-b194b9917a63\" [:disallowed {:time-secs 1467185407, :storm-id \"test-topology2-4-1467185073\", :executors [[3 3] [6 6] [-1 -1]], :port 6703}], \r\n\"4630c4bf-9786-47ff-9f3b-6b42d9781b9d\" [:disallowed {:time-secs 1467185407, :storm-id \"test-topology2-4-1467185073\", :executors [[7 7] [1 1] [-1 -1] [4 4]], :port 6701}], \r\n\"b9a622d2-5e5b-4311-999c-8c8dd92da6b6\" [:disallowed {:time-secs 1467185406, :storm-id \"test-topology2-4-1467185073\", :executors [[2 2] [-1 -1] [5 5]], :port 6702}]}\r\n```\r\n\r\nNOTE: Due to forward reference, I have to move `sync-processes` to just before `mk-synchronize-supervisor`. Major changes are done in sync-processes so reviewers need to compare before & after manually. Sorry about that.\r\n\r\nSince supervisor.clj is already ported to Java in master branch, I should have time to read ported code, and modify to be in sync.\r\n\r\nPlease review and comment while I'm working against master branch. Thanks!","created_at":"2016-06-29T08:01:28Z","updated_at":"2016-07-05T00:52:49Z","closed_at":null,"merged_at":null,"merge_commit_sha":"730e50ab0e049abd4e08ff2c9efcc2562512c0fc","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/storm/pulls/1528/commits","review_comments_url":"https://api.github.com/repos/apache/storm/pulls/1528/comments","review_comment_url":"https://api.github.com/repos/apache/storm/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/storm/issues/1528/comments","statuses_url":"https://api.github.com/repos/apache/storm/statuses/39a9092367983d0b7bb9e902892498aa58d26997","head":{"label":"HeartSaVioR:STORM-1934-1.x","ref":"STORM-1934-1.x","sha":"39a9092367983d0b7bb9e902892498aa58d26997","user":{"login":"HeartSaVioR","id":1317309,"avatar_url":"https://avatars.githubusercontent.com/u/1317309?v=3","gravatar_id":"","url":"https://api.github.com/users/HeartSaVioR","html_url":"https://github.com/HeartSaVioR","followers_url":"https://api.github.com/users/HeartSaVioR/followers","following_url":"https://api.github.com/users/HeartSaVioR/following{/other_user}","gists_url":"https://api.github.com/users/HeartSaVioR/gists{/gist_id}","starred_url":"https://api.github.com/users/HeartSaVioR/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HeartSaVioR/subscriptions","organizations_url":"https://api.github.com/users/HeartSaVioR/orgs","repos_url":"https://api.github.com/users/HeartSaVioR/repos","events_url":"https://api.github.com/users/HeartSaVioR/events{/privacy}","received_events_url":"https://api.github.com/users/HeartSaVioR/received_events","type":"User","site_admin":false},"repo":{"id":24730103,"name":"storm","full_name":"HeartSaVioR/storm","owner":{"login":"HeartSaVioR","id":1317309,"avatar_url":"https://avatars.githubusercontent.com/u/1317309?v=3","gravatar_id":"","url":"https://api.github.com/users/HeartSaVioR","html_url":"https://github.com/HeartSaVioR","followers_url":"https://api.github.com/users/HeartSaVioR/followers","following_url":"https://api.github.com/users/HeartSaVioR/following{/other_user}","gists_url":"https://api.github.com/users/HeartSaVioR/gists{/gist_id}","starred_url":"https://api.github.com/users/HeartSaVioR/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HeartSaVioR/subscriptions","organizations_url":"https://api.github.com/users/HeartSaVioR/orgs","repos_url":"https://api.github.com/users/HeartSaVioR/repos","events_url":"https://api.github.com/users/HeartSaVioR/events{/privacy}","received_events_url":"https://api.github.com/users/HeartSaVioR/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/HeartSaVioR/storm","description":"Mirror of Apache Storm","fork":true,"url":"https://api.github.com/repos/HeartSaVioR/storm","forks_url":"https://api.github.com/repos/HeartSaVioR/storm/forks","keys_url":"https://api.github.com/repos/HeartSaVioR/storm/keys{/key_id}","collaborators_url":"https://api.github.com/repos/HeartSaVioR/storm/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/HeartSaVioR/storm/teams","hooks_url":"https://api.github.com/repos/HeartSaVioR/storm/hooks","issue_events_url":"https://api.github.com/repos/HeartSaVioR/storm/issues/events{/number}","events_url":"https://api.github.com/repos/HeartSaVioR/storm/events","assignees_url":"https://api.github.com/repos/HeartSaVioR/storm/assignees{/user}","branches_url":"https://api.github.com/repos/HeartSaVioR/storm/branches{/branch}","tags_url":"https://api.github.com/repos/HeartSaVioR/storm/tags","blobs_url":"https://api.github.com/repos/HeartSaVioR/storm/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/HeartSaVioR/storm/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/HeartSaVioR/storm/git/refs{/sha}","trees_url":"https://api.github.com/repos/HeartSaVioR/storm/git/trees{/sha}","statuses_url":"https://api.github.com/repos/HeartSaVioR/storm/statuses/{sha}","languages_url":"https://api.github.com/repos/HeartSaVioR/storm/languages","stargazers_url":"https://api.github.com/repos/HeartSaVioR/storm/stargazers","contributors_url":"https://api.github.com/repos/HeartSaVioR/storm/contributors","subscribers_url":"https://api.github.com/repos/HeartSaVioR/storm/subscribers","subscription_url":"https://api.github.com/repos/HeartSaVioR/storm/subscription","commits_url":"https://api.github.com/repos/HeartSaVioR/storm/commits{/sha}","git_commits_url":"https://api.github.com/repos/HeartSaVioR/storm/git/commits{/sha}","comments_url":"https://api.github.com/repos/HeartSaVioR/storm/comments{/number}","issue_comment_url":"https://api.github.com/repos/HeartSaVioR/storm/issues/comments{/number}","contents_url":"https://api.github.com/repos/HeartSaVioR/storm/contents/{+path}","compare_url":"https://api.github.com/repos/HeartSaVioR/storm/compare/{base}...{head}","merges_url":"https://api.github.com/repos/HeartSaVioR/storm/merges","archive_url":"https://api.github.com/repos/HeartSaVioR/storm/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/HeartSaVioR/storm/downloads","issues_url":"https://api.github.com/repos/HeartSaVioR/storm/issues{/number}","pulls_url":"https://api.github.com/repos/HeartSaVioR/storm/pulls{/number}","milestones_url":"https://api.github.com/repos/HeartSaVioR/storm/milestones{/number}","notifications_url":"https://api.github.com/repos/HeartSaVioR/storm/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/HeartSaVioR/storm/labels{/name}","releases_url":"https://api.github.com/repos/HeartSaVioR/storm/releases{/id}","deployments_url":"https://api.github.com/repos/HeartSaVioR/storm/deployments","created_at":"2014-10-02T18:00:01Z","updated_at":"2016-01-13T23:55:58Z","pushed_at":"2016-07-01T14:58:37Z","git_url":"git://github.com/HeartSaVioR/storm.git","ssh_url":"git@github.com:HeartSaVioR/storm.git","clone_url":"https://github.com/HeartSaVioR/storm.git","svn_url":"https://github.com/HeartSaVioR/storm","homepage":null,"size":104939,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"apache:1.x-branch","ref":"1.x-branch","sha":"af0c2a3d27797f3c2c6d4191a4f4637faf17bab2","user":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":14135470,"name":"storm","full_name":"apache/storm","owner":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/apache/storm","description":"Mirror of Apache Storm","fork":false,"url":"https://api.github.com/repos/apache/storm","forks_url":"https://api.github.com/repos/apache/storm/forks","keys_url":"https://api.github.com/repos/apache/storm/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/storm/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/storm/teams","hooks_url":"https://api.github.com/repos/apache/storm/hooks","issue_events_url":"https://api.github.com/repos/apache/storm/issues/events{/number}","events_url":"https://api.github.com/repos/apache/storm/events","assignees_url":"https://api.github.com/repos/apache/storm/assignees{/user}","branches_url":"https://api.github.com/repos/apache/storm/branches{/branch}","tags_url":"https://api.github.com/repos/apache/storm/tags","blobs_url":"https://api.github.com/repos/apache/storm/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/storm/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/storm/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/storm/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/storm/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/storm/languages","stargazers_url":"https://api.github.com/repos/apache/storm/stargazers","contributors_url":"https://api.github.com/repos/apache/storm/contributors","subscribers_url":"https://api.github.com/repos/apache/storm/subscribers","subscription_url":"https://api.github.com/repos/apache/storm/subscription","commits_url":"https://api.github.com/repos/apache/storm/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/storm/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/storm/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/storm/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/storm/contents/{+path}","compare_url":"https://api.github.com/repos/apache/storm/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/storm/merges","archive_url":"https://api.github.com/repos/apache/storm/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/storm/downloads","issues_url":"https://api.github.com/repos/apache/storm/issues{/number}","pulls_url":"https://api.github.com/repos/apache/storm/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/storm/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/storm/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/storm/labels{/name}","releases_url":"https://api.github.com/repos/apache/storm/releases{/id}","deployments_url":"https://api.github.com/repos/apache/storm/deployments","created_at":"2013-11-05T08:00:14Z","updated_at":"2016-07-05T00:21:18Z","pushed_at":"2016-07-05T00:05:11Z","git_url":"git://github.com/apache/storm.git","ssh_url":"git@github.com:apache/storm.git","clone_url":"https://github.com/apache/storm.git","svn_url":"https://github.com/apache/storm","homepage":null,"size":171194,"stargazers_count":3299,"watchers_count":3299,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":2462,"mirror_url":"git://git.apache.org/storm.git","open_issues_count":136,"forks":2462,"open_issues":136,"watchers":3299,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/storm/pulls/1528"},"html":{"href":"https://github.com/apache/storm/pull/1528"},"issue":{"href":"https://api.github.com/repos/apache/storm/issues/1528"},"comments":{"href":"https://api.github.com/repos/apache/storm/issues/1528/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/storm/pulls/1528/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/storm/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/storm/pulls/1528/commits"},"statuses":{"href":"https://api.github.com/repos/apache/storm/statuses/39a9092367983d0b7bb9e902892498aa58d26997"}}}}
{"actor":{"display_login":"HeartSaVioR"}}