{"action":"created","comment":{"url":"https://api.github.com/repos/NebulousLabs/Sia/pulls/comments/71014945","id":71014945,"diff_hunk":"@@ -313,7 +295,25 @@ func (h *Host) ProcessConsensusChange(cc modules.ConsensusChange) {\n \t\th.log.Println(err)\n \t}\n \n-\t// Handle the list of action items.\n+\t// Handle the list of action items. Action items require host locks and\n+\t// potentially require waiting for long periods of time while various\n+\t// network communications finish. To prevent the host lock from being held\n+\t// while the network I/O is blocking the action item execution, the action\n+\t// items are handled in their own thread. A wait group is used to prevent\n+\t// the host from releasing ThreadGroup control until all threads that this\n+\t// function have spawned have completed.\n+\twg := new(sync.WaitGroup)\n+\t// The host should not release thread group control until all spawned\n+\t// threads have completed execution, though waiting for them should not\n+\t// block this function. The host should also not release thread group\n+\t// control until this function has terminated, hence the use of 'defer\n+\t// func() { go func() }' to coordinate waiting and done-ing.\n+\tdefer func() {\n+\t\tgo func() {\n+\t\t\twg.Wait()","path":"modules/host/update.go","position":79,"original_position":57,"commit_id":"4bf97d4d9e12e5a41992ce7e20d22cbeeb0688d3","original_commit_id":"4bf97d4d9e12e5a41992ce7e20d22cbeeb0688d3","user":{"login":"DavidVorick","id":2316841,"avatar_url":"https://avatars.githubusercontent.com/u/2316841?v=3","gravatar_id":"","url":"https://api.github.com/users/DavidVorick","html_url":"https://github.com/DavidVorick","followers_url":"https://api.github.com/users/DavidVorick/followers","following_url":"https://api.github.com/users/DavidVorick/following{/other_user}","gists_url":"https://api.github.com/users/DavidVorick/gists{/gist_id}","starred_url":"https://api.github.com/users/DavidVorick/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DavidVorick/subscriptions","organizations_url":"https://api.github.com/users/DavidVorick/orgs","repos_url":"https://api.github.com/users/DavidVorick/repos","events_url":"https://api.github.com/users/DavidVorick/events{/privacy}","received_events_url":"https://api.github.com/users/DavidVorick/received_events","type":"User","site_admin":false},"body":"It's not unbounded, because you will only spawn a thread to handle an action item if there is an action item, and since the host is locked you know that action items won't be getting added while you are spinning up the threads.\r\n\r\nThere is a call to `tg.Add()` at the beginning of `ProcessConsensusChange`, which means all future calls will be blocked. It's important for `Flush()` to wait until all action items have been handled, otherwise you aren't going to know (during testing) if the storage proof has been submitted yet. The purpose of `Flush()` is definitely to make it easy to know that certain actions have completed.\r\n\r\nI could potentially see issues if some 'attacker' has formed 10,000 file contracts with you that all expire in the same, but they pay a small price for each contract they form, and that's an attack vector regardless of how the threading is operating.\r\n\r\nSo basically, every action item that you have to deal with this block would have been created prior to the call to `ProcessConsensusChange` and would mean at least a little bit of income each.\r\n\r\n***if it's not clear, `Flush()` and `Stop()` will block all incoming calls to `tg.Add()` which occurs at the top of the `ProcessConsensusChange` function. So even if the call to `ProcessConsensusChange` completes, you know that there will not be another one to add more action items.","created_at":"2016-07-15T17:59:00Z","updated_at":"2016-07-15T17:59:00Z","html_url":"https://github.com/NebulousLabs/Sia/pull/1356#discussion_r71014945","pull_request_url":"https://api.github.com/repos/NebulousLabs/Sia/pulls/1356","_links":{"self":{"href":"https://api.github.com/repos/NebulousLabs/Sia/pulls/comments/71014945"},"html":{"href":"https://github.com/NebulousLabs/Sia/pull/1356#discussion_r71014945"},"pull_request":{"href":"https://api.github.com/repos/NebulousLabs/Sia/pulls/1356"}}},"pull_request":{"url":"https://api.github.com/repos/NebulousLabs/Sia/pulls/1356","id":77236773,"html_url":"https://github.com/NebulousLabs/Sia/pull/1356","diff_url":"https://github.com/NebulousLabs/Sia/pull/1356.diff","patch_url":"https://github.com/NebulousLabs/Sia/pull/1356.patch","issue_url":"https://api.github.com/repos/NebulousLabs/Sia/issues/1356","number":1356,"state":"open","locked":false,"title":"better storage obligation safety","user":{"login":"DavidVorick","id":2316841,"avatar_url":"https://avatars.githubusercontent.com/u/2316841?v=3","gravatar_id":"","url":"https://api.github.com/users/DavidVorick","html_url":"https://github.com/DavidVorick","followers_url":"https://api.github.com/users/DavidVorick/followers","following_url":"https://api.github.com/users/DavidVorick/following{/other_user}","gists_url":"https://api.github.com/users/DavidVorick/gists{/gist_id}","starred_url":"https://api.github.com/users/DavidVorick/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DavidVorick/subscriptions","organizations_url":"https://api.github.com/users/DavidVorick/orgs","repos_url":"https://api.github.com/users/DavidVorick/repos","events_url":"https://api.github.com/users/DavidVorick/events{/privacy}","received_events_url":"https://api.github.com/users/DavidVorick/received_events","type":"User","site_admin":false},"body":"The host now locks storage obligations before ever pulling them from the\r\ndatabase, and keeps the locks until the host is no longer going to be\r\nadding them to the database. This resolves some issues relating to\r\nstorage obligation corruption, which can cause errors like 'Consensus\r\nConflict' and can cause the host to report the wrong revision number, or\r\nto have the wrong sector roots, etc.","created_at":"2016-07-13T06:34:34Z","updated_at":"2016-07-15T17:59:00Z","closed_at":null,"merged_at":null,"merge_commit_sha":"09b279f735a537ae6351511d7bd7560f7b3eb0d5","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/NebulousLabs/Sia/pulls/1356/commits","review_comments_url":"https://api.github.com/repos/NebulousLabs/Sia/pulls/1356/comments","review_comment_url":"https://api.github.com/repos/NebulousLabs/Sia/pulls/comments{/number}","comments_url":"https://api.github.com/repos/NebulousLabs/Sia/issues/1356/comments","statuses_url":"https://api.github.com/repos/NebulousLabs/Sia/statuses/4bf97d4d9e12e5a41992ce7e20d22cbeeb0688d3","head":{"label":"NebulousLabs:obligation-safety","ref":"obligation-safety","sha":"4bf97d4d9e12e5a41992ce7e20d22cbeeb0688d3","user":{"login":"NebulousLabs","id":7471422,"avatar_url":"https://avatars.githubusercontent.com/u/7471422?v=3","gravatar_id":"","url":"https://api.github.com/users/NebulousLabs","html_url":"https://github.com/NebulousLabs","followers_url":"https://api.github.com/users/NebulousLabs/followers","following_url":"https://api.github.com/users/NebulousLabs/following{/other_user}","gists_url":"https://api.github.com/users/NebulousLabs/gists{/gist_id}","starred_url":"https://api.github.com/users/NebulousLabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NebulousLabs/subscriptions","organizations_url":"https://api.github.com/users/NebulousLabs/orgs","repos_url":"https://api.github.com/users/NebulousLabs/repos","events_url":"https://api.github.com/users/NebulousLabs/events{/privacy}","received_events_url":"https://api.github.com/users/NebulousLabs/received_events","type":"Organization","site_admin":false},"repo":{"id":25370072,"name":"Sia","full_name":"NebulousLabs/Sia","owner":{"login":"NebulousLabs","id":7471422,"avatar_url":"https://avatars.githubusercontent.com/u/7471422?v=3","gravatar_id":"","url":"https://api.github.com/users/NebulousLabs","html_url":"https://github.com/NebulousLabs","followers_url":"https://api.github.com/users/NebulousLabs/followers","following_url":"https://api.github.com/users/NebulousLabs/following{/other_user}","gists_url":"https://api.github.com/users/NebulousLabs/gists{/gist_id}","starred_url":"https://api.github.com/users/NebulousLabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NebulousLabs/subscriptions","organizations_url":"https://api.github.com/users/NebulousLabs/orgs","repos_url":"https://api.github.com/users/NebulousLabs/repos","events_url":"https://api.github.com/users/NebulousLabs/events{/privacy}","received_events_url":"https://api.github.com/users/NebulousLabs/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/NebulousLabs/Sia","description":"Blockchain-based marketplace for file storage","fork":false,"url":"https://api.github.com/repos/NebulousLabs/Sia","forks_url":"https://api.github.com/repos/NebulousLabs/Sia/forks","keys_url":"https://api.github.com/repos/NebulousLabs/Sia/keys{/key_id}","collaborators_url":"https://api.github.com/repos/NebulousLabs/Sia/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/NebulousLabs/Sia/teams","hooks_url":"https://api.github.com/repos/NebulousLabs/Sia/hooks","issue_events_url":"https://api.github.com/repos/NebulousLabs/Sia/issues/events{/number}","events_url":"https://api.github.com/repos/NebulousLabs/Sia/events","assignees_url":"https://api.github.com/repos/NebulousLabs/Sia/assignees{/user}","branches_url":"https://api.github.com/repos/NebulousLabs/Sia/branches{/branch}","tags_url":"https://api.github.com/repos/NebulousLabs/Sia/tags","blobs_url":"https://api.github.com/repos/NebulousLabs/Sia/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/NebulousLabs/Sia/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/NebulousLabs/Sia/git/refs{/sha}","trees_url":"https://api.github.com/repos/NebulousLabs/Sia/git/trees{/sha}","statuses_url":"https://api.github.com/repos/NebulousLabs/Sia/statuses/{sha}","languages_url":"https://api.github.com/repos/NebulousLabs/Sia/languages","stargazers_url":"https://api.github.com/repos/NebulousLabs/Sia/stargazers","contributors_url":"https://api.github.com/repos/NebulousLabs/Sia/contributors","subscribers_url":"https://api.github.com/repos/NebulousLabs/Sia/subscribers","subscription_url":"https://api.github.com/repos/NebulousLabs/Sia/subscription","commits_url":"https://api.github.com/repos/NebulousLabs/Sia/commits{/sha}","git_commits_url":"https://api.github.com/repos/NebulousLabs/Sia/git/commits{/sha}","comments_url":"https://api.github.com/repos/NebulousLabs/Sia/comments{/number}","issue_comment_url":"https://api.github.com/repos/NebulousLabs/Sia/issues/comments{/number}","contents_url":"https://api.github.com/repos/NebulousLabs/Sia/contents/{+path}","compare_url":"https://api.github.com/repos/NebulousLabs/Sia/compare/{base}...{head}","merges_url":"https://api.github.com/repos/NebulousLabs/Sia/merges","archive_url":"https://api.github.com/repos/NebulousLabs/Sia/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/NebulousLabs/Sia/downloads","issues_url":"https://api.github.com/repos/NebulousLabs/Sia/issues{/number}","pulls_url":"https://api.github.com/repos/NebulousLabs/Sia/pulls{/number}","milestones_url":"https://api.github.com/repos/NebulousLabs/Sia/milestones{/number}","notifications_url":"https://api.github.com/repos/NebulousLabs/Sia/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/NebulousLabs/Sia/labels{/name}","releases_url":"https://api.github.com/repos/NebulousLabs/Sia/releases{/id}","deployments_url":"https://api.github.com/repos/NebulousLabs/Sia/deployments","created_at":"2014-10-17T17:50:25Z","updated_at":"2016-07-15T09:14:09Z","pushed_at":"2016-07-15T07:58:08Z","git_url":"git://github.com/NebulousLabs/Sia.git","ssh_url":"git@github.com:NebulousLabs/Sia.git","clone_url":"https://github.com/NebulousLabs/Sia.git","svn_url":"https://github.com/NebulousLabs/Sia","homepage":"https://sia.tech","size":29599,"stargazers_count":311,"watchers_count":311,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":51,"mirror_url":null,"open_issues_count":44,"forks":51,"open_issues":44,"watchers":311,"default_branch":"master"}},"base":{"label":"NebulousLabs:master","ref":"master","sha":"bb82dd82c64e84bd50702a646b334435f4f157ee","user":{"login":"NebulousLabs","id":7471422,"avatar_url":"https://avatars.githubusercontent.com/u/7471422?v=3","gravatar_id":"","url":"https://api.github.com/users/NebulousLabs","html_url":"https://github.com/NebulousLabs","followers_url":"https://api.github.com/users/NebulousLabs/followers","following_url":"https://api.github.com/users/NebulousLabs/following{/other_user}","gists_url":"https://api.github.com/users/NebulousLabs/gists{/gist_id}","starred_url":"https://api.github.com/users/NebulousLabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NebulousLabs/subscriptions","organizations_url":"https://api.github.com/users/NebulousLabs/orgs","repos_url":"https://api.github.com/users/NebulousLabs/repos","events_url":"https://api.github.com/users/NebulousLabs/events{/privacy}","received_events_url":"https://api.github.com/users/NebulousLabs/received_events","type":"Organization","site_admin":false},"repo":{"id":25370072,"name":"Sia","full_name":"NebulousLabs/Sia","owner":{"login":"NebulousLabs","id":7471422,"avatar_url":"https://avatars.githubusercontent.com/u/7471422?v=3","gravatar_id":"","url":"https://api.github.com/users/NebulousLabs","html_url":"https://github.com/NebulousLabs","followers_url":"https://api.github.com/users/NebulousLabs/followers","following_url":"https://api.github.com/users/NebulousLabs/following{/other_user}","gists_url":"https://api.github.com/users/NebulousLabs/gists{/gist_id}","starred_url":"https://api.github.com/users/NebulousLabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NebulousLabs/subscriptions","organizations_url":"https://api.github.com/users/NebulousLabs/orgs","repos_url":"https://api.github.com/users/NebulousLabs/repos","events_url":"https://api.github.com/users/NebulousLabs/events{/privacy}","received_events_url":"https://api.github.com/users/NebulousLabs/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/NebulousLabs/Sia","description":"Blockchain-based marketplace for file storage","fork":false,"url":"https://api.github.com/repos/NebulousLabs/Sia","forks_url":"https://api.github.com/repos/NebulousLabs/Sia/forks","keys_url":"https://api.github.com/repos/NebulousLabs/Sia/keys{/key_id}","collaborators_url":"https://api.github.com/repos/NebulousLabs/Sia/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/NebulousLabs/Sia/teams","hooks_url":"https://api.github.com/repos/NebulousLabs/Sia/hooks","issue_events_url":"https://api.github.com/repos/NebulousLabs/Sia/issues/events{/number}","events_url":"https://api.github.com/repos/NebulousLabs/Sia/events","assignees_url":"https://api.github.com/repos/NebulousLabs/Sia/assignees{/user}","branches_url":"https://api.github.com/repos/NebulousLabs/Sia/branches{/branch}","tags_url":"https://api.github.com/repos/NebulousLabs/Sia/tags","blobs_url":"https://api.github.com/repos/NebulousLabs/Sia/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/NebulousLabs/Sia/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/NebulousLabs/Sia/git/refs{/sha}","trees_url":"https://api.github.com/repos/NebulousLabs/Sia/git/trees{/sha}","statuses_url":"https://api.github.com/repos/NebulousLabs/Sia/statuses/{sha}","languages_url":"https://api.github.com/repos/NebulousLabs/Sia/languages","stargazers_url":"https://api.github.com/repos/NebulousLabs/Sia/stargazers","contributors_url":"https://api.github.com/repos/NebulousLabs/Sia/contributors","subscribers_url":"https://api.github.com/repos/NebulousLabs/Sia/subscribers","subscription_url":"https://api.github.com/repos/NebulousLabs/Sia/subscription","commits_url":"https://api.github.com/repos/NebulousLabs/Sia/commits{/sha}","git_commits_url":"https://api.github.com/repos/NebulousLabs/Sia/git/commits{/sha}","comments_url":"https://api.github.com/repos/NebulousLabs/Sia/comments{/number}","issue_comment_url":"https://api.github.com/repos/NebulousLabs/Sia/issues/comments{/number}","contents_url":"https://api.github.com/repos/NebulousLabs/Sia/contents/{+path}","compare_url":"https://api.github.com/repos/NebulousLabs/Sia/compare/{base}...{head}","merges_url":"https://api.github.com/repos/NebulousLabs/Sia/merges","archive_url":"https://api.github.com/repos/NebulousLabs/Sia/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/NebulousLabs/Sia/downloads","issues_url":"https://api.github.com/repos/NebulousLabs/Sia/issues{/number}","pulls_url":"https://api.github.com/repos/NebulousLabs/Sia/pulls{/number}","milestones_url":"https://api.github.com/repos/NebulousLabs/Sia/milestones{/number}","notifications_url":"https://api.github.com/repos/NebulousLabs/Sia/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/NebulousLabs/Sia/labels{/name}","releases_url":"https://api.github.com/repos/NebulousLabs/Sia/releases{/id}","deployments_url":"https://api.github.com/repos/NebulousLabs/Sia/deployments","created_at":"2014-10-17T17:50:25Z","updated_at":"2016-07-15T09:14:09Z","pushed_at":"2016-07-15T07:58:08Z","git_url":"git://github.com/NebulousLabs/Sia.git","ssh_url":"git@github.com:NebulousLabs/Sia.git","clone_url":"https://github.com/NebulousLabs/Sia.git","svn_url":"https://github.com/NebulousLabs/Sia","homepage":"https://sia.tech","size":29599,"stargazers_count":311,"watchers_count":311,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":51,"mirror_url":null,"open_issues_count":44,"forks":51,"open_issues":44,"watchers":311,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/NebulousLabs/Sia/pulls/1356"},"html":{"href":"https://github.com/NebulousLabs/Sia/pull/1356"},"issue":{"href":"https://api.github.com/repos/NebulousLabs/Sia/issues/1356"},"comments":{"href":"https://api.github.com/repos/NebulousLabs/Sia/issues/1356/comments"},"review_comments":{"href":"https://api.github.com/repos/NebulousLabs/Sia/pulls/1356/comments"},"review_comment":{"href":"https://api.github.com/repos/NebulousLabs/Sia/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/NebulousLabs/Sia/pulls/1356/commits"},"statuses":{"href":"https://api.github.com/repos/NebulousLabs/Sia/statuses/4bf97d4d9e12e5a41992ce7e20d22cbeeb0688d3"}}}}
{"actor":{"display_login":"DavidVorick"}}