{"action":"created","comment":{"url":"https://api.github.com/repos/tendermint/tendermint/pulls/comments/77665589","id":77665589,"diff_hunk":"@@ -185,6 +318,7 @@ func (voteSet *VoteSet) BitArray() *BitArray {\n \treturn voteSet.votesBitArray.Copy()\n }\n \n+// NOTE: if validator has conflicting votes, picks random.","path":"types/vote_set.go","position":null,"original_position":346,"commit_id":"fa676f7e6be7f8783e6e2831f4d65773b9e19092","original_commit_id":"5b3289a2e3455c0e499a1ffd7c047232cdac3c6f","user":{"login":"jaekwon","id":53785,"avatar_url":"https://avatars.githubusercontent.com/u/53785?v=3","gravatar_id":"","url":"https://api.github.com/users/jaekwon","html_url":"https://github.com/jaekwon","followers_url":"https://api.github.com/users/jaekwon/followers","following_url":"https://api.github.com/users/jaekwon/following{/other_user}","gists_url":"https://api.github.com/users/jaekwon/gists{/gist_id}","starred_url":"https://api.github.com/users/jaekwon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jaekwon/subscriptions","organizations_url":"https://api.github.com/users/jaekwon/orgs","repos_url":"https://api.github.com/users/jaekwon/repos","events_url":"https://api.github.com/users/jaekwon/events{/privacy}","received_events_url":"https://api.github.com/users/jaekwon/received_events","type":"User","site_admin":false},"body":"fixed","created_at":"2016-09-06T16:05:13Z","updated_at":"2016-09-06T16:05:13Z","html_url":"https://github.com/tendermint/tendermint/pull/229#discussion_r77665589","pull_request_url":"https://api.github.com/repos/tendermint/tendermint/pulls/229","_links":{"self":{"href":"https://api.github.com/repos/tendermint/tendermint/pulls/comments/77665589"},"html":{"href":"https://github.com/tendermint/tendermint/pull/229#discussion_r77665589"},"pull_request":{"href":"https://api.github.com/repos/tendermint/tendermint/pulls/229"}}},"pull_request":{"url":"https://api.github.com/repos/tendermint/tendermint/pulls/229","id":76106845,"html_url":"https://github.com/tendermint/tendermint/pull/229","diff_url":"https://github.com/tendermint/tendermint/pull/229.diff","patch_url":"https://github.com/tendermint/tendermint/pull/229.patch","issue_url":"https://api.github.com/repos/tendermint/tendermint/issues/229","number":229,"state":"open","locked":false,"title":"Fix issue with double-signing causing blockage","user":{"login":"jaekwon","id":53785,"avatar_url":"https://avatars.githubusercontent.com/u/53785?v=3","gravatar_id":"","url":"https://api.github.com/users/jaekwon","html_url":"https://github.com/jaekwon","followers_url":"https://api.github.com/users/jaekwon/followers","following_url":"https://api.github.com/users/jaekwon/following{/other_user}","gists_url":"https://api.github.com/users/jaekwon/gists{/gist_id}","starred_url":"https://api.github.com/users/jaekwon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jaekwon/subscriptions","organizations_url":"https://api.github.com/users/jaekwon/orgs","repos_url":"https://api.github.com/users/jaekwon/repos","events_url":"https://api.github.com/users/jaekwon/events{/privacy}","received_events_url":"https://api.github.com/users/jaekwon/received_events","type":"User","site_admin":false},"body":"This is not complete, but I'll document the thinking process while working on it.\r\n\r\nThe problem is that, the way things are, due to the way peers communicate to each other which votes it has, if a validator double-signs for a given height/round/step, then it can prevent its own other double-votes from propagating correct, e.g. \"occluding\".\r\n\r\nThe case in which this was discovered to be an issue was during a BFT fuzz test, where it halted the blockchain with less than 1/3 of Byzantine validators, due to the Commit being occluded.\r\n\r\nWhat are all the issues related to double-signing occlusion?\r\n- 2/3 Precommits, or a Commit, from being broadcast and letting the network progress in height\r\n- 2/3 Prevotes, or a POL, from being broadcast and letting validators unlock to let consensus progress\r\n\r\nWhat about when there are more than 1 2/3 majority for a prevote or precommit step, due to 1/3+ being Byzantine?  Currently we don't have logic to automatically broadcast this issue.  This would be a good time to implement that logic, so we fix BFT as well as the first step in identifying an attack.  This isn't the same as Tendermint V2, where a Commit is fully justified, but, it would be ideal to work toward that, while we fix this issue.\r\n\r\nSo, we want to:\r\n- prevent occlusion when < 1/3 have double-signed\r\n- ensure broadcasting of double-signing by each validator\r\n- ensure broadcasting of 2 conflicting 2/3 precommits (forks)\r\n- maybe, ensure broadcasting of 2 conflicting 2/3 prevotes\r\n\r\nI don't have the whole solution yet, but the first step might be to refactor the VoteSet structure to allow tracking of double-signing.  But we need to make sure this is done in such a way that double-signers can't sign a million votes and make nodes run out of memory, or cause some other issue.\r\n\r\nThe general solution I have in mind is to do the following:\r\n- By default, allow VoteSet to keep track of up to 2 conflicting votes for each validator\r\n- Implement a method on VoteSet to allow, for each unique peer, one 2/3-majority blockhash to be tracked as extra, beyond the 2 maximum conflicting votes.  So even though we have votes from ByzVal1 for BlockA and BlockB, if Peer1 says it has a 2/3-majority blockhash for BlockC, then we allow this VoteSet to also remember a third vote for ByzVal1 for BlockC.\r\n- It is the responsibility of the consensus reactor to manage calling the above method, as well as any cleanup methods, upon peer connection and disconnection, and communicate 2/3-majority blockhash announcements as necessary.\r\n\r\nAs I implement these features in VoteSet, I'll make sure the functionality is reusable for Tendermint V2, so that justifications can be requested for (H,R,S) as necessary.","created_at":"2016-07-04T02:38:37Z","updated_at":"2016-09-06T16:05:13Z","closed_at":null,"merged_at":null,"merge_commit_sha":"0926c36ed6b1fb22f9ac5874dfa895a24eb90f46","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/tendermint/tendermint/pulls/229/commits","review_comments_url":"https://api.github.com/repos/tendermint/tendermint/pulls/229/comments","review_comment_url":"https://api.github.com/repos/tendermint/tendermint/pulls/comments{/number}","comments_url":"https://api.github.com/repos/tendermint/tendermint/issues/229/comments","statuses_url":"https://api.github.com/repos/tendermint/tendermint/statuses/fa676f7e6be7f8783e6e2831f4d65773b9e19092","head":{"label":"tendermint:bft_fix_2","ref":"bft_fix_2","sha":"fa676f7e6be7f8783e6e2831f4d65773b9e19092","user":{"login":"tendermint","id":7572940,"avatar_url":"https://avatars.githubusercontent.com/u/7572940?v=3","gravatar_id":"","url":"https://api.github.com/users/tendermint","html_url":"https://github.com/tendermint","followers_url":"https://api.github.com/users/tendermint/followers","following_url":"https://api.github.com/users/tendermint/following{/other_user}","gists_url":"https://api.github.com/users/tendermint/gists{/gist_id}","starred_url":"https://api.github.com/users/tendermint/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tendermint/subscriptions","organizations_url":"https://api.github.com/users/tendermint/orgs","repos_url":"https://api.github.com/users/tendermint/repos","events_url":"https://api.github.com/users/tendermint/events{/privacy}","received_events_url":"https://api.github.com/users/tendermint/received_events","type":"Organization","site_admin":false},"repo":{"id":19799491,"name":"tendermint","full_name":"tendermint/tendermint","owner":{"login":"tendermint","id":7572940,"avatar_url":"https://avatars.githubusercontent.com/u/7572940?v=3","gravatar_id":"","url":"https://api.github.com/users/tendermint","html_url":"https://github.com/tendermint","followers_url":"https://api.github.com/users/tendermint/followers","following_url":"https://api.github.com/users/tendermint/following{/other_user}","gists_url":"https://api.github.com/users/tendermint/gists{/gist_id}","starred_url":"https://api.github.com/users/tendermint/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tendermint/subscriptions","organizations_url":"https://api.github.com/users/tendermint/orgs","repos_url":"https://api.github.com/users/tendermint/repos","events_url":"https://api.github.com/users/tendermint/events{/privacy}","received_events_url":"https://api.github.com/users/tendermint/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/tendermint/tendermint","description":"Tendermint Core in Go","fork":false,"url":"https://api.github.com/repos/tendermint/tendermint","forks_url":"https://api.github.com/repos/tendermint/tendermint/forks","keys_url":"https://api.github.com/repos/tendermint/tendermint/keys{/key_id}","collaborators_url":"https://api.github.com/repos/tendermint/tendermint/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/tendermint/tendermint/teams","hooks_url":"https://api.github.com/repos/tendermint/tendermint/hooks","issue_events_url":"https://api.github.com/repos/tendermint/tendermint/issues/events{/number}","events_url":"https://api.github.com/repos/tendermint/tendermint/events","assignees_url":"https://api.github.com/repos/tendermint/tendermint/assignees{/user}","branches_url":"https://api.github.com/repos/tendermint/tendermint/branches{/branch}","tags_url":"https://api.github.com/repos/tendermint/tendermint/tags","blobs_url":"https://api.github.com/repos/tendermint/tendermint/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/tendermint/tendermint/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/tendermint/tendermint/git/refs{/sha}","trees_url":"https://api.github.com/repos/tendermint/tendermint/git/trees{/sha}","statuses_url":"https://api.github.com/repos/tendermint/tendermint/statuses/{sha}","languages_url":"https://api.github.com/repos/tendermint/tendermint/languages","stargazers_url":"https://api.github.com/repos/tendermint/tendermint/stargazers","contributors_url":"https://api.github.com/repos/tendermint/tendermint/contributors","subscribers_url":"https://api.github.com/repos/tendermint/tendermint/subscribers","subscription_url":"https://api.github.com/repos/tendermint/tendermint/subscription","commits_url":"https://api.github.com/repos/tendermint/tendermint/commits{/sha}","git_commits_url":"https://api.github.com/repos/tendermint/tendermint/git/commits{/sha}","comments_url":"https://api.github.com/repos/tendermint/tendermint/comments{/number}","issue_comment_url":"https://api.github.com/repos/tendermint/tendermint/issues/comments{/number}","contents_url":"https://api.github.com/repos/tendermint/tendermint/contents/{+path}","compare_url":"https://api.github.com/repos/tendermint/tendermint/compare/{base}...{head}","merges_url":"https://api.github.com/repos/tendermint/tendermint/merges","archive_url":"https://api.github.com/repos/tendermint/tendermint/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/tendermint/tendermint/downloads","issues_url":"https://api.github.com/repos/tendermint/tendermint/issues{/number}","pulls_url":"https://api.github.com/repos/tendermint/tendermint/pulls{/number}","milestones_url":"https://api.github.com/repos/tendermint/tendermint/milestones{/number}","notifications_url":"https://api.github.com/repos/tendermint/tendermint/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/tendermint/tendermint/labels{/name}","releases_url":"https://api.github.com/repos/tendermint/tendermint/releases{/id}","deployments_url":"https://api.github.com/repos/tendermint/tendermint/deployments","created_at":"2014-05-14T23:21:35Z","updated_at":"2016-09-06T04:01:48Z","pushed_at":"2016-09-06T01:48:49Z","git_url":"git://github.com/tendermint/tendermint.git","ssh_url":"git@github.com:tendermint/tendermint.git","clone_url":"https://github.com/tendermint/tendermint.git","svn_url":"https://github.com/tendermint/tendermint","homepage":"http://tendermint.com/","size":5503,"stargazers_count":286,"watchers_count":286,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":48,"mirror_url":null,"open_issues_count":50,"forks":48,"open_issues":50,"watchers":286,"default_branch":"master"}},"base":{"label":"tendermint:master","ref":"master","sha":"64f773c88644650f0ca550ee744ad9adada5e994","user":{"login":"tendermint","id":7572940,"avatar_url":"https://avatars.githubusercontent.com/u/7572940?v=3","gravatar_id":"","url":"https://api.github.com/users/tendermint","html_url":"https://github.com/tendermint","followers_url":"https://api.github.com/users/tendermint/followers","following_url":"https://api.github.com/users/tendermint/following{/other_user}","gists_url":"https://api.github.com/users/tendermint/gists{/gist_id}","starred_url":"https://api.github.com/users/tendermint/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tendermint/subscriptions","organizations_url":"https://api.github.com/users/tendermint/orgs","repos_url":"https://api.github.com/users/tendermint/repos","events_url":"https://api.github.com/users/tendermint/events{/privacy}","received_events_url":"https://api.github.com/users/tendermint/received_events","type":"Organization","site_admin":false},"repo":{"id":19799491,"name":"tendermint","full_name":"tendermint/tendermint","owner":{"login":"tendermint","id":7572940,"avatar_url":"https://avatars.githubusercontent.com/u/7572940?v=3","gravatar_id":"","url":"https://api.github.com/users/tendermint","html_url":"https://github.com/tendermint","followers_url":"https://api.github.com/users/tendermint/followers","following_url":"https://api.github.com/users/tendermint/following{/other_user}","gists_url":"https://api.github.com/users/tendermint/gists{/gist_id}","starred_url":"https://api.github.com/users/tendermint/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tendermint/subscriptions","organizations_url":"https://api.github.com/users/tendermint/orgs","repos_url":"https://api.github.com/users/tendermint/repos","events_url":"https://api.github.com/users/tendermint/events{/privacy}","received_events_url":"https://api.github.com/users/tendermint/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/tendermint/tendermint","description":"Tendermint Core in Go","fork":false,"url":"https://api.github.com/repos/tendermint/tendermint","forks_url":"https://api.github.com/repos/tendermint/tendermint/forks","keys_url":"https://api.github.com/repos/tendermint/tendermint/keys{/key_id}","collaborators_url":"https://api.github.com/repos/tendermint/tendermint/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/tendermint/tendermint/teams","hooks_url":"https://api.github.com/repos/tendermint/tendermint/hooks","issue_events_url":"https://api.github.com/repos/tendermint/tendermint/issues/events{/number}","events_url":"https://api.github.com/repos/tendermint/tendermint/events","assignees_url":"https://api.github.com/repos/tendermint/tendermint/assignees{/user}","branches_url":"https://api.github.com/repos/tendermint/tendermint/branches{/branch}","tags_url":"https://api.github.com/repos/tendermint/tendermint/tags","blobs_url":"https://api.github.com/repos/tendermint/tendermint/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/tendermint/tendermint/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/tendermint/tendermint/git/refs{/sha}","trees_url":"https://api.github.com/repos/tendermint/tendermint/git/trees{/sha}","statuses_url":"https://api.github.com/repos/tendermint/tendermint/statuses/{sha}","languages_url":"https://api.github.com/repos/tendermint/tendermint/languages","stargazers_url":"https://api.github.com/repos/tendermint/tendermint/stargazers","contributors_url":"https://api.github.com/repos/tendermint/tendermint/contributors","subscribers_url":"https://api.github.com/repos/tendermint/tendermint/subscribers","subscription_url":"https://api.github.com/repos/tendermint/tendermint/subscription","commits_url":"https://api.github.com/repos/tendermint/tendermint/commits{/sha}","git_commits_url":"https://api.github.com/repos/tendermint/tendermint/git/commits{/sha}","comments_url":"https://api.github.com/repos/tendermint/tendermint/comments{/number}","issue_comment_url":"https://api.github.com/repos/tendermint/tendermint/issues/comments{/number}","contents_url":"https://api.github.com/repos/tendermint/tendermint/contents/{+path}","compare_url":"https://api.github.com/repos/tendermint/tendermint/compare/{base}...{head}","merges_url":"https://api.github.com/repos/tendermint/tendermint/merges","archive_url":"https://api.github.com/repos/tendermint/tendermint/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/tendermint/tendermint/downloads","issues_url":"https://api.github.com/repos/tendermint/tendermint/issues{/number}","pulls_url":"https://api.github.com/repos/tendermint/tendermint/pulls{/number}","milestones_url":"https://api.github.com/repos/tendermint/tendermint/milestones{/number}","notifications_url":"https://api.github.com/repos/tendermint/tendermint/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/tendermint/tendermint/labels{/name}","releases_url":"https://api.github.com/repos/tendermint/tendermint/releases{/id}","deployments_url":"https://api.github.com/repos/tendermint/tendermint/deployments","created_at":"2014-05-14T23:21:35Z","updated_at":"2016-09-06T04:01:48Z","pushed_at":"2016-09-06T01:48:49Z","git_url":"git://github.com/tendermint/tendermint.git","ssh_url":"git@github.com:tendermint/tendermint.git","clone_url":"https://github.com/tendermint/tendermint.git","svn_url":"https://github.com/tendermint/tendermint","homepage":"http://tendermint.com/","size":5503,"stargazers_count":286,"watchers_count":286,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":48,"mirror_url":null,"open_issues_count":50,"forks":48,"open_issues":50,"watchers":286,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/tendermint/tendermint/pulls/229"},"html":{"href":"https://github.com/tendermint/tendermint/pull/229"},"issue":{"href":"https://api.github.com/repos/tendermint/tendermint/issues/229"},"comments":{"href":"https://api.github.com/repos/tendermint/tendermint/issues/229/comments"},"review_comments":{"href":"https://api.github.com/repos/tendermint/tendermint/pulls/229/comments"},"review_comment":{"href":"https://api.github.com/repos/tendermint/tendermint/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/tendermint/tendermint/pulls/229/commits"},"statuses":{"href":"https://api.github.com/repos/tendermint/tendermint/statuses/fa676f7e6be7f8783e6e2831f4d65773b9e19092"}}}}
{"actor":{"display_login":"jaekwon"}}