{"action":"created","issue":{"url":"https://api.github.com/repos/coreos/etcd/issues/5004","repository_url":"https://api.github.com/repos/coreos/etcd","labels_url":"https://api.github.com/repos/coreos/etcd/issues/5004/labels{/name}","comments_url":"https://api.github.com/repos/coreos/etcd/issues/5004/comments","events_url":"https://api.github.com/repos/coreos/etcd/issues/5004/events","html_url":"https://github.com/coreos/etcd/issues/5004","id":146778559,"number":5004,"title":"raft - questions regard to membership change","user":{"login":"EasonLiao","id":4032971,"avatar_url":"https://avatars.githubusercontent.com/u/4032971?v=3","gravatar_id":"","url":"https://api.github.com/users/EasonLiao","html_url":"https://github.com/EasonLiao","followers_url":"https://api.github.com/users/EasonLiao/followers","following_url":"https://api.github.com/users/EasonLiao/following{/other_user}","gists_url":"https://api.github.com/users/EasonLiao/gists{/gist_id}","starred_url":"https://api.github.com/users/EasonLiao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/EasonLiao/subscriptions","organizations_url":"https://api.github.com/users/EasonLiao/orgs","repos_url":"https://api.github.com/users/EasonLiao/repos","events_url":"https://api.github.com/users/EasonLiao/events{/privacy}","received_events_url":"https://api.github.com/users/EasonLiao/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-04-08T00:24:46Z","updated_at":"2016-04-08T00:36:42Z","closed_at":null,"body":"Hi etcd team, \r\n\r\nI was reading the raft code regards to dynamic membership change and I have a question about it. I have limited familiarity with the raft code so first I want to verify if my understanding about current membership implementation is correct:\r\n\r\n- Only one node reconfiguration at a time and one node change per reconfiguration. This guarantees two consecutive configurations has at least one node overlap in quorums.\r\n- Raft proposes the reconfiguration as \"normal\" command and adopts the new configuration only after it's committed in old quorum.\r\n\r\nThis implies that the membership is something in \"state\" because raft only uses it after it's committed, however, raft uses log history for leader election and I wonder if it's possible that current \"applied membership\" and the latest membership in log differs more than one node? If so, is this safe?\r\n\r\nOne example:\r\n\r\nLet's say there're 3 consecutive reconfigurations:\r\n```\r\nS1: [A, B, C]\r\nS2: [A, B, C, D]\r\nS3: [A, B, C, D, E]\r\n```\r\n\r\nAnd this is current log state:\r\n```\r\nA(Term: 1): S1, S2, S3 [S1]\r\nB(Term: 1): S1, S2, S3 [S3]\r\nC(Term: 1): S1, S2, S3 [S3]\r\nD(Term: 1): S1, S2, S3 [S3]\r\nE(Term: 1): S1, S2, S3 [S3]\r\n```\r\nConfiguration S3 is committed because it has been persisted on the quorum of S2, besides that, it has been persisted by all nodes in S3 as well. However, S3 is not applied to A yet, S2 is not applied to A as well(this is possible that A received entry S2 and S3 but has not received the commit message yet)so A thinks it's still belong to S1. \r\n\r\nNow let's say B is partitioned from rest of the cluster and C, D, E form a new quorum in next term and C is the leader: \r\n```\r\nA(Term: 1): S1, S2, S3 [S1]\r\nB(Term: 1): S1, S2, S3 [S3]\r\nC(Term: 2): S1, S2, S3 [S3] L\r\nD(Term: 2): S1, S2, S3 [S3]\r\nE(Term: 2): S1, S2, S3 [S3]\r\n```\r\nThen A starts a leader election and it got the vote from B because A has a sufficient log history(it has S3 in log). So A thinks it's the leader of term2.\r\n```\r\nA(Term: 2): S1, S2, S3 [S1] L\r\nB(Term: 2): S1, S2, S3 [S3]\r\nC(Term: 2): S1, S2, S3 [S3] L\r\nD(Term: 2): S1, S2, S3 [S3]\r\nE(Term: 2): S1, S2, S3 [S3]\r\n```\r\n\r\nI was wondering can this happen? Thanks in advance!"},"comment":{"url":"https://api.github.com/repos/coreos/etcd/issues/comments/207149267","html_url":"https://github.com/coreos/etcd/issues/5004#issuecomment-207149267","issue_url":"https://api.github.com/repos/coreos/etcd/issues/5004","id":207149267,"user":{"login":"xiang90","id":4479947,"avatar_url":"https://avatars.githubusercontent.com/u/4479947?v=3","gravatar_id":"","url":"https://api.github.com/users/xiang90","html_url":"https://github.com/xiang90","followers_url":"https://api.github.com/users/xiang90/followers","following_url":"https://api.github.com/users/xiang90/following{/other_user}","gists_url":"https://api.github.com/users/xiang90/gists{/gist_id}","starred_url":"https://api.github.com/users/xiang90/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xiang90/subscriptions","organizations_url":"https://api.github.com/users/xiang90/orgs","repos_url":"https://api.github.com/users/xiang90/repos","events_url":"https://api.github.com/users/xiang90/events{/privacy}","received_events_url":"https://api.github.com/users/xiang90/received_events","type":"User","site_admin":false},"created_at":"2016-04-08T00:36:42Z","updated_at":"2016-04-08T00:36:42Z","body":"> however, raft uses log history for leader election and I wonder if it's possible that current \"applied membership\" and the latest membership in log differs more than one node? If so, is this safe?\r\n\r\nThis should not be possible.\r\n\r\n> Then A starts a leader election and it got the vote from B because A has a sufficient log history(it has S3 in log). So A thinks it's the leader of term2.\r\n\r\nBefore A starts the election, it MUST apply all pending configuration entries."}}