{"action":"created","issue":{"url":"https://api.github.com/repos/jpmorganchase/quorum/issues/683","repository_url":"https://api.github.com/repos/jpmorganchase/quorum","labels_url":"https://api.github.com/repos/jpmorganchase/quorum/issues/683/labels{/name}","comments_url":"https://api.github.com/repos/jpmorganchase/quorum/issues/683/comments","events_url":"https://api.github.com/repos/jpmorganchase/quorum/issues/683/events","html_url":"https://github.com/jpmorganchase/quorum/issues/683","id":434157044,"node_id":"MDU6SXNzdWU0MzQxNTcwNDQ=","number":683,"title":"Data is not syncing (err=\"peer is unknown or unhealthy\")","user":{"login":"vaibhav-eleven","id":43638901,"node_id":"MDQ6VXNlcjQzNjM4OTAx","avatar_url":"https://avatars2.githubusercontent.com/u/43638901?v=4","gravatar_id":"","url":"https://api.github.com/users/vaibhav-eleven","html_url":"https://github.com/vaibhav-eleven","followers_url":"https://api.github.com/users/vaibhav-eleven/followers","following_url":"https://api.github.com/users/vaibhav-eleven/following{/other_user}","gists_url":"https://api.github.com/users/vaibhav-eleven/gists{/gist_id}","starred_url":"https://api.github.com/users/vaibhav-eleven/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vaibhav-eleven/subscriptions","organizations_url":"https://api.github.com/users/vaibhav-eleven/orgs","repos_url":"https://api.github.com/users/vaibhav-eleven/repos","events_url":"https://api.github.com/users/vaibhav-eleven/events{/privacy}","received_events_url":"https://api.github.com/users/vaibhav-eleven/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"prd-fox","id":31221824,"node_id":"MDQ6VXNlcjMxMjIxODI0","avatar_url":"https://avatars1.githubusercontent.com/u/31221824?v=4","gravatar_id":"","url":"https://api.github.com/users/prd-fox","html_url":"https://github.com/prd-fox","followers_url":"https://api.github.com/users/prd-fox/followers","following_url":"https://api.github.com/users/prd-fox/following{/other_user}","gists_url":"https://api.github.com/users/prd-fox/gists{/gist_id}","starred_url":"https://api.github.com/users/prd-fox/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/prd-fox/subscriptions","organizations_url":"https://api.github.com/users/prd-fox/orgs","repos_url":"https://api.github.com/users/prd-fox/repos","events_url":"https://api.github.com/users/prd-fox/events{/privacy}","received_events_url":"https://api.github.com/users/prd-fox/received_events","type":"User","site_admin":false},"assignees":[{"login":"prd-fox","id":31221824,"node_id":"MDQ6VXNlcjMxMjIxODI0","avatar_url":"https://avatars1.githubusercontent.com/u/31221824?v=4","gravatar_id":"","url":"https://api.github.com/users/prd-fox","html_url":"https://github.com/prd-fox","followers_url":"https://api.github.com/users/prd-fox/followers","following_url":"https://api.github.com/users/prd-fox/following{/other_user}","gists_url":"https://api.github.com/users/prd-fox/gists{/gist_id}","starred_url":"https://api.github.com/users/prd-fox/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/prd-fox/subscriptions","organizations_url":"https://api.github.com/users/prd-fox/orgs","repos_url":"https://api.github.com/users/prd-fox/repos","events_url":"https://api.github.com/users/prd-fox/events{/privacy}","received_events_url":"https://api.github.com/users/prd-fox/received_events","type":"User","site_admin":false}],"milestone":null,"comments":14,"created_at":"2019-04-17T08:31:14Z","updated_at":"2019-05-21T08:40:56Z","closed_at":null,"author_association":"NONE","body":"Geth\r\nVersion: 1.8.12-stable\r\nGit Commit: 4baffd64bda63208f9a9d153e975b52dabbfa32f\r\nQuorum Version: 2.2.0\r\nArchitecture: amd64\r\nProtocol Versions: [63 62]\r\nNetwork Id: 1337\r\nGo Version: go1.9.7\r\nOperating System: linux\r\nNumber of nodes - 5\r\nConsensus - RAFT\r\n\r\nAfter few transactions stopped a node did 13000 transaction on a network and then started a node. but its not syncing and when fetched block number its value is zero. \r\n\r\nLogs:-\r\nINFO [04-17|07:54:20.461] Loaded local transaction journal         transactions=0 dropped=0\r\nINFO [04-17|07:54:20.461] Regenerated local transaction journal    transactions=0 accounts=0\r\nDEBUG[04-17|07:54:20.462] Recalculated downloader QoS values       rtt=20s confidence=1.000 ttl=1m0s\r\nINFO [04-17|07:54:20.490] Starting P2P networking \r\nINFO [04-17|07:54:20.490] RLPx listener up                         self=\"enode://87767cbbe9840a2d46e22152414db8fbc93952f37ce04953dee691440ba928\r\na213512be7d55b36b9efd5fdf283bf26f645bddd9a3cc234415a652df72fa90c75@[::]:22001?discport=0\"\r\nINFO [04-17|07:54:20.490] starting raft protocol handler \r\nINFO [04-17|07:54:20.491] loaded the latest applied index          lastAppliedIndex=9717\r\nINFO [04-17|07:54:20.491] loading snapshot \r\nINFO [04-17|07:54:20.491] applying snapshot to raft storage \r\nINFO [04-17|07:54:20.491] updating cluster membership per raft snapshot \r\nINFO [04-17|07:54:20.491] adding new raft peer                     raft id=1\r\nTRACE[04-17|07:54:20.491] New dial task                            task=\"staticdial 87767cbbe9840a2d 52.175.216.141:22001\"\r\nTRACE[04-17|07:54:20.491] Adding static node                       node=\"enode://7020366ffb38ab6bd32648d0a3efadfb8529e5f5ee6780bd79ae3bfb85ecfa\r\nf7bed625d172fbfad283beb2ed79e79fb6b4f35b094e4419743cf3c991199267ed@52.175.245.187:22001?discport=0\"\r\nTRACE[04-17|07:54:20.491] New dial task                            task=\"staticdial 7020366ffb38ab6b 52.175.245.187:22001\"\r\nTRACE[04-17|07:54:20.492] Accepted connection                      addr=52.175.216.141:53772\r\nINFO [04-17|07:54:20.494] adding new raft peer                     raft id=3\r\nINFO [04-17|07:54:20.496] adding new raft peer                     raft id=4\r\nTRACE[04-17|07:54:20.497] Adding static node                       node=\"enode://dd995d46e592051e3485448c78a23542e8e0fe88063696cd2f3c5eed77fa42\r\nf9c039ce73715be8f63e559dd70f9b4d90c51976a2ed2afd4cd537caddca5b7ef9@51.143.1.36:22001?discport=0\"\r\nTRACE[04-17|07:54:20.497] New dial task                            task=\"staticdial dd995d46e592051e 51.143.1.36:22001\"\r\nTRACE[04-17|07:54:20.497] Adding static node                       node=\"enode://6f0093de30ab22e9d1da94693f3166baf966c54c2cbfe9129564ccb965324f44b1f7acf44b821dda673c4333217ccb6817feb8a5d8dc14f3524463890ffd01c7@52.183.74.248:22001?discport=0\"\r\nTRACE[04-17|07:54:20.497] New dial task                            task=\"staticdial 6f0093de30ab22e9 52.183.74.248:22001\"\r\nINFO [04-17|07:54:20.500] adding new raft peer                     raft id=5\r\nTRACE[04-17|07:54:20.502] Quorum permissioning                     id=7020366ffb38ab6b addr=52.175.245.187:22001 conn=staticdial EnableNodePermission=false DataDir=/home/eleven01/deployment-scripts/TestNet/qdata/node1/node Current Node ID=87767cbbe9840a2d46e22152414db8fbc93952f37ce04953dee691440ba928a213512be7d55b36b9efd5fdf283bf26f645bddd9a3cc234415a652df72fa90c75 Node Name=Geth/v1.8.12-stable-4baffd64/linux-amd64/go1.9.7 Dialed Dest=\"enode://7020366ffb38ab6bd32648d0a3efadfb8529e5f5ee6780bd79ae3bfb85ecfaf7bed625d172fbfad283beb2ed79e79fb6b4f35b094e4419743cf3c991199267ed@52.175.245.187:22001?discport=0\" Connection ID=7020366ffb38ab6b Connection String=7020366ffb38ab6bd32648d0a3efadfb8529e5f5ee6780bd79ae3bfb85ecfaf7bed625d172fbfad283beb2ed79e79fb6b4f35b094e4419743cf3c991199267ed\r\nTRACE[04-17|07:54:20.502] Node Permissioning is Disabled.          id=7020366ffb38ab6b addr=52.175.245.187:22001 conn=staticdial\r\nINFO [04-17|07:54:20.502] updated cluster membership \r\nINFO [04-17|07:54:20.502] synchronizing with peer                  peer id=1 hash=3f5694…7b6c5d\r\nINFO [04-17|07:54:20.503] Block synchronisation started \r\nWARN [04-17|07:54:20.503] Synchronisation failed, retrying         err=\"peer is unknown or unhealthy\"\r\nINFO [04-17|07:54:20.503] failed to synchronize with peer          peer id=7020366ffb38ab6bd32648d0a3efadfb8529e5f5ee6780bd79ae3bfb85ecfaf7bed625d172fbfad283beb2ed79e79fb6b4f35b094e4419743cf3c991199267ed\r\nTRACE[04-17|07:54:20.503] Adding static node                       node=\"enode://5a9870aca5d5014ef251d0a0e755cd433487d83ec0ce2a94c80661e12e0790267207dee4ba61d53d7dbc9f83535019856a091407c8cf14dfb5f2f115f219131e@52.191.137.15:22001?discport=0\"\r\nTRACE[04-17|07:54:20.503] New dial task                            task=\"staticdial 5a9870aca5d5014e 52.191.137.15:22001\"\r\nDEBUG[04-17|07:54:20.504] Adding p2p peer                          name=Geth/v1.8.12-stable-... addr=52.175.245.187:22001 peers=1\r\nTRACE[04-17|07:54:20.504] Starting protocol eth/63                 id=7020366ffb38ab6b conn=staticdial\r\nDEBUG[04-17|07:54:20.504] Ethereum peer connected                  id=7020366ffb38ab6b conn=staticdial name=Geth/v1.8.12-stable-4baffd64/linux-amd64/go1.9.7\r\nTRACE[04-17|07:54:20.504] connection set up                        id=7020366ffb38ab6b addr=52.175.245.187:22001 conn=staticdial inbound=false\r\nTRACE[04-17|07:54:20.504] Dial task done                           task=\"staticdial 7020366ffb38ab6b 52.175.245.187:22001\"\r\nTRACE[04-17|07:54:20.505] Registering sync peer                    peer=7020366ffb38ab6b\r\nTRACE[04-17|07:54:20.508] Quorum permissioning                     id=87767cbbe9840a2d addr=52.175.216.141:53772 conn=inbound    EnableNodePermission=false DataDir=/home/eleven01/deployment-scripts/TestNet/qdata/node1/node Current Node ID=87767cbbe9840a2d46e22152414db8fbc93952f37ce04953dee691440ba928a213512be7d55b36b9efd5fdf283bf26f645bddd9a3cc234415a652df72fa90c75 Node Name=Geth/v1.8.12-stable-4baffd64/linux-amd64/go1.9.7 Dialed Dest=nil                                                                                                                                                                        Connection ID=87767cbbe9840a2d Connection String=87767cbbe9840a2d46e22152414db8fbc93952f37ce04953dee691440ba928a213512be7d55b36b9efd5fdf283bf26f645bddd9a3cc234415a652df72fa90c75\r\nTRACE[04-17|07:54:20.508] Node Permissioning is Disabled.          id=87767cbbe9840a2d addr=52.175.216.141:53772 conn=inbound\r\nTRACE[04-17|07:54:20.508] Rejected peer before protocol handshake  id=87767cbbe9840a2d addr=52.175.216.141:53772 conn=inbound    err=\"connected to self\"\r\nTRACE[04-17|07:54:20.508] Setting up connection failed             id=87767cbbe9840a2d err=\"connected to self\"\r\nTRACE[04-17|07:54:20.513] Quorum permissioning                     id=6f0093de30ab22e9 addr=52.183.74.248:22001  conn=staticdial EnableNodePermission=false DataDir=/home/eleven01/deployment-scripts/TestNet/qdata/node1/node Current Node ID=87767cbbe9840a2d46e22152414db8fbc93952f37ce04953dee691440ba928a213512be7d55b36b9efd5fdf283bf26f645bddd9a3cc234415a652df72fa90c75 Node Name=Geth/v1.8.12-stable-4baffd64/linux-amd64/go1.9.7 Dialed Dest=\"enode://6f0093de30ab22e9d1da94693f3166baf966c54c2cbfe9129564ccb965324f44b1f7acf44b821dda673c4333217ccb6817feb8a5d8dc14f3524463890ffd01c7@52.183.74.248:22001?discport=0\"  Connection ID=6f0093de30ab22e9 Connection String=6f0093de30ab22e9d1da94693f3166baf966c54c2cbfe9129564ccb965324f44b1f7acf44b821dda673c4333217ccb6817feb8a5d8dc14f3524463890ffd01c7\r\nTRACE[04-17|07:54:20.513] Node Permissioning is Disabled.          id=6f0093de30ab22e9 addr=52.183.74.248:22001  conn=staticdial\r\nDEBUG[04-17|07:54:20.513] Adding p2p peer                          name=Geth/v1.8.12-stable-...                          addr=52.183.74.248:22001  peers=2\r\nTRACE[04-17|07:54:20.513] connection set up                        id=6f0093de30ab22e9 addr=52.183.74.248:22001  conn=staticdial inbound=false\r\nTRACE[04-17|07:54:20.513] Dial task done                           task=\"staticdial 6f0093de30ab22e9 52.183.74.248:22001\"\r\nTRACE[04-17|07:54:20.513] Starting protocol eth/63                 id=6f0093de30ab22e9 conn=staticdial\r\nDEBUG[04-17|07:54:20.513] Ethereum peer connected                  id=6f0093de30ab22e9 conn=staticdial name=Geth/v1.8.12-stable-4baffd64/linux-amd64/go1.9.7"},"comment":{"url":"https://api.github.com/repos/jpmorganchase/quorum/issues/comments/494297644","html_url":"https://github.com/jpmorganchase/quorum/issues/683#issuecomment-494297644","issue_url":"https://api.github.com/repos/jpmorganchase/quorum/issues/683","id":494297644,"node_id":"MDEyOklzc3VlQ29tbWVudDQ5NDI5NzY0NA==","user":{"login":"prd-fox","id":31221824,"node_id":"MDQ6VXNlcjMxMjIxODI0","avatar_url":"https://avatars1.githubusercontent.com/u/31221824?v=4","gravatar_id":"","url":"https://api.github.com/users/prd-fox","html_url":"https://github.com/prd-fox","followers_url":"https://api.github.com/users/prd-fox/followers","following_url":"https://api.github.com/users/prd-fox/following{/other_user}","gists_url":"https://api.github.com/users/prd-fox/gists{/gist_id}","starred_url":"https://api.github.com/users/prd-fox/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/prd-fox/subscriptions","organizations_url":"https://api.github.com/users/prd-fox/orgs","repos_url":"https://api.github.com/users/prd-fox/repos","events_url":"https://api.github.com/users/prd-fox/events{/privacy}","received_events_url":"https://api.github.com/users/prd-fox/received_events","type":"User","site_admin":false},"created_at":"2019-05-21T08:40:56Z","updated_at":"2019-05-21T08:40:56Z","author_association":"CONTRIBUTOR","body":"Hey.\r\n\r\nIt seems the root cause is that the upgrade to 1.8.12 introduced state caching, which means that state isn't written to disk immediately.\r\n\r\nThe way raft works means it requires the state to be written before committing the raft log, meaning the data was out of sync and couldn't recover.\r\n\r\nSetting the `--gcmode=archive` option should fix the issue, and we will make sure that a raft node is always started with this flag in the future.\r\n\r\nIf you could try and let me know the results, that would be great."}}
{"actor":{"display_login":"prd-fox"}}