{"action":"opened","issue":{"url":"https://api.github.com/repos/moby/moby/issues/36382","repository_url":"https://api.github.com/repos/moby/moby","labels_url":"https://api.github.com/repos/moby/moby/issues/36382/labels{/name}","comments_url":"https://api.github.com/repos/moby/moby/issues/36382/comments","events_url":"https://api.github.com/repos/moby/moby/issues/36382/events","html_url":"https://github.com/moby/moby/issues/36382","id":299423104,"number":36382,"title":"Docker swarm join fails - 17.12","user":{"login":"instantlinux","id":3667505,"avatar_url":"https://avatars1.githubusercontent.com/u/3667505?v=4","gravatar_id":"","url":"https://api.github.com/users/instantlinux","html_url":"https://github.com/instantlinux","followers_url":"https://api.github.com/users/instantlinux/followers","following_url":"https://api.github.com/users/instantlinux/following{/other_user}","gists_url":"https://api.github.com/users/instantlinux/gists{/gist_id}","starred_url":"https://api.github.com/users/instantlinux/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/instantlinux/subscriptions","organizations_url":"https://api.github.com/users/instantlinux/orgs","repos_url":"https://api.github.com/users/instantlinux/repos","events_url":"https://api.github.com/users/instantlinux/events{/privacy}","received_events_url":"https://api.github.com/users/instantlinux/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-02-22T16:37:59Z","updated_at":"2018-02-22T16:37:59Z","closed_at":null,"author_association":"NONE","body":"**Description**\r\n\r\nI have a 3-node cluster upgraded twice since 17.06. Not long after the 17.12 update, after a couple of server crashes from which recovery happened OK, I had one go down and I cannot join a new node to the swarm which continues to run on two nodes. Even a freshly-installed Linux cannot join.\r\n\r\n**Steps to reproduce the issue:**\r\n\r\nI lack the hardware to attempt to reproduce this. \r\n\r\n**Describe the results you received:**\r\n```\r\n# docker swarm join --token SWMTKN-1-37xxxxx 192.168.2.79:2377\r\nError response from daemon: Timeout was reached before node joined. The attempt to join the swarm will continue in the background. Use the \"docker info\" command to see the current swarm status of your node.\r\n# tail /var/log/syslog\r\nFeb 22 08:23:30 sys144b dockerd[17000]: time=\"2018-02-22T08:23:30.547171772-08:00\" level=error msg=\"Handler for POST /v1.35/swarm/join returned error: Timeout was reached before node joined. The attempt to join the swarm will continue in the background. Use the \\\"docker info\\\" command to see the current swarm status of your node.\"\r\nFeb 22 08:23:45 sys144b dockerd[17000]: time=\"2018-02-22T08:23:45.634304108-08:00\" level=error msg=\"failed to request and save new certificate\" error=\"rpc error: code = DeadlineExceeded desc = context deadline exceeded\" module=node/tls\r\n```\r\n\r\n**Describe the results you expected:**\r\n\r\nJoin OK. \r\n\r\n** Additional Info **\r\n\r\nIn my attempts to deal with this, I think I have lost manager quorum. This is a very nasty problem if it happens to anyone else; unless I learn of a workaround, my only recourse is to discard my current swarm with all its data and rebuild under an older more-trusted version (17.09).\r\n\r\n**Output of `docker version`:**\r\n\r\n```\r\nClient:\r\n Version:\t17.12.0-ce\r\n API version:\t1.35\r\n Go version:\tgo1.9.2\r\n Git commit:\tc97c6d6\r\n Built:\tWed Dec 27 20:11:19 2017\r\n OS/Arch:\tlinux/amd64\r\n\r\nServer:\r\n Engine:\r\n  Version:\t17.12.0-ce\r\n  API version:\t1.35 (minimum version 1.12)\r\n  Go version:\tgo1.9.2\r\n  Git commit:\tc97c6d6\r\n  Built:\tWed Dec 27 20:09:53 2017\r\n  OS/Arch:\tlinux/amd64\r\n  Experimental:\tfalse\r\n```\r\n\r\n**Output of `docker info`:**\r\n\r\n(Fresh node trying to join)\r\n```\r\nContainers: 0\r\n Running: 0\r\n Paused: 0\r\n Stopped: 0\r\nImages: 1\r\nServer Version: 17.12.0-ce\r\nStorage Driver: devicemapper\r\n Pool Name: data01-thinpool\r\n Pool Blocksize: 524.3kB\r\n Base Device Size: 10.74GB\r\n Backing Filesystem: ext4\r\n Udev Sync Supported: true\r\n Data Space Used: 334MB\r\n Data Space Total: 2.097GB\r\n Data Space Available: 1.763GB\r\n Metadata Space Used: 135.2kB\r\n Metadata Space Total: 104.9MB\r\n Metadata Space Available: 104.7MB\r\n Thin Pool Minimum Free Space: 209.7MB\r\n Deferred Removal Enabled: false\r\n Deferred Deletion Enabled: false\r\n Deferred Deleted Device Count: 0\r\n Library Version: 1.02.110 (2015-10-30)\r\nLogging Driver: json-file\r\nCgroup Driver: cgroupfs\r\nPlugins:\r\n Volume: local\r\n Network: bridge host macvlan null overlay\r\n Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog\r\nSwarm: error\r\n NodeID: \r\n Error: rpc error: code = DeadlineExceeded desc = context deadline exceeded\r\n Is Manager: false\r\n Node Address: 192.168.2.125\r\nRuntimes: runc\r\nDefault Runtime: runc\r\nInit Binary: docker-init\r\ncontainerd version: 89623f28b87a6004d4b785663257362d1658a729\r\nrunc version: b2567b37d7b75eb4cf325b77297b140ea686ce8f\r\ninit version: 949e6fa\r\nSecurity Options:\r\n apparmor\r\n seccomp\r\n  Profile: default\r\nKernel Version: 4.11.10-041110-generic\r\nOperating System: Ubuntu 16.04.3 LTS\r\nOSType: linux\r\nArchitecture: x86_64\r\nCPUs: 1\r\nTotal Memory: 2.936GiB\r\nName: sys144b\r\nID: TTPJ:JLWV:IZOC:OLAH:O45U:PWX6:XGIM:6MRI:VAKQ:QJLA:AXK5:CZVL\r\nDocker Root Dir: /var/lib/docker\r\nDebug Mode (client): false\r\nDebug Mode (server): false\r\nRegistry: https://index.docker.io/v1/\r\nLabels:\r\nExperimental: false\r\nInsecure Registries:\r\n 127.0.0.0/8\r\nLive Restore Enabled: false\r\n\r\nWARNING: No swap limit support\r\n```\r\n\r\n(One of existing nodes)\r\n```\r\nContainers: 85\r\n Running: 33\r\n Paused: 0\r\n Stopped: 52\r\nImages: 1039\r\nServer Version: 17.12.0-ce\r\nStorage Driver: devicemapper\r\n Pool Name: data01-thinpool\r\n Pool Blocksize: 524.3kB\r\n Base Device Size: 10.74GB\r\n Backing Filesystem: ext4\r\n Udev Sync Supported: true\r\n Data Space Used: 49.42GB\r\n Data Space Total: 145GB\r\n Data Space Available: 95.6GB\r\n Metadata Space Used: 18.38MB\r\n Metadata Space Total: 104.9MB\r\n Metadata Space Available: 86.48MB\r\n Thin Pool Minimum Free Space: 14.5GB\r\n Deferred Removal Enabled: false\r\n Deferred Deletion Enabled: false\r\n Deferred Deleted Device Count: 0\r\n Library Version: 1.02.110 (2015-10-30)\r\nLogging Driver: json-file\r\nCgroup Driver: cgroupfs\r\nPlugins:\r\n Volume: local\r\n Network: bridge host macvlan null overlay\r\n Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog\r\nSwarm: active\r\n NodeID: c8kaeqrq2ogb2g34zfuh73hhe\r\n Is Manager: true\r\n ClusterID: 48s4nkfptgk54do81dxqz58ny\r\n Managers: 2\r\n Nodes: 2\r\n Orchestration:\r\n  Task History Retention Limit: 5\r\n Raft:\r\n  Snapshot Interval: 10000\r\n  Number of Old Snapshots to Retain: 0\r\n  Heartbeat Tick: 1\r\n  Election Tick: 3\r\n Dispatcher:\r\n  Heartbeat Period: 5 seconds\r\n CA Configuration:\r\n  Expiry Duration: 3 months\r\n  Force Rotate: 0\r\n Autolock Managers: false\r\n Root Rotation In Progress: false\r\n Node Address: 192.168.2.79\r\n Manager Addresses:\r\n  192.168.2.64:2377\r\n  192.168.2.69:2377\r\n  192.168.2.79:2377\r\nRuntimes: runc\r\nDefault Runtime: runc\r\nInit Binary: docker-init\r\ncontainerd version: 89623f28b87a6004d4b785663257362d1658a729\r\nrunc version: b2567b37d7b75eb4cf325b77297b140ea686ce8f\r\ninit version: 949e6fa\r\nSecurity Options:\r\n apparmor\r\n seccomp\r\n  Profile: default\r\nKernel Version: 4.11.12-041112-generic\r\nOperating System: Ubuntu 16.04.3 LTS\r\nOSType: linux\r\nArchitecture: x86_64\r\nCPUs: 4\r\nTotal Memory: 15.17GiB\r\nName: vinson\r\nID: G7HB:LHJD:ZJKL:G34H:TQSE:WM55:FBGT:U3GR:6SPM:CR25:KHA3:3ACD\r\nDocker Root Dir: /var/lib/docker\r\nDebug Mode (client): false\r\nDebug Mode (server): false\r\nRegistry: https://index.docker.io/v1/\r\nLabels:\r\nExperimental: false\r\nInsecure Registries:\r\n 127.0.0.0/8\r\nLive Restore Enabled: false\r\n\r\nWARNING: No swap limit support\r\n```\r\n**Additional environment details (AWS, VirtualBox, physical, etc.):**\r\nBare-metal three-node cluster."}}
{"actor":{"display_login":"instantlinux"}}