{"action":"created","issue":{"url":"https://api.github.com/repos/docker/docker/issues/25321","repository_url":"https://api.github.com/repos/docker/docker","labels_url":"https://api.github.com/repos/docker/docker/issues/25321/labels{/name}","comments_url":"https://api.github.com/repos/docker/docker/issues/25321/comments","events_url":"https://api.github.com/repos/docker/docker/issues/25321/events","html_url":"https://github.com/docker/docker/issues/25321","id":168820726,"number":25321,"title":"Docker daemon hanging","user":{"login":"chbatey","id":1866779,"avatar_url":"https://avatars.githubusercontent.com/u/1866779?v=3","gravatar_id":"","url":"https://api.github.com/users/chbatey","html_url":"https://github.com/chbatey","followers_url":"https://api.github.com/users/chbatey/followers","following_url":"https://api.github.com/users/chbatey/following{/other_user}","gists_url":"https://api.github.com/users/chbatey/gists{/gist_id}","starred_url":"https://api.github.com/users/chbatey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chbatey/subscriptions","organizations_url":"https://api.github.com/users/chbatey/orgs","repos_url":"https://api.github.com/users/chbatey/repos","events_url":"https://api.github.com/users/chbatey/events{/privacy}","received_events_url":"https://api.github.com/users/chbatey/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/docker/docker/labels/area/runtime","name":"area/runtime","color":"fbca04"},{"url":"https://api.github.com/repos/docker/docker/labels/version/1.11","name":"version/1.11","color":"ededed"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":15,"created_at":"2016-08-02T07:48:26Z","updated_at":"2016-08-03T08:39:12Z","closed_at":null,"body":"I have seen the daemon hang (1.8.3) under high load: https://github.com/docker/docker/issues/13885. However this appears different. I am now running 1.11.2 and had a hang under little to no load.\r\n---------------------------------------------------\r\nBUG REPORT INFORMATION\r\n---------------------------------------------------\r\n\r\n\r\n**Output of `docker version`:**\r\n\r\n```\r\ncentos@ip-10-50-185-106 ~]$ docker version\r\nClient:\r\n Version:      1.11.2\r\n API version:  1.23\r\n Go version:   go1.5.4\r\n Git commit:   b9f10c9\r\n Built:        Wed Jun  1 21:23:11 2016\r\n OS/Arch:      linux/amd64\r\nCannot connect to the Docker daemon. Is the docker daemon running on this host?\r\n[centos@ip-10-50-185-106 ~]$ sudo docker version\r\nClient:\r\n Version:      1.11.2\r\n API version:  1.23\r\n Go version:   go1.5.4\r\n Git commit:   b9f10c9\r\n Built:        Wed Jun  1 21:23:11 2016\r\n OS/Arch:      linux/amd64\r\n\r\nServer:\r\n Version:      1.11.2\r\n API version:  1.23\r\n Go version:   go1.5.4\r\n Git commit:   b9f10c9\r\n Built:        Wed Jun  1 21:23:11 2016\r\n OS/Arch:      linux/amd64\r\n\r\n```\r\n\r\n\r\n**Output of `docker info`:**\r\n\r\n```\r\nContainers: 13\r\n Running: 12\r\n Paused: 0\r\n Stopped: 1\r\nImages: 11\r\nServer Version: 1.11.2\r\nStorage Driver: devicemapper\r\n Pool Name: direct_lvm-thin_pool\r\n Pool Blocksize: 65.54 kB\r\n Base Device Size: 107.4 GB\r\n Backing Filesystem: xfs\r\n Data file: \r\n Metadata file: \r\n Data Space Used: 2.39 GB\r\n Data Space Total: 66.57 GB\r\n Data Space Available: 64.18 GB\r\n Metadata Space Used: 4.375 MB\r\n Metadata Space Total: 1.074 GB\r\n Metadata Space Available: 1.069 GB\r\n Udev Sync Supported: true\r\n Deferred Removal Enabled: false\r\n Deferred Deletion Enabled: false\r\n Deferred Deleted Device Count: 0\r\n Library Version: 1.02.107-RHEL7 (2016-06-09)\r\nLogging Driver: json-file\r\nCgroup Driver: cgroupfs\r\nPlugins: \r\n Volume: local\r\n Network: null host bridge\r\nKernel Version: 3.10.0-327.22.2.el7.x86_64\r\nOperating System: CentOS Linux 7 (Core)\r\nOSType: linux\r\nArchitecture: x86_64\r\nCPUs: 4\r\nTotal Memory: 6.897 GiB\r\nName: ip-10-50-185-106.internal\r\nID: DDCD:TC7W:6V5N:QDUA:YQF6:24EU:5SVR:WY3L:VZ7X:4BRW:NKM4:INSA\r\nDocker Root Dir: /var/lib/docker\r\nDebug mode (client): false\r\nDebug mode (server): false\r\nHttp Proxy: http://10.50.185.193:80\r\nHttps Proxy: http://10.50.185.193:80\r\nNo Proxy: 10.50.185.0/24,.internal,registry-k8.api.bskyb.com,master-test-k8.api.bskyb.com,localhost,127.0.0.0/8,::1,/var/run/docker.sock,169.254.169.254\r\nRegistry: https://index.docker.io/v1/\r\n\r\n```\r\n\r\n**Additional environment details (AWS, VirtualBox, physical, etc.):**\r\nAWS, Centos 7\r\n\r\n\r\n**Steps to reproduce the issue:**\r\nUnknown\r\n\r\n\r\n**Describe the results you received:**\r\nAll docker client commands hang\r\n\r\nStracing docker client:\r\n\r\n\r\n```\r\nread(6, 0xc8203ea000, 4096)             = -1 EAGAIN (Resource temporarily unavailable)\r\nwrite(6, \"GET /v1.23/containers/json HTTP/\"..., 89) = 89\r\nfutex(0x21ef2b0, FUTEX_WAIT, 0, NULL\r\n```\r\n\r\nSo basically no response back from the daemon.\r\n\r\n\r\nStracing docker daemon\r\n```\r\n[root@ip-10-50-185-112 ~]# strace -p 1095\r\nProcess 1095 attached\r\nread(46, \r\n\r\n```\r\n\r\nSo the daemon look to be reading on FD 46\r\n\r\n```\r\n[centos@ip-10-50-185-112 ~]$ sudo lsof -d 46                                                                                                                                                                                                                                                                                                                               \r\nCOMMAND     PID USER   FD   TYPE             DEVICE SIZE/OFF      NODE NAME\r\nkube-prox   991 root   46u  IPv6              24429      0t0       TCP *:31624 (LISTEN)\r\ndocker     1095 root   46r  FIFO               0,18      0t0   3836156 /run/docker/libcontainerd/110d033df2a6bf66ddffce6aeef574148f04b25b61a4f931978933cec4f51116/init-stderr\r\nmaster     1540 root   46u  unix 0xffff8800e787c740      0t0     24830 public/flush\r\ndocker-co  1583 root   46r  FIFO               0,18      0t0     36330 /run/containerd/d6bd98f8998e1e5638a8d0122e44ae452a30ff89a7e6733ab962188762b3785e/init/exit\r\nkubelet    2244 root   46u  sock                0,6      0t0   6102239 protocol: TCPv6\r\n\r\n```\r\n\r\n\r\n**Describe the results you expected:**\r\nDocker to work\r\n\r\n\r\n**Additional information you deem important (e.g. issue happens only occasionally):**\r\nHappens occasionally\r\n\r\nAny additional information I can add today. I have taken this node out of use and left it hung. The only fix is to restart docker which I can hold off for 24 hours."},"comment":{"url":"https://api.github.com/repos/docker/docker/issues/comments/237177981","html_url":"https://github.com/docker/docker/issues/25321#issuecomment-237177981","issue_url":"https://api.github.com/repos/docker/docker/issues/25321","id":237177981,"user":{"login":"thaJeztah","id":1804568,"avatar_url":"https://avatars.githubusercontent.com/u/1804568?v=3","gravatar_id":"","url":"https://api.github.com/users/thaJeztah","html_url":"https://github.com/thaJeztah","followers_url":"https://api.github.com/users/thaJeztah/followers","following_url":"https://api.github.com/users/thaJeztah/following{/other_user}","gists_url":"https://api.github.com/users/thaJeztah/gists{/gist_id}","starred_url":"https://api.github.com/users/thaJeztah/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thaJeztah/subscriptions","organizations_url":"https://api.github.com/users/thaJeztah/orgs","repos_url":"https://api.github.com/users/thaJeztah/repos","events_url":"https://api.github.com/users/thaJeztah/events{/privacy}","received_events_url":"https://api.github.com/users/thaJeztah/received_events","type":"User","site_admin":false},"created_at":"2016-08-03T08:39:12Z","updated_at":"2016-08-03T08:39:12Z","body":"@krisives could you share what commands you put the sleep between?"}}
{"actor":{"display_login":"thaJeztah"}}