{"action":"created","issue":{"url":"https://api.github.com/repos/docker/docker/issues/27381","repository_url":"https://api.github.com/repos/docker/docker","labels_url":"https://api.github.com/repos/docker/docker/issues/27381/labels{/name}","comments_url":"https://api.github.com/repos/docker/docker/issues/27381/comments","events_url":"https://api.github.com/repos/docker/docker/issues/27381/events","html_url":"https://github.com/docker/docker/issues/27381","id":183022791,"number":27381,"title":"Driver devicemapper failed to remove root filesystem. Device is busy","user":{"login":"ceecko","id":8225242,"avatar_url":"https://avatars.githubusercontent.com/u/8225242?v=3","gravatar_id":"","url":"https://api.github.com/users/ceecko","html_url":"https://github.com/ceecko","followers_url":"https://api.github.com/users/ceecko/followers","following_url":"https://api.github.com/users/ceecko/following{/other_user}","gists_url":"https://api.github.com/users/ceecko/gists{/gist_id}","starred_url":"https://api.github.com/users/ceecko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceecko/subscriptions","organizations_url":"https://api.github.com/users/ceecko/orgs","repos_url":"https://api.github.com/users/ceecko/repos","events_url":"https://api.github.com/users/ceecko/events{/privacy}","received_events_url":"https://api.github.com/users/ceecko/received_events","type":"User","site_admin":false},"labels":[{"id":77260064,"url":"https://api.github.com/repos/docker/docker/labels/area/storage/devicemapper","name":"area/storage/devicemapper","color":"FBCA04","default":false},{"id":194268680,"url":"https://api.github.com/repos/docker/docker/labels/status/needs-attention","name":"status/needs-attention","color":"e11d21","default":false},{"id":375714123,"url":"https://api.github.com/repos/docker/docker/labels/version/1.12","name":"version/1.12","color":"ededed","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":23,"created_at":"2016-10-14T11:02:24Z","updated_at":"2016-10-31T15:10:32Z","closed_at":null,"body":"<!--\nIf you are reporting a new issue, make sure that we do not have any duplicates\nalready open. You can ensure this by searching the issue list for this\nrepository. If there is a duplicate, please close your issue and add a comment\nto the existing issue instead.\n\nIf you suspect your issue is a bug, please edit your issue description to\ninclude the BUG REPORT INFORMATION shown below. If you fail to provide this\ninformation within 7 days, we cannot debug your issue and will close it. We\nwill, however, reopen it if you later provide the information.\n\nFor more information about reporting issues, see\nhttps://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues\n\n---------------------------------------------------\nBUG REPORT INFORMATION\n---------------------------------------------------\nUse the commands below to provide key information from your environment:\nYou do NOT have to include this information if this is a FEATURE REQUEST\n-->\n\n**Description**\nCannot remove containers, docker reports `Driver devicemapper failed to remove root filesystem. Device is busy`. This leaves containers in `Dead` state.\n\n<!--\nBriefly describe the problem you are having in a few paragraphs.\n-->\n\n**Steps to reproduce the issue:**\n1. `docker rm container_id`\n\n**Describe the results you received:**\nError message is displayed: `Error response from daemon: Driver devicemapper failed to remove root filesystem ce2ea989895b7e073b9c3103a7312f32e70b5ad01d808b42f16655ffcb06c535: Device is Busy`\n\n**Describe the results you expected:**\nContainer should be removed.\n\n**Additional information you deem important (e.g. issue happens only occasionally):**\nThis started to occur after upgrade from 1.11.2 to 1.12.2 and happens occasionally (10% of removals)\n\n**Output of `docker version`:**\n\n```\nClient:\n Version:      1.12.2\n API version:  1.24\n Go version:   go1.6.3\n Git commit:   bb80604\n Built:\n OS/Arch:      linux/amd64\n\nServer:\n Version:      1.12.2\n API version:  1.24\n Go version:   go1.6.3\n Git commit:   bb80604\n Built:\n OS/Arch:      linux/amd64\n```\n\n**Output of `docker info`:**\n\n```\nContainers: 83\n Running: 72\n Paused: 0\n Stopped: 11\nImages: 49\nServer Version: 1.12.2\nStorage Driver: devicemapper\n Pool Name: data-docker_thin\n Pool Blocksize: 65.54 kB\n Base Device Size: 107.4 GB\n Backing Filesystem: ext4\n Data file:\n Metadata file:\n Data Space Used: 33.66 GB\n Data Space Total: 86.72 GB\n Data Space Available: 53.06 GB\n Metadata Space Used: 37.3 MB\n Metadata Space Total: 268.4 MB\n Metadata Space Available: 231.1 MB\n Thin Pool Minimum Free Space: 8.672 GB\n Udev Sync Supported: true\n Deferred Removal Enabled: false\n Deferred Deletion Enabled: false\n Deferred Deleted Device Count: 0\n Library Version: 1.02.107-RHEL7 (2016-06-09)\nLogging Driver: journald\nCgroup Driver: cgroupfs\nPlugins:\n Volume: local\n Network: bridge null overlay host\nSwarm: inactive\nRuntimes: runc\nDefault Runtime: runc\nSecurity Options: seccomp\nKernel Version: 3.10.0-327.10.1.el7.x86_64\nOperating System: CentOS Linux 7 (Core)\nOSType: linux\nArchitecture: x86_64\nCPUs: 2\nTotal Memory: 7.305 GiB\nName: us-2.c.evennode-1234.internal\nID: HVU4:BVZ3:QYUQ:IJ6F:Q2FP:Z4T3:MBKH:I4KC:XFIF:W5DV:4HZW:45NJ\nDocker Root Dir: /var/lib/docker\nDebug Mode (client): false\nDebug Mode (server): false\nRegistry: https://index.docker.io/v1/\nWARNING: bridge-nf-call-iptables is disabled\nWARNING: bridge-nf-call-ip6tables is disabled\nInsecure Registries:\n 127.0.0.0/8\n```\n\n**Additional environment details (AWS, VirtualBox, physical, etc.):**\nAll environments we run servers in - AWS, gcloud, physical, etc.\n"},"comment":{"url":"https://api.github.com/repos/docker/docker/issues/comments/257320198","html_url":"https://github.com/docker/docker/issues/27381#issuecomment-257320198","issue_url":"https://api.github.com/repos/docker/docker/issues/27381","id":257320198,"user":{"login":"rhvgoyal","id":9487534,"avatar_url":"https://avatars.githubusercontent.com/u/9487534?v=3","gravatar_id":"","url":"https://api.github.com/users/rhvgoyal","html_url":"https://github.com/rhvgoyal","followers_url":"https://api.github.com/users/rhvgoyal/followers","following_url":"https://api.github.com/users/rhvgoyal/following{/other_user}","gists_url":"https://api.github.com/users/rhvgoyal/gists{/gist_id}","starred_url":"https://api.github.com/users/rhvgoyal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rhvgoyal/subscriptions","organizations_url":"https://api.github.com/users/rhvgoyal/orgs","repos_url":"https://api.github.com/users/rhvgoyal/repos","events_url":"https://api.github.com/users/rhvgoyal/events{/privacy}","received_events_url":"https://api.github.com/users/rhvgoyal/received_events","type":"User","site_admin":false},"created_at":"2016-10-31T15:10:32Z","updated_at":"2016-10-31T15:10:32Z","body":"You docker-pid and host both seem to be sharing same mount namespace. And that means docker daemon is running in host mount namespace. And that probably means that nginx started at some point after container start and it seems to be running in its own mount namespace. And at that time mount points leaked into nginx mount namespace and that's preventing deletion of container.\r\n\r\nPlease make sure MountFlags=slave is working for you. Once it is working, /proc/<pid>/ns/mnt will give different output for docker daemon and bash shell running in host mount namespace."}}
{"actor":{"display_login":"rhvgoyal"}}