{"action":"created","issue":{"url":"https://api.github.com/repos/docker/for-mac/issues/371","repository_url":"https://api.github.com/repos/docker/for-mac","labels_url":"https://api.github.com/repos/docker/for-mac/issues/371/labels{/name}","comments_url":"https://api.github.com/repos/docker/for-mac/issues/371/comments","events_url":"https://api.github.com/repos/docker/for-mac/issues/371/events","html_url":"https://github.com/docker/for-mac/issues/371","id":172224434,"number":371,"title":"Docker.qcow2 never shrinks - disk space usage leak in docker for mac","user":{"login":"yakticus","id":1957660,"avatar_url":"https://avatars.githubusercontent.com/u/1957660?v=3","gravatar_id":"","url":"https://api.github.com/users/yakticus","html_url":"https://github.com/yakticus","followers_url":"https://api.github.com/users/yakticus/followers","following_url":"https://api.github.com/users/yakticus/following{/other_user}","gists_url":"https://api.github.com/users/yakticus/gists{/gist_id}","starred_url":"https://api.github.com/users/yakticus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yakticus/subscriptions","organizations_url":"https://api.github.com/users/yakticus/orgs","repos_url":"https://api.github.com/users/yakticus/repos","events_url":"https://api.github.com/users/yakticus/events{/privacy}","received_events_url":"https://api.github.com/users/yakticus/received_events","type":"User","site_admin":false},"labels":[{"id":415727199,"url":"https://api.github.com/repos/docker/for-mac/labels/area/storage","name":"area/storage","color":"fbca04","default":false},{"id":415702584,"url":"https://api.github.com/repos/docker/for-mac/labels/kind/bug","name":"kind/bug","color":"FF8C94","default":false},{"id":415702586,"url":"https://api.github.com/repos/docker/for-mac/labels/kind/enhancement","name":"kind/enhancement","color":"E8CaAF","default":false},{"id":438330965,"url":"https://api.github.com/repos/docker/for-mac/labels/status/1-acknowledged","name":"status/1-acknowledged","color":"5319e7","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":47,"created_at":"2016-08-19T21:30:04Z","updated_at":"2016-11-24T18:04:01Z","closed_at":null,"body":"### Expected behavior\n\nAmount of disk space used under ~/Library/Containers/com.docker.docker/ should shrink when deleting images and/or containers\n### Actual behavior\n\nAmount of disk space used under ~/Library/Containers/com.docker.docker/ doesn't shrink after deleting ~50 containers and ~141 images (out of a total of 194). My Docker.qcow2 was 41GB both before and after doing these deletions.\n\nOn the Diagnose & Feedback screen I see an error that looks relevant:\n\n```\nDocker for Mac: version: mac-v1.12.0.1\nOS X: version 10.11.6 (build: 15G31)\nlogs: /tmp/5F7A165B-83AC-4514-AFC4-DB53430D2E9D/20160819-141130.tar.gz\n[OK]     docker-cli\n[OK]     app\n[OK]     moby-syslog\n[ERROR]  disk\n         disk check failed with: Failure(\"exec: /Applications/Docker.app/Contents/Resources/bin/../../../Contents/MacOS/qemu-img check /Users/julie/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2 > /tmp/5F7A165B-83AC-4514-AFC4-DB53430D2E9D/20160819-141130/qemu-img-check.stdout 2> /tmp/5F7A165B-83AC-4514-AFC4-DB53430D2E9D/20160819-141130/qemu-img-check.stderr: exit 2\")\n[OK]     virtualization\n[OK]     system\n[OK]     menubar\n[OK]     osxfs\n[OK]     db\n[OK]     slirp\n[OK]     moby-console\n[OK]     logs\n[OK]     vmnetd\n[OK]     env\n[OK]     moby\n[OK]     driver.amd64-linux\n```\n### Information\n\nFor some history, see https://github.com/docker/docker/issues/23437\n- Diagnostic ID from \"Diagnose & Feedback\" in the menu.\n  I might not have this anymore since I had to do a factory reset in order to reclaim the disk space. After the factory reset, it is: 5F7A165B-83AC-4514-AFC4-DB53430D2E9D\n- a reproducible case if this is a bug, Dockerfiles FTW\n\nEasy to reproduce by pulling some images, observing the size of Docker.qcow2, deleting the images and seeing that the size of Docker.qcow2 does not change. Let me know if more info is needed\n- page URL if this is a docs issue or the name of a man page\n- host distribution and version (OSX 10.10.x, OSX 10.11.x)\n\nOSX 10.11.6\n### Steps to reproduce the behavior\n1. pull several images\n2. run a couple of containers\n3. observe size of ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2\n4. stop and delete all containers\n5. delete all images\n6. repeat step 3. Size has not shrunk despite there being no images or containers\n\nTranscript:\n\n```\n$ docker pull frolvlad/alpine-scala\nUsing default tag: latest\nlatest: Pulling from frolvlad/alpine-scala\ne110a4a17941: Pull complete\n052edde983dc: Pull complete\n44b3390a7725: Pull complete\n0f6b9d2879c1: Pull complete\nDigest: sha256:1d9f6fd716526764a4ba6e389b635cbef8c5d413216bf5d00ca5175fc0d9bac8\nStatus: Downloaded newer image for frolvlad/alpine-scala:latest\n$ du -sm ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2\n1290    /Users/julie/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2\n$ docker pull ubuntu\nUsing default tag: latest\nlatest: Pulling from library/ubuntu\n2f0243478e1f: Pull complete\nd8909ae88469: Pull complete\n820f09abed29: Pull complete\n01193a8f3d88: Pull complete\nDigest: sha256:8e2324f2288c26e1393b63e680ee7844202391414dbd48497e9a4fd997cd3cbf\nStatus: Downloaded newer image for ubuntu:latest\n$ du -sm ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2\n1456    /Users/julie/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2\n$ docker pull python\nUsing default tag: latest\nlatest: Pulling from library/python\n357ea8c3d80b: Pull complete\n52befadefd24: Pull complete\n3c0732d5313c: Pull complete\nceb711c7e301: Pull complete\n4211bb537697: Pull complete\n71f9074c0739: Pull complete\n3e5349707036: Pull complete\nDigest: sha256:a755ad5a30b2146f9a132bf74107aa5be9a26b7cc77956b379af678e0f311b3c\nStatus: Downloaded newer image for python:latest\n$ docker run ubuntu\n$ docker run -it ubuntu bash\nroot@a2ae24adfd12:/# exit\n$ docker run -itd ubuntu bash\n95f5bed47a790d2c8d60544cbc1fa870e1ef4b8c04653508bed34fb25e650536\n$ docker ps -a\nCONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                      PORTS               NAMES\n95f5bed47a79        ubuntu              \"bash\"              9 seconds ago       Up 9 seconds                                    hungry_lalande\na2ae24adfd12        ubuntu              \"bash\"              28 seconds ago      Exited (0) 21 seconds ago                       sharp_borg\na567a8a7aed3        ubuntu              \"/bin/bash\"         53 seconds ago      Exited (0) 52 seconds ago                       ecstatic_poitras\n$ du -sm ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2\n2371    /Users/julie/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2\n$ docker stop hungry_lalande\nhungry_lalande\n$ docker rm -v $(docker ps -a -q -f status=exited)\n95f5bed47a79\na2ae24adfd12\na567a8a7aed3\n$ docker rmi $(docker images -q)\nUntagged: ubuntu:latest\nUntagged: ubuntu@sha256:8e2324f2288c26e1393b63e680ee7844202391414dbd48497e9a4fd997cd3cbf\nDeleted: sha256:f8d79ba03c00bbcd8079cf05b7526ac8f4f422744aad8c3747a29a38ed8c4a41\nDeleted: sha256:b4119cffabb62bd6374f5b06adcb820344490252b621cb8f6a8fad7f6f61b3ae\nDeleted: sha256:83a17331cb4259e2690395586b641aabadf30256f25a032019b816a53a06ae6e\nDeleted: sha256:30ae5404a524e7a713f86ebe832b1e198d1270d62b27c860721055b890a3ae9c\nDeleted: sha256:d8d865b23727fa63dc7d4e9f02efc3e2433c8f220972689c6b6ba09e8136c162\nUntagged: python:latest\nUntagged: python@sha256:a755ad5a30b2146f9a132bf74107aa5be9a26b7cc77956b379af678e0f311b3c\nDeleted: sha256:a08871375facdc1619fc6d19608a4555db479219b2853e2e11de05eddf97e4cb\nDeleted: sha256:0878d93537f4951aa5fa1ca6fd7ef085aa993e520b5af01148d84704ef49abcf\nDeleted: sha256:2d5707b912a332c89665a078a2dabb088cbb4e6982b0d3db36217c89640074da\nDeleted: sha256:5f6d1bd4cff3a349d2b5989bf04295987b6d679c411357612523341d4b6f2264\nDeleted: sha256:0e54104f61d3ae37dc10fbc275796123a40b54859776fb6e937bb5c39655c3e6\nDeleted: sha256:d3c72419ca593d81f207884b70670fd69c4fa3bdecf926ecbbbd9d3bbdd6c7bf\nDeleted: sha256:016fc440b1e6b49d889b0f325d2439c82c92f554d3f015db67ffde823bd2ff3e\nDeleted: sha256:2f71b45e4e254ddceb187b1467f5471f0e14d7124ac2dd7fdd7ddbc76e13f0e5\nUntagged: frolvlad/alpine-scala:latest\nUntagged: frolvlad/alpine-scala@sha256:1d9f6fd716526764a4ba6e389b635cbef8c5d413216bf5d00ca5175fc0d9bac8\nDeleted: sha256:ac1474efe52e017303f24209fff310c734eed81cdd947604545f160d03f2caad\nDeleted: sha256:a08f99075408eb40ce74942b01d08003c8d923b8df0b0e593b73b13540850171\nDeleted: sha256:5c86a5cfb5b67143c7ec277ac533176abb9906072eac82de512f8e822b06e769\nDeleted: sha256:2615ff2c2f8c27913239080269f5be78d541b2564592856548b435f5daa42514\nDeleted: sha256:4fe15f8d0ae69e169824f25f1d4da3015a48feeeeebb265cd2e328e15c6a869f\n$ docker info\nContainers: 0\n Running: 0\n Paused: 0\n Stopped: 0\nImages: 0\nServer Version: 1.12.0\nStorage Driver: aufs\n Root Dir: /var/lib/docker/aufs\n Backing Filesystem: extfs\n Dirs: 0\n Dirperm1 Supported: true\nLogging Driver: json-file\nCgroup Driver: cgroupfs\nPlugins:\n Volume: local\n Network: null host bridge overlay\nSwarm: inactive\nRuntimes: runc\nDefault Runtime: runc\nSecurity Options: seccomp\nKernel Version: 4.4.15-moby\nOperating System: Alpine Linux v3.4\nOSType: linux\nArchitecture: x86_64\nCPUs: 4\nTotal Memory: 1.954 GiB\nName: moby\nID: KQ7I:O3AX:CY77:DKU4:6LZR:Y6RF:ZM2P:U2YA:TZUV:3YDV:DQPU:OTJV\nDocker Root Dir: /var/lib/docker\nDebug Mode (client): false\nDebug Mode (server): true\n File Descriptors: 18\n Goroutines: 29\n System Time: 2016-08-19T21:24:33.294921776Z\n EventsListeners: 1\nNo Proxy: *.local, 169.254/16\nRegistry: https://index.docker.io/v1/\nInsecure Registries:\n 127.0.0.0/8\n$ du -sm ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2\n2371    /Users/julie/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2\n```\n"},"comment":{"url":"https://api.github.com/repos/docker/for-mac/issues/comments/262826610","html_url":"https://github.com/docker/for-mac/issues/371#issuecomment-262826610","issue_url":"https://api.github.com/repos/docker/for-mac/issues/371","id":262826610,"user":{"login":"djs55","id":198586,"avatar_url":"https://avatars.githubusercontent.com/u/198586?v=3","gravatar_id":"","url":"https://api.github.com/users/djs55","html_url":"https://github.com/djs55","followers_url":"https://api.github.com/users/djs55/followers","following_url":"https://api.github.com/users/djs55/following{/other_user}","gists_url":"https://api.github.com/users/djs55/gists{/gist_id}","starred_url":"https://api.github.com/users/djs55/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/djs55/subscriptions","organizations_url":"https://api.github.com/users/djs55/orgs","repos_url":"https://api.github.com/users/djs55/repos","events_url":"https://api.github.com/users/djs55/events{/privacy}","received_events_url":"https://api.github.com/users/djs55/received_events","type":"User","site_admin":false},"created_at":"2016-11-24T18:04:01Z","updated_at":"2016-11-24T18:04:01Z","body":"Quick update on this: configuring the qcow2 size cap is possible in the latest betas (e.g. 30.1) with the caveats:\r\n- that it won't grow automatically yet: the `Docker.qcow2` needs to be deleted (loosing containers and images) and then the app restarted\r\n- there is no UI element yet\r\n\r\nFor example:\r\n\r\n```\r\n# my disk is currently 64GiB\r\n$ /Applications/Docker.app/Contents/MacOS/qemu-img info ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2\r\nimage: /Users/djs/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2\r\nfile format: qcow2\r\nvirtual size: 64G (68719476736 bytes)\r\ndisk size: 28G\r\ncluster_size: 65536\r\nFormat specific information:\r\n    compat: 1.1\r\n    lazy refcounts: true\r\n    refcount bits: 16\r\n    corrupt: false\r\n\r\n# I can increase or decrease the default cap by:\r\n$ cd ~/Library/Containers/com.docker.docker/Data/database/\r\n$ git reset --hard\r\nHEAD is now at 8635944 last-start-time changed at 1480009710\r\n$ cat com.docker.driver.amd64-linux/disk/size\r\n65536\r\n$ echo 131072 > com.docker.driver.amd64-linux/disk size\r\n$ git add com.docker.driver.amd64-linux/disk/size\r\n$ git commit -s -m 'increase size to 128GiB'\r\n[master 60861de] increase size to 128GiB\r\n 1 file changed, 1 insertion(+), 1 deletion(-)\r\n\r\n# delete the disk -- this unfortunately deletes all images and containers\r\n$ rm ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2\r\n# now I need to restart the app\r\n```\r\nFor anyone wanting to use an external drive, you can also change the location of the `Docker.qcow2` by editing the database key `com.docker.driver.amd64-linux/disk/path`. There is no UI element for this yet either.\r\n\r\nRegarding automatic shrinkage via TRIM: I'm working on this at the moment. There are early patches which expose the TRIM feature to the VM; and for the VM to invoke TRIM; the next step is to modify the `qcow2` code to record the TRIMmed blocks somewhere where they can be read by a compactor. I'll let you know when there's some code available to test.\r\n\r\n"}}
{"actor":{"display_login":"djs55"}}