{"action":"opened","issue":{"url":"https://api.github.com/repos/docker/distribution/issues/2094","repository_url":"https://api.github.com/repos/docker/distribution","labels_url":"https://api.github.com/repos/docker/distribution/issues/2094/labels{/name}","comments_url":"https://api.github.com/repos/docker/distribution/issues/2094/comments","events_url":"https://api.github.com/repos/docker/distribution/issues/2094/events","html_url":"https://github.com/docker/distribution/issues/2094","id":193298055,"number":2094,"title":"Strange behaviour when pushing an image after deleting the manifest from the registry","user":{"login":"lorenzobenvenuti","id":7447650,"avatar_url":"https://avatars.githubusercontent.com/u/7447650?v=3","gravatar_id":"","url":"https://api.github.com/users/lorenzobenvenuti","html_url":"https://github.com/lorenzobenvenuti","followers_url":"https://api.github.com/users/lorenzobenvenuti/followers","following_url":"https://api.github.com/users/lorenzobenvenuti/following{/other_user}","gists_url":"https://api.github.com/users/lorenzobenvenuti/gists{/gist_id}","starred_url":"https://api.github.com/users/lorenzobenvenuti/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lorenzobenvenuti/subscriptions","organizations_url":"https://api.github.com/users/lorenzobenvenuti/orgs","repos_url":"https://api.github.com/users/lorenzobenvenuti/repos","events_url":"https://api.github.com/users/lorenzobenvenuti/events{/privacy}","received_events_url":"https://api.github.com/users/lorenzobenvenuti/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-12-03T16:22:53Z","updated_at":"2016-12-03T16:22:53Z","closed_at":null,"body":"Hi,\r\n\r\nI'm experiencing an unexpected behaviour when deleting a manifest. I'm doing some experiments in order to understand how to cleanup a local registry thats hosts some images that are build by periodic tasks. The idea was to:\r\n\r\n* Create image with tag 1\r\n* Run the container\r\n* Remove image with tag 1 from the registry (= remove the manifest and run garbage collector to remove \"dangling\" layers)\r\n* Repeat\r\n\r\nTo validate this strategy I tried to setup a testing environment.\r\n\r\n```bash\r\n$ docker run -d -p 5000:5000 -e REGISTRY_STORAGE_DELETE_ENABLED=true -v /home/lorenzob/registry:/var/lib/registry --name reg registry:2.5.1\r\n\r\n$ docker tag ubuntu:latest localhost:5000/lorenzo/ubuntu:1.0\r\n\r\n$ docker push localhost:5000/lorenzo/ubuntu:1.0\r\n\r\n$ du -sh registry/\r\n47M\tregistry/\r\n```\r\nAt this point I tried to remove the image:\r\n\r\n```bash\r\n$ curl -I -X HEAD http://localhost:5000/v2/lorenzo/ubuntu/manifests/1.0\r\nHTTP/1.1 200 OK\r\nContent-Length: 9033\r\nContent-Type: application/vnd.docker.distribution.manifest.v1+prettyjws\r\nDocker-Content-Digest: sha256:27f101313deea7d60456d92440dc7e1584b5ef3d9984d5564e588ee3ee760042\r\nDocker-Distribution-Api-Version: registry/2.0\r\nEtag: \"sha256:27f101313deea7d60456d92440dc7e1584b5ef3d9984d5564e588ee3ee760042\"\r\nX-Content-Type-Options: nosniff\r\nDate: Sat, 03 Dec 2016 16:02:27 GMT\r\n\r\n$ curl -X DELETE http://localhost:5000/v2/lorenzo/ubuntu/manifests/sha256:27f101313deea7d60456d92440dc7e1584b5ef3d9984d5564e588ee3ee760042\r\n\r\n$ curl -X GET http://localhost:5000/v2/lorenzo/ubuntu/manifests/1.0\r\n{\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"manifest unknown\",\"detail\":{\"Tag\":\"1.0\"}}]}\r\n\r\n$ du -sh registry/\r\n47M\tregistry/\r\n```\r\nThe manifest is gone, the layers are still there. Running the garbage collector does the trick:\r\n\r\n```bash\r\n$ docker exec reg /bin/registry garbage-collect /etc/docker/registry/config.yml\r\ntime=\"2016-12-03T16:05:13Z\" level=info msg=\"Deleting blob: /docker/registry/v2/blobs/sha256/5b/5ba4f30e5bea63dcc2e7054b8b4f41ab1e5fcc7db0a88fc79359b890bcfe2258\" go.version=go1.6.3 instance.id=7320555a-9170-4b25-8d07-425a5cc7d25f \r\ntime=\"2016-12-03T16:05:13Z\" level=info msg=\"Deleting blob: /docker/registry/v2/blobs/sha256/9d/9d7d19c9dc56f51e04f598d8bb8dba1f5cde61434935959ef827ced31f1eb80c\" go.version=go1.6.3 instance.id=7320555a-9170-4b25-8d07-425a5cc7d25f \r\ntime=\"2016-12-03T16:05:13Z\" level=info msg=\"Deleting blob: /docker/registry/v2/blobs/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4\" go.version=go1.6.3 instance.id=7320555a-9170-4b25-8d07-425a5cc7d25f \r\ntime=\"2016-12-03T16:05:13Z\" level=info msg=\"Deleting blob: /docker/registry/v2/blobs/sha256/ac/ac6ad7efd0f97fbaa04d98cbfa6deb994382723ba32b23e46c0ef5b0afcc543e\" go.version=go1.6.3 instance.id=7320555a-9170-4b25-8d07-425a5cc7d25f \r\ntime=\"2016-12-03T16:05:13Z\" level=info msg=\"Deleting blob: /docker/registry/v2/blobs/sha256/e7/e7491a747824095463fa464d4d5f7941ff65277012f8102464e9531eb59db7a5\" go.version=go1.6.3 instance.id=7320555a-9170-4b25-8d07-425a5cc7d25f \r\ntime=\"2016-12-03T16:05:13Z\" level=info msg=\"Deleting blob: /docker/registry/v2/blobs/sha256/27/27f101313deea7d60456d92440dc7e1584b5ef3d9984d5564e588ee3ee760042\" go.version=go1.6.3 instance.id=7320555a-9170-4b25-8d07-425a5cc7d25f\r\n\r\n$ du -sh registry/\r\n132K\tregistry/\r\n```\r\n\r\nFine, I thought: it's working! But when I tried to add a new version of the image...\r\n\r\n```bash\r\n$ docker tag ubuntu:latest localhost:5000/lorenzo/ubuntu:2.0 \r\n$ docker push localhost:5000/lorenzo/ubuntu:2.0\r\n$ du -sh registry/\r\n188K\tregistry/\r\n```\r\n\r\nWhat? I was expecting this to be **47MB**! The strange thing is that the manifest at `http://localhost:5000/v2/lorenzo/ubuntu/manifests/2.0` seems ok, and I can even run a container from that image: \r\n\r\n```bash\r\n$ docker run --rm localhost:5000/lorenzo/ubuntu:2.0 echo \"Hi\"\r\nHi\r\n```\r\nAnyway if I try to delete it I receive an error:\r\n\r\n```\r\n$ curl -X DELETE http://localhost:5000/v2/lorenzo/ubuntu/manifests/sha256:27f101313deea7d60456d92440dc7e1584b5ef3d9984d5564e588ee3ee760042\r\n{\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"manifest unknown\"}]}\r\n```\r\n\r\nThis looks like a bug to me. Is this related to the fact that I'm actually adding the same image twice with different tags (so the digest is the same)? \r\n\r\nThank you,\r\n\r\nlorenzo\r\n\r\nPS: I think this has more to do with the registry than with docker itself, but to provide as much informations as I can I'm using docker 1.6.2:\r\n\r\n```\r\n$ docker version\r\nClient version: 1.6.2\r\nClient API version: 1.18\r\nGo version (client): go1.5.1\r\nGit commit (client): 7c8fca2\r\nOS/Arch (client): linux/amd64\r\nServer version: 1.6.2\r\nServer API version: 1.18\r\nGo version (server): go1.5.1\r\nGit commit (server): 7c8fca2\r\nOS/Arch (server): linux/amd64\r\n```\r\n"}}
{"actor":{"display_login":"lorenzobenvenuti"}}