{"action":"created","issue":{"url":"https://api.github.com/repos/dotnet/dotnet-docker/issues/237","repository_url":"https://api.github.com/repos/dotnet/dotnet-docker","labels_url":"https://api.github.com/repos/dotnet/dotnet-docker/issues/237/labels{/name}","comments_url":"https://api.github.com/repos/dotnet/dotnet-docker/issues/237/comments","events_url":"https://api.github.com/repos/dotnet/dotnet-docker/issues/237/events","html_url":"https://github.com/dotnet/dotnet-docker/issues/237","id":227518135,"number":237,"title":"NuGetFallbackFolder adds unnecessary bloat to build containers","user":{"login":"natemcmaster","id":2696087,"avatar_url":"https://avatars0.githubusercontent.com/u/2696087?v=4","gravatar_id":"","url":"https://api.github.com/users/natemcmaster","html_url":"https://github.com/natemcmaster","followers_url":"https://api.github.com/users/natemcmaster/followers","following_url":"https://api.github.com/users/natemcmaster/following{/other_user}","gists_url":"https://api.github.com/users/natemcmaster/gists{/gist_id}","starred_url":"https://api.github.com/users/natemcmaster/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/natemcmaster/subscriptions","organizations_url":"https://api.github.com/users/natemcmaster/orgs","repos_url":"https://api.github.com/users/natemcmaster/repos","events_url":"https://api.github.com/users/natemcmaster/events{/privacy}","received_events_url":"https://api.github.com/users/natemcmaster/received_events","type":"User","site_admin":false},"labels":[{"id":287524926,"url":"https://api.github.com/repos/dotnet/dotnet-docker/labels/bug","name":"bug","color":"fc2929","default":true},{"id":631613828,"url":"https://api.github.com/repos/dotnet/dotnet-docker/labels/triaged","name":"triaged","color":"006b75","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":10,"created_at":"2017-05-09T22:37:02Z","updated_at":"2018-02-01T06:19:24Z","closed_at":null,"author_association":"MEMBER","body":"In .NET Core 2.0, the CLI started shipping a feature that adds an offline package feed called the NuGetFallbackFolder. This adds unnecessary bloat to SDK images in the following ways:\r\n\r\n - ~660 MB of xmldocs are stored in `~/.dotnet/NuGetFallbackFolder`. xmldocs are unnecessary in build images.\r\n - first run of dotnet-restore creates about 5,700 duplicate files which adds about 326 MB. This duplication is caused by copying files from `~/.dotnet/NuGetFallbackFolder` to `~/.nuget/packages`\r\n - Any users that build inside the image and don't clear out the `~/.nuget/packages` folder end up with a bloated image. Here is the resulting duplication between the nuget cache and fallback folders:\r\n   ```\r\n   fdupes -r -m ~/.dotnet/NuGetFallbackFolder/ ~/.nuget/packages/\r\n   11439 duplicate files (in 2809 sets), occupying 876.3 megabytes\r\n   ```\r\n\r\n**Suggestions**\r\n- Exclude xmldocs from NuGet fallback folder\r\n- Disable the offline package cache feature altogether.\r\n- Warmup the NuGet cache by running dotnet-restore as a part of the image creation.\r\n\r\nOr some combination of the ideas above.\r\n\r\n**Details**\r\nHow I measured the numbers\r\nUsing: `microsoft/dotnet-nightly:2.0.0-preview1-sdk`\r\n\r\n - Measure the effect of a dotnet-restore\r\n   1.  Install fdupes tool. `apt-get update -qq && apt-get install -y fdupes`\r\n   1. `mkdir /app && cd /app`\r\n   1. `ls ~/.nuget/`. The `~/.nuget/packages/` folder does not yet exist.\r\n   1.  `dotnet new web` (automatically launches `dotnet restore`)\r\n   1. `fdupes -r -m ~/.dotnet/NuGetFallbackFolder/ ~/.nuget/packages/`\r\n   1. Count files added to nuget cache: `find ~/.nuget/packages -type f | wc -l`\r\n - Count xmldoc files \r\n   `find ~/.dotnet -name \"*.xml\" -type f | wc -l `\r\n - Count xmldoc filesize:\r\n   `find ~/.dotnet/ -name \"*.xml\" -type f -print0 | du -sb --files0-from=-  | awk '{ total += $1} END { print total/1024/1024 }'` (number is in MB)\r\n\r\ncc @emgarten @rrelyea @dotnet/dotnet-cli @glennc"},"comment":{"url":"https://api.github.com/repos/dotnet/dotnet-docker/issues/comments/362170046","html_url":"https://github.com/dotnet/dotnet-docker/issues/237#issuecomment-362170046","issue_url":"https://api.github.com/repos/dotnet/dotnet-docker/issues/237","id":362170046,"user":{"login":"natemcmaster","id":2696087,"avatar_url":"https://avatars0.githubusercontent.com/u/2696087?v=4","gravatar_id":"","url":"https://api.github.com/users/natemcmaster","html_url":"https://github.com/natemcmaster","followers_url":"https://api.github.com/users/natemcmaster/followers","following_url":"https://api.github.com/users/natemcmaster/following{/other_user}","gists_url":"https://api.github.com/users/natemcmaster/gists{/gist_id}","starred_url":"https://api.github.com/users/natemcmaster/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/natemcmaster/subscriptions","organizations_url":"https://api.github.com/users/natemcmaster/orgs","repos_url":"https://api.github.com/users/natemcmaster/repos","events_url":"https://api.github.com/users/natemcmaster/events{/privacy}","received_events_url":"https://api.github.com/users/natemcmaster/received_events","type":"User","site_admin":false},"created_at":"2018-02-01T06:19:24Z","updated_at":"2018-02-01T06:19:24Z","author_association":"MEMBER","body":"It looks like the fallback folder behavior is working now. However, I think there is still an opportunity here to trim some bloat. If I've done my bash/math right, here are the (uncompressed) size of files in `/usr/share/dotnet`\r\n\r\n<img src=\"https://user-images.githubusercontent.com/2696087/35663936-a806dc0a-06d4-11e8-922d-07d921d7a591.png\" width=\"240\" />\r\n"}}
{"actor":{"display_login":"natemcmaster"}}