{"action":"created","issue":{"url":"https://api.github.com/repos/opencontainers/runc/issues/861","repository_url":"https://api.github.com/repos/opencontainers/runc","labels_url":"https://api.github.com/repos/opencontainers/runc/issues/861/labels{/name}","comments_url":"https://api.github.com/repos/opencontainers/runc/issues/861/comments","events_url":"https://api.github.com/repos/opencontainers/runc/issues/861/events","html_url":"https://github.com/opencontainers/runc/issues/861","id":157637271,"number":861,"title":"Joining cgroups blindly causes performance problems","user":{"login":"cyphar","id":2888411,"avatar_url":"https://avatars.githubusercontent.com/u/2888411?v=3","gravatar_id":"","url":"https://api.github.com/users/cyphar","html_url":"https://github.com/cyphar","followers_url":"https://api.github.com/users/cyphar/followers","following_url":"https://api.github.com/users/cyphar/following{/other_user}","gists_url":"https://api.github.com/users/cyphar/gists{/gist_id}","starred_url":"https://api.github.com/users/cyphar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cyphar/subscriptions","organizations_url":"https://api.github.com/users/cyphar/orgs","repos_url":"https://api.github.com/users/cyphar/repos","events_url":"https://api.github.com/users/cyphar/events{/privacy}","received_events_url":"https://api.github.com/users/cyphar/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-05-31T10:17:01Z","updated_at":"2016-06-07T23:20:49Z","closed_at":null,"body":"It turns out that joining cgroups that we don't use has a non-zero performance degredation. The most obvious case is with `blkio` which can cause operations to become **10 times** slower. The following test assumes you have some block device `/dev/sdX` that is formatted as `ext4` (this used a spinning hard drive, but you could also use a flash usb):\r\n\r\n```\r\n# mount /dev/sdX /workspace\r\n# echo $$ > /sys/fs/cgroup/blkio/cgroup.procs\r\n# time dd if=/dev/zero of=/workspace/test.bin bs=512 count=1000 oflag=dsync\r\n1000+0 records in\r\n1000+0 records out\r\n512000 bytes (512 kB) copied, 2.09553 s, 244 kB/s\r\n\r\nreal    0m2.097s\r\nuser    0m0.000s\r\nsys     0m0.144s\r\n# mkdir /sys/fs/cgroup/blkio/test\r\n# echo $$ >/sys/fs/cgroup/blkio/test/cgroup.procs\r\n# time dd if=/dev/zero of=/workspace/test.bin bs=512 count=1000 oflag=dsync\r\n1000+0 records in\r\n1000+0 records out\r\n512000 bytes (512 kB) copied, 19.5512 s, 26.2 kB/s\r\n\r\nreal    0m19.553s\r\nuser    0m0.000s\r\nsys     0m0.132s \r\n```\r\n\r\nThis is already a known issue upstream (in the kernel), but it is a general problem (most cgroup controllers have special cases for their root cgroup to maximise performance -- but few have optimisations to reduce the hierarchy based on which cgroups have limits set).\r\n\r\nUnfortunately, the naive solution (not joining cgroups if we don't intend to use them in `config.json`) causes obvious issues with `runc update` (and commands that make assumptions about which cgroups we've joined). So we'd have to write quite a bit of code to create new cgroups and join container processes to them if the user requests a limit that wasn't required before. We could do it with the freezer cgroup and some enumeration.\r\n\r\nThis is (somewhat) related to the lazy cgroup work that we should do as a part of #774. \r\n\r\nThe performance issue described in docker/docker#21485 occurs because of this."},"comment":{"url":"https://api.github.com/repos/opencontainers/runc/issues/comments/224443446","html_url":"https://github.com/opencontainers/runc/issues/861#issuecomment-224443446","issue_url":"https://api.github.com/repos/opencontainers/runc/issues/861","id":224443446,"user":{"login":"cyphar","id":2888411,"avatar_url":"https://avatars.githubusercontent.com/u/2888411?v=3","gravatar_id":"","url":"https://api.github.com/users/cyphar","html_url":"https://github.com/cyphar","followers_url":"https://api.github.com/users/cyphar/followers","following_url":"https://api.github.com/users/cyphar/following{/other_user}","gists_url":"https://api.github.com/users/cyphar/gists{/gist_id}","starred_url":"https://api.github.com/users/cyphar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cyphar/subscriptions","organizations_url":"https://api.github.com/users/cyphar/orgs","repos_url":"https://api.github.com/users/cyphar/repos","events_url":"https://api.github.com/users/cyphar/events{/privacy}","received_events_url":"https://api.github.com/users/cyphar/received_events","type":"User","site_admin":false},"created_at":"2016-06-07T23:20:49Z","updated_at":"2016-06-07T23:20:49Z","body":"This could potentially also be fixed by the lazy cgroup handling (by only attaching to cgroups that we are using, and then attaching later if a user tries to update the limits).\r\n\r\n/cc @chb @monstermunchkin"}}
{"actor":{"display_login":"cyphar"}}