{"action":"created","issue":{"url":"https://api.github.com/repos/docker/docker/issues/27453","repository_url":"https://api.github.com/repos/docker/docker","labels_url":"https://api.github.com/repos/docker/docker/issues/27453/labels{/name}","comments_url":"https://api.github.com/repos/docker/docker/issues/27453/comments","events_url":"https://api.github.com/repos/docker/docker/issues/27453/events","html_url":"https://github.com/docker/docker/issues/27453","id":183462248,"number":27453,"title":"Add CPU hotplug support","user":{"login":"ddingel","id":20914710,"avatar_url":"https://avatars.githubusercontent.com/u/20914710?v=3","gravatar_id":"","url":"https://api.github.com/users/ddingel","html_url":"https://github.com/ddingel","followers_url":"https://api.github.com/users/ddingel/followers","following_url":"https://api.github.com/users/ddingel/following{/other_user}","gists_url":"https://api.github.com/users/ddingel/gists{/gist_id}","starred_url":"https://api.github.com/users/ddingel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ddingel/subscriptions","organizations_url":"https://api.github.com/users/ddingel/orgs","repos_url":"https://api.github.com/users/ddingel/repos","events_url":"https://api.github.com/users/ddingel/events{/privacy}","received_events_url":"https://api.github.com/users/ddingel/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-10-17T16:20:24Z","updated_at":"2016-10-18T02:00:47Z","closed_at":null,"body":"### Description\r\n\r\nHotplugged CPUs are currently not recognized by Docker.\r\n\r\n### Steps to reproduce the issue:\r\n\r\n**Environment:**\r\n\r\nThe tests have been done on a s390x Ubuntu Xenial z/VM guest with three virtual CPUs and latest Docker built via `make deb`:\r\n\r\n    root@r3545010:~# cat /etc/os-release \r\n    NAME=\"Ubuntu\"\r\n    VERSION=\"16.04.1 LTS (Xenial Xerus)\"\r\n\r\n    root@r3545010:~# uname -a\r\n    Linux r3545010 4.4.0-21-generic #37-Ubuntu SMP Mon Apr 18 18:31:26 UTC 2016 s390x s390x s390x GNU/Linux\r\n\r\n    root@r3545010:~# docker version\r\n    Client:\r\n     Version:      1.13.0-dev\r\n     API version:  1.25\r\n     Go version:   go1.7.1\r\n     Git commit:   c505486\r\n     Built:        Tue Sep 27 16:49:25 2016\r\n     OS/Arch:      linux/s390x\r\n\r\n**Test procedure:**\r\n\r\n-   Stop docker daemon\r\n\r\n-   Enable one CPU\r\n\r\n-   Start docker daemon\r\n\r\n-   Start first container and start two CPU burner processes\r\n\r\n-   Enable second CPU\r\n\r\n-   Check if second CPU is used by first container\r\n\r\n-   Start second container and start two CPU burner processes\r\n\r\n-   Check if second CPU is used by second container\r\n\r\n**Describe the results you received:**\r\n\r\n    root@r3545010:~# lscpu -e\r\n    CPU NODE BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS\r\n    0   0    0    0      0    0:0:0:0         yes    yes        horizontal   0\r\n    1   0    -    -      -    :::             no     yes        horizontal   1\r\n    2   0    -    -      -    :::             no     yes        horizontal   2\r\n\r\n    root@r3545010:~# systemctl start docker \r\n\r\n    root@r3545010:~# docker run -ti --rm rhel:7.1 bash\r\n\r\n    bash-4.2# while [ true ]; do true; done &\r\n    [1] 7\r\n    bash-4.2# while [ true ]; do true; done &\r\n    [2] 8\r\n\r\n    root@r3545010:~# top\r\n    top - 19:22:08 up 1 day,  1:13,  2 users,  load average: 0.59, 0.83, 0.78\r\n    Tasks: 113 total,   3 running, 110 sleeping,   0 stopped,   0 zombie\r\n    %Cpu0  : 97.4 us,  2.3 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.3 st\r\n    KiB Mem :   881040 total,   385944 free,    78736 used,   416360 buff/cache\r\n    KiB Swap:        0 total,        0 free,        0 used.   755592 avail Mem \r\n\r\n      PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND\r\n    23703 root      20   0    3536   1756   1416 R 50.0  0.2   0:08.83 bash\r\n    23702 root      20   0    3536   1760   1420 R 49.7  0.2   0:10.43 bash\r\n\r\n    root@r3545010:~# chcpu -e 1\r\n    CPU 1 enabled\r\n\r\n    root@r3545010:~# lscpu -e\r\n    CPU NODE BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS\r\n    0   0    0    0      0    0:0:0:0         yes    yes        horizontal   0\r\n    1   0    1    1      1    1:1:1:1         yes    yes        horizontal   1\r\n    2   0    -    -      -    :::             no     yes        horizontal   2\r\n\r\n    root@r3545010:~# top\r\n    top - 19:23:19 up 1 day,  1:14,  2 users,  load average: 1.56, 1.08, 0.87\r\n    Tasks: 110 total,   3 running, 107 sleeping,   0 stopped,   0 zombie\r\n    %Cpu0  : 97.3 us,  2.3 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.3 st\r\n    %Cpu1  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st\r\n    KiB Mem :   881040 total,   384772 free,    79844 used,   416424 buff/cache\r\n    KiB Swap:        0 total,        0 free,        0 used.   754436 avail Mem \r\n\r\n      PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND\r\n    23703 root      20   0    3536   1756   1416 R  50.0  0.2   0:44.16 bash\r\n    23702 root      20   0    3536   1760   1420 R  49.7  0.2   0:45.75 bash\r\n\r\nLooking at the cpusets it turned out that Docker and its containers only use CPU 0. The global cpuset also contains the new CPU 1:\r\n\r\n    root@r3545010:/sys/fs/cgroup/cpuset# cat ./cpuset.cpus\r\n    0-1\r\n    root@r3545010:/sys/fs/cgroup/cpuset# cat ./docker/cpuset.cpus\r\n    0\r\n    root@r3545010:/sys/fs/cgroup/cpuset# cat ./docker/c22..217/cpuset.cpus\r\n    0\r\n\r\nIt is possible to manually enable the CPU in the cpusets. First enable docker/cpuset.cpus and then do it for the container:\r\n\r\n    root@r3545010:/sys/fs/cgroup/cpuset# echo \"0-1\" > ./docker/cpuset.cpus\r\n    root@r3545010:/sys/fs/cgroup/cpuset# echo \"0-1\" > ./docker/c22..217/cpuset.cpus\r\n\r\nAfter this change the container uses the new CPU:\r\n\r\n    top - 19:55:28 up 1 day,  1:46,  2 users,  load average: 2.00, 1.99, 1.86\r\n    Tasks: 110 total,   3 running, 107 sleeping,   0 stopped,   0 zombie\r\n    %Cpu0  : 97.7 us,  2.3 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st\r\n    %Cpu1  : 97.7 us,  2.3 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st\r\n    KiB Mem :   881040 total,   384200 free,    80144 used,   416696 buff/cache\r\n    KiB Swap:        0 total,        0 free,        0 used.   754060 avail Mem \r\n\r\n      PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND\r\n    23703 root      20   0    3536   1756   1416 R 100.0  0.2  19:46.40 bash\r\n    23702 root      20   0    3536   1760   1420 R 100.0  0.2  19:47.84 bash\r\n\r\nWhen disabling and afterwards enabling CPU 1 again, the kernel (and Docker) do not include that CPU in the cpuset:\r\n\r\n    root@r3545010:/sys/fs/cgroup/cpuset# chcpu -d 1\r\n    CPU 1 disabled\r\n    root@r3545010:/sys/fs/cgroup/cpuset# chcpu -e 1\r\n    CPU 1 enabled\r\n    root@r3545010:/sys/fs/cgroup/cpuset# cat docker/cpuset.cpus \r\n    0\r\n    root@r3545010:/sys/fs/cgroup/cpuset# cat ./docker/c22..217/cpuset.cpus\r\n    0\r\n\r\nAlso newly created containers only use CPU 0.\r\n\r\n### Describe the results you expected:\r\n\r\nAlready running and newly started containers should be able to use additional online CPUs.\r\n\r\n### Initial proposal\r\nContainers which are not started with a `--cpuset-cpus` or `--cpuset-mems` option are called **unrestricted** in the following, the others are called **restricted**.\r\n\r\nFor each unrestricted container we skip the current creation of a cpuset cgroup sub-directory under the Docker cgroup directory. Then all tasks within that container will be automatically associated to the root cpuset cgroup. Because the root cpuset cgroup is managed by the kernel its cpuset.cpus will always contain all online CPUs. Therefore unrestricted containers will use all online CPUs.\r\n\r\nThis issue was created together with @michael-holzheu."},"comment":{"url":"https://api.github.com/repos/docker/docker/issues/comments/254385821","html_url":"https://github.com/docker/docker/issues/27453#issuecomment-254385821","issue_url":"https://api.github.com/repos/docker/docker/issues/27453","id":254385821,"user":{"login":"hqhq","id":6415670,"avatar_url":"https://avatars.githubusercontent.com/u/6415670?v=3","gravatar_id":"","url":"https://api.github.com/users/hqhq","html_url":"https://github.com/hqhq","followers_url":"https://api.github.com/users/hqhq/followers","following_url":"https://api.github.com/users/hqhq/following{/other_user}","gists_url":"https://api.github.com/users/hqhq/gists{/gist_id}","starred_url":"https://api.github.com/users/hqhq/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hqhq/subscriptions","organizations_url":"https://api.github.com/users/hqhq/orgs","repos_url":"https://api.github.com/users/hqhq/repos","events_url":"https://api.github.com/users/hqhq/events{/privacy}","received_events_url":"https://api.github.com/users/hqhq/received_events","type":"User","site_admin":false},"created_at":"2016-10-18T02:00:47Z","updated_at":"2016-10-18T02:00:47Z","body":"We had this discussing before, see if https://github.com/opencontainers/runc/pull/254 helps."}}
{"actor":{"display_login":"hqhq"}}