{"action":"created","issue":{"url":"https://api.github.com/repos/coreos/bugs/issues/1281","repository_url":"https://api.github.com/repos/coreos/bugs","labels_url":"https://api.github.com/repos/coreos/bugs/issues/1281/labels{/name}","comments_url":"https://api.github.com/repos/coreos/bugs/issues/1281/comments","events_url":"https://api.github.com/repos/coreos/bugs/issues/1281/events","html_url":"https://github.com/coreos/bugs/issues/1281","id":154599622,"number":1281,"title":"Alpha version exhausting processes/threads when limits are not reached","user":{"login":"victorgp","id":593300,"avatar_url":"https://avatars.githubusercontent.com/u/593300?v=3","gravatar_id":"","url":"https://api.github.com/users/victorgp","html_url":"https://github.com/victorgp","followers_url":"https://api.github.com/users/victorgp/followers","following_url":"https://api.github.com/users/victorgp/following{/other_user}","gists_url":"https://api.github.com/users/victorgp/gists{/gist_id}","starred_url":"https://api.github.com/users/victorgp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/victorgp/subscriptions","organizations_url":"https://api.github.com/users/victorgp/orgs","repos_url":"https://api.github.com/users/victorgp/repos","events_url":"https://api.github.com/users/victorgp/events{/privacy}","received_events_url":"https://api.github.com/users/victorgp/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/coreos/bugs/labels/area/usability","name":"area/usability","color":"bfe5bf"},{"url":"https://api.github.com/repos/coreos/bugs/labels/component/systemd","name":"component/systemd","color":"0052cc"},{"url":"https://api.github.com/repos/coreos/bugs/labels/team/os","name":"team/os","color":"ffffff"}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":13,"created_at":"2016-05-12T22:56:51Z","updated_at":"2016-09-29T11:51:17Z","closed_at":"2016-05-24T19:20:27Z","body":"The current alpha versions 1032.1.0 and 1045.0.0 (the ones i tried) are exhausting the processes/threads very quickly even when the limits are not reached.\r\n\r\nThis problem doesn't occur in stable(899.17.0) and beta (1010.3.0) versions\r\n\r\nI also increased the limits in systemd to discard that, but fails anyway.\r\n\r\nEasy to reproduce:\r\n\r\n1- Install alpha version\r\n2- Create file /etc/systemd/system/docker.service.d/10-settings.conf with content:\r\n```\r\n[Service]\r\nTasksAccounting=true\r\nTasksMax=infinity\r\nLimitNPROC=1048600\r\nLimitNOFILE=1048601\r\n```\r\n3- Restart docker\r\n```\r\nsystemctl daemon-reload\r\nsystemctl restart docker\r\n```\r\n\r\n4- Run a container:\r\n```\r\ndocker run --rm -ti ubuntu:trusty /bin/bash\r\n```\r\n\r\n5- Check limits inside the container\r\n```\r\nroot@b48a00426ee9:/# ulimit -a\r\ncore file size          (blocks, -c) unlimited\r\ndata seg size           (kbytes, -d) unlimited\r\nscheduling priority             (-e) 0\r\nfile size               (blocks, -f) unlimited\r\npending signals                 (-i) 256425\r\nmax locked memory       (kbytes, -l) 64\r\nmax memory size         (kbytes, -m) unlimited\r\nopen files                      (-n) 65536\r\npipe size            (512 bytes, -p) 8\r\nPOSIX message queues     (bytes, -q) 819200\r\nreal-time priority              (-r) 0\r\nstack size              (kbytes, -s) 8192\r\ncpu time               (seconds, -t) unlimited\r\nmax user processes              (-u) 1048600\r\nvirtual memory          (kbytes, -v) unlimited\r\nfile locks                      (-x) unlimited\r\n```\r\n\r\n6- Loop inside the container that creates a lot of threads\r\n```\r\nroot@b48a00426ee9:/# for i in {1..3000}; do sleep 5 & done\r\n```\r\n\r\nOutput:\r\n\r\n```\r\n[0] 24\r\n[...]\r\n[500] 524\r\n[501] 525\r\n[502] 526\r\n[503] 527\r\n[504] 528\r\n[505] 529\r\nbash: fork: retry: Resource temporarily unavailable\r\nbash: fork: retry: Resource temporarily unavailable\r\nbash: fork: retry: Resource temporarily unavailable\r\nbash: fork: retry: No child processes\r\n\r\netc.\r\n```\r\n\r\nFor our case, alpha version is unusable right now, running around 20 containers make them fail because they cannot create more threads.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/coreos/bugs/issues/comments/250444895","html_url":"https://github.com/coreos/bugs/issues/1281#issuecomment-250444895","issue_url":"https://api.github.com/repos/coreos/bugs/issues/1281","id":250444895,"user":{"login":"nicorevin","id":2492836,"avatar_url":"https://avatars.githubusercontent.com/u/2492836?v=3","gravatar_id":"","url":"https://api.github.com/users/nicorevin","html_url":"https://github.com/nicorevin","followers_url":"https://api.github.com/users/nicorevin/followers","following_url":"https://api.github.com/users/nicorevin/following{/other_user}","gists_url":"https://api.github.com/users/nicorevin/gists{/gist_id}","starred_url":"https://api.github.com/users/nicorevin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nicorevin/subscriptions","organizations_url":"https://api.github.com/users/nicorevin/orgs","repos_url":"https://api.github.com/users/nicorevin/repos","events_url":"https://api.github.com/users/nicorevin/events{/privacy}","received_events_url":"https://api.github.com/users/nicorevin/received_events","type":"User","site_admin":false},"created_at":"2016-09-29T11:51:17Z","updated_at":"2016-09-29T11:51:17Z","body":"looks like the limit is back with us at least in the Stable channel after updating from 1068 to 1122.2.0\r\nhttps://github.com/coreos/systemd/blob/e859aa9e993453be321450148d45d08fcc55c3f5/src/core/main.c#L88-L117\r\nCompare the above with:\r\nhttps://github.com/coreos/systemd/blob/e9fa78159bf392f12347c9d7709053ff2146e88c/src/core/main.c#L126-L127"}}
{"actor":{"display_login":"nicorevin"}}