{"action":"created","comment":{"url":"https://api.github.com/repos/moby/moby/pulls/comments/238905240","pull_request_review_id":181560557,"id":238905240,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIzODkwNTI0MA==","diff_hunk":"@@ -207,68 +216,88 @@ func (daemon *Daemon) restore() error {\n \t\treturn err\n \t}\n \n+\tloadWg := syncext.NewLimitedWaitGroup(parallelStartupJobs)","path":"daemon/daemon.go","position":34,"original_position":34,"commit_id":"4ff66cefa9aed645576d531544ddd94fd36864bc","original_commit_id":"4ff66cefa9aed645576d531544ddd94fd36864bc","user":{"login":"cyphar","id":2888411,"node_id":"MDQ6VXNlcjI4ODg0MTE=","avatar_url":"https://avatars1.githubusercontent.com/u/2888411?v=4","gravatar_id":"","url":"https://api.github.com/users/cyphar","html_url":"https://github.com/cyphar","followers_url":"https://api.github.com/users/cyphar/followers","following_url":"https://api.github.com/users/cyphar/following{/other_user}","gists_url":"https://api.github.com/users/cyphar/gists{/gist_id}","starred_url":"https://api.github.com/users/cyphar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cyphar/subscriptions","organizations_url":"https://api.github.com/users/cyphar/orgs","repos_url":"https://api.github.com/users/cyphar/repos","events_url":"https://api.github.com/users/cyphar/events{/privacy}","received_events_url":"https://api.github.com/users/cyphar/received_events","type":"User","site_admin":false},"body":"Sure, though note that this will change\r\n\r\n```go\r\n\tloadWg := syncext.NewLimitedWaitGroup(parallelStartupJobs)\r\n\tfor _, v := range dir {\r\n\t\tloadWg.Add()\r\n\t\tgo func(id string) {\r\n\t\t\tloadWg.Up()\r\n\t\t\tdefer loadWg.Down()\r\n\t\t\t// ...\r\n\t\t}(v.Name())\r\n\t}\r\n\tloadWg.Wait()\r\n```\r\n\r\nto\r\n\r\n```go\r\n\tvar loadWg sync.WaitGroup\r\n\tloadSem := make(chan struct{}, parallelStartupJobs)\r\n\tfor _, v := range dir {\r\n\t\tloadWg.Add(1)\r\n\t\tgo func(id string) {\r\n\t\t\tloadSem <- struct{}{}\r\n\t\t\tdefer func() {\r\n\t\t\t\t<-loadSem\r\n\t\t\t\tloadWg.Done()\r\n\t\t\t}()\r\n\t\t\t// ...\r\n\t\t}(v.Name())\r\n\t}\r\n\tloadWg.Wait()\r\n```\r\n\r\nWhich is the main reason I combined them (the latter seems far easier to mess up on copy-paste and would be repeated quite a bit). But sure, I can do the conversion if you prefer.","created_at":"2018-12-05T01:41:53Z","updated_at":"2018-12-05T01:41:54Z","html_url":"https://github.com/moby/moby/pull/38301#discussion_r238905240","pull_request_url":"https://api.github.com/repos/moby/moby/pulls/38301","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/moby/moby/pulls/comments/238905240"},"html":{"href":"https://github.com/moby/moby/pull/38301#discussion_r238905240"},"pull_request":{"href":"https://api.github.com/repos/moby/moby/pulls/38301"}},"in_reply_to_id":238883114},"pull_request":{"url":"https://api.github.com/repos/moby/moby/pulls/38301","id":235020503,"node_id":"MDExOlB1bGxSZXF1ZXN0MjM1MDIwNTAz","html_url":"https://github.com/moby/moby/pull/38301","diff_url":"https://github.com/moby/moby/pull/38301.diff","patch_url":"https://github.com/moby/moby/pull/38301.patch","issue_url":"https://api.github.com/repos/moby/moby/issues/38301","number":38301,"state":"open","locked":false,"title":" daemon: switch to Semaphore for startup tasks","user":{"login":"cyphar","id":2888411,"node_id":"MDQ6VXNlcjI4ODg0MTE=","avatar_url":"https://avatars1.githubusercontent.com/u/2888411?v=4","gravatar_id":"","url":"https://api.github.com/users/cyphar","html_url":"https://github.com/cyphar","followers_url":"https://api.github.com/users/cyphar/followers","following_url":"https://api.github.com/users/cyphar/following{/other_user}","gists_url":"https://api.github.com/users/cyphar/gists{/gist_id}","starred_url":"https://api.github.com/users/cyphar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cyphar/subscriptions","organizations_url":"https://api.github.com/users/cyphar/orgs","repos_url":"https://api.github.com/users/cyphar/repos","events_url":"https://api.github.com/users/cyphar/events{/privacy}","received_events_url":"https://api.github.com/users/cyphar/received_events","type":"User","site_admin":false},"body":"Many startup tasks have to run for each container, and thus using a\r\nWaitGroup (which doesn't have a limit to the number of parallel tasks)\r\ncan result in Docker exceeding the NOFILE limit quite trivially. A more\r\noptimal solution is to have a parallelism limit.\r\n\r\nIn addition, several startup tasks were not parallelised previously\r\nwhich resulted in very long startup times. According to my testing, 20K\r\ndead containers resulted in ~6 minute startup times (during which time\r\nDocker is completely unusable).\r\n\r\nThis patch fixes both issues, and the parallelStartupTimes factor chosen\r\n(128 * NumCPU) is based on my own significant testing of the 20K\r\ncontainer case. This patch (on my machines) reduces the startup time\r\nfrom 6 minutes to less than a minute (ideally this could be further\r\nreduced by removing the need to scan all dead containers on startup --\r\nbut that's beyond the scope of this patchset).\r\n\r\nIn order to avoid the NOFILE limit problem, we also detect this\r\non-startup and if NOFILE < 2*128*NumCPU we will reduce the parallelism\r\nfactor to avoid hitting NOFILE limits (but also emit a warning since\r\nthis is almost certainly a mis-configuration).\r\n\r\n![benchmarks](https://user-images.githubusercontent.com/2888411/49456934-04fab480-f83e-11e8-969d-69f85873bad4.png)\r\n\r\n###### Benchmarks were done on an 8-core machine.\r\n\r\n[![cat by Danny VB](https://farm4.staticflickr.com/3488/4051998735_5b4863ac11.jpg)](https://flic.kr/p/7b4y4a)\r\n\r\n###### [cat](https://flic.kr/p/7b4y4a) by [Danny VB](https://www.flickr.com/photos/parismadrid/)\r\n\r\nSigned-off-by: Aleksa Sarai <asarai@suse.de>","created_at":"2018-11-30T15:18:16Z","updated_at":"2018-12-05T01:41:54Z","closed_at":null,"merged_at":null,"merge_commit_sha":"6bc46610eba463984efd88a4c045eb3cd55773f4","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":175375407,"node_id":"MDU6TGFiZWwxNzUzNzU0MDc=","url":"https://api.github.com/repos/moby/moby/labels/status/2-code-review","name":"status/2-code-review","color":"fad8c7","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/moby/moby/pulls/38301/commits","review_comments_url":"https://api.github.com/repos/moby/moby/pulls/38301/comments","review_comment_url":"https://api.github.com/repos/moby/moby/pulls/comments{/number}","comments_url":"https://api.github.com/repos/moby/moby/issues/38301/comments","statuses_url":"https://api.github.com/repos/moby/moby/statuses/4ff66cefa9aed645576d531544ddd94fd36864bc","head":{"label":"cyphar:waitgroup-limits","ref":"waitgroup-limits","sha":"4ff66cefa9aed645576d531544ddd94fd36864bc","user":{"login":"cyphar","id":2888411,"node_id":"MDQ6VXNlcjI4ODg0MTE=","avatar_url":"https://avatars1.githubusercontent.com/u/2888411?v=4","gravatar_id":"","url":"https://api.github.com/users/cyphar","html_url":"https://github.com/cyphar","followers_url":"https://api.github.com/users/cyphar/followers","following_url":"https://api.github.com/users/cyphar/following{/other_user}","gists_url":"https://api.github.com/users/cyphar/gists{/gist_id}","starred_url":"https://api.github.com/users/cyphar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cyphar/subscriptions","organizations_url":"https://api.github.com/users/cyphar/orgs","repos_url":"https://api.github.com/users/cyphar/repos","events_url":"https://api.github.com/users/cyphar/events{/privacy}","received_events_url":"https://api.github.com/users/cyphar/received_events","type":"User","site_admin":false},"repo":{"id":19424861,"node_id":"MDEwOlJlcG9zaXRvcnkxOTQyNDg2MQ==","name":"docker","full_name":"cyphar/docker","private":false,"owner":{"login":"cyphar","id":2888411,"node_id":"MDQ6VXNlcjI4ODg0MTE=","avatar_url":"https://avatars1.githubusercontent.com/u/2888411?v=4","gravatar_id":"","url":"https://api.github.com/users/cyphar","html_url":"https://github.com/cyphar","followers_url":"https://api.github.com/users/cyphar/followers","following_url":"https://api.github.com/users/cyphar/following{/other_user}","gists_url":"https://api.github.com/users/cyphar/gists{/gist_id}","starred_url":"https://api.github.com/users/cyphar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cyphar/subscriptions","organizations_url":"https://api.github.com/users/cyphar/orgs","repos_url":"https://api.github.com/users/cyphar/repos","events_url":"https://api.github.com/users/cyphar/events{/privacy}","received_events_url":"https://api.github.com/users/cyphar/received_events","type":"User","site_admin":false},"html_url":"https://github.com/cyphar/docker","description":"Docker - the open-source application container engine","fork":true,"url":"https://api.github.com/repos/cyphar/docker","forks_url":"https://api.github.com/repos/cyphar/docker/forks","keys_url":"https://api.github.com/repos/cyphar/docker/keys{/key_id}","collaborators_url":"https://api.github.com/repos/cyphar/docker/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/cyphar/docker/teams","hooks_url":"https://api.github.com/repos/cyphar/docker/hooks","issue_events_url":"https://api.github.com/repos/cyphar/docker/issues/events{/number}","events_url":"https://api.github.com/repos/cyphar/docker/events","assignees_url":"https://api.github.com/repos/cyphar/docker/assignees{/user}","branches_url":"https://api.github.com/repos/cyphar/docker/branches{/branch}","tags_url":"https://api.github.com/repos/cyphar/docker/tags","blobs_url":"https://api.github.com/repos/cyphar/docker/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/cyphar/docker/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/cyphar/docker/git/refs{/sha}","trees_url":"https://api.github.com/repos/cyphar/docker/git/trees{/sha}","statuses_url":"https://api.github.com/repos/cyphar/docker/statuses/{sha}","languages_url":"https://api.github.com/repos/cyphar/docker/languages","stargazers_url":"https://api.github.com/repos/cyphar/docker/stargazers","contributors_url":"https://api.github.com/repos/cyphar/docker/contributors","subscribers_url":"https://api.github.com/repos/cyphar/docker/subscribers","subscription_url":"https://api.github.com/repos/cyphar/docker/subscription","commits_url":"https://api.github.com/repos/cyphar/docker/commits{/sha}","git_commits_url":"https://api.github.com/repos/cyphar/docker/git/commits{/sha}","comments_url":"https://api.github.com/repos/cyphar/docker/comments{/number}","issue_comment_url":"https://api.github.com/repos/cyphar/docker/issues/comments{/number}","contents_url":"https://api.github.com/repos/cyphar/docker/contents/{+path}","compare_url":"https://api.github.com/repos/cyphar/docker/compare/{base}...{head}","merges_url":"https://api.github.com/repos/cyphar/docker/merges","archive_url":"https://api.github.com/repos/cyphar/docker/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/cyphar/docker/downloads","issues_url":"https://api.github.com/repos/cyphar/docker/issues{/number}","pulls_url":"https://api.github.com/repos/cyphar/docker/pulls{/number}","milestones_url":"https://api.github.com/repos/cyphar/docker/milestones{/number}","notifications_url":"https://api.github.com/repos/cyphar/docker/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/cyphar/docker/labels{/name}","releases_url":"https://api.github.com/repos/cyphar/docker/releases{/id}","deployments_url":"https://api.github.com/repos/cyphar/docker/deployments","created_at":"2014-05-04T11:14:30Z","updated_at":"2016-01-15T12:08:46Z","pushed_at":"2018-12-04T17:38:24Z","git_url":"git://github.com/cyphar/docker.git","ssh_url":"git@github.com:cyphar/docker.git","clone_url":"https://github.com/cyphar/docker.git","svn_url":"https://github.com/cyphar/docker","homepage":"http://www.docker.io","size":135111,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":1,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"moby:master","ref":"master","sha":"65d9a5dde589f96f4728782fd0fa7de7bb6b3146","user":{"login":"moby","id":27259197,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI3MjU5MTk3","avatar_url":"https://avatars1.githubusercontent.com/u/27259197?v=4","gravatar_id":"","url":"https://api.github.com/users/moby","html_url":"https://github.com/moby","followers_url":"https://api.github.com/users/moby/followers","following_url":"https://api.github.com/users/moby/following{/other_user}","gists_url":"https://api.github.com/users/moby/gists{/gist_id}","starred_url":"https://api.github.com/users/moby/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/moby/subscriptions","organizations_url":"https://api.github.com/users/moby/orgs","repos_url":"https://api.github.com/users/moby/repos","events_url":"https://api.github.com/users/moby/events{/privacy}","received_events_url":"https://api.github.com/users/moby/received_events","type":"Organization","site_admin":false},"repo":{"id":7691631,"node_id":"MDEwOlJlcG9zaXRvcnk3NjkxNjMx","name":"moby","full_name":"moby/moby","private":false,"owner":{"login":"moby","id":27259197,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI3MjU5MTk3","avatar_url":"https://avatars1.githubusercontent.com/u/27259197?v=4","gravatar_id":"","url":"https://api.github.com/users/moby","html_url":"https://github.com/moby","followers_url":"https://api.github.com/users/moby/followers","following_url":"https://api.github.com/users/moby/following{/other_user}","gists_url":"https://api.github.com/users/moby/gists{/gist_id}","starred_url":"https://api.github.com/users/moby/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/moby/subscriptions","organizations_url":"https://api.github.com/users/moby/orgs","repos_url":"https://api.github.com/users/moby/repos","events_url":"https://api.github.com/users/moby/events{/privacy}","received_events_url":"https://api.github.com/users/moby/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/moby/moby","description":"Moby Project - a collaborative project for the container ecosystem to assemble container-based systems","fork":false,"url":"https://api.github.com/repos/moby/moby","forks_url":"https://api.github.com/repos/moby/moby/forks","keys_url":"https://api.github.com/repos/moby/moby/keys{/key_id}","collaborators_url":"https://api.github.com/repos/moby/moby/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/moby/moby/teams","hooks_url":"https://api.github.com/repos/moby/moby/hooks","issue_events_url":"https://api.github.com/repos/moby/moby/issues/events{/number}","events_url":"https://api.github.com/repos/moby/moby/events","assignees_url":"https://api.github.com/repos/moby/moby/assignees{/user}","branches_url":"https://api.github.com/repos/moby/moby/branches{/branch}","tags_url":"https://api.github.com/repos/moby/moby/tags","blobs_url":"https://api.github.com/repos/moby/moby/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/moby/moby/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/moby/moby/git/refs{/sha}","trees_url":"https://api.github.com/repos/moby/moby/git/trees{/sha}","statuses_url":"https://api.github.com/repos/moby/moby/statuses/{sha}","languages_url":"https://api.github.com/repos/moby/moby/languages","stargazers_url":"https://api.github.com/repos/moby/moby/stargazers","contributors_url":"https://api.github.com/repos/moby/moby/contributors","subscribers_url":"https://api.github.com/repos/moby/moby/subscribers","subscription_url":"https://api.github.com/repos/moby/moby/subscription","commits_url":"https://api.github.com/repos/moby/moby/commits{/sha}","git_commits_url":"https://api.github.com/repos/moby/moby/git/commits{/sha}","comments_url":"https://api.github.com/repos/moby/moby/comments{/number}","issue_comment_url":"https://api.github.com/repos/moby/moby/issues/comments{/number}","contents_url":"https://api.github.com/repos/moby/moby/contents/{+path}","compare_url":"https://api.github.com/repos/moby/moby/compare/{base}...{head}","merges_url":"https://api.github.com/repos/moby/moby/merges","archive_url":"https://api.github.com/repos/moby/moby/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/moby/moby/downloads","issues_url":"https://api.github.com/repos/moby/moby/issues{/number}","pulls_url":"https://api.github.com/repos/moby/moby/pulls{/number}","milestones_url":"https://api.github.com/repos/moby/moby/milestones{/number}","notifications_url":"https://api.github.com/repos/moby/moby/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/moby/moby/labels{/name}","releases_url":"https://api.github.com/repos/moby/moby/releases{/id}","deployments_url":"https://api.github.com/repos/moby/moby/deployments","created_at":"2013-01-18T18:10:57Z","updated_at":"2018-12-05T01:00:04Z","pushed_at":"2018-12-05T00:58:56Z","git_url":"git://github.com/moby/moby.git","ssh_url":"git@github.com:moby/moby.git","clone_url":"https://github.com/moby/moby.git","svn_url":"https://github.com/moby/moby","homepage":"https://mobyproject.org/","size":140410,"stargazers_count":51374,"watchers_count":51374,"language":"Go","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":14934,"mirror_url":null,"archived":false,"open_issues_count":3498,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":14934,"open_issues":3498,"watchers":51374,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/moby/moby/pulls/38301"},"html":{"href":"https://github.com/moby/moby/pull/38301"},"issue":{"href":"https://api.github.com/repos/moby/moby/issues/38301"},"comments":{"href":"https://api.github.com/repos/moby/moby/issues/38301/comments"},"review_comments":{"href":"https://api.github.com/repos/moby/moby/pulls/38301/comments"},"review_comment":{"href":"https://api.github.com/repos/moby/moby/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/moby/moby/pulls/38301/commits"},"statuses":{"href":"https://api.github.com/repos/moby/moby/statuses/4ff66cefa9aed645576d531544ddd94fd36864bc"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"cyphar"}}