{"action":"created","issue":{"url":"https://api.github.com/repos/docker/docker/issues/26507","repository_url":"https://api.github.com/repos/docker/docker","labels_url":"https://api.github.com/repos/docker/docker/issues/26507/labels{/name}","comments_url":"https://api.github.com/repos/docker/docker/issues/26507/comments","events_url":"https://api.github.com/repos/docker/docker/issues/26507/events","html_url":"https://github.com/docker/docker/pull/26507","id":176474555,"number":26507,"title":"Prevent stdout / stderr race condition in limitedBuffer","user":{"login":"drakenator","id":405056,"avatar_url":"https://avatars.githubusercontent.com/u/405056?v=3","gravatar_id":"","url":"https://api.github.com/users/drakenator","html_url":"https://github.com/drakenator","followers_url":"https://api.github.com/users/drakenator/followers","following_url":"https://api.github.com/users/drakenator/following{/other_user}","gists_url":"https://api.github.com/users/drakenator/gists{/gist_id}","starred_url":"https://api.github.com/users/drakenator/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drakenator/subscriptions","organizations_url":"https://api.github.com/users/drakenator/orgs","repos_url":"https://api.github.com/users/drakenator/repos","events_url":"https://api.github.com/users/drakenator/events{/privacy}","received_events_url":"https://api.github.com/users/drakenator/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/docker/docker/labels/status/2-code-review","name":"status/2-code-review","color":"fad8c7"}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/docker/docker/milestones/58","html_url":"https://github.com/docker/docker/milestones/1.12.2","labels_url":"https://api.github.com/repos/docker/docker/milestones/58/labels","id":1973095,"number":58,"title":"1.12.2","description":"","creator":{"login":"tiborvass","id":827131,"avatar_url":"https://avatars.githubusercontent.com/u/827131?v=3","gravatar_id":"","url":"https://api.github.com/users/tiborvass","html_url":"https://github.com/tiborvass","followers_url":"https://api.github.com/users/tiborvass/followers","following_url":"https://api.github.com/users/tiborvass/following{/other_user}","gists_url":"https://api.github.com/users/tiborvass/gists{/gist_id}","starred_url":"https://api.github.com/users/tiborvass/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tiborvass/subscriptions","organizations_url":"https://api.github.com/users/tiborvass/orgs","repos_url":"https://api.github.com/users/tiborvass/repos","events_url":"https://api.github.com/users/tiborvass/events{/privacy}","received_events_url":"https://api.github.com/users/tiborvass/received_events","type":"User","site_admin":false},"open_issues":30,"closed_issues":20,"state":"open","created_at":"2016-08-30T16:26:40Z","updated_at":"2016-09-14T22:53:17Z","due_on":null,"closed_at":null},"comments":4,"created_at":"2016-09-12T20:22:46Z","updated_at":"2016-09-15T08:00:46Z","closed_at":"2016-09-12T22:57:55Z","pull_request":{"url":"https://api.github.com/repos/docker/docker/pulls/26507","html_url":"https://github.com/docker/docker/pull/26507","diff_url":"https://github.com/docker/docker/pull/26507.diff","patch_url":"https://github.com/docker/docker/pull/26507.patch"},"body":"There is a possibility of a race condition in the stdout / stderr stream capture processing for container health checks. Output from the stdout and stderr streams are collected into the same bytes.Buffer via limitedBuffer, however bytes.Buffer is not thread-safe.\r\n\r\nThis change, which Fixes #26384, introduces a mutex in limitedBuffer to synchronize writes.\r\n\r\n### Steps to reproduce\r\n\r\nWhen run as a HEALTHCHECK, the following example program will spontaneously cause a panic.\r\n\r\nstdio.go:\r\n\r\n```\r\npackage main\r\n\r\nimport (\r\n\t\"fmt\"\r\n\t\"io\"\r\n\t\"os\"\r\n\t\"strings\"\r\n\t\"sync\"\r\n)\r\n\r\nfunc main() {\r\n\tvar doneWait sync.WaitGroup\r\n\tvar readyWait sync.WaitGroup\r\n\tvar barrierWait sync.WaitGroup\r\n\r\n\tbarrierWait.Add(1)\r\n\r\n\tout := func(target io.Writer, str string) {\r\n\t\treadyWait.Done()\r\n\r\n\t\t// Collect all goroutines here so stdio/err is written to as close to the same time as possible\r\n\t\tbarrierWait.Wait()\r\n\r\n\t\tfmt.Fprintf(target, \"%s\\n\", str)\r\n\t\tdoneWait.Done()\r\n\t}\r\n\r\n\tfor i := 0; i < 16; i++ {\r\n\t\treadyWait.Add(2)\r\n\t\tdoneWait.Add(2)\r\n\r\n\t\tgo out(os.Stdout, strings.Repeat(\".\", 40))\r\n\t\tgo out(os.Stderr, strings.Repeat(\"!\", 40))\r\n\t}\r\n\r\n\treadyWait.Wait()\r\n\tbarrierWait.Done()\r\n\tdoneWait.Wait()\r\n}\r\n\r\n```\r\n\r\nAn example of the panic, This is similar to those mentioned in #26384.\r\n\r\n```\r\npanic: runtime error: slice bounds out of range\r\n\r\ngoroutine 584795 [running]:\r\npanic(0x1b1e1c0, 0xc82000e070)\r\n        /usr/local/go/src/runtime/panic.go:481 +0x3e6\r\nbytes.(*Buffer).grow(0xc823e45d00, 0x11f, 0xc821ae4db0)\r\n        /usr/local/go/src/bytes/buffer.go:112 +0x141\r\nbytes.(*Buffer).Write(0xc823e45d00, 0xc823816000, 0x11f, 0x8000, 0x5f, 0x0, 0x0)\r\n        /usr/local/go/src/bytes/buffer.go:134 +0x4b\r\ngithub.com/docker/docker/daemon.(*limitedBuffer).Write(0xc823e45d00, 0xc823816000, 0x11f, 0x8000, 0x11f, 0x0, 0x0)\r\n        /go/src/github.com/docker/docker/daemon/health.go:288 +0xae\r\nio.copyBuffer(0x7f6598480a48, 0xc823e45d00, 0x7f659852f710, 0xc821925840, 0xc823816000, 0x8000, 0x8000, 0x0, 0x0, 0x0)\r\n        /usr/local/go/src/io/io.go:382 +0x2c9\r\nio.Copy(0x7f6598480a48, 0xc823e45d00, 0x7f659852f710, 0xc821925840, 0xc821925840, 0x0, 0x0)\r\n        /usr/local/go/src/io/io.go:350 +0x64\r\ngithub.com/docker/docker/container.AttachStreams.func2(0x1e03aa0, 0x6, 0x7f6598480a48, 0xc823e45d00, 0x7f659852f6e0, 0xc821925840)\r\n        /go/src/github.com/docker/docker/container/container.go:433 +0x1cc\r\ncreated by github.com/docker/docker/container.AttachStreams\r\n        /go/src/github.com/docker/docker/container/container.go:450 +0x37e\r\n```\r\n\r\nAs this is intermittent, it may take a while before the panic occurs, running more containers (e.g. scaling the service to 100) improves the chances of the issue occurring.\r\n\r\nDockerfile used for testing, uses stdio.go from above.\r\n```\r\nFROM busybox:latest\r\n\r\nCOPY stdio /stdio\r\nHEALTHCHECK --interval=1s CMD /stdio\r\n\r\nENTRYPOINT [\"busybox\"]\r\nCMD [\"sleep\", \"99999999\"]\r\n\r\n```\r\n\r\nSigned-off-by: Stephen Drake <stephen@xenolith.net>"},"comment":{"url":"https://api.github.com/repos/docker/docker/issues/comments/247262658","html_url":"https://github.com/docker/docker/pull/26507#issuecomment-247262658","issue_url":"https://api.github.com/repos/docker/docker/issues/26507","id":247262658,"user":{"login":"drakenator","id":405056,"avatar_url":"https://avatars.githubusercontent.com/u/405056?v=3","gravatar_id":"","url":"https://api.github.com/users/drakenator","html_url":"https://github.com/drakenator","followers_url":"https://api.github.com/users/drakenator/followers","following_url":"https://api.github.com/users/drakenator/following{/other_user}","gists_url":"https://api.github.com/users/drakenator/gists{/gist_id}","starred_url":"https://api.github.com/users/drakenator/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drakenator/subscriptions","organizations_url":"https://api.github.com/users/drakenator/orgs","repos_url":"https://api.github.com/users/drakenator/repos","events_url":"https://api.github.com/users/drakenator/events{/privacy}","received_events_url":"https://api.github.com/users/drakenator/received_events","type":"User","site_admin":false},"created_at":"2016-09-15T08:00:46Z","updated_at":"2016-09-15T08:00:46Z","body":"@tonistiigi Will post to `master` shortly."}}
{"actor":{"display_login":"drakenator"}}