{"action":"created","issue":{"url":"https://api.github.com/repos/dabeaz/curio/issues/106","repository_url":"https://api.github.com/repos/dabeaz/curio","labels_url":"https://api.github.com/repos/dabeaz/curio/issues/106/labels{/name}","comments_url":"https://api.github.com/repos/dabeaz/curio/issues/106/comments","events_url":"https://api.github.com/repos/dabeaz/curio/issues/106/events","html_url":"https://github.com/dabeaz/curio/issues/106","id":188957793,"number":106,"title":"The eager read/write optimization leaves curio programs highly susceptible to event loop starvation","user":{"login":"njsmith","id":609896,"avatar_url":"https://avatars.githubusercontent.com/u/609896?v=3","gravatar_id":"","url":"https://api.github.com/users/njsmith","html_url":"https://github.com/njsmith","followers_url":"https://api.github.com/users/njsmith/followers","following_url":"https://api.github.com/users/njsmith/following{/other_user}","gists_url":"https://api.github.com/users/njsmith/gists{/gist_id}","starred_url":"https://api.github.com/users/njsmith/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/njsmith/subscriptions","organizations_url":"https://api.github.com/users/njsmith/orgs","repos_url":"https://api.github.com/users/njsmith/repos","events_url":"https://api.github.com/users/njsmith/events{/privacy}","received_events_url":"https://api.github.com/users/njsmith/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-11-13T07:55:31Z","updated_at":"2016-11-13T22:49:30Z","closed_at":null,"body":"There was some discussion in #83 about removing the \"eager write\" optimization that curio does, where `await sock.send` will attempt to synchronously write to the socket directly, and only yield to the event loop if that fails. It turned out that the original reason we were considering removing it didn't actually apply, plus it significantly slows down the echo server microbenchmark, so for now the optimization survived.\r\n\r\nBut... I've just realized that actually the eager read/write optimizations create a serious correctness issue, in that they can cause ordinary-looking code to starve out the event loop in realistic situations.\r\n\r\nHere's a [standard curio echo server](https://gist.github.com/njsmith/d6bb8d7fb517f6248b9fb940294d63bf), along with a particularly aggressive echo client. The test case is thta we run two copies of the client, and watch to see how well the server switches between servicing both of them. The server is instrumented to report how often it switches.\r\n\r\nThe server output pretty regularly contains lines like:\r\n```\r\nSwitched after 220 loops (100.7 ms)\r\n```\r\nWhat this is saying is that one of the `echo_handler`s managed to complete 220 passes through its read/write loop before yielding to the kernel, so all other tasks were starved out for 100.7 ms.\r\n\r\nI even saw a\r\n```\r\nSwitched after 340 loops (171.6 ms)\r\n```\r\nwithout trying particularly hard. And in theory the only limit on how high those numbers can go is that at some point some scheduling quirk or something on my laptop causes one of the clients to be slow to read/write and makes the server block; under the right conditions it could go arbitrarily high.\r\n\r\n@glyph has some [interesting comments on this optimization in Twisted](https://github.com/python/asyncio/issues/446#issuecomment-256498678), and how they removed it.\r\n\r\nThere's also an important difference between curio and twisted: in twisted, reads are *only* triggered by callbacks, so they always go through the event loop; it's only writes where you can even contemplate doing the eager-write optimization. So for them, this starvation problem is less likely to happen, because you can only hit it if you're doing an endless series of writes without any reads in between (and as glyph says, \"/dev/urandom as a service has pretty limited utility\"). In curio, we currently have *both* eager-read and eager-write optimizations, which makes the problem much easier to hit -- all kinds of realistic protocols are susceptible, starting with echo servers and proxies and moving on to things like an HTTP/1.1 server talking to a pipelining client.\r\n\r\nI'm particularly worried about this because as you know I've been poking at HTTP server support for curio, and it's completely unacceptable for a fast client to totally lock up the server for unbounded amounts of time, even if it's a low-probability event :-(. (I.e., I don't think this is something where tweaking some heuristics will be enough; we need to able to actually put a hard bound on how long a task can monopolize the kernel.) It can be worked around by manually inserting calls to `curio.sleep(0)` (now that [that's fixed](https://github.com/dabeaz/curio/commit/b144a5d1ae6e0cdaf8f8b7c67fb44bcf82912bfa)), but that's not at all a nice solution in general.\r\n\r\nThe obvious solution is to make sure to always unschedule-then-reschedule on reads and writes. I can't immediately think of any cleverer (= faster) solution that is guaranteed to actually work."},"comment":{"url":"https://api.github.com/repos/dabeaz/curio/issues/comments/260219051","html_url":"https://github.com/dabeaz/curio/issues/106#issuecomment-260219051","issue_url":"https://api.github.com/repos/dabeaz/curio/issues/106","id":260219051,"user":{"login":"dabeaz","id":350836,"avatar_url":"https://avatars.githubusercontent.com/u/350836?v=3","gravatar_id":"","url":"https://api.github.com/users/dabeaz","html_url":"https://github.com/dabeaz","followers_url":"https://api.github.com/users/dabeaz/followers","following_url":"https://api.github.com/users/dabeaz/following{/other_user}","gists_url":"https://api.github.com/users/dabeaz/gists{/gist_id}","starred_url":"https://api.github.com/users/dabeaz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dabeaz/subscriptions","organizations_url":"https://api.github.com/users/dabeaz/orgs","repos_url":"https://api.github.com/users/dabeaz/repos","events_url":"https://api.github.com/users/dabeaz/events{/privacy}","received_events_url":"https://api.github.com/users/dabeaz/received_events","type":"User","site_admin":false},"created_at":"2016-11-13T22:49:30Z","updated_at":"2016-11-13T22:49:30Z","body":"I understand the theory of it and agree that it might be a problem, but I'm not yet convinced that this would actually be a common situation--it seems that it would require the task in question to be pegged at 100% CPU load continuously and to never block *ever*.   Given all the variances on a typical network, it seems like this would be pretty hard to achieve except in carefully controlled tests.  And even if that did happen, it seems like you'd have a much more serious problem on your hands related to the scaling of whatever thing you were running. \r\n\r\nI could be wrong though.  I just don't know.   How are other frameworks dealing with this?   How do programs that use threads deal with this?     \r\n\r\nAll things equal, I don't want to completely discount the performance overhead of constantly doing everything on the event loop by doing away with eager read/write.    I could possibly envision some kind of hybrid approach that allows a certain small number of eager reads to take place first before blocking on the event loop.   That might allow more normal workloads to take advantage of eager read/write while forcing the extreme case to more properly schedule itself. \r\n"}}
{"actor":{"display_login":"dabeaz"}}