{"action":"created","issue":{"url":"https://api.github.com/repos/Yelp/yelp_clog/issues/23","repository_url":"https://api.github.com/repos/Yelp/yelp_clog","labels_url":"https://api.github.com/repos/Yelp/yelp_clog/issues/23/labels{/name}","comments_url":"https://api.github.com/repos/Yelp/yelp_clog/issues/23/comments","events_url":"https://api.github.com/repos/Yelp/yelp_clog/issues/23/events","html_url":"https://github.com/Yelp/yelp_clog/pull/23","id":153091011,"number":23,"title":"Make ScribeLogger fork-safe","user":{"login":"tiras-j","id":6816898,"avatar_url":"https://avatars.githubusercontent.com/u/6816898?v=3","gravatar_id":"","url":"https://api.github.com/users/tiras-j","html_url":"https://github.com/tiras-j","followers_url":"https://api.github.com/users/tiras-j/followers","following_url":"https://api.github.com/users/tiras-j/following{/other_user}","gists_url":"https://api.github.com/users/tiras-j/gists{/gist_id}","starred_url":"https://api.github.com/users/tiras-j/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tiras-j/subscriptions","organizations_url":"https://api.github.com/users/tiras-j/orgs","repos_url":"https://api.github.com/users/tiras-j/repos","events_url":"https://api.github.com/users/tiras-j/events{/privacy}","received_events_url":"https://api.github.com/users/tiras-j/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-05-04T19:07:33Z","updated_at":"2016-05-04T21:45:38Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/Yelp/yelp_clog/pulls/23","html_url":"https://github.com/Yelp/yelp_clog/pull/23","diff_url":"https://github.com/Yelp/yelp_clog/pull/23.diff","patch_url":"https://github.com/Yelp/yelp_clog/pull/23.patch"},"body":"We've had to work around forking issues with scribelogger quite a few times and I've wondering if we could simply make the scribelogger forksafe. It looks like the only issue is that after a fork the socket fd is shared and there's potentials for write collisions between the children processes, so we should be able to simply close and reopen the socket in order to bind and different ephemeral port and thus avoid the issue.\r\n\r\nIt's very important that you don't try forking after creating multiple threads within the python interpreter, unless you can be sure that you own the RLock on the ScribeLogger. This is because only the current thread gets forked and if the RLock is in an acquired state it will never be released for the forked process and they will block forever on the next attempt to acquire the lock.\r\n\r\nLastly, I can't think of a good way to test this with pytest (I'm not sure if it can handle tests with forking?) but I threw together a simple model of the ScribeLogger behavior in order to functionally test this, see the attached file (had to change it to .txt to satisfy github). If anyone has ideas for adding some unit or integration tests I'd be happy to put it together.\r\n\r\n[fork_socket_logger_test.txt](https://github.com/Yelp/yelp_clog/files/249408/fork_socket_logger_test.txt)"},"comment":{"url":"https://api.github.com/repos/Yelp/yelp_clog/issues/comments/217013396","html_url":"https://github.com/Yelp/yelp_clog/pull/23#issuecomment-217013396","issue_url":"https://api.github.com/repos/Yelp/yelp_clog/issues/23","id":217013396,"user":{"login":"jolynch","id":1420460,"avatar_url":"https://avatars.githubusercontent.com/u/1420460?v=3","gravatar_id":"","url":"https://api.github.com/users/jolynch","html_url":"https://github.com/jolynch","followers_url":"https://api.github.com/users/jolynch/followers","following_url":"https://api.github.com/users/jolynch/following{/other_user}","gists_url":"https://api.github.com/users/jolynch/gists{/gist_id}","starred_url":"https://api.github.com/users/jolynch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jolynch/subscriptions","organizations_url":"https://api.github.com/users/jolynch/orgs","repos_url":"https://api.github.com/users/jolynch/repos","events_url":"https://api.github.com/users/jolynch/events{/privacy}","received_events_url":"https://api.github.com/users/jolynch/received_events","type":"User","site_admin":false},"created_at":"2016-05-04T21:45:38Z","updated_at":"2016-05-04T21:45:38Z","body":"@tiras-j Reproing what we talked about in person\r\n\r\nI think we might be solving the wrong problem here. Doing absolutely anything before a fork in python is a _bad plan_, and scribe logger has often been the \"canary\" that let's folk know that they're breaking this golden rule of python. \r\n\r\nMy favorite \"nothing is actually fork safe in python\" bugs:\r\n- [Python 6721](http://bugs.python.org/issue6721); If you end up with even a single single thread before a fork (3rd party libraries create these all the time) that initializes logging libraries and then fork; there is a race condition that can lead to total lockup.\r\n- [Python 13697](http://bugs.python.org/issue13697); Not strictly related to forking, but it's the same kind of thing where RLock is just simply broken"}}