{"action":"created","issue":{"url":"https://api.github.com/repos/zeromq/libzmq/issues/3057","repository_url":"https://api.github.com/repos/zeromq/libzmq","labels_url":"https://api.github.com/repos/zeromq/libzmq/issues/3057/labels{/name}","comments_url":"https://api.github.com/repos/zeromq/libzmq/issues/3057/comments","events_url":"https://api.github.com/repos/zeromq/libzmq/issues/3057/events","html_url":"https://github.com/zeromq/libzmq/issues/3057","id":318069465,"number":3057,"title":"Out-of-order message and slow performance when closing and reconnecting socket","user":{"login":"will133","id":1441568,"avatar_url":"https://avatars0.githubusercontent.com/u/1441568?v=4","gravatar_id":"","url":"https://api.github.com/users/will133","html_url":"https://github.com/will133","followers_url":"https://api.github.com/users/will133/followers","following_url":"https://api.github.com/users/will133/following{/other_user}","gists_url":"https://api.github.com/users/will133/gists{/gist_id}","starred_url":"https://api.github.com/users/will133/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/will133/subscriptions","organizations_url":"https://api.github.com/users/will133/orgs","repos_url":"https://api.github.com/users/will133/repos","events_url":"https://api.github.com/users/will133/events{/privacy}","received_events_url":"https://api.github.com/users/will133/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-04-26T15:01:05Z","updated_at":"2018-04-26T17:07:10Z","closed_at":null,"author_association":"NONE","body":"*Please use this template for reporting suspected bugs or requests for help.*\r\n\r\n# Issue description\r\nI have posted this on pyzmq but it may be an issue in the library itself (https://github.com/zeromq/pyzmq/issues/1171).  Essentially I have an application where it would do a connect, send a message, close.  It then can potentially reconnect, send another message, and close the socket.  I've experienced messages being dropped occasionally even with linger set to what I think is reasonable.\r\n\r\n\r\n# Environment\r\n\r\n* libzmq version (commit hash if unreleased): 4.2.5 (from conda-forge) with pyzmq 17.0.0\r\n* OS: Linux\r\n* Python: 3.6.5\r\n\r\n# Minimal test code / Steps to reproduce the issue\r\n```python\r\nimport threading\r\n\r\n_NUM_MESSAGE = 1000\r\n\r\n\r\nclass ReconnectThread(threading.Thread):\r\n    def __init__(self, ctx, server_url):\r\n        super(ReconnectThread, self).__init__()\r\n        self.ctx = ctx\r\n        self.server_url = server_url\r\n\r\n    def _get_new_socket(self):\r\n        socket = self.ctx.socket(zmq.DEALER)\r\n        socket.linger = 1000\r\n        socket.connect(self.server_url)\r\n        return socket\r\n\r\n    def run(self):\r\n        for i in range(_NUM_MESSAGE):\r\n            sock = self._get_new_socket()\r\n            sock.send_multipart([('blah_%s' % i).encode('utf8')])\r\n            sock.close()\r\n\r\n\r\nif __name__ == \"__main__\":\r\n    import zmq\r\n    ctx = zmq.Context()\r\n    try:\r\n        server_socket = ctx.socket(zmq.DEALER)\r\n        server_url = 'tcp://127.0.0.1:7384'\r\n        server_socket.bind(server_url)\r\n        thread = ReconnectThread(ctx, server_url)\r\n        thread.start()\r\n        for i in range(_NUM_MESSAGE):\r\n            print(server_socket.recv_multipart())\r\n    finally:\r\n        ctx.destroy()\r\n```\r\n\r\n# What's the actual result? (include assertion message & call stack if applicable)\r\n\r\nI found two issues with the results.  First is that it can appear out of order:\r\n\r\n```\r\n[b'blah_0']\r\n[b'blah_1']\r\n[b'blah_3']\r\n[b'blah_10']\r\n[b'blah_7']\r\n[b'blah_8']\r\n...\r\n```\r\nAccording to @minrk, this can be due to close() being async (which doesn't really close the socket), which causes multiple sockets being connected at the same time.  I think this behavior is somewhat confusing.  Is there some what to actually enforce the close() and linger to happen in a synchronous way?\r\n\r\nI'd have to emphasize that in my actual code there shouldn't be that many connections (with close() happening in parallel).  It's typically only a few, but it's just very difficult to reproduce the problem so I had to write this simple case.\r\n\r\nThe second problem, which is more serious, is that the code can hang.  It seems like the linger of 1000 (1 second) is not sufficient.  Thus messages are dropped and the main thread (recv_multipart()) can not received the same number of message.  If I adjust the linger to 5000 I can get through with no hanging, but the output sometimes can stutter for a few seconds, which I didn't expect to happen.\r\n\r\n\r\n# What's the expected result?\r\nI'd expect a more predictable close() or an alternative to make sure the linger happen before close() is returning.  I understand the linger can drop messages, but since the alternative can be hanging (when the other end point goes away), I found myself always trying to set the appropriate linger.  It does seem though with the close() behavior being unpredictable it's hard for me to set the correct value. \r\n"},"comment":{"url":"https://api.github.com/repos/zeromq/libzmq/issues/comments/384716984","html_url":"https://github.com/zeromq/libzmq/issues/3057#issuecomment-384716984","issue_url":"https://api.github.com/repos/zeromq/libzmq/issues/3057","id":384716984,"user":{"login":"bluca","id":782193,"avatar_url":"https://avatars3.githubusercontent.com/u/782193?v=4","gravatar_id":"","url":"https://api.github.com/users/bluca","html_url":"https://github.com/bluca","followers_url":"https://api.github.com/users/bluca/followers","following_url":"https://api.github.com/users/bluca/following{/other_user}","gists_url":"https://api.github.com/users/bluca/gists{/gist_id}","starred_url":"https://api.github.com/users/bluca/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bluca/subscriptions","organizations_url":"https://api.github.com/users/bluca/orgs","repos_url":"https://api.github.com/users/bluca/repos","events_url":"https://api.github.com/users/bluca/events{/privacy}","received_events_url":"https://api.github.com/users/bluca/received_events","type":"User","site_admin":false},"created_at":"2018-04-26T17:07:10Z","updated_at":"2018-04-26T17:07:10Z","author_association":"OWNER","body":"Yes, depending on the linger value - to test it, simply connect to a non-existing endpoint and close the socket.\r\nIf you don't want that, you can set the linger to unlimited."}}
{"actor":{"display_login":"bluca"}}