{"action":"created","issue":{"url":"https://api.github.com/repos/Parsely/pykafka/issues/471","repository_url":"https://api.github.com/repos/Parsely/pykafka","labels_url":"https://api.github.com/repos/Parsely/pykafka/issues/471/labels{/name}","comments_url":"https://api.github.com/repos/Parsely/pykafka/issues/471/comments","events_url":"https://api.github.com/repos/Parsely/pykafka/issues/471/events","html_url":"https://github.com/Parsely/pykafka/pull/471","id":136186471,"number":471,"title":"__del__ should only stop consumer if it is running","user":{"login":"suicidejack","id":1060658,"avatar_url":"https://avatars.githubusercontent.com/u/1060658?v=3","gravatar_id":"","url":"https://api.github.com/users/suicidejack","html_url":"https://github.com/suicidejack","followers_url":"https://api.github.com/users/suicidejack/followers","following_url":"https://api.github.com/users/suicidejack/following{/other_user}","gists_url":"https://api.github.com/users/suicidejack/gists{/gist_id}","starred_url":"https://api.github.com/users/suicidejack/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/suicidejack/subscriptions","organizations_url":"https://api.github.com/users/suicidejack/orgs","repos_url":"https://api.github.com/users/suicidejack/repos","events_url":"https://api.github.com/users/suicidejack/events{/privacy}","received_events_url":"https://api.github.com/users/suicidejack/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Parsely/pykafka/labels/backlog","name":"backlog","color":"009800"}],"state":"open","locked":false,"assignee":{"login":"emmett9001","id":723615,"avatar_url":"https://avatars.githubusercontent.com/u/723615?v=3","gravatar_id":"","url":"https://api.github.com/users/emmett9001","html_url":"https://github.com/emmett9001","followers_url":"https://api.github.com/users/emmett9001/followers","following_url":"https://api.github.com/users/emmett9001/following{/other_user}","gists_url":"https://api.github.com/users/emmett9001/gists{/gist_id}","starred_url":"https://api.github.com/users/emmett9001/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/emmett9001/subscriptions","organizations_url":"https://api.github.com/users/emmett9001/orgs","repos_url":"https://api.github.com/users/emmett9001/repos","events_url":"https://api.github.com/users/emmett9001/events{/privacy}","received_events_url":"https://api.github.com/users/emmett9001/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/Parsely/pykafka/milestones/16","html_url":"https://github.com/Parsely/pykafka/milestones/2.3.0","labels_url":"https://api.github.com/repos/Parsely/pykafka/milestones/16/labels","id":1567373,"number":16,"title":"2.3.0","description":"","creator":{"login":"emmett9001","id":723615,"avatar_url":"https://avatars.githubusercontent.com/u/723615?v=3","gravatar_id":"","url":"https://api.github.com/users/emmett9001","html_url":"https://github.com/emmett9001","followers_url":"https://api.github.com/users/emmett9001/followers","following_url":"https://api.github.com/users/emmett9001/following{/other_user}","gists_url":"https://api.github.com/users/emmett9001/gists{/gist_id}","starred_url":"https://api.github.com/users/emmett9001/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/emmett9001/subscriptions","organizations_url":"https://api.github.com/users/emmett9001/orgs","repos_url":"https://api.github.com/users/emmett9001/repos","events_url":"https://api.github.com/users/emmett9001/events{/privacy}","received_events_url":"https://api.github.com/users/emmett9001/received_events","type":"User","site_admin":false},"open_issues":7,"closed_issues":0,"state":"open","created_at":"2016-02-05T22:56:29Z","updated_at":"2016-02-24T21:34:42Z","due_on":null,"closed_at":null},"comments":5,"created_at":"2016-02-24T21:07:42Z","updated_at":"2016-03-03T18:10:02Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/Parsely/pykafka/pulls/471","html_url":"https://github.com/Parsely/pykafka/pull/471","diff_url":"https://github.com/Parsely/pykafka/pull/471.diff","patch_url":"https://github.com/Parsely/pykafka/pull/471.patch"},"body":"Ran into this error when attempting to gracefully shutdown my application.  What happens is when `__del__` gets called (I believe when the GC is cleaning up) and the consumer has already been stopped then your application hangs (using python 3.5.1).  Here is an example of what you see when the consumer hangs (notice how the consumer's stop method is called twice - expectation is once):\r\n```\r\nDEBUG [2016-02-24 15:41:20,334] stop: Stopping <pykafka.balancedconsumer.BalancedConsumer at 0x7f7762b25710 (consumer_group=b'nathanhale')>\r\nDEBUG [2016-02-24 15:41:20,335] commit_offsets: Committing offsets for 3 partitions to broker id 0\r\nDEBUG [2016-02-24 15:41:20,337] _submit: Sending request(xid=13): Close()\r\nDEBUG [2016-02-24 15:41:20,339] _read_watch_event: Received EVENT: Watch(type=4, state=3, path=\"/consumers/b'nathanhale'/ids\")\r\nINFO [2016-02-24 15:41:20,340] _connect_attempt: Closing connection to localhost:2181\r\nINFO [2016-02-24 15:41:20,340] _session_callback: Zookeeper session lost, state: CLOSED\r\nDEBUG [2016-02-24 15:41:20,341] __del__: Finalising <pykafka.balancedconsumer.BalancedConsumer at 0x7f7762b25710 (consumer_group=b'nathanhale')>\r\nDEBUG [2016-02-24 15:41:20,341] stop: Stopping <pykafka.balancedconsumer.BalancedConsumer at 0x7f7762b25710 (consumer_group=b'nathanhale')>\r\nDEBUG [2016-02-24 15:41:20,341] commit_offsets: Committing offsets for 3 partitions to broker id 0\r\n*** this is where it hangs...\r\n```\r\nAnd this is what you get with the fix (stop is now called once):\r\n```\r\nDEBUG [2016-02-24 15:52:05,508] stop: Stopping <pykafka.balancedconsumer.BalancedConsumer at 0x7f29634697f0 (consumer_group=b'nathanhale')>\r\nDEBUG [2016-02-24 15:52:05,508] commit_offsets: Committing offsets for 3 partitions to broker id 0\r\nDEBUG [2016-02-24 15:52:05,511] _submit: Sending request(xid=13): Close()\r\nDEBUG [2016-02-24 15:52:05,513] _read_watch_event: Received EVENT: Watch(type=4, state=3, path=\"/consumers/b'nathanhale'/ids\")\r\nINFO [2016-02-24 15:52:05,514] _connect_attempt: Closing connection to localhost:2181\r\nINFO [2016-02-24 15:52:05,514] _session_callback: Zookeeper session lost, state: CLOSED\r\n```\r\nNote that calling the consumer's stop method twice does not seem the hang consumer so the change to `__del__` should be sufficient to address this bug I think.\r\n\r\nI'm not 100% sure this is the best solution so let me know if there would be a better way to solve this issue and I can make those changes.  Since this is a minor change I am submitting to master (I did read the guidelines ;P)."},"comment":{"url":"https://api.github.com/repos/Parsely/pykafka/issues/comments/191890994","html_url":"https://github.com/Parsely/pykafka/pull/471#issuecomment-191890994","issue_url":"https://api.github.com/repos/Parsely/pykafka/issues/471","id":191890994,"user":{"login":"emmett9001","id":723615,"avatar_url":"https://avatars.githubusercontent.com/u/723615?v=3","gravatar_id":"","url":"https://api.github.com/users/emmett9001","html_url":"https://github.com/emmett9001","followers_url":"https://api.github.com/users/emmett9001/followers","following_url":"https://api.github.com/users/emmett9001/following{/other_user}","gists_url":"https://api.github.com/users/emmett9001/gists{/gist_id}","starred_url":"https://api.github.com/users/emmett9001/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/emmett9001/subscriptions","organizations_url":"https://api.github.com/users/emmett9001/orgs","repos_url":"https://api.github.com/users/emmett9001/repos","events_url":"https://api.github.com/users/emmett9001/events{/privacy}","received_events_url":"https://api.github.com/users/emmett9001/received_events","type":"User","site_admin":false},"created_at":"2016-03-03T18:10:02Z","updated_at":"2016-03-03T18:10:02Z","body":"@erickt @suicidejack I'm mainly concerned with the Travis tests, which now are passing after a few retries. Thanks for the contribution!"}}