{"action":"created","issue":{"url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/issues/412","repository_url":"https://api.github.com/repos/confluentinc/confluent-kafka-python","labels_url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/issues/412/labels{/name}","comments_url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/issues/412/comments","events_url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/issues/412/events","html_url":"https://github.com/confluentinc/confluent-kafka-python/issues/412","id":336740527,"node_id":"MDU6SXNzdWUzMzY3NDA1Mjc=","number":412,"title":"Synchronous Functions Should Release the GIL","user":{"login":"bpowers39","id":40673070,"node_id":"MDQ6VXNlcjQwNjczMDcw","avatar_url":"https://avatars2.githubusercontent.com/u/40673070?v=4","gravatar_id":"","url":"https://api.github.com/users/bpowers39","html_url":"https://github.com/bpowers39","followers_url":"https://api.github.com/users/bpowers39/followers","following_url":"https://api.github.com/users/bpowers39/following{/other_user}","gists_url":"https://api.github.com/users/bpowers39/gists{/gist_id}","starred_url":"https://api.github.com/users/bpowers39/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bpowers39/subscriptions","organizations_url":"https://api.github.com/users/bpowers39/orgs","repos_url":"https://api.github.com/users/bpowers39/repos","events_url":"https://api.github.com/users/bpowers39/events{/privacy}","received_events_url":"https://api.github.com/users/bpowers39/received_events","type":"User","site_admin":false},"labels":[{"id":357948251,"node_id":"MDU6TGFiZWwzNTc5NDgyNTE=","url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/labels/bug","name":"bug","color":"ee0701","default":true}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2018-06-28T18:57:33Z","updated_at":"2018-07-03T13:13:46Z","closed_at":"2018-07-03T13:13:46Z","author_association":"NONE","body":"Description\r\n===========\r\nSome functions that take a timeout do not release the GIL. This will cause all python threads to block until the function returns.\r\n\r\nFunctions that seem to do this include:\r\nConsumer.committed()\r\nConsumer.get_watermark_offsets()\r\nConsumer.offsets_for_times()\r\nProducer.flush()\r\n\r\nConsumer.get_watermark_offsets() is the only function I'm completely sure of. The others are just a guess from looking at the code.\r\n\r\nHow to reproduce\r\n================\r\nCreate a topic with one partition. Run the below example. Once the example has consumed a few messages, kill the broker hosting the partition. Notice the other thread is longer printing it's debug message.\r\n\r\n```python\r\nimport time\r\nimport sys\r\nimport threading\r\nimport confluent_kafka\r\nfrom confluent_kafka import Consumer, KafkaError\r\nfrom uuid import uuid4\r\n\r\ndef debug_thread_func():\r\n    while True:\r\n        print(\"Test...\")\r\n        time.sleep(1)\r\n\r\nif __name__ == '__main__':\r\n    print(confluent_kafka.version(), confluent_kafka.libversion())\r\n\r\n    debug_thread = threading.Thread(target=debug_thread_func)\r\n    debug_thread.start()\r\n\r\n    client = Consumer({'bootstrap.servers': 'gateway:9092', 'group.id': str(uuid4()),\r\n              'default.topic.config': {'auto.offset.reset': 'smallest'}})\r\n\r\n    def assigned(consumer, partitions):\r\n        print(\"Assigned:\", partitions)\r\n        \r\n\r\n    client.subscribe(['ibbot'], on_assign=assigned)\r\n\r\n    while True:\r\n        msg = client.poll(timeout=1)\r\n        for partition in client.assignment():\r\n            print(client.get_watermark_offsets(partition))\r\n\r\n        if msg is not None:\r\n            if msg.error():\r\n                print(\"Error: \", msg.error())\r\n            else:\r\n                print(\"Data\")\r\n\r\n    client.close()\r\n   \r\n\r\n```\r\n\r\nNote that this example is probably dependent on a bug in rdkafka, so it may not be reproducible 100% of the time. You can also reproduce this with a multi-partition and broker setup. In this case, the application only deadlocks for as long as it takes for a new leader to be elected.\r\n\r\n\r\nChecklist\r\n=========\r\nPlease provide the following information:\r\n\r\n - [x] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`):  kafka-python: ('0.11.4', 721920)  rdkafka: ('0.11.4', 722175)\r\n - [x] Apache Kafka broker version: 0.11.0.2\r\n - [x] Client configuration: `{'bootstrap.servers': 'gateway:9092', 'group.id': str(uuid4()),\r\n              'default.topic.config': {'auto.offset.reset': 'smallest'}}`\r\n - [X] Operating system: RHEL 7\r\n - [ ] Provide client logs (with `'debug': '..'` as necessary)\r\n - [ ] Provide broker log excerpts\r\n - [ ] Critical issue\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/issues/comments/402152544","html_url":"https://github.com/confluentinc/confluent-kafka-python/issues/412#issuecomment-402152544","issue_url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/issues/412","id":402152544,"node_id":"MDEyOklzc3VlQ29tbWVudDQwMjE1MjU0NA==","user":{"login":"rnpridgeon","id":5313961,"node_id":"MDQ6VXNlcjUzMTM5NjE=","avatar_url":"https://avatars0.githubusercontent.com/u/5313961?v=4","gravatar_id":"","url":"https://api.github.com/users/rnpridgeon","html_url":"https://github.com/rnpridgeon","followers_url":"https://api.github.com/users/rnpridgeon/followers","following_url":"https://api.github.com/users/rnpridgeon/following{/other_user}","gists_url":"https://api.github.com/users/rnpridgeon/gists{/gist_id}","starred_url":"https://api.github.com/users/rnpridgeon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rnpridgeon/subscriptions","organizations_url":"https://api.github.com/users/rnpridgeon/orgs","repos_url":"https://api.github.com/users/rnpridgeon/repos","events_url":"https://api.github.com/users/rnpridgeon/events{/privacy}","received_events_url":"https://api.github.com/users/rnpridgeon/received_events","type":"User","site_admin":false},"created_at":"2018-07-03T13:13:45Z","updated_at":"2018-07-03T13:13:45Z","author_association":"CONTRIBUTOR","body":"Thanks for reporting this Brendan, marking this as closed with the merger of #415 "}}
{"actor":{"display_login":"rnpridgeon"}}