{"action":"opened","issue":{"url":"https://api.github.com/repos/dpkp/kafka-python/issues/1417","repository_url":"https://api.github.com/repos/dpkp/kafka-python","labels_url":"https://api.github.com/repos/dpkp/kafka-python/issues/1417/labels{/name}","comments_url":"https://api.github.com/repos/dpkp/kafka-python/issues/1417/comments","events_url":"https://api.github.com/repos/dpkp/kafka-python/issues/1417/events","html_url":"https://github.com/dpkp/kafka-python/issues/1417","id":303409662,"number":1417,"title":"KafkaConsumer ","user":{"login":"vaibhav293","id":13231328,"avatar_url":"https://avatars3.githubusercontent.com/u/13231328?v=4","gravatar_id":"","url":"https://api.github.com/users/vaibhav293","html_url":"https://github.com/vaibhav293","followers_url":"https://api.github.com/users/vaibhav293/followers","following_url":"https://api.github.com/users/vaibhav293/following{/other_user}","gists_url":"https://api.github.com/users/vaibhav293/gists{/gist_id}","starred_url":"https://api.github.com/users/vaibhav293/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vaibhav293/subscriptions","organizations_url":"https://api.github.com/users/vaibhav293/orgs","repos_url":"https://api.github.com/users/vaibhav293/repos","events_url":"https://api.github.com/users/vaibhav293/events{/privacy}","received_events_url":"https://api.github.com/users/vaibhav293/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-03-08T09:22:50Z","updated_at":"2018-03-08T09:22:50Z","closed_at":null,"author_association":"NONE","body":"I have a kafka setup with 3 brokers and each topic distributed in 3 partitions. \r\n\r\nMy KafkaConsumer (kafka-python version=1.3.5)  implementation looks something like this:  \r\n\r\n`consumer = KafkaConsumer(config.TOPIC,\\\r\n                        bootstrap_servers=config.KAFKA_BOOTSTRAP_SERVERS,\\\r\n                        api_version=config.KAFKA_API_VERSION,\\\r\n                        group_id=config.CONSUMER_NAME + \"_group\")\r\n\r\nfor msg in consumer:\r\n    if 'event_name' in message and message['event_name'] == config.TOPIC:\r\n        with MyController() as myController:\r\n            myController.do_something(**message)`\r\n\r\n\r\nBut in the log file, I see the same message getting processed multiple times. It start with first message getting processed onetime and then slowly goes upto 4 parallel processing thread which printing exact same logs. My do_something function is a heavy operation and takes times. \r\n\r\nI am kind of clueless about this situation. \r\n\r\nHere is the kafka-consumer-groups o/p:\r\n\r\n`\r\nkafka-consumer-groups.sh --bootstrap-server \"...\" --describe --group my_group\r\nNote: This will not show information about old Zookeeper-based consumers.\r\n\r\n\r\nTOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG        CONSUMER-ID                                       HOST                           CLIENT-ID\r\nmytopic         0          152             152             0          kafka-python-1.3.5-00b0b957-65f6-4cfc-9ff9-0ddad708716d/10.31.45.230                  kafka-python-1.3.5\r\nmytopic         1          105             105             0          kafka-python-1.3.5-00b0b957-65f6-4cfc-9ff9-0ddad708716d/10.31.45.230                  kafka-python-1.3.5\r\nmytopic         2          147             147             0          kafka-python-1.3.5-00b0b957-65f6-4cfc-9ff9-0ddad708716d/10.31.45.230                  kafka-python-1.3.5\r\n`\r\n\r\n Please help me troubleshoot this issue. \r\n"}}
{"actor":{"display_login":"vaibhav293"}}