{"action":"created","issue":{"url":"https://api.github.com/repos/zendesk/ruby-kafka/issues/288","repository_url":"https://api.github.com/repos/zendesk/ruby-kafka","labels_url":"https://api.github.com/repos/zendesk/ruby-kafka/issues/288/labels{/name}","comments_url":"https://api.github.com/repos/zendesk/ruby-kafka/issues/288/comments","events_url":"https://api.github.com/repos/zendesk/ruby-kafka/issues/288/events","html_url":"https://github.com/zendesk/ruby-kafka/issues/288","id":184108917,"number":288,"title":"busy error log spamming with unreachable kafka node","user":{"login":"MartinNowak","id":288976,"avatar_url":"https://avatars.githubusercontent.com/u/288976?v=3","gravatar_id":"","url":"https://api.github.com/users/MartinNowak","html_url":"https://github.com/MartinNowak","followers_url":"https://api.github.com/users/MartinNowak/followers","following_url":"https://api.github.com/users/MartinNowak/following{/other_user}","gists_url":"https://api.github.com/users/MartinNowak/gists{/gist_id}","starred_url":"https://api.github.com/users/MartinNowak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MartinNowak/subscriptions","organizations_url":"https://api.github.com/users/MartinNowak/orgs","repos_url":"https://api.github.com/users/MartinNowak/repos","events_url":"https://api.github.com/users/MartinNowak/events{/privacy}","received_events_url":"https://api.github.com/users/MartinNowak/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-10-20T00:40:05Z","updated_at":"2016-10-24T10:01:44Z","closed_at":null,"body":"If this is a bug report, please fill out the following:\r\n\r\n* Version of Ruby: 2.2.4\r\n* Version of Kafka: 0.9.0.1\r\n* Version of ruby-kafka: 0.3.15\r\n\r\nPlease verify that the problem you're seeing hasn't been fixed by the current `master` of ruby-kafka.\r\n\r\n###### Steps to reproduce\r\n\r\n```ruby\r\nkafka = Kafka.new(\r\n  seed_brokers: ['kafka1:9092', 'kafka2:9092', 'kafka3:9092'],\r\n  logger: logger)\r\nconsumer.subscribe('topic')\r\nconsumer.each_batch(min_bytes: 64 * 1024, max_wait_time: 5) do |kafka_batch|\r\n  kafka_batch.messages.each do |msg|\r\n    # process message\r\n  end\r\nend\r\n```\r\n\r\n###### Expected outcome\r\n\r\nWhen one of the Kafka nodes went down, the consumer should have logged an error and tried to reconnect to the cluster/find the current coordinator.\r\n\r\n###### Actual outcome\r\n\r\nWhile restarting one of the Kafka replicas (might have been the leader for the partition), the consumer spammed the error log w/ about 2 messages / 4 ms (500 msg / s).\r\n\r\n```\r\n2016-10-20 02:17:40.753\tcassandra-writer\r\nConnection error while fetching messages: Connection refused - connect(2) for 10.1.24.5:9092\r\n2016-10-20 02:17:40.749\tcassandra-writer\r\nFailed to connect to kafka3:9092: Connection refused - connect(2) for 10.1.24.5:9092\r\n2016-10-20 02:17:40.749\tcassandra-writer\r\nConnection error while fetching messages: Connection refused - connect(2) for 10.1.24.5:9092\r\n2016-10-20 02:17:40.745\tcassandra-writer\r\nFailed to connect to kafka3:9092: Connection refused - connect(2) for 10.1.24.5:9092\r\n2016-10-20 02:17:40.745\tcassandra-writer\r\nConnection error while fetching messages: Connection refused - connect(2) for 10.1.24.5:9092\r\n2016-10-20 02:17:40.741\tcassandra-writer\r\nFailed to connect to kafka3:9092: Connection refused - connect(2) for 10.1.24.5:9092\r\n2016-10-20 02:17:40.741\tcassandra-writer\r\nConnection error while fetching messages: Connection refused - connect(2) for 10.1.24.5:9092\r\n```\r\n\r\nThe error log end with this before the consumer apparently managed to reconnect to the restarted server or another one.\r\n```\r\n2016-10-20 02:17:46.167\tcassandra-writer\r\nFailed to find coordinator for group `cassandra_writer`; retrying...\r\n2016-10-20 02:17:46.138\tcassandra-writer\r\nError committing offsets: Kafka::NotCoordinatorForGroup\r\n```"},"comment":{"url":"https://api.github.com/repos/zendesk/ruby-kafka/issues/comments/255699111","html_url":"https://github.com/zendesk/ruby-kafka/issues/288#issuecomment-255699111","issue_url":"https://api.github.com/repos/zendesk/ruby-kafka/issues/288","id":255699111,"user":{"login":"mensfeld","id":392754,"avatar_url":"https://avatars.githubusercontent.com/u/392754?v=3","gravatar_id":"","url":"https://api.github.com/users/mensfeld","html_url":"https://github.com/mensfeld","followers_url":"https://api.github.com/users/mensfeld/followers","following_url":"https://api.github.com/users/mensfeld/following{/other_user}","gists_url":"https://api.github.com/users/mensfeld/gists{/gist_id}","starred_url":"https://api.github.com/users/mensfeld/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mensfeld/subscriptions","organizations_url":"https://api.github.com/users/mensfeld/orgs","repos_url":"https://api.github.com/users/mensfeld/repos","events_url":"https://api.github.com/users/mensfeld/events{/privacy}","received_events_url":"https://api.github.com/users/mensfeld/received_events","type":"User","site_admin":false},"created_at":"2016-10-24T10:01:44Z","updated_at":"2016-10-24T10:01:44Z","body":"Same issue here"}}
{"actor":{"display_login":"mensfeld"}}