{"action":"created","issue":{"url":"https://api.github.com/repos/edenhill/librdkafka/issues/186","repository_url":"https://api.github.com/repos/edenhill/librdkafka","labels_url":"https://api.github.com/repos/edenhill/librdkafka/issues/186/labels{/name}","comments_url":"https://api.github.com/repos/edenhill/librdkafka/issues/186/comments","events_url":"https://api.github.com/repos/edenhill/librdkafka/issues/186/events","html_url":"https://github.com/edenhill/librdkafka/issues/186","id":55617424,"number":186,"title":"Failures to resolve addresses are logged continuously instead of once, like connection failures","user":{"login":"thelamb","id":537058,"avatar_url":"https://avatars.githubusercontent.com/u/537058?v=3","gravatar_id":"","url":"https://api.github.com/users/thelamb","html_url":"https://github.com/thelamb","followers_url":"https://api.github.com/users/thelamb/followers","following_url":"https://api.github.com/users/thelamb/following{/other_user}","gists_url":"https://api.github.com/users/thelamb/gists{/gist_id}","starred_url":"https://api.github.com/users/thelamb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thelamb/subscriptions","organizations_url":"https://api.github.com/users/thelamb/orgs","repos_url":"https://api.github.com/users/thelamb/repos","events_url":"https://api.github.com/users/thelamb/events{/privacy}","received_events_url":"https://api.github.com/users/thelamb/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-01-27T13:29:48Z","updated_at":"2016-06-08T21:17:07Z","closed_at":"2015-01-31T15:49:56Z","body":"The text below describes the original question: librdkafka continuously tries to reconnect to brokers it cannot connect to. The question included a note that address resolve errors are logged continously. \r\nThe continuous reconnect is by design, and no longer part of the 'issue' but I left it for reference\r\n\r\n-----\r\nIt appears to me that when librdkafka fails to connect (or fails to resolve) a broker, it keeps retrying indefinitely. Am I overlooking a configuration parameter or is this expected behavior?\r\n\r\nI'm in a scenario where my users can send Kafka message to a broker (Lua wrapper around librdkafka). The user can configure the broker's address. In case the broker is down, or if the user entered an invalid broker address I'd rather not have constant retries in the background.\r\n\r\nInternally I solve the problem now by removing inactive Producers after a certain timeout.\r\nThis works alright for connection failures (i.e. Connection refused), but it produces a lot of spam when there is a failure in resolving the broker's address. In this case the library prints the error message to terminal continuously.\r\n\r\nTo reproduce:\r\n```\r\n $ ./rdkafka_example -P -t test -b localhost:9092\r\n% Type stuff and hit enter to send\r\n1422363572.561 RDKAFKA-3-FAIL: rdkafka#producer-0: localhost:9092/bootstrap: Failed to connect to broker at localhost:9092: Connection refused\r\n1422363572.561 RDKAFKA-3-ERROR: rdkafka#producer-0: localhost:9092/bootstrap: Failed to connect to broker at localhost:9092: Connection refused\r\n1422363572.561 RDKAFKA-3-ERROR: rdkafka#producer-0: 1/1 brokers are down\r\n```\r\nThis 'Connection refused' is only printed once because of the duplicate-error-check in `rd_kafka_broker_connect`. \r\n \r\n```\r\n$ ./rdkafka_example -P -t test -b foo:9092\r\n% Type stuff and hit enter to send\r\n1422363764.196 RDKAFKA-3-ERROR: rdkafka#producer-0: Failed to resolve 'foo:9092': Name or service not known\r\n1422363764.196 RDKAFKA-3-GETADDR: rdkafka#producer-0: foo:9092/bootstrap: Failed to resolve 'foo:9092': Name or service not known\r\n1422363765.197 RDKAFKA-3-ERROR: rdkafka#producer-0: Failed to resolve 'foo:9092': Name or service not known\r\n1422363765.197 RDKAFKA-3-GETADDR: rdkafka#producer-0: foo:9092/bootstrap: Failed to resolve 'foo:9092': Name or service not known\r\n```\r\n\r\nThe resolve errors are printed continuously.\r\n\r\nSo my questions are:\r\n  1. Is there a reason for the indefinite retries?\r\n    1.1. If not, would you be interested in a pull request for this if I find the time to do it?\r\n  2. I suppose I could work around the log-spam by implemented the log_cb. I just feel like it should be handled the same way as a connection failure. \r\n\r\n\r\nI must admit that I'm in no way an expert when it comes to Kafka, so there may very well be a good reason for which you must continuously try to reconnect. \r\n\r\nThanks for your time/efforts\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/edenhill/librdkafka/issues/comments/224730836","html_url":"https://github.com/edenhill/librdkafka/issues/186#issuecomment-224730836","issue_url":"https://api.github.com/repos/edenhill/librdkafka/issues/186","id":224730836,"user":{"login":"mmanoj","id":1105057,"avatar_url":"https://avatars.githubusercontent.com/u/1105057?v=3","gravatar_id":"","url":"https://api.github.com/users/mmanoj","html_url":"https://github.com/mmanoj","followers_url":"https://api.github.com/users/mmanoj/followers","following_url":"https://api.github.com/users/mmanoj/following{/other_user}","gists_url":"https://api.github.com/users/mmanoj/gists{/gist_id}","starred_url":"https://api.github.com/users/mmanoj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mmanoj/subscriptions","organizations_url":"https://api.github.com/users/mmanoj/orgs","repos_url":"https://api.github.com/users/mmanoj/repos","events_url":"https://api.github.com/users/mmanoj/events{/privacy}","received_events_url":"https://api.github.com/users/mmanoj/received_events","type":"User","site_admin":false},"created_at":"2016-06-08T21:17:06Z","updated_at":"2016-06-08T21:17:06Z","body":"@edenhill \r\n\r\nI got below error once i kill the one of the kafka process (I have 3, with replication-factor 3)\r\n\r\n1465420107.985 RDKAFKA-3-ERROR: rdkafka#producer-5109: localhost:9094/bootstrap: Connect to ipv4#127.0.0.1:9094 failed: Connection refused\r\n\r\nOriginal Kafka State:\r\n===========================\r\n./kafka-topics.sh --describe --zookeeper localhost:2181 --topic PCRF\r\nTopic:PCRF\tPartitionCount:1\tReplicationFactor:3\tConfigs:\r\n\tTopic: PCRF\tPartition: 0\tLeader: 2\tReplicas: 2,0,1\tIsr: 0,2,1\r\n\r\nAfter Kill one server process:\r\n------------------------------------------------------------------------------------------------\r\n./kafka-topics.sh --describe --zookeeper localhost:2181 --topic PCRF\r\nTopic:PCRF\tPartitionCount:1\tReplicationFactor:3\tConfigs:\r\n\tTopic: PCRF\tPartition: 0\tLeader: 0\tReplicas: 2,0,1\tIsr: 0,1\r\n\r\n"}}
{"actor":{"display_login":"mmanoj"}}