title: console-producer Error connecting to node kafka1:9092 (org.apache.kafka.clients.NetworkClient) java.net.UnknownHostException
body: <p>I am starting a container with kafka via docker-compose successfully. By successfully I mean, I can log in such kafka container and produce and consume messages straight from there. Also I can see another container (filebeat) depending on such kafka container connecting successfully. </p>
  
  <p>docker-compose.yml</p>
  
  <pre><code>version: '3.2'
  services:
  
    zoo1:
      image: elevy/zookeeper:latest
      environment:
        MYID: 1
        SERVERS: zoo1
      ports:
        - "2181:2181"
  
    kafka1:
      image: wurstmeister/kafka
      command: [start-kafka.sh]
      depends_on:
        - zoo1
      links:
        - zoo1
      ports:
        - "9092:9092"
      environment:
        KAFKA_LISTENERS: PLAINTEXT://:9092
        KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092
        KAFKA_BROKER_ID: 1
        KAFKA_ADVERTISED_PORT: 9092
        KAFKA_LOG_RETENTION_HOURS: "168"
        KAFKA_LOG_RETENTION_BYTES: "100000000"
        KAFKA_ZOOKEEPER_CONNECT:  zoo1:2181
        KAFKA_CREATE_TOPICS: "log:1:1"
        KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
  
    filebeat:
      image: docker.elastic.co/beats/filebeat:7.5.2
      command: filebeat -e -strict.perms=false
      volumes:
        - "//c/Users/Cast/megalog-try-1/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro"
        - "//c/Users/Cast/megalog-try-1/sample-logs:/sample-logs"
      links:
        - kafka1
      depends_on:
        - kafka1
  </code></pre>
  
  <p>Nevertheless, when I try produce a very simple message from my local kafka to this container kafka I get connection refused.</p>
  
  <p>I found someone with same exception but a bit similar scenario. I tried <a href="https://stackoverflow.com/a/57965829/4148175">his solution</a> without success. Basically it points to "adjust advertised.listeners to be the external IP". I did it in my docker-compose but nothing changed at all. And even if this had worked, I guess it would not be a final solution because for me since the guy is hardcoding his computer ip address in KAFKA_ADVERTISED_LISTENERS</p>
  
  <p>So my question is how can I connect from my local kafka running in my Windows 10 to a kafka container in order to send a message?</p>
  
  <p>Here are everything I have tried so far:</p>
  
  <p>1 - trying with local ip</p>
  
  <pre><code>ipconfig
  ...
  Endereço IPv4. . . . . . . .  . . . . . . . : 192.168.129.97
  
  kafka-console-producer.bat --broker-list 192.168.129.97:9092 --topic log
  
  C:\tools\kafka\bin\windows&gt;kafka-console-producer.bat --broker-list 192.168.129.97:9092 --topic log
  &gt;[2020-03-02 16:54:44,155] WARN [Producer clientId=console-producer] Connection to node -1 (/192.168.129.97:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
  [2020-03-02 16:54:46,212] WARN [Producer clientId=console-producer] Connection to node -1 (/192.168.129.97:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
  </code></pre>
  
  <p>2 - trying with docker machine ip</p>
  
  <pre><code>C:\tools\kafka\bin\windows&gt;docker-machine ip
  192.168.99.100
  
  kafka-console-producer.bat --broker-list 192.168.99.100:9092 --topic log
  
  
  [2020-03-02 17:00:22,500] WARN [Producer clientId=console-producer] Error connecting to node kafka1:9092 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient)
  java.net.UnknownHostException: kafka1
          at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:797)
          at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
          at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1364)
          at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
          at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:104)
          at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:403)
          at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:363)
          at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:151)
          at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:955)
          at org.apache.kafka.clients.NetworkClient.access$600(NetworkClient.java:69)
          at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1126)
          at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1017)
          at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:538)
          at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:335)
          at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)
          at java.base/java.lang.Thread.run(Thread.java:834)
  </code></pre>
  
  <p>3 - trying with container ip</p>
  
  <pre><code>docker ps
  ...
  3eb114e24b53        wurstmeister/kafka                                    "start-kafka.sh"      0.0.0.0:9092-&gt;9092/tcp                                     megalog-try-1_kafka1_1
  
  docker inspect 3eb114e24b53
  ...
  "IPAddress": "172.18.0.4",
  
  C:\tools\kafka\bin\windows&gt;kafka-console-producer.bat --broker-list 172.18.0.4:9092 --topic log
  &gt;testing
  [2020-03-02 16:57:54,471] WARN [Producer clientId=console-producer] Connection to node -1 (/172.18.0.4:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
  org.apache.kafka.common.KafkaException: Producer closed while send in progress
          at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:888)
  </code></pre>
  
  <p>*** added after CRU's suggestion</p>
  
  <pre><code>C:\tools\kafka\bin\windows&gt;docker exec -it megalog-try-1_zoo1_1 bash
  bash-4.3# ls
  bin            entrypoint.sh  home           lib64          mnt            root           sbin           sys            usr            zookeeper
  dev            etc            lib            media          proc           run            srv            tmp            var
  bash-4.3# cd zookeeper/
  bash-4.3# ls
  bin                   conf                  data                  lib                   log                   wal                   zookeeper-3.4.10.jar
  bash-4.3# cd conf
  bash-4.3# ls
  log4j.properties  zoo.cfg
  bash-4.3# cat zoo.cfg
  tickTime=2000
  initLimit=10
  syncLimit=5
  dataDir=/zookeeper/data
  dataLogDir=/zookeeper/wal
  #snapCount=100000
  autopurge.purgeInterval=1
  clientPort=2181
  quorumListenOnAllIPs=true
  server.1=zoo1:2888:3888bash-4.3#
  </code></pre>
  
  <p>and from kafka container</p>
  
  <pre><code>C:\tools\kafka\bin\windows&gt;docker exec -it megalog-try-1_kafka1_1 bash
  bash-4.4# pwd
  /
  bash-4.4# ls
  bin    dev    etc    home   kafka  lib    lib64  media  mnt    opt    proc   root   run    sbin   srv    sys    tmp    usr    var
  bash-4.4# cd kafka
  bash-4.4# ls
  kafka-logs-0f385d1d435e
  bash-4.4# cd kafka-logs-0f385d1d435e/
  bash-4.4# ls
  app_logs-0                        log-0                             meta.properties                   replication-offset-checkpoint
  cleaner-offset-checkpoint         log-start-offset-checkpoint       recovery-point-offset-checkpoint  request_logs-0
  bash-4.4# cat meta.properties
  #
  #Mon Mar 02 21:09:33 GMT 2020
  cluster.id=-qitBVmjSUGo7Zd7P5cetw
  version=0
  broker.id=1
  bash-4.4#
  </code></pre>
  
  <p>*** added</p>
  
  <p>After stumbled in <a href="https://medium.com/@marcelo.hossomi/running-kafka-in-docker-machine-64d1501d6f0b" rel="nofollow noreferrer">this article</a> I could write a docker-compose to run kafka container that my local SpringBoot can send message. Nevertheless, I have to hard code the docker-machine ip manually in my docker-compose. Well, I don't think this is the only way otherwise everytime I start Docker ToolBox I will have to run docker-machine ip and replace the IP Address. I am wondering if there is a more friendly solution. </p>
  
  <p>Bellow 192.168.99.100 is my docker-machine ip.</p>
  
  <pre><code>version: "3.2"
  services:
    zookeeper:
      image: wurstmeister/zookeeper
      ports:
        - "2181:2181"
    kafka:
      image: wurstmeister/kafka
      ports:
        - "9092:9092"
      environment:
        KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
        KAFKA_LISTENERS:
          "INTERNAL://kafka:9090,\
           EXTERNAL://:9092"
        KAFKA_ADVERTISED_LISTENERS:
          "INTERNAL://kafka:9090,\
           EXTERNAL://192.168.99.100:9092"
        KAFKA_LISTENER_SECURITY_PROTOCOL_MAP:
          "INTERNAL:PLAINTEXT,\
           EXTERNAL:PLAINTEXT"
        KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
  </code></pre>
  
  <p>Based on Cricket_007's suggestion, I read carefully <a href="https://de.confluent.io/blog/kafka-listeners-explained/" rel="nofollow noreferrer">another article</a> but it drove me the conclusion that I must use DNS resolution but I can't setup an exclusive dns resolution in my Windows machine as far as I understand.</p>