title: Kafka Docker - Can't produce or consume from outside of docker container
body: <p>Kafka works fine in the docker container. I can use <code>docker exec -it [container name] [kafkascript]</code> and successfully create topics, produce/consume messages, but when I try from outside of the docker container using local kafka scripts I can only create and list topics.  Producing and consuming messages throws errors:</p>
  
  <p><strong>Producing</strong>:</p>
  
  <p><code>~/development/lib/kafka/kafka_2.11-0.10.0.0$ bin/kafka-console-producer.sh --broker-list $(docker-machine ip kafka):9092 --topic test
  asdf</code></p>
  
  <pre><code>[2016-09-18 10:13:48,999] ERROR Error when sending message to topic test with key: null, value: 4 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
  org.apache.kafka.common.errors.TimeoutException: Batch containing 1 record(s) expired due to timeout while requesting metadata from brokers for test-0
  </code></pre>
  
  <p><strong>Consuming:</strong></p>
  
  <p><code>~/development/lib/kafka/kafka_2.11-0.10.0.0$ bin/kafka-console-consumer.sh --zookeeper $(docker-machine ip kafka):2181 --topic test --from-beginning</code></p>
  
  <pre><code>[2016-09-18 09:57:10,389] WARN Fetching topic metadata with correlation id 0 for topics [Set(test)] from broker [BrokerEndPoint(0,ba762186182f,9092)] failed (kafka.client.ClientUtils$)
  java.nio.channels.ClosedChannelException
      at kafka.network.BlockingChannel.send(BlockingChannel.scala:110)
      at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80)
      at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)
      at kafka.producer.SyncProducer.send(SyncProducer.scala:124)
      at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)
      at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:94)
      at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
      at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
  [2016-09-18 09:57:10,392] WARN [console-consumer-34526_3c15c2c24040-1474210630122-9404562b-leader-finder-thread], Failed to find leader for Set([test,0]) (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread)
  kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(BrokerEndPoint(0,ba762186182f,9092))] failed
      at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:73)
      at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:94)
      at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
      at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
  Caused by: java.nio.channels.ClosedChannelException
      at kafka.network.BlockingChannel.send(BlockingChannel.scala:110)
      at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80)
      at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)
      at kafka.producer.SyncProducer.send(SyncProducer.scala:124)
      at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)
      ... 3 more
  </code></pre>
  
  <p>I'm using spotify/docker-kafka ,but I upgraded it to 0.10.0.0 and used some suggestions from <a href="https://github.com/spotify/docker-kafka/issues/40" rel="noreferrer">jshark</a> that sets up advertised.listeners. I'm running on a Mac. I've created a docker-machine called kafka. Here is my docker run:</p>
  
  <pre><code> docker run -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=`docker-machine ip kafka` --env ADVERTISED_PORT=9092 kafka
  </code></pre>
  
  <p>Here is my dockerfile:</p>
  
  <pre><code># Kafka and Zookeeper
  
  FROM java:openjdk-8-jre
  
  ENV DEBIAN_FRONTEND noninteractive
  ENV SCALA_VERSION 2.11
  ENV KAFKA_VERSION 0.10.0.0
  ENV KAFKA_HOME /opt/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION"
  
  # Install Kafka, Zookeeper and other needed things
  RUN apt-get update &amp;&amp; \
      apt-get install -y zookeeper wget supervisor dnsutils &amp;&amp; \
      rm -rf /var/lib/apt/lists/* &amp;&amp; \
      apt-get clean &amp;&amp; \
      wget -q http://apache.mirrors.spacedump.net/kafka/"$KAFKA_VERSION"/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION".tgz -O /tmp/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION".tgz &amp;&amp; \
      tar xfz /tmp/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION".tgz -C /opt &amp;&amp; \
      rm /tmp/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION".tgz
  
  ADD scripts/start-kafka.sh /usr/bin/start-kafka.sh
  
  # Supervisor config
  ADD supervisor/kafka.conf supervisor/zookeeper.conf /etc/supervisor/conf.d/
  
  # 2181 is zookeeper, 9092 is kafka
  EXPOSE 2181
  EXPOSE 9092
  
  CMD ["supervisord", "-n"]
  </code></pre>
  
  <p><a href="https://gist.github.com/JonathanAaron/0f21bcbefbda26e41e29d560ef97e48a" rel="noreferrer">scripts/start-kafka.sh</a></p>