<p>On a single Ubuntu 14.04 box</p>

<p>I've followed the same configuration as 
<a href="http://dojoblog.dellemc.com/dojo/deploy-kafka-cluster-kubernetes/" rel="nofollow noreferrer">http://dojoblog.dellemc.com/dojo/deploy-kafka-cluster-kubernetes/</a></p>

<p>I use Kubernetes version v1.10.2
( I also use apiVersion: apps/v1 in yml files. )</p>

<p>Basically I have setup a kubernetes service for kafka, and a kafka deployment  using image <strong>wurstmeister/kafka</strong>. Zookeeper is working ok. Zookeeper and Kafka services are up.
Kafka deployment is configured as per the blog :  <strong>KAFKA_ADVERTISED_HOST_NAME = the kafa service cluster IP</strong> which is for me <em>10.106.84.132</em></p>

<p>deployment config :</p>

<pre><code>....
 containers:
      - name: kafka
        image: wurstmeister/kafka
        ports:
        - containerPort: 9092
        env:
        - name: KAFKA_ADVERTISED_PORT
          value: "9092"
        - name: KAFKA_ADVERTISED_HOST_NAME
          value: 10.106.84.132
        - name: KAFKA_ZOOKEEPER_CONNECT
          value: zoo1:2181
        - name: KAFKA_BROKER_ID
          value: "1"
        - name: KAFKA_CREATE_TOPICS
          value: topic1:3:3
</code></pre>

<p>Then I test the kafka subscribe and publish from outside the kafka container on my host, but that fails as follow :</p>

<pre><code>root@edmitchell-virtual-machine:~# kafkacat -b 10.106.84.132:9092 -t topic1

% Auto-selecting Consumer mode (use -P or -C to override)
% ERROR: Topic topic1 error: Broker: Leader not available
</code></pre>

<p>The best I could do overall  was</p>

<p>I delete and recreate a kafka deployment with </p>

<ul>
<li>name: KAFKA_ADVERTISED_HOST_NAME
  value:  localhost</li>
</ul>

<p>I can then subscribe and publish but only from within the kafka container, it doesn't work from outside. If I change the value to anything else than localhost, nothing works.</p>

<p>Any idea ?
It looks as if Kafka is not good to be used with Kubernetes ?
maybe I should not deploy Kafka not using kubernetes..</p>

<p>many thanks
ed</p>

<hr>

<p>Thank you, I understand better now the nodeport function.</p>

<p>I still have the same issue :</p>

<pre><code>root@fnature-virtual-machine:~/Zookeeper# kafkacat -b 192.168.198.160:32748 -t topic1 % Auto-selecting Consumer mode (use -P or -C to override) % ERROR: Topic topic1 error: Broker: Leader not available
</code></pre>

<p>I created the nodeport service as you said.</p>

<pre><code>kafka-nodeport NodePort 10.111.234.104 9092:32748/TCP 27m

kafka-service LoadBalancer 10.106.84.132 9092:30351/TCP 1d
</code></pre>

<p>I also delete/create the kafka deployment with following env :  </p>

<pre><code> KAFKA_ADVERTISED_PORT: 32748

 KAFKA_ADVERTISED_HOST_NAME:  192.168.198.160

  KAFKA_ZOOKEEPER_CONNECT:     zoo1:2181

  KAFKA_BROKER_ID:             1

  KAFKA_CREATE_TOPICS:         topic1:3:3
</code></pre>

<p>—</p>

<p>also if I run the following from inside the kafka container, I get similar error  </p>

<blockquote>
  <p>"Leader not available". kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic topic1 --from-beginning</p>
</blockquote>

<p>if I create the kafka deployment with <code>KAFKA_ADVERTISED_HOST_NAME: localhost</code>, then above command works inside the kafka container</p>

<p>and 192.168.198.160 is the ip of default interface ens33 in my Ubuntu VM</p>

<p>I can’t seem to find any logs for kafka</p>

edmitchell
<p>Kafka broker registers an address to zookeeper via <code>KAFKA_ADVERTISED_HOST_NAME</code>. However, this address is a kubernetes cluster ip (10.106.84.132), which is only reachable within Kubernetes cluster. So a client outside the cluster can not reach Kafka broker using this address.</p>

<p>To resolve this problem, you can expose kafka service to a public ip, either through <code>NodePort</code> or <code>LoadBalancer</code>. For example, run <code>kubectl expose svc $YOUR_KAFKA_SERVICE_NAME --name=kafka-nodeport --type=NodePort</code>, then lookup what nodeport is exposed: <code>kubectl get svc kafka-nodeport -o yaml | grep nodePort</code>. In this example, kafka broker will be accessible via this address:  <code>$KUBERNETES_NODE_IP</code>:<code>$NODEPORT</code>.</p>
