title: Accessing bitnami/kafka outside the kubernetes cluster
body: <p>I am currently using bitnami/kafka image(<a href="https://hub.docker.com/r/bitnami/kafka" rel="nofollow noreferrer">https://hub.docker.com/r/bitnami/kafka</a>) and deploying it on kubernetes. </p>
  
  <ul>
  <li>kubernetes master: 1</li>
  <li>kubernetes workers: 3</li>
  </ul>
  
  <p>Within the cluster the other application are able to find kafka. The problem occurs when trying to access the kafka container from outside the cluster. When reading little bit I read that we need to set property "advertised.listener=PLAINTTEXT://hostname:port_number" for external kafka clients. </p>
  
  <p>I am currently referencing "<a href="https://github.com/bitnami/charts/tree/master/bitnami/kafka" rel="nofollow noreferrer">https://github.com/bitnami/charts/tree/master/bitnami/kafka</a>". Inside my values.yaml file I have added </p>
  
  <p><strong>values.yaml</strong></p>
  
  <ul>
  <li>advertisedListeners1: 10.21.0.191</li>
  </ul>
  
  <p>and  <strong>statefulset.yaml</strong></p>
  
  <pre><code>    - name: KAFKA_CFG_ADVERTISED_LISTENERS
        value: 'PLAINTEXT://{{ .Values.advertisedListeners }}:9092' 
  </code></pre>
  
  <p><strong>For a single kafka instance it is working fine.</strong></p>
  
  <p>But for 3 node kafka cluster, I changed some configuration like below:
  <strong>values.yaml</strong></p>
  
  <ul>
  <li>advertisedListeners1: 10.21.0.191 </li>
  <li>advertisedListeners2: 10.21.0.192</li>
  <li>advertisedListeners3: 10.21.0.193</li>
  </ul>
  
  <p>and <strong>Statefulset.yaml</strong></p>
  
  <pre><code>    - name: KAFKA_CFG_ADVERTISED_LISTENERS
        {{- if $MY_POD_NAME := "kafka-0" }}
        value: 'PLAINTEXT://{{ .Values.advertisedListeners1 }}:9092'
        {{- else if $MY_POD_NAME := "kafka-1" }}
        value: 'PLAINTEXT://{{ .Values.advertisedListeners2 }}:9092'
        {{- else if $MY_POD_NAME := "kafka-2" }}
        value: 'PLAINTEXT://{{ .Values.advertisedListeners3 }}:9092'
        {{- end }}
  </code></pre>
  
  <p>Expected result is that all the 3 kafka instances should get advertised.listener property set to worker nodes ip address.</p>
  
  <p>example:</p>
  
  <ul>
  <li><p>kafka-0 --> "PLAINTEXT://10.21.0.191:9092"</p></li>
  <li><p>kafka-1 --> "PLAINTEXT://10.21.0.192:9092"</p></li>
  <li><p>kafka-3 --> "PLAINTEXT://10.21.0.193:9092"</p></li>
  </ul>
  
  <p>Currently only one kafka pod in up and running and the other two are going to crashloopbackoff state. </p>
  
  <p>and the other two pods are showing error as:</p>
  
  <p>[2019-10-20 13:09:37,753] INFO [LogDirFailureHandler]: Starting (kafka.server.ReplicaManager$LogDirFailureHandler)
  [2019-10-20 13:09:37,786] ERROR [KafkaServer id=1002] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
  java.lang.IllegalArgumentException: requirement failed: Configured end points 10.21.0.191:9092 in advertised listeners are already registered by broker 1001
      at scala.Predef$.require(Predef.scala:224)
      at kafka.server.KafkaServer$$anonfun$createBrokerInfo$2.apply(KafkaServer.scala:399)
      at kafka.server.KafkaServer$$anonfun$createBrokerInfo$2.apply(KafkaServer.scala:397)
      at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
      at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
      at kafka.server.KafkaServer.createBrokerInfo(KafkaServer.scala:397)
      at kafka.server.KafkaServer.startup(KafkaServer.scala:261)
      at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
      at kafka.Kafka$.main(Kafka.scala:84)
      at kafka.Kafka.main(Kafka.scala)</p>
  
  <p>That means the logic applied in statefulset.yaml is not working. 
  Can anyone help me in resolving this..? </p>
  
  <p>Any help would be appreciated..</p>
  
  <p>The output of <code>kubectl get statefulset kafka -o yaml</code></p>
  
  <pre><code>apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    creationTimestamp: "2019-10-29T07:04:12Z"
    generation: 1
    labels:
      app.kubernetes.io/component: kafka
      app.kubernetes.io/instance: kafka
      app.kubernetes.io/managed-by: Tiller
      app.kubernetes.io/name: kafka
      helm.sh/chart: kafka-6.0.1
    name: kafka
    namespace: default
    resourceVersion: "12189730"
    selfLink: /apis/apps/v1/namespaces/default/statefulsets/kafka
    uid: d40cfd5f-46a6-49d0-a9d3-e3a851356063
  spec:
    podManagementPolicy: Parallel
    replicas: 3
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app.kubernetes.io/component: kafka
        app.kubernetes.io/instance: kafka
        app.kubernetes.io/name: kafka
    serviceName: kafka-headless
    template:
      metadata:
        creationTimestamp: null
        labels:
          app.kubernetes.io/component: kafka
          app.kubernetes.io/instance: kafka
          app.kubernetes.io/managed-by: Tiller
          app.kubernetes.io/name: kafka
          helm.sh/chart: kafka-6.0.1
        name: kafka
      spec:
        containers:
        - env:
          - name: MY_POD_IP
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: status.podIP
          - name: MY_POD_NAME
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: metadata.name
          - name: KAFKA_CFG_ZOOKEEPER_CONNECT
            value: kafka-zookeeper
          - name: KAFKA_PORT_NUMBER
            value: "9092"
          - name: KAFKA_CFG_LISTENERS
            value: PLAINTEXT://:$(KAFKA_PORT_NUMBER)
          - name: KAFKA_CFG_ADVERTISED_LISTENERS
            value: PLAINTEXT://10.21.0.191:9092
          - name: ALLOW_PLAINTEXT_LISTENER
            value: "yes"
          - name: KAFKA_CFG_BROKER_ID
            value: "-1"
          - name: KAFKA_CFG_DELETE_TOPIC_ENABLE
            value: "false"
          - name: KAFKA_HEAP_OPTS
            value: -Xmx1024m -Xms1024m
          - name: KAFKA_CFG_LOG_FLUSH_INTERVAL_MESSAGES
            value: "10000"
          - name: KAFKA_CFG_LOG_FLUSH_INTERVAL_MS
            value: "1000"
          - name: KAFKA_CFG_LOG_RETENTION_BYTES
            value: "1073741824"
          - name: KAFKA_CFG_LOG_RETENTION_CHECK_INTERVALS_MS
            value: "300000"
          - name: KAFKA_CFG_LOG_RETENTION_HOURS
            value: "168"
          - name: KAFKA_CFG_LOG_MESSAGE_FORMAT_VERSION
          - name: KAFKA_CFG_MESSAGE_MAX_BYTES
            value: "1000012"
          - name: KAFKA_CFG_LOG_SEGMENT_BYTES
            value: "1073741824"
          - name: KAFKA_CFG_LOG_DIRS
            value: /bitnami/kafka/data
          - name: KAFKA_CFG_DEFAULT_REPLICATION_FACTOR
            value: "1"
          - name: KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR
            value: "1"
          - name: KAFKA_CFG_TRANSACTION_STATE_LOG_REPLICATION_FACTOR
            value: "1"
          - name: KAFKA_CFG_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM
            value: https
          - name: KAFKA_CFG_TRANSACTION_STATE_LOG_MIN_ISR
            value: "1"
          - name: KAFKA_CFG_NUM_IO_THREADS
            value: "8"
          - name: KAFKA_CFG_NUM_NETWORK_THREADS
            value: "3"
          - name: KAFKA_CFG_NUM_PARTITIONS
            value: "1"
          - name: KAFKA_CFG_NUM_RECOVERY_THREADS_PER_DATA_DIR
            value: "1"
          - name: KAFKA_CFG_SOCKET_RECEIVE_BUFFER_BYTES
            value: "102400"
          - name: KAFKA_CFG_SOCKET_REQUEST_MAX_BYTES
            value: "104857600"
          - name: KAFKA_CFG_SOCKET_SEND_BUFFER_BYTES
            value: "102400"
          - name: KAFKA_CFG_ZOOKEEPER_CONNECTION_TIMEOUT_MS
            value: "6000"
          image: docker.io/bitnami/kafka:2.3.0-debian-9-r88
          imagePullPolicy: IfNotPresent
          livenessProbe:
            failureThreshold: 2
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            tcpSocket:
              port: kafka
            timeoutSeconds: 5
          name: kafka
          ports:
          - containerPort: 9092
            name: kafka
            protocol: TCP
          readinessProbe:
            failureThreshold: 6
            initialDelaySeconds: 5
            periodSeconds: 10
            successThreshold: 1
            tcpSocket:
              port: kafka
            timeoutSeconds: 5
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
          - mountPath: /bitnami/kafka
            name: data
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext:
          fsGroup: 1001
          runAsUser: 1001
        terminationGracePeriodSeconds: 30
    updateStrategy:
      type: RollingUpdate
    volumeClaimTemplates:
    - metadata:
        creationTimestamp: null
        name: data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 8Gi
        volumeMode: Filesystem
      status:
        phase: Pending
  status:
    collisionCount: 0
    currentReplicas: 3
    currentRevision: kafka-56ff499d74
    observedGeneration: 1
    readyReplicas: 1
    replicas: 3
    updateRevision: kafka-56ff499d74
    updatedReplicas: 3
  </code></pre>