<p>I would like to configure Kafka broker in Kubernetes. The docker image I am using is <code>confluentinc/cp-kafka:latest</code>. It requires <code>KAFKA_ADVERTISED_LISTENERS</code> environment variable which allows Kafka client to communicate with broker. </p>

<p>The problem is the difficulty to assign service endpoints IP to <code>KAFKA_ADVERTISED_LISTENERS</code>. If I am using <code>localhost</code> as this value, it is only working in local Kafka broker pod but it won't work for some Kafka client pods in kubernetes cluster to communicate with it. If I am using the service endpoint IP coming from <code>kubectl get endpoints -l app=kafka</code>, this is working but it is little overhead to use some audit script set this dynamic value every time.</p>

<p>I wonder is there a better way I can dynamically set this value in Kubernetes yaml file, so I don't need to programatically set this IP every time.</p>

<p>Here is the yaml file:</p>

<pre><code>---
apiVersion: v1
kind: Service
metadata:
  name: kafka-broker
  labels:
    app: kafka
spec:
  type: NodePort
  ports:
  - port: 9092
    targetPort: 9092
  selector:
    app: kafka

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka-broker
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka
  template:
    metadata:
      labels:
        app: kafka
    spec:
      hostname: broker
      containers:
      - name: kafka
        image: confluentinc/cp-kafka:latest
        ports:
        - containerPort: 9092
        env:
        - name: KAFKA_ADVERTISED_LISTENERS
          value: "PLAINTEXT://DYNAMIC_ENDPOINT_IP:9092"
        - name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR
          value: "1"
        - name: KAFKA_ZOOKEEPER_CONNECT
          value: zoo1:2181
</code></pre>

<p>Thanks in advance.</p>

<p><strong>Edit:</strong>  I tried to use Server name, Service host environment variable, service source IP and Pod IP. Unfortunately, I still get the error in kafka log: 
<code>
java.lang.IllegalArgumentException: Error creating broker listeners from 'PLAINTEXT://$KAFKA_BROKER_SERVICE_HOST:9092': Unable to parse PLAINTEXT://$KAFKA_BROKER_SERVICE_HOST:9092 to a broker endpoint
</code></p>

<p>If I am using <code>kubectl exec -it kafa-broker-ssfjks env</code>, those environment variables are actually set correctly in this pod. I guess it may be related to a Kafka broker configuration issue ?</p>

<p>You should let your clients connect through the service, so exposing the ip or dns of the service should work. By default services are exposed as variable names in the pod. If a dns plugin is configured dns can be used. More info: <a href="https://kubernetes.io/docs/concepts/services-networking/service/#environment-variables" rel="nofollow noreferrer">https://kubernetes.io/docs/concepts/services-networking/service/#environment-variables</a></p>
