<p>I'm trying to run confluent kafka image in kubernetes environment &amp; facing </p>

<pre><code>FATAL [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.KafkaException: Found directory /var/lib/kafka/data, 'data' is not in the form of topic-partition or topic-partition.uniqueId-delete (if marked for deletion).
Kafka's log directories (and children) should only contain Kafka topic data.
</code></pre>

<p>My deployment config:</p>

<pre><code>apiVersion: apps/v1beta2 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
  name: kafka-confluent
  labels:
    app: kafka-confluent
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka-confluent
  template:
    metadata:
      labels:
        app: kafka-confluent
    spec:
      containers:
      - name: zookeeper-kafka
        image: zookeeper:3.5
        ports:
        - containerPort: 2181
      - name: kafka-confluent
        image: confluentinc/cp-kafka:4.0.0
        ports:
        - containerPort: 9092
        command:
        - sh
        - -c
        - "exec kafka-server-start /etc/kafka/server.properties \
          --override reserved.broker.max.id=2147483647 \
          --override zookeeper.connect=localhost:2181 \
          --override listeners=PLAINTEXT://:9092 \
          "
</code></pre>

<p>To solve this I've tried to mount some ephemeral volume like this.</p>

<pre><code>volumes:
- name: kafka-data
  emptyDir: {}
...
volumeMounts:
- mountPath: /var/lib/kafka/data
  name: kafka-data
</code></pre>

<p>And clear the <code>data</code> dir with init container:</p>

<pre><code>containers:
- name: cleaner
  image: busybox
  command: ['rm', '-rf', '/var/lib/kafka/data/*']
</code></pre>

<p>Both tries failed with the same result.</p>

<p>Also If I run the image &amp; list the data <code>/var/lib/kafka/data/</code>
Looks like the directory is empty.</p>

<pre><code>docker run --rm -it confluentinc/cp-kafka:4.0.0 bash
root@35087653f43a:/# ls /var/lib/kafka/data/ -al
total 8
drwxrwxrwx 2 root root 4096 Apr 16 09:59 .
drwxr-xr-x 3 root root 4096 Jan  3 19:20 ..
</code></pre>

<p>is it your error? 
<a href="https://i.stack.imgur.com/al4yx.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/al4yx.png" alt="enter image description here"></a></p>

<p>Fix add this:</p>

<pre><code>log.dirs=/var/lib/kafka/data
</code></pre>

<p>in server.properties</p>
