title: Debezium with Postgres | Kafka Consumer not able to consume any message
body: <p>Here is my docker-compose file:</p>
  <pre><code>version: '3.7'
  
  services:
  
    postgres:
      image: debezium/postgres:12
      container_name: postgres
      networks: 
        - broker-kafka
      environment:
        POSTGRES_PASSWORD: admin
        POSTGRES_USER: antriksh
      ports:
        - 5499:5432
        
    zookeeper:
      image: confluentinc/cp-zookeeper:latest
      container_name: zookeeper
      networks: 
        - broker-kafka
      ports:
        - 2181:2181
      environment:
        ZOOKEEPER_CLIENT_PORT: 2181
        ZOOKEEPER_TICK_TIME: 2000
  
    kafka:
      image: confluentinc/cp-kafka:latest
      container_name: kafka
      networks: 
        - broker-kafka
      depends_on:
        - zookeeper
      ports:
        - 9092:9092
      environment:
        KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
        KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
        KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
        KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
        KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
        KAFKA_LOG_CLEANER_DELETE_RETENTION_MS: 5000
        KAFKA_BROKER_ID: 1
        KAFKA_MIN_INSYNC_REPLICAS: 1
  
    connector:
      image: debezium/connect:latest
      container_name: kafka_connect_with_debezium
      networks: 
        - broker-kafka
      ports:
        - &quot;8083:8083&quot;
      environment:
        GROUP_ID: 1
        CONFIG_STORAGE_TOPIC: my_connect_configs
        OFFSET_STORAGE_TOPIC: my_connect_offsets
        BOOTSTRAP_SERVERS: kafka:29092
      depends_on:
        - zookeeper
        - kafka
  
  networks: 
    broker-kafka:
      driver: bridge  
  </code></pre>
  <p>I am able to create table and insert data into it. I am also able to initialise connector using following config -</p>
  <pre><code>curl -X POST -H &quot;Accept:application/json&quot; -H &quot;Content-Type:application/json&quot; localhost:8083/connectors/ -d '
  {                                                
   &quot;name&quot;: &quot;payment-connector&quot;,
   &quot;config&quot;: {
   &quot;connector.class&quot;: &quot;io.debezium.connector.postgresql.PostgresConnector&quot;,
   &quot;tasks.max&quot;: &quot;1&quot;,
   &quot;database.hostname&quot;: &quot;postgres&quot;,
   &quot;database.port&quot;: &quot;5432&quot;,
   &quot;database.user&quot;: &quot;antriksh&quot;,
   &quot;database.password&quot;: &quot;admin&quot;,
   &quot;database.dbname&quot; : &quot;payment&quot;,
   &quot;database.server.name&quot;: &quot;dbserver1&quot;,
   &quot;database.whitelist&quot;: &quot;payment&quot;,
   &quot;database.history.kafka.bootstrap.servers&quot;: &quot;localhost:9092&quot;,
   &quot;database.history.kafka.topic&quot;: &quot;schema-changes.payment&quot;,
   &quot;publication.name&quot;: &quot;mytestpub&quot;,
   &quot;publication.autocreate.mode&quot;: &quot;all_tables&quot;
   }
  }'
  </code></pre>
  <p>I start my Kafka Consumer like this</p>
  <pre><code>kafka-console-consumer --bootstrap-server kafka:29092 --from-beginning --topic dbserver1.public.transaction --property print.key=true --property key.separator=&quot;-&quot;
  </code></pre>
  <p>But whenever I update or insert any changes inside my db I don't see the messages being relayed to Kafka Consumer.</p>
  <p>I have put the config property - <code>&quot;publication.autocreate.mode&quot;: &quot;all_tables&quot;</code> which will create a publication automatically for all tables. But when I do <code> select * from pg_publication</code> I see nothing. It's an empty table.</p>
  <p>There is a Debezium named replication slot so I don't know which config or step am I missing which is preventing Kafka Consumer to consume the message.</p>
  <p><strong>Update:</strong></p>
  <p>I found out that in order for Debezium to create publications automatically we will need pgoutput as the output plugin. Also after OneCricketer's comment this is my connector's config</p>
  <pre><code>curl -X POST -H &quot;Accept:application/json&quot; -H &quot;Content-Type:application/json&quot; localhost:8083/connectors/ -d '
  {
   &quot;name&quot;: &quot;payment-connector&quot;,
   &quot;config&quot;: {
   &quot;connector.class&quot;: &quot;io.debezium.connector.postgresql.PostgresConnector&quot;,
   &quot;tasks.max&quot;: &quot;1&quot;,
   &quot;database.hostname&quot;: &quot;postgres&quot;,
   &quot;database.port&quot;: &quot;5432&quot;,
   &quot;database.user&quot;: &quot;antriksh&quot;,
   &quot;database.password&quot;: &quot;admin&quot;,
   &quot;database.dbname&quot; : &quot;payment&quot;,
   &quot;database.server.name&quot;: &quot;dbserver1&quot;,
   &quot;database.whitelist&quot;: &quot;payment&quot;,
   &quot;database.history.kafka.bootstrap.servers&quot;: &quot;kafka:29092&quot;,
   &quot;database.history.kafka.topic&quot;: &quot;schema-changes.payment&quot;,
   &quot;plugin.name&quot;: &quot;pgoutput&quot;,
   &quot;publication.autocreate.mode&quot;: &quot;all_tables&quot;,
   &quot;publication.name&quot;: &quot;my_publication&quot;
   }
  }'
  </code></pre>
  <p>Now I am able to see the publication being created.</p>
  <pre><code>16395 | my_publication  |       10 | t            | t         | t         | t         | t
  
  </code></pre>
  <p>The issue now seems to be that the LSN is not moving ahead when I check the replication_slots</p>
  <pre><code>select * from pg_replication_slots;
   slot_name |  plugin  | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn 
  -----------+----------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
   debezium  | pgoutput | logical   |  16385 | payment  | f         | t      |        260 |      |          491 | 0/176F268   | 0/176F268
  (1 row)
  </code></pre>
  <p>It's stuck at 0/176F268 ever since the payment db was created.</p>
  <p>When I see the topics list I can see that Debezium has created the topic for the transaction table</p>
  <pre><code>[appuser@a112a33992d1 ~]$ kafka-topics --zookeeper zookeeper:2181 --list
  __consumer_offsets
  connect-status
  dbserver1.public.transaction
  my_connect_configs
  my_connect_offsets
  </code></pre>
  <p>I am unable to understand where is it going wrong.</p>