title: How to prepare tests using testcontainers and spring-kafka
body: <p>I'm trying to setup my integration tests for kafka messaging and to switch from using Embedded-Kafka in favor to the Testcontainers. Given following configuration for docker-compose and a base class for all integration tests:</p>
  <p>kafka-compose.yaml:</p>
  <pre><code>version: '3.3'
  
  services:
    zookeeper:
      image: &quot;wurstmeister/zookeeper&quot;
    kafka:
      image: &quot;wurstmeister/kafka:2.12-2.2.2&quot;
      ports:
        - &quot;9092:9092&quot;
      depends_on:
        - &quot;zookeeper&quot;
      environment:
        KAFKA_ZOOKEEPER_CONNECT: &quot;zookeeper:2181&quot;
        KAFKA_ADVERTISED_HOST_NAME: &quot;${KAFKA_HOST:-localhost}&quot;
        KAFKA_ADVERTISED_PORT: &quot;9092&quot;
        KAFKA_CREATE_TOPICS: &quot;recoverer-test:1:1,some-topic&quot;
        KAFKA_AUTO_CREATE_TOPICS_ENABLE: &quot;false&quot;
  </code></pre>
  <pre><code>@SpringBootTest
  @Slf4j
  public class IntegrationTest {
    private static final DockerComposeContainer kafkaContainer = initializeKafkaContainer();
  
    protected static DockerComposeContainer initializeKafkaContainer() {
      log.info(
          &quot;Initializing kafka container. Should be called only once. Current value of the kafkaContainer: {}&quot;,
          kafkaContainer);
      try {
        var kafkaContainer =
            new DockerComposeContainer(new File(&quot;src/test/resources/kafka-compose.yml&quot;))
                .withExposedService(&quot;kafka_1&quot;, 9092);
        kafkaContainer.start();
  
        var bootstrapServers =
            format(
                &quot;PLAINTEXT://%s:%s&quot;,
                kafkaContainer.getServiceHost(&quot;kafka_1&quot;, 9092),
                kafkaContainer.getServicePort(&quot;kafka_1&quot;, 9092));
  
        System.setProperty(&quot;spring.embedded.kafka.brokers&quot;, bootstrapServers);
  
        return kafkaContainer;
      } catch (Throwable t) {
        log.error(&quot;Can't initialize the Kafka test container.&quot;, t);
        throw t;
      }
    }
    
  </code></pre>
  <pre><code>@DirtiesContext(classMode = ClassMode.BEFORE_CLASS)
  class PerformSomethingInboundAdapterTest extends IntegrationTest {
  
    private static final String GROUP_ID = &quot;test-group-id&quot;;
    private static final TopicPartition PARTITION = new TopicPartition(SOME_TOPIC, 0);
    private static final Instant RECEIVED_AT = now();
    private static final CustomerNumber CUSTOMER_NUMBER = CustomerNumber.of(600830);
  
    @Autowired private KafkaListenerEndpointRegistry kafkaListenerRegistry;
  
    @Autowired private ConsumerFactory&lt;String, String&gt; consumerFactory;
  
    @Autowired private KafkaTemplate&lt;Object, Object&gt; kafkaTemplate;
  
    @MockBean private ActivateSomethingActivities activateCampaignActivities;
  
    private Consumer&lt;String, String&gt; consumer;
  
    private long initiallyCommittedOffset;
  
    @BeforeEach
    void startKafkaListener() {
      kafkaListenerRegistry.getListenerContainers().forEach(Lifecycle::start);
    }
  
    @AfterEach
    void stopKafkaListener() {
      kafkaListenerRegistry.getListenerContainers().forEach(Lifecycle::stop);
    }
  
    @Test
    void shouldPerformSomething() {
  ...
    }
  </code></pre>
  <p>There are few problems which I encountered:</p>
  <ol>
  <li>It seems that spring-kafka and it's <code>@KafkaListener</code>s are active during all possible tests annotated with @SpringBootTest and not only during the kafka specific ones. That means a message sent to a kafka topic can be consumed by an arbitrary test. First of all the question for the spring-kafka guys: is it possible to use spring-kafka-test along with Testcontainers at all? Are there some possibility to stop all the <code>@KafkaListener</code>'s for each test and enable them explicitly for specific <code>@SpringBootTest</code> tests?</li>
  <li>Testcontainers comes with a Kafka module on board. This one uses the confluent kafka docker image which is very stubborn in regards on configuration. For instance you can't set some broker properties and you can't tell the container which topic should be created after the start. After struggling with this module I decided to use the docker-compose module with the <code>wurstmeister/kafka</code> image. The problem with latter approach is that when I run the tests with command-line maven I get the error message telling that kafka is already running on 9092 port. It seems that maven starts few JVM during <code>mvn test</code> and as consequence the static field <code>kafkaContainer</code> gets initialised few times. Why does it happen?</li>
  </ol>