title: Dependency errors when running Apache Beam with Kafka
body: <p>When I run my Apache Beam code using the Direct Runner I get the following error:</p>
  
  <pre><code>Caused by: java.lang.NoSuchMethodError: org.slf4j.helpers.MessageFormatter.arrayFormat(Ljava/lang/String;[Ljava/lang/Object;)Lorg/slf4j/helpers/FormattingTuple;
      at org.apache.kafka.common.utils.LogContext$LocationAwareKafkaLogger.writeLog (LogContext.java:428)
      at org.apache.kafka.common.utils.LogContext$LocationAwareKafkaLogger.info (LogContext.java:382)
      at org.apache.kafka.clients.consumer.KafkaConsumer.assign (KafkaConsumer.java:1123)
      at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
  </code></pre>
  
  <p>Running it as follows works:</p>
  
  <pre class="lang-sh prettyprint-override"><code>mvn package
  
  java -cp target/myjar.jar \
      com.CLASSNAME \
      --runner=DirectRunner \
      ...
  </code></pre>
  
  <p>Running it as follows <strong>does not work</strong>:</p>
  
  <pre class="lang-sh prettyprint-override"><code>mvn compile exec:java \
      -Dexec.mainClass="com.CLASSNAME" \
      -Dexec.args="..."
  </code></pre>
  
  <p>It seems this is related to slf4j dependency conflicts. 
  I've been looking at this for hours and do not seem to make any progress.</p>
  
  <p>Things I've tried:</p>
  
  <ul>
  <li>changing versions of libraries</li>
  <li>putting the slf4j-api dependency on top</li>
  <li>excluding dependencies</li>
  <li>changing scopes</li>
  <li>inspecting dependencies using <code>mvn dependency:tree</code>, but all looks ok</li>
  <li>changing jdk versions</li>
  </ul>
  
  <p>I have noted the following conflict between <em>identical</em> versions:
  <a href="https://i.stack.imgur.com/GCWx8.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/GCWx8.png" alt="identical conflict"></a></p>
  
  <p>Any additional ideas or input are appreciated greatly.</p>
  
  <p>Cleaned version of pom file for completeness:</p>
  
  <pre class="lang-xml prettyprint-override"><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;project xmlns="http://maven.apache.org/POM/4.0.0"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"&gt;
      &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
  
      &lt;groupId&gt;...&lt;/groupId&gt;
      &lt;artifactId&gt;...&lt;/artifactId&gt;
      &lt;version&gt;...&lt;/version&gt;
  
      &lt;packaging&gt;jar&lt;/packaging&gt;
  
      &lt;properties&gt;
          &lt;maven.compiler.source&gt;1.8&lt;/maven.compiler.source&gt;
          &lt;maven.compiler.target&gt;1.8&lt;/maven.compiler.target&gt;
          &lt;beam.version&gt;2.19.0&lt;/beam.version&gt;
          &lt;google.cloud.core.version&gt;1.108.1&lt;/google.cloud.core.version&gt;
          &lt;maven-compiler-plugin.version&gt;3.7.0&lt;/maven-compiler-plugin.version&gt;
          &lt;maven-surefire-plugin.version&gt;2.21.0&lt;/maven-surefire-plugin.version&gt;
          &lt;maven-jar-plugin.version&gt;3.0.2&lt;/maven-jar-plugin.version&gt;
          &lt;maven-shade-plugin.version&gt;3.1.0&lt;/maven-shade-plugin.version&gt;
          &lt;maven-exec-plugin.version&gt;1.6.0&lt;/maven-exec-plugin.version&gt;
          &lt;slf4j.version&gt;1.7.30&lt;/slf4j.version&gt;
          &lt;project.build.sourceEncoding&gt;UTF-8&lt;/project.build.sourceEncoding&gt;
      &lt;/properties&gt;
  ...
      &lt;dependencies&gt;
  
          &lt;dependency&gt;
              &lt;groupId&gt;org.slf4j&lt;/groupId&gt;
              &lt;artifactId&gt;slf4j-api&lt;/artifactId&gt;
              &lt;version&gt;${slf4j.version}&lt;/version&gt;
          &lt;/dependency&gt;
  
          &lt;dependency&gt;
              &lt;groupId&gt;org.apache.beam&lt;/groupId&gt;
              &lt;artifactId&gt;beam-sdks-java-core&lt;/artifactId&gt;
              &lt;version&gt;${beam.version}&lt;/version&gt;
          &lt;/dependency&gt;
  
          &lt;dependency&gt;
              &lt;groupId&gt;org.apache.beam&lt;/groupId&gt;
              &lt;artifactId&gt;beam-sdks-java-extensions-google-cloud-platform-core&lt;/artifactId&gt;
              &lt;version&gt;${beam.version}&lt;/version&gt;
          &lt;/dependency&gt;
  
          &lt;dependency&gt;
              &lt;groupId&gt;org.apache.beam&lt;/groupId&gt;
              &lt;artifactId&gt;beam-runners-google-cloud-dataflow-java&lt;/artifactId&gt;
              &lt;version&gt;${beam.version}&lt;/version&gt;
          &lt;/dependency&gt;
  
          &lt;dependency&gt;
              &lt;groupId&gt;org.apache.beam&lt;/groupId&gt;
              &lt;artifactId&gt;beam-sdks-java-io-google-cloud-platform&lt;/artifactId&gt;
              &lt;version&gt;${beam.version}&lt;/version&gt;
          &lt;/dependency&gt;
  
          &lt;dependency&gt;
              &lt;groupId&gt;org.apache.beam&lt;/groupId&gt;
              &lt;artifactId&gt;beam-sdks-java-io-kafka&lt;/artifactId&gt;
              &lt;version&gt;${beam.version}&lt;/version&gt;
          &lt;/dependency&gt;
  
          &lt;dependency&gt;
              &lt;groupId&gt;org.apache.kafka&lt;/groupId&gt;
              &lt;artifactId&gt;kafka-clients&lt;/artifactId&gt;
              &lt;version&gt;2.4.0&lt;/version&gt;
          &lt;/dependency&gt;
  
          &lt;dependency&gt;
              &lt;groupId&gt;org.apache.avro&lt;/groupId&gt;
              &lt;artifactId&gt;avro&lt;/artifactId&gt;
              &lt;version&gt;1.9.2&lt;/version&gt;
          &lt;/dependency&gt;
  
          &lt;dependency&gt;
              &lt;groupId&gt;org.apache.beam&lt;/groupId&gt;
              &lt;artifactId&gt;beam-runners-direct-java&lt;/artifactId&gt;
              &lt;version&gt;${beam.version}&lt;/version&gt;
          &lt;/dependency&gt;
      &lt;/dependencies&gt;
  
  &lt;/project&gt;
  
  </code></pre>