<p>We are using <a href="https://github.com/camunda/zeebe" rel="nofollow noreferrer">zeebe</a> distributed workflow engine that uses log4j as its logging framework. Its a spring boot application deployed through helm on kubernetes. Unfortunately it does not include <code>log4j-layout-template-json</code> jar that enables json logging. Any approach that requires third party app code modifications doesn't look right since the project is under active development. We would need to constantly upgrade the third party app and hence forking it would make deployments tougher.</p>
<p>What are some possible ways to configure it for json logging so that the logs can be shipped to EFK stack?</p>
<p>Thanks</p>

Andy Dufresne
<p>The general solution is simple: you need to add <code>log4j-layout-template-json</code> to the classpath of the application and create a Docker image with the additional libraries.</p>
<p>The devil is in the details: every distribution has a different method to add libraries to the classpath.</p>
<p>Camunda Zeebe seems to be using <a href="https://www.mojohaus.org/appassembler/appassembler-maven-plugin/" rel="nofollow noreferrer">Appassembler</a> to create its binary distribution, which uses <a href="https://github.com/mojohaus/appassembler/blob/master/appassembler-maven-plugin/src/main/resources/org/codehaus/mojo/appassembler/daemon/script/unixBinTemplate" rel="nofollow noreferrer">this script</a> to boot up the application. So you have two choices:</p>
<ul>
<li>either you add libraries directly to <code>/usr/local/zeebe/lib</code> and your Log4j Core configuration file to <code>/usr/local/zeebe/config</code>,</li>
<li>or you add libraries in another location and set the <code>CLASSPATH_PREFIX</code> environment variable accordingly.</li>
</ul>
<p>For example you can add additional Log4j Core modules and a custom configuration file to the <code>/usr/local/log4j2</code> directory, with a Docker file like this:</p>
<pre class="lang-sh prettyprint-override"><code>FROM camunda/zeebe:latest

# Download additional Logj4 2.x artifacts
RUN apt -y update &amp;&amp; apt -y install curl
RUN curl -SL https://dist.apache.org/repos/dist/release/logging/log4j/2.20.0/apache-log4j-2.20.0-bin.tar.gz | \
  tar -xz --strip-components=1 --one-top-level=/usr/local/log4j2 \
    apache-log4j-2.20.0-bin/log4j-appserver-2.20.0.jar \
    apache-log4j-2.20.0-bin/log4j-jul-2.20.0.jar \
    apache-log4j-2.20.0-bin/log4j-layout-template-json-2.20.0.jar
# Add a custom configuration file
COPY log4j2.xml /usr/local/log4j2/

# Check artifacts
ARG LOG4J_APPSERVER_SUM=53d8e78277324145cde435b515b1c7f1ba02b93e7a1974411ce7c5511a8c6b69
ARG LOG4J_JUL_SUM=c9b33dffb40bd00d4889ea4700f79d87a2e4d9f92911a3a008ae18c0bb3fb167
ARG LOG4J_LAYOUT_TEMPLATE_JSON_SUM=62d2c2b8e80a74ca65d80cf2f9aa0eab3a1350349c7b03b428c5b53004cc751b
RUN sha256sum -c &lt;&lt;EOF
${LOG4J_APPSERVER_SUM} /usr/local/log4j2/log4j-appserver-2.20.0.jar
${LOG4J_JUL_SUM} /usr/local/log4j2/log4j-jul-2.20.0.jar
${LOG4J_LAYOUT_TEMPLATE_JSON_SUM} /usr/local/log4j2/log4j-layout-template-json-2.20.0.jar
EOF

# Add additional classpath entries
ENV CLASSPATH_PREFIX=/usr/local/log4j2/*:/usr/local/log4j2
# Use Log4j also for java.util.logging
ENV JAVA_OPTS=-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager

# Start the application
ENTRYPOINT [&quot;tini&quot;, &quot;--&quot;, &quot;/usr/local/bin/startup.sh&quot;]
</code></pre>
