<p>I have a Spark cluster running on Kubernetes using <a href="https://github.com/bitnami/charts/tree/main/bitnami/spark" rel="nofollow noreferrer">this</a> Bitnami Helm chart.</p>
<p>Looking at Spark documentation, I have created a log4j2 config file at the below location:</p>
<pre><code>/opt/bitnami/spark/conf/log4j2.properties
</code></pre>
<p>Configuration within this log file works fine as expected.</p>
<p>Now, I have a Spring boot based application which I am executing on this cluster using <code>spark-submit</code> command.</p>
<p>This application has a log4j2 config file at below location:</p>
<pre><code>src/main/resources/log4j2.xml
</code></pre>
<p>When I execute this jar using <code>spark-submit</code> command, configuration from this file doesn't seemed to work. I have also supply following argument from other Stackoverflow threads:</p>
<pre><code>--conf &quot;spark.driver.extraJavaOptions=-Dlog4j.configuration=log4j2.xml&quot;
--conf &quot;spark.executor.extraJavaOptions=-Dlog4j.configuration=log4j2.xml&quot;
</code></pre>
<p>Still has the same issue.</p>
<p>How can I override cluster level logging configuration from application level logging file?</p>

<p>You need to point to the path of the log4j2.xml in the system that contains the spark-submit something like this starting with file:///:</p>
<pre><code>spark.driver.extraJavaOptions=-Dlog4j.configurationFile=file:///opt/spark/log/log4j2.xml
</code></pre>
