title: Spark: spark-submit won't take custom log4j.properties
body: <p>This could be possible duplicate of <a href="https://stackoverflow.com/questions/31856532/spark-unable-to-load-custom-log4j-properties-from-fat-jar-resources">old post</a>, but that was relevant to <code>Spark</code> <code>1.3/4</code>. I am using <code>1.5.2</code>.</p>
  
  <p>I am packaging <code>log4j.properties</code> in my fat jar. It is showing different behavior in two scenarios. On ubuntu <code>machine 1</code>, I have spark in a user directory. On similar <code>machine 2</code>, I have it system installed. On  machine 2 a default <code>log4j.properties</code> is available at : <code>/etc/spark/conf/log4j.properties</code>. </p>
  
  <p>On <code>machine 2</code>, the packaged <code>log4j.properties</code> in the fat-jar is not getting loaded, but it does on <code>machine 1</code>. I am using the same command on both machines:</p>
  
  <pre><code>spark-submit 
  --master local[1]   
  --class com.myCompany.myMainClass  myFat.jar
  </code></pre>
  
  <p>Based on <a href="https://spark.apache.org/docs/1.5.2/running-on-yarn.html" rel="nofollow noreferrer">spark documentation</a>, I am though able to do it forcefully by providing it from outside by:</p>
  
  <pre><code>spark-submit 
  --master local[1]
  --driver-java-options "-Dlog4j.configuration=file:///mnt1/mohit/log4j.properties"
  --class com.myCompany.myMainClass 
  myFat.jar 
  </code></pre>
  
  <p>Why the behavior differs? How can I make <code>spark-submit</code> to use the packaged file?</p>