title: How to configure the log level of a specific logger using log4j in pyspark?
body: <p>From <a href="https://stackoverflow.com/questions/25407550/how-do-i-log-from-my-python-spark-script">this</a> StackOverflow thread, I know how to obtain and use the log4j logger in pyspark like so:</p>
  
  <pre><code>from pyspark import SparkContext
  sc = SparkContext()
  log4jLogger = sc._jvm.org.apache.log4j
  LOGGER = log4jLogger.LogManager.getLogger('MYLOGGER')
  LOGGER.info("pyspark script logger initialized")
  </code></pre>
  
  <p>Which works fine with the <em>spark-submit</em> script.</p>
  
  <p>My question is how to modify the <em>log4j.properties</em> file to configure the log level for <em>this particular</em> logger or how to configure it dynamically?</p>