title: How to disable logging with spark/scala project using gradle?
body: <p>I am using IntelliJ with gradle to build spark project. I tried multiple ways to disable logging but to no avail.</p>
  
  <p>Here are some of the things that I have tried.
  1. Add log4j.properties under src/main/resources with the following configuration</p>
  
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code># Set everything to be logged to the console
  log4j.rootCategory=WARN, console
  log4j.appender.console=org.apache.log4j.ConsoleAppender
  log4j.appender.console.target=System.err
  log4j.appender.console.layout=org.apache.log4j.PatternLayout
  log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n
  
  # Set the default spark-shell log level to WARN. When running the spark-shell, the
  # log level for this class is used to overwrite the root logger's log level, so that
  # the user can have different defaults for the shell and regular Spark apps.
  log4j.logger.org.apache.spark.repl.Main=WARN
  
  
  # Change this to set Spark log level
  log4j.logger.org.apache.spark=WARN
  
  # Silence akka remoting
  log4j.logger.Remoting=WARN
  
  # Ignore messages below warning level from Jetty, because it's a bit verbose
  log4j.logger.org.eclipse.jetty=WARN
  
  
  # Settings to quiet third party logs that are too verbose
  log4j.logger.org.spark_project.jetty=WARN
  log4j.logger.org.spark_project.jetty.util.component.AbstractLifeCycle=ERROR
  log4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=ERROR
  log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=ERROR
  log4j.logger.org.apache.parquet=ERROR
  log4j.logger.parquet=ERROR
  
  # SPARK-9183: Settings to avoid annoying messages when looking up nonexistent UDFs in SparkSQL with Hive support
  log4j.logger.org.apache.hadoop.hive.metastore.RetryingHMSHandler=FATAL
  log4j.logger.org.apache.hadoop.hive.ql.exec.FunctionRegistry=ERROR</code></pre>
  </div>
  </div>
  </p>
  
  <ol start="2">
  <li>Add these before and after sparkSession creation.</li>
  </ol>
  
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>Logger.getLogger("org").setLevel(Level.OFF)
  Logger.getLogger("akka").setLevel(Level.OFF)
  Logger.getLogger("spark").setLevel(Level.OFF)
  LogManager.getRootLogger.setLevel(Level.OFF)</code></pre>
  </div>
  </div>
  </p>
  
  <ol start="3">
  <li>Add this when creating sparkSession</li>
  </ol>
  
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>val spark: SparkSession = SparkSession.builder()
    .appName("test").master("local")
    .enableHiveSupport()
    .getOrCreate()
  spark.sparkContext.setLogLevel("OFF")</code></pre>
  </div>
  </div>
  </p>
  
  <p>I am running IntelliJ using mac OS, 
  scalaVersion=2.11.8
  sparkVersion=2.1.1</p>
  
  <p>Help me please, I have looked up and down in the Internet and still can't find anything that works</p>
  
  <p>Thanks a million!
  May.</p>