title: Exception in thread "main" java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveSessionState':
body: <p>I'm trying to connect to Hive through Intelliji. I'm using Scala version 2.11.4,the version of spark-core,spark-hive,spark-sql are 2.1.1.  here is the code snippt I'm using to connect remotely from my windows m/c. While connecting I'm getting following error, can someone help me to address this issue? </p>
  
  <p>Note: when I read some threads, they mentioned about checking the permission of tmp, in this case, /tmp/hive/warehouse. it has appropriate permission to the user xyz I'm using to connect. using this functional id, I'm able to manually connect from one of the unix server.  I even tried with spark.sql("show databases") but it was the same error.</p>
  
  <pre><code>def main(args: Array[String]): Unit = {
      createKerberosTicket()
      val spark: SparkSession = {
            SparkSession
              .builder()
              .master("local")
              .appName("SparkHiveTest")
              .config("hive.exec.dynamic.partition.mode", "nonstrict")
              .config("hive.exec.dynamic.partition", "true")
              .config("mapreduce.job.queuename", "root.XYZ_Pool")
              .enableHiveSupport()
              .getOrCreate()
          }
      spark.sparkContext.hadoopConfiguration.addResource(new Path("core-site.xml"))
      spark.sparkContext.hadoopConfiguration.addResource(new Path("hdfs-site.xml"))
      spark.sparkContext.hadoopConfiguration.addResource(new Path("hive-site.xml"))
      spark.sparkContext.hadoopConfiguration.set("fs.hdfs.impl", classOf[DistributedFileSystem].getName)
      spark.sparkContext.hadoopConfiguration.set("fs.file.impl", classOf[LocalFileSystem].getName)
      val listOfDBs = spark.sqlContext.sql("show databases")
  }
  
  18/05/02 23:59:13 INFO SharedState: spark.sql.warehouse.dir is not set, but hive.metastore.warehouse.dir is set. Setting spark.sql.warehouse.dir to the value of hive.metastore.warehouse.dir ('/tmp/hive/warehouse').
  18/05/02 23:59:13 INFO SharedState: Warehouse path is '/tmp/hive/warehouse'.
  18/05/02 23:59:14 INFO HiveUtils: Initializing HiveMetastoreConnection version 1.2.1 using Spark classes.
  18/05/02 23:59:14 INFO metastore: Trying to connect to metastore with URI thrift://xyz.net:1234
  18/05/02 23:59:14 INFO metastore: Connected to metastore.
  18/05/02 23:59:18 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  Exception in thread "main" java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveSessionState':
      at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:981)
      at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
      at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
      at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
      at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
      at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
      at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
      at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
      at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
      at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
      at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
      at spark.SparkPlusHive$.main(SparkPlusHive.scala:25)
      at spark.SparkPlusHive.main(SparkPlusHive.scala)
  Caused by: java.lang.reflect.InvocationTargetException
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
      ... 12 more
  Caused by: java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveExternalCatalog':
      at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:169)
      at org.apache.spark.sql.internal.SharedState.&lt;init&gt;(SharedState.scala:86)
      at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
      at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
      at scala.Option.getOrElse(Option.scala:120)
      at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
      at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
      at org.apache.spark.sql.internal.SessionState.&lt;init&gt;(SessionState.scala:157)
      at org.apache.spark.sql.hive.HiveSessionState.&lt;init&gt;(HiveSessionState.scala:32)
      ... 17 more
  Caused by: java.lang.reflect.InvocationTargetException
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
      ... 25 more
  Caused by: java.lang.reflect.InvocationTargetException
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
      at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:358)
      at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:262)
      at org.apache.spark.sql.hive.HiveExternalCatalog.&lt;init&gt;(HiveExternalCatalog.scala:66)
      ... 30 more
  Caused by: java.lang.RuntimeException: java.lang.NullPointerException
      at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
      at org.apache.spark.sql.hive.client.HiveClientImpl.&lt;init&gt;(HiveClientImpl.scala:188)
      ... 38 more
  Caused by: java.lang.NullPointerException
      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
      at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
      at org.apache.hadoop.util.Shell.run(Shell.java:478)
      at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)
      at org.apache.hadoop.util.Shell.execCommand(Shell.java:831)
      at org.apache.hadoop.util.Shell.execCommand(Shell.java:814)
      at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:712)
      at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:470)
      at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:510)
      at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:488)
      at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:309)
      at org.apache.hadoop.hive.ql.session.SessionState.createPath(SessionState.java:639)
      at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:567)
      at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:508)
      ... 39 more
  </code></pre>