title: Unable to access hive using spark
body: <p>I'm trying to access hive through spark-shell. I'm using Windows 8.
  Hive version - 2.1.1
  Spark version - 2.4.0
  Hadoop version - 2.7.7
  To begin with I've entered following code in Spark-shell</p>
  
  <pre><code>    import org.apache.spark.sql.hive.HiveContext
      import org.apache.spark.sql.SparkSession
      val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)
      scala&gt; sqlContext.sql("CREATE TABLE IF NOT EXISTS employees2(id INT,name STRING, age INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n'")
  </code></pre>
  
  <p>My hive-site.xml in the /spark/conf is as follows</p>
  
  <pre><code>&lt;?xml version="1.0"?&gt;
  &lt;?xml-stylesheet type="text/xsl" href="configuration.xsl"?&gt;
  &lt;configuration&gt;&lt;property&gt; &lt;name&gt;javax.jdo.option.ConnectionURL&lt;/name&gt; 
  &lt;value&gt;jdbc:derby://localhost:1527/metastore_db;create=true&lt;/value&gt; 
  &lt;description&gt;JDBC connect string for a JDBC metastore&lt;/description&gt;
  &lt;/property&gt;
  &lt;property&gt;
  &lt;name&gt;hive.metastore.uris&lt;/name&gt;
  &lt;value&gt;thrift://localhost:9083&lt;/value&gt;
  &lt;description&gt;URI for client to contact metastore server&lt;/description&gt;
  &lt;/property&gt;
  &lt;property&gt; 
  &lt;name&gt;javax.jdo.option.ConnectionDriverName&lt;/name&gt; 
  &lt;value&gt;org.apache.derby.jdbc.ClientDriver&lt;/value&gt; 
  &lt;description&gt;Driver class name for a JDBC metastore&lt;/description&gt;
  &lt;/property&gt;
  &lt;property&gt; 
  &lt;name&gt;hive.server2.enable.impersonation&lt;/name&gt; 
  &lt;description&gt;Enable user impersonation for HiveServer2&lt;/description&gt;
  &lt;value&gt;true&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
  &lt;name&gt;hive.server2.authentication&lt;/name&gt; 
  &lt;value&gt;NONE&lt;/value&gt;
  &lt;description&gt; Client authentication types. NONE: no authentication check LDAP: LDAP/AD based authentication KERBEROS: Kerberos/GSSAPI authentication CUSTOM: Custom authentication provider (Use with property hive.server2.custom.authentication.class) &lt;/description&gt;
  &lt;/property&gt;
  &lt;property&gt;
  &lt;name&gt;datanucleus.autoCreateTables&lt;/name&gt;
  &lt;value&gt;True&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
  &lt;name&gt;hive.metastore.schema.verification&lt;/name&gt;
  &lt;value&gt;False&lt;/value&gt;
  &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>Error logs that I'm getting is as follows</p>
  
  <pre><code>    2018-12-26 18:34:01 WARN  HiveConf:2753 - **HiveConf of name hive.server2.enable.impersonation does not exist**
      2018-12-26 18:34:03 WARN  metastore:428 - **Failed to connect to the MetaStore Server...**
      2018-12-26 18:34:04 WARN  metastore:428 - Failed to connect to the MetaStore Server...
     2018-12-26 18:34:01 WARN  HiveConf:2753 - HiveConf of name hive.server2.enable.impersonation does not exist
  2018-12-26 18:34:03 WARN  metastore:428 - Failed to connect to the MetaStore Server...
  2018-12-26 18:34:04 WARN  metastore:428 - Failed to connect to the MetaStore Server...
  77)
    at org.apache.spark.sql.catalyst.catalog.SessionCatalog.createTable(SessionCatalog.scala:311)
    at org.apache.spark.sql.execution.command.CreateTableCommand.run(tables.scala:128)
    at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
    at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
    at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
    at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:195)
    at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:195)
    at org.apache.spark.sql.Dataset$$anonfun$53.apply(Dataset.scala:3365)
    at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78)
    at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
    at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
    at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3364)
    at org.apache.spark.sql.Dataset.&lt;init&gt;(Dataset.scala:195)
    at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:80)
    at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642)
    at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:694)
    ... 49 elided
  Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
    at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:183)
    at org.apache.spark.sql.hive.client.HiveClientImpl.&lt;init&gt;(HiveClientImpl.scala:117)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:272)
    at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:384)
    at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:286)
    at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
    at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
    at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:215)
    at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:215)
    at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:215)
    at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
    ... 75 more
  Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.&lt;init&gt;(RetryingMetaStoreClient.java:86)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
    at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
    at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
    ... 90 more
  Caused by: java.lang.reflect.InvocationTargetException: org.apache.hadoop.hive.metastore.api.MetaException: Could not connect to meta store using any
  of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused: connect
          at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
          at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:420)
          at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.&lt;init&gt;(HiveMetaStoreClient.java:236)
          at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.&lt;init&gt;(SessionHiveMetaStoreClient.java:74)
          at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
          at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
          at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
          at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
          at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
          at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.&lt;init&gt;(RetryingMetaStoreClient.java:86)
          at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
          at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
          at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
          at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
          at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
          at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:183)
          at org.apache.spark.sql.hive.client.HiveClientImpl.&lt;init&gt;(HiveClientImpl.scala:117)
          at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
          at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
          at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
          at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
          at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:272)
          at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:384)
          at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:286)
          at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
          at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
          at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:215)
          at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:215)
          at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:215)
          at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
          at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:214)
          at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114)
          at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102)
          at org.apache.spark.sql.hive.HiveSessionStateBuilder.org$apache$spark$sql$hive$HiveSessionStateBuilder$$externalCatalog(HiveSessionStateBuilde
  r.scala:39)
          at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anonfun$1.apply(HiveSessionStateBuilder.scala:54)
          at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anonfun$1.apply(HiveSessionStateBuilder.scala:54)
          at org.apache.spark.sql.catalyst.catalog.SessionCatalog.externalCatalog$lzycompute(SessionCatalog.scala:90)
          at org.apache.spark.sql.catalyst.catalog.SessionCatalog.externalCatalog(SessionCatalog.scala:90)
          at org.apache.spark.sql.catalyst.catalog.SessionCatalog.databaseExists(SessionCatalog.scala:238)
          at org.apache.spark.sql.catalyst.catalog.SessionCatalog.org$apache$spark$sql$catalyst$catalog$SessionCatalog$$requireDbExists(SessionCatalog.s
  cala:177)
          at org.apache.spark.sql.catalyst.catalog.SessionCatalog.createTable(SessionCatalog.scala:311)
          at org.apache.spark.sql.execution.command.CreateTableCommand.run(tables.scala:128)
          at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
          at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
          at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
          at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:195)
          at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:195)
          at org.apache.spark.sql.Dataset$$anonfun$53.apply(Dataset.scala:3365)
          at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78)
          at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
          at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
          at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3364)
          at org.apache.spark.sql.Dataset.&lt;init&gt;(Dataset.scala:195)
          at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:80)
          at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642)
          at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:694)
          at &lt;init&gt;(&lt;console&gt;:28)
          at &lt;init&gt;(&lt;console&gt;:33)
          at &lt;init&gt;(&lt;console&gt;:35)
          at &lt;init&gt;(&lt;console&gt;:37)
          at &lt;init&gt;(&lt;console&gt;:39)
          at &lt;init&gt;(&lt;console&gt;:41)
          at &lt;init&gt;(&lt;console&gt;:43)
          at &lt;init&gt;(&lt;console&gt;:45)
          at &lt;init&gt;(&lt;console&gt;:47)
          at .&lt;init&gt;(&lt;console&gt;:51)
          at .&lt;clinit&gt;(&lt;console&gt;)
          at .$print$lzycompute(&lt;console&gt;:7)
          at .$print(&lt;console&gt;:6)
          at $print(&lt;console&gt;)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:793)
          at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1054)
          at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:645)
          at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:644)
          at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
          at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
          at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:644)
          at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:576)
          at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:572)
          at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:819)
          at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:691)
          at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:404)
          at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:425)
          at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:285)
          at org.apache.spark.repl.SparkILoop.runClosure(SparkILoop.scala:159)
          at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:182)
          at org.apache.spark.repl.Main$.doMain(Main.scala:78)
          at org.apache.spark.repl.Main$.main(Main.scala:58)
          at org.apache.spark.repl.Main.main(Main.scala)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
          at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
          at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
          at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
          at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
          at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
          at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
          at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  Caused by: java.net.ConnectException: Connection refused: connect
          at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
          at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
          at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
          at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
          at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
          at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
          at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
          at java.net.Socket.connect(Socket.java:589)
          at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
          ... 104 more
  
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
    ... 96 more
  Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Could not connect to meta store using any of the URIs provided. Most recent failure: or
  g.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused: connect
          at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
          at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:420)
          at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.&lt;init&gt;(HiveMetaStoreClient.java:236)
          at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.&lt;init&gt;(SessionHiveMetaStoreClient.java:74)
          at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
          at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
          at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
          at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
          at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
          at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.&lt;init&gt;(RetryingMetaStoreClient.java:86)
          at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
          at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
          at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
          at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
          at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
          at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:183)
          at org.apache.spark.sql.hive.client.HiveClientImpl.&lt;init&gt;(HiveClientImpl.scala:117)
          at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
          at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
          at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
          at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
          at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:272)
          at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:384)
          at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:286)
          at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
          at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
          at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:215)
          at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:215)
          at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:215)
          at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
          at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:214)
          at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114)
          at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102)
          at org.apache.spark.sql.hive.HiveSessionStateBuilder.org$apache$spark$sql$hive$HiveSessionStateBuilder$$externalCatalog(HiveSessionStateBuilde
  r.scala:39)
          at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anonfun$1.apply(HiveSessionStateBuilder.scala:54)
          at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anonfun$1.apply(HiveSessionStateBuilder.scala:54)
          at org.apache.spark.sql.catalyst.catalog.SessionCatalog.externalCatalog$lzycompute(SessionCatalog.scala:90)
          at org.apache.spark.sql.catalyst.catalog.SessionCatalog.externalCatalog(SessionCatalog.scala:90)
          at org.apache.spark.sql.catalyst.catalog.SessionCatalog.databaseExists(SessionCatalog.scala:238)
          at org.apache.spark.sql.catalyst.catalog.SessionCatalog.org$apache$spark$sql$catalyst$catalog$SessionCatalog$$requireDbExists(SessionCatalog.s
  cala:177)
          at org.apache.spark.sql.catalyst.catalog.SessionCatalog.createTable(SessionCatalog.scala:311)
          at org.apache.spark.sql.execution.command.CreateTableCommand.run(tables.scala:128)
          at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
          at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
          at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
          at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:195)
          at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:195)
          at org.apache.spark.sql.Dataset$$anonfun$53.apply(Dataset.scala:3365)
          at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78)
          at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
          at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
          at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3364)
          at org.apache.spark.sql.Dataset.&lt;init&gt;(Dataset.scala:195)
          at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:80)
          at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642)
          at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:694)
          at &lt;init&gt;(&lt;console&gt;:28)
          at &lt;init&gt;(&lt;console&gt;:33)
          at &lt;init&gt;(&lt;console&gt;:35)
          at &lt;init&gt;(&lt;console&gt;:37)
          at &lt;init&gt;(&lt;console&gt;:39)
          at &lt;init&gt;(&lt;console&gt;:41)
          at &lt;init&gt;(&lt;console&gt;:43)
          at &lt;init&gt;(&lt;console&gt;:45)
          at &lt;init&gt;(&lt;console&gt;:47)
          at .&lt;init&gt;(&lt;console&gt;:51)
          at .&lt;clinit&gt;(&lt;console&gt;)
          at .$print$lzycompute(&lt;console&gt;:7)
          at .$print(&lt;console&gt;:6)
          at $print(&lt;console&gt;)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:793)
          at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1054)
          at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:645)
          at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:644)
          at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
          at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
          at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:644)
          at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:576)
          at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:572)
          at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:819)
          at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:691)
          at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:404)
          at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:425)
          at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:285)
          at org.apache.spark.repl.SparkILoop.runClosure(SparkILoop.scala:159)
          at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:182)
          at org.apache.spark.repl.Main$.doMain(Main.scala:78)
          at org.apache.spark.repl.Main$.main(Main.scala:58)
          at org.apache.spark.repl.Main.main(Main.scala)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
          at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
          at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
          at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
          at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
          at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
          at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
          at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  Caused by: java.net.ConnectException: Connection refused: connect
          at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
          at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
          at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
          at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
          at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
          at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
          at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
          at java.net.Socket.connect(Socket.java:589)
          at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
          ... 104 more
  
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:466)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.&lt;init&gt;(HiveMetaStoreClient.java:236)
    at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.&lt;init&gt;(SessionHiveMetaStoreClient.java:74)
    ... 101 more
  </code></pre>
  
  <p>How to solve it. Am I missing something or is my configuration wrong?</p>