title: Error: You must build Spark with Hive
body: <p>I'm running Spark 1.6.2 with Hive 0.13.1 and Hadoop 2.6.0.</p>
  
  <p>I try to run this pyspark script:</p>
  
  <pre><code>import pyspark
  from pyspark.sql import HiveContext
  
  sc = pyspark.SparkContext('local[*]')
  hc = HiveContext(sc)
  hc.sql("select col from table limit 3")
  </code></pre>
  
  <p>with this command line:</p>
  
  <pre><code> ~/spark/bin/spark-submit script.py 
  </code></pre>
  
  <p>and I got this error message:</p>
  
  <pre><code> File "/usr/local/hadoop/spark/python/pyspark/sql/context.py", line &gt;552, in sql
   return DataFrame(self._ssql_ctx.sql(sqlQuery), self)
   File "/usr/local/hadoop/spark/python/pyspark/sql/context.py", line &gt;660, in _ssql_ctx
   "build/sbt assembly", e)
   Exception: ("You must build Spark with Hive. Export 'SPARK_HIVE=true' and run build/sbt assembly", Py4JJavaError(u'An error occurred while &gt;calling None.org.apache.spark.sql.hive.HiveContext.\n', JavaObject &gt;id=o18))
  </code></pre>
  
  <p>Doing what they asked, I saw a warning saying that "exporting SPARK_HIVE was deprecated" and to use instead "-Phive -Phive-thriftserver" 
  So I did this:</p>
  
  <pre><code> cd ~/spark/
   build/sbt -Pyarn -Phadoop-2.6 -Phive -Phive-thriftserver assembly
  </code></pre>
  
  <p>but I have slightly the same error:</p>
  
  <pre><code> [...]
   16/07/17 19:10:01 WARN metadata.Hive: Failed to access metastore. This class should not accessed in runtime.
   org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate      org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
       at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1236)   
   [...]
   Traceback (most recent call last):
     File "/home/hadoop/spark3/./script.py", line 6, in &lt;module&gt;
       hc.sql("select timestats from logweb limit 3")
     File "/usr/local/hadoop/spark/python/lib/pyspark.zip/pyspark/sql/context.py",      line 552, in sql
     File "/usr/local/hadoop/spark/python/lib/pyspark.zip/pyspark/sql/context.py", line 660, in _ssql_ctx
   Exception: ("You must build Spark with Hive. Export 'SPARK_HIVE=true' and run build/sbt assembly", Py4JJavaError(u'An error occurred while calling None.org.apache.spark.sql.hive.HiveContext.\n', JavaObject id=o19))
  </code></pre>
  
  <p>I searched on the web about this error, but none if the answers worked for me...</p>
  
  <p>Could someone help me please?</p>
  
  <hr>
  
  <p>I also tried to use a <a href="http://www-eu.apache.org/dist/spark/spark-1.6.2/spark-1.6.2-bin-hadoop2.6.tgz" rel="nofollow noreferrer">spark version which is suposed to work with Hadoop</a> (Suggested by <a href="https://stackoverflow.com/questions/38295021/error-you-must-build-spark-with-hive#answer-38295174">Joss</a>) , and I got this error :</p>
  
  <pre><code> Traceback (most recent call last):
   File "/home/hadoop/spark3/./script.py", line 6, in &lt;module&gt;
  hc.sql("select timestats from logweb limit 3")
   File "/usr/local/hadoop/spark/python/lib/pyspark.zip/pyspark/sql/context.py", line 552, in sql
   File "/usr/local/hadoop/spark/python/lib/pyspark.zip/pyspark/sql/context.py", line 660, in _ssql_ctx
   Exception: ("You must build Spark with Hive. Export 'SPARK_HIVE=true' and run build/sbt assembly", Py4JJavaError(u'An error occurred while calling None.org.apache.spark.sql.hive.HiveContext.\n', JavaObject id=o19))
  </code></pre>