title: Spark: Not able to read data from hive tables
body: <p>I have created a maven project as <code>pom.xml</code> </p>
  
  <pre><code>&lt;spark.version&gt;1.3.0&lt;/spark.version&gt;
  &lt;dependencies&gt;
      &lt;dependency&gt;
          &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
          &lt;artifactId&gt;spark-core_2.11&lt;/artifactId&gt;
          &lt;version&gt;${spark.version}&lt;/version&gt;
          &lt;exclusions&gt;
              &lt;exclusion&gt;
                  &lt;groupId&gt;org.scala-lang&lt;/groupId&gt;
                  &lt;artifactId&gt;scala-library&lt;/artifactId&gt;
              &lt;/exclusion&gt;
          &lt;/exclusions&gt;
      &lt;/dependency&gt;
      &lt;dependency&gt;
          &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
          &lt;artifactId&gt;spark-sql_2.11&lt;/artifactId&gt;
          &lt;version&gt;${spark.version}&lt;/version&gt;
      &lt;/dependency&gt;
      &lt;!-- &lt;dependency&gt;
          &lt;groupId&gt;mysql&lt;/groupId&gt;
          &lt;artifactId&gt;mysql-connector-java&lt;/artifactId&gt;
          &lt;version&gt;5.1.6&lt;/version&gt;
      &lt;/dependency&gt; --&gt;
      &lt;dependency&gt;
          &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
          &lt;artifactId&gt;spark-hive_2.11&lt;/artifactId&gt;
          &lt;version&gt;${spark.version}&lt;/version&gt;
      &lt;/dependency&gt;
  
  &lt;/dependencies&gt;
  </code></pre>
  
  <p>My class which is reading data from hive table :</p>
  
  <pre><code>import org.apache.spark.sql.SQLContext
  import org.apache.spark.SparkConf
  import org.apache.spark.SparkContext
  import org.apache.spark.sql.hive.HiveContext
  import org.apache.spark.sql.DataFrame
  
  class SparkHive {
    def createTable = {
      val conf = new SparkConf().setMaster("local").setAppName("My First spark app")
      val sparkCtxt = new SparkContext(conf)
      val hiveContext = new HiveContext(sparkCtxt)
      hiveContext.setConf("hive.metastore.uris", "thrift://127.0.0.1:9083")
      val table = hiveContext.sql("select * from test")
      table.show()
      val gpData = table.groupBy("col1")
      println(gpData.max("col2").show())
    }
  }
  </code></pre>
  
  <p>I am using spark to read data from a table present in hive metatore but facing a very strange issues.</p>
  
  <p>I have two question as described below:</p>
  
  <p><strong>Question 1.</strong> If I use <code>&lt;spark.version&gt;1.3.0&lt;/spark.version&gt;</code> spark is able to find hive table and it is able to print data on console with the help of this line</p>
  
  <pre><code>val table = hiveContext.sql("select * from test")
  table.show()
  </code></pre>
  
  <p>but if I do filter or group by as shown in the example spark can not find <code>col1</code> and throws exception as below</p>
  
  <pre><code>Exception in thread "main" java.util.NoSuchElementException: key not found: col1#0
  </code></pre>
  
  <p>so the question is why is that if data frame is able to find that table then why it is not letting me do group by on columns and how to solve this issue??</p>
  
  <p><strong>Question 2.</strong> If I use <code>&lt;spark.version&gt;1.6.0&lt;/spark.version&gt;</code> then spark is not able even find the table present in hive metastore so now why is this behavior ???</p>
  
  <p><strong>ENVIRONMENT: CLOUDERA QUICKSTART VM 5.8.0</strong></p>