title: Trying to execute a spark sql query from a UDF
body: <p>I am trying to write a inline function in spark framework using scala which will take a string input, execute a sql statement and return me a String value</p>
  
  <pre><code>val testfunc: (String=&gt;String)= (arg1:String) =&gt; 
  {val k = sqlContext.sql("""select c_code from r_c_tbl where x_nm = "something" """)                               
   k.head().getString(0)
  }
  </code></pre>
  
  <p>I am registering this scala function as an UDF</p>
  
  <pre><code>   val testFunc_test = udf(testFunc)
  </code></pre>
  
  <p>I have a dataframe over a hive table</p>
  
  <pre><code>    val df = sqlContext.table("some_table")
  </code></pre>
  
  <p>Then I am calling the udf in a withColumn and trying to save it in a new dataframe.</p>
  
  <pre><code>    val new_df = df.withColumn("test", testFunc_test($"col1"))
  </code></pre>
  
  <p>But everytime i try do this i get an error</p>
  
  <pre><code>16/08/10 21:17:08 WARN TaskSetManager: Lost task 0.0 in stage 1.0 (TID 1,       10.0.1.5): java.lang.NullPointerException
      at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:41)
      at org.apache.spark.sql.DataFrame.withNewExecutionId(DataFrame.scala:2086)
      at org.apache.spark.sql.DataFrame.foreach(DataFrame.scala:1434)
  </code></pre>
  
  <p>I am relatively new to spark and scala . But I am not sure why this code should not run. Any insights or an work around will be highly appreciated.</p>
  
  <p>Please note that I have not pasted the whole error stack . Please let me know if it is required.</p>