title: Spark + Sentry + Kerberos: org.apache.hadoop.security.UserGroupInformation.doAs()
body: <p>Getting following error stack:</p>
  
  <blockquote>
    <p>The Spark session could not be created in the cluster: 
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:160)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) 
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120) 
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ) 
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:466)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:234)
        at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
        ... 35 more</p>
  </blockquote>
  
  <p>My understanding that hive.server2.enable.impersonation and hive.server2.enable.doAs should be enabled to make UserGroupInformation.doAs() work?</p>
  
  <p>When I try to enable these parameters, Cloudera Manager shows error </p>
  
  <blockquote>
    <p>Hive Impersonation is enabled for Hive Server2 role 'HiveServer2
    (hostname)'.  Hive Impersonation should be disabled to enable Hive
    authorization using Sentry</p>
  </blockquote>
  
  <p>So Spark-Hive conflicts with Sentry!?</p>
  
  <p>Environment: Hue 3.9 Spark Notebooks + Livy Server (built from master). CDH 5.5.</p>
  
  <p>This is a kerberized cluster with Sentry. </p>
  
  <p>I was using hue's keytab as hue user is normally (by default in CDH) is allowed to impersonate to other users.
  So very convenient for Spark Notebooks.</p>
  
  <p>Any information to help solve this will be highly appreciated.</p>