<p>I'm trying to connect to hdfs which is kerberized which fails with the error </p>

<blockquote>
  <p>org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled.  Available:[TOKEN, KERBEROS]</p>
</blockquote>

<p>What additional parameters do I need to add while creating the spark setup  apart from the standard thing that you need to spawn Spark worker containers?</p>

Alok Gogate
<p>Check <code>&lt;property&gt;hadoop.security.authentication&lt;property&gt;</code> in your hdfs-site.xml properties file.<br>
In your case it should have value <code>kerberos</code> or <code>token</code>.<br>
Or you can configure it from code by specifying property explicitly:  </p>

<pre><code>Configuration conf = new Configuration();
conf.set("hadoop.security.authentication", "kerberos");
</code></pre>

<p>You can find more information about secure connection to hdfs <a href="https://community.hortonworks.com/articles/56702/a-secure-hdfs-client-example.html" rel="nofollow noreferrer">here</a></p>
