<p>I want to use Kubernetes as resource manager for spark.</p>
<p>so I wanted to submit a jar far to spark cluster with  <code>spark-submit</code>:</p>
<pre><code>./bin/spark-submit  \
    --master k8s://https://vm:6443 \
    --class com.example.WordCounter  \
    --conf spark.kubernetes.authenticate.driver.serviceAccountName=default  \
    --conf spark.kubernetes.container.image=private-docker-registery/spark/spark:3.2.1-3 \
    --conf spark.kubernetes.namespace=default \
    --conf spark.kubernetes.authenticate.submission.oauthToken=$TOKEN  \
    --conf spark.kubernetes.authenticate.caCertFile=api.cert \
    java-word-count-1.0-SNAPSHOT.jar
</code></pre>
<p>for <strong>service account</strong>:</p>
<pre><code>kubectl create serviceaccount spark
</code></pre>
<pre><code>kubectl create clusterrolebinding spark-role \
    --clusterrole=edit \
    --serviceaccount=default:default \
    --namespace=default
</code></pre>
<p>for <strong>caCertFile</strong> I used the <code>/etc/kubernetes/pki/apiserver.crt</code> content.</p>
<p>and for <strong>submission.oauthToken</strong>:</p>
<p><code>kubectl get  secret  spark-token-86tns -o yaml | grep token</code></p>
<p>and use the token part.</p>
<p>but still doesn't work and I <code>pods is forbidden: User &quot;system:anonymous&quot; cannot watch resource &quot;pods&quot; in API group &quot;&quot; in the namespace &quot;default&quot;</code> error</p>

Mohammad Abdollahzadeh
<p><code>spark.kubernetes.authenticate.caCertFile</code> has to be <code>service account</code> cert</p>
<p>and also <code>spark.kubernetes.authenticate.submission.oauthToken</code> has to be <code>service account</code> token.</p>
<p>both cert and token can be found in service account secret.</p>
<ul>
<li>be careful to decode <code>service account</code> cert and token (base64 -d).</li>
</ul>
