<p>I have Spark configured as below in kubernetes. I followed this link to setup -
<a href="https://testdriven.io/blog/deploying-spark-on-kubernetes/" rel="nofollow noreferrer">https://testdriven.io/blog/deploying-spark-on-kubernetes/</a></p>
<p><a href="https://i.stack.imgur.com/1fDmM.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/1fDmM.png" alt="enter image description here" /></a></p>
<p>Is it possible to run jobs in cluster mode in this configuration? Or can this be used only for local development?</p>

firvin
<p>Yes you can. Not all features are present since is experimental and more details can be seen here for full config and support: <a href="https://spark.apache.org/docs/latest/running-on-kubernetes.html#running-spark-on-kubernetes" rel="nofollow noreferrer">Running Spark on Kubernetes</a></p>
<p><a href="https://spark.apache.org/docs/latest/running-on-kubernetes.html#cluster-mode" rel="nofollow noreferrer">cluster mode</a></p>
<pre><code>To launch Spark Pi in cluster mode:

$ ./bin/spark-submit \
    --master k8s://https://&lt;k8s-apiserver-host&gt;:&lt;k8s-apiserver-port&gt; \
    --deploy-mode cluster \
    --name spark-pi \
    --class org.apache.spark.examples.SparkPi \
    --conf spark.executor.instances=5 \
    --conf spark.kubernetes.container.image=&lt;spark-image&gt; \
    local:///path/to/examples.jar
</code></pre>
<p>Adding details on how is used and monitored the jobs/apps when submitted and starting with: <a href="https://www.lightbend.com/blog/how-to-manage-monitor-spark-on-kubernetes-introduction-spark-submit-kubernetes-operator" rel="nofollow noreferrer">A Deeper Look At Spark-Submit</a> you can find the diagram and how does actually work. Also worth checking this info with custom images but the flow is similar <a href="https://docs.gigaspaces.com/latest/admin/kubernetes-spark-job.html" rel="nofollow noreferrer">kubernetes-spark-job</a></p>
