<p>I have a Scala application that is using Spark 2.1 in standalone mode. The application will run for 2 hours and finish. It should be run once a month.</p>

<p>I found several approaches to combine Spark and Kubernetes: </p>

<ol>
<li>Use Apache Spark Helm Chart: <a href="https://github.com/kubernetes/charts/tree/master/stable/spark" rel="nofollow noreferrer">https://github.com/kubernetes/charts/tree/master/stable/spark</a></li>
<li>There is a special branch of Spark for Kubernetes: <a href="https://github.com/apache-spark-on-k8s/spark" rel="nofollow noreferrer">https://github.com/apache-spark-on-k8s/spark</a> </li>
<li>Build my own Docker image of my application including the Spark binary: <a href="http://blog.madhukaraphatak.com/scaling-spark-with-kubernetes-part-5/" rel="nofollow noreferrer">http://blog.madhukaraphatak.com/scaling-spark-with-kubernetes-part-5/</a> Code example: <a href="https://github.com/phatak-dev/kubernetes-spark" rel="nofollow noreferrer">https://github.com/phatak-dev/kubernetes-spark</a></li>
</ol>

<p>Most of the documentation describe how to run a Spark cluster on Kubernetes. What is the approach for running Spark standalone on Kubernetes?</p>

<p>For standalone spark on Kubernetes, the two canonical samples that exist are:</p>

<ol>
<li><a href="https://github.com/kubernetes/charts/tree/master/stable/spark" rel="nofollow noreferrer">https://github.com/kubernetes/charts/tree/master/stable/spark</a></li>
<li><a href="https://github.com/kubernetes/examples/tree/master/staging/spark" rel="nofollow noreferrer">https://github.com/kubernetes/examples/tree/master/staging/spark</a></li>
</ol>

<p>These are currently running outdated versions of Spark, and require updating to 2.1 and soon 2.2. (PRs are welcome :)). </p>

<p>The <a href="https://github.com/apache-spark-on-k8s/spark" rel="nofollow noreferrer">https://github.com/apache-spark-on-k8s/spark</a> branch is not for standalone mode, but aims to enable Spark to directly launch on Kubernetes clusters. It will eventually be merged into upstream spark. Documentation, if you wish to make use of it, is <a href="https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html" rel="nofollow noreferrer">here</a>. </p>

<p>As of now, if you want to use Spark 2.1, options are: either to compile your own image, or packaging your application with the spark distribution in <a href="https://github.com/apache-spark-on-k8s/spark" rel="nofollow noreferrer">apache-spark-on-k8s</a> </p>
