<p>I have a Scala application that is using Spark 2.1 in standalone mode. The application will run for 2 hours and finish. It should be run once a month.</p>

<p>I found several approaches to combine Spark and Kubernetes: </p>

<ol>
<li>Use Apache Spark Helm Chart: <a href="https://github.com/kubernetes/charts/tree/master/stable/spark" rel="nofollow noreferrer">https://github.com/kubernetes/charts/tree/master/stable/spark</a></li>
<li>There is a special branch of Spark for Kubernetes: <a href="https://github.com/apache-spark-on-k8s/spark" rel="nofollow noreferrer">https://github.com/apache-spark-on-k8s/spark</a> </li>
<li>Build my own Docker image of my application including the Spark binary: <a href="http://blog.madhukaraphatak.com/scaling-spark-with-kubernetes-part-5/" rel="nofollow noreferrer">http://blog.madhukaraphatak.com/scaling-spark-with-kubernetes-part-5/</a> Code example: <a href="https://github.com/phatak-dev/kubernetes-spark" rel="nofollow noreferrer">https://github.com/phatak-dev/kubernetes-spark</a></li>
</ol>

<p>Most of the documentation describe how to run a Spark cluster on Kubernetes. What is the approach for running Spark standalone on Kubernetes?</p>

<p>Check my <a href="https://github.com/radanalyticsio/spark-operator" rel="nofollow noreferrer">https://github.com/radanalyticsio/spark-operator</a></p>

<p>It deploys standalone spark on Kubernetes and OpenShift and supports also spark-on-k8s native scheduler. The default Spark version is 2.4.0</p>

<p>You can find the very quick start in the project's readme file, however here is a way to deploy the spark cluster using the operator:</p>

<pre><code># create operator
kubectl apply -f https://raw.githubusercontent.com/radanalyticsio/spark-operator/master/manifest/operator.yaml

# create cluster
cat &lt;&lt;EOF | kubectl apply -f -
apiVersion: v1
kind: SparkCluster
metadata:
  name: my-cluster
spec:
  worker:
    instances: "2"
EOF
</code></pre>
