<p>We would like to test some <code>Spark</code> submission on a <code>Kubernetes</code> cluster;</p>
<p>However, the <a href="https://spark.apache.org/docs/2.3.0/running-on-kubernetes.html" rel="nofollow noreferrer">official documentation</a> is kind of ambiguous.</p>
<blockquote>
<p>Spark can run on clusters managed by Kubernetes. This feature makes use of native Kubernetes scheduler that has been added to Spark.</p>
<p><strong>The Kubernetes scheduler is currently experimental. In future versions, there may be behavioral changes around configuration, container images and entrypoints.</strong></p>
</blockquote>
<p>Does this mean that the <code>kubernetes</code> scheduler itself is experimental or some kind of its implementation related to spark?</p>
<p>Does it make sense to run spark on <code>Kubernetes</code> in production-grade environments?</p>

pkaramol
<ol>
<li><p>Yes, it's experimental if you are using the Spark Kubernetes scheduler like you mentioned <a href="https://spark.apache.org/docs/2.3.0/running-on-kubernetes.html" rel="nofollow noreferrer">here</a>. Use it at your own risk.</p></li>
<li><p>Not really, if you are running a standalone cluster in Kubernetes without the Kubernetes scheduler. This means create a master in a Kubernetes pod and then allocate a number of slave pods that talk to that master. Then submitting your jobs with the good old <code>spark-summit</code> without <code>--master k8s://</code> command and with the usual <code>--master spark://</code> command. The downside of this basically that your Spark cluster in Kubernetes is static.</p></li>
</ol>
