<p>I currently own only one computer, and I won't have another.</p>
<ol>
<li><p>I run <em>Spark</em> on its CPU cores : <code>master=local[5]</code>, using it directly : I set <code>spark-core</code> and <code>spark-sql</code> for dependencies, do quite no other configuration, and my programs start immediately. It's confortable, of course.</p>
</li>
<li><p>But should I attempt to create an architecture with a master and some workers by the mean of <em>Docker</em> containers or <em>minikube</em> (<em>Kubernetes</em>) on my computer ?</p>
</li>
</ol>
<p>Will solution <strong>#2</strong> - with all the settings it requires - reward me with better performances, because <em>Spark</em> is truly designed to work that way, even on a single computer,</p>
<p>or will I loose some time, because the mode I'm currently running it, without network usage, without need of data locality will always give me better performances, and solution <strong>#1</strong> will always be the best on a single computer ?</p>
<p>My hypothesis is that <strong>#1</strong> is fine. But I have no true measurement for that. No source of comparison. Who have experienced the two manners of doing things on a sigle computer ?</p>

<p>It really depends on your goals - if you always will run your Spark code on the single node with local master, then just use it.  But if you intend to run your resulting code in the distributed mode on multiple machines, then emulating cluster with Docker could be useful, as you'll get your code running in truly distributed manner, and you'll able to find problems that not always are found when you run your code with the local master.</p>
<p>Instead of direct Docker usage (that could be tricky to setup, although it's still possible), maybe you can consider to use Spark on Kubernetes, for example, via minikube - there is a plenty of articles found by Google on this topic.</p>
