<p>Currently I've deployed Spark to <code>minikube</code> in my local machine. Pod and its containers are up and running, and I've already checked that port <code>7077</code> is listening from the host machine (local machine).</p>
<p>Now I want to <code>spark-submit</code> from the host machine. Thus, I've downloaded Spark's binaries and I've moved them to <code>c:\bin\spark-3.2.1-bin-hadoop3.2</code>, and I've added <code>c:\bin\spark-3.2.1-bin-hadoop3.2\bin</code> to the <code>PATH</code>.</p>
<p>When I run <code>spark-submit</code>as follows...</p>
<pre><code>spark-submit --class org.apache.spark.deploy.dotnet.DotnetRunner --master spark.local:7077 microsoft-spark-3-2_2.12-2.1.1.jar dotnet C:\projects\xxx\xxx-dotnet-solution\xx-services/infrastructure/etl-service/Spark/bin/Debug/netcoreapp3.1/xxx.xx.Services.Infraestructure.ETLService.Spark.dll
</code></pre>
<p>...I get the following error <code>org.apache.spark.SparkException: Could not parse Master URL: 'spark.local'</code>.</p>
<p>I'm not sure if I'm mistaken, and maybe the issue is I can't <code>spark-submit</code> from my local machine to the remote Spark. Is this ever possible?</p>

Matías Fidemraizer
<p>According to the <a href="https://spark.apache.org/docs/latest/submitting-applications.html#master-urls" rel="nofollow noreferrer">master URL docs</a> that parameter accepts either some keywords like <code>local</code>, <code>yarn</code> or specific URL protocols, <code>spark://</code>, <code>mesos://</code>, <code>k8s://</code>. It can't handle machine or domain names.</p>
<p>In the <a href="https://learn.microsoft.com/en-us/dotnet/spark/tutorials/get-started?tabs=windows" rel="nofollow noreferrer">.NET for Apache Spark tutorial</a> the command uses the <code>local</code> keyword, not a host name :</p>
<pre><code>spark-submit ^
--class org.apache.spark.deploy.dotnet.DotnetRunner ^
--master local ^
microsoft-spark-3-0_2.12-&lt;version&gt;.jar ^
dotnet MySparkApp.dll &lt;path-of-input.txt&gt;
</code></pre>
<p>From the docs :</p>
<blockquote>
<p>The master URL passed to Spark can be in one of the following formats:</p>
<div class="s-table-container">
<table class="s-table">
<thead>
<tr>
<th>Master URL</th>
<th>Meaning</th>
</tr>
</thead>
<tbody>
<tr>
<td>local</td>
<td>Run Spark locally with one worker thread (i.e. no parallelism at all).</td>
</tr>
<tr>
<td>local[K]</td>
<td>Run Spark locally with K worker threads (ideally, set this to the number of cores on your machine).</td>
</tr>
<tr>
<td>local[K,F]</td>
<td>Run Spark locally with K worker threads and F maxFailures (see spark.task.maxFailures for an explanation of this variable).</td>
</tr>
<tr>
<td>local[*]</td>
<td>Run Spark locally with as many worker threads as logical cores on your machine.</td>
</tr>
<tr>
<td>local[*,F]</td>
<td>Run Spark locally with as many worker threads as logical cores on your machine and F maxFailures.</td>
</tr>
<tr>
<td>local-cluster[N,C,M]</td>
<td>Local-cluster mode is only for unit tests. It emulates a distributed cluster in a single JVM with N number of workers, C cores per worker and M MiB of memory per worker.</td>
</tr>
<tr>
<td>spark://HOST:PORT</td>
<td>Connect to the given Spark standalone cluster master. The port must be whichever one your master is configured to use, which is 7077 by default.</td>
</tr>
<tr>
<td>spark://HOST1:PORT1,HOST2:PORT2</td>
<td>Connect to the given Spark standalone cluster with standby masters with Zookeeper. The list must have all the master hosts in the high availability cluster set up with Zookeeper. The port must be whichever each master is configured to use, which is 7077 by default.</td>
</tr>
<tr>
<td>mesos://HOST:PORT</td>
<td>Connect to the given Mesos cluster. The port must be whichever one your is configured to use, which is 5050 by default. Or, for a Mesos cluster using ZooKeeper, use mesos://zk://.... To submit with --deploy-mode cluster, the HOST:PORT should be configured to connect to the MesosClusterDispatcher.</td>
</tr>
<tr>
<td>yarn</td>
<td>Connect to a YARN cluster in client or cluster mode depending on the value of --deploy-mode. The cluster location will be found based on the HADOOP_CONF_DIR or YARN_CONF_DIR variable.</td>
</tr>
<tr>
<td>k8s://HOST:PORT</td>
<td>Connect to a Kubernetes cluster in client or cluster mode depending on the value of --deploy-mode. The HOST and PORT refer to the Kubernetes API Server. It connects using TLS by default. In order to force it to use an unsecured connection, you can use k8s://http://HOST:PORT.</td>
</tr>
</tbody>
</table>
</div></blockquote>
