<pre><code>cpchung:small$ kubectl get pods
NAME                             READY   STATUS    RESTARTS   AGE
simpleapp-1581273108724-driver   1/1     Running   0          93m
simpleapp-1581273108724-exec-1   0/1     Pending   0          93m

</code></pre>

<p>My kubectl version:</p>

<pre><code>cpchung:spark-2.4.4-bin-hadoop2.7$ kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.2", GitCommit:"59603c6e503c87169aea6106f57b9f242f64df89", GitTreeState:"clean", BuildDate:"2020-01-21T22:17:28Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.0", GitCommit:"0ed33881dc4355495f623c6f22e7dd0b7632b7c0", GitTreeState:"clean", BuildDate:"2018-09-27T16:55:41Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
</code></pre>

<p>from kubectl logs :</p>

<pre><code>20/02/08 21:06:09 INFO DAGScheduler: Submitting 1 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[7] at count at SimpleApp.java:12) (first 15 tasks are for partitions Vector(0))
20/02/08 21:06:09 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
20/02/08 21:06:24 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

</code></pre>

<p>After running <code>minikube start --kubernetes-version v1.12.0 --cpus 4 --memory 8192</code> with more cpus and memories , the pod description for the executor pod is showing this:</p>

<pre><code>Events:
  Type     Reason            Age                  From               Message
  ----     ------            ----                 ----               -------
  Warning  FailedScheduling  83s (x541 over 91m)  default-scheduler  0/1 nodes are available: 1 Insufficient cpu, 1 Insufficient memory.


</code></pre>

<p>For the driver pod:</p>

<pre><code>Events:          &lt;none&gt;

</code></pre>

<p>What could go wrong here?</p>

<p>Here is the repo which can reproduce the problem:
<a href="https://github.com/pokeai/k8s_setup/tree/master/small" rel="nofollow noreferrer">https://github.com/pokeai/k8s_setup/tree/master/small</a></p>

cpchung
<p>As described in the description of your Pending pod: <code>0/1 nodes are available: 1 Insufficient cpu, 1 Insufficient memory.</code>, which means that you cluster has not enough resources to run the Pod.</p>

<p>To overcome it you have 2 options:</p>

<ul>
<li>Increase the resources amount of the Minikube cluster; you can do that by adding additional parameters to the cluster creation command: <code>minikube start ... --cpus 4 --memory 8192</code></li>
<li>Change CPU and memory requests for the Spark Pods; it's a bit trickier and to get the idea how Spark Driver and Executor resources requests are set please refer to <a href="https://github.com/apache/spark/blob/v2.4.4/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStep.scala" rel="nofollow noreferrer">driver</a> and <a href="https://github.com/apache/spark/blob/v2.4.4/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala" rel="nofollow noreferrer">executor</a> spec setups.</li>
</ul>

<p>Note that you cannot change Driver Pod CPU requests, its value is always set to 1 core. Also take into account that there is some memory <a href="https://github.com/apache/spark/blob/v2.4.4/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStep.scala#L54" rel="nofollow noreferrer">overhead</a> added to the requested amount for both driver and executor Pods.</p>

<p>Hope it helps.</p>
