<p>I am trying to follow this <a href="https://weidongzhou.wordpress.com/tag/spark/" rel="nofollow noreferrer">link</a> to run my jar as a spark job on google kubernetes engine. 
I have tried some things:-</p>

<ul>
<li>Copied my jar in /examples/jars and tried running.</li>
</ul>

<p>However when I run </p>

<p><code>sudo /opt/bin/spark-submit --master k8s://https://35.192.214.68 --deploy-mode cluster --name sparkIgnite --class org.blk.igniteSparkResearch.ScalarSharedRDDExample --conf spark.executor.instances=3 --conf spark.app.name=sharedSparkIgnite --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark --conf spark.kubernetes.container.image=us.gcr.io/nlp-research-198620/spark:k8s-spark-2.3 local:///opt/spark/examples/jars/igniteSpark-1.0-SNAPSHOT-jar-with-dependencies.jar
</code></p>

<p>and check the logs of my pod, I get </p>

<p><code>++ id -u
+ myuid=0
++ id -g
+ mygid=0
++ getent passwd 0
+ uidentry=root:x:0:0:root:/root:/bin/ash
+ '[' -z root:x:0:0:root:/root:/bin/ash ']'
+ SPARK_K8S_CMD=driver
+ '[' -z driver ']'
+ shift 1
+ SPARK_CLASSPATH=':/opt/spark/jars/*'
+ env
+ grep SPARK_JAVA_OPT_
+ sed 's/[^=]*=\(.*\)/\1/g'
+ readarray -t SPARK_JAVA_OPTS
+ '[' -n /opt/spark/examples/jars/igniteSpark-1.0-SNAPSHOT-jar-with-dependencies.jar:/opt/spark/examples/jars/igniteSpark-1.0-SNAPSHOT-jar-with-dependencies.jar ']'
+ SPARK_CLASSPATH=':/opt/spark/jars/*:/opt/spark/examples/jars/igniteSpark-1.0-SNAPSHOT-jar-with-dependencies.jar:/opt/spark/examples/jars/igniteSpark-1.0-SNAPSHOT-jar-with-dependencies.jar'
+ '[' -n '' ']'
+ case "$SPARK_K8S_CMD" in
+ CMD=(${JAVA_HOME}/bin/java "${SPARK_JAVA_OPTS[@]}" -cp "$SPARK_CLASSPATH" -Xms$SPARK_DRIVER_MEMORY -Xmx$SPARK_DRIVER_MEMORY -Dspark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS $SPARK_DRIVER_CLASS $SPARK_DRIVER_ARGS)
+ exec /sbin/tini -s -- /usr/lib/jvm/java-1.8-openjdk/bin/java -Dspark.kubernetes.driver.pod.name=sparkignite-a20d8d85b6b6389293be4b1fe8a12803-driver -Dspark.driver.port=7078 -Dspark.jars=/opt/spark/examples/jars/igniteSpark-1.0-SNAPSHOT-jar-with-dependencies.jar,/opt/spark/examples/jars/igniteSpark-1.0-SNAPSHOT-jar-with-dependencies.jar -Dspark.app.name=sparkIgnite -Dspark.driver.blockManager.port=7079 -Dspark.driver.host=sparkignite-a20d8d85b6b6389293be4b1fe8a12803-driver-svc.default.svc -Dspark.kubernetes.authenticate.driver.serviceAccountName=spark -Dspark.master=k8s://https://35.192.214.68 -Dspark.app.id=spark-b0523468df0b4751a2d94c3b9513c19f -Dspark.submit.deployMode=cluster -Dspark.executor.instances=3 -Dspark.kubernetes.container.image=us.gcr.io/nlp-research-198620/spark:k8s-spark-2.3 -Dspark.kubernetes.executor.podNamePrefix=sparkignite-a20d8d85b6b6389293be4b1fe8a12803 -cp ':/opt/spark/jars/*:/opt/spark/examples/jars/igniteSpark-1.0-SNAPSHOT-jar-with-dependencies.jar:/opt/spark/examples/jars/igniteSpark-1.0-SNAPSHOT-jar-with-dependencies.jar' -Xms1g -Xmx1g -Dspark.driver.bindAddress=10.8.2.54 org.blk.igniteSparkResearch.ScalarSharedRDDExample
Error: Could not find or load main class org.blk.igniteSparkResearch.ScalarSharedRDDExample</code></p>

<p>I am only able to run jars which are already packed with default spark 2.30 version. How to run any custom jar on spark is I want to know.</p>

<p>Thanks in advance.</p>

wadhwasahil
<p>It seems I need to create an image wherein I need to copy my jar and then run the image. When I follow this, it works.</p>
