<p>I'm running microk8s (installed via <code>snap install microk8s --classic</code>) across a local multinode cluster and <a href="https://snapcraft.io/spark-client" rel="nofollow noreferrer">spark-client</a> (installed via <code>snap install spark-client --edge</code>). Two of the nodes are WSL2 (Ubuntu) on Windows 11. I'm now adding a laptop running Ubuntu node. When I try to run <code>spark-client.spark-shell ...</code>, it will spin up executors successfully on the two WSL nodes, but it fails on the new laptop node. I know the laptop node is capable of running pods successfully because an hdfs pod is successfully running there.</p>
<p><code>spark-shell</code> immediately deletes and creates a new pod when it fails, so it's hard to see the error information. I was able to capture a log and it was only one line:
<code>error: unknown command &quot;executor&quot;, see 'pebble help'.</code></p>
<p>I notice that in the configuration for those pods there is an argument: executor, so that might be where that's coming from. But why would one node start up differently?</p>
<p>The image is: ghcr.io/canonical/charmed-spark:3.4.0-22.04_edge . I was able to run it directly.</p>
<p>Any ideas on how to resolve or further troubleshoot this?</p>
<p>Note: I did see 2 other questions here that are similar, but they do not have this particular error message, so I think this question is distinct.</p>
<p>Update: I just noticed in the node details, the sha256 is different across the nodes for that image.</p>

mentics
<p>Good news and bad news...</p>
<p>I deleted the images on all the nodes:</p>
<p><code>microk8s.ctr images delete ghcr.io/canonical/charmed-spark:3.4.0-22.04_edge</code></p>
<p>to force it to pull the latest.</p>
<p>Now all the nodes behave the same. They all fail. I'll assume that some bug was introduced last week. The latest edge release was last week (6/8), after I pulled the image for the old nodes, but before I pulled for the new node.</p>
<p>Mystery solved, though no solution, because I can't pull an older version via snap because it's on the same channel. I'll find something else to use.</p>
<p>Filed bug: <a href="https://github.com/canonical/spark-client-snap/issues/68" rel="nofollow noreferrer">https://github.com/canonical/spark-client-snap/issues/68</a>
There's an immediate workaround in the bug conversation, though it sounds like they'll have a fix out soon.</p>
