title: Zeppelin+Spark+Kubernetes: Let Zeppelin Job run on existing Spark Cluster
body: <p><strong>In a k8s cluster. How do you configure zeppelin to run spark jobs in an existing spark cluster instead of spinning up a new pod?</strong></p>
  <p>I've got a k8s cluster up and running in which I want to run Spark with Zeppelin.</p>
  <p>Spark is deployed using the official bitnami/spark helm chart (v 3.0.0). I got one Master and two Worker pods running fine, everything good.</p>
  <p>Zeppelin is deployed with the <code>zeppelin-server.yaml</code> from the official apache-zeppelin github.</p>
  <p>I've build my own zeppelin container without much modification from apache/zeppelin:0.9.0..</p>
  <p>Short pseudo Dockerfile:</p>
  <pre><code>FROM bitnami/spark:3.0.0 AS spark 
  FROM apache/zeppelin:0.9-0 AS Zeppelin 
  COPY --from spark /opt/btinami/spark/ /opt/bitnami/spark  
  RUN Install kubectl 
  END
  </code></pre>
  <p>I modified <code>zeppelin-server.yaml</code> slightly. (Image, imagePullSecret, setting spark master to the headless Service DNS of spark master)</p>
  <p>Now I want my zeppelin jobs to run on my existing spark cluster --- with no success.</p>
  <p>When I'm submitting zeppelin jobs (for the spark interpreter), zeppelin fires up a new spark pod and solely works with this one. Spark interpreter settings are like they should be. spark master url is set (<code>spark://\&lt;master-url\&gt;:\&lt;master-port\&gt;</code>), spark home as well.</p>
  <p>While this is kind of a sweet behaviour, it's not what I want.</p>
  <p>What I want (and what my question is) is: I want my zeppelin pod to submit the spark jobs to the existing cluster - not fire up a new pod. I am PRETTY sure that there has to be some <code>config/env/whatever</code> that I have to set but I simply can't find it.</p>
  <p>So, I wanna ask: Is there anyone out there, who knows how to run zeppelin spark jobs on an existing spark cluster? I thought setting the spark master should do the job...</p>
  <p>Kind regards
  Bob</p>