<p>Dockerfile:</p>

<pre><code>FROM openjdk:8-alpine

RUN apk update &amp;&amp; \
        apk add curl bash procps

ENV SPARK_VER 2.1.1
ENV HADOOP_VER 2.7
ENV SPARK_HOME /opt/spark

# Get Spark from US Apache mirror.
RUN mkdir -p /opt &amp;&amp; \
    cd /opt &amp;&amp; \
    curl http://www.us.apache.org/dist/spark/spark-${SPARK_VER}/spark-${SPARK_VER}-bin-hadoop${HADOOP_VER}.tgz | \
        tar -zx &amp;&amp; \
    ln -s spark-${SPARK_VER}-bin-hadoop${HADOOP_VER} spark &amp;&amp; \
    echo Spark ${SPARK_VER} installed in /opt

ADD start-common.sh start-worker.sh start-master.sh /
RUN chmod +x /start-common.sh /start-master.sh /start-worker.sh
ENV PATH $PATH:/opt/spark/bin

WORKDIR $SPARK_HOME
EXPOSE 4040 6066 7077 8080

CMD ["spark-shell", "--master", "local[2]"]
</code></pre>

<p>spark-master-service.yaml:</p>

<pre><code>apiVersion: v1
kind: Service
metadata:
  name: spark-master
  labels:
    name: spark-master
spec:
  type: NodePort
  ports:
    # the port that this service should serve on
  - name: webui
    port: 8080
    targetPort: 8080
  - name: spark
    port: 7077
    targetPort: 7077
  - name: rest
    port: 6066
    targetPort: 6066
  selector:
    name: spark-master
</code></pre>

<p>spark-master.yaml:</p>

<pre><code>apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    name: spark-master
  name: spark-master
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: spark-master
    spec:
      containers:
      - name : spark-master
        imagePullPolicy: "IfNotPresent"
        image: spark-2.1.1-bin-hadoop2.7
        name: spark-master
        ports:
        - containerPort: 8080
        - containerPort: 7077
        - containerPort: 6066
        command: ["/start-master.sh"]
</code></pre>

<p>Error:
 Back-off restarting failed docker container
Error syncing pod, skipping: failed to "StartContainer" for "spark-master" with CrashLoopBackOff: "Back-off 10s restarting failed container=spark-master pod=spark-master-286530801-7qv4l_default(34fecb5e-55eb-11e7-994e-525400f3f8c2)" </p>

<p>Any idea? Thanks</p>

<p><strong>UPDATE</strong></p>

<pre><code> 2017-06-20T19:43:56.300935235Z starting org.apache.spark.deploy.master.Master, logging to /opt/spark/logs/spark--org.apache.spark.deploy.master.Master-1-spark-master-1682838347-9927h.out
2017-06-20T19:44:03.414011228Z failed to launch: nice -n 0 /opt/spark/bin/spark-class org.apache.spark.deploy.master.Master --host spark-master-1682838347-9927h --port 7077 --webui-port 8080 --ip spark-master --port 7077
2017-06-20T19:44:03.418640516Z   nohup: can't execute '--': No such file or directory
2017-06-20T19:44:03.419814788Z full log in /opt/spark/logs/spark--org.apache.spark.deploy.maste


2017-06-20T19:43:50.343251857Z starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark--org.apache.spark.deploy.worker.Worker-1-spark-worker-243125562-0lh9k.out
2017-06-20T19:43:57.450929613Z failed to launch: nice -n 0 /opt/spark/bin/spark-class org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://spark-master:7077
2017-06-20T19:43:57.465409083Z   nohup: can't execute '--': No such file or directory
2017-06-20T19:43:57.466372593Z full log in /opt/spark/logs/spark--org.apache.spark.deploy.worker.Worker-1-spark-worker-243125562-0lh9k.out
r.Master-1-spark-master-1682838347-9927h.out 
</code></pre>

<p>The version of nohup that ships with alpine does not support '--'. You need to install a gnu version of nohup through the coreutils alpine package in your docker file like this:</p>

<p><code>RUN apk --update add coreutils
</code></p>

<p>Alternatively create your own start script that runs the class directly and run that instead</p>

<p><code>/usr/spark/bin/spark-submit --class org.apache.spark.deploy.master.Master $SPARK_MASTER_INSTANCE --port $SPARK_MASTER_PORT --webui-port $SPARK_WEBUI_PORT
</code></p>
