title: Spark High Availability
body: <p>I`m using spark 1.2.1 on three nodes that run three workers with slave configuration and run <strong>daily</strong> jobs by using:</p>
  
  <pre><code>./spark-1.2.1/sbin/start-all.sh
  
  //crontab configuration:
  ./spark-1.2.1/bin/spark-submit --master spark://11.11.11.11:7077 --driver-class-path home/ubuntu/spark-cassandra-connector-java-assembly-1.2.1-FAT.jar --class "$class" "$jar"
  </code></pre>
  
  <p>I want to keep spark master and slave workers available at all times, and even if it fail I need it to be restarted like a <strong><em>service</em></strong> (like cassandra does). </p>
  
  <p>Is there any way to do it?</p>
  
  <p><strong>EDIT:</strong></p>
  
  <p>I looked into start-all.sh script and it is only contains the setup for start-master.sh script and start-slaves.sh script.
  I tried to create a <strong>supervisor</strong> configuration file for it and only get the below errors:</p>
  
  <pre><code>11.11.11.11: ssh: connect to host 11.11.11.12 port 22: No route to host
  11.11.11.13: org.apache.spark.deploy.worker.Worker running as process 14627. Stop it first.
  11.11.11.11: ssh: connect to host 11.11.11.12 port 22: No route to host
  11.11.11.12: ssh: connect to host 11.11.11.13 port 22: No route to host
  11.11.11.11: org.apache.spark.deploy.worker.Worker running as process 14627. Stop it first.
  11.11.11.12: ssh: connect to host 11.11.11.12 port 22: No route to host
  11.11.11.13: ssh: connect to host 11.11.11.13 port 22: No route to host
  11.11.11.11: org.apache.spark.deploy.worker.Worker running as process 14627. Stop it first.
  </code></pre>