title: Spark Shell with Yarn - Error: Yarn application has already ended! It might have been killed or unable to launch application master
body: <p>As a follow-up of <a href="https://stackoverflow.com/questions/44458812/running-spark-on-yarn-on-single-node">this</a> question, I'm getting a new error when I try to use Spark 2.1.1 over Yarn (Hadoop 2.8.0) on my single node machine. If I launch the Spark Shell with</p>
  
  <pre><code>spark-shell
  </code></pre>
  
  <p>it starts without problems. After having started Hadoop with the usual <code>start-dfs.sh</code> and <code>start-yarn.sh</code>, if I use</p>
  
  <pre><code>spark-shell --master yarn
  </code></pre>
  
  <p>I get the following error:</p>
  
  <pre><code>17/06/10 12:00:07 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  17/06/10 12:00:12 ERROR SparkContext: Error initializing SparkContext.
  org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.
      at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApplication(YarnClientSchedulerBackend.scala:85)
      at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:62)
      at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:156)
      at org.apache.spark.SparkContext.&lt;init&gt;(SparkContext.scala:509)
      at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2320)
      at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
      at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
      at org.apache.spark.repl.Main$.createSparkSession(Main.scala:96)
      at $line3.$read$$iw$$iw.&lt;init&gt;(&lt;console&gt;:15)
      at $line3.$read$$iw.&lt;init&gt;(&lt;console&gt;:42)
      at $line3.$read.&lt;init&gt;(&lt;console&gt;:44)
      at $line3.$read$.&lt;init&gt;(&lt;console&gt;:48)
      at $line3.$read$.&lt;clinit&gt;(&lt;console&gt;)
      at $line3.$eval$.$print$lzycompute(&lt;console&gt;:7)
      at $line3.$eval$.$print(&lt;console&gt;:6)
      at $line3.$eval.$print(&lt;console&gt;)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:497)
      at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
      at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
      at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
      at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
      at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
      at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
      at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
      at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
      at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
      at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
      at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
      at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
      at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
      at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
      at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
      at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
      at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
      at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
      at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
      at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
      at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
      at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
      at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
      at org.apache.spark.repl.Main$.doMain(Main.scala:69)
      at org.apache.spark.repl.Main$.main(Main.scala:52)
      at org.apache.spark.repl.Main.main(Main.scala)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:497)
      at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:743)
      at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
      at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
      at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
      at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  17/06/10 12:00:12 WARN YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered!
  17/06/10 12:00:12 WARN MetricsSystem: Stopping a MetricsSystem that is not running
  </code></pre>
  
  <p>I'm new to Spark/Yarn, so I don't really know where to look for solutions. I tried what has been suggested <a href="https://stackoverflow.com/questions/42080446/setup-run-spark-spark-shell-on-yarn-client-mode">here</a> (which looks like a similar problem, since I'm using Java 8), but it didn't have any effect. I also tried using Java 7 (setting the <code>JAVA_HOME</code> variable to the JDK 7 installation folder), but I still got the same error. Do you have any ideas?</p>