title: Remotely connect to spark on yarn cluster in client mode
body: <p>I have a remote spark on yarn cluster that if I use rstudio server(web version) hosted on that cluster to connect in client mode I can do the following:</p>
  
  <pre><code>sc &lt;- SparkR::sparkR.init(master = "yarn-client") 
  </code></pre>
  
  <p>However if I try to use rstudio on my local machine to connect to that spark cluster the same way then I have errors:</p>
  
  <pre><code>ERROR SparkContext: Error initializing SparkContext.
  org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master
  ...
  ERROR Utils: Uncaught exception in thread nioEventLoopGroup-2-2
  java.lang.NullPointerException
  ...
  ERROR RBackendHandler: createSparkContext on org.apache.spark.api.r.RRDD failed
  Error in invokeJava(isStatic = TRUE, className, methodName, ...) : 
  org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.
  </code></pre>
  
  <p>A more detailed error message on hadoop application tracking page is like this:</p>
  
  <pre><code>User:   blueivy
  Name:   SparkR
  Application Type:   SPARK
  Application Tags:   
  State:  FAILED
  FinalStatus:    FAILED
  Started:    27-Oct-2015 11:07:09
  Elapsed:    4mins, 39sec
  Tracking URL:   History
  Diagnostics:    
  Application application_1445628650748_0027 failed 2 times due to AM Container for appattempt_1445628650748_0027_000002 exited with exitCode: 10
  For more detailed output, check application tracking page:http://master:8088/proxy/application_1445628650748_0027/Then, click on links to logs of each attempt.
  Diagnostics: Exception from container-launch.
  Container id: container_1445628650748_0027_02_000001
  Exit code: 10
  Stack trace: ExitCodeException exitCode=10:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
  at org.apache.hadoop.util.Shell.run(Shell.java:455)
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
  at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
  at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
  at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
  at java.util.concurrent.FutureTask.run(FutureTask.java:267)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1143)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:618)
  at java.lang.Thread.run(Thread.java:785)
  Container exited with a non-zero exit code 10
  Failing this attempt. Failing the application. 
  </code></pre>
  
  <p>I have the same configurations and environment for hadoop and spark with remote cluster: spark 1.5.1, hadoop 2.6.0 and ubuntu 14.04. Anyone can help me find what's my mistake here?</p>