title: Apache spark: permission denied for files uploaded to job's staging directory
body: <p>I wrote an apache spark job that uses some configuration file. When I run this job locally, it works fine. But when I submit this job to a YARN cluster, it fails with a <code>java.io.FileNotFoundException: (Permission denied)</code></p>
  
  <p>I submit my job with the following command:</p>
  
  <pre><code>bin/spark-submit --master yarn --deploy-mode cluster --num-executors 1 --files /home/user/app.conf --class org.myorg.PropTest assembly.jar
  </code></pre>
  
  <p>It uploads assembly.jar and app.conf file to a subdirectory of the .sparkStaging directory in my home directory on HDFS.</p>
  
  <p>I'm trying to access <code>app.conf</code> file on the following line:</p>
  
  <pre><code>ConfigFactory.parseFile(new File("app.conf"))
  </code></pre>
  
  <p>When I upload a file with a name other than <code>app.conf</code>, it fails with a FileNotFoundException as expected.</p>
  
  <p>But when I upload <code>app.conf</code>, it also fails with a FileNotFoundException, but with message that permission to ./app.conf is denied. So, it seems that it can gain access to this file, but can't gain required permissions.</p>
  
  <p>What can be wrong?</p>