title: Why does PySpark think my file does not exist locally?
body: <p>I am trying to read a textfile using pyspark that lives locally, and it is telling me the file does not exist:</p>
  
  <pre><code>sc = SparkContext()
  sc._conf.setMaster("local[*]")
  sc.setLogLevel("DEBUG")
  sqlContext = SQLContext(sc)
  
  inpath='file:///path/to/file'
  input_data = sqlContext.read.text(inpath)
  </code></pre>
  
  <p>and I get this: </p>
  
  <pre><code>Py4JJavaError: An error occurred while calling o52.showString.
  : org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, &lt;hostname&gt;): java.io.FileNotFoundException: File file:/path/to/file does not exist
  </code></pre>
  
  <p>I understand that you need to make sure that you change the configurations for spark when you are reading files locally, when you are running this on a cluster.  But, this is sitting on the master node, and the file does not need to be distributed across all nodes.  </p>
  
  <p>I checked out this question <a href="https://stackoverflow.com/questions/27299923/how-to-load-local-file-in-sc-textfile-instead-of-hdfs">How to load local file in sc.textFile, instead of HDFS</a> , and I tried the suggestion to set <code>sc._conf.setMaster("local[*]")</code> but that did not help - after restarting the spark context and rerunning it still does not work.</p>
  
  <p>Is there any other setting I can change so that this can work?</p>