title: PySpark on Databricks getting Relative path in absolute URI: when trying to read in Json Files with DateStamps
body: <p>I am trying to read in a directory of JSON files to a spark dataframe in databricks and whenever I use the the wildcard character ('*') or when I have multiline enabled I get the following error:</p>
  
  <pre><code>IllegalArgumentException: 'java.net.URISyntaxException: Relative path in 
  absolute URI: 2017-08-17T06:01:02Z-077d951a-3848-422b-9a4c-a3b648dde0ac.json'
  </code></pre>
  
  <p>First encountered this when trying to use the the wildcard character:</p>
  
  <pre><code>t= 'mnt/data/2017-08-17/06/*.json'
  rdf = spark.read.load(t, 'json')
  </code></pre>
  
  <p>and got the above error. Found this strange because when I specified the filepath directly:</p>
  
  <pre><code>'dbfs:/mnt/data/2017-08-17/06/2017-08-17T06:01:02Z-077d951a-3848-422b-9a4c-a3b648dde0ac.json'
  </code></pre>
  
  <p>So I implemented this fix:</p>
  
  <pre><code>rpaths = list(dbutils.fs.ls("mnt/data/2017-08-17/06/"))
  rdf = spark.read.load(rpaths[0].path, 'json')
  
  for i in range(1, len(rpaths)):
    temp = spark.read.load(rpaths[i].path, 'json')
    rdf = rdf.union(temp)
  </code></pre>
  
  <p>Which worked fine until I tried enable multiline json:</p>
  
  <pre><code>rpaths = list(dbutils.fs.ls("mnt/data/2017-08-17/06/"))
  rdf = spark.read.load(rpaths[0].path, 'json', multiline = 'true')
  
  for i in range(1, len(rpaths)):
    temp = spark.read.load(rpaths[i].path, 'json')
    rdf = rdf.union(temp)
  </code></pre>
  
  <p>and got the same error as above.</p>
  
  <p>Tried to URLEncode because I assumed it had to do with the ':' character in the file name, however it still gives me the same error.</p>
  
  <pre><code>from urllib.parse import urlparse
  rpaths = list(dbutils.fs.ls("mnt/data/2017-08-17/06/"))
  rdf = spark.read.load(urlparse(rpaths[0].path).path, 'json', multiline = True)
  </code></pre>
  
  <p>Any idea on why this is happening/ how I would fix this?</p>
  
  <p><strong>EDIT</strong></p>
  
  <p>Have looked extensively through their documentation and have confirmed that I am providing an absolute path, however it says nothing about ':'s being not supported.</p>
  
  <pre><code>t= '/mnt/data/2017-08-17/06/*.json' 
  rdf = spark.read.load(t, 'json', multiline = True) 
  </code></pre>
  
  <p>Still gives the same issue. The path when using DButils resolves to: </p>
  
  <pre><code> 'dbfs:/mnt/data/2017-08-17/06/2017-08-17T06:01:02Z-077d951a-3848-422b-9a4c-a3b648dde0ac.json'
  </code></pre>
  
  <p>Which is an absolute file path</p>
  
  <p>I think it may have to do with the colon (':') character. It seems that ':' has a special meaning in an URI, which is what HDFS uses. I Have tried to escape the character (%3A) but then it says there exists no such path?</p>
  
  <p>Any Ideas?</p>