title: Reading Blob Into Pyspark
body: <p>I'm trying to read in a series of json files stored in an azure blob into spark using the databricks notebook. I set the conf() with my account and key but it always returns the error</p>
  
  <pre><code>shaded.databricks.org.apache.hadoop.fs.azure.AzureException: java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.
  </code></pre>
  
  <p>I've followed along with the information provided here:</p>
  
  <p><a href="https://docs.databricks.com/_static/notebooks/data-import/azure-blob-store.html" rel="nofollow noreferrer">https://docs.databricks.com/_static/notebooks/data-import/azure-blob-store.html</a></p>
  
  <p>and here:</p>
  
  <p><a href="https://luminousmen.com/post/azure-blob-storage-with-pyspark" rel="nofollow noreferrer">https://luminousmen.com/post/azure-blob-storage-with-pyspark</a></p>
  
  <p>I can pull the data just fine using the azure sdk for python</p>
  
  <pre><code>storage_account_name = "name"
  storage_account_access_key = "key"
  
  spark.conf.set(
    "fs.azure.account.key."+storage_account_name+".blob.core.windows.net",
    storage_account_access_key)
  
  file_location = "wasbs://loc/locationpath"
  file_type = "json"
  
  df = spark.read.format(file_type).option("inferSchema", "true").load(file_location)
  </code></pre>
  
  <p>Should return a dataframe of the json file</p>