title: How to read bz2 files into dataframes using pyspark?
body: <p>I can read a json file into a dataframe in Pyspark using</p>
  
  <pre><code>spark = SparkSession.builder.appName('GetDetails').getOrCreate()
  df = spark.read.json("path to json file")
  </code></pre>
  
  <p>However, when i try to read a bz2(compressed csv) into a dataframe it gives me an error. I am using:</p>
  
  <pre><code>spark = SparkSession.builder.appName('GetDetails').getOrCreate()
  df = spark.read.load("path to bz2 file")
  </code></pre>
  
  <p>Could you please help correct me?</p>