title: Azure Databricks writing a file into Azure Data Lake Gen 2
body: <p>I have an Azure Data Lake gen1 and an Azure Data Lake gen2 (Blob Storage w/hierarchical) and I am trying to create a Databricks notebook (Scala) that reads 2 files and writes a new file back into the Data Lake.  In both Gen1 and Gen2 I am experiencing the same issue where the file name of the output csv I have specified is getting saved as a directory and inside that directory it's writing 4 files "<em>committed</em>, <em>started</em>, _SUCCESS, and part-00000-tid-
  
  <p><img src="https://i.imgur.com/Uc0e8TK.png" alt="databricks output screenshot"></p>
  
  <p>For the life of me, I can't figure out why it's doing it and not actually saving the csv to the location.
  Here's an example of the code I've written.  If I do a .show() on the df_join dataframe then it outputs the correct looking results.  But the .write is not working correctly.</p>
  
  <pre><code>val df_names = spark.read.option("header", "true").csv("/mnt/datalake/raw/names.csv")
  val df_addresses = spark.read.option("header", "true").csv("/mnt/datalake/raw/addresses.csv")
  
  val df_join = df_names.join(df_addresses, df_names.col("pk") === df_addresses.col("namepk"))
  
  
  df_join.write
  .format("com.databricks.spark.csv")
  .option("header", "true")
  .mode("overwrite")
  .save("/mnt/datalake/reports/testoutput.csv")
  </code></pre>