title: Can I read csv files from Google Storage using Spark in more than one executor?
body: <p>I am having several <strong>csv</strong> files compressed within a google bucket, their are grouped in folders by hour, meaning another application saves several of those files in folders having the hour in their name.</p>
  
  <p>I am basically then having a Spark application reading all of those files - thousands of them - with a simple code like the one below:</p>
  
  <pre><code>sparkSession.read
        .format("csv")
        .option("sep", "\t")
        .option("header", false)
        .option("inferSchema", false)
        .csv(path))
  </code></pre>
  
  <p>It takes more than an hour to read, is that because they are compressed?</p>
  
  <p>I also noted in the Spark UI I only have one executor, never more than one. Can't I use several executors to read those files in parallel and do the processing faster? How to do that? I am bascailly trying to create a temp view with the files for further SQL statements from Spark.</p>
  
  <p>I am running in Dataproc with the default Yarn configuration.</p>