title: Read all files in a nested folder in Spark
body: <p>If we have a folder <code>folder</code> having all <code>.txt</code> files, we can read them all using <code>sc.textFile("folder/*.txt")</code>. But what if I have a folder <code>folder</code> containing even more folders named datewise, like, <code>03</code>, <code>04</code>, ..., which further contain some <code>.log</code> files. How do I read these in Spark?</p>
  
  <p>In my case, the structure is even more nested &amp; complex, so a general answer is preferred.</p>