title: Loop over files in HDFS directory
body: <p>I need to loop over all csv files in a Hadoop file system. I can list all of the files in a HDFS directory with</p>
  
  <pre><code>&gt; hadoop fs -ls /path/to/directory
  Found 2 items
  drwxr-xr-x   - hadoop hadoop          2 2016-10-12 16:20 /path/to/directory/tmp
  -rwxr-xr-x   3 hadoop hadoop 4691945927 2016-10-12 19:37 /path/to/directory/myfile.csv
  </code></pre>
  
  <p>and can loop over all files in a standard directory with </p>
  
  <pre><code>for filename in /path/to/another/directory/*.csv; do echo $filename; done
  </code></pre>
  
  <p>but <strong>how can I combine the two?</strong> I've tried</p>
  
  <pre><code>for filename in `hadoop fs -ls /path/to/directory | grep csv`; do echo $filename; done
  </code></pre>
  
  <p>but that gives me some nonsense like </p>
  
  <pre><code>Found
  2
  items
  drwxr-xr-x
  
  hadoop
  hadoop
  2    
  2016-10-12
  ....
  </code></pre>