title: How to poll a remote directory for new files using kafka
body: <p>I have a folder on remote location and some other process keeps putting files in it.</p>
  
  <p>I am writing an application to download each new file and process it.</p>
  
  <p>For now I am downloading files with <code>rsync</code> that has been created in last hour and process it.</p>
  
  <p>But I need a solution where I want to put name of new file to be put onto a queue and then my consumer will take names from queue, download the file and process. </p>
  
  <p>In this solution I can run multiple producers which will poll only on kafka queue.</p>