title: Spring Batch - Reading a large flat file - Choices to scale horizontally?
body: <p>I have started researching Spring Batch in the last hour or two. And require your inputs.</p>
  
  <p>The problem : Read a/multiple csv file(s) with 20 million data, perform minor processing, store it in db and also write output to another flat file in the least time.</p>
  
  <p><strong>Most important</strong> : I need to make choices which will scale horizontally in the future.</p>
  
  <p>Questions :</p>
  
  <p>Use Remote Chunking or Partitioning to scale horizontally?</p>
  
  <p>Since data is in a flat file both Remote Chunking and Partitioning are bad choices?</p>
  
  <p>Which multi process solution will make it possible to read from a large file, spread processing across multiple servers and update Db but finally write/output to a single file?</p>
  
  <p>Does multiresourcepartitioner work across servers?</p>
  
  <p>Any good tutorials you know of where something like this has been accomplished/demonstrated?</p>
  
  <p>Your thoughts on how this needs to be attempted like 1) Split large file into smaller files before starting the job 2) Read one file at a time using the Item Reader...........</p>