title: Read CSV data in batches and Process it
body: <p>I have a csv file that looks like this </p>
  
  <pre><code>#DELTA,1#    
  Risk1,10
  Risk2,10
  Risk3,10
  Risk4,10
  Risk5,10
  #DELTA,1#    
  Risk6,10
  Risk7,10
  Risk8,10
  Risk9,10
  Risk10,10
  </code></pre>
  
  <p>and so on. These are very large files (in order of GBs).</p>
  
  <p>What I want to be able to do is to read them in batches like</p>
  
  <p>start streamreader from csv file from first line to just before next #Delta starts</p>
  
  <pre><code>---Batch 1---
  #DELTA,1
  Risk1,10
  Risk2,10
  Risk3,10
  Risk4,10
  Risk5,10
  --Batch 2-----
  #DELTA,1
  Risk6,10
  Risk7,10
  Risk8,10
  Risk9,10
  Risk10,10
  ----------------------
  </code></pre>
  
  <p>and once I get a batch put this subset for processing and come back and restart preparing another batch and so on till the end of file is reached.</p>
  
  <p>I have tried making the LINQ's take and take while but with my understanding of LINQ I am not getting far.</p>
  
  <p>Basically in summary it have to stream data in batches based on a pattern in my stream.. maybe my brain cells are dead or maybe it is too late in evening. 
  Really appreaicate anyone's help</p>