title: Finding mongoDB records in batches (using mongoid ruby adapter)
body: <p>Using rails 3 and mongoDB with the mongoid adapter, how can I batch finds to the mongo DB?  I need to grab all the records in a particular mongo DB collection and index them in solr (initial index of data for searching).</p>
  
  <p>The problem I'm having is that doing Model.all grabs all the records and stores them into memory.  Then when I process over them and index in solr, my memory gets eaten up and the process dies.</p>
  
  <p>What I'm trying to do is batch the find in mongo so that I can iterate over 1,000 records at a time, pass them to solr to index, and then process the next 1,000, etc...</p>
  
  <p>The code I currently have does this:</p>
  
  <pre><code>Model.all.each do |r|
    Sunspot.index(r)
  end
  </code></pre>
  
  <p>For a collection that has about 1.5 million records, this eats up 8+ GB of memory and kills the process.  In ActiveRecord, there is a find_in_batches method that allows me to chunk up the queries into manageable batches that keeps the memory from getting out of control.  However, I can't seem to find anything like this for mongoDB/mongoid.</p>
  
  <p>I would LIKE to be able to do something like this:</p>
  
  <pre><code>Model.all.in_batches_of(1000) do |batch|
    Sunpot.index(batch)
  end
  </code></pre>
  
  <p>That would alleviate my memory problems and query difficulties by only doing a manageable problem set each time.  The documentation is sparse, however, on doing batch finds in mongoDB.  I see lots of documentation on doing batch inserts but not batch finds.</p>