title: memory-efficient built-in SqlAlchemy iterator/generator?
body: <p>I have a ~10M record MySQL table that I interface with using SqlAlchemy.  I have found that queries on large subsets of this table will consume too much memory even though I thought I was using a built-in generator that intelligently fetched bite-sized chunks of the dataset:</p>
  
  <pre><code>for thing in session.query(Things):
      analyze(thing)
  </code></pre>
  
  <p>To avoid this, I find I have to build my own iterator that bites off in chunks:</p>
  
  <pre><code>lastThingID = None
  while True:
      things = query.filter(Thing.id &lt; lastThingID).limit(querySize).all()
      if not rows or len(rows) == 0: 
          break
      for thing in things:
          lastThingID = row.id
          analyze(thing)
  </code></pre>
  
  <p>Is this normal or is there something I'm missing regarding SA built-in generators?</p>
  
  <p>The answer to <a href="https://stackoverflow.com/questions/1078383/sqlalchemy-difference-between-query-and-query-all-in-for-loops">this question</a> seems to indicate that the memory consumption is not to be expected.</p>