title: Python & Pandas. How to work with a subset of the "chunks" in a TextFileReader object?
body: <p>I need to import a large <code>.txt</code> file (approx. 10GB) to do some calculations. I'm using Pandas in Python 2.7.</p>
  
  <p>Basically, I need to construct sums and averages of certain series (columns), conditional on the value of other series. To be more precise: I have basic information on individuals living in a country, and, for example, I want to take the average age of the people in each municipality.</p>
  
  <p>I cannot import the whole file (because it is too big), so I am doing it in "chunks" (using <code>read_table</code>, <code>chunksize</code>).
  For each calculation, I don't need all of the chunks, just a subset of them.</p>
  
  <p>Since the information might not be ordered, I first iterate over all chunks to identify which of them have information for each of the municipalities. Hence, for each municipality I have a list with the indices of the chunks that contain at least one observation belonging to it.</p>
  
  <p>I would then like to use this list to select only those chunks, but I'm not being able to do it in a fast way.
  The only thing that seems to work is to iterate over all chunks again.</p>
  
  <p>Is there a way to directly select a subset of the "chunks" in a <code>TextFileReader</code> object without having to iterate over all of them?</p>