title: A better way to load MongoDB data to a DataFrame using Pandas and PyMongo?
body: <p>I have a 0.7 GB MongoDB database containing tweets that I'm trying to load into a dataframe. However, I get an error.</p>
  
  <pre><code>MemoryError:    
  </code></pre>
  
  <p>My code looks like this:</p>
  
  <pre><code>cursor = tweets.find() #Where tweets is my collection
  tweet_fields = ['id']
  result = DataFrame(list(cursor), columns = tweet_fields)
  </code></pre>
  
  <p>I've tried the methods in the following answers, which at some point create a list of all the elements of the database before loading it.</p>
  
  <ul>
  <li><a href="https://stackoverflow.com/a/17805626/2297475">https://stackoverflow.com/a/17805626/2297475</a></li>
  <li><a href="https://stackoverflow.com/a/16255680/2297475">https://stackoverflow.com/a/16255680/2297475</a></li>
  </ul>
  
  <p>However, in another answer which talks about list(), the person said that it's good for small data sets, because everything is loaded into memory.</p>
  
  <ul>
  <li><a href="https://stackoverflow.com/a/13215411/2297475">https://stackoverflow.com/a/13215411/2297475</a></li>
  </ul>
  
  <p>In my case, I think it's the source of the error. It's too much data to be loaded into memory. What other method can I use?</p>