title: Why does this query slowdown in MongoDB using pymongo?
body: <p>I am running a similar query 374 times and up to the 367th time performance is reasonable but then the time to return results deteriorates dramatically. </p>
  
  <p>The collection I query stores posts where each post has a unique ID and there will be several versions of the same post in the database. The task is to get the last version of each post ID. The approach is to get a distinct list of post IDs and then for each post ID get the post ID with the highest ObjectID. </p>
  
  <p>This could also be done through the aggregation framework but it errors with <code>exception: aggregation result exceeds maximum document size (16MB)</code></p>
  
  <p>This is the code:</p>
  
  <pre><code>for oi in obj_ids: #obj_ids is a list of strings containing unique post IDs
      t1 = time.time()
      o = col.find({'object_id':oi}).sort('_id', -1).limit(1)[0]
      t2 = time.time()
  </code></pre>
  
  <p>The <code>col.find</code> function is timed and here is how this query's performance deteriorates over time:</p>
  
  <pre><code>364 of 374 in 0.00369000434875s
  365 of 374 in 0.0037579536438s
  366 of 374 in 0.00375485420227s
  367 of 374 in 0.00367307662964s
  368 of 374 in 0.735110998154s
  369 of 374 in 3.09494900703s
  370 of 374 in 5.16561698914s
  371 of 374 in 7.14517307281s
  372 of 374 in 8.3472340107s
  373 of 374 in 8.61702394485s
  374 of 374 in 8.07462406158s
  </code></pre>
  
  <p>Any ideas what is happening?</p>
  
  <p>UPDATE 2012/11/01</p>
  
  <p>Using the Python cprofile I found that there seems to be a network bottleneck</p>
  
  <p><img src="https://i.stack.imgur.com/jPZBh.png" alt="sorted by time"></p>
  
  <p>EDIT:
  spelling</p>