title: What is the right way to deal with Mongodb connections?
body: <p>I try node.js with mongodb (2.2.2) together using the native node.js drive by 10gen.</p>
  
  <p>At first everything went well. But when coming to the concurrency benchmarking part, a lot of errors occured. Frequent connect/close with 1000 concurrencies may cause mongodb reject any further requests with error like:</p>
  
  <pre><code>Error: failed to connect to [localhost:27017]
  
  Error: Could not locate any valid servers in initial seed list
  
  Error: no primary server found in set
  </code></pre>
  
  <p>Also, if a lot of clients shutdown without explicit close, it'll take mongodb minutes to detect and close them. Which will also cause similar connection problems. (Using /var/log/mongodb/mongodb.log to check the connection status)</p>
  
  <p>I have tried a lot. According to the manual, mongodb don't have connection limitation, but <strong>poolSize</strong> option seems to have no effects to me.</p>
  
  <p>As I have only worked with it in node-mongodb-native module, I'm not very sure what eventually caused the problem. What about the performance in other other languages and drivers?</p>
  
  <p>PS: Currently, using self maintained pool is the only solution I figured out, but using it can not can not solve the problem with replica set. According to my test, replica set seems take much less connections then standalone mongodb. But have no idea why this happens.</p>
  
  <p>Concurrency test code:</p>
  
  <pre><code>var MongoClient = require('mongodb').MongoClient;
  
  var uri = "mongodb://192.168.0.123:27017,192.168.0.124:27017/test";
  
  for (var i = 0; i &lt; 1000; i++) {
      MongoClient.connect(uri, {
          server: {
              socketOptions: {
                  connectTimeoutMS: 3000
              }
          },
      }, function (err, db) {
          if (err) {
              console.log('error: ', err);
          } else {
              var col = db.collection('test');
              col.insert({abc:1}, function (err, result) {
                  if (err) {
                      console.log('insert error: ', err);
                  } else {
                      console.log('success: ', result);
                  }
                  db.close()
              })
          }
      })
  }
  </code></pre>
  
  <p>Generic-pool solution:</p>
  
  <pre><code>var MongoClient = require('mongodb').MongoClient;
  var poolModule = require('generic-pool');
  
  var uri = "mongodb://localhost/test";
  
  var read_pool = poolModule.Pool({
      name     : 'redis_offer_payment_reader',
      create   : function(callback) {
          MongoClient.connect(uri, {}, function (err, db) {
              if (err) {
                  callback(err);
              } else {
                  callback(null, db);
              }
          });
      },
      destroy  : function(client) { client.close(); },
      max      : 400,
      // optional. if you set this, make sure to drain() (see step 3)
      min      : 200, 
      // specifies how long a resource can stay idle in pool before being removed
      idleTimeoutMillis : 30000,
      // if true, logs via console.log - can also be a function
      log : false 
  });
  
  
  var size = [];
  for (var i = 0; i &lt; 100000; i++) {
      size.push(i);
  }
  
  size.forEach(function () {
      read_pool.acquire(function (err, db) {
          if (err) {
              console.log('error: ', err);
          } else {
              var col = db.collection('test');
              col.insert({abc:1}, function (err, result) {
                  if (err) {
                      console.log('insert error: ', err);
                  } else {
                      //console.log('success: ', result);
                  }
                  read_pool.release(db);
              })
          }
      })
  })
  </code></pre>