title: Implementing general purpose long polling
body: <p>I've been trying to implement a simple long polling service for use in my own projects and maybe release it as a SAAS if I succeed. These are the two approaches I've tried so far, both using Node.js (polling PostgreSQL in the back).</p>
  
  <h2>1. Periodically check all the clients in the same interval</h2>
  
  <p>Every new connection is pushed onto a queue of connections, which is being walked through in an interval.</p>
  
  <pre><code>var queue = [];
  
  function acceptConnection(req, res) {
    res.setTimeout(5000);
    queue.push({ req: req, res: res });
  }
  
  function checkAll() {
    queue.forEach(function(client) {
      // respond if there is something new for the client
    });
  }
  
  // this could be replaced with a timeout after all the clients are served
  setInterval(checkAll, 500);
  </code></pre>
  
  <h2>2. Check each client at a separate interval</h2>
  
  <p>Every client gets his own <code>ticker</code> which checks for new data</p>
  
  <pre><code>function acceptConnection(req, res) {
    // something which periodically checks data for the client
    // and responds if there is anything new
    new Ticker(req, res);
  }
  </code></pre>
  
  <p>While this keeps the minimum latency for each client lower, it also introduces overhead by setting a lot of timeouts.</p>
  
  <h2>Conclusion</h2>
  
  <p>Both of these approaches solve the problem quite easily, but I don't feel that this will scale up easily to something like 10 million open connections, especially since I'm polling the database on every check for every client.</p>
  
  <p>I thought about doing this without the database and just immediately broadcast new messages to all open connections, but that will fail if a client's connection dies for a few seconds while the broadcast is happening, because it is not persistent. Which means I basically need to be able to look up messages in history when the client polls for the first time.</p>
  
  <p>I guess one step up here would be to have a data source where I can subscribe to new data coming in (CouchDB change notifications?), but maybe I'm missing something in the big picture here?</p>
  
  <p><strong>What is the usual approach for doing highly scalable long polling?</strong> I'm not specifically bound to Node.js, I'd actually prefer any other suggestion with a reasoning why.</p>