title: How can I prevent database connections from timing out in Rails?
body: <p>I have a Rails system in which every half hour, the following is done:</p>
  
  <ul>
  <li>There are 15 clients somewhere else on the network</li>
  <li>The server creates a record called <code>Measurement</code> for each of these clients</li>
  <li>The measurement records are configured, and then they are run asynchronously via Sidekiq, using <code>MeasurementWorker.perform_async(m.id)</code></li>
  <li>The connection to the client is done with Celluloid actors and a WebSocket client</li>
  <li>Each measurement, when run, creates a number of event records that are stored in the database</li>
  </ul>
  
  <p>The system has been running well with 5 clients, but now I am at 15, and many of the measurements don't run anymore when I start them at the same time, with the following error:</p>
  
  <pre><code>2015-02-04T07:30:10.410Z 35519 TID-owd4683iw MeasurementWorker JID-15f6b396ae9e3e3cb2ee3f66 INFO: fail: 5.001 sec
  2015-02-04T07:30:10.412Z 35519 TID-owd4683iw WARN: {"retry"=&gt;false, "queue"=&gt;"default", "backtrace"=&gt;true, "class"=&gt;"MeasurementWorker", "ar
  gs"=&gt;[6504], "jid"=&gt;"15f6b396ae9e3e3cb2ee3f66", "enqueued_at"=&gt;1423035005.4078047}
  2015-02-04T07:30:10.412Z 35519 TID-owd4683iw WARN: could not obtain a database connection within 5.000 seconds (waited 5.000 seconds)
  2015-02-04T07:30:10.412Z 35519 TID-owd4683iw WARN: /home/webtv/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_
  record/connection_adapters/abstract/connection_pool.rb:190:in `block in wait_poll'
  ....
  </code></pre>
  
  <p>Now, my production environment looks like this:</p>
  
  <h3><code>config/sidekiq.yml</code></h3>
  
  <pre><code>production:
    :verbose: false
    :logfile: ./log/sidekiq.log
    :poll_interval: 5
    :concurrency: 50
  </code></pre>
  
  <h3><code>config/unicorn.rb</code></h3>
  
  <pre><code>...
  worker_processes Integer(ENV["WEB_CONCURRENCY"] || 3)
  timeout 60
  ...
  </code></pre>
  
  <h3><code>config/database.yml</code></h3>
  
  <pre><code>production:
    adapter: postgresql
    database: ***
    username: ***
    password: ***
    host: 127.0.0.1
    pool: 50
  </code></pre>
  
  <h3><code>postgresql.conf</code></h3>
  
  <pre><code>max_connections = 100 # default
  </code></pre>
  
  <p>As you see, I've already increased the concurrency of Sidekiq to 50, to cater for a high number of possible concurrent measurements. I've set the database pool to 50, which already looks like overkill to me.</p>
  
  <p>I should add that the server itself is quite powerful, with 8 GB RAM and a quad-core Xeon E5-2403 1.8 GHz.</p>
  
  <p>What should these values ideally be set to? What formula can I use to calculate them? (E.g. <em>number of maximum DB connections = Unicorn workers &times; Sidekiq concurrency &times; N</em>)</p>