title: How can I efficiently bulk reindex with Searchkick without hitting the DB connection limit in a multitenant application?
body: <p>I have a multitenant application which was hitting connection limits and crashing Elasticsearch when using async reindexing via an <code>after_commit</code> hook. I've implemented bulk reindexing as suggested in the readme:
  <a href="https://github.com/ankane/searchkick#queuing" rel="nofollow noreferrer">https://github.com/ankane/searchkick#queuing</a></p>
  <p>I'm running a job every minute to keep things up to date. Elasticsearch is no longer crashing (most of the time), but I've hit the DB connection limit many times. This is the error in Sidekiq:</p>
  <pre><code>ActiveRecord::ConnectionTimeoutError: could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds); all pooled connections were in use
  </code></pre>
  <p>It's somewhat obvious why this is happening:</p>
  <pre><code>class DocumentReindexQueueWorker
    include Sidekiq::Worker
  
    def perform
      Account.find_each do |account|
        Apartment::Tenant.switch(account.subdomain) do
          Searchkick::ProcessQueueJob.perform_later(class_name: &quot;Document&quot;)
        end
      end
    end
  end
  </code></pre>
  <p>This bombards the DB with connections because all tenants are running their bulk reindex at the same moment. This is where I would normally introduce some kind of staggered delay using <code>delay_for</code> and setting <code>retry: false</code> so jobs don't pile up when they're just going to get queued again in another minute. These options don't seem to exist for <code>Searchkick::ProcessQueueJob</code> though.</p>
  <p>I have a concurrency limit of 3 with <code>ActiveJob::TrafficControl</code> (per <a href="https://github.com/ankane/searchkick#parallel-reindexing" rel="nofollow noreferrer">this</a>). The <code>max_connections</code> of the DB is 100 but it's ActiveRecord that's reporting the connection limit. Could it be that I just need to increase <code>RAILS_MAX_THREADS</code>? That's currently set to 5.</p>
  <p>FYI running <code>SELECT * FROM pg_stat_activity</code> shows 7 puma connections and 5 sidekiq connections.</p>