title: How does an `IN` instruction affect Druid SQL requests?
body: <p>The <a href="http://druid.io/docs/0.12.1/querying/filters.html#in-filter" rel="nofollow noreferrer">Druid Documentation of the IN Filter</a> does not say anything about efficiency. We're having problems with one SQL statement which keeps timing out and I think that the main culprit would be the <code>field IN (v, e, r, y, l, o, n, g, l, i, s, t, o, f, i, d, s)</code> from the <code>WHERE</code> clause.</p>
  
  <p>Is there documentation about the efficiency of the IN Filter in Druid? How to use it and how <strong>not</strong> to use it?</p>
  
  <p>The main reason why I think it's our culprit is because the list of elements in the <code>IN</code> list can be very large (thousands of identifiers) and the list probably changes (grows) daily. The growth is by one or two or three (I'm not too sure what the maximum is, I suspect that some customers may add as many as 10 new items in a day), which over the years ended up being thousands for some of our customers.</p>
  
  <p>It would be possible for us to use a <code>JOIN</code> or transform the <code>WHERE</code> to <em>calculate</em> the list of identifiers on the fly. Something that would look like this:</p>
  
  <pre><code>`WHERE ... object.customer_id = customer.id AND object.id = id ...`
  </code></pre>
  
  <p>I'm wondering how Druid is able to aggregate the data for us and whether the IN Filter would cause problems for the aggregation over time when a simpler <code>WHERE ...</code> clause may work much better and really aggregate the results automatically.</p>
  
  <p>Our queries use a time set to <em>Year to Date</em> (so between Jan 1st and Today inclusive).</p>