title: What's the most efficient way to filter a DataFrame
body: <p>... by checking whether a columns' value is in a <code>seq</code>.<br>
  Perhaps I'm not explaining it very well, I basically want this (to express it using regular SQL): <code>DF_Column IN seq</code>?</p>
  
  <p>First I did it using a <code>broadcast var</code> (where I placed the seq), <code>UDF</code> (that did the checking) and <code>registerTempTable</code>.<br>
  The problem is that I didn't get to test it since I ran into a <a href="https://issues.apache.org/jira/browse/SPARK-5281">known bug</a> that apparently only appears when using <code>registerTempTable</code> with <em>ScalaIDE</em>.</p>
  
  <p>I ended up creating a new <code>DataFrame</code> out of <code>seq</code> and doing inner join with it (intersection), but I doubt that's the most performant way of accomplishing the task.</p>
  
  <p>Thanks</p>
  
  <p><strong>EDIT:</strong> (in response to @YijieShen):<br>
  How to do <code>filter</code> based on whether elements of one <code>DataFrame</code>'s column are in another DF's column (like SQL <code>select * from A where login in (select username from B)</code>)?</p>
  
  <p>E.g:
  First DF:</p>
  
  <pre><code>login      count
  login1     192  
  login2     146  
  login3     72   
  </code></pre>
  
  <p>Second DF:</p>
  
  <pre><code>username
  login2
  login3
  login4
  </code></pre>
  
  <p>The result:</p>
  
  <pre><code>login      count
  login2     146  
  login3     72   
  </code></pre>
  
  <p><strong>Attempts:</strong><br>
  <em><strong>EDIT-2:</strong> I think, now that the bug is fixed, these should work. <strong>END EDIT-2</strong></em></p>
  
  <pre><code>ordered.select("login").filter($"login".contains(empLogins("username")))
  </code></pre>
  
  <p>and</p>
  
  <pre><code>ordered.select("login").filter($"login" in empLogins("username"))
  </code></pre>
  
  <p>which both throw <code>Exception in thread "main" org.apache.spark.sql.AnalysisException</code>, respectively:</p>
  
  <pre><code>resolved attribute(s) username#10 missing from login#8 in operator 
  !Filter Contains(login#8, username#10);
  </code></pre>
  
  <p>and</p>
  
  <pre><code>resolved attribute(s) username#10 missing from login#8 in operator 
  !Filter login#8 IN (username#10);
  </code></pre>