title: Does spark.sql.autoBroadcastJoinThreshold work for joins using Dataset's join operator?
body: <p>I'd like to know if <code>spark.sql.autoBroadcastJoinThreshold</code> property can be useful for broadcasting smaller table on all worker nodes (while making the join) even when the join scheme is using the Dataset API join instead of using Spark SQL.</p>
  
  <p>If my bigger table is 250 Gigs and Smaller is 20 Gigs, do I need to set this config: <code>spark.sql.autoBroadcastJoinThreshold</code> = 21 Gigs (maybe) in order for sending the whole table / <code>Dataset</code> to all worker nodes?</p>
  
  <p><strong>Examples</strong>:</p>
  
  <ul>
  <li><p><strong>Dataset API join</strong></p>
  
  <pre class="lang-scala prettyprint-override"><code>val result = rawBigger.as("b").join(
    broadcast(smaller).as("s"),
    rawBigger(FieldNames.CAMPAIGN_ID) === smaller(FieldNames.CAMPAIGN_ID), 
    "left_outer"
  )
  </code></pre></li>
  <li><p><strong>SQL</strong></p>
  
  <pre class="lang-sql prettyprint-override"><code>select * 
  from rawBigger_table b, smaller_table s
  where b.campign_id = s.campaign_id;
  </code></pre></li>
  </ul>