title: Spark 3.1 joining two datasets in Java
body: <p>I have 2 datasets as below,</p>
  <pre><code>dataset1
  +--------------------+----------------+
  |ids                 | names          |
  +--------------------+-----------------
  |1236015             | aaaaa          |
  |234567              | bbbbb          |
  |90909090            | ccccc          |
  +--------------------+-----------------
  
  and this is schema for dataset1
  
  root
   |-- ids: string (nullable = true)
   |-- names: string (nullable = true)
  
  
  dataset2
  +--------------------+
  |ids                 |
  +--------------------+
  |1236015             |
  |90909090            |
  |1345677             |
  +--------------------+
  and this is schema for dataset
  root
   |-- ids: string (nullable = true)
  
  </code></pre>
  <p>I want to remove rows from dataset1 if the ids are present in dataset2
  like this</p>
  <pre><code>+--------------------+----------------+
  |ids                 | names          |
  +--------------------+-----------------
  |234567              | bbbbb          |
  |90909090            | ccccc          |
  ---------------------------------------
  
  </code></pre>
  <p>I tried following</p>
  <pre><code> dataset1.join(dataset2,col(&quot;ids&quot;).notEqual(col(&quot;ids&quot;)), &quot;semi&quot;);
  </code></pre>
  <p>...but it returns all rows from dataset1. What could be the issue?</p>