title: Pyspark SQL WHERE NOT IN?
body: <p>I'm trying to find everything that is in <code>nodes2</code> but not <code>nodes1</code>.</p>
  <pre><code>spark.sql(&quot;&quot;&quot;
  SELECT COUNT(*) FROM 
      (SELECT * FROM nodes2 WHERE NOT IN  
      (SELECT * FROM nodes1))
  &quot;&quot;&quot;).show()
  </code></pre>
  <p>Getting the following error:
  <code>&quot;cannot resolve '</code>NOT<code>' given input columns: [nodes2.~id, nodes2.~label];</code></p>
  <p>Is it possible to do this sort of set difference operation in Pyspark?</p>