title: how to do intersection of list columns with pyspark dataframe columns?
body: <p>I have a below pyspark dataframe and i need to create new column (new_col) which is common items in column X and Y excluding items in Z.</p>
  
  <p>df</p>
  
  <pre><code>id X             Y                    Z            new_col
  1 [12,23,1,24]  [13,412,12,23,24]     [12]         [23,24]
  2 [1,2,3]       [2,4,5,6]             []           [2]
  </code></pre>